syzbot

 sign-in | mailing list | source | docs
🐞 Open [1496]
≡ Subsystems
🐞 Fixed [6528]
🐞 Invalid [16625]
⬇ Missing Backports [205]
≡ Crashes
Kernel Health Bugs/Month Bug Lifetimes Fuzzing
Total Repo Heatmap Subsystems Heatmap
💬 Send us feedback

possible deadlock in htab_lock_bucket (2)

Status: upstream: reported C repro on 2024/08/05 10:05
Subsystems: bpf
[Documentation on labels]
Reported-by: syzbot+ee7551b0640c5471e610@syzkaller.appspotmail.com
First crash: 384d, last: 152d
Cause bisection: failed (error log, bisect log)
  
Fix bisection: failed (error log, bisect log)
  
▶ ▼ Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [bpf?] possible deadlock in htab_lock_bucket (2) 0 (1) 2024/08/05 10:05
▶ ▼ Similar bugs (1)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 possible deadlock in htab_lock_bucket origin:upstream missing-backport 4 C inconclusive 5 30d 494d 0/3 upstream: reported C repro on 2024/04/17 13:40
▶ ▼ Last patch testing requests (10)
Created Duration User Patch Repo Result
2025/08/13 13:44 39m retest repro net OK log
2025/07/15 03:12 2h44m retest repro bpf-next OK log
2025/06/29 00:09 26m retest repro bpf-next OK log
2025/06/17 14:46 25m retest repro bpf-next report log
2025/06/03 21:15 25m retest repro net report log
2025/05/28 23:30 26m retest repro net-next OK log
2025/05/04 02:31 27m retest repro bpf OK log
2025/04/19 19:54 31m retest repro bpf-next report log
2025/04/08 13:17 20m retest repro bpf-next report log
2025/04/08 13:17 24m retest repro bpf-next report log
▶ ▼ Fix bisection attempts (2)
Created Duration User Patch Repo Result
2025/05/02 11:57 13h00m (2) bisect fix bpf-next error job log
2025/03/25 09:27 3h38m bisect fix net OK (0) job log log

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.11.0-rc4-syzkaller-00326-g5277d130947b #0 Not tainted
------------------------------------------------------
sshd/5231 is trying to acquire lock:
ffff88802f484458 (&htab->lockdep_key#20){....}-{2:2}, at: htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167

but task is already holding lock:
ffff88807bd1c140 (&htab->lockdep_key#21){....}-{2:2}, at: htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&htab->lockdep_key#21){....}-{2:2}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
       __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
       _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
       htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
       htab_lru_map_delete_elem+0x1f1/0x700 kernel/bpf/hashtab.c:1466
       0xffffffffa0001b9f
       bpf_dispatcher_nop_func include/linux/bpf.h:1257 [inline]
       __bpf_prog_run include/linux/filter.h:701 [inline]
       bpf_prog_run include/linux/filter.h:708 [inline]
       __bpf_trace_run kernel/trace/bpf_trace.c:2318 [inline]
       bpf_trace_run2+0x2ec/0x540 kernel/trace/bpf_trace.c:2359
       __traceiter_contention_begin+0x7b/0xb0 include/trace/events/lock.h:95
       trace_contention_begin+0x117/0x140 include/trace/events/lock.h:95
       __pv_queued_spin_lock_slowpath+0x114/0xdb0 kernel/locking/qspinlock.c:402
       pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:584 [inline]
       queued_spin_lock_slowpath+0x42/0x50 arch/x86/include/asm/qspinlock.h:51
       queued_spin_lock include/asm-generic/qspinlock.h:114 [inline]
       do_raw_spin_lock+0x272/0x370 kernel/locking/spinlock_debug.c:116
       htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
       htab_lru_map_delete_elem+0x1f1/0x700 kernel/bpf/hashtab.c:1466
       bpf_prog_6f5f05285f674219+0x43/0x4c
       bpf_dispatcher_nop_func include/linux/bpf.h:1257 [inline]
       __bpf_prog_run include/linux/filter.h:701 [inline]
       bpf_prog_run include/linux/filter.h:708 [inline]
       __bpf_trace_run kernel/trace/bpf_trace.c:2318 [inline]
       bpf_trace_run2+0x2ec/0x540 kernel/trace/bpf_trace.c:2359
       __traceiter_contention_begin+0x7b/0xb0 include/trace/events/lock.h:95
       trace_contention_begin+0xf5/0x120 include/trace/events/lock.h:95
       __mutex_lock_common kernel/locking/mutex.c:610 [inline]
       __mutex_lock+0x147/0xd70 kernel/locking/mutex.c:752
       pipe_read+0x12a/0x13e0 fs/pipe.c:264
       new_sync_read fs/read_write.c:395 [inline]
       vfs_read+0x9bd/0xbc0 fs/read_write.c:476
       ksys_read+0x1a0/0x2c0 fs/read_write.c:619
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (&htab->lockdep_key#20){....}-{2:2}:
       check_prev_add kernel/locking/lockdep.c:3133 [inline]
       check_prevs_add kernel/locking/lockdep.c:3252 [inline]
       validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3868
       __lock_acquire+0x137a/0x2040 kernel/locking/lockdep.c:5142
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
       __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
       _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
       htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
       htab_lru_map_delete_elem+0x1f1/0x700 kernel/bpf/hashtab.c:1466
       bpf_prog_6f5f05285f674219+0x43/0x4c
       bpf_dispatcher_nop_func include/linux/bpf.h:1257 [inline]
       __bpf_prog_run include/linux/filter.h:701 [inline]
       bpf_prog_run include/linux/filter.h:708 [inline]
       __bpf_trace_run kernel/trace/bpf_trace.c:2318 [inline]
       bpf_trace_run2+0x2ec/0x540 kernel/trace/bpf_trace.c:2359
       __traceiter_contention_begin+0x7b/0xb0 include/trace/events/lock.h:95
       trace_contention_begin+0x117/0x140 include/trace/events/lock.h:95
       __pv_queued_spin_lock_slowpath+0x114/0xdb0 kernel/locking/qspinlock.c:402
       pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:584 [inline]
       queued_spin_lock_slowpath+0x42/0x50 arch/x86/include/asm/qspinlock.h:51
       queued_spin_lock include/asm-generic/qspinlock.h:114 [inline]
       do_raw_spin_lock+0x272/0x370 kernel/locking/spinlock_debug.c:116
       htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
       htab_lru_map_delete_elem+0x1f1/0x700 kernel/bpf/hashtab.c:1466
       0xffffffffa0001b9f
       bpf_dispatcher_nop_func include/linux/bpf.h:1257 [inline]
       __bpf_prog_run include/linux/filter.h:701 [inline]
       bpf_prog_run include/linux/filter.h:708 [inline]
       __bpf_trace_run kernel/trace/bpf_trace.c:2318 [inline]
       bpf_trace_run2+0x2ec/0x540 kernel/trace/bpf_trace.c:2359
       __traceiter_contention_begin+0x7b/0xb0 include/trace/events/lock.h:95
       trace_contention_begin+0xf5/0x120 include/trace/events/lock.h:95
       __mutex_lock_common kernel/locking/mutex.c:610 [inline]
       __mutex_lock+0x147/0xd70 kernel/locking/mutex.c:752
       pipe_read+0x12a/0x13e0 fs/pipe.c:264
       new_sync_read fs/read_write.c:395 [inline]
       vfs_read+0x9bd/0xbc0 fs/read_write.c:476
       ksys_read+0x1a0/0x2c0 fs/read_write.c:619
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&htab->lockdep_key#21);
                               lock(&htab->lockdep_key#20);
                               lock(&htab->lockdep_key#21);
  lock(&htab->lockdep_key#20);

 *** DEADLOCK ***

4 locks held by sshd/5231:
 #0: ffff88807e1c2468 (&pipe->mutex){+.+.}-{3:3}, at: pipe_read+0x12a/0x13e0 fs/pipe.c:264
 #1: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
 #1: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
 #1: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: __bpf_trace_run kernel/trace/bpf_trace.c:2317 [inline]
 #1: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bpf_trace_run2+0x1fc/0x540 kernel/trace/bpf_trace.c:2359
 #2: ffff88807bd1c140 (&htab->lockdep_key#21){....}-{2:2}, at: htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
 #3: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:326 [inline]
 #3: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:838 [inline]
 #3: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: __bpf_trace_run kernel/trace/bpf_trace.c:2317 [inline]
 #3: ffffffff8e938320 (rcu_read_lock){....}-{1:2}, at: bpf_trace_run2+0x1fc/0x540 kernel/trace/bpf_trace.c:2359

stack backtrace:
CPU: 1 UID: 0 PID: 5231 Comm: sshd Not tainted 6.11.0-rc4-syzkaller-00326-g5277d130947b #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:93 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
 check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2186
 check_prev_add kernel/locking/lockdep.c:3133 [inline]
 check_prevs_add kernel/locking/lockdep.c:3252 [inline]
 validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3868
 __lock_acquire+0x137a/0x2040 kernel/locking/lockdep.c:5142
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
 htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
 htab_lru_map_delete_elem+0x1f1/0x700 kernel/bpf/hashtab.c:1466
 bpf_prog_6f5f05285f674219+0x43/0x4c
 bpf_dispatcher_nop_func include/linux/bpf.h:1257 [inline]
 __bpf_prog_run include/linux/filter.h:701 [inline]
 bpf_prog_run include/linux/filter.h:708 [inline]
 __bpf_trace_run kernel/trace/bpf_trace.c:2318 [inline]
 bpf_trace_run2+0x2ec/0x540 kernel/trace/bpf_trace.c:2359
 __traceiter_contention_begin+0x7b/0xb0 include/trace/events/lock.h:95
 trace_contention_begin+0x117/0x140 include/trace/events/lock.h:95
 __pv_queued_spin_lock_slowpath+0x114/0xdb0 kernel/locking/qspinlock.c:402
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:584 [inline]
 queued_spin_lock_slowpath+0x42/0x50 arch/x86/include/asm/qspinlock.h:51
 queued_spin_lock include/asm-generic/qspinlock.h:114 [inline]
 do_raw_spin_lock+0x272/0x370 kernel/locking/spinlock_debug.c:116
 htab_lock_bucket+0x1a4/0x370 kernel/bpf/hashtab.c:167
 htab_lru_map_delete_elem+0x1f1/0x700 kernel/bpf/hashtab.c:1466
 </TASK>

Crashes (6):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/09/21 18:25 bpf-next 5277d130947b 6f888b75 .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in htab_lock_bucket
2024/10/27 15:34 net e31a8219fbfc 65e8686b .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce possible deadlock in htab_lock_bucket
2024/09/02 00:16 bpf b408473ea01b 1eda0d14 .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in htab_lock_bucket
2024/12/24 22:41 net-next 9268abe611b0 444551c4 .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce possible deadlock in htab_lock_bucket
2024/12/21 17:21 bpf-next c5d2bac978c5 d7f584ee .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in htab_lock_bucket
2024/08/05 09:53 bpf-next 3d650ab5e7d9 1786a2a8 .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in htab_lock_bucket
* Struck through repros no longer work on HEAD.