syzbot


BUG: soft lockup in aoecmd_cfg (3)

Status: upstream: reported on 2025/04/30 11:13
Subsystems: block
[Documentation on labels]
Reported-by: syzbot+5dfe55156cc098033526@syzkaller.appspotmail.com
First crash: 79d, last: 2d11h
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [block?] BUG: soft lockup in aoecmd_cfg (3) 0 (1) 2025/04/30 11:13
Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in aoecmd_cfg 2 309d 399d 0/3 auto-obsoleted due to no activity on 2024/12/07 10:13
upstream BUG: soft lockup in aoecmd_cfg (2) block 3 214d 246d 0/29 auto-obsoleted due to no activity on 2025/03/02 19:34
upstream BUG: soft lockup in aoecmd_cfg block 1 533d 529d 0/29 auto-obsoleted due to no activity on 2024/04/17 16:28
upstream INFO: rcu detected stall in aoecmd_cfg (2) usb block C done 7 309d 422d 28/29 fixed on 2024/10/22 11:57

Sample crash report:
watchdog: BUG: soft lockup - CPU#0 stuck for 122s! [syz.1.205:6919]
Modules linked in:
irq event stamp: 10452345
hardirqs last  enabled at (10452344): [<ffffffff8b665394>] irqentry_exit+0x74/0x90 kernel/entry/common.c:310
hardirqs last disabled at (10452345): [<ffffffff8b663ede>] sysvec_apic_timer_interrupt+0xe/0xc0 arch/x86/kernel/apic/apic.c:1050
softirqs last  enabled at (10057082): [<ffffffff8185ae7a>] __do_softirq kernel/softirq.c:613 [inline]
softirqs last  enabled at (10057082): [<ffffffff8185ae7a>] invoke_softirq kernel/softirq.c:453 [inline]
softirqs last  enabled at (10057082): [<ffffffff8185ae7a>] __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
softirqs last disabled at (10057085): [<ffffffff8185ae7a>] __do_softirq kernel/softirq.c:613 [inline]
softirqs last disabled at (10057085): [<ffffffff8185ae7a>] invoke_softirq kernel/softirq.c:453 [inline]
softirqs last disabled at (10057085): [<ffffffff8185ae7a>] __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
CPU: 0 UID: 0 PID: 6919 Comm: syz.1.205 Not tainted 6.16.0-rc2-syzkaller-gd4adf1c9ee77 #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/07/2025
RIP: 0010:do_perf_trace_lock_acquire include/trace/events/lock.h:24 [inline]
RIP: 0010:perf_trace_lock_acquire+0x14e/0x410 include/trace/events/lock.h:24
Code: 83 c6 68 4c 89 f0 48 c1 e8 03 48 8b 4c 24 08 48 03 0b 48 89 4c 24 08 42 80 3c 38 00 74 08 4c 89 f7 e8 16 5a 86 00 49 83 3e 00 <75> 25 48 8b 5c 24 08 48 89 d8 48 c1 e8 03 42 80 3c 38 00 74 08 48
RSP: 0018:ffffc90000007320 EFLAGS: 00000282
RAX: 1ffffffff1c01bc9 RBX: ffffffff8dba7c60 RCX: ffffe8ffffc23f90
RDX: 0000000000000000 RSI: ffffffff8be28b60 RDI: ffffffff8be28b20
RBP: ffffc90000007418 R08: 0000000000000002 R09: 0000000000000000
R10: ffffc900000075f8 R11: ffffffff81acf690 R12: 1ffff92000000e6c
R13: 000000000000000d R14: ffffffff8e00de48 R15: dffffc0000000000
FS:  00007fcf21e876c0(0000) GS:ffff888125c51000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcf21e66d58 CR3: 0000000033400000 CR4: 00000000003526f0
DR0: 0000200000000300 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <IRQ>
 __do_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 trace_lock_acquire include/trace/events/lock.h:24 [inline]
 lock_acquire+0x311/0x360 kernel/locking/lockdep.c:5834
 rcu_lock_acquire include/linux/rcupdate.h:331 [inline]
 rcu_read_lock include/linux/rcupdate.h:841 [inline]
 class_rcu_constructor include/linux/rcupdate.h:1155 [inline]
 unwind_next_frame+0xc2/0x2390 arch/x86/kernel/unwind_orc.c:479
 arch_stack_walk+0x11c/0x150 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x9c/0xe0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:47 [inline]
 kasan_save_track+0x3e/0x80 mm/kasan/common.c:68
 unpoison_slab_object mm/kasan/common.c:319 [inline]
 __kasan_slab_alloc+0x6c/0x80 mm/kasan/common.c:345
 kasan_slab_alloc include/linux/kasan.h:250 [inline]
 slab_post_alloc_hook mm/slub.c:4148 [inline]
 slab_alloc_node mm/slub.c:4197 [inline]
 kmem_cache_alloc_node_noprof+0x1bb/0x3c0 mm/slub.c:4249
 kmalloc_reserve+0xbd/0x290 net/core/skbuff.c:579
 __alloc_skb+0x142/0x2d0 net/core/skbuff.c:670
 alloc_skb include/linux/skbuff.h:1336 [inline]
 new_skb+0x2f/0x2b0 drivers/block/aoe/aoecmd.c:66
 aoecmd_cfg_pkts drivers/block/aoe/aoecmd.c:430 [inline]
 aoecmd_cfg+0x28b/0x7c0 drivers/block/aoe/aoecmd.c:1370
 call_timer_fn+0x17e/0x5f0 kernel/time/timer.c:1747
 expire_timers kernel/time/timer.c:1798 [inline]
 __run_timers kernel/time/timer.c:2372 [inline]
 __run_timer_base+0x61a/0x860 kernel/time/timer.c:2384
 run_timer_base kernel/time/timer.c:2393 [inline]
 run_timer_softirq+0xb7/0x180 kernel/time/timer.c:2403
 handle_softirqs+0x286/0x870 kernel/softirq.c:579
 __do_softirq kernel/softirq.c:613 [inline]
 invoke_softirq kernel/softirq.c:453 [inline]
 __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:696
 instr_sysvec_irq_work arch/x86/kernel/irq_work.c:17 [inline]
 sysvec_irq_work+0xa3/0xc0 arch/x86/kernel/irq_work.c:17
 </IRQ>
 <TASK>
 asm_sysvec_irq_work+0x1a/0x20 arch/x86/include/asm/idtentry.h:738
RIP: 0010:rcu_read_unlock_special+0x87/0x4c0 kernel/rcu/tree_plugin.h:694
Code: f1 f1 f1 00 f2 f2 f2 4a 89 04 2b 66 42 c7 44 2b 09 f3 f3 42 c6 44 2b 0b f3 65 44 8b 35 b2 15 f5 10 41 f7 c6 00 00 f0 00 74 49 <48> c7 44 24 40 0e 36 e0 45 4a c7 04 2b 00 00 00 00 66 42 c7 44 2b
RSP: 0018:ffffc900032b7520 EFLAGS: 00000206
RAX: 08ccfbe347308c00 RBX: 1ffff92000656eac RCX: 08ccfbe347308c00
RDX: 0000000000000000 RSI: ffffffff8d9820ce RDI: ffffffff8be28b80
RBP: ffffc900032b7620 R08: ffffffff8fa10df7 R09: 1ffffffff1f421be
R10: dffffc0000000000 R11: fffffbfff1f421bf R12: ffffffff8e144100
R13: dffffc0000000000 R14: 0000000000000246 R15: 0000000000000001
 __rcu_read_unlock+0x84/0xe0 kernel/rcu/tree_plugin.h:438
 rcu_read_unlock include/linux/rcupdate.h:873 [inline]
 trace_call_bpf+0x5ae/0x850 kernel/trace/bpf_trace.c:148
 perf_trace_run_bpf_submit+0x78/0x170 kernel/events/core.c:10866
 do_perf_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 perf_trace_lock_acquire+0x335/0x410 include/trace/events/lock.h:24
 __do_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 trace_lock_acquire include/trace/events/lock.h:24 [inline]
 lock_acquire+0x311/0x360 kernel/locking/lockdep.c:5834
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
 futex_wait_setup+0xd7/0x590 kernel/futex/waitwake.c:625
 __futex_wait+0x148/0x3e0 kernel/futex/waitwake.c:682
 futex_wait+0x104/0x360 kernel/futex/waitwake.c:715
 do_futex+0x333/0x420 kernel/futex/syscalls.c:102
 __do_sys_futex kernel/futex/syscalls.c:179 [inline]
 __se_sys_futex+0x36f/0x400 kernel/futex/syscalls.c:160
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xfa/0x3b0 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fcf20f8e929
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fcf21e870e8 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: ffffffffffffffda RBX: 00007fcf211b5fa8 RCX: 00007fcf20f8e929
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 00007fcf211b5fa8
RBP: 00007fcf211b5fa0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fcf211b5fac
R13: 0000000000000000 R14: 00007ffe8b9d35c0 R15: 00007ffe8b9d36a8
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 6862 Comm: syz.4.197 Not tainted 6.16.0-rc2-syzkaller-gd4adf1c9ee77 #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/07/2025
RIP: 0010:strlen+0x3f/0x70 lib/string.c:420
Code: 00 00 00 00 fc ff df 48 89 fb 49 89 c7 48 89 d8 48 c1 e8 03 42 0f b6 04 30 84 c0 75 11 48 ff c3 49 8d 47 01 42 80 7c 3f 01 00 <75> de eb 19 89 d9 80 e1 07 38 c1 7c e6 49 89 fc 48 89 df e8 29 a6
RSP: 0018:ffffc90000a07d18 EFLAGS: 00000006
RAX: 0000000000000000 RBX: ffffffff8b8bd581 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffffffff8e13eda0 RDI: ffffffff8b8bd580
RBP: ffffc90000a07e40 R08: 0000000000000002 R09: 0000000000000000
R10: ffffc90000a080c0 R11: fffff5200014101f R12: 1ffff92000140fb0
R13: ffffffff8e13eda0 R14: dffffc0000000000 R15: ffffffffffffffff
FS:  0000000000000000(0000) GS:ffff888125d51000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1e3dcd0020 CR3: 000000006dbac000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <IRQ>
 __fortify_strlen include/linux/fortify-string.h:268 [inline]
 trace_event_get_offsets_lock_acquire include/trace/events/lock.h:24 [inline]
 do_perf_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 perf_trace_lock_acquire+0xcb/0x410 include/trace/events/lock.h:24
 __do_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 trace_lock_acquire include/trace/events/lock.h:24 [inline]
 lock_acquire+0x311/0x360 kernel/locking/lockdep.c:5834
 rcu_lock_acquire include/linux/rcupdate.h:331 [inline]
 rcu_read_lock include/linux/rcupdate.h:841 [inline]
 __perf_output_begin kernel/events/ring_buffer.c:167 [inline]
 perf_output_begin_forward+0xcd/0xa80 kernel/events/ring_buffer.c:277
 __perf_event_output kernel/events/core.c:8434 [inline]
 perf_event_output_forward+0x2b2/0x430 kernel/events/core.c:8452
 __perf_event_overflow+0x7cc/0xe10 kernel/events/core.c:10327
 perf_swevent_hrtimer+0x3c5/0x550 kernel/events/core.c:11717
 __run_hrtimer kernel/time/hrtimer.c:1761 [inline]
 __hrtimer_run_queues+0x4e0/0xc60 kernel/time/hrtimer.c:1825
 hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1887
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1039 [inline]
 __sysvec_apic_timer_interrupt+0x10b/0x410 arch/x86/kernel/apic/apic.c:1056
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1050 [inline]
 sysvec_apic_timer_interrupt+0x52/0xc0 arch/x86/kernel/apic/apic.c:1050
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:skb_queue_tail+0x4/0xf0 net/core/skbuff.c:4030
Code: e1 07 80 c1 03 38 c1 7c d8 48 89 df e8 75 45 dc f8 eb ce 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa <55> 41 57 41 56 41 55 41 54 53 50 49 89 f4 48 89 fb 49 bf 00 00 00
RSP: 0018:ffffc90000a08878 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffff88807cb08e40 RCX: dffffc0000000000
RDX: 0000000000000100 RSI: ffff88802fa7cb40 RDI: ffff88807cb09630
RBP: ffffc90000a08970 R08: 0006382e212f1d8b R09: 0000003e334a5829
R10: 0006382e212f1d8b R11: 0000003e334a5829 R12: ffff88802fa7cb40
R13: dffffc0000000000 R14: ffff88802fa7cb40 R15: ffff88802fa7cbb0
 ieee80211_rx_irqsafe+0x55/0xd0 net/mac80211/rx.c:5463
 mac80211_hwsim_rx+0x279/0x630 drivers/net/wireless/virtual/mac80211_hwsim.c:1751
 mac80211_hwsim_tx_frame_no_nl+0xd46/0x11c0 drivers/net/wireless/virtual/mac80211_hwsim.c:1881
 mac80211_hwsim_tx_frame+0x1b5/0x200 drivers/net/wireless/virtual/mac80211_hwsim.c:2222
 mac80211_hwsim_beacon_tx+0x3f0/0x860 drivers/net/wireless/virtual/mac80211_hwsim.c:2323
 __iterate_interfaces+0x2a8/0x590 net/mac80211/util.c:761
 ieee80211_iterate_active_interfaces_atomic+0xdb/0x180 net/mac80211/util.c:797
 mac80211_hwsim_beacon+0xbb/0x1c0 drivers/net/wireless/virtual/mac80211_hwsim.c:2353
 __run_hrtimer kernel/time/hrtimer.c:1761 [inline]
 __hrtimer_run_queues+0x52c/0xc60 kernel/time/hrtimer.c:1825
 hrtimer_run_softirq+0x187/0x2b0 kernel/time/hrtimer.c:1842
 handle_softirqs+0x286/0x870 kernel/softirq.c:579
 __do_softirq kernel/softirq.c:613 [inline]
 invoke_softirq kernel/softirq.c:453 [inline]
 __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:696
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1050 [inline]
 sysvec_apic_timer_interrupt+0xa6/0xc0 arch/x86/kernel/apic/apic.c:1050
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:lookup_page_ext mm/page_ext.c:252 [inline]
RIP: 0010:page_ext_lookup+0xd5/0x180 mm/page_ext.c:520
Code: 83 e3 7f c1 e3 05 49 01 de eb 0f e8 35 73 91 ff eb 05 e8 2e 73 91 ff 45 31 f6 49 83 c6 10 4c 89 f0 48 c1 e8 03 42 80 3c 28 00 <74> 08 4c 89 f7 e8 21 f2 f4 ff 49 8b 1e e8 f9 0b 77 ff 89 c5 31 ff
RSP: 0018:ffffc90003407318 EFLAGS: 00000246
RAX: 1ffff11027fff4de RBX: 00000000000002e0 RCX: ffff88802733da00
RDX: 0000000000000000 RSI: 0000000000001000 RDI: 0000000000000000
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffff822ef9b7
R10: ffffc900034070c0 R11: ffffffffa0201750 R12: 00000000000b868e
R13: dffffc0000000000 R14: ffff88813fffa6f0 R15: 0000000000001000
 page_ext_iter_begin include/linux/page_ext.h:132 [inline]
 page_table_check_clear+0x278/0x700 mm/page_table_check.c:78
 ptep_get_and_clear_full arch/x86/include/asm/jump_label.h:-1 [inline]
 get_and_clear_full_ptes include/linux/pgtable.h:714 [inline]
 zap_present_folio_ptes mm/memory.c:1505 [inline]
 zap_present_ptes mm/memory.c:1590 [inline]
 do_zap_pte_range mm/memory.c:1691 [inline]
 zap_pte_range mm/memory.c:1735 [inline]
 zap_pmd_range mm/memory.c:1827 [inline]
 zap_pud_range mm/memory.c:1856 [inline]
 zap_p4d_range mm/memory.c:1877 [inline]
 unmap_page_range+0x3249/0x41c0 mm/memory.c:1898
 unmap_single_vma mm/memory.c:1941 [inline]
 unmap_vmas+0x399/0x580 mm/memory.c:1985
 exit_mmap+0x248/0xb50 mm/mmap.c:1284
 __mmput+0x118/0x420 kernel/fork.c:1121
 exit_mm+0x1da/0x2c0 kernel/exit.c:581
 do_exit+0x640/0x22e0 kernel/exit.c:943
 do_group_exit+0x21c/0x2d0 kernel/exit.c:1104
 get_signal+0x1286/0x1340 kernel/signal.c:3034
 arch_do_signal_or_restart+0x9a/0x750 arch/x86/kernel/signal.c:337
 exit_to_user_mode_loop+0x75/0x110 kernel/entry/common.c:111
 exit_to_user_mode_prepare include/linux/entry-common.h:330 [inline]
 syscall_exit_to_user_mode_work include/linux/entry-common.h:414 [inline]
 syscall_exit_to_user_mode include/linux/entry-common.h:449 [inline]
 do_syscall_64+0x2bd/0x3b0 arch/x86/entry/syscall_64.c:100
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f975858e929
Code: Unable to access opcode bytes at 0x7f975858e8ff.
RSP: 002b:00007f975936f0e8 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: fffffffffffffe00 RBX: 00007f97587b6168 RCX: 00007f975858e929
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 00007f97587b6168
RBP: 00007f97587b6160 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f97587b616c
R13: 0000000000000000 R14: 00007fff7daca530 R15: 00007fff7daca618
 </TASK>

Crashes (10):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/06/22 19:30 bpf d4adf1c9ee77 d6cdfb8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/02 15:20 bpf-next 212ec9229567 0cd59a8f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/06/23 01:21 bpf-next 99fe8af069a9 d6cdfb8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/05/28 05:36 bpf-next db22b1382b96 874a1386 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/05/27 14:44 bpf-next 079e5c56a5c4 874a1386 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/30 11:12 bpf-next 38d976c32d85 85a5a23f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/30 10:52 bpf-next 38d976c32d85 85a5a23f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/24 13:25 bpf-next 60400cd2b9be 9c80ffa0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/19 00:22 bpf-next 8582d9ab3efd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/16 07:41 bpf-next 7d0b43b68d1c 23b969b7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
* Struck through repros no longer work on HEAD.