INFO: rcu detected stall in nsim_dev_trap_report_work (3)

Status: upstream: reported C repro on 2025/08/20 06:57
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+adf3d9d216d554ccac29@syzkaller.appspotmail.com
First crash: 337d, last: 15d
▶ ▼ Bug presence (2)
Date Name Commit Repro Result
2025/12/08 linux-5.15.y (ToT) 68efe5a6c16a C [report] INFO: rcu detected stall in corrupted
2025/12/08 upstream (ToT) c2f2b01b74be C Didn't crash
▶ ▼ Similar bugs (8)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in nsim_dev_trap_report_work 1 2 748d 770d 0/3 auto-obsoleted due to no activity on 2024/10/13 18:52
linux-5.15 INFO: rcu detected stall in nsim_dev_trap_report_work (2) 1 1 535d 535d 0/3 auto-obsoleted due to no activity on 2025/05/14 15:54
upstream INFO: rcu detected stall in nsim_dev_trap_report_work (2) net 1 syz done 17 507d 995d 0/29 auto-obsoleted due to no activity on 2025/09/25 13:43
upstream INFO: rcu detected stall in nsim_dev_trap_report_work net 1 syz 4 1050d 1093d 23/29 fixed on 2023/10/12 12:48
linux-6.1 INFO: rcu detected stall in nsim_dev_trap_report_work (3) 1 1 33d 33d 0/3 upstream: reported on 2026/06/20 12:13
upstream INFO: rcu detected stall in nsim_dev_trap_report_work (3) net 1 2 220d 220d 0/29 auto-obsoleted due to no activity on 2026/03/15 13:28
linux-6.1 INFO: rcu detected stall in nsim_dev_trap_report_work 1 2 800d 845d 0/3 auto-obsoleted due to no activity on 2024/08/22 01:13
linux-6.1 INFO: rcu detected stall in nsim_dev_trap_report_work (2) 1 1 454d 454d 0/3 auto-obsoleted due to no activity on 2025/08/03 09:20
▶ ▼ Last patch testing requests (3)
Created Duration User Patch Repo Result
2026/05/25 04:11 26m retest repro linux-5.15.y report log
2026/02/28 20:22 14m retest repro linux-5.15.y report log
2025/12/06 16:49 54m retest repro linux-5.15.y report log
▶ ▼ Fix bisection attempts (5)
Created Duration User Patch Repo Result
2026/05/25 02:40 14m fix candidate upstream error job log
2026/04/13 14:52 9m fix candidate upstream error job log
2026/03/14 08:34 1m fix candidate upstream error job log
2026/02/12 04:36 1m fix candidate upstream error job log
2026/01/13 00:08 20m fix candidate upstream error job log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 ticks this GP) idle=e2f/1/0x4000000000000002 softirq=21889/21889 fqs=0 
	(detected by 1, t=10502 jiffies, g=36977, q=195)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 4309 Comm: kworker/0:17 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025
Workqueue: events nsim_dev_trap_report_work
RIP: 0010:check_kcov_mode kernel/kcov.c:172 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x13/0x60 kernel/kcov.c:206
Code: 48 8b 3d 50 d3 f1 0b 48 89 de 5b e9 57 9e 43 00 00 00 cc cc 00 00 cc 48 8b 04 24 65 48 8b 0d a4 a3 8a 7e 65 8b 15 a5 a3 8a 7e <81> e2 00 01 ff 00 74 11 81 fa 00 01 00 00 75 35 83 b9 34 16 00 00
RSP: 0018:ffffc90000006fa0 EFLAGS: 00000046
RAX: ffffffff810eaa56 RBX: ffff8880b90351c8 RCX: ffff8880296bbb80
RDX: 0000000000000104 RSI: 0000000000000001 RDI: 0000000000000000
RBP: 0000000000000001 R08: dffffc0000000000 R09: ffffc90000007120
R10: fffff52000000e2b R11: 1ffff92000000e24 R12: dffffc0000000000
R13: ffffc90000007186 R14: ffffffff8bf23a40 R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880b9000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b33463fff CR3: 0000000029b99000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 kvm_is_in_guest+0x6/0x30 arch/x86/kvm/x86.c:8595
 perf_misc_flags+0x8f/0x1b0 arch/x86/events/core.c:2964
 perf_prepare_sample+0x132/0x1cb0 kernel/events/core.c:7622
 __perf_event_output kernel/events/core.c:7802 [inline]
 perf_event_output_forward+0x146/0x280 kernel/events/core.c:7822
 __perf_event_overflow+0x364/0x530 kernel/events/core.c:9515
 perf_swevent_overflow kernel/events/core.c:9591 [inline]
 perf_swevent_event+0x4ad/0x530 kernel/events/core.c:9629
 perf_tp_event+0x32c/0xb50 kernel/events/core.c:10047
 perf_trace_run_bpf_submit+0xf3/0x1c0 kernel/events/core.c:10021
 perf_trace_preemptirq_template+0x2be/0x380 include/trace/events/preemptirq.h:14
 trace_irq_enable_rcuidle+0x108/0x180 include/trace/events/preemptirq.h:40
 trace_hardirqs_on+0x57/0x80 kernel/trace/trace_preemptirq.c:44
 asm_sysvec_irq_work+0x16/0x20 arch/x86/include/asm/idtentry.h:713
RIP: 0010:rcu_read_unlock_special+0x84/0x480 kernel/rcu/tree_plugin.h:674
Code: f1 f1 f1 00 f2 f2 f2 4a 89 04 2b 66 42 c7 44 2b 09 f3 f3 42 c6 44 2b 0b f3 65 44 8b 35 ed 11 9e 7e 41 f7 c6 00 00 f0 00 74 45 <48> c7 44 24 20 0e 36 e0 45 4a c7 04 2b 00 00 00 00 66 42 c7 44 2b
RSP: 0018:ffffc90000007780 EFLAGS: 00000206
RAX: 4b3c6d95c972ca00 RBX: 1ffff92000000ef4 RCX: 4b3c6d95c972ca00
RDX: dffffc0000000000 RSI: ffffffff8a0b1c60 RDI: ffffffff8a59e580
RBP: ffffc90000007860 R08: dffffc0000000000 R09: fffffbfff1ff7c32
R10: fffffbfff1ff7c32 R11: 1ffffffff1ff7c31 R12: ffffffff8c120c00
R13: dffffc0000000000 R14: 0000000000000246 R15: 0000000000000001
 __rcu_read_unlock+0x78/0xd0 kernel/rcu/tree_plugin.h:422
 rcu_read_unlock include/linux/rcupdate.h:772 [inline]
 __ip_queue_xmit+0x1244/0x1c00 net/ipv4/ip_output.c:533
 __tcp_transmit_skb+0x1ced/0x31c0 net/ipv4/tcp_output.c:1404
 tcp_delack_timer_handler+0x32e/0x730 net/ipv4/tcp_timer.c:314
 tcp_delack_timer+0x188/0x2e0 net/ipv4/tcp_timer.c:341
 call_timer_fn+0x16c/0x530 kernel/time/timer.c:1451
 expire_timers kernel/time/timer.c:1496 [inline]
 __run_timers+0x525/0x7c0 kernel/time/timer.c:1767
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:1780
 handle_softirqs+0x328/0x820 kernel/softirq.c:576
 __do_softirq kernel/softirq.c:610 [inline]
 invoke_softirq kernel/softirq.c:450 [inline]
 __irq_exit_rcu+0x12f/0x220 kernel/softirq.c:659
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:671
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1108 [inline]
 sysvec_apic_timer_interrupt+0xa0/0xc0 arch/x86/kernel/apic/apic.c:1108
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:676
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:161 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xa5/0x100 kernel/locking/spinlock.c:194
Code: 74 05 e8 de 0d cb f7 48 c7 44 24 20 00 00 00 00 9c 8f 44 24 20 f6 44 24 21 02 75 4b f7 c3 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> f6 95 9e f7 65 8b 05 87 93 4f 76 85 c0 74 3c 48 c7 04 24 0e 36
RSP: 0018:ffffc900031ef8c0 EFLAGS: 00000206
RAX: 4b3c6d95c972ca00 RBX: 0000000000000a06 RCX: 4b3c6d95c972ca00
RDX: dffffc0000000000 RSI: ffffffff8a0b1c60 RDI: 0000000000000001
RBP: ffffc900031ef948 R08: dffffc0000000000 R09: fffffbfff1ff7c3b
R10: fffffbfff1ff7c3b R11: 1ffffffff1ff7c3a R12: dffffc0000000000
R13: ffff888075c32000 R14: ffffffff96301880 R15: 1ffff9200063df18
 __debug_check_no_obj_freed lib/debugobjects.c:982 [inline]
 debug_check_no_obj_freed+0x515/0x530 lib/debugobjects.c:1003
 slab_free_hook mm/slub.c:1685 [inline]
 slab_free_freelist_hook+0x8b/0x170 mm/slub.c:1736
 slab_free mm/slub.c:3504 [inline]
 kfree+0xef/0x2a0 mm/slub.c:4564
 skb_free_head net/core/skbuff.c:655 [inline]
 skb_release_data+0x6fe/0x850 net/core/skbuff.c:677
 skb_release_all net/core/skbuff.c:742 [inline]
 __kfree_skb net/core/skbuff.c:756 [inline]
 consume_skb+0xa2/0x100 net/core/skbuff.c:914
 nsim_dev_trap_report drivers/net/netdevsim/dev.c:737 [inline]
 nsim_dev_trap_report_work+0x7cb/0xb40 drivers/net/netdevsim/dev.c:762
 process_one_work+0x863/0x1000 kernel/workqueue.c:2310
 worker_thread+0xaa8/0x12a0 kernel/workqueue.c:2457
 kthread+0x436/0x520 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>
rcu: rcu_preempt kthread timer wakeup didn't happen for 10501 jiffies! g36977 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=0 timer-softirq=19856
rcu: rcu_preempt kthread starved for 10502 jiffies! g36977 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:27328 pid:   15 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5049 [inline]
 __schedule+0x11bb/0x4390 kernel/sched/core.c:6395
 schedule+0x11b/0x1e0 kernel/sched/core.c:6478
 schedule_timeout+0x15c/0x280 kernel/time/timer.c:1914
 rcu_gp_fqs_loop+0x29e/0x11b0 kernel/rcu/tree.c:1972
 rcu_gp_kthread+0x98/0x350 kernel/rcu/tree.c:2145
 kthread+0x436/0x520 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 4309 Comm: kworker/0:17 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025
Workqueue: events nsim_dev_trap_report_work
RIP: 0010:rcu_dynticks_curr_cpu_in_eqs kernel/rcu/tree.c:331 [inline]
RIP: 0010:rcu_is_watching+0x36/0xa0 kernel/rcu/tree.c:1123
Code: e8 9f cc 45 08 89 c3 83 f8 08 73 5c 49 bf 00 00 00 00 00 fc ff df 4c 8d 34 dd 20 c8 bc 8b 4c 89 f0 48 c1 e8 03 42 80 3c 38 00 <74> 08 4c 89 f7 e8 c0 82 58 00 48 c7 c3 08 b3 03 00 49 03 1e 48 89
RSP: 0018:ffffc900000070c0 EFLAGS: 00000046
RAX: 1ffffffff1779904 RBX: 0000000000000000 RCX: 4b3c6d95c972ca00
RDX: 0000000000000100 RSI: ffffffff8a59e560 RDI: ffffffff8a59e520
RBP: ffffc900000071f8 R08: dffffc0000000000 R09: fffffbfff1ad33a6
R10: fffffbfff1ad33a6 R11: 1ffffffff1ad33a5 R12: 00000000ffffffe4
R13: 1ffff92000000e20 R14: ffffffff8bbcc820 R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880b9000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b33463fff CR3: 0000000029b99000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 rcu_read_unlock include/linux/rcupdate.h:769 [inline]
 __perf_event_output kernel/events/core.c:7813 [inline]
 perf_event_output_forward+0x1b0/0x280 kernel/events/core.c:7822
 __perf_event_overflow+0x364/0x530 kernel/events/core.c:9515
 perf_swevent_overflow kernel/events/core.c:9591 [inline]
 perf_swevent_event+0x4ad/0x530 kernel/events/core.c:9629
 perf_tp_event+0x32c/0xb50 kernel/events/core.c:10047
 perf_trace_run_bpf_submit+0xf3/0x1c0 kernel/events/core.c:10021
 perf_trace_preemptirq_template+0x2be/0x380 include/trace/events/preemptirq.h:14
 trace_irq_enable_rcuidle+0x108/0x180 include/trace/events/preemptirq.h:40
 trace_hardirqs_on+0x57/0x80 kernel/trace/trace_preemptirq.c:44
 asm_sysvec_irq_work+0x16/0x20 arch/x86/include/asm/idtentry.h:713
RIP: 0010:rcu_read_unlock_special+0x84/0x480 kernel/rcu/tree_plugin.h:674
Code: f1 f1 f1 00 f2 f2 f2 4a 89 04 2b 66 42 c7 44 2b 09 f3 f3 42 c6 44 2b 0b f3 65 44 8b 35 ed 11 9e 7e 41 f7 c6 00 00 f0 00 74 45 <48> c7 44 24 20 0e 36 e0 45 4a c7 04 2b 00 00 00 00 66 42 c7 44 2b
RSP: 0018:ffffc90000007780 EFLAGS: 00000206
RAX: 4b3c6d95c972ca00 RBX: 1ffff92000000ef4 RCX: 4b3c6d95c972ca00
RDX: dffffc0000000000 RSI: ffffffff8a0b1c60 RDI: ffffffff8a59e580
RBP: ffffc90000007860 R08: dffffc0000000000 R09: fffffbfff1ff7c32
R10: fffffbfff1ff7c32 R11: 1ffffffff1ff7c31 R12: ffffffff8c120c00
R13: dffffc0000000000 R14: 0000000000000246 R15: 0000000000000001
 __rcu_read_unlock+0x78/0xd0 kernel/rcu/tree_plugin.h:422
 rcu_read_unlock include/linux/rcupdate.h:772 [inline]
 __ip_queue_xmit+0x1244/0x1c00 net/ipv4/ip_output.c:533
 __tcp_transmit_skb+0x1ced/0x31c0 net/ipv4/tcp_output.c:1404
 tcp_delack_timer_handler+0x32e/0x730 net/ipv4/tcp_timer.c:314
 tcp_delack_timer+0x188/0x2e0 net/ipv4/tcp_timer.c:341
 call_timer_fn+0x16c/0x530 kernel/time/timer.c:1451
 expire_timers kernel/time/timer.c:1496 [inline]
 __run_timers+0x525/0x7c0 kernel/time/timer.c:1767
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:1780
 handle_softirqs+0x328/0x820 kernel/softirq.c:576
 __do_softirq kernel/softirq.c:610 [inline]
 invoke_softirq kernel/softirq.c:450 [inline]
 __irq_exit_rcu+0x12f/0x220 kernel/softirq.c:659
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:671
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1108 [inline]
 sysvec_apic_timer_interrupt+0xa0/0xc0 arch/x86/kernel/apic/apic.c:1108
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:676
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:161 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xa5/0x100 kernel/locking/spinlock.c:194
Code: 74 05 e8 de 0d cb f7 48 c7 44 24 20 00 00 00 00 9c 8f 44 24 20 f6 44 24 21 02 75 4b f7 c3 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> f6 95 9e f7 65 8b 05 87 93 4f 76 85 c0 74 3c 48 c7 04 24 0e 36
RSP: 0018:ffffc900031ef8c0 EFLAGS: 00000206
RAX: 4b3c6d95c972ca00 RBX: 0000000000000a06 RCX: 4b3c6d95c972ca00
RDX: dffffc0000000000 RSI: ffffffff8a0b1c60 RDI: 0000000000000001
RBP: ffffc900031ef948 R08: dffffc0000000000 R09: fffffbfff1ff7c3b
R10: fffffbfff1ff7c3b R11: 1ffffffff1ff7c3a R12: dffffc0000000000
R13: ffff888075c32000 R14: ffffffff96301880 R15: 1ffff9200063df18
 __debug_check_no_obj_freed lib/debugobjects.c:982 [inline]
 debug_check_no_obj_freed+0x515/0x530 lib/debugobjects.c:1003
 slab_free_hook mm/slub.c:1685 [inline]
 slab_free_freelist_hook+0x8b/0x170 mm/slub.c:1736
 slab_free mm/slub.c:3504 [inline]
 kfree+0xef/0x2a0 mm/slub.c:4564
 skb_free_head net/core/skbuff.c:655 [inline]
 skb_release_data+0x6fe/0x850 net/core/skbuff.c:677
 skb_release_all net/core/skbuff.c:742 [inline]
 __kfree_skb net/core/skbuff.c:756 [inline]
 consume_skb+0xa2/0x100 net/core/skbuff.c:914
 nsim_dev_trap_report drivers/net/netdevsim/dev.c:737 [inline]
 nsim_dev_trap_report_work+0x7cb/0xb40 drivers/net/netdevsim/dev.c:762
 process_one_work+0x863/0x1000 kernel/workqueue.c:2310
 worker_thread+0xaa8/0x12a0 kernel/workqueue.c:2457
 kthread+0x436/0x520 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>

Crashes (14):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/11/22 09:28 linux-5.15.y cc5ec8769306 4fb8ef37 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2026/07/08 05:05 linux-5.15.y c86c4726e7f0 76cdc0c0 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2026/05/10 15:23 linux-5.15.y de8dfb3f0278 29233ece .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2026/05/02 23:43 linux-5.15.y ef251c45f1cd a0d91488 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2026/04/22 04:19 linux-5.15.y b9d57c40a767 0b6ab7ec .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2026/02/14 19:00 linux-5.15.y e45d5d41c134 1e62d198 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2025/12/31 04:22 linux-5.15.y 68efe5a6c16a d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2025/12/28 21:00 linux-5.15.y 68efe5a6c16a d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2025/12/25 10:50 linux-5.15.y 68efe5a6c16a d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2025/12/25 08:54 linux-5.15.y 68efe5a6c16a d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2025/12/08 00:30 linux-5.15.y 68efe5a6c16a d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in nsim_dev_trap_report_work
2025/11/17 17:55 linux-5.15.y cc5ec8769306 ef766cd7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in nsim_dev_trap_report_work
2025/10/21 01:13 linux-5.15.y ac56c046adf4 d422939c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in nsim_dev_trap_report_work
2025/08/20 06:56 linux-5.15.y c79648372d02 79512909 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in nsim_dev_trap_report_work
* Struck through repros no longer work on HEAD.