INFO: rcu detected stall in try_check_zero (4)

Status: auto-obsoleted due to no activity on 2023/09/02 21:24
Subsystems: kernel
[Documentation on labels]
First crash: 1234d, last: 1167d
▶ ▼ Similar bugs (6)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in try_check_zero kernel 1 66 2289d 2732d 0/29 auto-closed as invalid on 2020/09/05 16:29
upstream INFO: rcu detected stall in try_check_zero (5) kernel 1 syz error error 2 214d 590d 0/29 upstream: reported syz repro on 2025/01/02 10:34
linux-6.1 INFO: rcu detected stall in try_check_zero (2) origin:lts-only 1 syz, C error 1 180d 180d 0/3 upstream: reported C repro on 2026/02/16 13:40
upstream INFO: rcu detected stall in try_check_zero (2) kernel 1 1 2159d 2159d 0/29 auto-closed as invalid on 2020/12/15 14:13
upstream INFO: rcu detected stall in try_check_zero (3) kernel 1 2 1675d 1678d 0/29 closed as invalid on 2022/02/08 10:10
linux-6.1 INFO: rcu detected stall in try_check_zero 1 1 637d 637d 0/3 auto-obsoleted due to no activity on 2025/02/23 21:01

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	0-...!: (10500 ticks this GP) idle=ad04/1/0x4000000000000000 softirq=104415/104415 fqs=1
rcu: 	(t=10502 jiffies g=136189 q=74 ncpus=2)
rcu: rcu_preempt kthread timer wakeup didn't happen for 10500 jiffies! g136189 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=1 timer-softirq=180863
rcu: rcu_preempt kthread starved for 10503 jiffies! g136189 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:28808 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5343 [inline]
 __schedule+0xc9a/0x5880 kernel/sched/core.c:6669
 schedule+0xde/0x1a0 kernel/sched/core.c:6745
 schedule_timeout+0x14e/0x2b0 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x190/0x910 kernel/rcu/tree.c:1609
 rcu_gp_kthread+0x23a/0x360 kernel/rcu/tree.c:1808
 kthread+0x344/0x440 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 19677 Comm: kworker/1:3 Not tainted 6.4.0-rc4-syzkaller-00371-g6f64a5ebe1dc #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/25/2023
Workqueue: rcu_gp process_srcu
RIP: 0010:kvm_wait+0xa8/0x110 arch/x86/kernel/kvm.c:1064
Code: fa 83 e2 07 38 d0 7f 04 84 c0 75 66 0f b6 07 40 38 c6 74 1b 48 83 c4 10 c3 c3 e8 53 9f 52 00 eb 07 0f 00 2d 0a 47 11 09 fb f4 <48> 83 c4 10 c3 eb 07 0f 00 2d fa 46 11 09 f4 48 83 c4 10 c3 89 74
RSP: 0018:ffffc900001e0b48 EFLAGS: 00000202
RAX: 0000000000089fa8 RBX: 0000000000000000 RCX: 1ffffffff22ae42e
RDX: 0000000000000000 RSI: 0000000000000102 RDI: 0000000000000000
RBP: ffff88807e70c0f0 R08: 0000000000000001 R09: ffffffff91529dc7
R10: 0000000000000001 R11: 0000000000094001 R12: 0000000000000000
R13: ffffed100fce181e R14: 0000000000000001 R15: ffff8880b993d440
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000002030c030 CR3: 0000000031c88000 CR4: 0000000000350ee0
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 pv_wait arch/x86/include/asm/paravirt.h:598 [inline]
 pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:470 [inline]
 __pv_queued_spin_lock_slowpath+0x8cb/0xb50 kernel/locking/qspinlock.c:511
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:586 [inline]
 queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:51 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:114 [inline]
 do_raw_spin_lock+0x204/0x2b0 kernel/locking/spinlock_debug.c:115
 spin_lock include/linux/spinlock.h:350 [inline]
 fq_pie_timer+0x96/0x2a0 net/sched/sch_fq_pie.c:377
 call_timer_fn+0x1a0/0x580 kernel/time/timer.c:1700
 expire_timers+0x29b/0x4b0 kernel/time/timer.c:1751
 __run_timers kernel/time/timer.c:2022 [inline]
 __run_timers kernel/time/timer.c:1995 [inline]
 run_timer_softirq+0x326/0x910 kernel/time/timer.c:2035
 __do_softirq+0x1d4/0x905 kernel/softirq.c:571
 invoke_softirq kernel/softirq.c:445 [inline]
 __irq_exit_rcu+0x114/0x190 kernel/softirq.c:650
 irq_exit_rcu+0x9/0x20 kernel/softirq.c:662
 sysvec_apic_timer_interrupt+0x97/0xc0 arch/x86/kernel/apic/apic.c:1106
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:rdtsc_ordered arch/x86/include/asm/msr.h:215 [inline]
RIP: 0010:delay_tsc+0x49/0xb0 arch/x86/lib/delay.c:72
Code: 0f 01 f9 66 90 48 c1 e2 20 48 09 c2 48 89 d5 eb 16 f3 90 bf 01 00 00 00 e8 d4 a0 53 f7 e8 0f 3d 0b 00 44 39 e0 75 36 0f 01 f9 <66> 90 48 c1 e2 20 48 89 d3 48 09 c3 48 89 d8 48 29 e8 4c 39 e8 73
RSP: 0018:ffffc9000448fb98 EFLAGS: 00000246
RAX: 000000003dae95d3 RBX: 000005ff3dae9562 RCX: 0000000000000001
RDX: 00000000000005ff RSI: 0000000000000001 RDI: 0000000000000000
RBP: 000005ff3dae7d7a R08: 0000000000000000 R09: ffff8880b9945e0f
R10: ffffed1017328bc1 R11: 0000000000094001 R12: 0000000000000001
R13: 0000000000002bf3 R14: ffffffff91e71be0 R15: ffff8880b9945e08
 try_check_zero+0xe1/0x4b0 kernel/rcu/srcutree.c:1091
 srcu_advance_state kernel/rcu/srcutree.c:1666 [inline]
 process_srcu+0x58d/0x1710 kernel/rcu/srcutree.c:1769
 process_one_work+0x99a/0x15e0 kernel/workqueue.c:2405
 worker_thread+0x67d/0x10c0 kernel/workqueue.c:2552
 kthread+0x344/0x440 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
CPU: 0 PID: 17375 Comm: kworker/u4:25 Not tainted 6.4.0-rc4-syzkaller-00371-g6f64a5ebe1dc #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/25/2023
Workqueue: netns cleanup_net
RIP: 0010:__list_del_entry include/linux/list.h:134 [inline]
RIP: 0010:list_move_tail include/linux/list.h:229 [inline]
RIP: 0010:fq_pie_qdisc_dequeue+0x672/0x9c0 net/sched/sch_fq_pie.c:248
Code: 03 0f 8e 06 03 00 00 8b 85 28 03 00 00 48 89 df 44 01 f8 89 43 f0 e8 5d 3c d8 fb 31 ff 41 89 c5 89 c6 e8 61 26 45 f9 45 84 ed <74> 64 e8 77 2a 45 f9 48 89 d8 48 c1 e8 03 42 80 3c 20 00 0f 85 e8
RSP: 0018:ffffc90000007620 EFLAGS: 00000202
RAX: 0000000000000000 RBX: ffff88809efa6b80 RCX: ffffffff883f1bbf
RDX: ffff88803832d940 RSI: 0000000000000100 RDI: 0000000000000001
RBP: ffff88807e70c000 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0200000000000000 R12: dffffc0000000000
R13: 0000000000000001 R14: ffff88807e70c2f0 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0edd46bfc0 CR3: 000000003875c000 CR4: 0000000000350ef0
Call Trace:
 <IRQ>
 dequeue_skb net/sched/sch_generic.c:292 [inline]
 qdisc_restart net/sched/sch_generic.c:397 [inline]
 __qdisc_run+0x1b2/0x1780 net/sched/sch_generic.c:415
 __dev_xmit_skb net/core/dev.c:3868 [inline]
 __dev_queue_xmit+0x2215/0x3b10 net/core/dev.c:4210
 dev_queue_xmit include/linux/netdevice.h:3085 [inline]
 neigh_connected_output+0x3c2/0x550 net/core/neighbour.c:1581
 neigh_output include/net/neighbour.h:544 [inline]
 ip6_finish_output2+0x55a/0x1560 net/ipv6/ip6_output.c:134
 __ip6_finish_output net/ipv6/ip6_output.c:195 [inline]
 ip6_finish_output+0x69a/0x1170 net/ipv6/ip6_output.c:206
 NF_HOOK_COND include/linux/netfilter.h:292 [inline]
 ip6_output+0x1f1/0x540 net/ipv6/ip6_output.c:227
 dst_output include/net/dst.h:458 [inline]
 NF_HOOK include/linux/netfilter.h:303 [inline]
 ndisc_send_skb+0xa63/0x1850 net/ipv6/ndisc.c:508
 ndisc_send_rs+0x132/0x6f0 net/ipv6/ndisc.c:718
 addrconf_rs_timer+0x3f1/0x870 net/ipv6/addrconf.c:3936
 call_timer_fn+0x1a0/0x580 kernel/time/timer.c:1700
 expire_timers+0x29b/0x4b0 kernel/time/timer.c:1751
 __run_timers kernel/time/timer.c:2022 [inline]
 __run_timers kernel/time/timer.c:1995 [inline]
 run_timer_softirq+0x326/0x910 kernel/time/timer.c:2035
 __do_softirq+0x1d4/0x905 kernel/softirq.c:571
 invoke_softirq kernel/softirq.c:445 [inline]
 __irq_exit_rcu+0x114/0x190 kernel/softirq.c:650
 irq_exit_rcu+0x9/0x20 kernel/softirq.c:662
 sysvec_apic_timer_interrupt+0x97/0xc0 arch/x86/kernel/apic/apic.c:1106
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:lock_acquire+0x1f5/0x520 kernel/locking/lockdep.c:5673
Code: 97 9c 7e 83 f8 01 0f 85 b9 02 00 00 9c 58 f6 c4 02 0f 85 a4 02 00 00 48 83 7c 24 08 00 74 01 fb 48 b8 00 00 00 00 00 fc ff df <48> 01 c3 48 c7 03 00 00 00 00 48 c7 43 08 00 00 00 00 48 8b 84 24
RSP: 0018:ffffc90006847a38 EFLAGS: 00000206
RAX: dffffc0000000000 RBX: 1ffff92000d08f49 RCX: b3110f0433bb069c
RDX: 1ffff11007065c80 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: ffffffff91529d07
R10: fffffbfff22a53a0 R11: 1ffffffff18f47d1 R12: 0000000000000002
R13: 0000000000000000 R14: ffffffff8c7990c0 R15: 0000000000000000
 rcu_lock_acquire include/linux/rcupdate.h:327 [inline]
 rcu_read_lock include/linux/rcupdate.h:773 [inline]
 inet_twsk_purge+0x132/0x900 net/ipv4/inet_timewait_sock.c:292
 ops_exit_list+0x125/0x170 net/core/net_namespace.c:175
 cleanup_net+0x4ee/0xb10 net/core/net_namespace.c:614
 process_one_work+0x99a/0x15e0 kernel/workqueue.c:2405
 worker_thread+0x67d/0x10c0 kernel/workqueue.c:2552
 kthread+0x344/0x440 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/06/04 21:21 upstream 6f64a5ebe1dc a4ae4f42 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in try_check_zero
2023/03/29 23:40 upstream ffe78bbd5121 f325deb0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in try_check_zero
* Struck through repros no longer work on HEAD.