INFO: rcu detected stall in neigh_periodic_work (3)

Status: auto-obsoleted due to no activity on 2024/08/09 21:19
Subsystems: net
[Documentation on labels]
First crash: 829d, last: 810d
▶ ▼ Similar bugs (7)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in neigh_periodic_work (5) prio:normal net 1 syz error error 1 213d 209d 0/29 upstream: reported syz repro on 2026/01/02 09:02
linux-5.15 INFO: rcu detected stall in neigh_periodic_work origin:lts-only 1 syz, C error 1 160d 160d 0/3 upstream: reported C repro on 2026/02/20 05:23
upstream INFO: rcu detected stall in neigh_periodic_work (2) net 1 syz, C error 2 872d 1044d 0/29 closed as invalid on 2024/03/13 18:06
upstream INFO: rcu detected stall in neigh_periodic_work (4) net 1 1 454d 454d 0/29 auto-obsoleted due to no activity on 2025/07/31 09:55
upstream BUG: soft lockup in neigh_periodic_work net 1 2 575d 607d 0/29 closed as invalid on 2025/01/28 16:26
upstream INFO: rcu detected stall in neigh_periodic_work net 1 1 1239d 1239d 0/29 auto-obsoleted due to no activity on 2023/06/07 17:40
linux-6.1 INFO: rcu detected stall in neigh_periodic_work origin:upstream 1 syz error 2 3d04h 421d 0/3 upstream: reported syz repro on 2025/06/04 08:32

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (3 ticks this GP) idle=fe7c/1/0x4000000000000000 softirq=108928/108928 fqs=120
rcu: 	(detected by 0, t=10502 jiffies, g=154297, q=557 ncpus=2)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 44 Comm: kworker/1:1 Not tainted 6.9.0-rc7-syzkaller-00183-gcf87f46fd34d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Workqueue: events_power_efficient neigh_periodic_work
RIP: 0010:hlock_class+0x56/0x130 kernel/locking/lockdep.c:228
Code: 20 66 81 e3 ff 1f 0f b7 db be 08 00 00 00 48 89 d8 48 c1 e8 06 48 8d 3c c5 80 51 fe 93 e8 22 20 7e 00 48 0f a3 1d 9a 4f 93 12 <73> 13 48 69 c3 c8 00 00 00 5b 48 05 a0 55 fe 93 c3 cc cc cc cc 48
RSP: 0018:ffffc90000a08ad8 EFLAGS: 00000047
RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffffffff816b01de
RDX: fffffbfff27fca31 RSI: 0000000000000008 RDI: ffffffff93fe5180
RBP: ffffffff93fea618 R08: 0000000000000000 R09: fffffbfff27fca30
R10: ffffffff93fe5187 R11: 0000000000000004 R12: ffffed10033ddc9a
R13: 0000000000000001 R14: ffff888019eee578 R15: 0000000000000002
FS:  0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b2fc4b000 CR3: 000000005f572000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __lock_acquire+0xc5d/0x3b30 kernel/locking/lockdep.c:5134
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0x3a/0x60 kernel/locking/spinlock.c:162
 debug_object_deactivate+0x13c/0x370 lib/debugobjects.c:763
 debug_hrtimer_deactivate kernel/time/hrtimer.c:428 [inline]
 debug_deactivate kernel/time/hrtimer.c:484 [inline]
 __run_hrtimer kernel/time/hrtimer.c:1660 [inline]
 __hrtimer_run_queues+0x47d/0xcc0 kernel/time/hrtimer.c:1756
 hrtimer_interrupt+0x31b/0x800 kernel/time/hrtimer.c:1818
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1032 [inline]
 __sysvec_apic_timer_interrupt+0x10f/0x450 arch/x86/kernel/apic/apic.c:1049
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline]
 sysvec_apic_timer_interrupt+0x90/0xb0 arch/x86/kernel/apic/apic.c:1043
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:__sanitizer_cov_trace_pc+0x58/0x60 kernel/kcov.c:225
Code: 82 f0 15 00 00 83 f8 02 75 20 48 8b 8a f8 15 00 00 8b 92 f4 15 00 00 48 8b 01 48 83 c0 01 48 39 d0 73 07 48 89 01 48 89 34 c1 <c3> cc cc cc cc 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
RSP: 0018:ffffc90000b474a8 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffffffff8fd23920 RCX: ffffffff813d0bf4
RDX: ffff888019eeda00 RSI: ffffffff813d0bfe RDI: 0000000000000006
RBP: ffffffff8fd23938 R08: 0000000000000006 R09: ffffffff81799744
R10: ffffffff81799771 R11: 0000000000000003 R12: ffffffff81799744
R13: ffffffff81799771 R14: dffffc0000000000 R15: ffffffff8fd2392c
 __orc_find+0xce/0x130 arch/x86/kernel/unwind_orc.c:106
 orc_find arch/x86/kernel/unwind_orc.c:227 [inline]
 unwind_next_frame+0x335/0x23a0 arch/x86/kernel/unwind_orc.c:494
 arch_stack_walk+0x100/0x170 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x95/0xd0 kernel/stacktrace.c:122
 kasan_save_stack+0x33/0x60 mm/kasan/common.c:47
 kasan_save_track+0x14/0x30 mm/kasan/common.c:68
 kasan_save_free_info+0x3b/0x60 mm/kasan/generic.c:579
 poison_slab_object mm/kasan/common.c:240 [inline]
 __kasan_slab_free+0x11d/0x1a0 mm/kasan/common.c:256
 kasan_slab_free include/linux/kasan.h:184 [inline]
 slab_free_hook mm/slub.c:2111 [inline]
 slab_free mm/slub.c:4286 [inline]
 kmem_cache_free+0x12e/0x390 mm/slub.c:4350
 kfree_skbmem+0x10e/0x200 net/core/skbuff.c:1159
 __kfree_skb net/core/skbuff.c:1217 [inline]
 consume_skb net/core/skbuff.c:1432 [inline]
 consume_skb+0xdf/0x170 net/core/skbuff.c:1426
 netlink_broadcast_filtered+0x3d5/0xf10 net/netlink/af_netlink.c:1546
 nlmsg_multicast_filtered include/net/netlink.h:1111 [inline]
 nlmsg_multicast include/net/netlink.h:1130 [inline]
 nlmsg_notify+0x9e/0x220 net/netlink/af_netlink.c:2602
 __neigh_notify+0xde/0x160 net/core/neighbour.c:3525
 neigh_cleanup_and_release+0x99/0x2d0 net/core/neighbour.c:101
 neigh_periodic_work+0x6a7/0xc40 net/core/neighbour.c:1005
 process_one_work+0x9a9/0x1ac0 kernel/workqueue.c:3267
 process_scheduled_works kernel/workqueue.c:3348 [inline]
 worker_thread+0x6c8/0xf70 kernel/workqueue.c:3429
 kthread+0x2c1/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: rcu_preempt kthread starved for 8585 jiffies! g154297 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:27856 pid:16    tgid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5409 [inline]
 __schedule+0xf15/0x5d00 kernel/sched/core.c:6746
 __schedule_loop kernel/sched/core.c:6823 [inline]
 schedule+0xe7/0x350 kernel/sched/core.c:6838
 schedule_timeout+0x136/0x2a0 kernel/time/timer.c:2582
 rcu_gp_fqs_loop+0x1eb/0xb00 kernel/rcu/tree.c:1663
 rcu_gp_kthread+0x271/0x380 kernel/rcu/tree.c:1862
 kthread+0x2c1/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 PID: 8541 Comm: kworker/u8:1 Not tainted 6.9.0-rc7-syzkaller-00183-gcf87f46fd34d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:queued_write_lock_slowpath+0x176/0x330 kernel/locking/qrwlock.c:85
Code: 00 00 8b 03 3d 00 01 00 00 74 37 48 b8 00 00 00 00 00 fc ff df 48 89 d9 48 89 da 48 c1 e9 03 83 e2 07 48 01 c1 83 c2 03 f3 90 <0f> b6 01 38 c2 7c 08 84 c0 0f 85 5f 01 00 00 8b 03 3d 00 01 00 00
RSP: 0018:ffffc900000075f8 EFLAGS: 00000206
RAX: 00000000000001ff RBX: ffffffff8f6dc560 RCX: fffffbfff1edb8ac
RDX: 0000000000000003 RSI: 0000000000000004 RDI: ffffffff8f6dc560
RBP: 1ffff92000000ec1 R08: 0000000000000001 R09: fffffbfff1edb8ac
R10: ffffffff8f6dc563 R11: 0000000000000008 R12: ffffffff8f6dc564
R13: 0000000000000003 R14: fffffbfff1edb8ac R15: ffffc90000007638
FS:  0000000000000000(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4891e27440 CR3: 000000000d77a000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 queued_write_lock include/asm-generic/qrwlock.h:101 [inline]
 do_raw_write_lock+0x1d4/0x3a0 kernel/locking/spinlock_debug.c:211
 ___neigh_create+0x9dd/0x2ae0 net/core/neighbour.c:682
 ip6_finish_output2+0x112f/0x18b0 net/ipv6/ip6_output.c:128
 __ip6_finish_output net/ipv6/ip6_output.c:211 [inline]
 ip6_finish_output+0x3f9/0x1300 net/ipv6/ip6_output.c:222
 NF_HOOK_COND include/linux/netfilter.h:303 [inline]
 ip6_output+0x1f8/0x540 net/ipv6/ip6_output.c:243
 dst_output include/net/dst.h:450 [inline]
 NF_HOOK include/linux/netfilter.h:314 [inline]
 ndisc_send_skb+0xa2d/0x1c30 net/ipv6/ndisc.c:509
 ndisc_send_rs+0x12b/0x690 net/ipv6/ndisc.c:719
 addrconf_rs_timer+0x422/0x850 net/ipv6/addrconf.c:4038
 call_timer_fn+0x1a0/0x610 kernel/time/timer.c:1793
 expire_timers kernel/time/timer.c:1844 [inline]
 __run_timers+0x74b/0xaf0 kernel/time/timer.c:2418
 __run_timer_base kernel/time/timer.c:2429 [inline]
 __run_timer_base kernel/time/timer.c:2422 [inline]
 run_timer_base+0x111/0x190 kernel/time/timer.c:2438
 run_timer_softirq+0x1a/0x40 kernel/time/timer.c:2448
 handle_softirqs+0x216/0x8f0 kernel/softirq.c:554
 __do_softirq kernel/softirq.c:588 [inline]
 invoke_softirq kernel/softirq.c:428 [inline]
 __irq_exit_rcu kernel/softirq.c:637 [inline]
 irq_exit_rcu+0xbb/0x120 kernel/softirq.c:649
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline]
 sysvec_apic_timer_interrupt+0x95/0xb0 arch/x86/kernel/apic/apic.c:1043
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:csd_lock_wait kernel/smp.c:311 [inline]
RIP: 0010:smp_call_function_many_cond+0x4e7/0x1420 kernel/smp.c:855
Code: 0c 00 85 ed 74 4d 48 b8 00 00 00 00 00 fc ff df 4d 89 f4 4c 89 f5 49 c1 ec 03 83 e5 07 49 01 c4 83 c5 03 e8 5b 07 0c 00 f3 90 <41> 0f b6 04 24 40 38 c5 7c 08 84 c0 0f 85 f7 0c 00 00 8b 43 08 31
RSP: 0018:ffffc9000327f910 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8880b9544700 RCX: ffffffff8181c58b
RDX: ffff888019bf8000 RSI: ffffffff8181c565 RDI: 0000000000000005
RBP: 0000000000000003 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000006 R12: ffffed10172a88e1
R13: 0000000000000001 R14: ffff8880b9544708 R15: ffff8880b943fc00
 on_each_cpu_cond_mask+0x40/0x90 kernel/smp.c:1023
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2086 [inline]
 text_poke_bp_batch+0x22b/0x760 arch/x86/kernel/alternative.c:2296
 text_poke_flush arch/x86/kernel/alternative.c:2487 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:2484 [inline]
 text_poke_finish+0x30/0x40 arch/x86/kernel/alternative.c:2494
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x1d7/0x400 kernel/jump_label.c:829
 static_key_enable_cpuslocked+0x1b7/0x270 kernel/jump_label.c:205
 static_key_enable+0x1a/0x20 kernel/jump_label.c:218
 toggle_allocation_gate mm/kfence/core.c:826 [inline]
 toggle_allocation_gate+0xf8/0x250 mm/kfence/core.c:818
 process_one_work+0x9a9/0x1ac0 kernel/workqueue.c:3267
 process_scheduled_works kernel/workqueue.c:3348 [inline]
 worker_thread+0x6c8/0xf70 kernel/workqueue.c:3429
 kthread+0x2c1/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/11 21:09 upstream cf87f46fd34d 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in neigh_periodic_work
2024/04/22 13:51 upstream ed30a4a51bb1 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in neigh_periodic_work
* Struck through repros no longer work on HEAD.