syzbot


INFO: task hung in worker_thread (5)

Status: upstream: reported C repro on 2024/09/30 12:46
Subsystems: kernel
[Documentation on labels]
Reported-by: syzbot+c27cd2da6c2dc003b4a1@syzkaller.appspotmail.com
First crash: 471d, last: 11h03m
Cause bisection: introduced by (bisect log) :
commit d15121be7485655129101f3960ae6add40204463
Author: Paolo Abeni <pabeni@redhat.com>
Date: Mon May 8 06:17:44 2023 +0000

  Revert "softirq: Let ksoftirqd do its job"

Crash: no output from test machine (log)
Repro: C syz .config
  
Fix bisection: fixed by (bisect log) :
commit 66951e4860d3c688bfa550ea4a19635b57e00eca
Author: Peter Zijlstra <peterz@infradead.org>
Date: Mon Jan 13 12:50:11 2025 +0000

  sched/fair: Fix update_cfs_group() vs DELAY_DEQUEUE

  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [kernel?] INFO: task hung in worker_thread (5) 0 (2) 2025/05/16 13:12
Similar bugs (7)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 INFO: task hung in worker_thread 1 1 1321d 1321d 0/1 auto-closed as invalid on 2022/05/08 06:12
upstream INFO: task hung in worker_thread (3) serial 1 1 914d 914d 0/29 auto-obsoleted due to no activity on 2023/05/20 16:01
upstream INFO: task hung in worker_thread (4) kernel 1 1 598d 598d 0/29 auto-obsoleted due to no activity on 2024/03/31 02:07
android-6-12 INFO: task hung in worker_thread 1 2 40d 48d 0/1 premoderation: reported on 2025/07/04 05:27
upstream INFO: task hung in worker_thread (2) fs 1 1 1674d 1674d 0/29 auto-closed as invalid on 2021/05/17 11:26
linux-5.15 INFO: task hung in worker_thread 1 1 290d 290d 0/3 auto-obsoleted due to no activity on 2025/02/12 21:25
upstream INFO: task hung in worker_thread fs 1 1 1839d 1839d 0/29 auto-closed as invalid on 2020/11/06 03:23
Last patch testing requests (2)
Created Duration User Patch Repo Result
2025/03/30 01:36 35m retest repro upstream report log
2024/11/16 06:51 25m retest repro upstream report log
Fix bisection attempts (4)
Created Duration User Patch Repo Result
2025/05/16 03:30 9h40m bisect fix upstream OK (1) job log
2025/04/11 13:59 3h04m bisect fix upstream OK (0) job log log
2025/01/18 20:55 4h38m bisect fix upstream OK (0) job log log
2024/12/19 13:14 3h01m bisect fix upstream OK (0) job log log

Sample crash report:
INFO: task kworker/u8:0:12 blocked for more than 159 seconds.
      Not tainted 6.16.0-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u8:0    state:I stack:22824 pid:12    tgid:12    ppid:2      task_flags:0x4208060 flags:0x00004000
Workqueue:  0x0 (bat_events)
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5397 [inline]
 __schedule+0x16aa/0x4c90 kernel/sched/core.c:6786
 __schedule_loop kernel/sched/core.c:6864 [inline]
 schedule+0x165/0x360 kernel/sched/core.c:6879
 worker_thread+0xa82/0xda0 kernel/workqueue.c:3417
 kthread+0x70e/0x8a0 kernel/kthread.c:464
 ret_from_fork+0x3fc/0x770 arch/x86/kernel/process.c:148
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
 </TASK>

Showing all locks held in the system:
2 locks held by kworker/u8:1/13:
1 lock held by rcu_tasks_trace/29:
 #0: ffffffff8e13fe10 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{4:4}, at: rcu_tasks_one_gp+0xaf9/0xdf0 kernel/rcu/tasks.h:614
1 lock held by khungtaskd/31:
 #0: ffffffff8e13f0e0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire include/linux/rcupdate.h:331 [inline]
 #0: ffffffff8e13f0e0 (rcu_read_lock){....}-{1:3}, at: rcu_read_lock include/linux/rcupdate.h:841 [inline]
 #0: ffffffff8e13f0e0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x2e/0x180 kernel/locking/lockdep.c:6770
3 locks held by kworker/u8:2/36:
2 locks held by kworker/u8:3/49:
6 locks held by kworker/0:2/977:
 #0: ffff888143ebe948 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3213 [inline]
 #0: ffff888143ebe948 ((wq_completion)usb_hub_wq){+.+.}-{0:0}, at: process_scheduled_works+0x9b4/0x17b0 kernel/workqueue.c:3321
 #1: ffffc90003a4fbc0 ((work_completion)(&hub->events)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3214 [inline]
 #1: ffffc90003a4fbc0 ((work_completion)(&hub->events)){+.+.}-{0:0}, at: process_scheduled_works+0x9ef/0x17b0 kernel/workqueue.c:3321
 #2: ffff888144722198 (&dev->mutex){....}-{4:4}, at: device_lock include/linux/device.h:884 [inline]
 #2: ffff888144722198 (&dev->mutex){....}-{4:4}, at: hub_event+0x184/0x4a20 drivers/usb/core/hub.c:5898
 #3: ffff88807cf06198 (&dev->mutex){....}-{4:4}, at: device_lock include/linux/device.h:884 [inline]
 #3: ffff88807cf06198 (&dev->mutex){....}-{4:4}, at: usb_disconnect+0xf8/0x950 drivers/usb/core/hub.c:2335
 #4: ffff88802171a160 (&dev->mutex){....}-{4:4}, at: device_lock include/linux/device.h:884 [inline]
 #4: ffff88802171a160 (&dev->mutex){....}-{4:4}, at: __device_driver_lock drivers/base/dd.c:1094 [inline]
 #4: ffff88802171a160 (&dev->mutex){....}-{4:4}, at: device_release_driver_internal+0xb6/0x7c0 drivers/base/dd.c:1292
 #5: ffffffff8ef805e8 (input_mutex){+.+.}-{4:4}, at: class_mutex_constructor include/linux/mutex.h:225 [inline]
 #5: ffffffff8ef805e8 (input_mutex){+.+.}-{4:4}, at: __input_unregister_device+0x2d8/0x5e0 drivers/input/input.c:2221
3 locks held by kworker/u8:7/3462:
3 locks held by kworker/u8:8/3509:
3 locks held by kworker/u8:9/3543:
2 locks held by getty/5596:
 #0: ffff88814c57f0a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x25/0x70 drivers/tty/tty_ldisc.c:243
 #1: ffffc9000333b2f0 (&ldata->atomic_read_lock){+.+.}-{4:4}, at: n_tty_read+0x43e/0x1400 drivers/tty/n_tty.c:2222
3 locks held by kworker/0:3/5824:
 #0: ffff88801a480d48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3213 [inline]
 #0: ffff88801a480d48 ((wq_completion)events){+.+.}-{0:0}, at: process_scheduled_works+0x9b4/0x17b0 kernel/workqueue.c:3321
 #1: ffffc90003fffbc0 (deferred_process_work){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3214 [inline]
 #1: ffffc90003fffbc0 (deferred_process_work){+.+.}-{0:0}, at: process_scheduled_works+0x9ef/0x17b0 kernel/workqueue.c:3321
 #2: ffffffff8f509f08 (rtnl_mutex){+.+.}-{4:4}, at: switchdev_deferred_process_work+0xe/0x20 net/switchdev/switchdev.c:104
1 lock held by udevd/5852:
4 locks held by kworker/1:3/5904:
 #0: ffff88801a481d48 ((wq_completion)events_power_efficient){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3213 [inline]
 #0: ffff88801a481d48 ((wq_completion)events_power_efficient){+.+.}-{0:0}, at: process_scheduled_works+0x9b4/0x17b0 kernel/workqueue.c:3321
 #1: ffffc9000492fbc0 ((reg_check_chans).work){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3214 [inline]
 #1: ffffc9000492fbc0 ((reg_check_chans).work){+.+.}-{0:0}, at: process_scheduled_works+0x9ef/0x17b0 kernel/workqueue.c:3321
 #2: ffffffff8f509f08 (rtnl_mutex){+.+.}-{4:4}, at: reg_check_chans_work+0x95/0xf00 net/wireless/reg.c:2483
 #3: ffff88802ff58768 (&rdev->wiphy.mtx){+.+.}-{4:4}, at: class_wiphy_constructor include/net/cfg80211.h:6062 [inline]
 #3: ffff88802ff58768 (&rdev->wiphy.mtx){+.+.}-{4:4}, at: reg_leave_invalid_chans net/wireless/reg.c:2471 [inline]
 #3: ffff88802ff58768 (&rdev->wiphy.mtx){+.+.}-{4:4}, at: reg_check_chans_work+0x164/0xf00 net/wireless/reg.c:2486
1 lock held by syz.2.194/6684:
 #0: ffffffff8e144bf8 (rcu_state.exp_mutex){+.+.}-{4:4}, at: exp_funnel_lock kernel/rcu/tree_exp.h:304 [inline]
 #0: ffffffff8e144bf8 (rcu_state.exp_mutex){+.+.}-{4:4}, at: synchronize_rcu_expedited+0x2f6/0x730 kernel/rcu/tree_exp.h:998
1 lock held by syz-executor/7237:
1 lock held by sed/7244:
2 locks held by syz.3.339/7246:
 #0: ffff88807e1d6808 (&sb->s_type->i_mutex_key#10){+.+.}-{4:4}, at: inode_lock include/linux/fs.h:869 [inline]
 #0: ffff88807e1d6808 (&sb->s_type->i_mutex_key#10){+.+.}-{4:4}, at: __sock_release net/socket.c:646 [inline]
 #0: ffff88807e1d6808 (&sb->s_type->i_mutex_key#10){+.+.}-{4:4}, at: sock_close+0x9b/0x240 net/socket.c:1391
 #1: ffffffff8e144bf8 (rcu_state.exp_mutex){+.+.}-{4:4}, at: exp_funnel_lock kernel/rcu/tree_exp.h:336 [inline]
 #1: ffffffff8e144bf8 (rcu_state.exp_mutex){+.+.}-{4:4}, at: synchronize_rcu_expedited+0x3b9/0x730 kernel/rcu/tree_exp.h:998
1 lock held by syz.4.340/7253:
 #0: ffffffff8f509f08 (rtnl_mutex){+.+.}-{4:4}, at: tcx_link_attach+0xe7/0x8a0 kernel/bpf/tcx.c:318

=============================================

NMI backtrace for cpu 0
CPU: 0 UID: 0 PID: 31 Comm: khungtaskd Not tainted 6.16.0-syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/12/2025
Call Trace:
 <TASK>
 dump_stack_lvl+0x189/0x250 lib/dump_stack.c:120
 nmi_cpu_backtrace+0x39e/0x3d0 lib/nmi_backtrace.c:113
 nmi_trigger_cpumask_backtrace+0x17a/0x300 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:158 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:307 [inline]
 watchdog+0xfee/0x1030 kernel/hung_task.c:470
 kthread+0x70e/0x8a0 kernel/kthread.c:464
 ret_from_fork+0x3fc/0x770 arch/x86/kernel/process.c:148
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Not tainted 6.16.0-syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/12/2025
RIP: 0010:pv_native_safe_halt+0x13/0x20 arch/x86/kernel/paravirt.c:82
Code: 53 de 02 00 cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d d3 ad 21 00 f3 0f 1e fa fb f4 <c3> cc cc cc cc cc cc cc cc cc cc cc cc 90 90 90 90 90 90 90 90 90
RSP: 0018:ffffc90000197de0 EFLAGS: 000002c2
RAX: 74ce113e412f4b00 RBX: ffffffff81976918 RCX: 74ce113e412f4b00
RDX: 0000000000000001 RSI: ffffffff8d982fba RDI: ffffffff8be1ba40
RBP: ffffc90000197f20 R08: ffff8880b8732f5b R09: 1ffff110170e65eb
R10: dffffc0000000000 R11: ffffed10170e65ec R12: ffffffff8fa0b3f0
R13: 0000000000000001 R14: 0000000000000001 R15: 1ffff11003a57b40
FS:  0000000000000000(0000) GS:ffff888125d57000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f111c9b1f98 CR3: 0000000076552000 CR4: 00000000003526f0
Call Trace:
 <TASK>
 arch_safe_halt arch/x86/include/asm/paravirt.h:107 [inline]
 default_idle+0x13/0x20 arch/x86/kernel/process.c:749
 default_idle_call+0x74/0xb0 kernel/sched/idle.c:117
 cpuidle_idle_call kernel/sched/idle.c:185 [inline]
 do_idle+0x1e8/0x510 kernel/sched/idle.c:325
 cpu_startup_entry+0x44/0x60 kernel/sched/idle.c:423
 start_secondary+0x101/0x110 arch/x86/kernel/smpboot.c:315
 common_startup_64+0x13e/0x147
 </TASK>

Crashes (47):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/08/21 12:21 upstream 038d61fd6422 0b9605c8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2025/08/17 06:25 upstream 99bade344cfa 1804e95e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2025/06/12 22:25 upstream 2c4a1f3fe03e 98683f8f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in worker_thread
2025/02/10 23:06 upstream a64dcfb451e2 43f51a00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in worker_thread
2025/02/02 02:28 upstream 69e858e0b8b2 0dff8567 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2025/01/27 03:31 upstream c2da8b3f914f 9fbd772e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2024/11/16 21:42 upstream e8bdb3c8be08 cfe3a04a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in worker_thread
2024/10/31 04:54 upstream 4236f913808c fb888278 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2024/10/07 11:08 upstream 8cf0b93919e1 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/10/05 16:15 upstream 27cc6fdf7201 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/09/28 01:19 upstream e477dba5442c 2b1784d6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2024/09/26 12:37 upstream aa486552a110 0d19f247 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in worker_thread
2024/09/20 14:21 upstream a430d95c5efa 6f888b75 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in worker_thread
2024/08/11 02:38 upstream 5189dafa4cf9 6f4edef4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/07/13 05:22 upstream 528dd46d0fc3 eaeb5c15 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in worker_thread
2024/07/09 08:21 upstream 4376e966ecb7 bc23a442 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/07/05 15:04 upstream 661e504db04c 2a40360c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/06/24 11:05 upstream f2661062f16b edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2024/06/24 05:24 upstream 7c16f0a4ed1c edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/06/15 14:24 upstream 2ccbdf43d5e7 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/06/14 12:39 upstream 2ccbdf43d5e7 a9616ff5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2024/06/09 18:43 upstream 771ed66105de 82c05ab8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2024/06/05 13:50 upstream 32f88d65f01b e1e2c66e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2024/05/30 04:05 upstream 4a4be1ad3a6e 34889ee3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/05/28 18:27 upstream 2bfcfd584ff5 34889ee3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in worker_thread
2024/05/25 07:19 upstream 0b32d436c015 a10a183e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in worker_thread
2024/10/08 16:41 upstream 87d6aab2389e 402f1df0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: task hung in worker_thread
2024/07/21 13:36 upstream 2c9b3512402e b88348e9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: task hung in worker_thread
2025/02/12 14:26 net e589adf5b70c b27c2402 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in worker_thread
2025/02/05 12:20 net a1300691aed9 5896748e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in worker_thread
2025/01/26 06:12 net 15a901361ec3 9fbd772e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in worker_thread
2024/11/19 12:17 net 4262bacb748f 571351cb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in worker_thread
2024/10/18 01:50 net cb560795c8c2 666f77ed .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in worker_thread
2024/07/11 10:13 net 76a0a3f9cc2f c699c2eb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in worker_thread
2024/06/20 02:19 net be27b8965297 41b7e219 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in worker_thread
2025/03/11 13:44 net-next 40587f749df2 16256247 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in worker_thread
2025/02/01 01:07 net-next c2933b2befe2 aa47157c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in worker_thread
2025/01/26 14:05 net-next 0ad9617c78ac 9fbd772e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in worker_thread
2025/01/23 01:01 net-next 0ad9617c78ac 25e17fd3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in worker_thread
2024/10/18 18:18 net-next 6d858708d465 cd6fc0a3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in worker_thread
2024/06/12 12:52 net-next 91579c93a9b2 f815599d .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce INFO: task hung in worker_thread
2024/11/02 06:34 linux-next c88416ba074a f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in worker_thread
2024/08/02 20:01 linux-next 931a3b3bccc9 53683cf2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in worker_thread
2024/05/27 16:35 linux-next 3689b0ef08b7 a10a183e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in worker_thread
2024/05/17 15:46 linux-next c75962170e49 a12e99e7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in worker_thread
2024/05/13 12:20 linux-next 6ba6c795dc73 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in worker_thread
2024/05/07 19:08 linux-next 93a39e476608 cb2dcc0e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in worker_thread
* Struck through repros no longer work on HEAD.