KCSAN: data-race in process_scheduled_works / queue_work_on

Status: moderation: reported on 2026/11/20 00:02
Subsystems: wireguard
Labels: race:benign prio:low
[Documentation on labels]
Reported-by: syzbot+809b5f50e670f71bc255@syzkaller.appspotmail.com
First crash: 2d22h, last: 4h18m
✨ AI-generated root cause summary:

KCSAN reports a data race on struct work_struct::data between an unmarked read in process_one_work() (via process_scheduled_works()) and a marked atomic bit operation in queue_work_on(). When set_printk_work is already queued and being claimed by a kworker in process_one_work() with pool->lock held (before WORK_STRUCT_PENDING_BIT is cleared), another CPU calls schedule_work(&set_printk_work) -> queue_work_on(), which executes test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work)) without holding pool->lock. Meanwhile, process_one_work() reads work_data = *work_data_bits(work) using a plain (unmarked) load instead of atomic_long_read(&work->data) or READ_ONCE(), triggering the KCSAN report despite the race being benign.

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 4405c18c95d2:

Root Cause Analysis

The KCSAN report is triggered by an unmarked plain load of struct work_struct::data in process_one_work() racing with a marked atomic read-modify-write (test_and_set_bit()) on the same data field in queue_work_on().

Involved Entities

Mechanism

  1. On CPU 1, kworker/1:4 runs worker_thread() -> process_scheduled_works() -> process_one_work() to execute set_printk_work while holding pool->lock with IRQs disabled.
  2. At kernel/workqueue.c:3419, WORK_STRUCT_PENDING_BIT (bit 0 of work->data) is still set; it is not cleared until set_work_pool_and_clear_pending(work, ...) is called later at line 3456.
  3. While get_work_pwq(work) at line 3388 reads work->data using atomic_long_read(&work->data), line 3419 performs an unmarked plain dereference *work_data_bits(work) to extract the flush color and WORK_STRUCT_INACTIVE flag for worker->current_color and pwq_dec_nr_in_flight().
  4. Concurrently on CPU 0, another task verifying a BPF program that uses bpf_trace_printk calls bpf_get_trace_printk_proto() -> schedule_work(&set_printk_work) -> queue_work_on(). Without acquiring pool->lock, queue_work_on() executes test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work)), which KCSAN instruments as an 8-byte atomic read-write (instrument_atomic_read_write()).
  5. Because queue_work_on() performs a marked read-write concurrently with the unmarked read *work_data_bits(work) in process_one_work(), KCSAN reports a data race on set_printk_work.data.

Invariant Analysis

The race is benign at the algorithmic level:

  • At kernel/workqueue.c:3419, pool->lock is held and WORK_STRUCT_PENDING_BIT is guaranteed to be 1 until set_work_pool_and_clear_pending() at line 3456 (which is bracketed by smp_wmb() and smp_mb() barriers). No other path can modify work->data while WORK_STRUCT_PENDING_BIT is set without acquiring pool->lock.
  • test_and_set_bit(WORK_STRUCT_PENDING_BIT, ...) in queue_work_on() atomically observes that bit 0 is already 1, leaves all bits of work->data unchanged, and returns true, causing queue_work_on() to return false without enqueueing or modifying the work item.
  • Replacing *work_data_bits(work) at kernel/workqueue.c:3419 (and similar unmarked reads such as try_to_grab_pending() at line 2265) with atomic_long_read(&work->data) or READ_ONCE(*work_data_bits(work)) would make the access marked and silence the KCSAN report.
▶ ▼ ✨ AI Jobs (2)
ID Workflow Result Correct Bug Created Started Finished Revision
63cc2620 assessment-kcsan Benign: ✅ ❓ KCSAN: data-race in process_scheduled_works / queue_work_on 2026/11/19 18:41 2026/11/19 18:41 2026/11/19 18:43 229865fb
0deb46e6 assessment-security DenialOfService: ❌ Exploitable: ❌ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ❌ NetworkNeedsSetup: ❌ NetworkTrigger: ❌ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ✅ UserNamespace: ✅ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ KCSAN: data-race in process_scheduled_works / queue_work_on 2026/11/19 18:40 2026/11/19 18:40 2026/11/19 18:47 229865fb

Sample crash report:
==================================================================
BUG: KCSAN: data-race in process_scheduled_works / queue_work_on

read-write (marked) to 0xffff888100a0cc90 of 8 bytes by task 4143 on cpu 0:
 instrument_atomic_read_write include/linux/instrumented.h:113 [inline]
 test_and_set_bit include/asm-generic/bitops/instrumented-atomic.h:71 [inline]
 queue_work_on+0x42/0x150 kernel/workqueue.c:2597
 queue_work include/linux/workqueue.h:700 [inline]
 loop_queue_work drivers/block/loop.c:769 [inline]
 loop_queue_rq+0x5d2/0x6a0 drivers/block/loop.c:1811
 __blk_mq_issue_directly block/blk-mq.c:2718 [inline]
 blk_mq_try_issue_directly+0x290/0x500 block/blk-mq.c:2778
 blk_mq_submit_bio+0xf7d/0x1310 block/blk-mq.c:3242
 __submit_bio+0x68/0x250 block/blk-core.c:674
 __submit_bio_noacct_mq block/blk-core.c:757 [inline]
 submit_bio_noacct_nocheck+0x168/0x650 block/blk-core.c:791
 submit_bio_noacct+0x810/0xa80 block/blk-core.c:929
 submit_bio+0x2a4/0x2c0 block/blk-core.c:970
 blk_crypto_submit_bio include/linux/blk-crypto.h:212 [inline]
 __bh_submit+0x427/0x460 fs/buffer.c:1148
 bh_submit fs/buffer.c:1162 [inline]
 block_read_full_folio+0x60e/0x650 fs/buffer.c:2386
 blkdev_read_folio+0x1c/0x30 block/fops.c:503
 filemap_read_folio+0x40/0x150 mm/filemap.c:2549
 filemap_update_page mm/filemap.c:2636 [inline]
 filemap_get_pages+0xe98/0x1020 mm/filemap.c:2766
 filemap_read+0x20d/0x8d0 mm/filemap.c:2853
 blkdev_read_iter+0x29c/0x390 block/fops.c:892
 new_sync_read fs/read_write.c:473 [inline]
 vfs_read+0x6ac/0x7d0 fs/read_write.c:554
 ksys_read+0xdc/0x1a0 fs/read_write.c:696
 __do_sys_read fs/read_write.c:705 [inline]
 __se_sys_read fs/read_write.c:703 [inline]
 __x64_sys_read+0x40/0x50 fs/read_write.c:703
 x64_sys_call+0x2135/0x2550 arch/x86/include/generated/asm/syscalls_64.h:1
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x11e/0x3a0 arch/x86/entry/syscall_64.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

read to 0xffff888100a0cc90 of 8 bytes by task 366 on cpu 1:
 process_one_work kernel/workqueue.c:3419 [inline]
 process_scheduled_works+0x2fd/0xab0 kernel/workqueue.c:3578
 worker_thread+0x56c/0x770 kernel/workqueue.c:3659
 kthread+0x248/0x2a0 kernel/kthread.c:436
 ret_from_fork+0x146/0x330 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

Reported by Kernel Concurrency Sanitizer on:
CPU: 1 UID: 0 PID: 366 Comm: kworker/u8:6 Not tainted syzkaller #0 PREEMPT(lazy) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/06/2026
Workqueue: loop2 loop_rootcg_workfn
==================================================================

Crashes (8):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/19 20:28 upstream 8dac6df92a2e 229865fb .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in process_scheduled_works / queue_work_on
2026/11/19 07:25 upstream 4405c18c95d2 f138241c .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in process_scheduled_works / queue_work_on
2026/11/18 20:44 upstream 4405c18c95d2 2006bdcc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in process_scheduled_works / queue_work_on
2026/11/18 18:58 upstream 20eef29b7c84 2006bdcc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in process_scheduled_works / queue_work_on
2026/11/18 06:57 upstream 20eef29b7c84 eb4895d7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in process_scheduled_works / queue_work_on
2026/11/17 21:35 upstream 63fdb511c27a 122f94d5 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in process_scheduled_works / queue_work_on
2026/11/17 10:46 upstream 666539eecdf4 43af6cac .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in process_scheduled_works / queue_work_on
2026/11/17 02:20 upstream 666539eecdf4 a3e64cfc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in process_scheduled_works / queue_work_on
* Struck through repros no longer work on HEAD.