KCSAN: data-race in __mod_timer / blk_add_timer (14)

Status: moderation: reported on 2026/11/15 18:04
Subsystems: block
Labels: prio:low race:benign
[Documentation on labels]
Reported-by: syzbot+9a91936621131872a036@syzkaller.appspotmail.com
First crash: 1h31m, last: 1h31m
✨ AI-generated root cause summary:

In blk-mq, blk_mq_start_request() calls blk_add_timer() concurrently across CPUs for requests submitted to the same request_queue without serialization. As an optimization to avoid calling mod_timer() on every request, blk_add_timer() performs unannotated, lockless reads of q->timeout.expires (in time_before(expiry, q->timeout.expires) and diff = q->timeout.expires - expiry). Concurrently, another CPU calling mod_timer(&q->timeout, expiry) executes __mod_timer(), which acquires the timer base lock and writes timer->expires = expires without memory ordering annotations. This leads to a KCSAN data race between the unmarked read in blk_add_timer() and the unmarked write in __mod_timer().

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 23c62304dde2:

Root Cause Analysis

1. Involved Components and Context

  • Source Files:
  • Data Structures and Fields:
    • struct request_queue: Holds the shared timer timeout (struct timer_list).
    • struct timer_list: Holds expires (unsigned long) representing the absolute expiration timestamp in jiffies.
    • struct request: Holds deadline (unsigned long) representing the specific request's timeout.

2. Mechanism of the Defect

In the multiqueue block layer (blk-mq), multiple hardware contexts and CPUs submit and start requests concurrently against the same struct request_queue. When a request is started, blk_mq_start_request() calls blk_add_timer(req) without holding any queue-wide lock that serializes timeout modifications across CPUs.

Each request stores its own timeout deadline in req->deadline using WRITE_ONCE(req->deadline, expiry). To wake up the block layer's timeout handling workqueue (q->timeout_work), a single shared timer, q->timeout, is maintained per request_queue.

In blk_add_timer() (block/blk-timeout.c:153-165), the code checks whether q->timeout needs to be armed or updated:

	if (!timer_pending(&q->timeout) ||
	    time_before(expiry, q->timeout.expires)) {
		unsigned long diff = q->timeout.expires - expiry;

		/*
		 * Due to added timer slack to group timers, the timer
		 * will often be a little in front of what we asked for.
		 * So apply some tolerance here too, otherwise we keep
		 * modifying the timer because expires for value X
		 * will be X + something.
		 */
		if (!timer_pending(&q->timeout) || (diff >= HZ / 2))
			mod_timer(&q->timeout, expiry);
	}

To avoid calling mod_timer() (which takes the timer wheel's per-CPU spinlock base->lock) on every request start, blk_add_timer() performs lockless reads of q->timeout.expires twice:

  1. time_before(expiry, q->timeout.expires)
  2. diff = q->timeout.expires - expiry

Neither of these reads uses READ_ONCE() or data_race().

Concurrently, another CPU processing another request on the same request_queue in blk_add_timer() can evaluate the condition as true and invoke mod_timer(&q->timeout, expiry) -> __mod_timer(). Inside __mod_timer() (kernel/time/timer.c:1127), after taking the timer base lock (base->lock), the timer's expiration time is updated via:

	timer->expires = expires;

This is an unmarked plain write to timer->expires.

3. Invariant Violation and KCSAN Detection

Because blk_add_timer() reads q->timeout.expires without holding base->lock and without using READ_ONCE(), and __mod_timer() writes to timer->expires without WRITE_ONCE(), concurrent execution across two CPUs produces a data race on q->timeout.expires (0xffff888102545e08), triggering the KCSAN report.

▶ ▼ ✨ AI Jobs (2)
ID Workflow Result Correct Bug Created Started Finished Revision
8784703c assessment-kcsan Benign: ✅ ❓ KCSAN: data-race in __mod_timer / blk_add_timer (14) 2026/11/15 17:34 2026/11/15 17:34 2026/11/15 18:29 819ebe34
fb8776b7 assessment-security DenialOfService: ❌ Exploitable: ❌ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ❌ NetworkNeedsSetup: ❌ NetworkTrigger: ❌ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ✅ UserNamespace: ✅ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ KCSAN: data-race in __mod_timer / blk_add_timer (14) 2026/11/15 17:33 2026/11/15 17:34 2026/11/15 18:02 819ebe34
▶ ▼ Similar bugs (13)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream KCSAN: data-race in __mod_timer / blk_add_timer (6) block 6 1 1321d 1301d 0/29 auto-obsoleted due to no activity on 2023/05/09 08:21
upstream KCSAN: data-race in __mod_timer / blk_add_timer (10) block 6 1 662d 657d 0/29 auto-obsoleted due to no activity on 2025/03/19 05:59
upstream KCSAN: data-race in __mod_timer / blk_add_timer (13) block 6 1 345d 345d 0/29 auto-obsoleted due to no activity on 2026/01/30 11:05
upstream KCSAN: data-race in __mod_timer / blk_add_timer (11) block 6 3 565d 601d 0/29 auto-obsoleted due to no activity on 2025/06/24 14:35
upstream KCSAN: data-race in __mod_timer / blk_add_timer block 6 2 2004d 2039d 0/29 auto-closed as invalid on 2021/06/25 00:11
upstream KCSAN: data-race in __mod_timer / blk_add_timer (3) block 6 8 1779d 1878d 0/29 auto-closed as invalid on 2022/02/04 22:18
upstream KCSAN: data-race in __mod_timer / blk_add_timer (2) block 6 2 1930d 1960d 0/29 auto-closed as invalid on 2021/09/07 18:24
upstream KCSAN: data-race in __mod_timer / blk_add_timer (4) block 6 2 1687d 1706d 0/29 auto-closed as invalid on 2022/05/08 11:46
upstream KCSAN: data-race in __mod_timer / blk_add_timer (7) block 6 2 1241d 1258d 0/29 auto-obsoleted due to no activity on 2023/07/28 08:54
upstream KCSAN: data-race in __mod_timer / blk_add_timer (5) block 6 1 1543d 1543d 0/29 auto-closed as invalid on 2022/09/29 13:43
upstream KCSAN: data-race in __mod_timer / blk_add_timer (12) block 6 4 407d 477d 0/29 auto-obsoleted due to no activity on 2025/11/29 15:07
upstream KCSAN: data-race in __mod_timer / blk_add_timer (9) block 6 1 884d 884d 0/29 auto-obsoleted due to no activity on 2024/07/19 18:23
upstream KCSAN: data-race in __mod_timer / blk_add_timer (8) block 6 7 1034d 1071d 0/29 auto-obsoleted due to no activity on 2024/02/20 17:38

Sample crash report:
==================================================================
BUG: KCSAN: data-race in __mod_timer / blk_add_timer

write to 0xffff888102545e08 of 8 bytes by task 14692 on cpu 0:
 __mod_timer+0x671/0x990 kernel/time/timer.c:1127
 mod_timer+0x1f/0x30 kernel/time/timer.c:1196
 blk_add_timer+0x222/0x2c0 block/blk-timeout.c:165
 blk_mq_start_request+0x1e5/0x3b0 block/blk-mq.c:1373
 loop_queue_rq+0x43/0x680 drivers/block/loop.c:1780
 __blk_mq_issue_directly block/blk-mq.c:2718 [inline]
 blk_mq_try_issue_directly+0x254/0x4b0 block/blk-mq.c:2778
 blk_mq_submit_bio+0xee4/0x1240 block/blk-mq.c:3242
 __submit_bio+0x54/0x3d0 block/blk-core.c:674
 __submit_bio_noacct_mq block/blk-core.c:757 [inline]
 submit_bio_noacct_nocheck+0x153/0x620 block/blk-core.c:791
 submit_bio_noacct+0x738/0x970 block/blk-core.c:929
 submit_bio+0x2a4/0x2c0 block/blk-core.c:970
 blk_crypto_submit_bio include/linux/blk-crypto.h:212 [inline]
 __bh_submit+0x373/0x3b0 fs/buffer.c:1148
 bh_submit+0x26/0x30 fs/buffer.c:1162
 __ext4_read_bh fs/ext4/super.c:192 [inline]
 ext4_read_bh+0x104/0x170 fs/ext4/super.c:217
 ext4_read_bh_lock fs/ext4/super.c:232 [inline]
 __ext4_sb_bread_gfp fs/ext4/super.c:254 [inline]
 ext4_sb_bread_unmovable+0x14b/0x180 fs/ext4/super.c:277
 ext4_load_super fs/ext4/super.c:5214 [inline]
 __ext4_fill_super fs/ext4/super.c:5420 [inline]
 ext4_fill_super+0x44b/0x37d0 fs/ext4/super.c:5899
 get_tree_bdev_flags+0x297/0x300 fs/super.c:1821
 get_tree_bdev+0x1f/0x30 fs/super.c:1844
 ext4_get_tree+0x1c/0x30 fs/ext4/super.c:5931
 vfs_get_tree+0x57/0x1d0 fs/super.c:1881
 fc_mount fs/namespace.c:1282 [inline]
 do_new_mount_fc fs/namespace.c:3910 [inline]
 do_new_mount+0x288/0x730 fs/namespace.c:3986
 path_mount+0x4ad/0xba0 fs/namespace.c:4337
 do_mount fs/namespace.c:4350 [inline]
 __do_sys_mount fs/namespace.c:4563 [inline]
 __se_sys_mount+0x28c/0x2e0 fs/namespace.c:4540
 __x64_sys_mount+0x67/0x80 fs/namespace.c:4540
 x64_sys_call+0xc3b/0x2550 arch/x86/include/generated/asm/syscalls_64.h:166
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x116/0x380 arch/x86/entry/syscall_64.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

read to 0xffff888102545e08 of 8 bytes by task 12254 on cpu 1:
 blk_add_timer+0x1b5/0x2c0 block/blk-timeout.c:-1
 blk_mq_start_request+0x1e5/0x3b0 block/blk-mq.c:1373
 loop_queue_rq+0x43/0x680 drivers/block/loop.c:1780
 __blk_mq_issue_directly block/blk-mq.c:2718 [inline]
 blk_mq_request_issue_directly+0x1f3/0x370 block/blk-mq.c:2805
 blk_mq_issue_direct+0x18a/0x4e0 block/blk-mq.c:2826
 blk_mq_dispatch_queue_requests+0x1f4/0x320 block/blk-mq.c:2901
 blk_mq_flush_plug_list+0x2ad/0x320 block/blk-mq.c:2984
 __blk_flush_plug+0x22f/0x2b0 block/blk-core.c:1274
 blk_finish_plug+0x47/0x70 block/blk-core.c:1301
 read_pages+0x4bf/0x540 mm/readahead.c:187
 page_cache_ra_unbounded+0x2cd/0x3a0 mm/readahead.c:315
 do_page_cache_ra mm/readahead.c:348 [inline]
 force_page_cache_ra+0x190/0x1d0 mm/readahead.c:378
 page_cache_sync_ra+0x11d/0x560 mm/readahead.c:602
 filemap_get_pages+0x2f2/0x1360 mm/filemap.c:2743
 filemap_read+0x21c/0xad0 mm/filemap.c:2853
 blkdev_read_iter+0x29c/0x390 block/fops.c:892
 new_sync_read fs/read_write.c:473 [inline]
 vfs_read+0x699/0x7c0 fs/read_write.c:554
 ksys_read+0xdc/0x1a0 fs/read_write.c:696
 __do_sys_read fs/read_write.c:705 [inline]
 __se_sys_read fs/read_write.c:703 [inline]
 __x64_sys_read+0x40/0x50 fs/read_write.c:703
 x64_sys_call+0x2135/0x2550 arch/x86/include/generated/asm/syscalls_64.h:1
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x116/0x380 arch/x86/entry/syscall_64.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

value changed: 0x000000010000c897 -> 0x000000010000cc05

Reported by Kernel Concurrency Sanitizer on:
CPU: 1 UID: 0 PID: 12254 Comm: udevd Not tainted syzkaller #0 PREEMPT(lazy) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/06/2026
==================================================================

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/15 17:33 upstream 23c62304dde2 819ebe34 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in __mod_timer / blk_add_timer
* Struck through repros no longer work on HEAD.