KCSAN: data-race in blk_mq_put_driver_tag / blk_mq_tagset_busy_iter

Status: moderation: reported on 2026/10/26 22:16
Subsystems: block
Labels: race:benign prio:low
[Documentation on labels]
Reported-by: syzbot+106207df4f48565a7c02@syzkaller.appspotmail.com
First crash: 3d20h, last: 3d20h
✨ AI-generated root cause summary:

A data race occurs on the request driver tag field (rq->tag) between blk_mq_find_and_get_req() and __blk_mq_put_driver_tag(). When iterating over active tags, blk_mq_find_and_get_req() performs a lockless, plain read of rq->tag to verify the tag matches the sbitmap bitnr before incrementing the request reference counter. Concurrently, __blk_mq_put_driver_tag() releases the tag into the sbitmap and performs an unsynchronized plain write resetting rq->tag to BLK_MQ_NO_TAG. In production kernels, this access is atomic at the word level and benign; it does not lead to memory corruption, information disclosure, or denial of service.

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 6b4b543fa187:

Root Cause Analysis

1. Raced Variable and Involved Components

  • Variable / Field: rq->tag (offset 32 bytes / 256 bits, size 4 bytes in struct request), holding the allocated block driver tag or BLK_MQ_NO_TAG (-1U / 0xffffffff).
  • Files:

2. Race Mechanism and Lifecycle Violation

When an I/O scheduler is configured (e.g. RQF_SCHED_TAGS), requests are assigned a scheduler tag (rq->internal_tag) upon creation, while the hardware driver tag (rq->tag) is dynamically acquired right before dispatch to the device driver via blk_mq_prep_dispatch_rq() $\to$ blk_mq_get_driver_tag() $\to$ __blk_mq_alloc_driver_tag().

During request dispatch in blk_mq_dispatch_rq_list():

  • If the target device driver returns BLK_STS_DEV_RESOURCE or BLK_STS_RESOURCE, blk_mq_handle_dev_resource() invokes __blk_mq_requeue_request(), which calls blk_mq_put_driver_tag() to release the driver tag while keeping the scheduler tag intact.
  • __blk_mq_put_driver_tag() in block/blk-mq.h:
    static inline void __blk_mq_put_driver_tag(struct blk_mq_hw_ctx *hctx,
                                               struct request *rq)
    {
        blk_mq_dec_active_requests(hctx);
        blk_mq_put_tag(hctx->tags, rq->mq_ctx, rq->tag);
        rq->tag = BLK_MQ_NO_TAG;
    }
    
    First, blk_mq_put_tag() clears the tag bit in the tag sbitmap (tags->bitmap_tags). Immediately after, a plain store rq->tag = BLK_MQ_NO_TAG overwrites rq->tag.

Concurrently on another CPU (task 426):

  • scsi_host_busy() runs blk_mq_tagset_busy_iter() to count in-flight commands.
  • The iterator iterates over all bits set in the tag sbitmap (bt_tags_for_each() $\to$ sbitmap_for_each_set() $\to$ bt_tags_iter()).
  • In bt_tags_iter(), blk_mq_find_and_get_req(tags, bitnr) is called locklessly (protected only by blk_mq_tags_srcu for tagset lifetime):
    static struct request *blk_mq_find_and_get_req(struct blk_mq_tags *tags,
            unsigned int bitnr)
    {
        struct request *rq;
    
        rq = tags->rqs[bitnr];
        if (!rq || rq->tag != bitnr || !req_ref_inc_not_zero(rq))
            rq = NULL;
        return rq;
    }
    
  • Because entries in tags->rqs[] are retained and not set to NULL upon request completion or requeue, blk_mq_find_and_get_req() performs a plain load rq->tag != bitnr to verify if the request cached in tags->rqs[bitnr] is indeed still holding tag bitnr.
  • Neither access uses READ_ONCE() or WRITE_ONCE(), and no lock is held between the requeue path and the lockless busy iteration path, causing KCSAN to report a concurrent data race.

Runtime Impact in Production

  • Memory Corruption (false): On production architectures (e.g., x86_64, arm64), 32-bit aligned reads and writes are single-copy atomic. The value read by blk_mq_find_and_get_req() will either be the old tag bitnr or BLK_MQ_NO_TAG. If BLK_MQ_NO_TAG is read, rq->tag != bitnr evaluates to true, rq is set to NULL, and iteration moves on. If bitnr is read, req_ref_inc_not_zero(rq) safely checks whether the request is still referenced before calling the iteration callback (scsi_host_check_in_flight()), which only inspects the command state. The request object belongs to the preallocated static request pool (sched_tags->static_rqs), so no memory safety or use-after-free issue can occur.
  • Information Leak (none): No kernel memory or pointer values are exposed to userspace or external channels.
  • Raw Denial of Service (false): The race only impacts whether a requeued command is transiently included in scsi_host_busy() or skipped. This does not cause kernel panics, deadlocks, infinite loops, or resource leaks.
  • Networking / KVM Candidates (false): The issue is entirely within the block multiqueue subsystem (block/blk-mq*) and SCSI core (drivers/scsi/), with no involvement of virtualization, hypervisors, or networking.
▶ ▼ ✨ AI Jobs (2)
ID Workflow Result Correct Bug Created Started Finished Revision
4425f6b1 assessment-security DenialOfService: ❌ Exploitable: ❌ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ❌ NetworkNeedsSetup: ❌ NetworkTrigger: ❌ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ✅ UserNamespace: ✅ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ KCSAN: data-race in blk_mq_put_driver_tag / blk_mq_tagset_busy_iter 2026/10/28 00:01 2026/10/28 00:01 2026/10/28 00:52 96923c60
5b1d185a assessment-kcsan Benign: ✅ ❓ KCSAN: data-race in blk_mq_put_driver_tag / blk_mq_tagset_busy_iter 2026/10/26 22:16 2026/10/26 22:16 2026/10/26 23:01 168cb7a2

Sample crash report:
==================================================================
BUG: KCSAN: data-race in blk_mq_put_driver_tag / blk_mq_tagset_busy_iter

write to 0xffff8881035f5e60 of 4 bytes by task 9669 on cpu 0:
 __blk_mq_put_driver_tag block/blk-mq.h:361 [inline]
 blk_mq_put_driver_tag+0x121/0x140 block/blk-mq.h:369
 __blk_mq_requeue_request+0x23/0x180 block/blk-mq.c:1521
 blk_mq_handle_dev_resource block/blk-mq.c:2019 [inline]
 blk_mq_dispatch_rq_list+0x749/0xef0 block/blk-mq.c:2140
 __blk_mq_sched_dispatch_requests+0x1db/0xce0 block/blk-mq-sched.c:299
 blk_mq_sched_dispatch_requests+0x86/0x120 block/blk-mq-sched.c:329
 blk_mq_run_hw_queue+0x145/0x1e0 block/blk-mq.c:2370
 blk_mq_dispatch_list+0x838/0x9f0 arch/x86/include/asm/bitops.h:-1
 blk_mq_flush_plug_list+0x2cd/0x320 block/blk-mq.c:2980
 blk_add_rq_to_plug+0xf0/0x3f0 block/blk-mq.c:1399
 blk_mq_submit_bio+0xd16/0x1210 block/blk-mq.c:3210
 __submit_bio+0x54/0x3d0 block/blk-core.c:674
 __submit_bio_noacct_mq block/blk-core.c:757 [inline]
 submit_bio_noacct_nocheck+0x1c6/0x620 block/blk-core.c:791
 submit_bio_noacct+0x738/0x970 block/blk-core.c:929
 submit_bio+0x2a4/0x2c0 block/blk-core.c:970
 bio_chain_and_submit+0x84/0xb0 block/bio.c:396
 __blkdev_issue_zero_pages+0x1f7/0x240 block/blk-lib.c:215
 blkdev_issue_zero_pages block/blk-lib.c:231 [inline]
 blkdev_issue_zeroout+0x39c/0x4e0 block/blk-lib.c:312
 blk_ioctl_zeroout block/ioctl.c:244 [inline]
 blkdev_common_ioctl+0x1a07/0x2060 block/ioctl.c:667
 blkdev_ioctl+0x31f/0x460 block/ioctl.c:792
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:597 [inline]
 __se_sys_ioctl+0xce/0x140 fs/ioctl.c:583
 __x64_sys_ioctl+0x43/0x50 fs/ioctl.c:583
 x64_sys_call+0x239d/0x2550 arch/x86/include/generated/asm/syscalls_64.h:17
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x112/0x360 arch/x86/entry/syscall_64.c:84
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

read to 0xffff8881035f5e60 of 4 bytes by task 426 on cpu 1:
 blk_mq_find_and_get_req block/blk-mq-tag.c:274 [inline]
 bt_tags_iter block/blk-mq-tag.c:369 [inline]
 __sbitmap_for_each_set include/linux/sbitmap.h:269 [inline]
 sbitmap_for_each_set include/linux/sbitmap.h:290 [inline]
 bt_tags_for_each block/blk-mq-tag.c:404 [inline]
 __blk_mq_all_tag_iter block/blk-mq-tag.c:415 [inline]
 blk_mq_tagset_busy_iter+0x643/0x860 block/blk-mq-tag.c:459
 scsi_host_busy+0x4e/0x80 drivers/scsi/hosts.c:631
 scsi_host_queue_ready drivers/scsi/scsi_lib.c:1502 [inline]
 scsi_queue_rq+0x4e0/0x19f0 drivers/scsi/scsi_lib.c:1900
 blk_mq_dispatch_rq_list+0x5d2/0xef0 block/blk-mq.c:2131
 __blk_mq_do_dispatch_sched block/blk-mq-sched.c:168 [inline]
 blk_mq_do_dispatch_sched block/blk-mq-sched.c:182 [inline]
 __blk_mq_sched_dispatch_requests+0x86c/0xce0 block/blk-mq-sched.c:307
 blk_mq_sched_dispatch_requests+0x86/0x120 block/blk-mq-sched.c:329
 blk_mq_run_hw_queue+0x145/0x1e0 block/blk-mq.c:2370
 blk_mq_run_hw_queues+0x18e/0x270 block/blk-mq.c:2418
 blk_mq_requeue_work+0x3ca/0x400 block/blk-mq.c:1585
 process_one_work kernel/workqueue.c:3495 [inline]
 process_scheduled_works+0x53f/0xa20 kernel/workqueue.c:3578
 worker_thread+0x56c/0x770 kernel/workqueue.c:3659
 kthread+0x223/0x270 kernel/kthread.c:436
 ret_from_fork+0x146/0x330 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

value changed: 0x000023d9 -> 0xffffffff

Reported by Kernel Concurrency Sanitizer on:
CPU: 1 UID: 0 PID: 426 Comm: kworker/1:1H Not tainted syzkaller #0 PREEMPT(lazy) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/16/2026
Workqueue: kblockd blk_mq_requeue_work
==================================================================

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/10/26 22:16 upstream 6b4b543fa187 168cb7a2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in blk_mq_put_driver_tag / blk_mq_tagset_busy_iter
* Struck through repros no longer work on HEAD.