possible deadlock in wbt_init_enable_default

Status: upstream: reported on 2026/10/31 23:30
Subsystems: block
Labels: prio:low
[Documentation on labels]
Reported-by: syzbot+e2e1def00b1be94f0a22@syzkaller.appspotmail.com
First crash: 17d, last: 4d11h
✨ AI-generated root cause summary:

A race between disk registration in dm_setup_md_queue() and concurrent device removal (dm_destroy() -> blk_mark_disk_dead()) causes the queue freeze lockdep state tracking in block/blk-mq.c and block/blk.h to become inconsistent. Although rq_qos_add() freezes and subsequently unfreezes the queue during wbt_init_enable_default(), blk_unfreeze_release_lock() skips releasing q->io_lockdep_map because the disk was concurrently marked dead/dying. When wbt_init_enable_default() immediately proceeds to acquire q->debugfs_mutex, lockdep detects an apparent circular locking dependency against the leaked lockdep map. In production kernels without CONFIG_LOCKDEP, the queue is properly unfrozen and no deadlock or memory corruption occurs.

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 151a134a4e6e:

Root Cause Analysis

1. Functions, Structs, Fields, and Files Involved


2. Mechanism of the Bug and How It Is Triggered

Lockdep detected a potential circular locking dependency: $$&q\to\text{debugfs_mutex} \longrightarrow &sb\to\text{s_type}\to\text{i_mutex_key} \longrightarrow \text{fs_reclaim} \longrightarrow &q\to\text{q_usage_counter(io)} \longrightarrow &q\to\text{debugfs_mutex}$$

The dependency chain is constructed as follows:

  1. fs_reclaim $\to$ &q->q_usage_counter(io) (q->io_lockdep_map): In block/blk-core.c:504-507, blk_alloc_queue() explicitly trains lockdep on lock ordering between memory reclaim and queue freezing via:
    fs_reclaim_acquire(GFP_KERNEL);
    rwsem_acquire_read(&q->io_lockdep_map, 0, 0, _RET_IP_);
    rwsem_release(&q->io_lockdep_map, _RET_IP_);
    fs_reclaim_release(GFP_KERNEL);
    
  2. &sb->s_type->i_mutex_key $\to$ fs_reclaim: Filesystem/debugfs operations (e.g. during debugfs_create_dir() via simple_start_creating() / lookup_one_qstr_excl()) hold the parent directory's inode i_rwsem while performing GFP_KERNEL slab allocations, which can invoke fs_reclaim.
  3. &q->debugfs_mutex $\to$ &sb->s_type->i_mutex_key: In blk_register_queue() (block/blk-sysfs.c:967), blk_debugfs_lock(q) acquires q->debugfs_mutex before creating debugfs_create_dir(disk->disk_name, ...), acquiring the debugfs directory inode rwsem while holding q->debugfs_mutex.
  4. The reverse edge (&q->q_usage_counter(io) $\to$ &q->debugfs_mutex): In wbt_init_enable_default() (block/blk-wbt.c:788-796), the kernel calls:
    wbt_init(disk, rwb);
    memflags = blk_debugfs_lock(q);
    blk_mq_debugfs_register_rq_qos(q);
    blk_debugfs_unlock(q, memflags);
    
    Inside wbt_init() $\to$ rq_qos_add() (block/blk-rq-qos.c:341), memflags = blk_mq_freeze_queue(q) is called, which invokes blk_freeze_acquire_lock(q). Under CONFIG_LOCKDEP, this acquires q->io_lockdep_map. At line 349, rq_qos_add() calls blk_mq_unfreeze_queue(q, memflags).

However, as indicated by the log device-mapper: ioctl: device has been removed from the dev hash table. at drivers/md/dm-ioctl.c:1680, a concurrent device removal operation (dev_remove() $\to$ dm_destroy() $\to$ __dm_destroy() $\to$ blk_mark_disk_dead()) was executing against the same mapped device while table_load() was setting up the queue.

blk_mark_disk_dead() sets QUEUE_FLAG_DYING and GD_DEAD (test_and_set_bit(GD_DEAD, &disk->state)). In block/blk-mq.c and block/blk.h, lockdep state tracking for queue freezing maintains:

static inline void blk_freeze_acquire_lock(struct request_queue *q)
{
    if (!q->mq_freeze_disk_dead)
        rwsem_acquire(&q->io_lockdep_map, 0, 1, _RET_IP_);
    if (!q->mq_freeze_queue_dying)
        rwsem_acquire(&q->q_lockdep_map, 0, 1, _RET_IP_);
}

static inline void blk_unfreeze_release_lock(struct request_queue *q)
{
    if (!q->mq_freeze_queue_dying)
        rwsem_release(&q->q_lockdep_map, _RET_IP_);
    if (!q->mq_freeze_disk_dead)
        rwsem_release(&q->io_lockdep_map, _RET_IP_);
}

Because the queue state flags (mq_freeze_disk_dead, mq_freeze_queue_dying) or freeze owner tracking (mq_freeze_owner, mq_freeze_owner_depth) are stored on the shared request_queue struct, concurrent state modifications during the window between freeze and unfreeze lead to an imbalance: blk_freeze_acquire_lock() acquired the lockdep maps, but blk_unfreeze_release_lock() skipped rwsem_release(&q->io_lockdep_map) (or blk_unfreeze_check_owner() returned false).

As a result, lockdep still believed the task held &q->q_usage_counter(io) when wbt_init_enable_default() immediately reached line 794 and called blk_debugfs_lock(q) (which acquires q->debugfs_mutex), triggering the circular locking dependency splat.


3. Production Runtime Consequences & Classification Rationales

  • MemoryCorruption: false This is a lockdep warning. There are no out-of-bounds memory accesses, use-after-free conditions, invalid frees, or memory safety violations.
  • InfoLeak: "none" No uninitialized kernel memory or pointer values are disclosed to userspace or external observers.
  • RawDenialOfService: false In a production kernel built without CONFIG_LOCKDEP:
    • The synthetic q->io_lockdep_map does not exist.
    • blk_mq_freeze_queue() increments q->mq_freeze_depth and kills the percpu refcount, while blk_mq_unfreeze_queue() cleanly decrements q->mq_freeze_depth back to zero, resurrects q->q_usage_counter, and wakes up waiters on q->mq_freeze_wq.
    • blk_debugfs_lock() enters memalloc_noio_save(), preventing filesystem direct reclaim from entering the block layer while q->debugfs_mutex is held.
    • In table_load(), when hc is discovered to be removed from the hash table, the code safely aborts, frees the pending table, and returns -ENXIO.
    • No deadlock, hang, BUG, or crash occurs in production.
  • IsNetCandidate: false The issue is entirely within the block layer core, blk-mq, blk-wbt, and Device Mapper (dm-ioctl). No networking components are involved.
  • IsKVMCandidate: false No virtualization, hypervisor, or KVM subsystem interfaces are involved.
▶ ▼ ✨ AI Jobs (2)
ID Workflow Result Correct Bug Created Started Finished Revision
5cd01b4a repro-c 💥 possible deadlock in wbt_init_enable_default 2026/10/30 15:10 2026/10/30 15:10 2026/10/31 04:52 0ec395f8
d8e6aa49 assessment-security DenialOfService: ❌ Exploitable: ❌ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ❌ NetworkNeedsSetup: ❌ NetworkTrigger: ❌ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ❌ UserNamespace: ❌ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ possible deadlock in wbt_init_enable_default 2026/10/29 04:10 2026/10/29 04:10 2026/10/29 06:40 b1d00f76
▶ ▼ Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [block?] possible deadlock in wbt_init_enable_default 0 (1) 2026/10/31 23:30

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
syzkaller #0 Tainted: G             L     
------------------------------------------------------
syz.3.568/7072 is trying to acquire lock:
ffff8880316b08a8 (&q->debugfs_mutex){+.+.}-{4:4}, at: blk_debugfs_lock_nomemsave block/blk.h:816 [inline]
ffff8880316b08a8 (&q->debugfs_mutex){+.+.}-{4:4}, at: blk_debugfs_lock block/blk.h:830 [inline]
ffff8880316b08a8 (&q->debugfs_mutex){+.+.}-{4:4}, at: wbt_init_enable_default+0xf6/0x170 block/blk-wbt.c:794

but task is already holding lock:
ffff8880316b0060 (&q->q_usage_counter(io)#69){++++}-{0:0}, at: blk_mq_freeze_queue_nomemsave+0x15/0x20 block/blk-mq.c:206

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (&q->q_usage_counter(io)#69){++++}-{0:0}:
       lock_acquire kernel/locking/lockdep.c:5942 [inline]
       lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
       blk_alloc_queue+0x610/0x790 block/blk-core.c:505
       __blk_alloc_disk+0xa0/0x170 block/genhd.c:1527
       alloc_dev drivers/md/dm.c:2349 [inline]
       dm_create+0x5e8/0x1530 drivers/md/dm.c:2563
       dev_create+0x121/0x2a0 drivers/md/dm-ioctl.c:977
       ctl_ioctl+0x4d2/0xcc0 drivers/md/dm-ioctl.c:2221
       dm_ctl_ioctl+0x22/0x30 drivers/md/dm-ioctl.c:2243
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:597 [inline]
       __se_sys_ioctl fs/ioctl.c:583 [inline]
       __x64_sys_ioctl+0x18e/0x210 fs/ioctl.c:583
       do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
       do_syscall_64+0x123/0x7b0 arch/x86/entry/syscall_64.c:89
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #2 (fs_reclaim){+.+.}-{0:0}:
       lock_acquire kernel/locking/lockdep.c:5942 [inline]
       lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
       __fs_reclaim_acquire mm/page_alloc.c:4365 [inline]
       fs_reclaim_acquire+0xc4/0x100 mm/page_alloc.c:4379
       might_alloc include/linux/sched/mm.h:316 [inline]
       slab_pre_alloc_hook mm/slub.c:4673 [inline]
       slab_alloc_node mm/slub.c:5011 [inline]
       kmem_cache_alloc_lru_noprof+0x5f/0x6a0 mm/slub.c:5066
       __d_alloc+0x34/0xad0 fs/dcache.c:1939
       d_alloc+0x4a/0x1e0 fs/dcache.c:2025
       lookup_one_qstr_excl+0x171/0x250 fs/namei.c:1812
       __start_dirop fs/namei.c:2926 [inline]
       start_dirop+0x59/0xb0 fs/namei.c:2948
       simple_start_creating+0xf9/0x110 fs/libfs.c:2310
       debugfs_start_creating.part.0+0x7b/0x150 fs/debugfs/inode.c:397
       debugfs_start_creating fs/debugfs/inode.c:371 [inline]
       debugfs_create_dir+0x6c/0x440 fs/debugfs/inode.c:575
       regulator_init+0x97/0x180 drivers/regulator/core.c:6787
       do_one_initcall+0x11c/0x6f0 init/main.c:1419
       do_initcall_level init/main.c:1481 [inline]
       do_initcalls init/main.c:1497 [inline]
       do_basic_setup init/main.c:1517 [inline]
       kernel_init_freeable+0x6ea/0x7b0 init/main.c:1732
       kernel_init+0x21/0x1e0 init/main.c:1622
       ret_from_fork+0x730/0xd60 arch/x86/kernel/process.c:158
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

-> #1 (&sb->s_type->i_mutex_key#3/1){+.+.}-{4:4}:
       lock_acquire kernel/locking/lockdep.c:5942 [inline]
       lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
       down_write_nested+0x8f/0x1f0 kernel/locking/rwsem.c:1757
       inode_lock_nested include/linux/fs.h:1071 [inline]
       __start_dirop fs/namei.c:2924 [inline]
       start_dirop+0x4b/0xb0 fs/namei.c:2948
       simple_start_creating+0xf9/0x110 fs/libfs.c:2310
       debugfs_start_creating.part.0+0x7b/0x150 fs/debugfs/inode.c:397
       debugfs_start_creating fs/debugfs/inode.c:371 [inline]
       debugfs_create_dir+0x6c/0x440 fs/debugfs/inode.c:575
       blk_register_queue+0x199/0x640 block/blk-sysfs.c:968
       __add_disk+0x7dd/0xf10 block/genhd.c:531
       add_disk_fwnode+0x279/0x480 block/genhd.c:604
       add_disk include/linux/blkdev.h:799 [inline]
       brd_alloc.isra.0+0x5ab/0x820 drivers/block/brd.c:340
       brd_init+0x167/0x190 drivers/block/brd.c:420
       do_one_initcall+0x11c/0x6f0 init/main.c:1419
       do_initcall_level init/main.c:1481 [inline]
       do_initcalls init/main.c:1497 [inline]
       do_basic_setup init/main.c:1517 [inline]
       kernel_init_freeable+0x6ea/0x7b0 init/main.c:1732
       kernel_init+0x21/0x1e0 init/main.c:1622
       ret_from_fork+0x730/0xd60 arch/x86/kernel/process.c:158
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

-> #0 (&q->debugfs_mutex){+.+.}-{4:4}:
       check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3209
       check_prevs_add kernel/locking/lockdep.c:3328 [inline]
       validate_chain kernel/locking/lockdep.c:3952 [inline]
       __lock_acquire+0x1528/0x1f40 kernel/locking/lockdep.c:5288
       lock_acquire kernel/locking/lockdep.c:5942 [inline]
       lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
       __mutex_lock_common kernel/locking/mutex.c:646 [inline]
       __mutex_lock+0x19f/0x1c10 kernel/locking/mutex.c:821
       blk_debugfs_lock_nomemsave block/blk.h:816 [inline]
       blk_debugfs_lock block/blk.h:830 [inline]
       wbt_init_enable_default+0xf6/0x170 block/blk-wbt.c:794
       blk_register_queue+0x4b8/0x640 block/blk-sysfs.c:995
       __add_disk+0x7dd/0xf10 block/genhd.c:531
       add_disk_fwnode+0x118/0x480 block/genhd.c:600
       add_disk include/linux/blkdev.h:799 [inline]
       dm_setup_md_queue+0x1e2/0x550 drivers/md/dm.c:2632
       table_load+0x4e8/0x7b0 drivers/md/dm-ioctl.c:1661
       ctl_ioctl+0x4d2/0xcc0 drivers/md/dm-ioctl.c:2221
       dm_ctl_ioctl+0x22/0x30 drivers/md/dm-ioctl.c:2243
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:597 [inline]
       __se_sys_ioctl fs/ioctl.c:583 [inline]
       __x64_sys_ioctl+0x18e/0x210 fs/ioctl.c:583
       do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
       do_syscall_64+0x123/0x7b0 arch/x86/entry/syscall_64.c:89
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

Chain exists of:
  &q->debugfs_mutex --> fs_reclaim --> &q->q_usage_counter(io)#69

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&q->q_usage_counter(io)#69);
                               lock(fs_reclaim);
                               lock(&q->q_usage_counter(io)#69);
  lock(&q->debugfs_mutex);

 *** DEADLOCK ***

locks held by syz.3.568/7072: 6, on CPU#0:
 #0: ffff88806aefa190 (&md->type_lock){+.+.}-{4:4}, at: table_load+0x22d/0x7b0 drivers/md/dm-ioctl.c:1644
 #1: ffff88806aefa0e8 (&md->table_devices_lock){+.+.}-{4:4}, at: dm_setup_md_queue+0x1ab/0x550 drivers/md/dm.c:2631
 #2: ffff888027a5f1c0 (&set->update_nr_hwq_lock){++++}-{4:4}, at: add_disk_fwnode+0x107/0x480 block/genhd.c:599
 #3: ffff8880316b0610 (&q->sysfs_lock){+.+.}-{4:4}, at: blk_register_queue+0x10a/0x640 block/blk-sysfs.c:965
 #4: ffff8880316b0060 (&q->q_usage_counter(io)#69){++++}-{0:0}, at: blk_mq_freeze_queue_nomemsave+0x15/0x20 block/blk-mq.c:206
 #5: ffff8880316b0098 (&q->q_usage_counter(queue)#52){+.+.}-{0:0}, at: blk_mq_freeze_queue_nomemsave+0x15/0x20 block/blk-mq.c:206

stack backtrace:
CPU: 0 UID: 0 PID: 7072 Comm: syz.3.568 Tainted: G             L      syzkaller #0 PREEMPT(full) 
Tainted: [L]=SOFTLOCKUP
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/06/2026
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120
 print_circular_bug.cold+0x178/0x1be kernel/locking/lockdep.c:2087
 check_noncircular+0x146/0x160 kernel/locking/lockdep.c:2219
 check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3209
 check_prevs_add kernel/locking/lockdep.c:3328 [inline]
 validate_chain kernel/locking/lockdep.c:3952 [inline]
 __lock_acquire+0x1528/0x1f40 kernel/locking/lockdep.c:5288
 lock_acquire kernel/locking/lockdep.c:5942 [inline]
 lock_acquire+0x1d1/0x380 kernel/locking/lockdep.c:5899
 __mutex_lock_common kernel/locking/mutex.c:646 [inline]
 __mutex_lock+0x19f/0x1c10 kernel/locking/mutex.c:821
 blk_debugfs_lock_nomemsave block/blk.h:816 [inline]
 blk_debugfs_lock block/blk.h:830 [inline]
 wbt_init_enable_default+0xf6/0x170 block/blk-wbt.c:794
 blk_register_queue+0x4b8/0x640 block/blk-sysfs.c:995
 __add_disk+0x7dd/0xf10 block/genhd.c:531
 add_disk_fwnode+0x118/0x480 block/genhd.c:600
 add_disk include/linux/blkdev.h:799 [inline]
 dm_setup_md_queue+0x1e2/0x550 drivers/md/dm.c:2632
 table_load+0x4e8/0x7b0 drivers/md/dm-ioctl.c:1661
 ctl_ioctl+0x4d2/0xcc0 drivers/md/dm-ioctl.c:2221
 dm_ctl_ioctl+0x22/0x30 drivers/md/dm-ioctl.c:2243
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:597 [inline]
 __se_sys_ioctl fs/ioctl.c:583 [inline]
 __x64_sys_ioctl+0x18e/0x210 fs/ioctl.c:583
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x123/0x7b0 arch/x86/entry/syscall_64.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f4478da8f19
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f4479cbb028 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f4479025fa0 RCX: 00007f4478da8f19
RDX: 0000200000000200 RSI: 00000000c138fd09 RDI: 0000000000000003
RBP: 00007f4478e37f6d R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f4479026038 R14: 00007f4479025fa0 R15: 00007fff5a8c85f8
 </TASK>
device-mapper: ioctl: device has been removed from the dev hash table.

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/10 03:23 upstream b3dfbde62cf1 70ec0380 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in wbt_init_enable_default
2026/11/07 21:39 upstream d164c7d754f0 5cecaf1f .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in wbt_init_enable_default
2026/11/03 19:11 upstream 05441d697f7f e2d35131 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in wbt_init_enable_default
2026/11/08 02:46 linux-next ddfe51b0a0f5 5cecaf1f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-rust-kasan-gce possible deadlock in wbt_init_enable_default
2026/10/27 23:19 linux-next 151a134a4e6e 96923c60 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-rust-kasan-gce possible deadlock in wbt_init_enable_default
* Struck through repros no longer work on HEAD.