====================================================== WARNING: possible circular locking dependency detected syzkaller #0 Not tainted ------------------------------------------------------ kworker/0:1H/52 is trying to acquire lock: ffff0000d038dd38 ((wq_completion)nvmet-wq){+.+.}-{0:0}, at: __flush_workqueue+0x108/0x1060 kernel/workqueue.c:2809 but task is already holding lock: ffff80001f157c40 ((work_completion)(&queue->io_work)){+.+.}-{0:0}, at: process_one_work+0x6e8/0x13bc kernel/workqueue.c:2267 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #2 ((work_completion)(&queue->io_work)){+.+.}-{0:0}: __flush_work+0x10c/0x1ec kernel/workqueue.c:3072 __cancel_work_timer+0x2f8/0x450 kernel/workqueue.c:3163 cancel_work_sync+0x24/0x38 kernel/workqueue.c:3199 nvmet_tcp_release_queue_work+0x270/0xc24 drivers/nvme/target/tcp.c:1558 process_one_work+0x7e4/0x13bc kernel/workqueue.c:2292 worker_thread+0x8cc/0xfe8 kernel/workqueue.c:2439 kthread+0x254/0x2e0 kernel/kthread.c:376 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:850 -> #1 ((work_completion)(&queue->release_work)){+.+.}-{0:0}: process_one_work+0x710/0x13bc kernel/workqueue.c:2268 worker_thread+0x8cc/0xfe8 kernel/workqueue.c:2439 kthread+0x254/0x2e0 kernel/kthread.c:376 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:850 -> #0 ((wq_completion)nvmet-wq){+.+.}-{0:0}: check_prev_add kernel/locking/lockdep.c:3162 [inline] check_prevs_add kernel/locking/lockdep.c:3281 [inline] validate_chain kernel/locking/lockdep.c:3897 [inline] __lock_acquire+0x3504/0x7280 kernel/locking/lockdep.c:5171 lock_acquire+0x210/0x63c kernel/locking/lockdep.c:5790 __flush_workqueue+0x134/0x1060 kernel/workqueue.c:2812 nvmet_tcp_install_queue+0x9c/0x2e8 drivers/nvme/target/tcp.c:1915 nvmet_install_queue+0x29c/0x654 drivers/nvme/target/fabrics-cmd.c:185 nvmet_execute_admin_connect+0x2a0/0x660 drivers/nvme/target/fabrics-cmd.c:266 nvmet_tcp_execute_request drivers/nvme/target/tcp.c:599 [inline] nvmet_tcp_try_recv_data drivers/nvme/target/tcp.c:1254 [inline] nvmet_tcp_try_recv_one drivers/nvme/target/tcp.c:1318 [inline] nvmet_tcp_try_recv drivers/nvme/target/tcp.c:1344 [inline] nvmet_tcp_io_work+0x11ec/0x6bd0 drivers/nvme/target/tcp.c:1394 process_one_work+0x7e4/0x13bc kernel/workqueue.c:2292 worker_thread+0x8cc/0xfe8 kernel/workqueue.c:2439 kthread+0x254/0x2e0 kernel/kthread.c:376 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:850 other info that might help us debug this: Chain exists of: (wq_completion)nvmet-wq --> (work_completion)(&queue->release_work) --> (work_completion)(&queue->io_work) Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock((work_completion)(&queue->io_work)); lock((work_completion)(&queue->release_work)); lock((work_completion)(&queue->io_work)); lock((wq_completion)nvmet-wq); *** DEADLOCK *** 2 locks held by kworker/0:1H/52: #0: ffff0000d038e538 ((wq_completion)nvmet_tcp_wq){+.+.}-{0:0}, at: process_one_work+0x69c/0x13bc kernel/workqueue.c:-1 #1: ffff80001f157c40 ((work_completion)(&queue->io_work)){+.+.}-{0:0}, at: process_one_work+0x6e8/0x13bc kernel/workqueue.c:2267 stack backtrace: CPU: 0 PID: 52 Comm: kworker/0:1H Not tainted syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/14/2026 Workqueue: nvmet_tcp_wq nvmet_tcp_io_work dump_backtrace(regs = 0000000000000000 tsk = 0000000000000000) Call trace: dump_backtrace+0x1c4/0x1f0 arch/arm64/kernel/stacktrace.c:158 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:165 __dump_stack+0x30/0x40 lib/dump_stack.c:88 dump_stack_lvl+0xf4/0x15c lib/dump_stack.c:106 dump_stack+0x1c/0x5c lib/dump_stack.c:113 print_circular_bug+0x148/0x1b0 kernel/locking/lockdep.c:2088 check_noncircular+0x290/0x370 kernel/locking/lockdep.c:2215 check_prev_add kernel/locking/lockdep.c:3162 [inline] check_prevs_add kernel/locking/lockdep.c:3281 [inline] validate_chain kernel/locking/lockdep.c:3897 [inline] __lock_acquire+0x3504/0x7280 kernel/locking/lockdep.c:5171 lock_acquire+0x210/0x63c kernel/locking/lockdep.c:5790 __flush_workqueue+0x134/0x1060 kernel/workqueue.c:2812 nvmet_tcp_install_queue+0x9c/0x2e8 drivers/nvme/target/tcp.c:1915 nvmet_install_queue+0x29c/0x654 drivers/nvme/target/fabrics-cmd.c:185 nvmet_execute_admin_connect+0x2a0/0x660 drivers/nvme/target/fabrics-cmd.c:266 nvmet_tcp_execute_request drivers/nvme/target/tcp.c:599 [inline] nvmet_tcp_try_recv_data drivers/nvme/target/tcp.c:1254 [inline] nvmet_tcp_try_recv_one drivers/nvme/target/tcp.c:1318 [inline] nvmet_tcp_try_recv drivers/nvme/target/tcp.c:1344 [inline] nvmet_tcp_io_work+0x11ec/0x6bd0 drivers/nvme/target/tcp.c:1394 process_one_work+0x7e4/0x13bc kernel/workqueue.c:2292 worker_thread+0x8cc/0xfe8 kernel/workqueue.c:2439 kthread+0x254/0x2e0 kernel/kthread.c:376 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:850 nvmet: creating nvm controller 1 for subsystem testsubsys for NQN nqn.2014-08.org.nvmexpress:uuid:a4c64857-2055-4056-aeed-201a1e3a8952. tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 000000002c3ef090 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nvmet: adding queue 1 to ctrl 1. tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nvmet: ctrl 1 update keep-alive timer for 5 secs tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nvmet: ctrl 1 update keep-alive timer for 5 secs tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nvmet: ctrl 1 update keep-alive timer for 5 secs tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nvmet: ctrl 1 update keep-alive timer for 5 secs tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nvmet: ctrl 1 update keep-alive timer for 5 secs tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nf_conntrack: normal packet for 00000000e1430935 tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3 nvmet: ctrl 1 update keep-alive timer for 5 secs tcp_conntracks: syn=0 ack=1 fin=0 rst=0 old=3 new=3