syzbot
possible deadlock in rds_tcp_recv_path (2)
Status: auto-obsoleted due to no activity on 2026/10/31 00:00
Subsystems:
rds
Labels:
prio:low
[Documentation on labels]
First crash: 110d, last: 110d
Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
syzkaller #0 Not tainted
------------------------------------------------------
kworker/u8:37/16253 is trying to acquire lock:
ffffffff8ec848e0 (fs_reclaim){+.+.}-{0:0}, at: might_alloc include/linux/sched/mm.h:317 [inline]
ffffffff8ec848e0 (fs_reclaim){+.+.}-{0:0}, at: slab_pre_alloc_hook mm/slub.c:4537 [inline]
ffffffff8ec848e0 (fs_reclaim){+.+.}-{0:0}, at: slab_alloc_node mm/slub.c:4897 [inline]
ffffffff8ec848e0 (fs_reclaim){+.+.}-{0:0}, at: kmem_cache_alloc_noprof+0x64/0x5f0 mm/slub.c:4931
but task is already holding lock:
ffff8880329f6e60 (k-sk_lock-AF_INET6){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1713 [inline]
ffff8880329f6e60 (k-sk_lock-AF_INET6){+.+.}-{0:0}, at: rds_tcp_recv_path+0xfe/0x340 net/rds/tcp_recv.c:304
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #7 (k-sk_lock-AF_INET6){+.+.}-{0:0}:
lock_sock_nested+0x41/0x100 net/core/sock.c:3828
lock_sock include/net/sock.h:1713 [inline]
__inet6_bind+0x301/0xf30 net/ipv6/af_inet6.c:292
mptcp_bind+0x110/0x1d0 net/mptcp/protocol.c:4122
__sys_bind_socket net/socket.c:1920 [inline]
__sys_bind+0x2e3/0x410 net/socket.c:1951
__do_sys_bind net/socket.c:1956 [inline]
__se_sys_bind net/socket.c:1954 [inline]
__x64_sys_bind+0x7a/0x90 net/socket.c:1954
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x174/0x580 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #6 (sk_lock-AF_INET6){+.+.}-{0:0}:
lock_sock_nested+0x41/0x100 net/core/sock.c:3828
lock_sock include/net/sock.h:1713 [inline]
inet_shutdown+0x6a/0x390 net/ipv4/af_inet.c:913
nbd_mark_nsock_dead+0x2cb/0x550 drivers/block/nbd.c:318
recv_work+0x1cee/0x1e10 drivers/block/nbd.c:1021
process_one_work kernel/workqueue.c:3322 [inline]
process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3405
worker_thread+0xa47/0xfb0 kernel/workqueue.c:3486
kthread+0x388/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
-> #5 (&nsock->tx_lock){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x19d/0x1550 kernel/locking/mutex.c:821
nbd_handle_cmd drivers/block/nbd.c:1143 [inline]
nbd_queue_rq+0x373/0x1150 drivers/block/nbd.c:1207
blk_mq_dispatch_rq_list+0x499/0x1990 block/blk-mq.c:2117
__blk_mq_do_dispatch_sched block/blk-mq-sched.c:168 [inline]
blk_mq_do_dispatch_sched block/blk-mq-sched.c:182 [inline]
__blk_mq_sched_dispatch_requests+0xd36/0x1580 block/blk-mq-sched.c:307
blk_mq_sched_dispatch_requests+0xd7/0x190 block/blk-mq-sched.c:329
blk_mq_run_work_fn+0x16c/0x300 block/blk-mq.c:2532
process_one_work kernel/workqueue.c:3322 [inline]
process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3405
worker_thread+0xa47/0xfb0 kernel/workqueue.c:3486
kthread+0x388/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
-> #4 (&cmd->lock){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x19d/0x1550 kernel/locking/mutex.c:821
nbd_queue_rq+0xc1/0x1150 drivers/block/nbd.c:1199
blk_mq_dispatch_rq_list+0x499/0x1990 block/blk-mq.c:2117
__blk_mq_do_dispatch_sched block/blk-mq-sched.c:168 [inline]
blk_mq_do_dispatch_sched block/blk-mq-sched.c:182 [inline]
__blk_mq_sched_dispatch_requests+0xd36/0x1580 block/blk-mq-sched.c:307
blk_mq_sched_dispatch_requests+0xd7/0x190 block/blk-mq-sched.c:329
blk_mq_run_work_fn+0x16c/0x300 block/blk-mq.c:2532
process_one_work kernel/workqueue.c:3322 [inline]
process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3405
worker_thread+0xa47/0xfb0 kernel/workqueue.c:3486
kthread+0x388/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
-> #3 (set->srcu){.+.+}-{0:0}:
srcu_lock_sync include/linux/srcu.h:199 [inline]
__synchronize_srcu+0xc9/0x2f0 kernel/rcu/srcutree.c:1481
elevator_switch+0x1e8/0x7b0 block/elevator.c:576
elevator_change+0x2fa/0x480 block/elevator.c:681
elevator_set_default+0x375/0x440 block/elevator.c:754
blk_register_queue+0x3f3/0x4e0 block/blk-sysfs.c:992
__add_disk+0x6cb/0xe30 block/genhd.c:524
add_disk_fwnode+0x100/0x3a0 block/genhd.c:593
add_disk include/linux/blkdev.h:800 [inline]
nbd_dev_add+0x733/0xb60 drivers/block/nbd.c:2021
nbd_init+0x15f/0x1e0 drivers/block/nbd.c:2729
do_one_initcall+0x250/0x870 init/main.c:1347
do_initcall_level+0x10a/0x1a0 init/main.c:1409
do_initcalls+0x59/0xa0 init/main.c:1425
kernel_init_freeable+0x29d/0x3e0 init/main.c:1658
kernel_init+0x1d/0x1d0 init/main.c:1548
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
-> #2 (&q->elevator_lock){+.+.}-{4:4}:
__mutex_lock_common kernel/locking/mutex.c:646 [inline]
__mutex_lock+0x19d/0x1550 kernel/locking/mutex.c:821
elevator_change+0x1af/0x480 block/elevator.c:679
elevator_set_none+0xb5/0x140 block/elevator.c:769
blk_mq_elv_switch_none block/blk-mq.c:5101 [inline]
__blk_mq_update_nr_hw_queues block/blk-mq.c:5146 [inline]
blk_mq_update_nr_hw_queues+0x5ef/0x19f0 block/blk-mq.c:5211
nbd_start_device+0x189/0xb30 drivers/block/nbd.c:1526
nbd_genl_connect+0x1597/0x1c10 drivers/block/nbd.c:2276
genl_family_rcv_msg_doit+0x233/0x340 net/netlink/genetlink.c:1114
genl_family_rcv_msg net/netlink/genetlink.c:1194 [inline]
genl_rcv_msg+0x614/0x7a0 net/netlink/genetlink.c:1209
netlink_rcv_skb+0x226/0x4a0 net/netlink/af_netlink.c:2556
genl_rcv+0x28/0x40 net/netlink/genetlink.c:1218
netlink_unicast_kernel net/netlink/af_netlink.c:1319 [inline]
netlink_unicast+0x7bb/0x940 net/netlink/af_netlink.c:1345
netlink_sendmsg+0x813/0xb40 net/netlink/af_netlink.c:1900
sock_sendmsg_nosec+0x13a/0x180 net/socket.c:775
__sock_sendmsg net/socket.c:790 [inline]
____sys_sendmsg+0x54e/0x850 net/socket.c:2684
___sys_sendmsg+0x2a5/0x360 net/socket.c:2738
__sys_sendmsg net/socket.c:2770 [inline]
__do_sys_sendmsg net/socket.c:2775 [inline]
__se_sys_sendmsg net/socket.c:2773 [inline]
__x64_sys_sendmsg+0x1b1/0x290 net/socket.c:2773
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x174/0x580 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
-> #1 (&q->q_usage_counter(io)#50){++++}-{0:0}:
blk_alloc_queue+0x544/0x690 block/blk-core.c:504
blk_mq_alloc_queue block/blk-mq.c:4420 [inline]
__blk_mq_alloc_disk+0x194/0x390 block/blk-mq.c:4467
nbd_dev_add+0x494/0xb60 drivers/block/nbd.c:1991
nbd_init+0x15f/0x1e0 drivers/block/nbd.c:2729
do_one_initcall+0x250/0x870 init/main.c:1347
do_initcall_level+0x10a/0x1a0 init/main.c:1409
do_initcalls+0x59/0xa0 init/main.c:1425
kernel_init_freeable+0x29d/0x3e0 init/main.c:1658
kernel_init+0x1d/0x1d0 init/main.c:1548
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
-> #0 (fs_reclaim){+.+.}-{0:0}:
check_prev_add kernel/locking/lockdep.c:3165 [inline]
check_prevs_add kernel/locking/lockdep.c:3284 [inline]
validate_chain kernel/locking/lockdep.c:3908 [inline]
__lock_acquire+0x1520/0x2cf0 kernel/locking/lockdep.c:5237
lock_acquire+0x106/0x350 kernel/locking/lockdep.c:5868
__fs_reclaim_acquire mm/page_alloc.c:4329 [inline]
fs_reclaim_acquire+0x71/0x100 mm/page_alloc.c:4343
might_alloc include/linux/sched/mm.h:317 [inline]
slab_pre_alloc_hook mm/slub.c:4537 [inline]
slab_alloc_node mm/slub.c:4897 [inline]
kmem_cache_alloc_noprof+0x64/0x5f0 mm/slub.c:4931
rds_tcp_data_recv+0x174/0xa00 net/rds/tcp_recv.c:174
__tcp_read_sock+0x18d/0x970 net/ipv4/tcp.c:1688
rds_tcp_read_sock net/rds/tcp_recv.c:277 [inline]
rds_tcp_recv_path+0x200/0x340 net/rds/tcp_recv.c:305
rds_tcp_conn_path_shutdown+0x29b/0xac0 net/rds/tcp_connect.c:269
rds_conn_shutdown+0x4b8/0xab0 net/rds/connection.c:414
process_one_work kernel/workqueue.c:3322 [inline]
process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3405
worker_thread+0xa47/0xfb0 kernel/workqueue.c:3486
kthread+0x388/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
other info that might help us debug this:
Chain exists of:
fs_reclaim --> sk_lock-AF_INET6 --> k-sk_lock-AF_INET6
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(k-sk_lock-AF_INET6);
lock(sk_lock-AF_INET6);
lock(k-sk_lock-AF_INET6);
lock(fs_reclaim);
*** DEADLOCK ***
3 locks held by kworker/u8:37/16253:
#0: ffff888054dff140 ((wq_completion)krds_cp_wq#14/0){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3297 [inline]
#0: ffff888054dff140 ((wq_completion)krds_cp_wq#14/0){+.+.}-{0:0}, at: process_scheduled_works+0xa20/0x14e0 kernel/workqueue.c:3405
#1: ffffc90006e47c40 ((work_completion)(&cp->cp_down_w)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3297 [inline]
#1: ffffc90006e47c40 ((work_completion)(&cp->cp_down_w)){+.+.}-{0:0}, at: process_scheduled_works+0xa20/0x14e0 kernel/workqueue.c:3405
#2: ffff8880329f6e60 (k-sk_lock-AF_INET6){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1713 [inline]
#2: ffff8880329f6e60 (k-sk_lock-AF_INET6){+.+.}-{0:0}, at: rds_tcp_recv_path+0xfe/0x340 net/rds/tcp_recv.c:304
stack backtrace:
CPU: 0 UID: 0 PID: 16253 Comm: kworker/u8:37 Not tainted syzkaller #0 PREEMPT(full)
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/24/2026
Workqueue: krds_cp_wq#14/0 rds_shutdown_worker
Call Trace:
<TASK>
dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120
print_circular_bug+0x2e1/0x300 kernel/locking/lockdep.c:2043
check_noncircular+0x12e/0x150 kernel/locking/lockdep.c:2175
check_prev_add kernel/locking/lockdep.c:3165 [inline]
check_prevs_add kernel/locking/lockdep.c:3284 [inline]
validate_chain kernel/locking/lockdep.c:3908 [inline]
__lock_acquire+0x1520/0x2cf0 kernel/locking/lockdep.c:5237
lock_acquire+0x106/0x350 kernel/locking/lockdep.c:5868
__fs_reclaim_acquire mm/page_alloc.c:4329 [inline]
fs_reclaim_acquire+0x71/0x100 mm/page_alloc.c:4343
might_alloc include/linux/sched/mm.h:317 [inline]
slab_pre_alloc_hook mm/slub.c:4537 [inline]
slab_alloc_node mm/slub.c:4897 [inline]
kmem_cache_alloc_noprof+0x64/0x5f0 mm/slub.c:4931
rds_tcp_data_recv+0x174/0xa00 net/rds/tcp_recv.c:174
__tcp_read_sock+0x18d/0x970 net/ipv4/tcp.c:1688
rds_tcp_read_sock net/rds/tcp_recv.c:277 [inline]
rds_tcp_recv_path+0x200/0x340 net/rds/tcp_recv.c:305
rds_tcp_conn_path_shutdown+0x29b/0xac0 net/rds/tcp_connect.c:269
rds_conn_shutdown+0x4b8/0xab0 net/rds/connection.c:414
process_one_work kernel/workqueue.c:3322 [inline]
process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3405
worker_thread+0xa47/0xfb0 kernel/workqueue.c:3486
kthread+0x388/0x470 kernel/kthread.c:436
ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
</TASK>
* Struck through repros no longer work on HEAD.