Root Cause Technical Analysis
1. Overview and Invariant Background
The Network Block Device (drivers/block/nbd.c) driver dispatches block device I/O over network sockets. Because block devices participate directly in dirty page writeback and swap I/O initiated during direct memory reclaim (fs_reclaim), block request handling must strictly adhere to memory reclaim lock ordering invariants.
Normally, userspace sockets establish the lock ordering:
sk_lock (e.g. sk_lock-AF_INET6) -> mm->mmap_lock -> fs_reclaim
This occurs because socket system calls (such as sendmsg() in rawv6_sendmsg() or tcp_sendmsg()) acquire lock_sock(sk) before copying data from userspace buffers (copy_from_user()). If the buffer is unmapped or swapped out, a page fault occurs under lock_sock(sk), acquiring &mm->mmap_lock. When page allocation occurs under mmap_lock (e.g., during faults or core dump generation in dump_user_range()), memory allocation may trigger fs_reclaim.
To prevent an inverted dependency where block I/O performed under fs_reclaim acquires sk_lock, drivers that use sockets for storage (NBD, NVMe-oF/TCP, CIFS, SunRPC) reclassify socket locks into dedicated lockdep classes (such as sk_lock-AF_INET6-NBD for NBD) using sock_lock_init_class_and_name().
2. The Defect in NBD Socket Management (drivers/block/nbd.c)
In drivers/block/nbd.c, sockets are provided from userspace file descriptors via sockfd_lookup().
-
Silent Failure of Reclassification in nbd_add_socket():
In nbd_add_socket() (drivers/block/nbd.c:1296–1300):
sock = nbd_get_socket(nbd, arg, &err);
if (!sock)
return err;
nbd_reclassify_socket(sock);
nbd_reclassify_socket() calls sock_allow_reclassification(sk) (include/net/sock.h:1861), which verifies:
return !sock_owned_by_user_nocheck(sk) && !spin_is_locked(&sk->sk_lock.slock);
If another thread in userspace concurrently performs an operation on the same socket (holding lock_sock(sk) or sk->sk_lock.slock, e.g. in connect(), sendmsg(), or setsockopt()), sock_allow_reclassification() returns false.
nbd_reclassify_socket() then silently returns without modifying the socket's lockdep class. nbd_add_socket() ignores this condition, does not return an error (such as -EBUSY), and attaches the socket to config->socks[] while its lockdep map still bears the generic sk_lock-AF_INET6 class.
-
Missing Reclassification in nbd_genl_reconnect_sock_cb():
In nbd_genl_reconnect_sock_cb() (drivers/block/nbd.c:1348–1385), when a socket is reconnected, nbd_get_socket() is called, but nbd_reclassify_socket() is never invoked at all.
3. How the Cycle Closes
When the device encounters an error or disconnects (e.g., in nbd_genl_connect() -> nbd_config_put() -> nbd_clear_sock() -> sock_shutdown()):
mutex_lock(&nsock->tx_lock);
nbd_mark_nsock_dead(nbd, nsock, 0);
mutex_unlock(&nsock->tx_lock);
Inside nbd_mark_nsock_dead(), kernel_sock_shutdown() is invoked:
kernel_sock_shutdown(nsock->sock, SHUT_RDWR);
This calls inet_shutdown() (net/ipv4/af_inet.c:913), which calls:
lock_sock(sk);
Because the socket was never reclassified to sk_lock-AF_INET6-NBD, this establishes the edge:
&nsock->tx_lock -> sk_lock-AF_INET6
Combined with the block layer's reclaim path:
fs_reclaim -> &q->q_usage_counter(io) (established in blk_alloc_queue() to model reclaim-time I/O)
&q->q_usage_counter(io) -> &q->elevator_lock (queue freeze in elevator_change())
&q->elevator_lock -> set->srcu (elevator_switch() calls synchronize_srcu(set->srcu))
set->srcu -> &cmd->lock (blk_mq_dispatch_rq_list() runs nbd_queue_rq() inside srcu_read_lock(set->srcu))
&cmd->lock -> &nsock->tx_lock (nbd_queue_rq() calls nbd_handle_cmd(), which acquires &nsock->tx_lock)
&nsock->tx_lock -> sk_lock-AF_INET6 (sock_shutdown() calls nbd_mark_nsock_dead() -> kernel_sock_shutdown() -> inet_shutdown() -> lock_sock())
sk_lock-AF_INET6 -> &mm->mmap_lock (rawv6_sendmsg() -> copy_from_user() -> __might_fault())
&mm->mmap_lock -> fs_reclaim (elf_core_dump() / dump_user_range() -> page fault / page allocation)
This creates the circular locking dependency reported by lockdep:
fs_reclaim -> &q->q_usage_counter(io) -> &q->elevator_lock -> set->srcu -> &cmd->lock -> &nsock->tx_lock -> sk_lock-AF_INET6 -> &mm->mmap_lock -> fs_reclaim