Technical Breakdown & Root Cause Analysis
1. Triggering Context and Call Path
When a BPF arena map (BPF_MAP_TYPE_ARENA) is created in kernel/bpf/arena.c:arena_map_alloc(), it reserves a kernel virtual address area of size KERN_VM_SZ (SZ_4G + GUARD_SZ, roughly 4 GiB):
kern_vm = get_vm_area(KERN_VM_SZ, VM_SPARSE | VM_USERMAP);
This area is a sparse vmalloc reservation used for shared memory between BPF programs and userspace.
When the BPF map is destroyed asynchronously via the deferred workqueue callback bpf_map_free_deferred() -> arena_map_free() (kernel/bpf/arena.c:400):
free_vm_area(arena->kern_vm);
free_vm_area() calls remove_vm_area() (mm/vmalloc.c:3352), which executes debug validation hooks across the full allocated range before freeing the vmap_area:
debug_check_no_locks_freed(vm->addr, get_vm_area_size(vm));
debug_check_no_obj_freed(vm->addr, get_vm_area_size(vm));
When CONFIG_DEBUG_OBJECTS_FREE is enabled, debug_check_no_obj_freed() calls __debug_check_no_obj_freed() (lib/debugobjects.c:1141):
static void __debug_check_no_obj_freed(const void *address, unsigned long size)
{
unsigned long flags, oaddr, saddr, eaddr, paddr, chunks;
...
saddr = (unsigned long) address;
eaddr = saddr + size;
paddr = saddr & ODEBUG_CHUNK_MASK;
chunks = ((eaddr - paddr) + (ODEBUG_CHUNK_SIZE - 1));
chunks >>= ODEBUG_CHUNK_SHIFT;
for (;chunks > 0; chunks--, paddr += ODEBUG_CHUNK_SIZE) {
db = get_bucket(paddr);
repeat:
cnt = 0;
raw_spin_lock_irqsave(&db->lock, flags);
hlist_for_each_entry_safe(obj, tmp, &db->list, node) {
...
}
raw_spin_unlock_irqrestore(&db->lock, flags);
...
}
ODEBUG_CHUNK_SHIFT is PAGE_SHIFT (PAGE_SIZE, typically 4 KiB).- For a 4 GiB range (
KERN_VM_SZ), chunks evaluates to (SZ_4G + GUARD_SZ) / PAGE_SIZE, which is approximately 1,048,592 iterations.
ODEBUG_HASH_SIZE is only 1 << 14 = 16,384 buckets. As chunks advances through the linear address space, get_bucket(paddr) repeatedly wraps around and revisits the exact same 16,384 hash buckets 64 times over.- In each iteration,
raw_spin_lock_irqsave(&db->lock, flags) disables local IRQs and takes the bucket lock.
- There is no
cond_resched() in the loop, nor is chunks capped to ODEBUG_HASH_SIZE (scanning every hash bucket once would suffice to detect any debug object falling in [saddr, eaddr)).
- Consequently, executing over 1 million iterations of spinlock acquire/release operations in kernel worker context monopolizes the CPU for more than 23 seconds, tripping the kernel watchdog (
watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/u8:8:4093]).
3. Production Kernel Impact
- Denial of Service (
RawDenialOfService = false): In standard production builds, kernel debugging options (CONFIG_DEBUG_OBJECTS and CONFIG_DEBUG_OBJECTS_FREE) are disabled. In include/linux/debugobjects.h:115, debug_check_no_obj_freed() compiles to a complete no-op (static inline void debug_check_no_obj_freed(...) {}). Therefore, the loop is entirely omitted, and no CPU stall or watchdog lockup occurs at runtime in production.
- Memory Corruption (
MemoryCorruption = false): No invalid memory access, write, use-after-free, or corrupted state occurs; this is purely an excessive CPU iteration stall in the debugobjects checker.
- Information Leak (
InfoLeak = "none"): No kernel memory contents or addresses are leaked.
- Network / KVM Candidacy (
IsNetCandidate = false, IsKVMCandidate = false): The issue resides in BPF arena virtual memory tear-down and the core debugobjects hash iteration logic; it is unrelated to networking (CPU 0's NMI backtrace was simply interrupted handling SSH traffic) or hypervisor virtualization.