syzbot
BUG: unable to handle kernel paging request in __run_timers (4)
Status: closed as invalid on 2026/11/16 17:46
Subsystems:
pci
Labels:
prio:normal
[Documentation on labels]
First crash: 7d21h, last: 7d17h
✨ AI-generated root cause summary:
Writing to /proc/bus/pci/00/00.0 via proc_bus_pci_write() allows a privileged user to write directly to the PCI configuration space of device 00:00.0 (the Host Bridge / Intel Q35 Memory Controller Hub). The write modified memory-decoding configuration registers, specifically lowering TOLUD (Top of Low Usable DRAM at offset 0x9c), which remapped a portion of physical DRAM into the 32-bit PCI MMIO hole. Subsequent CPU reads from kernel memory allocated within that physical range stopped reaching DRAM and instead returned 0xffffffffffffffff (PCI unassigned MMIO abort). When a hardware timer interrupt later invoked debug_object_deactivate(), lookup_object() dereferenced an hlist node pointer in that corrupted memory range, reading 0xffffffffffffffff and causing a general protection fault on the resulting non-canonical KASAN shadow address.
▶
▼
✨ Full AI root cause analysis
Root Cause Analysis
1. Functions, Structs, and Files Involved
proc_bus_pci_write() in drivers/pci/proc.c: The procfs handler that writes raw data directly to the PCI configuration space of a given PCI device.
lookup_object() in lib/debugobjects.c: Called by debug_object_deactivate() when deactivating an expiring timer; traverses b->list (struct hlist_head) of struct debug_obj entries to find the matching tracked object.
struct debug_obj (include/linux/debugobjects.h): Debug tracking structure containing struct hlist_node node at offset 0 and void *object at offset 0x18.
- Host Bridge / DRAM Controller (
0000:00:00.0): On QEMU (Q35 + ICH9), device 00:00.0 is the Intel Q35 MCH (Memory Controller Hub / DRAM Controller).
2. Trigger Mechanism and Lifecycle Invariant Violation
-
Unrestricted PCI Configuration Space Write:
The reproducer mounts procfs and opens /proc/bus/pci/00/00.0 (O_RDWR). Device 00:00.0 corresponds to the chipset host bridge (Intel Q35 MCH). A write of 0x10f (271) bytes is executed via proc_bus_pci_write().
-
Chipset Memory Decoding Corruption (TOLUD / PAM / PCIEXBAR):
- The write data modifies configuration registers of the Q35 MCH, notably register
0x9c (TOLUD – Top of Low Usable DRAM) as well as the PAM registers (0x90–0x96) and PCIEXBAR (0x60–0x67).
- Register
0x9c (TOLUD, 8-bit mask 0xf8) received 0x2f ('/'), lowering TOLUD to 0x28 (640 MB / 0x28000000).
- Under the Q35 memory map, addresses between
TOLUD (0x28000000) and 4 GB (0x100000000) are remapped to the 32-bit PCI MMIO hole. Any physical DRAM addresses above 640 MB are immediately disconnected from DRAM decoding.
-
Hardware Bus Abort on Memory Reads:
- The kernel itself, its page allocations, and active data structures were already resident across physical memory above 640 MB.
- Any CPU read directed at physical memory within the newly carved-out MMIO hole does not reach DRAM; it instead routes to unassigned MMIO space, which returns all-ones (
0xff, 0xffffffffffffffff) due to standard PCI master-abort termination.
-
Manifestation in lookup_object():
- A hardware APIC timer interrupt occurred while a worker thread was running in
cleanup_net.
- In the timer softirq,
expire_timers() invoked detach_timer() -> debug_deactivate() -> debug_object_deactivate() on a timer (ffff888025c16ec8, physical address 0x25c16ec8, ~604 MB, which remained in readable DRAM below 640 MB).
- In
lookup_object(), the hash bucket's first entry was inspected. The next pointer obj->node.next (mov (%rbx), %rbx), which was allocated in memory above 640 MB, read 0xffffffffffffffff instead of a valid pointer.
- On the next loop iteration, the code calculated
&obj->object at offset 0x18 from 0xffffffffffffffff, yielding effective address 0x0000000000000017.
- KASAN checked the shadow memory for
0x17 via cmpb $0, (%rax, %r13) (0x17 >> 3 + 0xdffffc0000000000 = 0xdffffc0000000002), triggering a general protection fault on the non-canonical shadow address.
Sample crash report:
Oops: general protection fault, probably for non-canonical address 0xdffffc0000000002: 0000 [#1] SMP KASAN NOPTI
KASAN: null-ptr-deref in range [0x0000000000000010-0x0000000000000017]
CPU: 0 UID: 0 PID: 14 Comm: kworker/u32:1 Not tainted syzkaller #0 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Workqueue: netns cleanup_net
RIP: 0010:lookup_object lib/debugobjects.c:440 [inline]
RIP: 0010:debug_object_deactivate+0x19b/0x3b0 lib/debugobjects.c:938
Code: 48 c1 e8 03 42 80 3c 28 00 0f 85 8e 01 00 00 48 8b 1b 48 85 db 0f 84 32 01 00 00 48 8d 7b 18 41 83 c4 01 48 89 f8 48 c1 e8 03 <42> 80 3c 28 00 0f 85 4c 01 00 00 4c 3b 73 18 75 c2 48 b8 00 00 00
RSP: 0018:ffffc90000007c60 EFLAGS: 00010003
RAX: 0000000000000002 RBX: ffffffffffffffff RCX: 0000000000000002
RDX: ffffffff9ce46d60 RSI: 0000000000000004 RDI: 0000000000000017
RBP: ffffc90000007d38 R08: 0000000000000001 R09: fffff52000000f7a
R10: 0000000000000003 R11: 0000000000000001 R12: 0000000000000002
R13: dffffc0000000000 R14: ffff888025c16ec8 R15: 1ffff92000000f8e
FS: 0000000000000000(0000) GS:ffff888095279000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00001def9e82a000 CR3: 0000000044d4a000 CR4: 0000000000352ef0
Call Trace:
<IRQ>
debug_timer_deactivate kernel/time/timer.c:799 [inline]
debug_deactivate kernel/time/timer.c:843 [inline]
detach_timer kernel/time/timer.c:890 [inline]
expire_timers kernel/time/timer.c:1783 [inline]
__run_timers+0x5de/0xb00 kernel/time/timer.c:2375
__run_timer_base kernel/time/timer.c:2387 [inline]
__run_timer_base kernel/time/timer.c:2379 [inline]
run_timer_base+0x114/0x190 kernel/time/timer.c:2396
run_timer_softirq+0x1a/0x50 kernel/time/timer.c:2406
handle_softirqs+0x216/0xa20 kernel/softirq.c:669
__do_softirq kernel/softirq.c:703 [inline]
invoke_softirq kernel/softirq.c:542 [inline]
__irq_exit_rcu+0x178/0x220 kernel/softirq.c:786
irq_exit_rcu+0x9/0x30 kernel/softirq.c:809
instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1062 [inline]
sysvec_apic_timer_interrupt+0xa3/0xc0 arch/x86/kernel/apic/apic.c:1062
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:674
RIP: 0010:lock_is_held_type+0xf5/0x140 kernel/locking/lockdep.c:6021
Code: ff ff ff ff 65 0f c1 15 f1 97 cd 09 83 fa 01 8b 44 24 04 75 2d 9c 5a 80 e6 02 75 47 41 f7 c5 00 02 00 00 74 01 fb 48 83 c4 08 <5b> 5d 41 5c 41 5d 41 5e 41 5f e9 8c 58 03 00 e9 87 58 03 00 31 c0
RSP: 0018:ffffc9000040f6a0 EFLAGS: 00000296
RAX: 0000000000000001 RBX: ffffffff8ffeae60 RCX: 0000000000000001
RDX: 0000000000000046 RSI: ffffffff8e5fe6e1 RDI: ffffffff8c835580
RBP: ffff88801df74b00 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000200 R11: 0000000000000001 R12: 00000000ffffffff
R13: 0000000000000246 R14: ffff88801df75778 R15: 0000000000000005
kernfs_root+0x20e/0x2a0 fs/kernfs/kernfs-internal.h:77
kernfs_active+0xb6/0x110 fs/kernfs/dir.c:42
kernfs_drain+0xd8/0x600 fs/kernfs/dir.c:517
__kernfs_remove+0x34c/0xbc0 fs/kernfs/dir.c:1578
kernfs_remove_by_name_ns+0x7c/0x120 fs/kernfs/dir.c:1796
kernfs_remove_by_name include/linux/kernfs.h:644 [inline]
remove_files+0x96/0x1c0 fs/sysfs/group.c:28
sysfs_remove_group+0x8b/0x1b0 fs/sysfs/group.c:328
sysfs_remove_groups fs/sysfs/group.c:352 [inline]
sysfs_remove_groups+0x60/0xb0 fs/sysfs/group.c:344
destroy_port drivers/infiniband/core/sysfs.c:1238 [inline]
ib_free_port_attrs+0x22e/0x480 drivers/infiniband/core/sysfs.c:1368
remove_one_compat_dev drivers/infiniband/core/device.c:1035 [inline]
rdma_dev_exit_net+0x2fd/0x6b0 drivers/infiniband/core/device.c:1173
ops_exit_list net/core/net_namespace.c:200 [inline]
ops_undo_list+0x2ee/0xab0 net/core/net_namespace.c:253
cleanup_net+0x499/0x920 net/core/net_namespace.c:708
process_one_work+0xac7/0x1b10 kernel/workqueue.c:3495
process_scheduled_works kernel/workqueue.c:3578 [inline]
worker_thread+0x5ef/0xe50 kernel/workqueue.c:3659
kthread+0x373/0x450 kernel/kthread.c:436
ret_from_fork+0x730/0xd60 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
</TASK>
Modules linked in:
---[ end trac
---[ end trace 0000000000000000 ]---
----------------
Code disassembly (best guess):
0: 48 c1 e8 03 shr $0x3,%rax
4: 42 80 3c 28 00 cmpb $0x0,(%rax,%r13,1)
9: 0f 85 8e 01 00 00 jne 0x19d
f: 48 8b 1b mov (%rbx),%rbx
12: 48 85 db test %rbx,%rbx
15: 0f 84 32 01 00 00 je 0x14d
1b: 48 8d 7b 18 lea 0x18(%rbx),%rdi
1f: 41 83 c4 01 add $0x1,%r12d
23: 48 89 f8 mov %rdi,%rax
26: 48 c1 e8 03 shr $0x3,%rax
* 2a: 42 80 3c 28 00 cmpb $0x0,(%rax,%r13,1) <-- trapping instruction
2f: 0f 85 4c 01 00 00 jne 0x181
35: 4c 3b 73 18 cmp 0x18(%rbx),%r14
39: 75 c2 jne 0xfffffffd
3b: 48 rex.W
3c: b8 .byte 0xb8
3d: 00 00 add %al,(%rax)
* Struck through repros no longer work on HEAD.