BUG: unable to handle kernel paging request in __run_timers (4)

Status: closed as invalid on 2026/11/16 17:46
Subsystems: pci
Labels: prio:normal
[Documentation on labels]
First crash: 7d21h, last: 7d17h
✨ AI-generated root cause summary:

Writing to /proc/bus/pci/00/00.0 via proc_bus_pci_write() allows a privileged user to write directly to the PCI configuration space of device 00:00.0 (the Host Bridge / Intel Q35 Memory Controller Hub). The write modified memory-decoding configuration registers, specifically lowering TOLUD (Top of Low Usable DRAM at offset 0x9c), which remapped a portion of physical DRAM into the 32-bit PCI MMIO hole. Subsequent CPU reads from kernel memory allocated within that physical range stopped reaching DRAM and instead returned 0xffffffffffffffff (PCI unassigned MMIO abort). When a hardware timer interrupt later invoked debug_object_deactivate(), lookup_object() dereferenced an hlist node pointer in that corrupted memory range, reading 0xffffffffffffffff and causing a general protection fault on the resulting non-canonical KASAN shadow address.

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 9d66f26fefac:

Root Cause Analysis

1. Functions, Structs, and Files Involved

  • proc_bus_pci_write() in drivers/pci/proc.c: The procfs handler that writes raw data directly to the PCI configuration space of a given PCI device.
  • lookup_object() in lib/debugobjects.c: Called by debug_object_deactivate() when deactivating an expiring timer; traverses b->list (struct hlist_head) of struct debug_obj entries to find the matching tracked object.
  • struct debug_obj (include/linux/debugobjects.h): Debug tracking structure containing struct hlist_node node at offset 0 and void *object at offset 0x18.
  • Host Bridge / DRAM Controller (0000:00:00.0): On QEMU (Q35 + ICH9), device 00:00.0 is the Intel Q35 MCH (Memory Controller Hub / DRAM Controller).

2. Trigger Mechanism and Lifecycle Invariant Violation

  1. Unrestricted PCI Configuration Space Write: The reproducer mounts procfs and opens /proc/bus/pci/00/00.0 (O_RDWR). Device 00:00.0 corresponds to the chipset host bridge (Intel Q35 MCH). A write of 0x10f (271) bytes is executed via proc_bus_pci_write().

  2. Chipset Memory Decoding Corruption (TOLUD / PAM / PCIEXBAR):

    • The write data modifies configuration registers of the Q35 MCH, notably register 0x9c (TOLUD – Top of Low Usable DRAM) as well as the PAM registers (0x90–0x96) and PCIEXBAR (0x60–0x67).
    • Register 0x9c (TOLUD, 8-bit mask 0xf8) received 0x2f ('/'), lowering TOLUD to 0x28 (640 MB / 0x28000000).
    • Under the Q35 memory map, addresses between TOLUD (0x28000000) and 4 GB (0x100000000) are remapped to the 32-bit PCI MMIO hole. Any physical DRAM addresses above 640 MB are immediately disconnected from DRAM decoding.
  3. Hardware Bus Abort on Memory Reads:

    • The kernel itself, its page allocations, and active data structures were already resident across physical memory above 640 MB.
    • Any CPU read directed at physical memory within the newly carved-out MMIO hole does not reach DRAM; it instead routes to unassigned MMIO space, which returns all-ones (0xff, 0xffffffffffffffff) due to standard PCI master-abort termination.
  4. Manifestation in lookup_object():

    • A hardware APIC timer interrupt occurred while a worker thread was running in cleanup_net.
    • In the timer softirq, expire_timers() invoked detach_timer() -> debug_deactivate() -> debug_object_deactivate() on a timer (ffff888025c16ec8, physical address 0x25c16ec8, ~604 MB, which remained in readable DRAM below 640 MB).
    • In lookup_object(), the hash bucket's first entry was inspected. The next pointer obj->node.next (mov (%rbx), %rbx), which was allocated in memory above 640 MB, read 0xffffffffffffffff instead of a valid pointer.
    • On the next loop iteration, the code calculated &obj->object at offset 0x18 from 0xffffffffffffffff, yielding effective address 0x0000000000000017.
    • KASAN checked the shadow memory for 0x17 via cmpb $0, (%rax, %r13) (0x17 >> 3 + 0xdffffc0000000000 = 0xdffffc0000000002), triggering a general protection fault on the non-canonical shadow address.
▶ ▼ ✨ AI Jobs (1)
ID Workflow Result Correct Bug Created Started Finished Revision
bc86ed98 assessment-security DenialOfService: ✅ Exploitable: ✅ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ✅ NetworkNeedsSetup: ❌ NetworkTrigger: ❌ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ❌ UserNamespace: ❌ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ BUG: unable to handle kernel paging request in __run_timers (4) 2026/11/12 21:26 2026/11/12 21:26 2026/11/12 22:02 c0087d7f
▶ ▼ Similar bugs (16)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream BUG: unable to handle kernel paging request in __run_timers (2) sound virt net 21 4 293d 389d 0/29 auto-obsoleted due to no activity on 2026/05/01 08:57
upstream BUG: unable to handle kernel paging request in __run_timers net 8 2 2870d 2879d 0/29 auto-closed as invalid on 2019/07/10 08:52
android-54 KASAN: use-after-free Write in __run_timers (2) 22 syz, C 752 543d 632d 0/2 upstream: reported C repro on 2025/02/25 14:17
upstream BUG: unable to handle kernel paging request in __run_timers (3) virt prio:normal 8 syz inconclusive 2 12d 8d07h 0/29 closed as invalid on 2026/11/12 12:41
android-54 KASAN: use-after-free Write in __run_timers 22 syz 449 870d 2322d 0/2 auto-obsoleted due to no activity on 2024/09/11 00:16
upstream KASAN: invalid-access Write in __run_timers kernel -1 8 2100d 2106d 0/29 auto-closed as invalid on 2021/05/20 13:15
android-6-1 KASAN: use-after-free Write in __run_timers origin:lts 22 syz, C 1190 16h39m 719d 0/2 upstream: reported C repro on 2024/12/01 07:19
upstream KMSAN: uninit-value in __run_timers (3) net 7 syz, C error 4 1172d 1208d 0/29 auto-obsoleted due to no activity on 2024/01/10 00:11
upstream KASAN: slab-use-after-free Write in __run_timers net 22 1 629d 629d 0/29 closed as invalid on 2025/03/30 10:14
upstream general protection fault in __run_timers (4) prio:high netfilter 2 1 112d 108d 0/29 auto-obsoleted due to no activity on 2026/10/28 15:26
linux-5.15 KASAN: use-after-free Write in __run_timers origin:lts-only 22 syz, C error 1 310d 343d 0/3 auto-obsoleted due to no activity on 2026/06/03 21:03
android-54 KASAN: slab-out-of-bounds Write in __run_timers 21 4 1971d 2161d 0/2 auto-closed as invalid on 2021/10/26 12:53
upstream BUG: unable to handle kernel NULL pointer dereference in __run_timers arm 10 1 905d 901d 0/29 auto-obsoleted due to no activity on 2024/08/27 10:03
upstream general protection fault in __run_timers kernel 2 35 3348d 3370d 0/29 closed as invalid on 2017/10/22 12:45
upstream general protection fault in __run_timers (2) kernel 2 syz, C 5 3009d 3024d 0/29 closed as invalid on 2018/09/05 12:51
upstream general protection fault in __run_timers (3) kernel 2 1 2262d 2258d 0/29 auto-closed as invalid on 2020/12/09 00:33

Sample crash report:
Oops: general protection fault, probably for non-canonical address 0xdffffc0000000002: 0000 [#1] SMP KASAN NOPTI
KASAN: null-ptr-deref in range [0x0000000000000010-0x0000000000000017]
CPU: 0 UID: 0 PID: 14 Comm: kworker/u32:1 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Workqueue: netns cleanup_net
RIP: 0010:lookup_object lib/debugobjects.c:440 [inline]
RIP: 0010:debug_object_deactivate+0x19b/0x3b0 lib/debugobjects.c:938
Code: 48 c1 e8 03 42 80 3c 28 00 0f 85 8e 01 00 00 48 8b 1b 48 85 db 0f 84 32 01 00 00 48 8d 7b 18 41 83 c4 01 48 89 f8 48 c1 e8 03 <42> 80 3c 28 00 0f 85 4c 01 00 00 4c 3b 73 18 75 c2 48 b8 00 00 00
RSP: 0018:ffffc90000007c60 EFLAGS: 00010003
RAX: 0000000000000002 RBX: ffffffffffffffff RCX: 0000000000000002
RDX: ffffffff9ce46d60 RSI: 0000000000000004 RDI: 0000000000000017
RBP: ffffc90000007d38 R08: 0000000000000001 R09: fffff52000000f7a
R10: 0000000000000003 R11: 0000000000000001 R12: 0000000000000002
R13: dffffc0000000000 R14: ffff888025c16ec8 R15: 1ffff92000000f8e
FS:  0000000000000000(0000) GS:ffff888095279000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00001def9e82a000 CR3: 0000000044d4a000 CR4: 0000000000352ef0
Call Trace:
 <IRQ>
 debug_timer_deactivate kernel/time/timer.c:799 [inline]
 debug_deactivate kernel/time/timer.c:843 [inline]
 detach_timer kernel/time/timer.c:890 [inline]
 expire_timers kernel/time/timer.c:1783 [inline]
 __run_timers+0x5de/0xb00 kernel/time/timer.c:2375
 __run_timer_base kernel/time/timer.c:2387 [inline]
 __run_timer_base kernel/time/timer.c:2379 [inline]
 run_timer_base+0x114/0x190 kernel/time/timer.c:2396
 run_timer_softirq+0x1a/0x50 kernel/time/timer.c:2406
 handle_softirqs+0x216/0xa20 kernel/softirq.c:669
 __do_softirq kernel/softirq.c:703 [inline]
 invoke_softirq kernel/softirq.c:542 [inline]
 __irq_exit_rcu+0x178/0x220 kernel/softirq.c:786
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:809
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1062 [inline]
 sysvec_apic_timer_interrupt+0xa3/0xc0 arch/x86/kernel/apic/apic.c:1062
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:674
RIP: 0010:lock_is_held_type+0xf5/0x140 kernel/locking/lockdep.c:6021
Code: ff ff ff ff 65 0f c1 15 f1 97 cd 09 83 fa 01 8b 44 24 04 75 2d 9c 5a 80 e6 02 75 47 41 f7 c5 00 02 00 00 74 01 fb 48 83 c4 08 <5b> 5d 41 5c 41 5d 41 5e 41 5f e9 8c 58 03 00 e9 87 58 03 00 31 c0
RSP: 0018:ffffc9000040f6a0 EFLAGS: 00000296
RAX: 0000000000000001 RBX: ffffffff8ffeae60 RCX: 0000000000000001
RDX: 0000000000000046 RSI: ffffffff8e5fe6e1 RDI: ffffffff8c835580
RBP: ffff88801df74b00 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000200 R11: 0000000000000001 R12: 00000000ffffffff
R13: 0000000000000246 R14: ffff88801df75778 R15: 0000000000000005
 kernfs_root+0x20e/0x2a0 fs/kernfs/kernfs-internal.h:77
 kernfs_active+0xb6/0x110 fs/kernfs/dir.c:42
 kernfs_drain+0xd8/0x600 fs/kernfs/dir.c:517
 __kernfs_remove+0x34c/0xbc0 fs/kernfs/dir.c:1578
 kernfs_remove_by_name_ns+0x7c/0x120 fs/kernfs/dir.c:1796
 kernfs_remove_by_name include/linux/kernfs.h:644 [inline]
 remove_files+0x96/0x1c0 fs/sysfs/group.c:28
 sysfs_remove_group+0x8b/0x1b0 fs/sysfs/group.c:328
 sysfs_remove_groups fs/sysfs/group.c:352 [inline]
 sysfs_remove_groups+0x60/0xb0 fs/sysfs/group.c:344
 destroy_port drivers/infiniband/core/sysfs.c:1238 [inline]
 ib_free_port_attrs+0x22e/0x480 drivers/infiniband/core/sysfs.c:1368
 remove_one_compat_dev drivers/infiniband/core/device.c:1035 [inline]
 rdma_dev_exit_net+0x2fd/0x6b0 drivers/infiniband/core/device.c:1173
 ops_exit_list net/core/net_namespace.c:200 [inline]
 ops_undo_list+0x2ee/0xab0 net/core/net_namespace.c:253
 cleanup_net+0x499/0x920 net/core/net_namespace.c:708
 process_one_work+0xac7/0x1b10 kernel/workqueue.c:3495
 process_scheduled_works kernel/workqueue.c:3578 [inline]
 worker_thread+0x5ef/0xe50 kernel/workqueue.c:3659
 kthread+0x373/0x450 kernel/kthread.c:436
 ret_from_fork+0x730/0xd60 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
 </TASK>
Modules linked in:
---[ end trac
---[ end trace 0000000000000000 ]---
----------------
Code disassembly (best guess):
   0:	48 c1 e8 03          	shr    $0x3,%rax
   4:	42 80 3c 28 00       	cmpb   $0x0,(%rax,%r13,1)
   9:	0f 85 8e 01 00 00    	jne    0x19d
   f:	48 8b 1b             	mov    (%rbx),%rbx
  12:	48 85 db             	test   %rbx,%rbx
  15:	0f 84 32 01 00 00    	je     0x14d
  1b:	48 8d 7b 18          	lea    0x18(%rbx),%rdi
  1f:	41 83 c4 01          	add    $0x1,%r12d
  23:	48 89 f8             	mov    %rdi,%rax
  26:	48 c1 e8 03          	shr    $0x3,%rax
* 2a:	42 80 3c 28 00       	cmpb   $0x0,(%rax,%r13,1) <-- trapping instruction
  2f:	0f 85 4c 01 00 00    	jne    0x181
  35:	4c 3b 73 18          	cmp    0x18(%rbx),%r14
  39:	75 c2                	jne    0xfffffffd
  3b:	48                   	rex.W
  3c:	b8                   	.byte 0xb8
  3d:	00 00                	add    %al,(%rax)

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/12 19:13 upstream 9d66f26fefac 5c462c66 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 general protection fault in __run_timers
2026/11/12 15:55 upstream 9d66f26fefac 5c462c66 .config console log report syz / log [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 BUG: unable to handle kernel paging request in __run_timers
* Struck through repros no longer work on HEAD.