Technical Root Cause Analysis

1. Bug Manifestation & Call Context

The bug report shows a lockdep warning firing at fs/eventpoll.c:2272:

__lockdep_enabled && !this_cpu_read(hardirqs_enabled)
WARNING: fs/eventpoll.c:2272 at ep_poll fs/eventpoll.c:2272 [inline], CPU#1: udevd/2859
WARNING: fs/eventpoll.c:2272 at do_epoll_wait+0x746/0x1790 fs/eventpoll.c:2837, CPU#1: udevd/2859

At the crash site:

• The CPU registers show EFLAGS: 00010293, where bit 9 (0x200, the Interrupt Flag IF) is set (1), indicating hardware interrupts are actively enabled on CPU#1.
• Register RBX holds 0, reading this_cpu_read(hardirqs_enabled).
• The system is Not tainted and __lockdep_enabled (debug_locks && !this_cpu_read(lockdep_recursion)) is true, confirming lockdep is active, debugging locks are enabled, and no prior assertions or warnings have triggered since boot.
• The task udevd invoked epoll_wait(2), traversing: entry_SYSCALL_64_after_hwframe → do_syscall_64() → __x64_sys_epoll_wait() → do_epoll_wait() → ep_poll().
• At the top of ep_poll(), lockdep_assert_irqs_enabled() checks WARN_ON_ONCE(__lockdep_enabled && !this_cpu_read(hardirqs_enabled)) and splats because lockdep's software tracking state disagrees with the hardware CPU flag state.

2. Root Cause Mechanism

1. Syscall Entry State Tracking: In arch/x86/entry/syscall_64.c, do_syscall_64() invokes syscall_enter_from_user_mode_randomize_stack() (include/linux/entry-common.h:199):

   #define syscall_enter_from_user_mode_randomize_stack(regs, syscall)	\
   ({									\
       enter_from_user_mode_randomize_stack(regs);			\
       instrumentation_begin();					\
       local_irq_enable();						\
       long _ret = syscall_enter_from_user_mode_work(regs, syscall);	\
       instrumentation_end();						\
       _ret;								\
   })
   

   First, enter_from_user_mode_randomize_stack() → enter_from_user_mode() (include/linux/irq-entry-common.h:98) calls lockdep_hardirqs_off(CALLER_ADDR0), setting hardirqs_enabled = 0 while hardware interrupts are disabled upon entering kernel space from usermode.

2. Interrupt Re-enabling and Lockless Execution in epoll_wait: At line 204 of include/linux/entry-common.h, local_irq_enable() runs:

   trace_hardirqs_on();
   raw_local_irq_enable();
   

   Under normal circumstances, trace_hardirqs_on() calls lockdep_hardirqs_on(), which writes __this_cpu_write(hardirqs_enabled, 1). Following this, do_syscall_64() executes the system call. Unlike many syscalls that acquire spinlocks or mutexes (which repeatedly execute local_irq_save() / local_irq_restore() or lockdep tracking hooks that resynchronize hardirqs_enabled), epoll_wait with valid arguments runs completely locklessly until ep_poll():

   • CLASS(fd, f)(epfd) acquires an RCU-based or reference-counted file pointer without IRQ-disabling spinlocks.
   • ep_check_params() performs validation checks.
   • ep_poll() immediately runs lockdep_assert_irqs_enabled().

3. Desynchronization Mechanism: Because lockdep_hardirqs_off() in kernel/locking/lockdep.c:4546 asserts DEBUG_LOCKS_WARN_ON(!irqs_disabled()), hardirqs_enabled can only be set to 0 while hardware interrupts are off (IF == 0). Therefore, either:

   • An interrupt or exception occurred immediately around or after local_irq_enable() in the entry path (include/linux/entry-common.h:204-205), entering via irqentry_enter_from_kernel_mode() (which called lockdep_hardirqs_off()), but returned to the kernel without properly invoking trace_hardirqs_on() / lockdep_hardirqs_on() on the exit path (irqentry_exit_to_kernel_mode_after_preempt()); OR
   • A race condition exists in the non-atomic sequence in local_irq_enable() where trace_hardirqs_on() marks hardirqs_enabled = 1 before raw_local_irq_enable() sets EFLAGS.IF = 1, and an exception/trap firing in that window hits an entry/exit path that inspects regs_irqs_disabled(regs) (finding IF == 0) and skips enabling tracing on exit, returning to raw_local_irq_enable() which sets IF = 1 while leaving hardirqs_enabled = 0.

3. Scope and Incomplete/Unverified Aspects

• Verified:
  • Hardware interrupts are definitely enabled (EFLAGS.IF == 1) at the point of failure.
  • No lockdep assertion fired prior to ep_poll(), and debug_locks was non-zero.
  • The failure only manifests visibly on syscalls like epoll_wait() that do not take spinlocks or change interrupt states before checking lockdep_assert_irqs_enabled().
• Inconclusive / Unverified:
  • The exact asynchronous event (e.g. reschedule IPI, timer tick, preemption check via irqentry_exit_cond_resched, or an in-kernel exception/probe) that executed between local_irq_enable() and ep_poll() leaving hardirqs_enabled cleared could not be definitively pinpointed due to disabled tools, but the bug definitely resides in the entry/irqentry common framework (include/linux/entry-common.h, include/linux/irq-entry-common.h, and kernel/entry/common.c).