kernel panic: corrupted stack end in worker_thread (2)

Status: moderation: reported on 2026/11/06 06:49
Subsystems: arm
Labels: prio:high
[Documentation on labels]
Reported-by: syzbot+562e88cd056d490257fc@syzkaller.appspotmail.com
First crash: 4d04h, last: 4d04h
✨ AI-generated root cause summary:

A kernel panic occurs in schedule_debug() due to task_stack_end_corrupted() detecting that the STACK_END_MAGIC canary at the bottom of the kworker task stack has been overwritten. The worker was processing tasks on WireGuard's packet_crypt_wq ('wg-crypt-wg1'), where operations like wg_packet_tx_worker(), wg_packet_encrypt_worker(), and wg_packet_decrypt_worker() combine local stack allocations with deep call chains through ChaCha20-Poly1305 cryptography, UDP tunneling (udp_tunnel_xmit_skb), IP routing, and network device transmission. This cumulative stack consumption exhausted the kernel stack space, overwriting the stack end canary before the worker yielded execution to schedule().

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 7a19f0cc94ad:

Root Cause Analysis

1. Failure Manifestation and Detection Mechanism

The reported kernel panic:

Kernel panic - not syncing: corrupted stack end detected inside scheduler
Workqueue:  0x0 (wg-crypt-wg1)
Call trace:
 ...
 vpanic+0x594/0xaf8 kernel/panic.c:651
 schedule_debug kernel/sched/core.c:6103 [inline]
 __schedule+0x18a0/0x2b00 kernel/sched/core.c:7133
 __schedule_loop kernel/sched/core.c:7361 [inline]
 schedule+0x13c/0x20c kernel/sched/core.c:7376
 worker_thread+0x924/0xbd0 kernel/workqueue.c:3575
 kthread+0x304/0x3d4 kernel/kthread.c:436

The check is performed in schedule_debug() (kernel/sched/core.c):

#ifdef CONFIG_SCHED_STACK_END_CHECK
	if (task_stack_end_corrupted(prev))
		panic("corrupted stack end detected inside scheduler\n");
#endif

task_stack_end_corrupted(task) evaluates *(end_of_stack(task)) != STACK_END_MAGIC. On architectures with downward-growing stacks (such as arm64), end_of_stack(task) is situated at the lowest address of the allocated kernel stack area.

When the worker thread in worker_thread() (kernel/workqueue.c:3575) completes processing work items via process_scheduled_works(), it marks itself as idle (__set_current_state(TASK_IDLE)) and invokes schedule(). Notice that worker->current_func and worker->current_pwq are cleared upon work completion in process_scheduled_works() (kernel/workqueue.c:3447-3449), but worker->desc still holds the workqueue name "wg-crypt-wg1", which is why print_worker_info() outputs Workqueue: 0x0 (wg-crypt-wg1).

2. WireGuard Workqueue Execution and Kernel Stack Exhaustion

WireGuard allocates its packet cryptography and transmission workqueue in drivers/net/wireguard/device.c:

wg->packet_crypt_wq = alloc_workqueue("wg-crypt-%s",
		WQ_CPU_INTENSIVE | WQ_MEM_RECLAIM | WQ_PERCPU, 0,
		dev->name);

Three work handlers execute on this workqueue:

  1. wg_packet_encrypt_worker (drivers/net/wireguard/send.c)
  2. wg_packet_decrypt_worker (drivers/net/wireguard/receive.c)
  3. wg_packet_tx_worker (drivers/net/wireguard/send.c)

Investigation of these paths revealed:

  • In encrypt_packet() and decrypt_packet(), local stack variables include struct scatterlist sg[MAX_SKB_FRAGS + 8], which consumes considerable stack space before invoking chacha20poly1305_encrypt_sg_inplace() / chacha20poly1305_decrypt_sg_inplace().
  • In wg_packet_tx_worker(), once packets are marked PACKET_STATE_CRYPTED, wg_packet_create_data_done() calls:
    wg_socket_send_skb_to_peer()
      -> send4() / send6()
        -> udp_tunnel_xmit_skb() / udp_tunnel6_xmit_skb()
          -> iptunnel_xmit()
            -> ip_local_out()
              -> ip_finish_output() -> __dev_queue_xmit() -> ...
    
  • Packet transmission through UDP tunnels into nested devices (which can recurse up to XMIT_RECURSION_LIMIT = 8 under dev_xmit_recursion()) or down through deep virtual device/netfilter/BPF/routing stacks drastically deepens the call chain.

Because this entire stack of crypto operations, UDP tunneling encapsulation, routing lookup, egress queuing, and hardware/virtual driver transmission executes in the synchronous context of the wg-crypt-%s kworker, cumulative stack frame consumption descended into the bottom boundary of the task stack, overwriting STACK_END_MAGIC. Because the stack pointer did not cross into the guard page beneath the stack (or CONFIG_VMAP_STACK was either bordered or just touched the canary word), execution successfully returned to worker_thread(), where schedule() caught the corrupted canary during the next context switch.

▶ ▼ ✨ AI Jobs (2)
ID Workflow Result Correct Bug Created Started Finished Revision
55a32770 repro-c Reproduced: ✅ ❓ kernel panic: corrupted stack end in worker_thread (2) 2026/11/05 11:59 2026/11/05 11:59 2026/11/05 19:36 5cecaf1f
ea5de86e assessment-security DenialOfService: ✅ Exploitable: ✅ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ✅ NetworkNeedsSetup: ✅ NetworkTrigger: ✅ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ❌ UserNamespace: ✅ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ kernel panic: corrupted stack end in worker_thread (2) 2026/11/02 06:37 2026/11/02 06:38 2026/11/02 06:56 25a0ae84
▶ ▼ Similar bugs (2)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream kernel panic: corrupted stack end in worker_thread kernel 2 1 2013d 1998d 0/29 auto-closed as invalid on 2021/07/02 05:00
upstream BUG: stack guard page was hit in worker_thread net -1 2 594d 592d 0/29 auto-obsoleted due to no activity on 2025/06/30 01:00
▶ ▼ Last patch testing requests (1)
Created Duration User Patch Repo Result
2026/11/06 08:03 1h28m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git b0958b1397fb5ffdb995c0ccc74349841929c08d report job log log

Sample crash report:
Kernel panic - not syncing: corrupted stack end detected inside scheduler
CPU: 1 UID: 0 PID: 4931 Comm: kworker/1:6 Not tainted syzkaller #0 PREEMPT 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/14/2026
Workqueue:  0x0 (wg-crypt-wg1)
Call trace:
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:499 (C)
 __dump_stack+0x30/0x40 lib/dump_stack.c:94
 dump_stack_lvl+0x30/0x12c lib/dump_stack.c:120
 dump_stack+0x1c/0x28 lib/dump_stack.c:129
 vpanic+0x594/0xaf8 kernel/panic.c:651
 vpanic+0x0/0xaf8 kernel/panic.c:788
 schedule_debug kernel/sched/core.c:6103 [inline]
 __schedule+0x18a0/0x2b00 kernel/sched/core.c:7133
 __schedule_loop kernel/sched/core.c:7361 [inline]
 schedule+0x13c/0x20c kernel/sched/core.c:7376
 worker_thread+0x924/0xbd0 kernel/workqueue.c:3575
 kthread+0x304/0x3d4 kernel/kthread.c:436
 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:854
SMP: stopping secondary CPUs
Kernel Offset: disabled
CPU features: 0x0,a0000000,00f00020,9c080a02,a84ff6a7
Memory Limit: none
Rebooting in 86400 seconds..

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/02 06:37 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 7a19f0cc94ad 25a0ae84 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 kernel panic: corrupted stack end in worker_thread
* Struck through repros no longer work on HEAD.