inconsistent lock state in rxrpc_put_client_conn

Status: fixed on 2020/03/30 09:03
Reported-by: syzbot+19114525476fe0be9e7b@syzkaller.appspotmail.com
Fix commit: 963485d436cc rxrpc: Fix call RCU cleanup using non-bh-safe locks
First crash: 2384d, last: 2367d
Fix bisection: fixed by (bisect log) :
commit 43cac315bec132e962e04c31fe888caac257ec0a
Author: David Howells <dhowells@redhat.com>
Date: Thu Feb 6 13:57:40 2020 +0000

  rxrpc: Fix call RCU cleanup using non-bh-safe locks

  
▶ ▼ Similar bugs (2)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream inconsistent lock state in rxrpc_put_client_conn 4 syz, C done 18198 2385d 2391d 15/29 fixed on 2020/02/18 14:31
linux-4.14 inconsistent lock state in rxrpc_put_client_conn 4 syz, C 15057 1265d 2380d 0/1 upstream: reported C repro on 2020/02/14 22:06

Sample crash report:
audit: type=1400 audit(1582739155.360:36): avc:  denied  { map } for  pid=7919 comm="syz-executor619" path="/root/syz-executor619664645" dev="sda1" ino=16484 scontext=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1
IPVS: ftp: loaded support on port[0] = 21
================================
WARNING: inconsistent lock state
4.19.106-syzkaller #0 Not tainted
--------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
swapper/1/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
00000000387f7d19 (&(&local->client_conns_lock)->rlock){+.?.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000387f7d19 (&(&local->client_conns_lock)->rlock){+.?.}, at: rxrpc_put_one_client_conn net/rxrpc/conn_client.c:949 [inline]
00000000387f7d19 (&(&local->client_conns_lock)->rlock){+.?.}, at: rxrpc_put_client_conn+0x666/0xc00 net/rxrpc/conn_client.c:1002
{SOFTIRQ-ON-W} state was registered at:
  __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
  _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
  spin_lock include/linux/spinlock.h:329 [inline]
  rxrpc_get_client_conn net/rxrpc/conn_client.c:309 [inline]
  rxrpc_connect_call+0x301/0x4630 net/rxrpc/conn_client.c:702
  rxrpc_new_client_call+0x8c6/0x1850 net/rxrpc/call_object.c:291
  rxrpc_new_client_call_for_sendmsg net/rxrpc/sendmsg.c:596 [inline]
  rxrpc_do_sendmsg+0xf2e/0x1bc1 net/rxrpc/sendmsg.c:652
  rxrpc_sendmsg+0x4a8/0x5b0 net/rxrpc/af_rxrpc.c:593
  sock_sendmsg_nosec net/socket.c:622 [inline]
  sock_sendmsg+0xcf/0x120 net/socket.c:632
  ___sys_sendmsg+0x3e2/0x920 net/socket.c:2115
  __sys_sendmmsg+0x195/0x470 net/socket.c:2210
  __do_sys_sendmmsg net/socket.c:2239 [inline]
  __se_sys_sendmmsg net/socket.c:2236 [inline]
  __x64_sys_sendmmsg+0x99/0x100 net/socket.c:2236
  do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
  entry_SYSCALL_64_after_hwframe+0x49/0xbe
irq event stamp: 144150
hardirqs last  enabled at (144150): [<ffffffff871f84d7>] __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:160 [inline]
hardirqs last  enabled at (144150): [<ffffffff871f84d7>] _raw_spin_unlock_irqrestore+0x67/0xe0 kernel/locking/spinlock.c:184
hardirqs last disabled at (144149): [<ffffffff871f8746>] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:108 [inline]
hardirqs last disabled at (144149): [<ffffffff871f8746>] _raw_spin_lock_irqsave+0x66/0xbf kernel/locking/spinlock.c:152
softirqs last  enabled at (144020): [<ffffffff813e3edd>] irq_enter+0xbd/0xd0 kernel/softirq.c:354
softirqs last disabled at (144021): [<ffffffff813e406b>] invoke_softirq kernel/softirq.c:372 [inline]
softirqs last disabled at (144021): [<ffffffff813e406b>] irq_exit+0x17b/0x1c0 kernel/softirq.c:412

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(&(&local->client_conns_lock)->rlock);
  <Interrupt>
    lock(&(&local->client_conns_lock)->rlock);

 *** DEADLOCK ***

1 lock held by swapper/1/0:
 #0: 000000009ffcac88 (rcu_callback){....}, at: __rcu_reclaim kernel/rcu/rcu.h:226 [inline]
 #0: 000000009ffcac88 (rcu_callback){....}, at: rcu_do_batch kernel/rcu/tree.c:2584 [inline]
 #0: 000000009ffcac88 (rcu_callback){....}, at: invoke_rcu_callbacks kernel/rcu/tree.c:2897 [inline]
 #0: 000000009ffcac88 (rcu_callback){....}, at: __rcu_process_callbacks kernel/rcu/tree.c:2864 [inline]
 #0: 000000009ffcac88 (rcu_callback){....}, at: rcu_process_callbacks+0xbff/0x17f0 kernel/rcu/tree.c:2881

stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.106-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x188/0x20d lib/dump_stack.c:118
 print_usage_bug.cold+0x327/0x425 kernel/locking/lockdep.c:2540
 valid_state kernel/locking/lockdep.c:2553 [inline]
 mark_lock_irq kernel/locking/lockdep.c:2747 [inline]
 mark_lock+0xc71/0x11b0 kernel/locking/lockdep.c:3127
 mark_irqflags kernel/locking/lockdep.c:3005 [inline]
 __lock_acquire+0xc62/0x49c0 kernel/locking/lockdep.c:3368
 lock_acquire+0x170/0x400 kernel/locking/lockdep.c:3903
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
 spin_lock include/linux/spinlock.h:329 [inline]
 rxrpc_put_one_client_conn net/rxrpc/conn_client.c:949 [inline]
 rxrpc_put_client_conn+0x666/0xc00 net/rxrpc/conn_client.c:1002
 rxrpc_put_connection net/rxrpc/ar-internal.h:951 [inline]
 rxrpc_rcu_destroy_call+0xb6/0x1e0 net/rxrpc/call_object.c:657
 __rcu_reclaim kernel/rcu/rcu.h:236 [inline]
 rcu_do_batch kernel/rcu/tree.c:2584 [inline]
 invoke_rcu_callbacks kernel/rcu/tree.c:2897 [inline]
 __rcu_process_callbacks kernel/rcu/tree.c:2864 [inline]
 rcu_process_callbacks+0xb2d/0x17f0 kernel/rcu/tree.c:2881
 __do_softirq+0x26c/0x93c kernel/softirq.c:292
 invoke_softirq kernel/softirq.c:372 [inline]
 irq_exit+0x17b/0x1c0 kernel/softirq.c:412
 exiting_irq arch/x86/include/asm/apic.h:544 [inline]
 smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1094
 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:893
 </IRQ>
RIP: 0010:native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:61
Code: fa eb 82 90 90 90 90 90 90 90 90 90 e9 07 00 00 00 0f 00 2d 04 76 48 00 f4 c3 66 90 e9 07 00 00 00 0f 00 2d f4 75 48 00 fb f4 <c3> 90 41 56 41 55 41 54 55 53 e8 33 26 37 fa e8 3e ec 35 fc 0f 1f
RSP: 0018:ffff8880a9eefd58 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff1164ac4 RBX: dffffc0000000000 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffff8880a9edac3c
RBP: 0000000000000001 R08: ffff8880a9eda3c0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff88b25610
R13: 0000000000000000 R14: 0000000000000000 R15: ffff8880a9eda3c0
 arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline]
 default_idle+0x49/0x320 arch/x86/kernel/process.c:565
 cpuidle_idle_call kernel/sched/idle.c:153 [inline]
 do_idle+0x2ee/0x4b0 kernel/sched/idle.c:263
 ? a

Crashes (6584):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/02/26 17:48 linux-4.19.y f25804f38984 251aabb7 .config console log report syz C ci2-linux-4-19
2020/02/24 13:54 linux-4.19.y f25804f38984 1253d6f0 .config console log report syz C ci2-linux-4-19
2020/02/24 13:32 linux-4.19.y f25804f38984 1253d6f0 .config console log report syz C ci2-linux-4-19
2020/02/24 13:11 linux-4.19.y f25804f38984 1253d6f0 .config console log report syz C ci2-linux-4-19
2020/02/24 12:46 linux-4.19.y f25804f38984 1253d6f0 .config console log report syz C ci2-linux-4-19
2020/02/24 12:26 linux-4.19.y f25804f38984 1253d6f0 .config console log report syz C ci2-linux-4-19
2020/02/24 00:16 linux-4.19.y 4fccc2503536 d801cb02 .config console log report syz C ci2-linux-4-19
2020/02/23 23:58 linux-4.19.y 4fccc2503536 d801cb02 .config console log report syz C ci2-linux-4-19
2020/02/23 01:13 linux-4.19.y 4fccc2503536 2c36e7a7 .config console log report syz C ci2-linux-4-19
2020/02/22 21:52 linux-4.19.y 4fccc2503536 2c36e7a7 .config console log report syz C ci2-linux-4-19
2020/02/22 15:16 linux-4.19.y 4fccc2503536 2c36e7a7 .config console log report syz C ci2-linux-4-19
2020/02/22 14:59 linux-4.19.y 4fccc2503536 2c36e7a7 .config console log report syz C ci2-linux-4-19
2020/02/22 06:42 linux-4.19.y 4fccc2503536 2ffa6679 .config console log report syz C ci2-linux-4-19
2020/02/21 21:40 linux-4.19.y 4fccc2503536 2ffa6679 .config console log report syz C ci2-linux-4-19
2020/02/21 07:27 linux-4.19.y 4fccc2503536 bd2a74a3 .config console log report syz C ci2-linux-4-19
2020/02/21 07:07 linux-4.19.y 4fccc2503536 bd2a74a3 .config console log report syz C ci2-linux-4-19
2020/02/21 06:19 linux-4.19.y 4fccc2503536 bd2a74a3 .config console log report syz C ci2-linux-4-19
2020/02/21 01:45 linux-4.19.y 4fccc2503536 81230308 .config console log report syz C ci2-linux-4-19
2020/02/21 00:50 linux-4.19.y 4fccc2503536 81230308 .config console log report syz C ci2-linux-4-19
2020/02/20 16:06 linux-4.19.y 4fccc2503536 81230308 .config console log report syz C ci2-linux-4-19
2020/02/20 15:26 linux-4.19.y 4fccc2503536 81230308 .config console log report syz C ci2-linux-4-19
2020/02/20 15:04 linux-4.19.y 4fccc2503536 81230308 .config console log report syz C ci2-linux-4-19
2020/02/20 07:17 linux-4.19.y 4fccc2503536 b690a6e3 .config console log report syz C ci2-linux-4-19
2020/02/20 06:45 linux-4.19.y 4fccc2503536 b690a6e3 .config console log report syz C ci2-linux-4-19
2020/02/20 03:22 linux-4.19.y 4fccc2503536 b690a6e3 .config console log report syz C ci2-linux-4-19
2020/02/20 03:02 linux-4.19.y 4fccc2503536 b690a6e3 .config console log report syz C ci2-linux-4-19
2020/02/19 22:31 linux-4.19.y 4fccc2503536 47fae6e9 .config console log report syz C ci2-linux-4-19
2020/02/19 21:54 linux-4.19.y 4fccc2503536 47fae6e9 .config console log report syz C ci2-linux-4-19
2020/02/19 17:20 linux-4.19.y 9b15f7fae677 47fae6e9 .config console log report syz C ci2-linux-4-19
2020/02/19 16:37 linux-4.19.y 9b15f7fae677 47fae6e9 .config console log report syz C ci2-linux-4-19
2020/02/19 15:59 linux-4.19.y 9b15f7fae677 47fae6e9 .config console log report syz C ci2-linux-4-19
2020/02/19 15:35 linux-4.19.y 9b15f7fae677 47fae6e9 .config console log report syz C ci2-linux-4-19
2020/02/19 06:44 linux-4.19.y 9b15f7fae677 135c18aa .config console log report syz C ci2-linux-4-19
2020/02/19 05:57 linux-4.19.y 9b15f7fae677 135c18aa .config console log report syz C ci2-linux-4-19
2020/02/19 05:16 linux-4.19.y 9b15f7fae677 135c18aa .config console log report syz C ci2-linux-4-19
2020/02/19 04:34 linux-4.19.y 9b15f7fae677 135c18aa .config console log report syz C ci2-linux-4-19
2020/02/19 03:51 linux-4.19.y 9b15f7fae677 135c18aa .config console log report syz C ci2-linux-4-19
2020/02/19 03:08 linux-4.19.y 9b15f7fae677 135c18aa .config console log report syz C ci2-linux-4-19
2020/02/17 22:57 linux-4.19.y 9b15f7fae677 72bfa6f2 .config console log report syz C ci2-linux-4-19
2020/02/17 14:18 linux-4.19.y 9b15f7fae677 72bfa6f2 .config console log report syz C ci2-linux-4-19
2020/02/11 22:14 linux-4.19.y 357668399cf7 4d1ab643 .config console log report syz C ci2-linux-4-19
2020/02/28 15:29 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 14:45 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 13:38 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 12:33 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 12:06 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 10:52 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 09:47 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 08:58 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 07:44 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 06:41 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 05:02 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 04:53 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 03:52 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 03:49 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 02:42 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/28 01:28 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/27 23:41 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/27 22:34 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/27 22:05 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/27 21:40 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/27 21:04 linux-4.19.y f25804f38984 c88c7b75 .config console log report ci2-linux-4-19
2020/02/27 20:32 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 19:25 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 18:24 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 17:22 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 16:14 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 15:13 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 14:08 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 12:41 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 12:34 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 11:28 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 10:24 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 10:00 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 09:25 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 08:52 linux-4.19.y f25804f38984 40bcfdd5 .config console log report ci2-linux-4-19
2020/02/27 07:59 linux-4.19.y f25804f38984 251aabb7 .config console log report ci2-linux-4-19
2020/02/27 07:40 linux-4.19.y f25804f38984 251aabb7 .config console log report ci2-linux-4-19
2020/02/27 06:39 linux-4.19.y f25804f38984 251aabb7 .config console log report ci2-linux-4-19
2020/02/27 05:40 linux-4.19.y f25804f38984 251aabb7 .config console log report ci2-linux-4-19
2020/02/27 04:40 linux-4.19.y f25804f38984 251aabb7 .config console log report ci2-linux-4-19
2020/02/27 03:39 linux-4.19.y f25804f38984 251aabb7 .config console log report ci2-linux-4-19
2020/02/11 15:22 linux-4.19.y 357668399cf7 084454ae .config console log report ci2-linux-4-19
* Struck through repros no longer work on HEAD.