Root Cause Analysis

Involved Functions, Structs, and Files

• Files: fs/gfs2/quota.c
• Functions: gfs2_quota_init(), do_qc(), qd_grab_sync(), do_sync(), gfs2_quota_sync(), qd_unlock(), qdsb_put(), qd_put(), gfs2_qd_isolate()
• Structs & Fields:
  • struct gfs2_quota_data: qd_lockref.count, qd_flags (QDF_CHANGE, QDF_LOCKED), qd_slot_ref, qd_lru, qd_list
  • struct gfs2_sbd: sd_quota_list

Technical Mechanism and Bug Trigger

1. Quota Change Reference Lifecycle Invariant: In GFS2, an active quota change recorded in the quota change file (sdp->sd_qc_inode) represents an in-use slot and holds two references on the associated struct gfs2_quota_data (qd):

   • A slot reference tracked in qd->qd_slot_ref (acquired via slot_hold(qd) and released via slot_put(qd)).
   • An object reference tracked in qd->qd_lockref.count (acquired via __qd_hold(qd) and released via qd_put(qd)).

   This lifecycle invariant is demonstrated in runtime quota changes in do_qc() (fs/gfs2/quota.c):

   /* When a slot becomes used */
   set_bit(QDF_CHANGE, &qd->qd_flags);
   __qd_hold(qd);
   slot_hold(qd);
   

   and symmetrically when do_qc() writes back quota changes and the change drops back to 0:

   /* When the slot becomes unused */
   clear_bit(QDF_CHANGE, &qd->qd_flags);
   ...
   if (needs_put) {
       slot_put(qd);
       qd_put(qd);
   }
   

2. The Defect in gfs2_quota_init(): During mount, gfs2_quota_init() reads existing quota change records from the quota change file. For any non-zero change (qc_change != 0), it creates a qd structure:

   hash = gfs2_qd_hash(sdp, qc_id);
   qd = qd_alloc(hash, sdp, qc_id);
   ...
   set_bit(QDF_CHANGE, &qd->qd_flags);
   qd->qd_change = qc_change;
   qd->qd_slot = slot;
   qd->qd_slot_ref = 1;
   

   qd_alloc() initializes qd->qd_lockref.count = 0;. Unlike qd_get() (which increments new_qd->qd_lockref.count++ to 1 upon allocation) and unlike do_qc() (which calls __qd_hold(qd) upon setting QDF_CHANGE), gfs2_quota_init() sets QDF_CHANGE and qd->qd_slot_ref = 1, but fails to increment qd->qd_lockref.count. As a result, qd->qd_lockref.count remains 0.

3. Triggering the Refcount Underflow in gfs2_quota_sync(): When gfs2_quota_sync() runs (either via gfs2_quotad or during filesystem shutdown in sync_filesystem() / gfs2_sync_fs()):

   • qd_grab_sync() sees test_bit(QDF_CHANGE, &qd->qd_flags) is set, increments qd->qd_lockref.count++ (so count moves from 0 to 1), sets QDF_LOCKED, calls slot_hold(qd) (qd_slot_ref becomes 2), and adds qd to qda[num_qd++].
   • do_sync() is called and invokes do_qc(qd, -qd->qd_change_sync). The change reaches 0, clearing QDF_CHANGE, and do_qc() calls slot_put(qd) (qd_slot_ref drops to 1) and qd_put(qd).
   • Because qd->qd_lockref.count was only 1, qd_put()'s call to lockref_put_or_lock(&qd->qd_lockref) sees count <= 1, acquires qd->qd_lockref.lock, and returns 0.
   • qd_put() sets qd->qd_lockref.count = 0; and adds qd to gfs2_qd_lru. If the filesystem is withdrawing/withdrawn or SDF_JOURNAL_LIVE is cleared, qd_put() instead calls lockref_mark_dead(&qd->qd_lockref) and gfs2_qd_dispose(qd). Alternatively, if the slab shrinker (gfs2_qd_isolate(), e.g., invoked via drop_caches) scans gfs2_qd_lru while qd is sitting with count == 0, it calls lockref_mark_dead(&qd->qd_lockref). In both cases, qd->qd_lockref.count becomes -128 (0xffffff80).

4. Failure Manifestation: After do_sync() returns, gfs2_quota_sync() proceeds to line 1363:

   for (x = 0; x < num_qd; x++)
       qd_unlock(qda[x]);
   

   qd_unlock() calls qdsb_put(qd), which invokes qd_put(qd) to release the sync reference grabbed earlier in qd_grab_sync().

   Inside qd_put():

   if (lockref_put_or_lock(&qd->qd_lockref))
       return;
   
   BUG_ON(__lockref_is_dead(&qd->qd_lockref));
   

   Because count is already -128, lockref_put_or_lock() acquires the spinlock and returns 0. Execution hits line 333: BUG_ON(__lockref_is_dead(&qd->qd_lockref)); triggering the kernel BUG at fs/gfs2/quota.c:333! invalid opcode crash.