Marc Vef mvef · review.whamcloud.com

Needs your action

4 new · 101 longstanding

64320 master J –B ✓T ✓R 0/2

LU-19921 target: verify GSSIAM tokens on the target

reviewing wangdi · PS16→PS97 since your -1 — your objection was dropped

4 unresolved XL +1829 −35

build #135119 SUCCESS · tests all Maloo sessions

no vote yet: Jinshan Xiong, Sebastien Buisson, Bruno Faccini, Andreas Dilger, Lai Siyao

patchsets:

+92 earlier

PS1 03-03 PS2 03-03 PS3 03-10 PS4 03-12 PS5 03-12 PS6 03-12 PS7 03-28 PS8 03-29 PS9 03-29 PS10 03-29 PS11 03-30 PS12 04-03 PS13 04-03 PS14 04-04 PS15 04-10 PS16 04-12 PS17 04-17 PS18 04-20 PS19 04-23 PS20 04-24 PS21 04-25 PS22 05-13 PS23 05-14 PS24 05-17 PS25 05-17 PS26 05-17 PS27 05-19 PS28 05-20 PS29 05-21 PS30 05-21 PS31 05-25 PS32 05-29 PS33 05-30 PS34 06-02 PS35 06-03 PS36 06-03 PS37 06-04 PS38 06-04 PS39 06-06 PS40 06-08 PS41 06-10 PS42 06-10 PS43 06-11 PS44 06-12 PS45 06-13 PS46 06-15 PS47 06-15 PS48 06-18 PS49 06-19 PS50 06-22 PS51 07-10 PS52 07-12 PS53 07-20 PS54 07-24 PS55 07-27 PS56 07-30 PS57 08-02 PS58 08-02 PS59 08-02 PS60 08-02 PS61 08-02 PS62 08-04 PS63 08-05 PS64 08-05 PS65 08-06 PS66 08-06 PS67 08-07 PS68 08-07 PS69 08-08 PS70 08-09 PS71 08-09 PS72 08-11 PS73 08-13 PS74 08-13 PS75 08-13 PS76 08-13 PS77 08-14 PS78 08-15 PS79 08-16 PS80 08-17 PS81 08-18 PS82 08-18 PS83 08-18 PS84 08-19 PS85 08-20 PS86 08-21 PS87 08-22 PS88 08-22 PS89 08-22 PS90 08-23 PS91 08-24 PS92 08-25

PS93 08-26 PS94 08-27 PS95 09-07 PS96 10-23 PS97 11-01

your previous vote: -1 on PS16 (2026-04-15 09:27) — now at PS97

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS97 2026-11-02 10:53 Gerrit AI review for Lustre l_gssiam_auth.8:30 — (minor) The helper does not return a Project ID. The downcall it writes is `<key> <mode>:<expire>:<principal>`, which tgt_gssiam_parse_downc

Gerrit AI review for Lustre PS97 · 2026-11-02 10:53

(minor) The helper does not return a Project ID. The downcall it writes is `<key> <mode>:<expire>:<principal>`, which tgt_gssiam_parse_downcall() parses as deny|ro|rw|expired, a relative expiry in seconds and the principal. The projid is resolved by MDT0 from the namespace (mdt_gssiam_projid_cb()). The DESCRIPTION paragraph says the same thing ("associated Project ID").

The downcall format itself is not documented anywhere on the page. That matters because a production site is expected to supply its own helper, and tgt_gssiam_parse_downcall() rejects ro/rw results that do not include a principal. Could the page describe the result format, including the required principal, and drop the Project ID wording?

reply PS97 2026-11-02 10:53 Gerrit AI review for Lustre sec_gssiam_srv.c:484 — (minor) On success gssiam_verify_target() leaves major = GSS_S_COMPLETE. Every later failure in this function then goes through `out:` with

Gerrit AI review for Lustre PS97 · 2026-11-02 10:53

(minor) On success gssiam_verify_target() leaves major = GSS_S_COMPLETE. Every later failure in this function then goes through `out:` with that value: rsc_entry_get(), the gctx allocation, gssiam_get_nodemap() (for example a nodemap_fileset_add() error other than -EPERM or -EEXIST), the gsc_nm_name allocation and lustre_pack_reply_v2(). The result is

    gss_pack_err_notify(req, GSS_S_COMPLETE, 0)

so the client gets a PROC_ERR reply with a success major and logs "GSSIAM init failed: 0/0". Before this patch, verify always failed, so these paths were never reached. Should major be reset to GSS_S_FAILURE after the verify call, or set at each later error?

reply PS97 2026-11-02 10:53 Gerrit AI review for Lustre tgt_gssiam.c:58 — (minor) If this OBD_STRDUP() fails, the entry continues with gd_subdir == NULL. alloc_entry() only drops an entry when init_entry marked it

Gerrit AI review for Lustre PS97 · 2026-11-02 10:53

(minor) If this OBD_STRDUP() fails, the entry continues with gd_subdir == NULL. alloc_entry() only drops an entry when init_entry marked it UC_CACHE_SET_INVALID(), which the client-side gssiam_entry_init() does on its own strdup failure.

Here a request for "/foo" is then handled as if the subdir were empty:

    tgt_gssiam_do_upcall()->mdt_gssiam_projid_cb(path = NULL)
      ->nodemap_fileset_get_root(NULL) -> "" (or the primary fileset)
      ->mdt_lookup_fileset("") -> ROOT projid
    l_gssiam_auth is asked about subdir ""
    tgt_gssiam_info_alloc() -> lid_subdir "" -> nodemap_fileset_clear()

An allocation failure therefore quietly changes the authorization scope and the projid rather than failing the request. Could this, and the gd_token allocation above, set UC_CACHE_SET_INVALID(entry) instead?

reply PS97 2026-11-02 10:53 Gerrit AI review for Lustre tgt_gssiam.c:771 — (minor) The refcount test runs without uc_lock, so a concurrent lookup can take a reference between this check and the state change:

Gerrit AI review for Lustre PS97 · 2026-11-02 10:53

(minor) The refcount test runs without uc_lock, so a concurrent lookup can take a reference between this check and the state change:

    detach: gd_export_count -> 0, ue_refcount == 1
                         ctx init, same token:
                         upcall_cache_get_entry() -> ref 2, entry valid
    detach: upcall_cache_update_entry(UC_CACHE_INVALID)
                         tgt_gssiam_verify(): !UC_CACHE_IS_VALID -> -EACCES

The new mount then gets GSS_S_DEFECTIVE_CREDENTIAL, and the client fails with -EPERM rather than retrying. This can happen when the last mount for a token goes away while another mount with the same token connects. The UC_CACHE_EXPIRED path in tgt_gssiam_attach_cache_entry() has the same window. Would it be simpler not to invalidate here, and let the entry age out or be unlinked by check_unlink_entry() once nobody holds it?

owner wangdi · uploader wangdi · PS97 uploaded 20h ago · NEW · open in Gerrit ↗

commit message

LU-19921 target: verify GSSIAM tokens on the target

When the target receives a SEC_CTX_INIT RPC, it calls
tgt_gssiam_verify() to authenticate the token with the external
GSSIAM server via the l_gssiam_auth upcall. Once authenticated,
the token and its resolved identity/permissions are cached and
attached to the export.

This patch includes:
- Target GSSIAM verification framework (tgt_gssiam.c) and upcall
  cache interactions to validate tokens and manage the gssiam_server
  cache.
- l_gssiam_auth utility and man page to perform external
  authentication and downcall results to the kernel.
- Remote authentication forwarding via MDS_GET_INFO RPCs
  (tgt_gssiam_rpc_authenticate, mdt_get_info_gssiam_authenticate,
  ofd_gssiam_authenticate_remote) delegating verification to MDT0.
- Dynamic per-identity nodemap creation and property configuration
  (gssiam_get_nodemap) derived from the authenticated principal.
- Wire format definition (struct gssiam_auth_rep) in lustre_idl.h
  along with corresponding wiretest entries.
- Move DEFAULT_NODEMAP definition from nodemap_internal.h to
  lustre_nodemap.h as LUSTRE_DEFAULT_NODEMAP for cross-subsystem
  access.
- Rework tgt_mod_init() error-unwind paths in tgt_main.c to ensure
  complete rollback on initialization failures, including
  tgt_gssiam_init().

Signed-off-by: Di Wang <ddiwang@google.com>
Change-Id: I9fbd92e7e5dcdbc0039f90f547ce3841b005615e

66481 master J –B ✓T ✓R ✗

LU-12669 ec: fix EC recovery read for encrypted files

reviewing Zhenyu Xu · PS21→PS24 since your -1 — your objection was dropped

CR -1 ec2 pt_ecro218_review_blocker 5 unresolved L +309 −10

PS21→PS24 since your -1 — your objection was dropped

blocked on the owner: Zhenyu Xu voted -1

build #134661 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-11-02 00:14 · Zhenyu Xu -1 2026-11-02 10:01

no vote yet: Keguang Xu

patchsets:

+19 earlier

PS1 06-05 PS2 06-06 PS3 06-06 PS4 06-06 PS5 06-07 PS6 06-08 PS7 06-11 PS8 06-11 PS9 06-11 PS10 06-12 PS11 06-30 PS12 07-16 Marc Vef PS13 07-26 Marc Vef PS14 07-26 Marc Vef PS15 08-01 Marc Vef PS16 08-10 Keguang Xu PS17 08-10 PS18 08-10 PS19 08-11

PS20 08-11 PS21 08-19 PS22 08-20 PS23 10-08 Andreas Dilger PS24 10-28 Andreas Dilger

your previous vote: -1 on PS21 (2026-08-19 20:14) — now at PS24

threads: 5 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS24 2026-10-30 11:22 Gerrit AI review for Lustre lov_io.c:1984 — (suggestion) This isn't a bug, but this loop is a copy of the page-cache branch of the decrypt loop in osc_brw_fini_request(): it stops at t

Gerrit AI review for Lustre PS24 · 2026-10-30 11:22

(suggestion) This isn't a bug, but this loop is a copy of the page-cache branch of the decrypt loop in osc_brw_fini_request(): it stops at the first all-zero unit, clears private_2 when the zero unit is at offset 0, and calls llcrypt_decrypt_pagecache_blocks() per LUSTRE_ENCRYPTION_UNIT_SIZE. If the patch is refreshed, could both callers share one helper? Then a later change to the hole handling can't update only one of them.

reply PS24 2026-11-02 10:01 Zhenyu Xu lov_io.c:1994 — (defect) for 64K page size client (arm64), 4K empty does not mean a empty page.

Zhenyu Xu PS24 · 2026-11-02 10:01

(defect) for 64K page size client (arm64), 4K empty does not mean a empty page.
```
bool empty = true;

for (..) {
      if (is_empty_folio(folio, pgno, offs, unit))
                  continue;
      empty = false;
      ...
}
if (empty)
      folio_clear_private_2(folio);```

reply PS24 2026-11-02 10:11 Zhenyu Xu lov_io.c:2232 — (suggestion) as this function is also used to read raw data from data stripe, could change its name to lov_ec_read_raw_stripe(), and lov_ec_

Zhenyu Xu PS24 · 2026-11-02 10:11

(suggestion) as this function is also used to read raw data from data stripe, could change its name to lov_ec_read_raw_stripe(), and lov_ec_read_partiy_stripe() can be routed to it.

reply PS24 2026-10-30 11:22 Gerrit AI review for Lustre lov_io.c:2771 — (style) This isn't a bug, but lov_ec_read_parity_stripe() now also reads data stripes. Its name and its parameters (pcomp_i, parity_idx, par

Gerrit AI review for Lustre PS24 · 2026-10-30 11:22

(style) This isn't a bug, but lov_ec_read_parity_stripe() now also reads data stripes. Its name and its parameters (pcomp_i, parity_idx, parity_foff) still say parity, so this call reads as if it fetched parity stripe k. If the patch is refreshed, consider a neutral name such as lov_ec_read_raw_stripe() with comp_i/stripe_i/rg_off parameters.

reply PS24 2026-10-30 11:22 Gerrit AI review for Lustre lov_io.c:2922 — (suggestion) This fails some reads with -EIO even though the data could be rebuilt. Take 4+2 with data stripes A and B both down. A's page a

Gerrit AI review for Lustre PS24 · 2026-10-30 11:22

(suggestion) This fails some reads with -EIO even though the data could be rebuilt. Take 4+2 with data stripes A and B both down. A's page at position j is still cached from before the failure, and B's page at j is EC_DPG_ERROR:

- err_nr == 2, so two parity stripes are read
- lov_ec_read_raw_cached() skips A because lov_ec_readable_sub() returns NULL, so raw_folios[A] stays NULL
- at j, r_j == 1 and no_raw is set, so this returns -EIO and the whole read fails

The plaintext of A can't be used here. But A's page at j could be counted as one more erasure: its recov ptr would go to a scratch buffer, and the cached page would be left alone. Then r_j == err_nr and the shared recover_ctx already covers {A, B}. Is that worth handling here or in a follow-up? Before this patch the same case returned garbage, so -EIO is still an improvement.

tags: pt_ecro 218_review_blocker

owner Zhenyu Xu · uploader Andreas Dilger · PS24 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-12669 ec: fix EC recovery read for encrypted files

The parity of an encrypted file is computed from its data as stored
on the OSTs, but an EC recovery read rebuilds a stripe from data
pages that OSC has already decrypted, together with the parity as
stored, and returns garbage.

Rebuild the stripe from the data as stored instead: have OSC leave
the data pages read for the recovery encrypted and not uptodate,
rebuild the missing pages from them and the parity, then decrypt the
pages read and rebuilt, leaving an all-zero block (a hole) as OSC
does.  A data page already in the page cache holds the plaintext, so
read the stripe again from the OST for the positions being rebuilt,
rather than encrypting the plaintext, which cannot recreate the
stored data of a hole.  Without the key, leave the pages encrypted,
as a normal read does.  Keep these pages out of extents that OSC
decrypts, and send the encryption flag with the parity reads.

Add sanity-ec test_56a to recover an encrypted file with a hole in
a surviving stripe, both from the OSTs and with that stripe cached.

Test-Parameters: testlist=sanity-ec
Fixes: 783c13bb0b8f ("LU-12669 ec: recover data from parity")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: Ida4722d05d79bcbee4e680e75e94b005579adcac

69762 master J –B ✓T ✓R +2/2

LU-20287 lov: don't block setattr on unavailable OST

reviewing Keguang Xu · PS5→PS7 since your -1 — your objection was dropped

pt_ecro218_review_priority master-next 1 unresolved L +319 −25

build #134595 SUCCESS · tests all Maloo sessions

reviews: Ronnie Sahlberg +1 2026-10-29 03:43 · Andreas Dilger +1 2026-11-02 00:58

patchsets:

+2 earlier

PS1 10-17 PS2 10-18

PS3 10-22 PS4 10-25 PS5 10-27 PS6 10-27 Marc Vef PS7 10-28

your previous vote: -1 on PS5 (2026-10-27 09:34) — now at PS7

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS7 2026-11-02 00:58 Andreas Dilger cl_object.h:2567 — seems reasonable if patch is refreshed

Gerrit AI review for Lustre PS7 · 2026-10-28 16:35

(minor) The new mtime and the new atime are both compared with one shared baseline, max(ctime, sa_cur_time). sa_cur_time is the max of the old mtime and atime, the cached OST mtime and atime, and the inode mtime and atime that vvp_attr_get() reads at iter_init. ll_md_setattr() has already written the new values into the inode by then. ll_merge_attr() merges each field on its own (atime with atime, mtime with mtime), so a newer value in the *other* field can never win the max() for this one.

With the shared baseline:

    touch -m -d '+1 hour' f   # mtime ahead, no lock (new >= ctime)
    touch -a f                # atime = now < base (mtime +1h)

The second command now counts as backward. vvp_io_setattr_lock() takes a CLM_WRITE [0, EOF] lock, which revokes other clients' extent locks even when every OST is up. If an OST is down, the command also blocks. Before this patch it took no lock. touch -m behaves the same way when the atime is ahead. So does utimensat() with atime < mtime when both are later than ctime, because the new mtime is already in the inode.

Would it be safer to keep separate mtime and atime baselines (sa_cur_mtime / sa_cur_atime), so that each new time is compared only with its own field? The comments on sa_cur_time, in cl_setattr_ost() and in vvp_io_setattr_iter_init() would then also match what is actually collected.

Andreas Dilger PS7 · 2026-11-02 00:58

seems reasonable if patch is refreshed

tags: pt_ecro 218_review_priority

owner Keguang Xu · uploader Keguang Xu · PS7 uploaded 5d ago · NEW · open in Gerrit ↗

commit message

LU-20287 lov: don't block setattr on unavailable OST

chown/chgrp/chmod/touch wait in cl_setattr_ost() when a stripe OST
is down: VFS sets ATTR_CTIME and the client sends OST_SETATTR to
every stripe. That RPC only carries times/flags; UID/GID/PROJID are
authoritative on the MDT and are pushed by MDS osp_sync (persistent
llog, retried until the OST returns). Skipping the client RPC
therefore does not leave stale ownership on OST objects.

For non-truncate/non-fallocate setattr on any layout, mark LOV
sub-IOs ci_ndelay so OSC sends OST_SETATTR with rq_no_delay and
rq_no_resend (plus a 5s delay limit for idle import reconnect),
and ignore -EAGAIN/-ENOTCONN/-ESHUTDOWN/-ETIMEDOUT for such sub-IOs
in osc_io_setattr_end(). Ndelay applies only when the new mtime/atime
are >= max(ctime, sa_cur_time), where sa_cur_time is the known prior
inode/cached OST times: ll_merge_attr() takes max(MDT, OST), so
skipping an OST that already holds a newer time would leave that
value visible. Missed updates are not repaired later, and on a slow
but connected OST the client update may also be dropped.

The CLM_WRITE [0, EOF] lock in vvp_io_setattr_lock() uses that same
baseline, not ctime alone. Plain touch/utime(NULL) therefore takes
the lock when sa_cur_time is ahead of the client clock (writer clock
skew, or a future timestamp) and revokes other clients' cached
extent locks; that path does not set ndelay. That is intentional:
without it ndelay would skip the newer OST and max() would keep the
wrong time.

Still block when:
- mtime/atime moves earlier than that baseline;
- OP_XVALID_FLAGS is set (chattr, encryption flag): MDS never
  pushes flags to OSTs.

This only unblocks the setattr RPC itself (including chmod(2)).
stat/glimpse can still hang when no usable stripe/mirror is
reachable: e.g. coreutils chmod stats via FTS before chmod(2),
so the command may block on plain files even though the syscall
would not. That is unchanged here (see LU-11962 for MDT SOM).

Also re-push UID/GID/PROJID from the MDT inode on FLR RESYNC_DONE
in lod_layout_change() with the layout version update. osp_sync
normally retries ownership pushes; this is belt-and-suspenders if
an MDS->OST ownership setattr was skipped (e.g. exercised with
OBD_FAIL_LFSCK_BAD_OWNER; layout LFSCK can also repair that).

Add sanity 39v (plain file) and sanity-flr 100a/100b.

Test-Parameters: testlist=sanity env=ONLY=39v
Test-Parameters: testlist=sanity-flr
Assisted-by: Cursor:Grok-4.6/Opus5.5 llm_code_and_review_tools
Signed-off-by: Keguang Xu <kxu@ddn.com>
Change-Id: I5005451e10f293279c441c1fc6d012fb5337d95d

64086 master J –B ✓T ✓R 0/2

LU-20958 lov: handle unavailable EC stripes in lseek

reviewing Patrick Farrell · PS47→PS48 since your -1 — your objection was dropped

ec2 pt_ecro218_review_priority 5 unresolved L +453 −12

build #134718 SUCCESS · tests all Maloo sessions

no vote yet: Keguang Xu, Andreas Dilger, Zhenyu Xu

patchsets:

+43 earlier

PS1 02-17 PS2 03-29 PS3 03-29 PS4 03-29 PS5 03-29 Ronnie Sahlberg PS6 03-30 Ronnie Sahlberg PS7 03-31 Ronnie Sahlberg PS8 03-31 PS9 03-31 Ronnie Sahlberg PS10 04-01 Ronnie Sahlberg PS11 04-01 Ronnie Sahlberg PS12 04-07 Ronnie Sahlberg PS13 04-08 PS14 04-09 PS15 04-09 PS16 04-09 PS17 04-09 PS18 04-09 PS19 04-09 PS20 04-09 PS21 04-09 PS22 04-10 PS23 04-10 PS24 04-10 PS25 04-10 PS26 04-10 PS27 04-14 PS28 04-14 PS29 04-14 PS30 04-15 PS31 04-15 PS32 04-15 PS33 04-25 PS34 05-20 Marc Vef PS35 06-04 Ronnie Sahlberg PS36 07-13 Marc Vef PS37 07-13 Marc Vef PS38 07-13 Marc Vef PS39 07-15 Marc Vef PS40 07-16 Marc Vef PS41 07-26 Marc Vef PS42 07-26 Marc Vef PS43 08-17 Marc Vef

PS44 08-19 Maximilian Dilger PS45 10-14 Andreas Dilger PS46 10-16 Andreas Dilger PS47 10-21 Andreas Dilger PS48 10-29 Andreas Dilger

your previous vote: -1 on PS47 (2026-10-27 10:04) — now at PS48

threads: 4 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS47 2026-10-27 10:04 Marc Vef patchset comment — Highlighting a conflict with 69111. This patch (and the parent) needs to be refreshed due to 68361 since the code is no longer compatible. (

Marc Vef PS47 · 2026-10-27 10:04

Highlighting a conflict with 69111. This patch (and the parent) needs to be refreshed due to 68361 since the code is no longer compatible. (-1)

Does this patch need the current parent patches or can it be rebased and updatd for the RAID10 patch now?

reply PS48 2026-10-31 02:29 Gerrit AI review for Lustre file.c:5435 — (defect) The code here depends on lli_attr_valid losing OBD_MD_FLSIZE when the UPDATE lock is cancelled. That clearing is added by 69828 ("L

Gerrit AI review for Lustre PS48 · 2026-10-31 02:29

(defect) The code here depends on lli_attr_valid losing OBD_MD_FLSIZE when the UPDATE lock is cancelled. That clearing is added by 69828 ("LU-20958 llite: drop strict SOM size with the UPDATE lock"), but 69828 is not an ancestor of this patch set. The parent is 8bef60849e (an older patch set of 69694), which sits directly on 40b80bc048 on master. In this tree, lli_attr_valid is written only in ll_update_inode(), and nothing clears it in ll_md_blocking_ast().

Without 69828, this sequence goes wrong:

    client A: open/stat while the file is in sync -> OBD_MD_FLSIZE, i_size = S
    client B: writes the file up to T > S (MDS revokes LAYOUT|UPDATE, marks the SOM stale)
    an OST of a data stripe becomes unavailable
    client A: lseek(SEEK_DATA, S) -> ci_ec_unavail, but FLSIZE is still set
      -> no revalidate; vvp_io_lseek_start() skips ll_merge_attr()
      -> eof = S, so the result is -ENXIO although [S, T) has data

Because ll_merge_attr() is skipped, this also misses B's data on the stripes that are available. Should this patch be rebased onto 69828 so that the two cannot land in the wrong order?

reply PS47 2026-10-29 01:23 Andreas Dilger lov_io.c:3833 — Not yet in PS48: 68361 is still at PS17 and its stack does not apply to master. This will be done when 64086 is rebased onto the refreshed

Andreas Dilger PS47 · 2026-10-27 03:43

This needs to be updated for the LU-20690 parity layout in https://review.whamcloud.com/68361 ("LU-20690 ec: like-RAID10 parity layout"), which removes RAID0|PARITY: parity components become LOV_PATTERN_PARITY with the p parity objects of a raid set sharing one offset.  Once that lands, this check returns -EOPNOTSUPP for every EC file and the parity SEEK_DATA/SEEK_HOLE path is never used.  Please check for the parity flag/pattern instead and use the new mapping below.

Andreas Dilger PS47 · 2026-10-29 01:23

Not yet in PS48: 68361 is still at PS17 and its stack does not apply to master.  This will be done when 64086 is rebased onto the refreshed 68361.

reply PS47 2026-10-29 01:23 Andreas Dilger lov_io.c:3862 — Not yet in PS48, for the same reason. The slot helpers and absolute-offset row/set mapping will be used once 68361 is refreshed.

Andreas Dilger PS47 · 2026-10-27 03:43

The parity offset needs to change with 68361 in two ways:
1. RAID10 layout: one stripe_size per raid set per row, and the parity index selects the object rather than the offset, i.e. poff = e_start + (row * R + set) * ss + intra, pstripe = (stripe of that slot) + pi with R = esc_n0 + esc_n1, via the 69516/68361 slot helpers (lsme_parity_slots_per_raidset()) rather than lov_stripe_number() on the concatenated offset.
2. The row/set must come from the absolute data offset, not offset - e_start (see my lov_io.c comment on 68361 PS17): row = off / (c * ss), offset_i = (off / ss) % c, and poff relative to row0 = e_start / (c * ss).  Otherwise when e_start is not a multiple of the stripe width this looks at the parity of a different raid set than the one holding the data.
Ideally use the shared helper that 68361 adds for this mapping so the kernel recovery, llapi resync, hedge (64919) and lseek all agree.  Rebase onto that series once it is updated.

Andreas Dilger PS47 · 2026-10-29 01:23

Not yet in PS48, for the same reason.  The slot helpers and absolute-offset row/set mapping will be used once 68361 is refreshed.

reply PS48 2026-10-31 02:29 Gerrit AI review for Lustre lov_io.c:3865 — (minor) lov_parity_mirror_index_from_data() looks up the parity link from the data component at lio->lis_pos, which is the lseek start, not

Gerrit AI review for Lustre PS48 · 2026-10-31 02:29

(minor) lov_parity_mirror_index_from_data() looks up the parity link from the data component at lio->lis_pos, which is the lseek start, not from dsub's component dindex. If the seek starts in an earlier component of the mirror that has no parity link (e.g. a non-EC first PFL component), pmirror is < 0, and the stripe is reported as all data even though in-sync parity exists. The result is still correct, only less precise. If this is changed to use lsm_entries[dindex]->lsme_mirror_link_id, note that the LASSERTF in lov_io_sub_init() does the same lis_pos lookup, so it would need to stay in step for lov_sub_get() on the parity stripe.

tags: pt_ecro 218_review_priority

owner Patrick Farrell · uploader Andreas Dilger · PS48 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-20958 lov: handle unavailable EC stripes in lseek

SEEK_DATA/SEEK_HOLE on an EC file with a data stripe on an OST that
was deactivated would fail with -ESHUTDOWN, and with an OST that was
down, e.g. during an OSS failover, the lseek would wait for the OST to
return, although reads of the file are rebuilt from parity at once.

Do the lseek with no-delay lock enqueues, as reads do.  When the
sub-lock enqueue of a stripe in an EC data component fails, mark it in
this IO's lov_io_sub::sub_ec_err and do not start its sub-IO.  Retry
with blocking enqueues once every mirror has been tried.  The
unavailable stripe leaves its size out of the merged stripe
attributes, so take EOF from the strict SOM size from the MDS instead,
fetched again as stat() does if the UPDATE lock was dropped.  Without
one, wait for the OST, or return -EIO if it stays unavailable, rather
than miss data past a smaller EOF.  A stripe whose extent lock is
still cached is locked locally, so its OST_SEEK still waits for the
OST.

Resync writes parity only where a data stripe of its raid set has
data, page aligned, and punches the rest, so a hole in an in-sync
parity stripe is a hole in every data stripe of the raid set.  Seek an
available parity stripe of the unavailable stripe's raid set from the
matching offset, and map the result back, for the RAID0|PARITY parity
layout.  Each row of a raid set holds one stripe_size chunk in every
data and parity object, so offsets in the two objects advance
together.  The result can report data where only another stripe of the
raid set has data, but never a hole where the stripe may have data.
With no parity stripe that can answer, report the stripe as data from
where the IO starts in it, with no hole before its end, which is
always correct: a seek must not skip data, but may report a hole as
data.  Since such a stripe has no hole at its end, report the hole at
EOF for SEEK_HOLE if nothing before it is found, and -ENXIO for
SEEK_DATA at EOF.

The parity object is not covered by the lseek lock, so seek it with a
server lock, and do not trust the LVB size of an unlocked object in
osc_io_lseek_start().  Seek parity without delay, failing at once if
its OST is disconnected; an idle import still reconnects for the seek.

Add sanity-ec test_79a, which stops the OST of a stripe, and
test_79b, which checks holes found from parity, and that the stripe
is reported as data when the parity OSTs are stopped too.

Test-Parameters: testlist=sanity-ec ostcount=8
Fixes: 783c13bb0b8f ("LU-12669 ec: recover data from parity")
Signed-off-by: Patrick Farrell <patrick@thelustrecollective.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I5cce4e0ea51c68b0c6fda1d83b694af19cad57bd

▶

Longstanding (signal older than a week)

(101)

62389 master J –B ✓T ✓R 0/2

LU-19566 lustre: add layout flags to lfsck

reviewing Patrick Farrell · PS14→PS20 since your -1 — your objection was dropped

ec2 pw_ownpt_ecro L +761 −149

build #133811 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger, Lai Siyao, Alex Zhuravlev, Hongchao Zhang, Timothy Day

patchsets:

+15 earlier

PS1 11-05 PS2 11-05 PS3 11-06 PS4 11-07 PS5 11-07 PS6 11-07 PS7 11-08 PS8 11-08 PS9 11-09 PS10 11-11 PS11 11-11 PS12 11-26 PS13 03-30 Marc Vef PS14 03-30 Marc Vef PS15 10-07

PS16 10-08 PS17 10-08 PS18 10-09 patrick bot PS19 10-10 patrick bot PS20 10-16 patrick bot

your previous vote: -1 on PS14 (2026-04-02 10:21) — now at PS20

tags: pw_own pt_ecro

owner Patrick Farrell · uploader patrick bot · PS20 uploaded 17d ago · NEW · open in Gerrit ↗

commit message

LU-19566 lustre: add layout flags to lfsck

EC parity mirror components hold parity, not file data, so
LFSCK must restore them as parity components when it
rebuilds a layout from orphan OST objects. It restored each
component with only LCME_FL_INIT, losing the parity flag,
the EC geometry and the link between the data and parity
mirrors, so a rebuilt EC file could not use its parity.

For EC components, store the long-lived component flags
(prefer, nosync, parity, nocompr), the EC data and code
stripe counts and the mirror link id at the end of
filter_fid. The client sends them on writes and setattr in
the obdo fields that were o_uid_h and o_gid_h, never used,
flagged by the never-used 0x00200000 valid bit (formerly
OBD_MD_FLQOS), now OBD_MD_FLCOMPFLAGS, and the OST passes
them to the MDT in the orphan record, so LFSCK restores
them. The objects only get them from a write or setattr, so
a change of flags is stored once the file is next written. A
rebuilt parity component also gets the parity pattern, which
clients use to keep reads off it, and LFSCK accepts that
pattern when it adds further orphans to such a file. When
LFSCK adds a component to an EC file its parity is marked
stale, since the objects do not show whether the parity was
current, so it is only used again after a resync. LFSCK
scans still skip EC files: repairing a missing parity object
there would need the parity regenerated or marked stale.

STALE is not stored: resync writes an object while its
mirror is still stale, so a stored STALE is backward and can
rebuild a layout whose mirrors are all stale, which LBUGs
the MDS when it loads that layout.

Other components keep the 52-byte filter_fid, so that
servers before this change and older tools still read it,
and only EC files, which need the new servers anyway, get
the 60-byte form. Servers without this change do not handle
it: their LFSCK and MDT migration get -ERANGE reading it,
and an OST keeping the parent FID in LMA (256-byte inodes)
refuses to have it written, unless they have "lfsck: read a
filter_fid longer than ours", which this patch is based on.
An OST keeping the parent FID in LMA has no room for these
fields, and does not store them.

sanity-lfsck test_47 and test_48 cover mirrors after resync
and EC files; test_48 checks the rebuilt EC geometry and,
after a resync, reads the file with a data OST failing.
test_45 now waits for LFSCK to finish on every target, not
only on the OST it checks, so that the next test can start
LFSCK.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Assisted-by: ClaudeCode:claude-opus-5-5
Change-Id: I716dbf24db48cc50385a2fa9bcc446d291380814

68801 master J !B ✗T –R +1/2

LU-10026 ofd: fix punch vs BRW deadlock on compressed files

mine build failure (#131557)

csdc XS +8 −1

10d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #131557 FAILURE

reviews: Sebastien Buisson +1 2026-10-23 09:56

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: fix punch vs BRW deadlock on compressed files

ofd_object_punch_compr() converts a compressed chunk back into a
plain one when a truncate lands in the middle of it.  It asks the
OSD for the chunk pages first and takes the chunk range lock
afterwards, while ofd_preprw_read_compr() and
ofd_preprw_write_compr() take the range lock first and only then
ask the OSD for the pages.

A truncate and a BRW touching the same chunk deadlock on that: the
punch thread holds the page locks that osd_bufs_get() acquired and
sleeps in range_lock(), while the BRW thread holds the chunk range
lock and sleeps in osd_get_page() waiting for one of those pages.
Nothing breaks the cycle, so the OST_PUNCH never completes and the
client stays blocked in ftruncate().  sanity-compr test_fsx trips
this a minute or two after fsx starts and then eats the whole
session timeout.

Take the chunk range lock before mapping the chunks to OSD buffers
so that all three paths lock the range before the pages.

Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11952
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6095f9533c23a376cf0b5c84bb75c4d956922204

65737 master J –B ✓T ✗2R -1/2−

LU-10026 csdc: set compress component for file

mine needs rebase — checkpatch: cannot be cherry-picked

needs rebase csdc 14 unresolved XL +1163 −16

11d

needs rebase — checkpatch: cannot be cherry-picked

enforced failed: custom-1001, review-dne-part-7

14 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

failed enforced test	platform	detail
custom-1001	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-pfl.	session
review-dne-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-pcc.	session

build #132858 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Roman Bolshakov, Andreas Dilger, Artem Blagodarenko

patchsets:

+10 earlier

PS1 05-07 PS2 05-07 PS3 05-28 PS4 05-28 PS5 06-11 PS6 06-11 PS7 06-16 Artem Blagodarenko PS8 06-16 Artem Blagodarenko PS9 07-24 Artem Blagodarenko PS10 08-19 Artem Blagodarenko

PS11 08-19 Artem Blagodarenko PS12 08-20 Artem Blagodarenko PS13 08-20 Artem Blagodarenko PS14 08-21 Artem Blagodarenko PS15 10-05 Roman Bolshakov

threads: 14 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS15 2026-10-06 08:14 Gerrit AI review for Lustre COMMIT_MSG:56 — (minor) The body doesn't mention the client-side gate added by squashing Ib1e2123ff: the new lustre_check_lov_user_md() and lustre_check_lov

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(minor) The body doesn't mention the client-side gate added by squashing Ib1e2123ff: the new lustre_check_lov_user_md() and lustre_check_lov_comp_md_v1() in ptlrpc/pack_generic.c, and their calls from ll_lov_setstripe_ea_info() and ll_xattr_set(). These reject a compress layout when `llite.*.enable_compression=0`, which is a user-visible behaviour change. Could a short paragraph describing them be added?

reply PS11 2026-08-20 12:36 Artem Blagodarenko patchset comment — Hi Andreas. Thanks for poining to this. I have reverted all fixed by Patchset 11 pushing wrong branch. I have reverted to Patchset 10 where

Andreas Dilger PS11 · 2026-08-19 23:52

@ablagodarenko@thelustrecollective.com it looks like the patches were rebased, but did not address any of the AI review comments?  While we can't go overboard changing every detail of the patches, I think it does make sense to fix the code where possible so that we don't carry technical debt later.

Artem Blagodarenko PS11 · 2026-08-20 12:36

Hi Andreas. Thanks for poining to this. I have reverted all fixed by Patchset 11 pushing wrong branch. I have reverted to Patchset 10 where I addressed AI bot comments and fixed build.

reply PS14 2026-10-05 17:51 Roman Bolshakov patchset comment — Squashed Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f into the change. I'll address a couple more comments in another push.

Roman Bolshakov PS14 · 2026-10-05 17:51

Squashed Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f into the change. I'll address a couple more comments in another push.

reply PS9 2026-08-19 23:52 Andreas Dilger lfs-getstripe.1:331 — Should be fixed if patch is refreshed.

Gerrit AI review for Lustre PS9 · 2026-07-28 09:23

(minor) There is no `.TP` before this entry, so `--compress-type|--compr-type` and its description get folded into the `-z, --extension-size` paragraph above instead of starting their own tagged item. The two entries below it do have theirs.

Oleg Drokin PS9 · 2026-08-12 02:29

weird but in my man output it looks ok.

Artem Blagodarenko PS10 · 2026-08-19 19:47

The entry was missing its .TP in the troff source. Some man implementations are lenient and reflow correctly, but others fold the entry into the previous paragraph. Added .TP for correctness.

Artem Blagodarenko PS10 · 2026-08-19 19:47

Done. Added the missing .TP before the --compress-type|--compr-type entry.

Andreas Dilger PS9 · 2026-08-19 23:52

Should be fixed if patch is refreshed.

reply PS14 2026-08-22 12:47 Gerrit AI review for Lustre llapi_layout_compress_set.3:105 — (minor) Neither of the two new man3 pages has an EXAMPLES section. Lustre llapi pages are expected to carry a short usage example showing th

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) Neither of the two new man3 pages has an EXAMPLES section. Lustre llapi pages are expected to carry a short usage example showing the required header and a realistic call.

reply PS15 2026-10-06 08:14 Gerrit AI review for Lustre llapi_layout_compress_set.3:71 — (minor) This, and the matching EINVAL entry under ERRORS, says a `stripe_size` that is not a multiple of `chunk_size` is an error. llapi_lay

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(minor) This, and the matching EINVAL entry under ERRORS, says a `stripe_size` that is not a multiple of `chunk_size` is an error. llapi_layout_compress_set() never fails for that: it decrements `log_bits` until the chunk divides the stripe and stores the smaller chunk (sanity-pfl test_100c expects 3*64KiB to become 128KiB). Similarly, gzip/lz4/lz4fast levels outside the listed ranges are clamped by their from_*_level() helpers, not rejected with EINVAL as the text above says (test_100c expects gzip:15 to read back as 9). If the patch is refreshed, could the page describe the rounding and clamping instead?

reply PS14 2026-08-22 12:47 Gerrit AI review for Lustre llapi_parse_compress_type.3:62 — (minor) llapi_parse_compress_type() also returns -EINVAL - for a level that isn't a valid number or falls outside +/-COMPR_LEVEL_MAX - which

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) llapi_parse_compress_type() also returns -EINVAL - for a level that isn't a valid number or falls outside +/-COMPR_LEVEL_MAX - which neither RETURN VALUES nor ERRORS mentions. Mixing -1 with -Exxx returns in one function is also awkward for callers; would returning -EINVAL for the unknown-type case be more consistent?

reply PS14 2026-08-22 12:47 Gerrit AI review for Lustre lustreapi.h:1457 — (minor) The 9 used as the lz4 default collides with a real level. from_lz4_level(10) returns 9, and to_lz4_level(9) returns 9 because 9 == L

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) The 9 used as the lz4 default collides with a real level. from_lz4_level(10) returns 9, and to_lz4_level(9) returns 9 because 9 == LZ4HC_DEFAULT_CLEVEL, so 'lfs setstripe -Z lz4:10' reports back as lz4:9 and level 10 can never be displayed - even though lfs-setstripe.1 documents 1-16 as the valid lz4 range.

reply PS15 2026-10-06 08:14 Gerrit AI review for Lustre file.c:2838 — (defect) This gate runs only for regular files: ll_lov_setstripe_ea_info(), and the `lov.` branch in ll_xattr_set(). A directory default goe

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) This gate runs only for regular files: ll_lov_setstripe_ea_info(), and the `lov.` branch in ll_xattr_set(). A directory default goes through ll_setstripe_ea() -> ll_dir_setstripe() (and the LL_IOC_LOV_SETSTRIPE case in ll_dir_ioctl()), which never calls lustre_check_lov_user_md(). So on a client with `enable_compression=0`:

    lfs setstripe -E eof -Z lz4 $dir    # accepted
    touch $dir/f                        # MDS instantiates a compress layout

This still produces a file the client can't handle. lsm_lmm_verify_v1v3() rejects it with "file compression not enabled", and the PS14 report shows a later unlink or stripe change hitting the lov_io_layout_at() LBUG. Should ll_dir_setstripe() apply the same check? Or, if allowing compressed defaults on a disabled client is intended, should the client reject the layout cleanly instead of asserting?

reply PS15 2026-10-06 08:14 Gerrit AI review for Lustre xattr.c:425 — (defect) On this `lov.` path nothing validates `size` before lustre_check_lov_user_md() walks the entries. lustre_check_lov_comp_md_v1() rea

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) On this `lov.` path nothing validates `size` before lustre_check_lov_user_md() walks the entries. lustre_check_lov_comp_md_v1() reads `lcm_entry_count` and then up to 65535 `lcm_entries[]`, but the buffer here is only the `size` bytes copied in by the VFS.

The `lustre.` xattr namespace needs only write permission on the inode, so any file owner can trigger it:

    setxattr(f, "lustre.lov.add", buf, 8, 0);  /* buf starts with LOV_USER_MAGIC_COMP_V1 */

That reads lcm_entry_count past the end of an 8-byte allocation and keeps reading for up to ~3MB. KASAN reports it as slab-out-of-bounds. Before this patch, little-endian clients only read `lmm_magic` here.

The next change in the series ("utils: add 'lfs find' support for compressed file") adds a `size < need` check just above this call. Since this patch introduces the read, should that check move here? Passing `size` into lustre_check_lov_user_md(), the way lustre_swab_lov_user_md() takes one, would also let it validate the buffer itself.

reply PS14 2026-08-22 12:47 Gerrit AI review for Lustre lod_qos.c:3191 — (minor) This applies the chunk adjustment to every component including extension ones, where llc_stripe_size holds the extension size rather

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) This applies the chunk adjustment to every component including extension ones, where llc_stripe_size holds the extension size rather than a real stripe size. lod_declare_layout_add() deliberately skips LCME_FL_EXTENSION components for exactly that reason, so the same input layout gets different chunk_lum_bits depending on whether it arrives via create or via --component-add. Should this site skip extension components as well?

While there: the comment added in lod_declare_layout_add() says chunk adjustment is deferred until instantiation, but nothing calls lod_adjust_compr_chunk_size() on the instantiation path - the only three callers are here, lod_declare_layout_add() and lod_striping_from_default().

reply PS15 2026-10-06 08:14 Gerrit AI review for Lustre pack_generic.c:2672 — (defect) -ENOTSUPP (524) is kernel-internal and has no userspace errno, so this reaches `lfs setstripe` through ioctl()/setxattr() as "Unkno

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) -ENOTSUPP (524) is kernel-internal and has no userspace errno, so this reaches `lfs setstripe` through ioctl()/setxattr() as "Unknown error 524". The erasure-coding check just above it in ll_lov_setstripe_ea_info() returns -EOPNOTSUPP for the same kind of rejection. Should this be -EOPNOTSUPP too?

While here, a couple of nits: `rc` is never changed from 0, there's a double blank line after the EXPORT_SYMBOL(), and the default case in lustre_check_lov_user_md() logs "Invalid LOV magic" but still returns 0.

tags: csdc

owner Marc Vef · uploader Roman Bolshakov · PS15 uploaded 27d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: set compress component for file

Introduce per-component compression support for composite LOV layouts.
A new LCME_FL_COMPRESS component flag marks entries that carry
compress fields; the flag is set when the user supplies a compression
type via 'lfs setstripe -Z' and is propagated through the LOD and LOV
layers so that compressed components are distinguished from plain
ones.

LOD layer:
- lod_parse_striping(): read lcme_compr_type/lvl/chunk_lum_bits from
  the user-supplied lov_comp_md_v1 when LOV_PATTERN_COMPRESS is set.
- lod_use_defined_striping(): likewise for pre-defined layouts.
- lod_get_default_lov_striping(): carry compress fields into the LOD
  component when inheriting a default layout that includes
  compression.
- lod_generate_lovea(): serialise llc_compr_* fields into the on-wire
  lcm entry when LOV_PATTERN_COMPRESS is set.
- lod_adjust_compr_chunk_size(): new helper that rounds down the
  requested chunk-size exponent so the chunk fits evenly within one
  stripe and within one RPC (clamped to COMPR_CHUNK_MAX_BITS in the
  following changes).

API / utils layer:
- 'lfs setstripe' gains --compress|-Z <type>[:<level>] and
  --compress-chunk=<size> options to set the compression type, level,
  and chunk size on a new component.  Like -S and -c, -Z is inherited
  by subsequent -E components unless overridden by another -Z.

  Example:
	$ lfs setstripe -Eeof -Z lz4:5 --compress-chunk=512 <file>

- 'lfs getstripe' gains --compress-type, --compress-level, and
  --compress-chunk options to print individual compress parameters:

  Example:
	$ lfs getstripe <file>
    lcme_compr_type:     lz4
    lcme_compr_lvl:      5
    lcme_compr_chunk_kb: 512
      lmm_pattern:       raid0,compress

- llapi_layout_merge(): copy compress fields when merging layouts.
- Two new man3 pages: llapi_layout_compress_set.3 and
  llapi_parse_compress_type.3.

Misc:
- dump_lsm() in lov_ea.c prints compress fields for debugging.
- compr_name_from_type() helper added in lov_internal.h and wired up
  in dump_lsm() to print compression type names in debug output.

Test-Parameters: testlist=sanity-pfl serverversion=2.17
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Signed-off-by: Roman Bolshakov <rbolshakov@ddn.com>
Change-Id: Ife0382469cbc5099e0c6dc96534bb169ddeff61e
Was-Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f

69026 master J ✗1B ✓T ✗1R -2/2−

LU-17217 nodemap: add client version policy

cc Timothy Day · Andreas Dilger pinged you: “@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here? If this pa”

needs rebase 1 unique failure 12 unresolved L +805 −3

20d

Andreas Dilger pinged you: “@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here? If this pa”

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; janitor: test failures unique to this patch; enforced tests failed

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-hsm@ldiskfs+DNE:test_254b	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

failed enforced test	platform	detail
review-ldiskfs-dne-arm	RHEL 8.10 / x86_64, Rocky 9.8 / aarch64	ran 5 tests. 1 tests failed: sanity.	session

build #131922 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger -1 2026-09-18 04:21

no vote yet: Patrick Farrell

1 ping waiting on you:

@you PS1 2026-09-19 08:08 Andreas Dilger lustre_idl.h:4001 — @sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here? If this patch is introducing the mismatch between the flags

Gerrit AI review for Lustre PS1 · 2026-09-18 09:49

(minor) Adding the bit changes NODEMAP_RAISE_PRIV_NONE from 0xfffffe00 to 0xfffffc00, and that value is stored on disk rather than being a pure in-memory sentinel.

nodemap_idx_cluster_roles_modify() writes a NODEMAP_CLUSTER_ROLES record whenever rbac != NODEMAP_RBAC_ALL, and the record carries ncrr_privs = nmf_raise_privs, i.e. the old 0xfffffe00 for a nodemap whose raise_privs is 'none'. nodemap_cluster_roles_helper() then assigns it back verbatim with no masking.

So after an upgrade, a nodemap that merely had a non-default rbac comes back with bit 0x200 set, and

    if ((nodemap->nm_parent_nm->nmf_raise_privs & priv) && priv != NODEMAP_RAISE_PRIV_RBAC)
            return true;

short-circuits the new NODEMAP_RAISE_PRIV_VERSION check, so a dynamic child can override a parent policy that was meant to be locked. raise_privs would also read back as 'version_policy' for a nodemap the admin set to 'none'.

This encoding predates the patch (CAPS and DENY_MNT had the same effect when they were added) so it probably belongs in its own change rather than blocking this one - but is it worth masking the loaded value to the known bits, or is the intent that unknown bits stay raised?

Andreas Dilger PS1 · 2026-09-19 08:08

@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here?  If this patch is introducing the mismatch between the flags then it makes sense to fix it here as well instead of separate patch.

Sebastien Buisson PS1 · 2026-09-25 15:55

`NODEMAP_RAISE_PRIV_NONE` was made on the same model as `NODEMAP_RBAC_NONE`, but the sematic is quite the opposite actually. Rbac roles are all present by default, and we remove some of them to lower client capabilities. 'Raise privs' are all absent by default, and we add some of them to grant privileges. So I think we should set `ncrr_privs` to 0 when `nmf_raise_privs` is `NODEMAP_RAISE_PRIV_NONE`. That would avoid the problem mentioned here when we upgrade a system after new privs have been added, and that would be a simple fix. I could push a quick patch, so that it is fixed right now and does not wait for this bigger patch.

threads: 12 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-09-18 09:49 Gerrit AI review for Lustre COMMIT_MSG:27 — (minor) "older servers never see it" only holds while no policy is set. Once one is set the NODEMAP_CLUSTER_VERSION record is written to the

Gerrit AI review for Lustre PS1 · 2026-09-18 09:49

(minor) "older servers never see it" only holds while no policy is set. Once one is set the NODEMAP_CLUSTER_VERSION record is written to the MGS index and distributed to every server, including older ones - which is exactly the case that matters.

The subid-unknown fallback in nodemap_process_keyrec() that ignores it with a warning only landed in 2.15.55, so a 2.15.x LTS server does not have it. Could the sentence say instead that the policy is simply not enforced on servers that predate it?

reply PS1 2026-09-18 04:21 Andreas Dilger lctl-nodemap-set-version-policy.8:18 — Is there a reason not to include the 4th version digit? That is needed for versions like 2.18.0-tlc7 or whatever.

Andreas Dilger PS1 · 2026-09-18 04:21

Is there a reason not to include the 4th version digit?  That is needed for versions like 2.18.0-tlc7 or whatever.

reply PS1 2026-09-18 04:21 Andreas Dilger lctl-nodemap-set-version-policy.8:80 — I'm not sure that "glob" is actually the right paradigm for this, if it ends up being converted to a number anyway. I think that storing a m

Timothy Day PS1 · 2026-09-17 20:20

We probably want more glob examples.

Andreas Dilger PS1 · 2026-09-18 04:21

I'm not sure that "glob" is actually the right paradigm for this, if it ends up being converted to a number anyway. I think that storing a min and max numerical range for allow or deny would match the functionality better:

    --policy allow --min 2.16.0.22 --max 2.18.1.12

reply PS1 2026-09-18 04:21 Andreas Dilger lustre_disk.h:388 — This could be reduced to `__u32 nvpr_min` and `__u32 nvpr_max`. If this is the only such record per nodemap, then it could contain a min/ma

Andreas Dilger PS1 · 2026-09-18 04:21

This could be reduced to `__u32 nvpr_min` and `__u32 nvpr_max`.  If this is the only such record per nodemap, then it could contain a min/max allow and min/max block version (16 bytes) and still have space left for some other parameters.  It would be possible to have separate allow and block records do describe multiple non-overlapping regions (e.g. blacklist specific versions), but I don't know if that is needed yet?

I'm not sure if there are other parameters we could store within a single record?

reply PS1 2026-09-18 04:21 Andreas Dilger lustre_disk.h:392 — (style) align fields in struct declarations

Andreas Dilger PS1 · 2026-09-18 04:21

(style) align fields in struct declarations

reply PS1 2026-09-25 15:55 Sebastien Buisson lustre_idl.h:4001 — `NODEMAP_RAISE_PRIV_NONE` was made on the same model as `NODEMAP_RBAC_NONE`, but the sematic is quite the opposite actually. Rbac roles are

Gerrit AI review for Lustre PS1 · 2026-09-18 09:49

(minor) Adding the bit changes NODEMAP_RAISE_PRIV_NONE from 0xfffffe00 to 0xfffffc00, and that value is stored on disk rather than being a pure in-memory sentinel.

nodemap_idx_cluster_roles_modify() writes a NODEMAP_CLUSTER_ROLES record whenever rbac != NODEMAP_RBAC_ALL, and the record carries ncrr_privs = nmf_raise_privs, i.e. the old 0xfffffe00 for a nodemap whose raise_privs is 'none'. nodemap_cluster_roles_helper() then assigns it back verbatim with no masking.

So after an upgrade, a nodemap that merely had a non-default rbac comes back with bit 0x200 set, and

    if ((nodemap->nm_parent_nm->nmf_raise_privs & priv) && priv != NODEMAP_RAISE_PRIV_RBAC)
            return true;

short-circuits the new NODEMAP_RAISE_PRIV_VERSION check, so a dynamic child can override a parent policy that was meant to be locked. raise_privs would also read back as 'version_policy' for a nodemap the admin set to 'none'.

This encoding predates the patch (CAPS and DENY_MNT had the same effect when they were added) so it probably belongs in its own change rather than blocking this one - but is it worth masking the loaded value to the known bits, or is the intent that unknown bits stay raised?

Andreas Dilger PS1 · 2026-09-19 08:08

@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here?  If this patch is introducing the mismatch between the flags then it makes sense to fix it here as well instead of separate patch.

Sebastien Buisson PS1 · 2026-09-25 15:55

`NODEMAP_RAISE_PRIV_NONE` was made on the same model as `NODEMAP_RBAC_NONE`, but the sematic is quite the opposite actually. Rbac roles are all present by default, and we remove some of them to lower client capabilities. 'Raise privs' are all absent by default, and we add some of them to grant privileges. So I think we should set `ncrr_privs` to 0 when `nmf_raise_privs` is `NODEMAP_RAISE_PRIV_NONE`. That would avoid the problem mentioned here when we upgrade a system after new privs have been added, and that would be a simple fix. I could push a quick patch, so that it is fixed right now and does not wait for this bigger patch.

reply PS1 2026-09-17 20:43 patrick bot mdt_handler.c:7593 — **[AI review - opus]**

patrick bot PS1 · 2026-09-17 20:43

**[AI review - opus]**

(suggestion) Refusing on the reconnect path means a client that is in the recovery set and is now blocked can never come back: ptlrpc_connect_interpret() sets IMPF_DEACTIVE on -EACCES, so the target waits on it for the whole recovery window before evicting it. nmf_deny_mount avoids this by checking in mdt_get_root() rather than at connect.

Is it worth releasing the recovery slot (or only enforcing on an initial connect) so a version policy change does not stretch out the next recovery?

reply PS1 2026-09-18 04:21 Andreas Dilger nodemap_handler.c:390 — With numeric ranges this would be possible - the child nodemaps could be allowed to increase the deny range or decrease the accept range.

Andreas Dilger PS1 · 2026-09-18 04:21

With numeric ranges this would be possible - the child nodemaps could be allowed to increase the deny range or decrease the accept range.

reply PS1 2026-09-17 20:43 patrick bot nodemap_handler.c:3935 — **[AI review - opus]**

patrick bot PS1 · 2026-09-17 20:43

**[AI review - opus]**

(defect) Deleting the record when the policy goes back to allow:* assumes the nodemap's inherited value is also the default, but nodemap_inherit_properties() copies nmf_version_policy/nm_version_glob from the 'default' nodemap for every non-dynamic nodemap.

So with a policy set on 'default':

    lctl nodemap_set_version_policy --name default --policy block --glob '2.15.*'
    lctl nodemap_add c0                 # c0 inherits block:2.15.* in memory, no record written
    lctl nodemap_set_version_policy --name c0 --policy allow

the last command takes the is_default() branch, deletes a key that was never inserted, and leaves no NODEMAP_CLUSTER_VERSION record for c0. nodemap_idx_delete_batch() swallows -ENOENT and still calls nodemap_inc_version(), so the other servers re-read the config and rebuild c0 through nodemap_cluster_rec_helper() -> nodemap_create() -> nodemap_inherit_properties(), which hands it back block:2.15.*. The MGS reports allow:* for c0 while the MDTs and OSTs keep refusing those clients, and the MGS agrees again after a restart.

nm_sepol is deliberately not inherited from 'default' (see the src->nm_id == LUSTRE_NODEMAP_DEFAULT_ID case); would the same treatment work here, or should the record be written whenever the policy differs from what would be inherited?

reply PS1 2026-09-18 04:21 Andreas Dilger nodemap_handler.c:4002 — we don't really need such a verbose comment

Andreas Dilger PS1 · 2026-09-18 04:21

we don't really need such a verbose comment

reply PS1 2026-09-17 20:20 Timothy Day nodemap_handler.c:4007 — This might be a bit spammy.

Timothy Day PS1 · 2026-09-17 20:20

This might be a bit spammy.

reply PS1 2026-09-18 04:21 Andreas Dilger sanity-sec.sh:12297 — One option would be to have a fail_loc plus fail_val that sets the client version sent to the servers at connect time. The 32-bit fail_val

Timothy Day PS1 · 2026-09-17 20:21

There's probably a smarter, more concise way to test this using some kind of fake client.

Andreas Dilger PS1 · 2026-09-18 04:21

One option would be to have a fail_loc plus fail_val that sets the client version sent to the servers at connect time.  The 32-bit fail_val can store the encoded version so it would be possible to report any arbitrary version code.  This doesn't allow emulating ocd_connect_flags, but I don't think that matters in this case.

owner Timothy Day · uploader Timothy Day · PS1 uploaded 45d ago · NEW · open in Gerrit ↗

commit message

LU-17217 nodemap: add client version policy

Let admins control which Lustre client versions may connect,
using nodemap rather than a separate policy framework. Each
nodemap gets a version_policy property, shown as POLICY:GLOB,
where GLOB is matched against the client "major.minor.patch"
version when it connects to an MDT or OST:

  allow:GLOB       only clients matching GLOB may connect
  block:GLOB       clients matching GLOB are refused

The default is allow:*, which accepts every client. A refused
client gets -EACCES at connect time, so the mount fails and the
import is deactivated rather than retried forever. Server to
server connections (MDS, MDS_MDS and lightweight) are never
checked, so a policy cannot partition the servers.

The policy is stored as a new NODEMAP_CLUSTER_VERSION record in
the nodemap config, so it is distributed to all servers like the
other nodemap properties. The record is only written when a
policy is set, so older servers never see it. Dynamic nodemaps
inherit the policy from their parent and may only change it if
the parent has no policy, or raises the new version_policy
privilege via child_raise_privileges.

Add "lctl nodemap_set_version_policy --name NM --policy
{allow|block} [--glob GLOB]", a man page, and sanity-sec
tests 86a (mount enforcement) and 86b (dynamic nodemap
privilege).

Test-Parameters: trivial testlist=sanity-sec env=ONLY="86a 86b"
Signed-off-by: Timothy Day <timday@thelustrecollective.com>
Change-Id: I380802f736a87dd29ad746efff9d3c386a6a6964

68797 master J ✗1B ✗T –R +2/2

LU-10026 ofd: properly locate root object

mine build failure (#133377)

1 unique failure csdc 1 unresolved S +17 −8

22d

build failure (#133377)

janitor: 1 test failure unique to this patch — sanity1@ldiskfs+DNE:test_56od

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity1@ldiskfs+DNE:test_56od	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #133377 FAILURE

reviews: Andreas Dilger +1 2026-10-11 04:49 · Mikhail Pershin +1 2026-09-11 06:17

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

On this branch ofd_tunables_init() already ran before the compr_stats block, so the new failure paths jump to a new err_fini_tunables label (ofd_procfs_fini()) ahead of the existing teardown. When the compr_stats file cannot be obtained the stored error pointer is cleared so ofd_fini() does not try to save/put it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: properly locate root object

ofd_init0() gets root object by using dt_locate_at() with
an OSD device as top. As result lu_object can be incomplete
if it doesn't exist yet. Instead dt_locate() should be used.

Patch fixes that and adds error handling in related places.
When the compr_stats file cannot be obtained the reference is
cleared, so that it is neither saved nor released at fini.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-13567
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9f5eaaae9c4ea23225f574b8e7cde64b3c5f47b6

62859 master J ✓B ✓T ✗12R -1/2−

LU-12668 ec: Add tests for computing the parity coverage

carrying Ronnie Sahlberg · needs rebase — checkpatch: cannot be cherry-picked

needs rebase ec2 pt_ecro L +460 −0

44d

needs rebase — checkpatch: cannot be cherry-picked

enforced failed: custom-1001, custom-1002, review-dne-subtest-change, review-dne-zfs-subtest-change, review-ldiskfs-dne-arm

failed enforced test	platform	detail
custom-1001	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
custom-1002	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-ldiskfs-dne-arm	RHEL 8.10 / x86_64, Rocky 9.5 / aarch64	ran 5 tests. 1 tests failed: sanity.	session

build #127014 SUCCESS · tests all Maloo sessions

no vote yet: Keguang Xu, Qian Yingjin, Maximilian Dilger

patchsets:

+42 earlier

PS1 12-03 PS2 12-03 PS3 12-03 PS4 12-03 PS5 12-03 Patrick Farrell PS6 12-03 Patrick Farrell PS7 12-03 Patrick Farrell PS8 12-04 Patrick Farrell PS9 12-04 PS10 12-04 Patrick Farrell PS11 12-04 Patrick Farrell PS12 12-05 PS13 12-05 PS14 12-05 PS15 12-05 PS16 12-06 PS17 12-09 PS18 12-09 Patrick Farrell PS19 12-09 Patrick Farrell PS20 12-10 PS21 12-10 Patrick Farrell PS22 12-10 PS23 12-10 PS24 12-11 PS25 12-12 Patrick Farrell PS26 12-15 PS27 12-15 PS28 12-16 Patrick Farrell PS29 01-13 Patrick Farrell PS30 01-13 Patrick Farrell PS31 01-14 Patrick Farrell PS32 01-15 Patrick Farrell PS33 01-16 Patrick Farrell PS34 01-29 Patrick Farrell PS35 03-29 Patrick Farrell PS36 03-29 Patrick Farrell PS37 03-31 Patrick Farrell PS38 04-08 Patrick Farrell PS39 04-09 Patrick Farrell PS40 04-10 Patrick Farrell PS41 04-21 Zhenyu Xu PS42 05-05 Patrick Farrell

PS43 05-06 PS44 05-14 Marc Vef PS45 05-20 Marc Vef PS46 06-22 Marc Vef PS47 06-26 Marc Vef

your previous vote: -1 on PS45 (2026-05-21 08:46) — now at PS47

tags: pt_ecro

owner Ronnie Sahlberg · uploader Marc Vef · PS47 uploaded 128d ago · NEW · open in Gerrit ↗

commit message

LU-12668 ec: Add tests for computing the parity coverage

When we write or verify the parities we no longer
use the whole stripe, instead we compute the coverage
of what ranges of parity is important and what can be
ignored.
This is based on SEEK_DATA/SEEK_HOLE and EOF.

Add tests that we compute these ranges correctly.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: Iaee9d0ccb0875f6d515a8adc322fa3cd37bfe0f6

57776 master J ✗2B ✓T ✓R +1/2−

LU-10026 lustre: add lz4 and lz4hc kernel modules

carrying Artem Blagodarenko · needs rebase — checkpatch: cannot be cherry-picked

needs rebase csdc 5 unresolved XL +4422 −2

46d

needs rebase — checkpatch: cannot be cherry-picked

Timothy Day voted -1

3 unresolved thread(s) await your reply — last from Timothy Day

2 thread(s) waiting on others

no vote yet from Sebastien Buisson, Oleg Drokin

janitor flagged test failures unique to this patch — superseded (tests have since passed) — full janitor results

unique failing test	history
sanity-lfsck@zfs:test_18c	seen in 25 other reviews
sanity-scrub@ldiskfs+DNE:test_9	seen in 5 other reviews

build #125233 SUCCESS · tests all Maloo sessions

reviews: James Simmons +1 2026-05-29 01:12 · Marc Vef +1 2026-05-19 09:37 · Andreas Dilger +1 2026-05-19 20:23 · Timothy Day -1 2026-06-05 20:49

no vote yet: Sebastien Buisson, Oleg Drokin

patchsets:

+27 earlier

PS1 01-15 PS2 01-15 PS3 01-15 PS4 01-16 PS5 01-16 PS6 01-16 PS7 01-17 PS8 01-17 PS9 01-20 PS10 01-21 PS11 01-22 PS12 04-08 Timothy Day PS13 04-08 Timothy Day PS14 04-09 Timothy Day PS15 04-09 Timothy Day PS16 04-09 Timothy Day PS17 04-10 Timothy Day PS18 05-12 Timothy Day PS19 05-21 Timothy Day PS20 05-05 Marc Vef PS21 05-05 Marc Vef PS22 05-05 Marc Vef PS23 05-05 Marc Vef PS24 05-05 Marc Vef PS25 05-07 Marc Vef PS26 05-07 Marc Vef PS27 05-07 Marc Vef

PS28 05-11 Marc Vef PS29 05-11 Marc Vef PS30 05-11 Marc Vef PS31 05-13 Marc Vef PS32 05-18 Marc Vef

your previous vote: -1 on PS27 (2026-05-07 21:28) — now at PS32

threads: 3 need your reply · 2 waiting on others · 0 own notes · 0 bot

reply PS32 2026-05-29 17:08 Timothy Day COMMIT_MSG:36 — This is false.

Timothy Day PS32 · 2026-05-29 17:08

This is false.

reply PS32 2026-05-29 17:08 Timothy Day COMMIT_MSG:60 — These modules do not compile on newer kernels (6.18, at least). I think it should be possible to build without these.

Timothy Day PS32 · 2026-05-29 17:08

These modules do not compile on newer kernels (6.18, at least). I think it should be possible to build without these.

reply PS32 2026-05-29 21:34 Oleg Drokin kcompr.c:86 — just as the smatch implies this should really be IS_ERR, I don't think filp_open can return NULL?

Oleg Drokin PS32 · 2026-05-29 21:34

just as the smatch implies this should really be IS_ERR, I don't think filp_open can return NULL?

tags: csdc

owner Artem Blagodarenko · uploader Marc Vef · PS32 uploaded 168d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lustre: add lz4 and lz4hc kernel modules

lz4 and lz4hc kernel modules implement compression according to the
lz4 and lz4hc algorithms respectively, through the kernel Crypto API.
lz4 module provides 2 cipher drivers under the generic name 'lz4':
* lz4-lustre-generic of type compression
* lz4-lustre-scomp of type scomp
lz4hc module provides 2 cipher drivers under the generic name 'lz4hc':
* lz4hc-lustre-generic of type compression
* lz4hc-lustre-scomp of type scomp

lz4 and lz4hc kernel module sources are copied from linux v6.1-rc5,
and renamed to llz4.c and llz4hc.c respectively to avoid name
collisions. Use of vmalloc has been changed to kvmalloc since it is
faster in most cases. They implement the Crypto API interface, and
rely on the lz4/lz4hc kernel library for compression implementation.
They have been modified to grok a compression acceleration/level, as
read from the top 4 bits of the crypto_tfm flags, and pass it to the
underlying library.
The lz4/lz4hc library sources are also copied from linux v6.1-rc5 and
built statically, so lz4_compress, lz4_decompress and lz4hc_compress
sources have been "de-modulified", and EXPORT_SYMBOLs removed. Headers
have also been copied from linux v6.1-rc5 for consistency, and source
files modified to include the copied headers instead of the system
headers.

All aforementioned sources are located in the lustre_compat/crypto/lz4
directory.

The lz4/lz4hc modules are built only if the kernel does not provide
them. This is verified by checking if the CONFIG_CRYPTO_LZ4 and
CONFIG_CRYPTO_LZ4HC kernel config options are defined.

This patch provides unit testing of several compression modules, such
as lz4, lz4hc, lzo and gzip. This is done via a new test kernel module
kcompr.ko, and new sanity test_84. File lustre/tests/kernel/kcompr.c
contains examples of how to call the compression/decompression
routines.

EX-7998 lustre: Fix build with crypto_tfm_ctx on kernel 6.2

In recent kernels, commit e634ac4
"crypto: api - Add crypto_tfm_ctx_dma"
moved crypto_tfm_ctx into algapi.h. So this file must be included
from lustre_crypto.h.
Was-Change-Id: If10f721355da3e58ca541c17615e978334e8d718

EX-7683 utils: always try to use our own lz4/lz4hc

lz4/lz4hc provided by the kernel do not grok a compression level.
The built-in lz4/lz4hc do, so always build them as dedicated kernel
modules llz4.ko and llz4hc.ko, with the same .cra_name but with a
slightly higher .cra_priority = 110, so that they are preferred over
the in-kernel modules if any.

And try to manually load the llz4/llz4hc kernel modules when a file
requires compression with the corresponding alg. This is a "one-shot"
try that allows us to prefer our modules that has level support, but
continues to at least compress/decompress files even if our own
modules are not available.
Was-Change-Id: I0bdf267f998e21df81e460250a653aed34e3215d

EX-bug-id: EX-6275
Test-Parameters: testlist=sanity env=ONLY=84
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I74ce95ff18194c6052d291588d7e8c79596a7f23

68391 master J !B ✗T –R 0/2

LU-10026 csdc: migrate to another compression level

mine build failure (#130363)

csdc 2 unresolved M +70 −0

47d

build failure (#130363)

1 unresolved thread(s) await your reply — last from Patrick Farrell

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130363 FAILURE

no vote yet: Sergey Cheremencev, Artem Blagodarenko, Alex Zhuravlev

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

Version gate converted to compression_supported.

reply PS1 2026-09-15 18:51 Patrick Farrell sanity-compr.sh:2033 — **[AI review]**

Patrick Farrell PS1 · 2026-09-15 18:51

**[AI review]**

test_1030 reads $tmp_hdf and $hdf, but neither is set here. Every other test that uses them declares them as locals in its own function (test_1007 at sanity-compr.sh:552-554, and five more sites), so they are empty by the time test_1030 runs.

So [[ ! -f $tmp_hdf ]] is always true, [[ -f $hdf.bz2 ]] tests the literal path ".bz2" and is always false, and the test always takes the else branch. test_1030 never executes, and the skip reason it prints is misleading.

Seen on a full sanity-compr run of this series on Rocky 9, where bunzip2, bzcat and lustre/tests/AMSR_E_L3_DailyOcean_V05_20111003.hdf.bz2 were all present, and 1007 and 1008 -- which do declare the locals -- had already run and passed:

  SKIP: sanity-compr test_1030 bunzip2 is not installed

Declaring the same locals here as the other tests do should be enough:

  local hdf=$LUSTRE/tests/AMSR_E_L3_DailyOcean_V05_20111003.hdf
  local tmp_hdf=$TMP/$tfile.hdf

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: migrate to another compression level

Here is a test there lfs migrate with changed
compression level.

tests: fix sanity-compr 1030

Set compress_check_bytes to 2 times larger
than uncompressed file size. Without that
file might be considered incompressible.

Add extra debugging to help understand test failures.

DDN-bug-id: EX-10512
Was-Change-Id: I559f94b24bd2abb821a07775198bb4e511793a72

tests: change algo in 1030

to compare apples with apples. skip (i.e. do no fail) if
less data were compressed in the 2nd approach as this may
happen due to external conditions we don't control well
(grants, etc).
take into account that higher compression level in lz4fast
means less efforts and larger result.

DDN-bug-id: EX-10512
Was-Change-Id: Ia4c655ec99c1b45ff9c273311e5d567b8201cd5a

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9889
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia67316a247fed99f6ad002c8b67053a3c2fa0cff

68411 master J !B ✗T –R 0/2

LU-10026 csdc: parameter to disable compression on HDD OSTs

mine build failure (#130383)

csdc 2 unresolved L +471 −11

48d

build failure (#130383)

1 unresolved thread(s) await your reply — last from Patrick Farrell

janitor run: 40 failing config(s), none unique to this patch — janitor results

build #130383 FAILURE

no vote yet: Yang Sheng, Sergey Cheremencev, Artem Blagodarenko

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- In lod_ost_alloc_rr the -EBADR early-out jumps to this tree's out_pool label (the original's 'out' label; identical cleanup).
- The HDD check in lod_ost_alloc_qos sits after the additional allocation filters this tree has in that loop.
- The sysfs attribute is inserted in the alphabetically sorted attr table (the original appended).
- The three sanity-compr 2020* version gates converted to compression_supported; the statistics-init helper keeps the unconditional obdfilter access.
- Subject shortened to the 62-column limit; the tunable paragraph in the message and man page rewrapped.

reply PS1 2026-09-14 22:19 Patrick Farrell sanity-compr.sh:3230 — test_2020a needs at least two OSTs and has no guard for it: it marks OST0001 as SSD with 'do_facet ost2' below, waits on an OST0001 line fro

Patrick Farrell PS1 · 2026-09-14 22:19

test_2020a needs at least two OSTs and has no guard for it: it marks OST0001 as SSD with 'do_facet ost2' below, waits on an OST0001 line from 'lfs df -v', and calls subtest_2020a with '-i 1'. On a single-OST setup it fails rather than skipping.

Adding the existing idiom next to the compression_supported check would cover it:

	(( OSTCOUNT >= 2 )) || skip_env "needs >= 2 OSTs"

Confirmed by running it both ways: it fails with one OST and passes with two. test_2020b and test_2020c only use osts_nodes, so they are fine as they are.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: parameter to disable compression on HDD OSTs

The current CSDC compression does not perform well with HDD OST
storage due to fragmentation of the block allocation for a
compressed file. This is acceptable for NVMe OSTs because they
do not have seek latency, but can cause significant performance
impact for HDD OSTs, both for the compressed objects, as well as
uncompressed objects that are being written concurrently due to
IOPS starvation and fragmented free space.

This patch optimizes compression for systems with HDD OSTs.
By default, compression is not supported on HDD OSTs. Such an OST
is skipped and the next SSD-based OSD is chosen. This behaviour can
be adjusted using lod.*.enable_compr_rotational. Allowed values are:
 * -1 - return -EBADR error when trying to allocate compressed files
        on HDD OSTs
 * 1  - allow compression on HDD OSTs
 * 0  - skip an HDD OST

tests: fix sanity-compr/2020 for one-node setup

The check in sanity-compr test_2020() if an OST was flash or HDD
was unreliable if the MDT was also on a flash device, or if there
were other statfs flags on the target devices.

Improve this check to only look for the 'f' (flash/nonrotational)
status on the specific OSTs being used for the test.

DDN-bug-id: EX-10989
Was-Change-Id: If8ec5e6ce02ce4e5eee4ffb00bc71cda577d98e0

lod: dereference the device after check

Ensure the index is valid before dereferencing it.

DDN-bug-id: EX-10989
Was-Change-Id: Iefab5ae2036c09819016b2e569c870ba36174e38

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10989
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibc485aaed6fe719591f64e55b1203ddd2e646f09

68796 master J ✗1B ✗T –R 0/2

LU-19794 csdc: use acomp compression API

mine build failure (#131552)

1 unique failure csdc 2 unresolved M +199 −8

49d

build failure (#131552)

janitor: 1 test failure unique to this patch — sanity-quota@ldiskfs+DNE:test_1k

1 unresolved thread(s) await your reply — last from Nikos Papakonstantinou

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-quota@ldiskfs+DNE:test_1k	seen in 11 other reviews

build #131552 FAILURE

no vote yet: Nikos Papakonstantinou

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

reply PS1 2026-09-14 09:36 Nikos Papakonstantinou COMMIT_MSG:1 — I am not aware of this tbh. I tried to build this but I didn't got this.

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-09-10 20:19

ldlm/ldlm_lib.c:
 warn: client_connect_import():duplicate check 'rc' (previous on line 745)

Nikos Papakonstantinou PS1 · 2026-09-14 09:36

I am not aware of this tbh. I tried to build this but I didn't got this.
I am using a Ubuntu 25.10 vm with 6.17.0-14 kernel.
Of course I am building it with --disable-server option.

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- Only the series files are ported here (lustre_compr.h/.c, lustre_crypto.h, ofd_compress.c, osc_compress.c, configure probe); the hunks to the lz4/gzip/zstd modules, kcompr kunit and compr_private/scompress.h belong to the standalone module changes (57776/62408/66001) and are left for new patchsets there, together with the LC_HAVE_CRYPTO_STREAM_IN_SCOMP probe they need.
- The HAVE_CRYPTO_REQUEST_SET_DMA probe sits under the 6.15 section (kernel commit b67a0260037, v6.15-rc1).
- The acomp branch of ofd_decompress_chunk() was completed: the request/wait variables are declared, and a failed request allocation sets -ENOMEM and falls through to the existing error handling (the original referenced undeclared variables and a missing label there). acomp_request_alloc() returns NULL on failure, so the allocation checks test for NULL instead of IS_ERR().
- The request buffers are set with the non-DMA variants of the acomp setters (acomp_request_set_src/dst_nondma): every buffer handed in is a vm_map_ram() mapping or a large pool allocation, which the DMA variants' contract does not allow (inert for scomp-backed algorithms, wrong for a hardware acomp provider).
- ll_crypto_comp_set_level() is compiled out when the legacy comp API is absent; the acomp variant is unconditional on this branch.
- Caveat: no build available to the porter compiles the acomp side (el8 4.18; the janitor uses rocky 8/9); verified by reading against the v6.15 headers. Reviewers with a 6.15+ kernel, please build it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-19794 csdc: use acomp compression API

For kernels 6.15 and beyond use the acomp compression API with DMA
source and destination virtual addresses, instead of the legacy comp
API which was removed there.

The acomp request allocation returns NULL on failure, so the
allocation checks test for NULL and fail with -ENOMEM. The buffers
handed to the request are vm_map_ram() mappings or large pool
allocations, which are not usable for DMA, so they are set with the
non-DMA variants of the request setters.

Test-Parameters: forjanitoronly
Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I1fa60a7a933e6cdd2baa525d0520e7372def29df

68799 master J ✗1B ✗T –R +1/2

LU-10026 ofd: fix csdc optimizations

mine build failure (#131555)

1 unique failure csdc 1 unresolved S +17 −4

49d

build failure (#131555)

janitor: 1 test failure unique to this patch — sanity-slow@ldiskfs+DNE:test_255c

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-slow@ldiskfs+DNE:test_255c	seen in 5 other reviews

build #131555 FAILURE

reviews: Alex Zhuravlev +1 2026-09-14 08:25

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The fix Alex Zhuravlev asked to land together with 68366/68378; `llch.` member access follows the stack-copy form of the header introduced by the checksum change.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: fix csdc optimizations

we shouldn't ignore pre-existing chunks doing RMW if new data
don't cover full range (i.e. have gap(s))

Test-Parameters: forjanitoronly
DDN-bug-id: EX-14659
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I08e7d3bf9a263ed31ed5ea82a4f0800eeb0529c7

68786 master J !B ✗T –R 0/2

LU-10026 tests: move verify_comp_attr to test-framework

mine build failure (#131542)

csdc 1 unresolved M +79 −74

51d

janitor run: 9 failing config(s), none unique to this patch — janitor results

build #131542 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

Preparatory, new on master: verify_mirror_count()/verify_comp_attr() move from sanity-flr.sh to test-framework.sh (sanity-pfl needs them, including the already-present test_100k) and verify_comp_attr() learns compr-type/compr-level/compr-chunk/pattern/mirror-id. The string comparison is kept (the internal variant compares arithmetically, which is always true for non-numeric attributes).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: move verify_comp_attr to test-framework

Move verify_mirror_count() and verify_comp_attr() from sanity-flr.sh
to test-framework.sh so that other suites can use them, and teach
verify_comp_attr() the compression attributes (compr-type,
compr-level, compr-chunk) as well as pattern and mirror-id.

sanity-pfl test_100k already uses verify_comp_attr(), and the
following compression mirror tests need it too.

Test-Parameters: forjanitoronly
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0a679215a0848e5d348c4175b09d39fb125c7607

68792 master J !B ✗T –R 0/2

LU-10026 tests: FID retrieval from getstripe

mine build failure (#131548)

csdc 1 unresolved S +33 −6

51d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #131548 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The sanityn.sh hunk (a trash-can test) does not apply on this branch and was dropped; the rest as the original.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: FID retrieval from getstripe

Some tests rely on the getstripe output to obtain
an object's FID. However, for compressed files, the
getstripe output format differs significantly.

This patch introduces a function that handles both
output formats. All functions potentially affected
by this change have been updated accordingly.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8258
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia9ce932a023066c78aa819bdf502191c5ecc2044

68790 master J !B ✗T –R 0/2

LU-10026 csdc: compress_request memory optimisation

mine build failure (#131546)

csdc 1 unresolved M +52 −55

51d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #131546 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- merge_chunk() fills the page array with brw_folio_page() (this branch's brw_page carries a folio + page index instead of a page); otherwise as the original.
- Notes for reviewers (inherited, unchanged): the vm_map_ram() NULL path in compress_request() returns -ENOMEM without releasing the pool destination buffer; decompress_request() leaves the mapping in place on two error paths after it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: compress_request memory optimisation

Map brw pages with vm_map_ram instead of copying them to a
buffer allocated by sptlrpc_pool_get_pages. This should save
memory and some CPU cycles.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10553
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3a9280a70a7ac25e29b4e68f0810fe89a5ca69fc

68791 master J !B ✗T –R 0/2

LU-10026 lov: disable compression for DoM

mine build failure (#131547)

csdc 1 unresolved S +45 −0

51d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #131547 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

test_2003 relocated after test_2002 (numeric order); its version gate converted. Its fix "distinguish the -Z none parameter for DoM" follows three changes later (source landing order kept).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: disable compression for DoM

Disallow enabling compression on the DoM component until
the related feature is properly fixed.

Added a test to ensure this combination is prohibited,
while confirming that DoM continues to function normally.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8718
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I338b375ab386826d08b5610105b14d7496e33aa7

68794 master J ✗2B ✗T –R 0/2

LU-10026 csdc: distinguish the -Z none parameter for DoM

mine build failure (#131550)

2 unique failures csdc XS +3 −2

51d

build failure (#131550)

janitor: 2 test failures unique to this patch — recovery-small@ldiskfs+DNE:test_115b, sanity-slow@zfs:test_255c

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
recovery-small@ldiskfs+DNE:test_115b	seen in 7 other reviews
sanity-slow@zfs:test_255c	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #131550 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: distinguish the -Z none parameter for DoM

When "-Z none" is passed, LOV_PATTERN_COMPRESS is set
with LL_COMPR_TYPE_SPEC_NONE, this is how this case
can be distinguished for DoM with compression
prevention.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-13161
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0718af523efe2952b844e2d474acc1c02493303a

66001 master J !B ✗T –R 0/2

LU-10026 lustre: add lzstd kernel module

mine build failure

csdc 34 unresolved XL +33132 −11

51d

build failure

34 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 4 failing config(s), none unique to this patch — janitor results

no vote yet: Andreas Dilger, Sebastien Buisson, Artem Blagodarenko, Oleg Drokin

patchsets: PS1 05-19 PS2 05-19 PS3 09-10 Sebastien Buisson PS4 09-10 Sebastien Buisson

threads: 34 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) debug.c:24 — warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) debug.c:24 — warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) huf_compress.c:460 — warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) huf_compress.c:460 — warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_compress.c:4684 — warn: ZSTD_initCDict_internal():inconsistent indenting

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_initCDict_internal():inconsistent indenting

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) zstd_compress.c:4684 — warn: ZSTD_initCDict_internal():inconsistent indenting

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warn: ZSTD_initCDict_internal():inconsistent indenting

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1577 — warn: ZSTD_compressBlock_lazy_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1605 — parse ZSTD_compressBlock_lazy_generic():error: Function too hairy. Giving up. 3 seconds

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

parse ZSTD_compressBlock_lazy_generic():error: Function too hairy.  Giving up. 3 seconds
warn: ZSTD_compressBlock_lazy_generic():Function too hairy.  No more merges.

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1915 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1937 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1947 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1979 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

tags: csdc

owner Marc Vef · uploader Sebastien Buisson · PS4 uploaded 53d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lustre: add lzstd kernel module

lzstd kernel module implements compression according to the zstd
algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'zstd':
* zstd-lustre-generic of type compression
* zstd-lustre-scomp of type scomp
And 2 cipher drivers under the generic name 'zstdfast':
* zstdfast-lustre-generic of type compression
* zstdfast-lustre-scomp of type scomp

Note the 'zstd' name is identical to the in-kernel module, but lzstd
registers it with a slightly higher .cra_priority = 110, so that it is
preferred over the in-kernel module.
Our 'zstd' is also different in that it accepts a compression level
as explained below. 'zstdfast' is handling the 'negative' compression
levels, interpreted as an acceleration factor.

lzstd kernel module sources are copied from linux v6.9. This module
implements the Crypto API interface, and rely on the zstd kernel
library for compression implementation. It has been modified to grok
a compression level, as read from the top 4 bits of the crypto_tfm
flags, and pass it to the underlying library.
The zstd library sources are also copied from linux v6.9 and built
statically. Headers have also been copied from linux v6.9 for
consistency, and source files modified to include the copied headers
instead of the system headers.
In order to maximize compatibility with various distributions, we copy
xxhash sources from linux v6.9, and link these statically.
All aforementioned sources are located in the
lustre_compat/crypto/zstd directory.
The lzstd module is always built with Lustre.

This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'zstd' module.
It also tries to manually load the lzstd kernel module when a file
requires compression with the 'zstd' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.

EX-9678 csdc: defer memory allocation in zstd/zstdfast

Zstandard normally proceeds to memory allocation for its context
(workspace) while in the init phase, which corresponds to a call to
crypto_alloc_comp() when used via the Crypto API.
At this stage we cannot be aware of the compression level or the chunk
size. So we have to make zstd allocate a workspace that can fit any
level/chunk. And the higher the level is, the larger the workspace
needs to be. Problem is zstd is very memory eager, and by default for
the higher compression level it requires a workspace of 650 MiB. This
is insane compared to our usual compression chunk sizes.

So we defer workspace initialization to the first time zstd actually
compresses data. This corresponds to a call to crypto_comp_compress()
when used via the Crypto API. And by that time we know the compression
level and chunk size, which enables us to make zstd allocate a finer
grained workspace, that just fits our actual needs. This saves a lot
on memory allocation, as can be seen from these values (workspace size
expressed in bytes):

                      no src hint   64kB src
   level 1 cwksp size      582392    303864
   level 2 cwksp size      779000    402168
   level 3 cwksp size     1303288    664312
   level 4 cwksp size     2614008   1057528
   level 5 cwksp size     3662584   1057528
   level 6 cwksp size     3662584   1057528
   level 7 cwksp size     6808312   1057528
   level 8 cwksp size     6808312   1057528
   level 9 cwksp size    13099768   1057528
   level 11 cwksp size   25682680   1319672
   level 13 cwksp size   34071288   1468856
   level 15 cwksp size   67625720   1791095
   level 17 cwksp size   50997688   1791095
   level 19 cwksp size   85196598   1791095
   level 21 cwksp size  336854838   1791095
   level 22 cwksp size  680943414   1791095

And of course because we reduce contention on memory a lot, we improve
performance when zstd is used via the Crypto API, which is done in
osc_brw_prep_request() for ever RPC that involves a file compressed
with zstd.
Was-Change-Id: I470331ea0f2e550761ce7223e7be7e1373701bd2

Was-Change-Id: Idf7382093feec21bed5b6e62001a085a386736d2
Was-Change-Id: Ifa9d38d807d0ab55f85f6c6c8672c340778c2855

EX-bug-id: EX-9678
Test-Parameters: testlist=sanity env=ONLY=84
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I113345cd10f41197683bfe127d373b8628364f88

68798 master J !B ✗T –R 0/2

LU-10026 osc: do not expand if overflow compression

mine build failure (#131554)

csdc S +23 −0

51d

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #131554 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: do not expand if overflow compression

The extent expansion function osc_extent_expand() can
expand extents beyond the compression chunk limit, completely
bypassing the limit enforced in osc_extent_find() and
try_to_add_extent_for_io().

Let's add the check to the osc_extent_expand() to
make this situation impossible.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12849
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id0240c2078840a250d169a946e1894f4375fd0b9

68795 master J !B ✗T –R 0/2

LU-10026 utils: add compression type completion

mine build failure (#131551)

csdc 1 unresolved M +59 −0

52d

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #131551 FAILURE

no vote yet: Chakshu Kansal

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

Applied cleanly; test_440b's compression_enabled gate replaced by compression_supported (the test only reads osc.*.compress_types).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: add compression type completion

Add bash completion support for compression types in lfs setstripe
command. Also adds a test case to verify the functionality works
correctly.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12299
Signed-off-by: Chakshu Kansal <ckansal@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id19e27a3088023dca4dda03b19054e5fdf0a4385

68787 master J !B ✗T –R 0/2

LU-10026 lov: pack layout_gen and compr info into lcme

mine build failure (#131543)

csdc 1 unresolved M +122 −0

52d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #131543 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- Field renames to *_chunk_lum_bits; the two sanity-pfl gates converted to compression_supported; subject shortened.
- test_100n expects the pattern "raid0,compress" for the compressed mirror (the original compared "compress" arithmetically, which always passed; the string comparison used here needs the full pattern name, as in test_100k). Depends on the preceding helper commit (verify_comp_attr in test-framework.sh with the compression attributes).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: pack layout_gen and compr info into lcme

Creating a compressed mirror with "lfs mirror -N" did not produce a
compressed component. Even when the compression parameters were fully
specified, the new mirror was created without compression, and lfs
getstripe showed the mirror entry lacking compression fields.

lov_lsm_pack() did not populate the composite entry fields
lcme_layout_gen and lcme_compr_* when packing a lov_comp_md_v1. As a
result the MDS/client logic took the "no layout/compression in entry"
path and defaulted to an uncompressed layout for the new mirror.
llapi_layout_mirror_inherit() should set compression fields.

Set these fields for each composite entry and set during layout mirror
inherit to fix the problem.

sanity-pfl 100n and 100m were added.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12672
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I5bfcd2d942e9eeb853674763f8ea14d97e4a966c

68788 master J ✓B ✗T –R 0/2

LU-10026 tests: compression component via --component-add

mine build failure (#131544)

csdc 1 unresolved S +15 −0

52d

build #131544 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The lod fix this test was written for is already on master (90c093b7850, LU-17468); only the sanity-compr test_1a part was unlanded - ported under a new Change-Id with the original as Was-Change-Id.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: compression component via --component-add

Add sanity-compr test_1a: create a file with a plain first component,
append a compressed component with "lfs setstripe --component-add
-Z gzip" and verify the added component carries the compression
type. This covers the compression branch of the component-add path
in lod_declare_layout_add().

The lod fix this test was written for is already on master as
commit 90c093b7850 ("LU-17468 lod: component add missed pattern
info"); only the test part is new here.

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I7ad746a550f1afea54a6f5b68823a79a85a44082
Change-Id: I0624bf3f5f9c965f826e72656a8cd8198c5624fd

68793 master J –B ✗T –R 0/2

LU-10026 tests: enable CSDC layout in sanity-compr.sh

mine build failure (#131549)

csdc 1 unresolved M +146 −56

52d

build #131549 FAILURE

no vote yet: Sarah Liu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- This branch's sanity-compr.sh header already used the compr_STRIPEPARAMS/setstripe_getstripe form; the default layout now carries compression ("-E EOF -Z lz4:3") and the layout is passed to sanity.sh/sanityn.sh via fs_STRIPEPARAMS with ALWAYS_EXCEPT/fs_STRIPEPARAMS saved and restored around the runs. Note: master landed 6602cc38ea ("LU-16904 tests: allow sanity to run with PFL layout", per-suite <suite>_STRIPEPARAMS) after this series' base - the next rebase needs to re-plumb to that mechanism.
- The compression-specific exceptions are conditional on a "-Z" layout; tags renumbered to LU-10026. The list gained 119r (this branch's test_119r has no compression skip; unaligned DIO) and lost 360 (no such test on this branch). `always_except LU-16904 906` (added earlier in this series) is dropped as in the original.
- sanity.sh test_127a: the original's guard referenced a variable local to another test and would have disabled the sum check for every layout; the port guards on COMPRESSED (set together with the compression exceptions).
- sanityn.sh test_93: the original's awk printed a fixed field of the composite getstripe output (never the OST index), so the duplicate check was vacuous under compression and lost for plain layouts; the port extracts the OST index from both output formats.
- test_460a already had no enable/disable_compression on this branch; 460a-460g gained the compression_enabled skip as in the original.
- The sanityn.sh test_117u hunk (trash-can test) does not apply here. The original also flipped sanity.sh to mode 0644; the port keeps it executable. cleanup_80b: the added directory removal follows this branch's `wait $migrate_pid`.
- The original's iozone always_except is not carried: a later change in the source landing order removes it again.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: enable CSDC layout in sanity-compr.sh

Enable CSDC layout on root mount in sanity-compr.sh and run the
sanity and sanityn suites with that layout. Subtests that do not
work with a compressed root layout yet are excepted when the layout
carries a compression component, and a few tests are adjusted to
either disable compression for the file under test or to account
for compressed data (block usage, statistics, FID parsing).

sanity test_127a skips its sum check only when the layout is
compressed, and sanityn test_93 reads the OST index from both the
plain and the composite getstripe output.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8258
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2e0e0c3e34d398156a56128925f02336d59ecfca

68372 master J –B ✗T –R -1/2−

LU-10026 tests: add sanity-compr test for compression

mine build failure (#130385)

csdc 1 unresolved L +309 −9

52d

build #130385 FAILURE

no vote yet: Artem Blagodarenko, Alex Zhuravlev

patchsets: PS1 08-26 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- ll_compression_scan moved out of the server-only install block (bin_SCRIPTS under UTILS), matching the patch intent against master's restructured scripts/Makefile.am.
- The report_error() changes were already on master; only error_ignore_no_dump() is new.
- The ubuntu always_except for test 1008 is removed as in the original (the test source dir change makes it unnecessary).
- test_1007/test_1008 gates converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: add sanity-compr test for compression

This patch adds a sanity-compr test to validate that
we get space usage reduction with compression.
The test uses ll_compression_scan tool to calculate
the compressed size of the source file and compares
it with the size of the Lustre CSDC compressed file.

tests: add sanity-compr test for dir compression

This patch adds a sanity-compr test to validate that
we get directory space usage reduction with compression.

DDN-bug-id: EX-7795
Was-Change-Id: I16f3a3f1e413e4884b3973829df36500667271ce

tests: skip sanity-compr 1007 and 1008

if needed tools are not installed

DDN-bug-id: EX-9871
Was-Change-Id: I0dc3d44c300708f3a25bfce06b81993cdd30c418

tests: change source dir for sanity-compr/1008

While the source directory contains almost all small files,
sanity-compr test_1008 will hit the
"failed estimates > 50% of total estimates" failure on SLES15.

The patch fixes the issue by changing the source dir to
contain large files.

DDN-bug-id: EX-9125
Was-Change-Id: Iad661750cba7c9d2204f2306e73169deb012ddf4

utils: fix ll_compression_scan overflow

Fix the total size estimate calculation to avoid overflow for
large total sizes.  Otherwise, the intermediate register used
for the calculation will overflow and produce an invalid estimte.

DDN-bug-id: EX-9125
Was-Change-Id: I12099cea1268abede17c92e0422b6c1c25ac66d5

tests: sanity-compr/1008 CentOS 7.9 more margin

For sanity-compr test_1008 allow a higher margin of error for
CentOS 7.9 clients because they consistently show slightly
worse compression ratios and fail the test.  It isn't clear
if this is because of worse/partial RPC formation or because
the compression code in this kernel is less optimal.

These tests should still be run to ensure the code is working, but
not fail so much, because fixing these old clients is not critical.

DDN-bug-id: EX-9125
Was-Change-Id: I6c1c6044daf5d0706cfb661b4f0f051769174ff9

tests: get stats_compr param in sanity-compr/1007

This patch adds getting stats_compr param in sanity-compr
test 1007 and 1008 to track data compression statistics.

DDN-bug-id: EX-9125
Was-Change-Id: I7df25d36c8c89ec5a568bf1e8d5694a97dd18ecc

tests: ignore sanity-compr/1008 failures

Temporarily ignore test failures for sanity-compr.sh test_1008 on
SLES15 and aarch64/ppc64le due to very high failure rates on those
systems.

Rename the *second* test_1008 to test_1080 so it can run, and allow
other new tests to start using disjoint numbers to avoid conflicts.

DDN-bug-id: EX-9125
Was-Change-Id: I199613b1e5b1ee8ea7ca4287a6dfe090257ed72f
Was-Change-Id: I3611d1a8b41267f5c38b8088e06eb8f0cad9dd48

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7795
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icf763331205a3e937b794f90444f756fc59f9050

68800 master J ✓B ✗T –R 0/2

LU-10026 tests: skip sanity-compr/1010 on EL10 clients

mine build failure (#131556)

csdc 1 unresolved XS +6 −0

52d

build #131556 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip sanity-compr/1010 on EL10 clients

sanity-compr test_1010 ("still can read the broken chunk?") fails when
clients run Rocky Linux 10.x. Skip the test until a fix is available
as it blocks testing.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12865
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id84d8bc4922f8fd37b9e42ae64234a0593d79f7f

68785 master J –B ✗T –R -1/2−

LU-10026 csdc: add compression header checksum

mine build failure (#131541)

csdc 1 unresolved L +241 −48

52d

build #131541 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- This is series entry 81 (ex-8688), which the original port skipped by an off-by-one in the entry numbering; ported now on top of the series. sanity-compr test_1080 (dropped from "tests: more tests for csdc" for that reason) returns with it.
- The header checksum is gated on the LLCH_FL_HDR_CSUM flag that this branch's ll_compr_hdr reserves for it (set by compress_chunk(), checked by is_chunk_start()) instead of the original's "csum != 0" heuristic. The pre-built sample lustre/tests/compressed.bin has the flag set and its header checksum recomputed accordingly; the checksum constant in test_1080's corrupted-size case was recomputed the same way (algorithm unchanged: crc32 over the header with the checksum field zeroed). test_1080's last case clears the flag (instead of zeroing the checksum) so that the no-checksum path is what gets exercised.
- is_chunk_start() takes the folio and gained the int *rc out-param; the four ofd call sites and the osc call site adapted. The ofd functions keep a stack copy of the header as in the original.
- The header-size sanity limit uses sizeof(*header) (the original compared against the size of a pointer).
- The decompress_rnb() prototype hunk does not apply (that helper was removed earlier in the series).
- The forced-decompression fail-loc (OBD_FAIL_OSC_FORCE_DECOMPR) keeps 0x420, which is free on this branch; the forced type is LL_COMPR_TYPE_LZ4.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: add compression header checksum

This commit adds functionality to verify and calculate the checksum
of the header in the `ll_compr_hdr` struct and some another sanity
check.

The client sets LLCH_FL_HDR_CSUM in llch_flags when it stores the
header checksum, and the checksum is only verified when that flag
is set. A chunk with a valid magic and checksum but inconsistent
header fields is treated as corrupted and the read fails with
-EUCLEAN; without a valid checksum such a chunk is passed through
as plain data.

Add sanity-compr test_1080, which exercises these checks with a
pre-built compressed sample (lustre/tests/compressed.bin).

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8688
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I24a9ab9cb7bea1208ada23aa6550127fe6a55017

68789 master J !B ✗T –R 0/2

LU-10026 osc: prevent oversized extents for compressed objects

mine build failure (#131545)

csdc 1 unresolved M +165 −1

52d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #131545 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- test_2030 anchored after test_2020c. The original flipped the mode of sanity-compr.sh to 0644; the port keeps it executable.
- Note for reviewers: `if (( -n "$brw_check" ))` in test_2030 is a bash arithmetic expression, not a string test (inherited).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: prevent oversized extents for compressed objects

When compressed objects have high compress_rpc_fragment_ratio values,
large extents could contain too many compression chunks to fit in a
single RPC, causing try_to_add_extent_for_io() to crash with
LASSERT(data->erd_page_count > 0) when the oversized extent was the
first one being considered.

This client-side bug also caused server-side buffer overflows where
the client would send thousands of compression chunks in a single
RPC, exceeding the server's PTLRPC_MAX_BRW_PAGES limit and
triggering 'too large request' errors.

Instead of splitting extents after they are created, this fix
prevents the problem at the source by limiting extent size during
extent creation in osc_extent_find(). For compressed objects, the
maximum extent size is now calculated based on compression chunk
limits to ensure extents never exceed the RPC fragment ratio.

This proactive approach prevents both client crashes and server
buffer overflows while maintaining optimal performance by avoiding
the overhead of extent splitting operations.

tests: improve sanity-compr test_2030

Some improvements:

- Use save_lustre_params/restore_lustre_params for brw_size params
- Use save_lustre_params/restore_lustre_params for client OSC params
- Add comprehensive parameter validation to ensure settings applied
- Improve brw_size verification to check all OSTs not just first
- Replace cat > /dev/null with MD5 checksum for data integrity

DDN-bug-id: EX-12849
Was-Change-Id: I3a8dfd393289f5be0c638e84e1097b18a4303240

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12849
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id161a17b4b45a6cad39ccd55e18da0f6d4f58957

68397 master J –B ✗T –R -1/2−

LU-10026 tests: more tests for csdc

mine build failure (#130369)

csdc 2 unresolved XL +1436 −534

52d

build #130369 FAILURE

no vote yet: Artem Blagodarenko, Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 2 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] correction to the earlier remark: test_1080 is not excluded because its prerequisite is outside the series - the header-checksum

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] correction to the earlier remark: test_1080 is not excluded because its prerequisite is outside the series - the header-checksum change IS series entry 81 and was skipped by an off-by-one in the original port. It is now ported as https://review.whamcloud.com/c/fs/lustre-release/+/68785 ("LU-10026 csdc: add compression header checksum"), which brings test_1080 back together with the compressed.bin sample and the fail-loc it needs.

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- This patch reorganizes sanity-compr.sh wholesale (renumbers 1000->1000a, fsx->1000b, adds 1000c, adds tests 1009a-1016, moves tests into numeric order, adds iozone/dbench/fsx wrappers, file-scope compression gates and DIRECTIO/RAMKB/THREADS/SIZE setup). The port reconstructed the patch's intended post-state and re-applied this tree's conversions on top:
- internal version gates -> compression_supported (or dropped where they guard functionality this series itself provides);
- test_2000 keeps the master client version (2.17.53) for the OOM fix;
- OBD_FAIL_OST_BRW_WRITE_BULK3 uses 0x258 on this tree (0x256 is taken); OBD_FAIL_OST_CSDC_WRITE_BAD stays 0x257;
- the local enable/disable_compression definitions removed by this patch were already provided by test-framework.sh here;
- test_1123 keeps the compression_supported gate added when it was introduced;
- the local RAMKB in subtest_2000 gave way to the new file-scope RAMKB.
- test_1080 is excluded: it depends on the header-checksum change (Was-Change-Id: I24a9ab9cb7bea1208ada23aa6550127fe6a55017), which is not part of this series; it should return together with a port of that change.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: more tests for csdc

- synthetic tests to cover all simple cases like full/partial
  write/overwrite/append/read which verifies the operations
  doing same IO to tmpfs
- a test simulating a bulk error
  to ensure all bufs/pages are released
- few tests simulating read/write/truncate to corrupted chunks
- a test checking how much IO different cases need
- add own fsx with compression enforced
- add own dbench with compression enforced
- add own iozone with compression enforced (slow)

tests: cleanup sanity-compr.sh

Fix style issues in sanity-compr (whitespace, variables).
Add check_and_set_extents_dense() helper in a few places.
Re-enable test_1012 now that server-side code is fixed.

DDN-bug-id: EX-10107
Was-Change-Id: I7f5e5b759a8081efed4700d1c963afde135e690f

tests: improve sanity-compr/1000c

the existing test doesn't take into account that allocations can be
done in few racing threads, thus allocation is not serialized. thus
it can be non-sequential, but still dense.
the test should sort filefrag's output by physical location first,
then count gaps.

DDN-bug-id: EX-10763
Was-Change-Id: Ie3e1122e549da0a181f8cf5dbb0aa4304d1c3157

tests: Use less data in csdc DIO iozone test

"EX-6266 llite: DIO fallback on compressed files" forces
to switch any DIO operation to the buffered one. This behaviour
breaks any iozone expectations and the test goes too slowly.
Having such an iozone test with DIO and compression is a good
idea, but let's reduce the testing file size so this test pass
for reasonable amount of time.

DDN-bug-id: EX-10445
Was-Change-Id: I8dee83398a4e5249ad254d061bafa89240feb2e2

tests: Use less data in iozone test

"EX-10445 tests: Use less data in csdc DIO iozone test" sets to
use 1/10 of available space for DIO iozone test. The same chage
should be done for buffered iozone test too, because the test it
too slow for large disks and slow VMs.

DDN-bug-id: EX-10445
Was-Change-Id: I40ca9783826448885cb4d8d3ca889d94df26f257

csdc: execute tests if compression is enabled

"csdc: Provide finer grained enable_compression control" patch
enables CSDC by default on all archs except aarch64 and ppc64le.
No need to execute enable_compression/disable_compression for
each test, so this test changes them to compression_enabled
check that skips test if compression is disabled for the arch.

This patch forces to skip compression tests for ppc64le and
aarch64 so they stop adding noise to the test results.

DDN-bug-id: EX-8981
Was-Change-Id: I2b5deba0672f3bad028f383cf1205e66c2b05529

tests: disable compressibility checks

some tests want to mix compressible and incompressible data for
verification. EX-8355 can disable compression for a period when
incompressible data met. disable this functionality in few
tests.

DDN-bug-id: EX-10653
Was-Change-Id: I1da67af9802e92b29651b8b532ab21c3cc75dff2

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9858
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icbc13ee239a68d165b77481678665e80690564b2

68408 master J ✓B ✗T –R 0/2

LU-10026 tests: disable sanity-compr/1011

mine build failure (#130380)

csdc 1 unresolved XS +1 −0

53d

build failure (#130380)

1 unresolved thread(s) await your reply — last from Timothy Day

build #130380 FAILURE

no vote yet: Alex Zhuravlev

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-09-10 14:36 Timothy Day patchset comment — What's the significance of these 'upstream remarks for reference'? Differences between your in-house change vs. the version in Gerrit?

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

Remainder of a later snapshot of the test_2001 change (same original Change-Id): a single always_except for test_1011, tag renumbered to LU-10026. New Change-Id, original kept as Was-Change-Id.

Timothy Day PS1 · 2026-09-10 14:36

What's the significance of these 'upstream remarks for reference'? Differences between your in-house change vs. the version in Gerrit?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: disable sanity-compr/1011

Add sanity-compr test_1011 (grants let a compressed file fill
OST) to the always-except list; the failure is tracked
separately.

This is the remaining delta of a later snapshot of the change
that added sanity-compr test_2001.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10142
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I6dd2643c93739fd6abaeb7add27f530a57fdcf64
Change-Id: I9570a6f71145f4f75c2bb2a09f8387936c88b114

52166 master J ✓B ✓T ✓R -1/2−

LU-13814 osc: add osc_dio_page_submit

reviewing Patrick Farrell · PS34→PS36 since your -1 — your objection was dropped

needs rebase M +42 −16

59d

PS34→PS36 since your -1 — your objection was dropped

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked

failed enforced test	platform	detail
review-dne-part-3	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: conf-sanity.	session
review-dne-part-5	RHEL 8.10 / x86_64	ran 5 tests. 1 tests failed: sanityn.	session
review-dne-part-8	RHEL 9.5 / x86_64	ran 4 tests. 1 tests failed: replay-dual.	session
review-dne-zfs-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-zfs-part-2	RHEL 8.10 / x86_64	ran 11 tests. 1 tests failed: sanity-lnet.	session
review-dne-zfs-part-7	RHEL 8.10 / x86_64	ran 5 tests. 1 tests failed: sanity-scrub.	session
review-ldiskfs	RHEL 8.10 / x86_64, RHEL 9.5 / x86_64	ran 5 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64	ran 8 tests. 1 tests failed: sanity.	session
review-zfs	RHEL 8.10 / x86_64	ran 10 tests. 1 tests failed: replay-single.	session

8 optional test failure(s) — informational

build #119017 SUCCESS · tests all Maloo sessions

no vote yet: Qian Yingjin, Andreas Dilger, Zhenyu Xu

patchsets:

+31 earlier

PS1 08-29 PS2 08-30 PS3 08-30 PS4 08-30 PS5 08-31 PS6 09-05 PS7 09-09 PS8 09-10 PS9 09-10 PS10 09-10 PS11 09-10 PS12 09-10 PS13 09-10 PS14 09-12 PS15 09-14 PS16 09-15 PS17 09-24 PS18 09-25 PS19 09-26 PS20 09-26 PS21 10-05 PS22 10-05 PS23 10-20 PS24 10-22 PS25 10-22 PS26 02-27 PS27 10-28 PS28 11-03 PS29 04-28 PS30 04-28 PS31 05-01

PS32 07-04 PS33 07-14 PS34 11-06 PS35 11-09 PS36 11-16

your previous vote: -1 on PS34 (2025-11-07 08:23) — now at PS36

owner Patrick Farrell · uploader Patrick Farrell · PS36 uploaded 350d ago · NEW · open in Gerrit ↗

commit message

LU-13814 osc: add osc_dio_page_submit

This patch adds the osc_dio_page_submit and specializes
osc_page_submit to take only BIO pages.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I048bbcd25ffee28fbbd8fee26017796b5e3586bd

68385 master J !B ✗T –R 0/2

LU-10026 utils: lfs setstripe adds -E with -Z

mine build failure (#130357)

csdc 1 unresolved S +22 −5

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130357 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- test_1014 is anchored after the 1013x tests (numeric order; the original anchored it after test_1002, but this series adds 1013a-1013e in between).
- Version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: lfs setstripe adds -E with -Z

When specifying a layout with "lfs setstripe -Z" it will ignore
this option if no PFL component is specified with "-E".

Instead, "lfs setstripe -Z" should automatically upgrade the file
layout to a PFL layout so the compression parameters are saved.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9708
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I29cc373fabd352d6f8b6781c238806b75cce7057

68389 master J !B ✗T –R 0/2

LU-10026 csdc: inherit CSDC params unless overriden

mine build failure (#130361)

csdc 1 unresolved L +258 −25

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130361 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- Rewritten onto the current naming: LL_COMPR_TYPE_SPEC_NONE = 9 added (MAX moves to 10); wire asserts extended accordingly (this tree carries LL_COMPR_TYPE_MAX asserts the original did not have).
- compr_type_table: the NONE row loses its name, SPEC_NONE row ("none") added before the MAX sentinel - the compress_type_2str()/parse helpers were already in the matching form.
- The inheritance-hint struct uses cd_compr_chunk_lum_bits with this series' lum-bits semantics (offset above COMPR_CHUNK_MIN_BITS - no +16 at use sites), and lod_adjust_compr_chunk_size() is called in its take-by-value/return-by-value form.
- lod_qos_parse_config: the convert_plain arm skips all lcme-derived assignments including fields this tree has that the original context did not (llc_mirror_link_id, llc_dstripe_count, llc_cstripe_count).
- SPEC_NONE clearing blocks in lod_parse_striping/lsm_unpackmd applied at the current field naming.
- test_1000d version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: inherit CSDC params unless overriden

If a parent directory has been set CSDC parameters, the file created
under it needs to inherit the CSDC parameters unless another CSDC
parameters or an empty CDSC has been specified for a component of the
file. The file create method includes touch, setstripe a plain layout
file, or setstripe a composite layout file.

As "-Z none" would be used to set compression type to 0 and server
would treat it as no CSDC parameters has been set, so "-Z none"
can not be used to stop inherit parent directory's CSDC inheritance,
so this patch introduces another compression type
LL_COMPR_TYPE_SPEC_NONE to associate with "none" compresstion type,
and it can be used to stop the CSDC inheritance from parent
directories, when user uses "-Z none" to reset compression type,
this patch would set the compression type to LL_COMPR_TYPE_SPEC_NONE.

If we use patched new client to create a file with "-Z none" upon old
servers, the old servers cannot recognize this new
LL_COMPR_TYPE_SPEC_NONE compression type, and it just records the
type as is in the layout.

On old clients "lfs getstripe" will show "unknown" to the component's
lcme_compr_type, and on new clients, "lfs getstripe" will show "none"
to the lcme_compr_type.

When we write data to this file, both clients work but with different
behavior. On old client, with "unknown" compression type, it chooses
a fallback algorithm to compress the data, twisting the original
intention not to compress the component; while on new client, it will
treat "none" component as uncompressed one as intended.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10201
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibbdb3f6489cf7e19bfebe075bf88580f39ed79e4

68392 master J ✗1B ✗T –R 0/2

LU-10026 lov: overstriping + compression is OK

mine build failure (#130364)

1 unique failure csdc 1 unresolved XS +6 −0

66d

build failure (#130364)

janitor: 1 test failure unique to this patch — sanity-slow@zfs:test_64b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-slow@zfs:test_64b	seen in 2 other reviews

build #130364 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The lov_pattern_supported() hunk re-applied against the current helper (which has a RAID0|PARITY row and no MDT|COMPRESS row).
- lov_pattern_available() postdates this patch's original base and needed the same OVERSTRIPING|COMPRESS row added - without it the pattern is "supported" but never "available" and setstripe fails with -EINVAL.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: overstriping + compression is OK

it should be OK to enable compression on overstriped files.

lov: extend supported normal striping

where compression + overstriping is OK

DDN-bug-id: EX-10854
Was-Change-Id: I2474329d5b1d8684353efe8423f225e7ee779da3

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10854
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I45d14b3fc5badcb04652df2b64524c143b45c528

68394 master J !B ✗T –R 0/2

LU-10026 csdc: Added client parameters for compression types.

mine build failure (#130438)

csdc 1 unresolved L +243 −50

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130438 FAILURE

no vote yet: Chakshu Kansal

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The original's common target-side helpers are seq_file/procfs-based; on this branch mdt.*.compress_types and obdfilter.*.compress_types are sysfs attributes, so the common helpers were adapted to a mask-based sysfs form (lprocfs_compress_types_mask_show/store in lprocfs_status.c) and the mdt/ofd show/store delegate to them (renamed with mdt_/ofd_ prefixes to coexist with the new common client compress_types_show).
- The client-side compress_types_show (osc/mdc, reads ocd_compress_types, "disabled" without the connect flag) is as in the original.
- Tests 1013d/1013e anchored after test_1013c. The MDC-specific connect-flag check in test_1013e is kept as-is: it probes the MDC import, and compression_supported() probes the OSC import, so it is not a substitute (MDT-side compression negotiation stays disabled).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: Added client parameters for compression types.

Added new parameters 'osc.*.compress_types' and 'mdc.*.compress_types'
on the client side to report the list of available compression
algorithms.

DDN-bug-id: EX-12289
Was-Change-Id: Iba832e5351fc159e49916df2c06b75ad5ad37638

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12289
Signed-off-by: Chakshu Kansal <ckansal@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I07cd0531e9c2934c28dd999b21310d18a988e2f7

68406 master J !B ✗T –R 0/2

LU-10026 llite: skip layout changes if update is held

mine build failure (#130378)

csdc S +15 −0

66d

janitor run: 9 failing config(s), none unique to this patch — janitor results

build #130378 FAILURE

no vote yet: Alex Zhuravlev

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: skip layout changes if update is held

skip minor layout changes (like incompressability) when the client
helds UPDATE lock as MDT grabs UPDATE lock for this kind of changes.

llite: do not update layout from fsync

fsync can be called from within write. in this case there would be one
active IO (from LOV's point of view). if we update the layout at this
point, then fsync would need to refresh the layout to proceed, but
given there is an active IO, LOV would block layout refresh waiting
for all IOs to finish - a dealock.

DDN-bug-id: EX-10979
Was-Change-Id: I6d1cd7787f4c0d9f75f01030ed1235279bb6d301

llite: do not update layout at umount

umount can race with ptlrpcd finishing IOs and locks.
at the same time, umount resets sb->s_root to NULL which
is needed to form new RPCs, so it's safer to skip minor
layout update.

DDN-bug-id: EX-11166
Was-Change-Id: Ie9df33e95317bf38452ddd3afdfa30b31932bdb4

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10857
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I52e31a9c15967dd92714e6102e6b16d22695f81b

68410 master J !B ✗T –R 0/2

LU-10026 llite: issue compressiblity change in workqueue

mine build failure (#130382)

csdc 3 unresolved M +118 −56

66d

build failure (#130382)

2 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130382 FAILURE

no vote yet: Artem Blagodarenko, Zhenyu Xu

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The test_1021 tweaks (sleeps, echo moves) re-applied against the unwrapped assertion blocks; ci_need_change_layout renamed as in the original.

reply PS1 2026-08-26 13:39 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:292 — warn: vvp_layout_minor_change():passing zero to 'ERR_CAST'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:39

warn: vvp_layout_minor_change():passing zero to 'ERR_CAST'

reply PS1 2026-08-26 13:39 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:294 — error: vvp_layout_minor_change():'vlcw' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:39

error: vvp_layout_minor_change():'vlcw' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: issue compressiblity change in workqueue

This patch uses kernel-global workqueue to handle the file's minor
layout change RPC equest. At vvp_io_fini() when it find that the
file's compressibility should be changed, it creates a work and
insert the request to the kernel workqueue.

This would prevent the layout minor change RPC issued from within an
IO, and it could avoid the possible double lock the LOV conf lock.

lov: replace switch compressibility

The switch compressibility flag results in weird behavior -
it tells us we are to flip what's there, but results in some
very hard to read code.

I *think* the existing code is probably correct in absence
of race conditions, but an instruction to "flip" rather than
set is very confusing and could easily do the wrong thing
if there's a race which changes the compressibility value.

Replace it with two explicit bit flags.

DDN-bug-id: EX-10857
Was-Change-Id: I3b1d9f8d635e0bdbe67d5806127727a7264d26be

Singed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10857
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic4345b1387288ab11ce9061f40b056b07cb976bf

68364 master J !B ✗T –R 0/2

LU-10026 ofd: avoid assertion on large read request

mine build failure (#130339)

csdc 5 unresolved S +24 −5

66d

build failure (#130339)

4 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130339 FAILURE

no vote yet: Artem Blagodarenko

threads: 4 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Commit message no longer references a specific downstream release version for the affected old clients.

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:500 — warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:504 — warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:684 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:688 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: avoid assertion on large read request

In case an old client using CSDC sends a
read IO that will overflow buffers on OST, OST should
return an error rather than crash.

A similar check was previously added to the write path,
but was not added to the read path.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10359
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I7dc4c1a7a19c71d757d0a505003c3012edbf987a

68366 master J !B ✗T –R +1/2

LU-10026 ofd: skip reading compressed tails

mine build failure (#130341)

csdc 2 unresolved M +37 −28

66d

build failure (#130341)

1 thread(s) waiting on others

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130341 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:02

threads: 0 need your reply · 1 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied on the integration patch with only the series' field naming (ol_compr_chunk_lum_bits) where the original said _log_bits.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: skip reading compressed tails

if a tail is being overwritten, then we don't need to read
and decompress old content.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10116
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icde88d9fbd0ba0bdf53497c0c14d1215fa9cad73

68363 master J !B ✗T –R +1/2

LU-10026 ofd: reject non-aligned compressed niobufs

mine build failure (#130338)

csdc 1 unresolved S +7 −3

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130338 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 10:24

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied on the integration patch with only the series' field naming (ol_compr_chunk_lum_bits) where the original said _log_bits.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: reject non-aligned compressed niobufs

a client can send non-aligned compressed niobuf by mistake,
OST should reject it with an error.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11092
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2a61f13b5be17ca22c3a92b7d5b7b8bfd739c1eb

68365 master J !B ✗T –R 0/2

LU-10026 csdc: finer enable_compression control

mine build failure (#130340)

csdc 1 unresolved S +20 −0

66d

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130340 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Note for reviewers: the #if (PAGE_SIZE > 4096) guards in userspace (liblustreapi_layout.c) rely on the build host's page size, not the runtime client's; probably wants a runtime sysconf(_SC_PAGESIZE) check. Kept as the original for now.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: finer enable_compression control

On all architectures other than aarch64 and ppc64le enable_compression
is now enabled by default. lfs warning message is gone.

To use CSDC on aarch64/ppc64le (on your own risk)
llite.*.enable_compression=1 should be set. lfs
set_stripe command still prints a warning message in this case.

Test-Parameters: forjanitoronly
Test-Parameters: trivial
DDN-bug-id: EX-9074
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic8edc5bbeb8f9a3cd34ad3fc4e8c78e59f4cc34f

68367 master J !B ✗T –R +1/2

LU-10026 ofd: don't write blocks after compressed size

mine build failure (#130342)

csdc S +11 −0

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130342 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:00

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: don't write blocks after compressed size

due to limits in ib/rdma/lnet, client has to send full pages.
if client's pagesize is bigger then server's pagesize, then
we want to skip writing useless bytes from those full pages
containing compressed data.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10712
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I61c6fe72af1d2e8679d9fde303a561e80a287746

68368 master J !B ✗T –R 0/2

LU-10026 csdc: tune allowed compression type on server

mine build failure (#130343)

csdc 1 unresolved L +258 −18

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130343 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- The patch re-created a libcfs_string.h/.c pair to host string-mask helpers; master already provides cfs_str2mask()/cfs_mask2str() with u64 masks (lnet/libcfs/debug.c), so both files were dropped and the new compress_types stores call the existing helper (~0ULL masks). A hunk that narrowed the existing capability-mask store from ~0ULL to ~0 was dropped for the same reason.
- mdt.*.compress_types and obdfilter.*.compress_types are implemented as sysfs attributes (the target-side procfs vars tables the original patch extended no longer exist on master).
- The obd_compress_types mask enum / COMPRESS_NAMES use the series' type naming (lz4 for the HC type) and already include zstd, which is a valid layout type at this point in the series.
- OST-side only, as in the original: OBD_CONNECT2_COMPRESS is advertised in OST_CONNECT_SUPPORTED2 and requested by the OSC; MDT enabling remains pending (test_1013b keeps its unconditional skip; its old-version negative-check block, meaningless on master, was dropped).
- compression_supported() in test-framework.sh now probes the OSC import for the compressed_file connect flag instead of comparing versions. (The OSC import, not the MDC one, because MDT support is not enabled.)
- test_1013a/b were placed before test_1015 (the tests they were anchored to arrive later in the series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: tune allowed compression type on server

Use lctl get_param {mdt|obdfiler}.*.compress_types to list supported
compression types on server.

Use lctl set_param {mdt|obdfilter}.*.compress_types="+gzip-lzo" to
add gzip to and delete lzo from existing compression types on server.

Server will negotiate supported compression types with client in
ocd_compress_types and client import stats could show the negotiated
supported compression types in "lctl get_param {mdc|odc}.*.import"
connect_data section:

import:
  ....
  connect_data:
     ....
     compress_types: [ fast,best,gzip,lz4fast,lz4hc,lzo ]

The OST support for the connect flags is enabled in this patch, but
MDT enabling is pending unaligned compression support for DoM files.

test: refine sanity-compr.sh/test_0a

Only change and verify compression types negotiation between client
and OST0000.

DDN-bug-id: EX-9802
Was-Change-Id: I307f3826973f48791b7fbf4cdfa121401b059b0e
Was-Change-Id: Ic22685d2b21e872c01d90f2c3062b5d0d9b92ec2

test: sanity-compr.sh/test_0a failure

* move sanity-compr.sh test_0a/test_0b to test_1013a/test_1013b
* explicitly specify client's hostname to get client's negotiated
  compression types, as test environtment could mount several clients

DDN-bug-id: EX-9802
Was-Change-Id: Ic091f55a59ad9cfdaf3a94a58d44b372070c7875

Test-Parameters: forjanitoronly
DDN-bug-id: EX-6269
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I18943352e25ed9d5fe82442df9f00a7ef388f242

68369 master J !B ✗T –R 0/2

LU-10026 llite: set STATX_ATTR_COMPRESSED in ll_iocontrol()

mine build failure (#130344)

csdc 1 unresolved S +10 −0

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130344 FAILURE

no vote yet: Jian Yu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

LLIF_COMPR_FL renumbered 6 -> 8 (bit 6 is LLIF_ACL_VALID on master). Subject shortened to fit the 62-column limit.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: set STATX_ATTR_COMPRESSED in ll_iocontrol()

This patch extracts the compression flag LUSTRE_COMPR_FL from
mbo_flags and set STATX_ATTR_COMPRESSED flag in ll_iocontrol()
to please lsattr and other e2fsprogs tools.

Test-Parameters: forjanitoronly
Test-Parameters: trivial testlist=sanity-compr
DDN-bug-id: EX-8669
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I14d2082a6719a1ca5708f7aef7a2fb0f085ca63c

68374 master J ✗2B ✗T –R 0/2

LU-10026 csdc: store compression info in FID EA

mine build failure (#130348)

2 unique failures csdc 1 unresolved M +37 −65

66d

build failure (#130348)

janitor: 2 test failures unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b, sanity-quota@ldiskfs+DNE:test_80

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 4 other reviews
sanity-quota@ldiskfs+DNE:test_80	seen in 73 other reviews

build #130348 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied cleanly. Note for reviewers: this changes where the PFID EA is stored for small inodes (no longer embedded in LMA for <=256-byte inodes) - needs interop/lfsck review on master.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: store compression info in FID EA

Store compression information in OST-object's FID EA, and lfsck could
use it to recover the MDT-object layout EA from orphan OST-object(s).

2.15 Lustre may embed PFID and layout stripe info in LMA EA, this
patch would clear them from LMA EA and store them with compression
info directly into FID EA thereafter.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8038
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iacac04601b73f85d9bc057b8dd34a5004248dac4

66612 master J !B ✗T –R 0/2

LU-10026 csdc: sending compression info to server

mine build failure (#130320)

csdc 4 unresolved M +100 −13

66d

build failure (#130320)

3 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 11 failing config(s), none unique to this patch — janitor results

build #130320 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 3 need your reply · 0 waiting on others · 1 own notes · 0 bot

reply PS1 2026-07-15 00:58 Gerrit AI review for Lustre COMMIT_MSG:12 — (minor) This is an on-wire change: the client now fills bytes in the obdo (formerly o_padding_4) that are sent to the OST/MDS. Test-Paramete

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This is an on-wire change: the client now fills bytes in the obdo (formerly o_padding_4) that are sent to the OST/MDS. Test-Parameters: forjanitoronly doesn't exercise interoperability. Consider requesting old-server/new-client and new-server/old-client runs with serverjob/serverbuildno and clientjob/clientbuildno so the padding-reuse stays wire-compatible with unpatched peers.

note PS2 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS2 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase notes: kept the base's comment wording for the chunk-bits field
(2^(16+chunk_lum_bits)); the lov_lsm2layout() wrapper is added alongside
the base's compr_name_from_type() helper in lov_internal.h.

reply PS1 2026-07-15 00:58 Gerrit AI review for Lustre lustre_user.h:1128 — (minor) This comment reword looks unrelated to sending compression info to the server; is it meant to be in this patch? Also the code comput

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This comment reword looks unrelated to sending compression info to the server; is it meant to be in this patch? Also the code computes the chunk size with COMPR_CHUNK_MIN_BITS (COMPR_GET_CHUNK_SIZE()), not LOV_MIN_STRIPE_BITS; they happen to both be 16, but referencing the constant the code actually uses would be clearer.

reply PS1 2026-07-15 00:58 Gerrit AI review for Lustre lov_internal.h:408 — This isn't a bug, but after this patch lov_lsm2layout() has no remaining callers (both lov_io.c and lovsub_object.c now call lov_lsm2layout2

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

This isn't a bug, but after this patch lov_lsm2layout() has no remaining callers (both lov_io.c and lovsub_object.c now call lov_lsm2layout2()). Folding the olc parameter into lov_lsm2layout() and updating those two call sites would avoid the version-suffixed name and the now-dead wrapper.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: sending compression info to server

Client fills in layout compression info into obdo and passes it
to server.

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ieb5d7b3609da41f35f8622ed6116f19ce7567ddb

68371 master J ✗1B ✗T –R -1/2−

LU-10026 osc: Add counters for compressed data

mine build failure (#130346)

1 unique failure csdc 1 unresolved L +544 −265

66d

build failure (#130346)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 5 other reviews

build #130346 FAILURE

no vote yet: Sebastien Buisson, Artem Blagodarenko, Zhenyu Xu, Vitaliy Kuznetsov

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- The removed dead decompression helpers (decompress_chunk_in_lnb / decompress_rnb) were deleted against their current (folio/accessor) form.
- obt_compr_stats placed after the histogram member master added since.
- The osc stats_compr file and the compress_type_fast/best entries live in the debugfs vars table; the stats_compr fops therefore use LDEBUGFS_SEQ_FOPS (LPROC_SEQ_FOPS produces struct proc_ops on kernels >= 5.6, which the CI -Werror build rejects for debugfs entries). The ofd counter-init sits next to the reworked kernel-doc of ofd_stats_counter_init().
- New tests' old-version gates converted to compression_supported; the counter-reset calls are unconditional (the parameters always exist on a server with this series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: Add counters for compressed data

This patch is the first of two patches that add counters
to track client/server-side data compression statistics.
This patch add new compr_stats file in osc.*.compr_stats.

From added counters:
1. Size of compressed/uncompressed chunks written/read by
   client to compressed files, in chunk/bytes;
2. Compressed page counter;

ofd: Add counters for compressed data

This patch is the second of two patches that add counters
to track server-side data compression statistics.
From added counters:
1. Size of compressed/uncompressed chunks read by
   client to compressed files, in chunk/bytes;

Example:
	obdfilter.lustre-OST0000.stats_compr=
	snapshot_time             1715876501.602669648 secs.nsecs
	start_time                1715873670.819396144 secs.nsecs
	elapsed_time              2830.783273504 secs.nsecs
	compressed_objects        2 samples [reqs]
	uncompressed_objects      1 samples [reqs]
	compr_bytes_read          885 samples [bytes] 557 126974 52174403
	compr_bytes_read_raw      893 samples [bytes] 16384 131104 115885042
	incompr_bytes_read        8 samples [bytes] 16384 16384 131072
	compr_chunks_read         885 samples [reqs]
	incompr_chunks_read       8 samples [reqs]

DDN-bug-id: EX-7729
Was-Change-Id: I0998b1facaffd3e84b8d9429c5674d8d9ad60020

csdc: rename client-side stats

Some CSDC client-side stats can be confusing for the uninitiated:
- write_pages_compr is the number of pages from the 'raw' file that
  could actually be compressed;
- write_pages_uncompr is the number of pages from the 'raw' file that
  count not be compressed;
- write_chunks_compr is the number of chunks from the 'raw' file that
  could actually be compressed;
- write_chunks_no_compr is the number of chunks from the 'raw' file
  that count not be compressed;
- read_pages_compr is the number of pages from the 'raw' file that
  were actually compressed;
- read_chunks_compr is the number of chunks from the 'raw' file that
  were actually compressed;

So rename those parameters as follows:

    write_pages_compr -> write_pages_compressible
    write_pages_uncompr -> write_pages_incompressible
    write_chunks_compr -> write_chunks_compressible
    write_chunks_no_compr -> write_chunks_incompressible
    read_pages_compr -> read_pages_compressed
    read_chunks_compr -> read_chunks_compressed

And also change the corresponding field names in struct client_obd to
make them more explicit.

DDN-bug-id: EX-10914
Was-Change-Id: Ia876e0cde435095493e0b7457ce7f363ea86146d

test: sanity-compr.sh/test_1001 error

sanity-compr test_1001: operand expected (error token is "+ ")

Some compression stat counters could be 0 and not shows in the
stats_compr stats, so we need to assign 0 to the counter if
necessary.

DDN-bug-id: EX-10088
Was-Change-Id: I87948a61e591680d105a8a98a4cde4fba634e5fe

tests: fix check compr_chunk in test 1020

Skip test 1020 in sanity-compr, which depends on fixes
applied to the code that were not available in older
versions.

Temporarily add an additional tolerance of 1 unit when
checking the number of chunks in test 1020 in
sanity-compr.sh

DDN-bug-id: EX-10914
Was-Change-Id: Ia1c071d754217127decbf4d4397a58b512808843

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7729
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I091153480e53309c641d39f271bef536296dc09e

68378 master J ✗1B ✗T –R +1/2

LU-10026 ofd: skip decompression

mine build failure (#130352)

1 unique failure csdc 2 unresolved XS +5 −0

66d

build failure (#130352)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

1 thread(s) waiting on others

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 3 other reviews

build #130352 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 10:43

threads: 0 need your reply · 1 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

One context line updated to the folio form of is_chunk_start().

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: skip decompression

we can skip decompression if new data overwrite all raw bytes of
compressed chunk.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10103
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3511c6bff2a8c4c8d98532d025beead78c597a5b

68377 master J ✗1B ✗T –R 0/2

LU-10026 csdc: process all compressed chunks in rnb

mine build failure (#130351)

1 unique failure csdc S +22 −12

66d

build failure (#130351)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 2 other reviews

build #130351 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: process all compressed chunks in rnb

It is supposed, that every rnb contains only
continuous pages. There is always a gap after
a compressed chunk, so it is not expected to
have any other chunk in the same rnb after the
compressed chunk.

Compressing with chunk size >= than page size
is an exception to the rule above. After the
compression, data size is rounded up to page
size. So, it is possible that another compressed
chunk goes after the compressed one.

To support such a sutuation an aditional
loop was added to ofd_map_niobuf_to_chunks().

This patch is useful for ARM archs with 64k page.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibaac18e4d046e020a7ad60b8a436c789ba4c990b

68380 master J !B ✗T –R 0/2

LU-10026 csdc: exponential backoff of incompressiblity check

mine build failure (#130354)

csdc 1 unresolved M +144 −90

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130354 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:38 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:38

[Marc Bot] upstream remarks for reference:

Chunk-bits field naming follows the series (llc_compr_chunk_lum_bits).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: exponential backoff of incompressiblity check

If a file is incompressible and after a round of compressibility
check, if it remains incompressible, this patch would doubles the
skip bytes before running into another round of compressiblity
check.

This patch adds a state machine for update_compression_info(), the
states could be CS_INCOMPR, CS_COMPR, CS_CHECK, CS_CHECK_INC.

CS_INCOMPR    --  file is not compressible
CS_COMPR      --  file is  compressible
CS_CHECK      --  was in CS_COMPR, is now under check phase
CS_CHECK_INC  --  was in CS_INCOMPR, is now under check phase

state transition:
INCOMPR --> INCOMPR   : skip bytes has not been passed
INCOMPR --> CHECK_INC : skip bytes has been reached, in check phase
COMPR   --> CHECK     : in check phase, checked bytes has not been
                        reached
CHECK*  --> CHECK*    : in check phase, checked bytes has not been
                        reached
CHECK*/COMPR
          --> COMPR   : file is determined to be compressible
CHECK/COMPR
          --> INCOMPR : file is determined to be incompressible, reset
                        the skip bytes check length
CHECK_INC --> INCOMPR : file is determined to be incompressible,
                        double the skip bytes check length

====

Test with 4 OSTs system, target file layout is set as:
lfs setstripe -Eeof -c-1 -Z lz4 --compress-chunk=64 $tf

and src file is 336M bz2 file, using "dd if=$src of=$tf bs=4M" copy
this bz2 file to target file.

w/o the patch, copy uses about 7 seconds
** start 1724216088 **
84+1 records in
84+1 records out
352625400 bytes (353 MB, 336 MiB) copied, 3.76366 s, 93.7 MB/s
** end 1724216095 **
...7 seconds

w/ the patch, copy uses about 4 seconds
** start 1724215237 **
84+1 records in
84+1 records out
352625400 bytes (353 MB, 336 MiB) copied, 2.07946 s, 170 MB/s
** end 1724215241 **
...4 seconds

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I42105d3e137bf50897ff5c67654ca33552b23102

68379 master J !B ✗T –R 0/2

LU-10026 utils: return error on invalid compress level setting

mine build failure (#130353)

csdc 1 unresolved S +11 −4

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130353 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:38 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:38

[Marc Bot] upstream remarks for reference:

- The strtoul/end-pointer validation this patch adds was already carried (in stricter form) by the earlier level-handling patches of this series; those hunks were dropped.
- The parse-time rewrite of low lz4 levels to the lz4fast default was removed here (matching the original series' final behaviour): the stored level is kept and the compression engine falls back to lz4fast when allocating the compressor. This also makes the negotiated-type test lists in sanity-pfl roundtrip correctly.
- The new negative test was renamed test_1013a -> test_1013c: 1013a already exists (type negotiation test); the duplicate name would have silently shadowed it (latent issue in the original series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: return error on invalid compress level setting

lfs setstripe -Z <compress_type>:<compress_level> does not report
error on invalid <compress_level>.
An explicit low level for the lz4 type (below the lz4hc minimum) is
no longer silently rewritten to the lz4fast default at parse time;
the stored level is kept and the compression engine falls back to
lz4fast for such levels when allocating the compressor.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10186
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie794a6d4cf208a9c66d1d179c157fee756477aee

66279 master J !B ✗T –R +1/2

LU-10026 sec: disable compression for encrypted files

mine build failure (#130315)

csdc 2 unresolved S +10 −3

66d

build failure (#130315)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #130315 FAILURE

reviews: Sebastien Buisson +1 2026-08-26 09:47

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre COMMIT_MSG:7 — (style) The body explains why compression must be disabled for encrypted files, but that logic (compressed = 0; goto skip_compression) alrea

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(style) The body explains why compression must be disabled for encrypted files, but that logic (compressed = 0; goto skip_compression) already lives in the parent compression patch. What this change actually does — throttling the existing CWARN to once per 24h and adding `always_except LU-10026 66` in sanity-sec.sh — isn't mentioned. Could the message describe the change it makes?

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre sanity-sec.sh:18 — (minor) There is no test_66 in sanity-sec.sh (tests jump 65 -> 68), so this exclusion is a no-op in this tree. Which patch adds sanity-sec t

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) There is no test_66 in sanity-sec.sh (tests jump 65 -> 68), so this exclusion is a no-op in this tree. Which patch adds sanity-sec test_66, and should the always_except land together with it? Worth also confirming sanity-sec is the intended suite rather than sanity-compr.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 sec: disable compression for encrypted files

In case a read-modify-write io pattern is carried out on a compressed
file, it has to be handled on server side.
But because encryption cannot be done on server side for security
reasons, we are not able to handle that kind of io pattern if the file
is encrypted + compressed.
So just disable compression for all encrypted files.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I84881fb1235f015d022751d4cce2d43a7231c2b4

66280 master J !B ✗T –R 0/2

LU-10026 osc: disable CPU-access features for RDMA only pages

mine build failure (#130316)

csdc 1 unresolved S +27 −10

66d

build failure (#130316)

1 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130316 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre osc_request.c:1611 — (defect) `req` was just allocated above (ptlrpc_request_alloc_pool for writes, ptlrpc_request_alloc for reads) and every other error path in

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) `req` was just allocated above (ptlrpc_request_alloc_pool for writes, ptlrpc_request_alloc for reads) and every other error path in this function frees it with ptlrpc_request_free(req) before returning. These two early returns leave without freeing it, and *reqp is only assigned on the success path, so the caller can't release it either:

    if (compressed && !page_access_allowed)
        RETURN(-EINVAL);
    if (inode && IS_ENCRYPTED(inode) && !page_access_allowed)
        RETURN(-EINVAL);

For OST_WRITE the request comes from osc_rq_pool, so a GPU-direct write to a compressed or encrypted file leaks a pooled request each time and can eventually drain the pool. Should both routes free req first?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: disable CPU-access features for RDMA only pages

Pages which cannot be accessed by the CPU are referred to
as RDMA only pages.  If pages cannot be accessed by the
CPU, it is impossible for us to do compression,
encryption, checksums, or short-io (data-in-RPC) on them.

This patch disables compression and encryption for these
pages and cleans up the code so checksums and short-io
are disabled by the same code.

The only user of RDMA only pages today is Nvidia's GPU
direct, so this patch disables compression and
encryption with GPU direct.

NB: We eventually intend to handle compression for
GPU direct with server side compress/decompress.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iad9311617cddf27d3ff75a17429499c573067ea0

66611 master J !B ✗T –R 0/2

LU-10026 lfs: migrate compressed file without stripe info

mine build failure (#130319)

csdc 2 unresolved M +76 −14

66d

build failure (#130319)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #130319 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-07-15 00:58 Gerrit AI review for Lustre COMMIT_MSG — (minor) The diff also touches Documentation/man1/lfs-mirror-extend.1 (documenting compression options for SETSTRIPE_OPTIONS), which the mess

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) The diff also touches Documentation/man1/lfs-mirror-extend.1 (documenting compression options for SETSTRIPE_OPTIONS), which the message doesn't mention and which is unrelated to the migrate fix. Is that hunk meant to be part of this patch, or should it be split out / called out in the body?

reply PS1 2026-07-15 00:58 Gerrit AI review for Lustre liblustreapi_layout.c:2166 — (minor) This prints `fd`, which is always -1 on this path, so the message reads "Cannot open file X: -1" with no useful detail. The error is

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This prints `fd`, which is always -1 on this path, so the message reads "Cannot open file X: -1" with no useful detail. The error is in errno from open() – strerror(errno) would be the informative value.

Also note the caller (lfs_migrate/lfs.c) already emits its own llapi_error() when this returns < 0, so this adds a duplicate line. Using llapi_error(LLAPI_MSG_ERROR, errno, ...) here (as the ENOSPC path below does) would decode errno and respect the message level instead of writing to stderr unconditionally.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lfs: migrate compressed file without stripe info

lfs migrate file without specifying stripe info will get layout info
from the file as the target layout template, and
llapi_layout_get_by_xattr() tries to convert LOV_PATTERN_* values
to user scope LLAPI_LAYOUT_* values, while LOV_PATTERN_COMPRESS
is missed in this conversion.

This patch add a function llapi_pattern_from_lov() to handle this
conversion specifically.

This patch also add more error messages for llapi_layout_file_open().

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I49a43cc7761cd2baed7a5da7d4e7cff2152ff9bb

66615 master J ✗1B ✗T –R 0/2

LU-10026 csdc: prohibit set compression upon encrypted file

mine build failure (#130322)

1 unique failure csdc 1 unresolved L +247 −35

66d

build failure (#130322)

janitor: 1 test failure unique to this patch — conf-sanity4@zfs:test_123aj

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
conf-sanity4@zfs:test_123aj	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #130322 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS2 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS2 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: kept the base's newer compression-parameter assignment block
in lod_use_defined_striping() (gated on LOV_PATTERN_COMPRESS, with the
extension-component chunk-adjustment deferral); this patch's encryption
checks and attr threading apply on top unchanged.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: prohibit set compression upon encrypted file

Setting compression layout component upon encrypted file is not
allowed for now.

This patch add this check on MDS in creating file with layout,
adding/merging new mirror to existing file.

Test-Parameters: forjanitoronly
Test-Parameters: testlist=sanity-sec env=ONLY=67,PTLDEBUG=-1
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I60d9f4bfce3a498f1eb3994c6276afb9d89c99a7

68356 master J !B ✗T –R +1/2

LU-10026 ofd: remove unused csdc bits

mine build failure (#130335)

csdc 1 unresolved L +150 −594

66d

janitor run: 15 failing config(s), none unique to this patch — janitor results

build #130335 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:10

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- The dt_read_prep() signature revert (dropping the decompression flag) was
  applied to both the header prototype and the function body in
  obdclass/dt_object.c (the bodies moved there upstream, LU-13139); the
  osd-wbcfs dbo_read_prep implementation lost the parameter as well.
- In ofd_preprw_write(), this patch removes the separate err_put label; the
  early "BRW to missing obj" failure (which happens after
  tgt_grant_prepare_write()) now puts the object and jumps to err_commit so
  the prepared grants are still committed.
- A whitespace-only obd.h hunk (tab/space churn) was dropped.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: remove unused csdc bits

some bits are not used with the ofd csdc integration patch
implemented, we can remove
those now. also the patch tries to sync the code with master
branch a bit.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9996
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8edb25fe322f4fafaac1b9a832dd5b4b72e52a42

68346 master J –B ✗T –R -1/2−

LU-10026 tests: improve/skip sanity test_460a

mine build failure (#130326)

M +108 −27

66d

build #130326 FAILURE

patchsets: PS1 08-25 PS2 08-26

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: improve/skip sanity test_460a

Skip sanity test_460a for el9.2 clients, since they appear to be
failing that test regularly, but no other distro client is.
Improve the log messages to see what stage is currently running.
Limit the "cmp --verbose" messages to one chunk, otherwise it
may print the entire 14MB test file (about 80 MiB of ASCII).

Move enable_compression() and disable_compression() functions
into test-framework.sh so that they can be used for all tests.

Set LFS_SETSTRIPE_COMPR_OK=y in enable_compression() since we
already know this is a preview and don't need it printed.

Allow sanity-compr.sh to specify SANITY_ONLY and/or SANITYN_ONLY,
and skip the other test script run if only one of them is set.

Test-Parameters: forjanitoronly
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8cb2f67689824513335f3fa65e9ea7519e3ebbe5

66263 master J !B ✗T –R 0/2

LU-10026 osc: add compression to can_merge_pages

mine build failure (#130261)

csdc XS +5 −2

66d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #130261 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: add compression to can_merge_pages

Some BRW flags are OK to have on only some pages in a BRW,
others are not.  can_merge_pages has a whitelist of the
flags which are safe to have on only some pages in a BRW,
and prints a warning if other flags are seen.

Add compression to the white list, because while all pages
in an niobuf must be compressed, it is normal to have only
some pages in a BRW compressed.

Prior to this patch, this warning was printing during
normal usage of compression.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia262d4fc878e5328bd956865047e997aa77946f0

66266 master J !B ✗T –R 0/2

LU-10026 osc: clear oi_write_osclock in lock fini func

mine build failure (#130263)

csdc S +12 −4

66d

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130263 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: clear oi_write_osclock in lock fini func

Move osc_io::oi_write_osclock clearance in osc_lock_fini() as
it's set in osc_lock_init().

Compression IO could possibly expand lock region and
osc_lock_set_writer() could access a osc_io that is not accessed
in osc_io_iter_init(), so that osc_io_rw_iter_fini() miss clearing
osc_io's oi_writer_osclock.

This patch moves the oi_write_osclock clearance in lock fini function
to match its creation in osc_lock_init().

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ied42f5befc1abd76aa10a7666eadb9a58e1f1783

66267 master J !B ✗T –R +1/2

LU-10026 llite: getting stripe info optimization

mine build failure (#130264)

csdc 2 unresolved M +63 −0

66d

build failure (#130264)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130264 FAILURE

reviews: Sebastien Buisson +1 2026-08-25 21:12

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre rw.c:794 — (minor) These CDEBUG() pairs print start_idx/end_idx before and after the rounding with no descriptive text, on the D_SEC (security) mask wh

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) These CDEBUG() pairs print start_idx/end_idx before and after the rounding with no descriptive text, on the D_SEC (security) mask which is unrelated to readahead. Same at lines 813 and 816. They look like leftover debugging scaffolding; if kept, D_READA would be the right mask.

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre rw.c:815 — (defect) vui_ra_pages is initialized once per read syscall in vvp_io_read_start() and only reset at IO teardown, but ll_readahead() runs onc

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) vui_ra_pages is initialized once per read syscall in vvp_io_read_start() and only reset at IO teardown, but ll_readahead() runs once per page (ll_readpage()->ll_io_read_page()->ll_readahead()). Won't this += accumulate across every not-uptodate page of a compressed read, progressively inflating the RA window enlarge at lines 806-808 and ria_end_idx_min at line 863?

Also, the delta is taken from end_idx before the EOF clamp just below (lines 819-828), so vui_ra_pages can grow past the file size. ria_end_idx_min then feeds ll_read_ahead_pages() at lines 478-479, which raises ria_end_idx without an ria_eof guard - so readahead can extend past eof_index, the opposite of the "restrict readahead to eof" goal in the commit message.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: getting stripe info optimization

ll_lov_getstripe_ea_info() is expensive call and should be avoided
if possible. Let's use cached chunk size rather than get it from
stripe info every time.

llite: restrict readahead to eof

Compressed file readahead rounding needs to come before
readahead is limited to EOF.

DDN-bug-id: EX-7601
Was-Change-Id: I4e9e7fe63301c08efcb05f170726735593a9431d

llite: fix chunk_bits usage in readahead

For the minimum compression chunk size, chunk bits is zero,
so we cannot use if (chunk_bits) to determine if we're
doing compression.

This also fixes two other things:
1. A rounding error when rounding to chunk
2. Move rounding of end_idx to before first usage of
end_idx, so calculation of number of pages is correct
Without this, when the user reads 1 page or less, readahead
will calculate the readahead page count as 0 and will exit
without reading the chunk.

DDN-bug-id: EX-6127
Was-Change-Id: I273506fd4f6ed5f0b8b5020357fd7caf0531e61c

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id08487ec782f797e242e3f673c4a4dd8d526c9cc

68375 master J –B ✗T –R -1/2−

LU-10026 tests: Skip sanity 430c for CSDC layout

mine build failure (#130349)

csdc 1 unresolved XS +1 −0

66d

build #130349 FAILURE

no vote yet: Sarah Liu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: Skip sanity 430c for CSDC layout

Skip sanity test_430c until SEEK_HOLE is implemented for CSDC

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7680
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I7359262de9e3d1644d2a45b5336328bd8253f91b

68390 master J !B ✗T –R 0/2

LU-10026 mmap: add mmap support for compression

mine build failure (#130362)

csdc 1 unresolved L +446 −78

66d

janitor run: 31 failing config(s), none unique to this patch — janitor results

build #130362 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- OBD_FAIL_LOV_SKIP_CHECK_COMPR renumbered to 0x143a: the original's 0x1436 collides with OBD_FAIL_LLITE_TRUNC_PAUSE on this branch, and the original's sanity.sh test actually set fail_loc=0x1429, which is OBD_FAIL_LLITE_FORCE_BIO_AS_DIO - both the define and the test now use 0x143a consistently.
- The osd_lseek() compressed-object rejection was merged into the current form of the function (osd_get_filp_for_inode(), not osd_alloc_file_pseudo()); osd_xattr_get() prototype added to osd_internal.h.
- mmap_copy_file added to the current THETESTS line in tests/Makefile.am.
- test_1003 anchored before test_1004; its gate and the sanity.sh 460g gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 mmap: add mmap support for compression

This removes the EOPNOTSUPP for compression with mmap and
adds an mmap sanity test for compression.  This patch
removes all the restrictions for mmap, but we actually only
have unaligned read support right now, so the test is
deliberately simplified to only test reads.

A more complicated version which also tests mmap writes
comes later in the series, once read-modify-write is
supported.

The test tests mmap by copying data at several different
block sizes with several different compression chunk sizes.

tests: add mmap write test

This improves the existing mmap test to test mmap writing
as well as mmap reading.

DDN-bug-id: EX-7601
Was-Change-Id: I81840c7bbbefbc5c3bae6b270c2d94297a254d19

csdc: server reject FIEMAP/SEEK_HOLE|DATA on compr obj

Server return -EOPNOTSUPP if they get the FIEMAP and
SEEK_HOLE/SEEK_DATA requests upon compressed file objects.

DDN-bug-id: EX-8927
Was-Change-Id: I9f04fbb13a22cc83402d9989daab63a59367ff33

csdc: check ocd flag for fiemap/lseek

Currently client will not sending fiemap/lseek request to OST if
the file is a compressed one. This patch will check the
OBD_CONNECT2_COMPRESS flags and send the request if OST supports
compression as server would do the fiemap/lseek check instead.

DDN-bug-id: EX-8927
Was-Change-Id: I478cbf161044165fa31d4caa2336e9949fc626fe

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7644
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4a37b106831a903d90e8a8871e9a93baac4e201e

68407 master J !B ✗T –R 0/2

LU-10026 tests: minor cleanups to sanity-compr

mine build failure (#130379)

csdc 1 unresolved XS +6 −0

66d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130379 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The test_1001 hunk re-anchored (its context had a version guard this tree already dropped). One sub-section sentence of the original message did not apply to this content and was dropped.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: minor cleanups to sanity-compr

add/fix few subtests to remove big temporary files

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11112
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id04692cbd2443721070a7152186b38829b075907

68403 master J –B ✗T –R -1/2−

LU-10026 tests: fix 1009e and 1122 for arm clients

mine build failure (#130375)

csdc 1 unresolved S +12 −12

66d

build #130375 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

Applied against the LU-10026-tagged aarch64 except block.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: fix 1009e and 1122 for arm clients

arm clients use 64k page size, this should be reflected in
the tests checking stats on different operations.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10755
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia84ceef06c7ce96ae464abef919bf368583a70e5

68404 master J !B ✗T –R 0/2

LU-10026 tests: drop redundant gate in sanity-compr/2002

mine build failure (#130376)

csdc 1 unresolved XS +0 −3

66d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130376 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The original is a later snapshot of the change that introduced test_2002 (same Change-Id) and re-added subtest_2002/test_2002 at the file end, duplicating the definitions (the shell silently uses the second copy, and run_test 2002 would execute twice). The port keeps a single copy - the later version - in place; the effective delta is dropping the per-test gate now covered by the suite-wide gates. New Change-Id, original kept as Was-Change-Id.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: drop redundant gate in sanity-compr/2002

sanity-compr now checks compression support and enablement at
file scope, so test_2002 does not need its own skip gate.

This is the remaining delta of a later snapshot of the change
that introduced test_2002; the rest of it is already applied.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11120
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I7daec2d28fa32646e607291a6fcdd0e303351f67
Change-Id: I6b3bd30b8009fa702f9ddae83fdaf005fd40fd09

68350 master J ✗8B ✗T –R 0/2

LU-10026 tests: memory check depends on RAM

mine build failure (#130330)

8 unique failures csdc 1 unresolved XS +2 −7

66d

build failure (#130330)

janitor: 8 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1, sanity1@ldiskfs+DNE:test_36f +5 more

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 2 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews
sanity1@ldiskfs+DNE:test_36f	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_36h	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_44b	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_56ra	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_56x	seen in 37 other reviews
sanity1@ldiskfs+DNE:test_56xa	seen in 38 other reviews

build #130330 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- The Fixes: reference was remapped to the master commit carrying the same
  change (14fc034e983e "LU-18086 obdclass: page_pools shrinker fix").
- The debug echos this patch removes were already absent from the ported
  subtest_2000; the local count declaration is kept (count is still used
  only inside the function).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: memory check depends on RAM

Each open file consumes about 4.7KB of RAM, 300000
files consume about 1.3GB. At systems with 2.6GB RAM
it might cause an OOM with SLOW=yes option.
1. Consume 40% of RAM to check that page pools would be
shrinked.
2. Change createmany with memhog.
3. Export MEMHOG in init_test_env.

Test-Parameters: forjanitoronly
Fixes: 14fc034e983e ("LU-18086 obdclass: page_pools shrinker fix")
DDN-bug-id: EX-10278
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I58ae0a201456ed084db5c528007fe8d7a3782315

66277 master J !B ✗T –R 0/2

LU-10026 llite: restrict overwrites during preview

mine build failure (#130314)

csdc 1 unresolved M +80 −2

67d

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130314 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: ci_size_extend_compression:1 is placed inside the cl_io
bitfield next to the new ci_parity_io:1 member that master added in the
meantime (the non-bitfield loff_t ci_parity_eof follows the bitfield).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: restrict overwrites during preview

EX-7601 is an issue where when modifying a compressed file
we do not correctly read-up and re-write existing
compressed data.

To avoid this, we can only allow writes which are not
aligned to compression chunk size when they are not
overwriting existing data, ie, when they are extending the
file.

This returns EINVAL for all writes to compressed files
which are not either chunk aligned or extending the file.
This should prevent users from hitting the data corruption
issue but still allows some basic usage.  This is intended
just for the preview period.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0eea01e2249866a074afd0d0642fe6dce9a49664

65767 master J !B ✗T –R 0/2

LU-10026 utils: update handling of compr level for lz4

mine build failure (#130258)

csdc 6 unresolved M +107 −49

67d

build failure (#130258)

5 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130258 FAILURE

patchsets:

+1 earlier

PS1 05-08

PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 08-25

threads: 5 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS6 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS6 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebased onto the current series base (65769). Notes on this patchset:

- The level-mapping helpers in lustreapi.h were reconciled with the current
  base's version of the file: the base had switched the numeric bound 0xf to
  the named LL_LZ4FAST_MAX_LEVEL constant; this patch keeps the named
  constant while applying the new +1 level-shift semantics.
- The crypto algorithm-name helper this patch previously extended no longer
  exists at this point in the series (the earlier patches carry a
  display-only compr_name_from_type() in lov instead). The level-dependent
  algorithm selection ("lz4" vs "lz4fast" by level) now lands together with
  the compression engine in 65770, which carries the final form of that
  helper in obdclass.
- In llapi_parse_compress_type(), the "no level given" branch now guards the
  '*level = -1' store with a NULL check: the function accepts a NULL level
  out-pointer and the unguarded store would dereference NULL.
- test_100d: merged this patch's "lz4 twice (std + HC)" coverage with the
  base's newer type list. Final list is "lz4 gzip lz4 lz4fast" plus
  no-level types "lzo zstd", and the compress-file creation passes -S4M so
  the doubling chunk sizes (64K..2M) stay below the stripe size (the chunk
  size is clamped to the stripe size, which would break the per-chunk find
  counts). File names carry _${lvl}_${cs} suffixes so the duplicate lz4
  entries do not collide.

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre lustreapi.h:1419 — (minor) The comment above still says the acceleration factor is mapped "keeping 1-9 as-is", but from/to now apply a +1/-1 offset, so the enc

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(minor) The comment above still says the acceleration factor is mapped "keeping 1-9 as-is", but from/to now apply a +1/-1 offset, so the encode and decode boundaries no longer line up at 9. from_lz4fast_level(9) returns 10 (the `level < 10` path, level+1), but to_lz4fast_level(10) takes the `>= 10` path and returns 11. So `lfs setstripe -Z lz4fast:9` reads back as level 11 via `lfs getstripe --compr-level` (8 round-trips, 9 does not). Was the boundary meant to be `level <= 10` on decode (or `level < 9` on encode)?

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre lustre_crypto.h:197 — (style) These crypto headers are #include'd in the middle of the file, interleaved between the new static inline definitions, rather than gr

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(style) These crypto headers are #include'd in the middle of the file, interleaved between the new static inline definitions, rather than grouped with the other includes near the top. If the patch is refreshed, consider moving them up. (There's also a stray double blank line at the end of the block.)

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre lustre_crypto.h:200 — (typo) "Cyrpto" -> "Crypto" here and in the ll_crypto_acomp_set_level() comment below.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(typo) "Cyrpto" -> "Crypto" here and in the ll_crypto_acomp_set_level() comment below.

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre lustre_crypto.h:203 — (defect) ll_crypto_comp_set_level() and ll_crypto_acomp_set_level() (and COMPR_LEVEL_MASK / the *_MOD_NAME macros) don't appear to be called

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) ll_crypto_comp_set_level() and ll_crypto_acomp_set_level() (and COMPR_LEVEL_MASK / the *_MOD_NAME macros) don't appear to be called or referenced anywhere in the tree. If nothing uses them yet, this looks like it belongs in the kernel patch that consumes them rather than in a utils level-mapping change.

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre liblustreapi_layout.c:1657 — (defect) This branch is entered when `level` is NULL (`!level`), but then dereferences it. The pre-patch code returned early for exactly the

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) This branch is entered when `level` is NULL (`!level`), but then dereferences it. The pre-patch code returned early for exactly the `!level` case, so a caller of this exported liblustreapi function that passes NULL for `level` (only interested in the type) now crashes. Both in-tree callers happen to pass non-NULL, but the guard still contradicts the body. Something like:

    if (!level)
        return 0;
    if (!ptr) {
        *level = -1;
        return 0;
    }

tags: csdc

owner Marc Vef · uploader Marc Vef · PS6 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: update handling of compr level for lz4

The way lz4 compression level or acceleration factor is handled needs
to be adapted in order to match what is provided by the lz4 userspace
tool:
- any level between 0 and 2 is interpreted as the default lz4
  acceleration factor of 1;
- any level from 3 and up to 16 is interpreted as a compression level
  for internal lz4hc. Increasing the compression level trades CPU
  time for improved compression ratio;
- acceleration factor can be specified for the lz4fast compression
  type, from 1 to 26. This acceleration factor trades compression
  ratio for faster speed.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4711217c1a6601f29f78d262567da5998f657fc9

66282 master J !B ✗T –R 0/2

LU-10026 llite: skip hole/data lseek() on compressed file

mine build failure (#130317)

csdc M +48 −5

67d

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #130317 FAILURE

no vote yet: Mikhail Pershin

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: skip hole/data lseek() on compressed file

Doesn't execute real lseek() on compressed file with
SEEK_HOLE/SEEK_DATA origin but consider file always
non-sparsed and do generic lseek() only

test: get size before fallback to generic

If ll_file_seek() falls back to generic_file_llseek_size()
then make sure inode size is reliable

DDN-bug-id: EX-7786
Was-Change-Id: Ieb8c90e66fb19675ba41f0147c0d9cdaf29ea20a

Test-Parameters: forjanitoronly
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2cd587cc0205e85758e06bbaafafe0e2959e0ade

68351 master J ✗2B ✗T –R 0/2

LU-10026 llite: don't queue async IO page for inode in freeing

mine build failure (#130329)

2 unique failures csdc 1 unresolved S +19 −0

67d

build failure (#130329)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 5 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130329 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- Registered as .cio_commit_async with the current signature (the
  enum cl_io_priority argument was added upstream), next to master's
  renamed .cio_read_ahead_prep hook.
- Reviewer note: master meanwhile carries LU-17364 ("discard un-flushable
  pages on evicted inode teardown"), which addresses the same teardown
  crash from another angle; this patch is kept as an additional guard and
  may be droppable after evaluation.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: don't queue async IO page for inode in freeing

When inode is being freed, do not try to queue async IO pages as
there is a race window between ll_delete_inode() and
ll_truncate_inode_pages_final(), and the queued async IO pages
cannot be teardown.

(osc_page.c:182:osc_page_delete()) Trying to teardown failed: -16
(osc_page.c:184:osc_page_delete()) ASSERTION( 0 ) failed:
(osc_page.c:184:osc_page_delete()) LBUG

    lbug_with_loc+0x64/0xb0 [libcfs]
    osc_page_delete+0x290/0x574 [osc]
    cl_page_delete0+0xb8/0x404 [obdclass]
    cl_page_delete+0x64/0x22c [obdclass]
    ll_invalidate_folio+0x114/0x2a0 [lustre]
    truncate_cleanup_folio+0x6c/0xf0
    truncate_inode_pages_range+0xf8/0x4e0
    truncate_inode_pages_final+0x54/0x80
    ll_truncate_inode_pages_final+0x40/0x520 [lustre]
    ll_delete_inode+0x68/0x25c [lustre]
    evict+0xb8/0x1a0

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11006
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: If655a89faf6f979167c05c91c7b52a6c49a7884b

68352 master J ✗2B ✗T –R +1/2

LU-10026 ofd: space is always granted to server

mine build failure (#130332)

2 unique failures csdc XS +1 −1

67d

build failure (#130332)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 8 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130332 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 09:47

patchsets: PS1 08-25 PS2 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: space is always granted to server

when it's converting a compressed chunk to uncompressed one for
partial truncate.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10391
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic636525e76c544b55dede13133f423cc0aed534f

68354 master J ✗2B ✗T –R 0/2

LU-10026 sec: set page sync flags to avoid ENOMEM

mine build failure (#130334)

2 unique failures csdc 1 unresolved S +24 −6

67d

build failure (#130334)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 6 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130334 FAILURE

no vote yet: Artem Blagodarenko

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port note: fill_cpga() debug output adapted to the folio-based brw_page
(bp_folio).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 sec: set page sync flags to avoid ENOMEM

Set OBD_BRW_SOFT_SYNC page flag when appropriate page_pool
is at least 75% full. This would cause a "soft sync" update
at OST side. When pool is at least 81% full clear OBD_BRW_ASYNC
flag to trigger a sync commit. This needs to not stop
compression at client side when page pool becomes empty,
i.e. no free elements.

There is sanity-compr_2001 to check that all chunks from written
1GB data have been successfully compressed, but it will be
landed later,
in a separate patch, after the statistics patches.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10481
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I715d2fb95a459b47c003dad3339f8b6f775245a8

68353 master J ✗2B ✗T –R 0/2

LU-10026 scripts: Compression estimate script

mine build failure (#130331)

2 unique failures csdc 1 unresolved L +472 −1

67d

build failure (#130331)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 7 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130331 FAILURE

no vote yet: Raphael Druon, Jian Yu

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- always_except tag renumbered to LU-10026.
- ll_compression_scan added to EXTRA_DIST so it survives make dist (the
  tarball would otherwise fail to build with the script listed in
  bin_SCRIPTS but absent).
- Reviewer note: the script is installed under the server-only block
  although it can run on any client; left as the original patch had it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 scripts: Compression estimate script

ll_compression_scan is a simple tool which can be run on any
Linux system to estimate the space usage reduction from the
Lustre Client Side Data Compression (CSDC) feature with
particular compression settings (algorithm, chunk size,
and compression level).

When run on one or more directories, it will recursively
examine a percentage of files under that directory, sampling
data in those files to estimate how the files will compress.

This tools samples data throughout the file, so it should
avoid problems with poor estimates for files with headers
which differ from the bulk data in the file.

However, if the directory tree is particularly imbalanced,
with a few large uncompressible files in one directory, and
many small files in other directories, then scanning a small
percentage of files may give a misleading compression estimate.
Sampling a larger percentage of files will improve this.

This tool requires the lz4, lzop, and gzip utilities to
be installed in order to test those compression types.
(lzop is the command line utility for lzo compression)

scripts: add whole file to compression scan

Add a mode where the compression scan script compresses the
entire file, which in theory should 100% match the
compression results from using CSDC and allow a test to
calculate the exact space usage reduction expected by
using CSDC.

This is intended to be used mostly for testing.

Change help documentation slightly to make clear this can
also accept a path to a single file.

DDN-bug-id: EX-7795
Was-Change-Id: I606a33d686d87dd631bf5b33dc85ee8c24fe9f67

scripts: Improve estimated ratio

ll_compression_scan does not take in account the size of the
sampled files, this might lead to uncorrect estimated ratio for non
homogeneous file.

This patch apply the compression ratio estimated with the sampled data
and applies it to the entire file size, assuming the file will have
the same compression ratio across it.

DDN-bug-id: EX-8362
Was-Change-Id: Ic4a26460e17c666b9edf4c0d8d450a06fad5920f

scripts: improve ll_compression_scan functionality

Improve ll_compression_scan script functionality without
changing the compression estimates.
- add a version string to the output to allow tracking
- handle pathnames with spaces in them
- handle the lz4fast compression type
- allow running on MacOS for testing

DDN-bug-id: EX-8362
Was-Change-Id: I0b8442a2590fdb9c718b1404cba1d73c26cff03c

scripts: improve ll_compression_scan estimate

Improve ll_compression_scan script to give a better estimate of
actual compression ratios.
- add a '-d' debug option for verbose output during testing
- log and report incompressible small files < 4096
- log and report incompressible file count and size
- include small/incompressible/large files in compression estimate
- add a correction factor to calculations for safety margin

DDN-bug-id: EX-8362
Was-Change-Id: If561b0273e38e4821de228c81291859c7bb1a0d2

tests: exclude sanity-compr/1008 on Ubuntu

This subtest is failing consistently on Ubuntu.  Disable until it
can be fixed.

DDN-bug-id: EX-9125
Was-Change-Id: Ie71462e7f033be914523ca96b22478a53b81b882

utils: always print ll_compression_scan summary

Print the summary block for ll_compression_scan even if it is
killed with CTRL-C before finishing the scan.  Otherwise, only
the running total compression ratio estimate will be printed.

DDN-bug-id: EX-9125
Was-Change-Id: I64e135fa6729767d279a52df767252103a2540e5

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7681
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Raphael Druon <rdruon@ddn.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I092f9608553eba10bacfcc3c4a3fafc9a454c287

68399 master J –B ✗T –R -1/2−

LU-10026 tests: sanity-compr/1000d failure for ARM client

mine build failure (#130371)

csdc S +9 −7

67d

build #130371 FAILURE

no vote yet: Zhenyu Xu

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: sanity-compr/1000d failure for ARM client

aarch64 has 64K page size and compression chunk size could be adjusted
upward if it's original setting value is equal to page size, so
the test should be refined to get the actual chunk size be set.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ide2f91e255ba886f9c1e18d1e1fe89c589f3bca1

68348 master J ✗2B ✗T –R 0/2

LU-10026 csdc: do not mix compressed and plain in niobuf

mine build failure (#130328)

2 unique failures csdc 1 unresolved XS +1 −1

67d

build failure (#130328)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 3 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130328 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port note: applied as a hand edit - the mask line context had been
reflowed by the earlier can_merge_pages() changes. Content unchanged
(removes OBD_BRW_COMPRESSED from the merge-safe mask).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: do not mix compressed and plain in niobuf

It is important to prevent placing
compresed and uncompressed chunks to the
same rnb, because rnb has only one set flags
for all chunks contained.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: If3fbe160a9f32af2d51f4ba694f444563feb82f6

68401 master J !B ✗T –R 0/2

LU-10026 tests: Add compression statistics output

mine build failure (#130373)

csdc 1 unresolved S +25 −0

67d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #130373 FAILURE

no vote yet: Sergey Cheremencev, Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

The two OST-version guards around the obdfilter stats access were dropped - compression_enabled() already implies the server supports compression on this tree. The interop paragraph of the message was reworded accordingly.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: Add compression statistics output

sanity-comr executes sanity, sanityn large set of its tests

Currently, it is not even easy to understand that data was
compressed/decompressed. Adding compression statists gives a sign
that compression is enabled and works, showing that parts
(client/server, any features etc.) were executed during the test.

tests: skip CSDC stats errors in interop

This also avoids trying to clear or fetch compression statistics
from servers that do not support compression, always returning
success in that case.
Let's avoid trying to clear or fetch these stats from servers that
do not have them and always return success when clear/read stats.

DDN-bug-id: EX-10124
Was-Change-Id: Ide3e69dfd0268041591c3445aff39532ff00f019

tests: osc compr stats for all OSTs

Show osc compression statistic for all OSTs in
the end of the test session.

DDN-bug-id: EX-11007
Was-Change-Id: Id6ae864438931d9ce50300911490760bf24dd88a

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10124
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9856bad32b0f421ce8cb8ff45a171e229f7c8f02

68370 master J ✗1B ✗T –R -1/2−

LU-10026 csdc: enable zstd/zstdfast as compr type in layout

mine build failure (#130345)

1 unique failure csdc 1 unresolved M +107 −31

67d

build failure (#130345)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 1 other review

build #130345 FAILURE

no vote yet: Sebastien Buisson

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- Rewritten onto the current naming: only LL_COMPR_TYPE_ZSTDFAST=8 is new (zstd=7 already existed); the existing zstd table row gained its level converters (renamed to from_/to_zstd_level, from_/to_zstdfast_level); zstdfast row added before the MAX sentinel; wire asserts extended (ZSTDFAST==8, MAX 8->9 in both wiretest copies + wirecheck).
- The duplicate COMPR_LEVEL_MAX definition was removed while raising it to 100 (previously the header defined it twice; the values would have diverged).
- Man page: zstdfast added to the type list and level-range sentence (including correcting the lz4fast range to 1-26, matching the current mapping); zstd/zstdfast bullets added after lzo.
- sanity-pfl test_100b/test_100d switched to deriving the compression type lists from the negotiated compress_types in the OSC import, as in the original; adapted to the parametric test structure used by this series.
- The old-version gate around the zstd line in sanity.sh compress_content() is meaningless on master and was dropped (line unconditional).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: enable zstd/zstdfast as compr type in layout

The Zstandard compression algorithm shows good promise for both higher
compression than gzip, while also being faster.
Available compression levels for zstd are 1 through 22. In addition,
faster levels supported by zstdfast are 1-10, 20-100 (in increments of
10).

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9678
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Idc320a9039d2d974683339b8033b23736fe2a20f

68402 master J !B ✗T –R 0/2

LU-10026 csdc: enable compression on aarch64

mine build failure (#130374)

csdc 1 unresolved S +13 −19

67d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #130374 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- Removes the PAGE_SIZE-conditional default (llite_enable_compression now unconditionally 1) and the userspace PAGE_SIZE preview-warning blocks (which were dead in userspace, since PAGE_SIZE is not a preprocessor macro there).
- aarch64 always_except tags renumbered to LU-10026; one comment reworded to drop an internal change reference.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: enable compression on aarch64

This patch enables CSDC by default on aarch64 client
with >4k PAGE_SIZE.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6172e569c0ae5eb0b823f703c640847479909bdd

66274 master J !B ✗T –R 0/2

LU-10026 osc: don't do decomp. with compression disabled

mine build failure (#130267)

csdc 3 unresolved XS +3 −1

67d

build failure (#130267)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130267 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-05-28 20:13 Marc Vef COMMIT_MSG:15 — this is not helpful and needs to be updated

Marc Vef PS1 · 2026-05-28 20:13

this is not helpful and needs to be updated

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre COMMIT_MSG:9 — (minor) The body says decompress_request() has "a significant performance cost for checking if data is compressed", but it already returns i

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) The body says decompress_request() has "a significant performance cost for checking if data is compressed", but it already returns immediately when the first page's cp_compr_type is LL_COMPR_TYPE_NONE. For uncompressed files that check is cheap; the scan cost only applies to compressed data, which is the case that still needs decompressing. Could the justification be tightened, and the read/write asymmetry this creates (writes still compress off the layout while reads stop decompressing) be noted?

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre osc_request.c:2537 — (defect) This gates decompression on the global llite_enable_compression instead of the data's own cp_compr_type.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) This gates decompression on the global llite_enable_compression instead of the data's own cp_compr_type.

decompress_request() already returns early when pga[0]'s cp_compr_type is LL_COMPR_TYPE_NONE:

    type = clpage->cp_compr_type;
    /* no compression */
    if (type == LL_COMPR_TYPE_NONE)
        RETURN(0);

so uncompressed reads were never paying for a full scan.

The concern is a genuinely compressed read reaching here with the tunable off. lsm_lmm_verify_v1v3() only rejects a fresh compressed-layout unpack when compression is disabled; a cached lsm survives a runtime toggle. On read, set_page_compression()->__set_page_compression() still fills cp_compr_type from that cached lsme, and the write path still compresses off cp_compr_type regardless of the global.

After echo 0 > ...enable_compression on a client that already holds the compressed file's layout, does a cache-cold read here hand back the still-compressed bytes undecompressed?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: don't do decomp. with compression disabled

decompress_request likely has a significant performance
cost for checking if data is compressed, so we should not
call it when compression is disabled.

This is a stop-gap solution for the preview until we can
have the server tell the client if data is compressed, as
described in EX-7818.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iba8feba3ab0fe620d0594f59c2c6ddea25faeb4f

66265 master J !B ✗T –R 0/2

LU-10026 llite: allow mmap reads of compressed files

mine build failure (#130262)

csdc 4 unresolved M +107 −3

67d

build failure (#130262)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130262 FAILURE

no vote yet: Artem Blagodarenko

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-07-15 00:56 Gerrit AI review for Lustre llite_mmap.c:576 — (minor) `off` is computed at the call site as vma->vm_pgoff<<PAGE_SHIFT and passed in, but never used in the body. Drop the parameter, or us

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(minor) `off` is computed at the call site as vma->vm_pgoff<<PAGE_SHIFT and passed in, but never used in the body. Drop the parameter, or use it.

reply PS3 2026-07-15 00:56 Gerrit AI review for Lustre llite_mmap.c:587 — (suggestion) The block is keyed on the file open mode, so an fd opened O_RDWR that only creates a PROT_READ mapping is rejected with -EOPNOT

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(suggestion) The block is keyed on the file open mode, so an fd opened O_RDWR that only creates a PROT_READ mapping is rejected with -EOPNOTSUPP even though it never writes the file (a common open-O_RDWR-then-mmap-read pattern). Since read mmap of compressed files is supported, would keying on the mapping's write intent (vma->vm_flags VM_WRITE/VM_SHARED) rather than the open mode let those read-only mappings through while still avoiding the write SIGBUS?

reply PS3 2026-07-15 00:56 Gerrit AI review for Lustre llite_mmap.c:613 — (minor) The message names cl_io_rw_init(), but this path calls cl_io_init().

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(minor) The message names cl_io_rw_init(), but this path calls cl_io_init().

reply PS3 2026-08-26 11:52 Alex Zhuravlev multiop.c:585 — what do we do with this?

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(defect) These treat O_RDONLY/O_WRONLY/O_RDWR as bitmask flags, but the access mode is a 2-bit field. O_RDONLY is 0, so `fd_flags & O_RDONLY` is always 0, and for an O_RDONLY fd `fd_flags & O_RDWR` is also 0, so mmap_mode stays 0 (PROT_NONE) instead of PROT_READ.

Mask and compare instead:

    switch (fd_flags & O_ACCMODE) {
    case O_RDONLY: mmap_mode = PROT_READ; break;
    case O_WRONLY: mmap_mode = PROT_WRITE; break;
    case O_RDWR:   mmap_mode = PROT_READ | PROT_WRITE; break;
    }

Because of this, the `oSMc` case in test_460e maps PROT_NONE, so "readonly mmap should work" passes without ever exercising a readable mapping.

Alex Zhuravlev PS3 · 2026-08-26 11:52

what do we do with this?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: allow mmap reads of compressed files

mmap reads of compressed files work, so we should only
block writes.

We cannot block the actual fault operations because that
will cause the application to get a SIGBUS, so we check
the file open mode when we go to create the memory mapping.

llite: disable mmap on compressed files

disable mmap(2) on compressed files until well tested.

DDN-bug-id: EX-6265
Was-Change-Id: I3464a03b16708edcd0692bc9db337eb8473ea047

llite: use CIT_MISC in check_compression

ll_mmap_check_compression uses a CIT_FAULT io type and
initializes it with cl_io_rw_init.  This doesn't
initialize the IO correctly and this sometimes results
in the following crash during io_fini, because the ft_page
pointer is initialized to something else by cl_io_rw_init:

BUG: unable to handle kernel NULL pointer dereference
at 000000000000004a
RIP: 0010:cl_pagevec_put+0x9f/0x3a0 [obdclass]
 vvp_io_fault_fini+0x21/0x40 [lustre]
 cl_io_fini+0x7a/0x230 [obdclass]
 ll_mmap_check_compression+0x403/0x540 [lustre]

DDN-bug-id: EX-7812
Was-Change-Id: Ib1f270f071370d6c045bb9d799ab5b7b41a6c6be

Test-Parameters: forjanitoronly
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I636f398fd247ddcd153f94bf8116440540e8469c

66268 master J !B ✗T –R 0/2

LU-10026 utils: error for invalid compression level

mine build failure (#130265)

csdc 3 unresolved M +78 −45

67d

build failure (#130265)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130265 FAILURE

no vote yet: Sebastien Buisson

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre COMMIT_MSG:9 — The message (and the sanity-pfl/sanity test updates) cover only lzo, but the new error path also fires for zstd, since it likewise has no le

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

The message (and the sanity-pfl/sanity test updates) cover only lzo, but the new error path also fires for zstd, since it likewise has no level mapping. Worth naming zstd here and adding coverage for it, or clarifying why zstd is excluded from the documented 1-21 level range.

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase notes:

- The base meanwhile added zstd to the compression type table (no level
  converter). Since this patch makes an explicit level on a no-converter
  type an error, zstd was added to test_100d's type_nolvl list next to lzo,
  and the compress-file creation now passes -S4M so the six doubling chunk
  sizes (64K..2M) stay below the stripe size (the chunk size is clamped to
  the stripe size, which would break the per-chunk find counts).
- test_100d's chunk-verification loop now iterates all types with a
  doubling chunk size (the previous patchset's loop only ran one
  iteration).

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre liblustreapi_layout.c:1723 — (defect) This whitelists only fast/best to carry a level; every other type with a NULL ctn_from_compr_level falls through to the error. That

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) This whitelists only fast/best to carry a level; every other type with a NULL ctn_from_compr_level falls through to the error. That is lzo (intended) but also zstd. So `lfs setstripe -Z zstd:5` now fails, whereas pre-patch it stored the level. lfs-setstripe.1:232 (added in this same LU-10026 series) says zstd takes a level 1-21. Is the zstd rejection intended? If zstd should not take a level, the man page needs updating too; if it should, zstd wants a from_compr_level mapping rather than the error path.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: error for invalid compression level

Not all compression types accept a compression level (e.g. lzo in its
kernel implementation). For those, return an error if a compression
level is provided on 'lfs setstripe' command-line.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6c367c4bfd76cc81c890a89ba9f994f4fd9f4f80

68349 master J ✗2B ✗T –R +0/2−

LU-10026 osc: limit compression chunks in RPC

mine build failure (#130327)

2 unique failures csdc 2 unresolved L +283 −22

67d

build failure (#130327)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

1 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 10 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 10 other reviews

build #130327 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:31

no vote yet: Artem Blagodarenko, Zhenyu Xu

patchsets: PS1 08-25 PS2 08-26

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- try_to_add_extent_for_io()/get_write_extents() gained their env parameter
  on top of master's reworked extent-RPC code: the erd_max_compr_chunks
  member was added to the write_rpc_def()/read_rpc_def() initializer macros
  (which master introduced since), with the macros now taking the osc
  object.
- Page indices in the chunk-counting loops use osc_index() - struct
  brw_page no longer has a bp_page member on master (folio-based).
- The RPC-fragment limiting in the submit path is applied in
  osc_io_submit() only; master has since split direct-IO submission into
  osc_dio_submit(), which keeps its existing behaviour (DIO on compressed
  files is restricted to chunk-aligned IO anyway).
- cp_compr_chunk_log_bits -> cp_compr_chunk_lum_bits (series field naming).
- Fixed a short memcpy in the chunk-ledger growth helper
  (expand_osc_compr_chunk_array copied the element count as bytes,
  losing 3/4 of the saved entries on expansion).
- The new sanity-compr test_1016 gained a compression_supported skip guard
  (it had none).

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) osc_io.c:138 — error: osc_io_submit():'osc' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

error: osc_io_submit():'osc' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: limit compression chunks in RPC

Limit the number of partial compressed chunks that can be sent to
the server in a single RPC, so that it doesn't need to allocate
buffers for read-modify-write on a huge number of chunks at once.
It is better to handle that in separate server threads.

In the worst case, each extent in the BRW RPC belongs to a different
compression chunk, so the worst_case_chunks would expand on the OSS
to worst_case_chunks * compression_chunk_size bytes IO on the server.
That might be hundreds of MB in size per thread and LASSERT or OOM.

Add tunable osc.*.compress_rpc_fragment_ratio to reduce the maximum
compressed chunks by the given factor, its value ranges in [1, 32].

osc: scan extents to find compression chunks

as we can't trust oe_start and oe_end, we have to scan all the
pages covered by extent and so find all compression chunks to
estimate and limit potential read-modify-write IO on OST.

osc_submit_io() should not create too big extents we can't send
due to limits for compressed chunks.

DDN-bug-id: EX-10141
Was-Change-Id: I00b104ded6cdb8e66414eb2f759f6dca1b7b4d11

osc: make oti_compr_chunks ledger expandable

This patch makes osc_thread_info::oti_compr_chunks an expandable
array.

DDN-bug-id: EX-10361
Was-Change-Id: Ib2ad5407815c733687eadd4a46014d0a694dbf41

osc: expand_osc_compr_chunks_array() use atomic alloc

This can be called in "spinlock" context and generate a stack dump:

  BUG: sleeping function called from invalid context
  in_atomic(): 1, irqs_disabled(): 0, name: ptlrpcd_00_02
  Call Trace:
    ___might_sleep.cold.23+0x104/0x118
    kmem_cache_alloc_trace+0x1cd/0x2a0
    expand_osc_compr_chunk_array+0xc6/0x5d0 [osc]
    try_to_add_extent_for_io.isra.10+0x11f6/0x16b0 [osc]
    osc_io_unplug0+0x6a4/0x1b50 [osc]
    brw_queue_work+0x29/0xb0 [osc]
    work_interpreter+0x2b/0x100 [ptlrpc]
    ptlrpc_check_set+0x583/0x2fe0 [ptlrpc]
    ptlrpcd_check+0x511/0x570 [ptlrpc]
    ptlrpcd+0x1b5/0x4b0 [ptlrpc]

DDN-bug-id: EX-10361
Was-Change-Id: I72f30b5185be0417c1bb7170ac278f48b04e7d0b

osc: revert to correct cl_max_extent_pages

otherwise grant overhead calculation breaks.

DDN-bug-id: EX-10267
Was-Change-Id: I4e74524b17c7834c432a708bb917f291fd773023

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10047
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4d0b4bdaaa91f46147d2ca1c52741117533f8f7a

68355 master J ✗2B ✗T –R +1/2

LU-10026 ofd: make oti_chunk_bufs expandable

mine build failure (#130333)

2 unique failures csdc M +60 −70

67d

build failure (#130333)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 4 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130333 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 09:47

patchsets: PS1 08-25 PS2 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: make oti_chunk_bufs expandable

to be able to handle very large IO with lots of chunks.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10394
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8ad24acd61037cf2804612940cdb74662c0c5303

68373 master J –B ✗T –R +0/2−

LU-10026 csdc: stop compressing incompressible file data

mine build failure (#130347)

csdc 2 unresolved L +413 −17

67d

build failure (#130347)

1 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

build #130347 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 12:21

no vote yet: Zhenyu Xu

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

ci_incompressible placed in the relocated cl_io bitfield next to ci_size_extend_compression; test_1021 gate converted to compression_supported.

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) osc_io.c:566 — error: osc_io_iter_init():'osc' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

error: osc_io_iter_init():'osc' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: stop compressing incompressible file data

The reduced_ratio (original_size/compress_reduced_size) represents
the minimum fraction of pages that are compressed out of each chunk,
namely the compressed chunk needs to shrink by at least
1/reduced_ratio blocks for it to be "compressible".

Let size compression_ratio be defined as
original_size/after_compression_size, so

reduced_ratio = compression_ratio / (compression_ratio - 1)

and we set its default value to 16, equivalent to 1.07 of compression
ratio (i.e. needs to shrink at least one 4KB block out of each 64KB
chunk).

After every compress_check_bytes of data being compressed, file's
compressibility would be re-calculated based on average
compress_reduced and average compress_orig data size.

Stop compressing file data if it is deemed to be incompressible, and
after compress_skip_bytes data have been written uncompressed , retry
the file compressibility check.

compress_reduced_ratio, compress_check_bytes, compress_skip_bytes
are tunable parameters:
	osc.*.compress_reduced_ratio
	osc.*.compress_check_bytes
	osc.*.compress_skip_bytes
their default values are 16, 1M and 32M respectively.

tests: use /dev/zero as compressible source

in test 1021 as hdf file has few incompressible chunks and this
breaks the test infrequently.

DDN-bug-id: EX-10857
Was-Change-Id: Iba29d9e654a245d3c0cbb14a8b7dbb9f847f52d5

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4ce3d752c67f18ba7b100c72a2bb61a91258c6e8

68409 master J –B ✗T –R -1/2−

LU-10026 csdc: tunable to disable compression on fallocate

mine build failure (#130381)

csdc 1 unresolved L +238 −12

67d

build #130381 FAILURE

no vote yet: Sebastien Buisson

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- The connect-time obd_set_info_async(KEY_FALLOCATE_COMPR) block was merged into the current client_common_fill_super() flow (placed before cl_sb_init; the error path uses this function's current label and puts the pending request like its sibling paths).
- test_1031's version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: tunable to disable compression on fallocate

fallocate is denied for compressed files because preallocating space
for a compressed file makes the compression useless. But sometimes it
could be preferable to deactivate compression and honor fallocate
instead.
So add a tunable for disabling compression on fallocate, 0 by default:
llite.*.prefer_fallocate_over_compr
Note disabling compression is possible only if none of the components
have been already initialized.

Add sanity-compr test_1031 to exercise this.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10053
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3f272f39166287cd459ebcfbb9bd61ed0fefd30f

68405 master J !B ✗T –R 0/2

LU-10026 cdsc: update layout compressibility

mine build failure (#130377)

csdc 4 unresolved L +335 −54

67d

build failure (#130377)

3 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130377 FAILURE

no vote yet: Zhenyu Xu, Alex Zhuravlev

threads: 3 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- OBD_CONNECT2_UPDATE_LAYOUT was already reserved on master at the same bit; only the MDT_CONNECT_SUPPORTED2 and client connect lists needed the addition.
- ci_need_change_layout added to the current cl_io bitfield; CAT_COMPRESSIBLE handling merged into the current vvp_attr_update().
- mdd_layout_update_pccro was renamed/extended per the patch on the current (retry-aware) master form.
- FORMATTED_BUF_LEN (used by the new getstripe formatting) is defined locally - the source branch carries it from an unrelated change.
- test_1021 layout-flag assertions applied unconditionally (the original wrapped them in version gates).

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:297 — error: vvp_layout_minor_change():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_layout_minor_change():'vob' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:429 — error: vvp_io_fini():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_io_fini():'vob' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:1973 — error: vvp_io_init():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_io_init():'vob' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 cdsc: update layout compressibility

When compressing write changes the file's compressibility, it updates
the file layout flags so that its compressibility persists.

lod: reset la_attr at declare

to avoid unplanned local attributes update during minor layout change.

DDN-bug-id: EX-10786
Was-Change-Id: I093efeb9cf110adfe6a7590091e60b1e9f1eb9ed

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia2964817d1b5fee2d84ef15e9f1036882d70adb2

66284 master J –B ✗T –R -1/2−

LU-10026 csdc: handle data compression on MDT

mine build failure (#130318)

csdc 4 unresolved S +39 −1

67d

build failure (#130318)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

build #130318 FAILURE

no vote yet: Zhenyu Xu

patchsets:

+2 earlier

PS1 05-28 PS2 06-11

PS3 06-11 PS4 06-11 PS5 06-11 PS6 08-25 PS7 08-26

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS5 2026-07-15 00:57 Gerrit AI review for Lustre COMMIT_MSG:9 — (minor) The body explains the DoM+compress block, but two hunks pull the other way and aren't mentioned: lov_pattern_available() gains LOV_P

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(minor) The body explains the DoM+compress block, but two hunks pull the other way and aren't mentioned: lov_pattern_available() gains LOV_PATTERN_MDT | LOV_PATTERN_COMPRESS, and llapi_pattern_to_lov() gains MDT|COMPRESS and RAID0|COMPRESS cases. Both make the combination more acceptable, not less. Could the message cover why they belong in a patch whose stated purpose is to disallow it?

reply PS5 2026-07-15 00:57 Gerrit AI review for Lustre lustre_user.h:837 — (defect) This makes the server accept an MDT|COMPRESS component (lod_lov.c / lov_ea.c call lov_pattern_available()), yet lov_pattern_support

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(defect) This makes the server accept an MDT|COMPRESS component (lod_lov.c / lov_ea.c call lov_pattern_available()), yet lov_pattern_supported() still excludes it, so client IO marks such an entry invalid (lov_object.c sets lle_valid = 0). The only guard against creating it lives in lfs.c; a liblustreapi caller doing llapi_layout_pattern_set(MDT) + llapi_layout_compress_set() bypasses that and produces a layout the server stores but IO cannot use. If DoM+compress is unsupported, should the block live in the library/server rather than only the CLI?

reply PS5 2026-07-15 00:57 Gerrit AI review for Lustre lfs.c:3459 — (defect) errno is set to a negative value here; every other error path in comp_args_to_layout() uses errno = EINVAL. A negative errno breaks

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(defect) errno is set to a negative value here; every other error path in comp_args_to_layout() uses errno = EINVAL. A negative errno breaks strerror(errno)/perror on this -1 return. Should be errno = EINVAL.

reply PS5 2026-07-15 00:57 Gerrit AI review for Lustre liblustreapi_layout.c:831 — (minor) These two cases look unreachable. The COMPRESS bit never reaches llc_pattern: llapi_layout_pattern_set() rejects any pattern carryin

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(minor) These two cases look unreachable. The COMPRESS bit never reaches llc_pattern: llapi_layout_pattern_set() rejects any pattern carrying it, and llapi_layout_from_lum() stores compression in llc_compr_type, not the pattern. The bit is OR'd into lmm_pattern separately in llapi_layout_to_lum() (blob->lmm_pattern |= LOV_PATTERN_COMPRESS). Was a real caller intended, or can these be dropped?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS7 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: handle data compression on MDT

Do not support setting data compression component on DoM until
data compression on MDT implemented.

DDN-bug-id: EX-7806
Was-Change-Id: I3794460140f08a073377c418dd56e7dda907d96d

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8b5c4d5fd18033e796b1c7555c37251e4f7f9a49

68400 master J –B ✗T –R -1/2−

LU-10026 tests: skip 1000d if not enough OSTs

mine build failure (#130372)

csdc 1 unresolved XS +1 −0

67d

build #130372 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

OSTCOUNT gate added after the converted compression_supported gate.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip 1000d if not enough OSTs

sanity-compr/1000d needs 2 OSTs at least

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10785
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I93d7a2c0178549980348ab41c2fcb62669571f6a

68398 master J !B ✗T –R 0/2

LU-10026 tests: one more compressed truncate

mine build failure (#130370)

csdc 1 unresolved S +14 −0

67d

janitor run: 21 failing config(s), none unique to this patch — janitor results

build #130370 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

The version guard inside the new truncate case (guarding a fix contained in this series) was dropped; its bug-reference comment renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: one more compressed truncate

check one more scenario of compressed file's truncation.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10391
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I186b92f36dd93afe943ea8f87a6e334f73fd9d26

68396 master J !B ✗T –R 0/2

LU-10026 osc: mem leak in compress_request

mine build failure (#130368)

csdc 1 unresolved M +70 −13

67d

janitor run: 25 failing config(s), none unique to this patch — janitor results

build #130368 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

Applied cleanly; test_2002's client-version gate converted to compression_supported (dropped again later in the series once the suite gained file-scope gates). The commit message's review pointer was generalized.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: mem leak in compress_request

1. Fix memory leak in compress_request.
2. In case of ENOMEM compress_request should return 0
if it has been able successfully compress at least
one chunk.

osc: fix OBD_FAIL_OSC_COMPR_ENOMEM

Set cfs_fail_loc to OBD_FAIL_OSC_COMPR_ENOMEM to
always fail with ENOMEM after skipping fail_val times.

Several cosmetic changes in sanity-compr test_2002 to address
review comments on an earlier version.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11120
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I745e17978d885d40218efe13e68e7e41b4b33d31

68395 master J !B ✗T –R 0/2

LU-10026 tests: check that all chunks are compressed

mine build failure (#130367)

csdc 1 unresolved S +35 −0

67d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130367 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The osc half of the original change ("osc: improve can_merge_pages()") is already on master as 366fe06822, so this ports only the test remainder under a new Change-Id, with the original kept as Was-Change-Id.
- Adds test_2001; its client-version gate (a fix contained in this series) converted to compression_supported. test_1030 is left in place: the series' final state keeps both tests.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: check that all chunks are compressed

Add sanity-compr test_2001, which streams a large sequential
write to a compressed file and verifies from the client-side
compression stats that (nearly) every chunk was written
compressed, i.e. that the OSC forms full compressible chunks.

This exercises the can_merge_pages() behaviour that landed on
master in commit 366fe06822 ("LU-10026 osc: improve
can_merge_pages()"): pages with different OBD_BRW_ASYNC /
OBD_BRW_SOFT_SYNC hints must still merge into full chunks.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11063
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I6dd2643c93739fd6abaeb7add27f530a57fdcf64
Change-Id: Ice73943d4a2c297c1a95e0754b75b1f157df6c6f

68393 master J ✓B ✗T –R 0/2

LU-10026 tests: disable 1081 in sanity-compr

mine build failure (#130365)

csdc 1 unresolved XS +2 −0

67d

build #130365 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: disable 1081 in sanity-compr

causing too many failures.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10077
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2b389c2ae2c733c3e6b8b100cdd0ca84df303246

68388 master J –B ✗T –R -1/2−

LU-10026 ofd: don't increment csdc counters

mine build failure (#130360)

csdc M +64 −21

67d

build #130360 FAILURE

no vote yet: Alex Zhuravlev

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: don't increment csdc counters

if corresponding event doesn't happen on the server.

add few more tests for compression counters.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10647
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id9badab250a3337d6a968680ce37e41e74b9c269

68387 master J !B ✗T –R 0/2

LU-10026 ofd: persistent compression counters

mine build failure (#130359)

csdc 10 unresolved L +333 −1

67d

build failure (#130359)

9 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130359 FAILURE

no vote yet: Alex Zhuravlev

threads: 9 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- Context-only adaptation: struct ofd_device gained members on master (new fields appended at the end), and the include list and module-init surroundings in ofd_dev.c differ.
- test_1123 gained a compression_supported gate (the original had none; without it the test would run and fail on servers without compression).
- lprocfs_load_stats/lprocfs_save_stats land in lprocfs_status_server.c as in the original; both EXPORT_SYMBOL'd (used from ofd.ko).

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1618 — warn: lprocfs_save_stats():passing zero to 'ERR_CAST'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():passing zero to 'ERR_CAST'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1629 — error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1630 — warn: lprocfs_save_stats():possible memory leak of 'buf'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():possible memory leak of 'buf'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1641 — error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1672 — warn: lprocfs_save_stats():possible memory leak of 'buf'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():possible memory leak of 'buf'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1716 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1723 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1737 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1750 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: persistent compression counters

ofd to save compression stats every hour and at umount,
then to load the saved stats at mount time so compression
stats can be found persistent and repsresenting overall
compression usage.
the file is called "compr_stats", stored in the root dir
of underlaying fs and uses YAML format:
- { name: uncompressed_objects, count:0, min:92, max:0, sum:0 }
- { name: compressed_objects, count:1, min:92233, max:0, sum:0 }
- { name: compr_chunks_read_disk, count:0, min:9223, max:0, sum:0 }
- { name: compr_bytes_read_disk, count:0, min:92233, max:0, sum:0 }
- { name: compr_bytes_read_user, count:0, min:92233, max:0, sum:0 }
- { name: compr_chunks_read_decompress, count:0, min:9, max:0, sum:0 }
- { name: compr_bytes_read_decompresse, count:0, min:9, max:0, sum:0 }
- { name: compr_chunks_write_disk, count:4, min:1, max:8, sum:11 }
- { name: compr_bytes_write_disk, count:4, min:8192, max:1310, sum:20}
- { name: compr_bytes_write_user, count:4, min:8192, max:10486, sum:1}

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10130
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I59016c473607fe7e4aa2583c28dfc223bc3d9b7c

68386 master J !B ✗T –R 0/2

LU-10026 ofd: support for compression counters

mine build failure (#130358)

csdc 1 unresolved L +257 −38

67d

janitor run: 21 failing config(s), none unique to this patch — janitor results

build #130358 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The two ofd_compress.c hunks around is_chunk_start() re-applied against the folio form of the call sites.
- test_1122 anchored after test_1121; its version gate converted to compression_supported.
- Commit message reworded to drop downstream release-version references in the two test-skip sub-sections.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: support for compression counters

the patch adds additional compression related counters:
compr_bytes_read_disk -  bytes were read from disk
compr_chunks_read_disk -  chunks were read from disk
compr_bytes_read_user -  bytes were sent to clients
compr_bytes_read_decompressed -  bytes were sent after decompression
compr_chunks_read_decompressed -  chunks were decompressed
compr_bytes_write_disk -  bytes were sent to disk
compr_chunks_write_disk -  chunks were sent to disk
compr_bytes_write_user -  bytes clients send to server
compr_chunks_rmw -  chunks were involved in read-modify-write
compr_bytes_rmw -  disk bytes were involved in read-modify-write
compr_bytes_rmw_user -  bytes clients sent causing read-modify-write
decompression_time - time in usec spent in decompression

tests: skip sanity-compr/1122 with old OST

skip sanity-compr test_1122 when the OST does not provide the
server-side compression counters.

DDN-bug-id: EX-10586
Was-Change-Id: I2f5608e93880ad6569d68e83110144bfcf791f80

tests: adjust the sanity-compr/1122 version check

adjust the version check for the server-side counters.

DDN-bug-id: EX-10890
Was-Change-Id: I4f58cac84a7c7dcbec667292528ae4667396f1f0

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10128
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie974cbd15451ae790310ec8e3d5c729a849c882a

68376 master J ✗1B ✗T –R -1/2−

LU-10026 tests: skip sanity-pfl tests if no compression

mine build failure (#130350)

1 unique failure csdc 1 unresolved S +17 −29

67d

build failure (#130350)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #130350 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Re-derived by hand - every hunk's context had version gates this series already replaced with compression_supported. Each test_100* gained 'compression_enabled || skip' and lost its per-test llite.*.enable_compression save/set/restore block (100a keeps a bare enable, 100e keeps its disable + re-enable trap), relying on the default-on behaviour from the finer-grained enable_compression control patch.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip sanity-pfl tests if no compression

Skip sanity-pfl test_100* for servers that do not understand CSDC.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9482
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ib3fcfd77e9e7ffb122ed6ade9015b02d42ea8319

66276 master J !B ✗T –R 0/2

LU-10026 lov: fallocate is not allowed for compressed files

mine build failure (#130268)

csdc 4 unresolved S +30 −0

68d

build failure (#130268)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130268 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre COMMIT_MSG — The commit message only describes disabling fallocate, but the diff also adds a `rsync --sparse` decompression round-trip (the `decomp2` / "

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

The commit message only describes disabling fallocate, but the diff also adds a `rsync --sparse` decompression round-trip (the `decomp2` / "fiomap case" block) to compress_type() in sanity.sh. That change is about the sparse-copy/FIEMAP read path, not fallocate, and isn't mentioned here. Is it meant to be part of this patch, or should it be split into its own change with its own Change-Id?

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre lov_io.c:1005 — (suggestion) This only checks lsm_entries[0]. lsme_compr_type is set per component (lov_ea.c copies lcme_compr_type for each LOV_PATTERN_COM

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(suggestion) This only checks lsm_entries[0]. lsme_compr_type is set per component (lov_ea.c copies lcme_compr_type for each LOV_PATTERN_COMPRESS entry), so a composite/PFL file could have an uncompressed component 0 and a compressed later component. Would fallocate over such a file still be allowed even though it targets a compressed component? (The ll_fiemap() guard has the same entry[0]-only limitation.)

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre lov_io.c:1006 — (minor) `lio` and `loo` here just re-derive values already in scope: the outer `lio` (declared at the top of the function) is the same `cl2l

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) `lio` and `loo` here just re-derive values already in scope: the outer `lio` (declared at the top of the function) is the same `cl2lov_io(env, ios)`, and `loo` is `lio->lis_object`. The inner `struct lov_io *lio` also shadows the outer one. This could just use the existing `lio->lis_object` without the extra declarations.

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre lov_io.c:1009 — (minor) fallocate(2) is directly user-triggerable, so a CWARN here lets an unprivileged caller flood the console by repeatedly calling fallo

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) fallocate(2) is directly user-triggerable, so a CWARN here lets an unprivileged caller flood the console by repeatedly calling fallocate on a compressed file. The equivalent compressed-file guard in ll_fiemap() (lov_object.c around line 2021) returns -EOPNOTSUPP silently. Consider CDEBUG(D_VFSTRACE, ...) or a rate-limited message instead.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: fallocate is not allowed for compressed files

Client Side Data Compression allocates blocks after a compression.
It is impossible to preallocate blocks for the whole file, so
fallocate should be disabled in case of compression.

Test-Parameters: forjanitoronly
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie834ace183fdcec0d7d6f747237e0964c3c4120b

66264 master J –B ✗T –R 0/2

LU-10026 man: document supported compression types

mine build failure (#130260)

csdc 3 unresolved S +16 −0

68d

build failure (#130260)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

build #130260 FAILURE

no vote yet: Sebastien Buisson

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: the lfs-setstripe.1 sentence this patch anchors on was
reformatted on the base (.B "lfs getstripe" roff style); the
compression-type list is inserted after the reformatted sentence, content
unchanged.

reply PS3 2026-07-15 00:56 Gerrit AI review for Lustre lfs-setstripe.1:236 — (suggestion) This new list of currently supported types omits zstd (and fast/best), but the untouched line just above still advertises "1-21

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(suggestion) This new list of currently supported types omits zstd (and fast/best), but the untouched line just above still advertises "1-21 for zstd" as a valid level range. If zstd is not currently supported, consider reconciling the two so the level-range hint doesn't contradict this list.

reply PS3 2026-07-15 00:56 Gerrit AI review for Lustre lfs-setstripe.1:242 — (defect) Is the lz4 default really 1? When no level is given, llapi_parse_compress_type() passes level = -1, and from_lz4_level(-1) returns

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(defect) Is the lz4 default really 1? When no level is given, llapi_parse_compress_type() passes level = -1, and from_lz4_level(-1) returns LZ4HC_DEFAULT_CLEVEL - 1 = 8, which lfs getstripe maps back via to_lz4_level() to 9. So the effective default looks like 9 (highest ratio / slowest), not 1. The LZ4_ACCELERATION_DEFAULT = 1 case only applies when an explicit level < 3 is given, not for the unspecified default. Compare gzip just above, where the documented default of 6 does match from_gzip_level(-1).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 man: document supported compression types

Update lfs-setstripe to mention the currently supported compression
types: gzip, lz4, lz4fast and lzo.

Test-Parameters: forjanitoronly
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9b587516d27b882bec5855da4948f489e5a0041f

65762 master J ✗1B ✓T ✗1R 0/2

LU-10026 csdc: prefer uncompressed mirror for read and write

mine janitor: 1 test failure unique to this patch — sanity3@zfs:test_907

1 unique failure csdc 4 unresolved L +213 −47

72d

janitor: 1 test failure unique to this patch — sanity3@zfs:test_907

enforced failed: review-dne-selinux-ssk-part-2

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity3@zfs:test_907	seen in 57 other reviews

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #129912 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Andreas Dilger, Artem Blagodarenko, Zhenyu Xu

patchsets:

+7 earlier

PS1 05-08 PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 07-27 Artem Blagodarenko PS7 07-28 Artem Blagodarenko

PS8 08-19 Artem Blagodarenko PS9 08-19 Artem Blagodarenko PS10 08-20 Artem Blagodarenko PS11 08-20 Artem Blagodarenko PS12 08-21 Artem Blagodarenko

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS12 2026-08-22 12:56 Gerrit AI review for Lustre lod_lov.c:622 — This bonus is added once per allocated stripe, which is exactly the weight the OS_STATFS_NONROT bonus a few lines above uses, so the two can

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

This bonus is added once per allocated stripe, which is exactly the weight the OS_STATFS_NONROT bonus a few lines above uses, so the two can cancel out.

Two mirrors, no LCME_FL_PREF_WR anywhere, one stripe each:

    mirror 0: compressed,   stripe on a NONROT OST -> pref = 1
    mirror 1: uncompressed, stripe on a rotating OST -> pref = 1

lme_preference ties, so the `!found_preferred` fallback below keeps the lower index (`>` is strict) and sets lme_prefer on mirror 0. lod_primary_pick() then classifies mirror 0 as case B (preferred + compressed) and mirror 1 as case C, and B outranks C, so the write goes to the compressed mirror.

A wider compressed mirror wins outright rather than tying: 4 NONROT stripes compressed = 4 vs 1 NONROT stripe uncompressed = 2.

Mixed flash/disk mirrors are the case the non-rotational preference was added for, so this is not an exotic layout. Should the uncompressed bonus be on a different scale than the per-stripe NONROT count (or applied once per mirror) so it can't be traded away against OST speed?

The comment also says "mirror", but this sits in the per-stripe loop of one component.

reply PS12 2026-08-22 12:56 Gerrit AI review for Lustre lod_object.c:8060 — (style) Not a bug, but this comment has a UTF-8 em dash in it; the rest of the tree is plain ASCII. A plain '-' would do.

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

(style) Not a bug, but this comment has a UTF-8 em dash in it; the rest of the tree is plain ASCII. A plain '-' would do.

reply PS12 2026-08-22 12:56 Gerrit AI review for Lustre lod_object.c:8088 — Ranking B (preferred + compressed) above C (uncompressed) reads as "an explicit `lfs setstripe --flags=prefer` beats the compression heurist

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

Ranking B (preferred + compressed) above C (uncompressed) reads as "an explicit `lfs setstripe --flags=prefer` beats the compression heuristic". But lme_prefer is not only user intent: when no component carries LCME_FL_PREF_WR, lod_fill_mirrors() picks the highest lme_preference non-stale mirror and sets lme_prefer = 1 on it.

So tier B is reachable on a file the user never flagged, and the heuristic ends up overriding its own compression preference. Should case B be restricted to mirrors that actually carry LCME_FL_PREF_WR, or should the synthesized preference be tracked separately from the user flag?

reply PS12 2026-08-22 12:56 Gerrit AI review for Lustre lov_io.c:662 — (minor) ci_ndelay_tried > 0 means "not the first lov_io_mirror_init() for this env", which is broader than "this is an FLR error retry".

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

(minor) ci_ndelay_tried > 0 means "not the first lov_io_mirror_init() for this env", which is broader than "this is an FLR error retry".

ll_file_io_generic() restarts the io for reasons unrelated to mirror failure and preserves the counter:

    partial_io -> io->ci_need_restart = 1   (file.c)
    retried = io->ci_ndelay_tried; goto restart;

A single buffered read larger than max_io_bytes therefore takes the uncompressed mirror for the first chunk only; from the second chunk on ci_ndelay_tried is non-zero, lis_mirror_index has been advanced by one, and the first in-range mirror is accepted even when it is compressed. The layout-change restart in vvp_io_iter_init() has the same shape.

Would gating on something that only the -EAGAIN mirror-failover path sets (io->ci_tried_all_mirrors, or a dedicated flag set where cl_io_loop() retries) keep the failover fix without giving up the preference on ordinary io continuation?

tags: csdc

owner Marc Vef · uploader Artem Blagodarenko · PS12 uploaded 73d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: prefer uncompressed mirror for read and write

When accessing a mirrored file with both compressed and uncompressed
components, choose uncompressed components for read (lov_io.c).

When writing to mirrored files with both compressed and uncompressed
mirrors, prefer the uncompressed components to write; this is better
for performance, more compatible with older clients, and better fits
the model of compressing files after initial write. The write-side
preference is implemented via lod_fill_mirrors() and lod_primary_pick()
using a new lme_compressed flag.

Note: the changes to lod_fill_mirrors() and lod_primary_pick() affect
all FLR files, not only those with compressed mirrors. Plain FLR writes
will also use the updated preferred-mirror selection logic.

DDN-bug-id: EX-6510
Was-Change-Id: I62a117d5cc3d34e2c0c96d1a9ade8eef0a2d1291

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@gmail.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I043b27bd891c039901075a08c76630f8f0f9f182

65761 master J !B ✓T ✗1R 0/2

LU-10026 utils: add 'lfs find' support for compressed file

mine enforced failed: review-dne-selinux-ssk-part-2

csdc 9 unresolved L +715 −67

72d

enforced failed: review-dne-selinux-ssk-part-2

9 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 4 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #129911 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Andreas Dilger, Artem Blagodarenko, Zhenyu Xu

patchsets:

+12 earlier

PS1 05-08 PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 06-17 Artem Blagodarenko PS7 06-18 Artem Blagodarenko PS8 07-24 Artem Blagodarenko PS9 07-27 Artem Blagodarenko PS10 07-27 Andreas Dilger PS11 07-27 Andreas Dilger PS12 07-27 Andreas Dilger

PS13 08-19 Artem Blagodarenko PS14 08-19 Artem Blagodarenko PS15 08-20 Artem Blagodarenko PS16 08-20 Artem Blagodarenko PS17 08-21 Artem Blagodarenko

threads: 9 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre COMMIT_MSG:9 — (minor) The first three bullets read as new functionality, but `--comp-flags=[^]compress` and `--comp-flags=[^]nocompr` already work (LCME_F

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(minor) The first three bullets read as new functionality, but `--comp-flags=[^]compress` and `--comp-flags=[^]nocompr` already work (LCME_FL_COMPRESS/LCME_FL_NOCOMPR are in comp_flags_table[]), and so does `-L compress` (LOV_PATTERN_COMPRESS is in lov_pattern_names[]). This patch documents them and adds test coverage rather than adding them. Could the wording say "document" for those three, so the message matches the diff?

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre lustre_user.h:3278 — (style) COMPR_LEVEL_MAX is already defined with the same value earlier in this header, just after `enum ll_compr_type`, with a comment expla

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) COMPR_LEVEL_MAX is already defined with the same value earlier in this header, just after `enum ll_compr_type`, with a comment explaining it. Two definitions can drift apart later; can this one be dropped?

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre sanity-pfl.sh:3078 — (style) extra blank line. Also z1..z4 here, and `expect` in test_100d, are not declared `local`.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) extra blank line. Also z1..z4 here, and `expect` in test_100d, are not declared `local`.

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre lfs.c:4637 — (style) not a bug, but `compress-chunk` already appears in this long_opts[] further down, so it is listed twice now. getopt_long takes the f

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) not a bug, but `compress-chunk` already appears in this long_opts[] further down, so it is listed twice now. getopt_long takes the first match so it still works; `compr-chunk` looks like the only new alias actually needed here.

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre lfs.c:8226 — (typo) this comment uses a UTF-8 em dash; the tree is otherwise plain ASCII. Same in the `%LZ` directory-branch comment in liblustreapi_pfin

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(typo) this comment uses a UTF-8 em dash; the tree is otherwise plain ASCII. Same in the `%LZ` directory-branch comment in liblustreapi_pfind.c.

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre lfs.c:8248 — (defect) atoi() cannot report a parse failure, so `--compr-level=abc` silently becomes level 0, and `--compr-level=+abc` becomes "level > 0"

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) atoi() cannot report a parse failure, so `--compr-level=abc` silently becomes level 0, and `--compr-level=+abc` becomes "level > 0", i.e. every compressed component. The unsigned range check above cannot catch it.

`-Z gzip:abc` is rejected by llapi_parse_compress_type(), and the setstripe `--compress-level` path uses strtol() with an end-pointer check, so the three spellings of the same thing disagree. strtol() plus `*end != '\0'` here would line them up.

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre liblustreapi_pfind.c:509 — (minor) This helper backs every numeric `lfs find` predicate, and the new exact-match branch only changes behaviour when `margin == 0`. `lfs

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(minor) This helper backs every numeric `lfs find` predicate, and the new exact-match branch only changes behaviour when `margin == 0`. `lfs find -z N` with a plain number is such a caller: lfs.c does `fp_ext_size_units /= SEL_UNIT_SIZE`, so 1/1024 becomes 0, and an exact `-z` match that previously reported "no match" now matches. That may well be desirable, but it is a silent change to an unrelated option.

The new compression predicates are the only callers passing margin 0; the rest pass 1 or a unit size, and with margin 1 this function already returns 1 on an exact match. Passing 1 there would avoid touching the shared helper, and would also make `--compr-level=-N` / `--compr-chunk=-N` mean "< N" like `-c -N`, `--size -N` and friends, rather than "<= N".

The table above the function (rows 5 and 6, and the note about the interval `(limit - margin, limit]`) no longer describes the code either.

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre liblustreapi_pfind.c:975 — (defect) Negation is handled differently for level/chunk than for type. `fp_exclude_compr_lvl` is passed into find_value_cmp() as `negopt`,

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) Negation is handled differently for level/chunk than for type. `fp_exclude_compr_lvl` is passed into find_value_cmp() as `negopt`, so `found_lvl` ends up meaning "some component does not match" instead of "no component matches", and the aggregation here then ignores the exclude flag entirely - unlike the fp_check_compr_type block just above.

For a file with two compressed components, level 3 and level 5:

    lfs find ! --compr-level=3 dir

entry(lvl 5) makes find_value_cmp() return -1, negopt flips it to 1, found_lvl is set, and the file is reported - even though it does have a level-3 component. The equivalent `! --compr-type=gzip` on a gzip+lz4 file correctly does not match.

Should the level and chunk comparisons drop the negopt argument and be inverted here the same way type is?

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre liblustreapi_pfind.c:1771 — (defect) `fp_lum_size` is the size common_param_init() allocated for the lmd buffer (>= XATTR_SIZE_MAX, 64KiB), not the length of the layout

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) `fp_lum_size` is the size common_param_init() allocated for the lmd buffer (>= XATTR_SIZE_MAX, 64KiB), not the length of the layout stored in it.

For a non-composite lum, llapi_layout_get_by_xattr() never narrows that value, so llapi_layout_objects_in_lum() computes (65536 - 32) / 24 = 2729 objects and __llapi_comp_alloc() rejects it with EINVAL because 2729 > LOV_MAX_STRIPE_COUNT (2000).

So this call fails for every plain V1/V3 file and always falls through to llapi_layout_get_by_path(); only composite layouts reach the new path, because the entry loop resets the size to `ent->lcme_size`. Passing the actual layout length instead would make the fast path work for non-PFL files too.

tags: csdc

owner Marc Vef · uploader Artem Blagodarenko · PS17 uploaded 73d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: add 'lfs find' support for compressed file

* Add "--comp-flags=[^]compress" to locate file with/without
  compressed components.
* Add "--comp-flags=[^]nocompr" to locate file with/without setting
  component compress preference.
* Add "[!] --layout=compress" to locate file with/without compressed
  components.
* Add "[!] --compress-type=<compress-type>" to locate compressed file
  with/without specified compress algorithm.
* Add "[!] --compress-level=[+-]<compress-level>" to locate compressed
  file with/without specified compress level.
* Add "[!] --compress-chunk=[+-]<compress-chunk>" to locate compressed
  file with/without specified compress chunk in KiB.

utils: add -Z option for 'lfs getstripe/find'

Add support for "lfs getstripe -Z" to get the last instantiated
component compression information.

Add support for "lfs find -Z <type>[:[+-]<level>]" to keep consistent
options with "lfs setstripe -Z".
DDN-bug-id: EX-6856
Was-Change-Id: Ia5e2a2ea6937dc2b46e224cec808504a196b974c

utils: logical AND for 'lfs find' compression exprs

All search expressions provided to 'lfs find' must be combined as a
logical AND. Fix newly added options for compression support, so that
they comply with this logical AND.
DDN-bug-id: EX-7714
Was-Change-Id: I3b28cd87c1d304df6d04753b413d46f5abcfe16e

csdc: don't set compression layout when disabled

When llite_enable_compression is disabled
(lfs set_param llite.*.enable_compression=0), we should check
it before sending it to MDS lest we get a file with compressed
component which we cannot handle.
DDN-bug-id: EX-7593
Was-Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f

utils: support 'lfs find --printf %LZ'

Add support for "lfs find --printf %LZ" to print the compression
type:level of the last instantiated component of a file.
DDN-bug-id: EX-6856
Was-Change-Id: Iaf1b6c031b06c70e7b5be51354697aa6bdcc9850

csdc: right error code in case compression is disabled

Currently, if compression is disabled on a client the error
message is confusing "Cannot set layout EA: Unknown error 524"

Let's replace the error code from  ENOTSUPP to EOPNOTSUPP,
so error messages become more informative

ctl get_param -n llite.*.enable_compression
0
lfs setstripe -i 0 -c 1 -E -1 -Z lz4 /mnt/lustre/foo.txt
Cannot set layout EA: Operation not supported
lfs setstripe: cannot create composite file
'/mnt/lustre/foo.txt': Inappropriate ioctl for device

This message could be even more informative, but this version
is already much better.
DDN-bug-id: EX-10577
Was-Change-Id: I7fd67d5d3d20bc7de169f35f6fa63b53f2bfe718

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic36946738c3463fd862aeca4ee2e2c2ed85eff84

52776 master J !B ✓T ✓R -1/2−

LU-13802 llite: add file nonrotational check

carrying Patrick Farrell · needs rebase — checkpatch: cannot be cherry-picked

needs rebase hio stalled 133d 1 unresolved S +17 −2

133d

needs rebase — checkpatch: cannot be cherry-picked

1 unresolved thread(s) await your reply — last from Andreas Dilger

CI green — no vote yet from Shaun Tancheff, Andreas Dilger

janitor run: 13 failing config(s), none unique to this patch — janitor results

6 optional test failure(s) — informational

build #124324 SUCCESS · tests all Maloo sessions

no vote yet: Shaun Tancheff, Andreas Dilger

patchsets:

+16 earlier

PS1 10-20 PS2 10-20 PS3 10-20 PS4 10-24 PS5 10-24 PS6 01-16 PS7 01-16 PS8 02-10 PS9 02-12 PS10 02-12 PS11 02-12 PS12 02-13 PS13 02-13 PS14 03-13 PS15 03-13 PS16 03-21

PS17 03-21 PS18 03-27 PS19 04-01 PS20 04-23 Marc Vef PS21 04-23 Marc Vef

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS19 2026-04-26 02:56 Andreas Dilger cl_object.h:1963 — Actually, an OST is (generally) only going to be rotational or non-rotational, and this is already returned to the client via statfs, so the

Andreas Dilger PS19 · 2024-04-11 23:20

We aggregate attributes from multiple objects for other reasons (e.g. maxbytes, timestamps, size, etc.).

The current implementation looks "mostly OK" and will work for basic linear IO patterns, where the file transitions from non-rotational to rotational.  However, any other kind of IO pattern (e.g. HDF5 where some writes are at the beginning of the file and others at the end) might have a serious issue?

IMHO, it would be better to aggregate this information once from the object (maybe from the OSC's the object is allocated on) and then be done with it?  Storing a threshold when the file transitions from non-rotational to rotational would be best, as this could be computed once based on the layout.  It is fairly unlikely that a file would have HDD stripes in the middle and flash at the beginning and end.

Alternately (probably better) is to store the nonrot state in each layout component (which is a perfect 1:1 mapping), and then this can be checked at IO submission time to see what type of storage it is covering.

Patrick Farrell PS19 · 2024-04-11 23:58

Hm, OK.  I don't think we can do this as easily as we'd like - the issue is we have to have this information very early.  I can consider this, though, your points are good ones.

The good thing is this isn't essential to merging the core feature, particularly since we're leaving it off by default for 2.16.

Andreas Dilger PS19 · 2026-04-26 02:56

Actually, an OST is (generally) only going to be rotational or non-rotational, and this is already returned to the client via statfs, so the clients should have full knowledge whether any IO is on flash or disk.

owner Patrick Farrell · uploader Marc Vef · PS21 uploaded 192d ago · NEW · open in Gerrit ↗

commit message

LU-13802 llite: add file nonrotational check

This patch adds the ability to note whether or not the
last IO to a file hit a rotational or a non-rotational
device.

This will be used in selecting the cutover thresholds for
hybrid IO, since DIO is synchronous and much slower on
spinning disk.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I75a9970f91b1776ed6f04ac0d000a9ba576df75a

50687 master J ✗1B ✓T ✗12R -2/2−

LU-12916 osd: use writeback for small writes in ldiskfs

carrying Qian Yingjin · needs rebase — checkpatch: cannot be cherry-picked

needs rebase 1 unique failure stalled 186d 1 unresolved L +639 −14

186d

needs rebase — checkpatch: cannot be cherry-picked

janitor: 1 test failure unique to this patch — sanity2@ldiskfs+DNE:test_127f

enforced failed: review-dne-part-1, review-dne-part-2, review-dne-part-3, review-dne-part-5, review-dne-selinux-ssk-part-1, review-dne-subtest-change, review-ldiskfs, review-ldiskfs-ubuntu

1 unresolved thread(s) await your reply — last from Qian Yingjin

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity2@ldiskfs+DNE:test_127f	seen in 1 other review

failed enforced test	platform	detail
review-dne-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-part-1	RHEL 9.5 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-part-2	RHEL 8.10 / x86_64	ran 11 tests. 1 tests failed: sanity-sec.	session
review-dne-part-3	RHEL 9.5 / x86_64	ran 3 tests. 1 tests failed: conf-sanity.	session
review-dne-part-5 crashed	RHEL 9.5 / x86_64	ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %%	session
review-dne-part-5	RHEL 8.10 / x86_64	ran 5 tests. 1 tests failed: sanityn.	session
review-dne-selinux-ssk-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanityn.	session
review-ldiskfs	RHEL 8.10 / x86_64, RHEL 9.5 / x86_64	ran 5 tests. 1 tests failed: sanity.	session
review-ldiskfs	RHEL 8.10 / x86_64, SLES 15.5 / x86_64	ran 5 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64	ran 6 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64	ran 6 tests. 1 tests failed: sanity.	session

12 optional test failure(s) — informational

build #124337 SUCCESS · tests all Maloo sessions

no vote yet: Sohei Koyama, Shaun Tancheff, Shuichi Ihara, Andreas Dilger, Alex Zhuravlev, Patrick Farrell

patchsets:

+68 earlier

PS1 04-19 PS2 04-19 PS3 04-19 PS4 04-20 PS5 04-20 PS6 04-20 PS7 04-20 PS8 04-20 PS9 04-21 PS10 04-21 PS11 04-21 PS12 04-21 PS13 04-22 PS14 04-22 PS15 04-22 PS16 04-23 PS17 04-23 PS18 04-23 PS19 04-23 PS20 04-23 PS21 04-23 PS22 04-23 PS23 04-23 PS24 04-23 PS25 04-23 PS26 04-24 PS27 04-24 PS28 04-24 PS29 04-24 PS30 04-24 PS31 04-24 PS32 04-24 PS33 04-25 PS34 04-25 PS35 04-25 PS36 04-26 PS37 04-26 PS38 04-27 PS39 04-27 PS40 04-28 PS41 05-11 PS42 09-22 PS43 10-25 PS44 12-15 PS45 12-15 PS46 12-15 PS47 01-17 PS48 01-19 PS49 03-15 PS50 03-22 PS51 03-26 PS52 04-01 PS53 05-10 PS54 03-06 Sohei Koyama PS55 04-15 Sohei Koyama PS56 04-24 Sohei Koyama PS57 04-24 Sohei Koyama PS58 04-24 Sohei Koyama PS59 04-24 Sohei Koyama PS60 05-06 Sohei Koyama PS61 05-06 Sohei Koyama PS62 05-07 Sohei Koyama PS63 05-07 Sohei Koyama PS64 07-09 Sohei Koyama PS65 07-09 Sohei Koyama PS66 07-09 Sohei Koyama PS67 10-02 Sohei Koyama PS68 10-02 Janitor Bot

PS69 10-02 Sohei Koyama PS70 10-29 Sohei Koyama PS71 10-29 Janitor Bot PS72 04-23 Marc Vef PS73 04-24 Marc Vef

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS72 2026-04-24 08:41 Qian Yingjin COMMIT_MSG:19 — Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext

Andreas Dilger PS72 · 2026-04-24 08:29

(typo) ".patch"

Writes for fscrypt files *should* always be full-block writes?  Why/how would writes to fscrypt files be partial blocks?  The client should be sending a full encrypted 4KiB block, but then set the size on the inode without truncating it.

Qian Yingjin PS72 · 2026-04-24 08:41

Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext4 for fscypt with write through (direct I/O) mode.

Although writes from clients are always full-block writes, but the size calls via lnb offset + len may be not full-block aligned, we use this size for the actual size of the file?

owner Qian Yingjin · uploader Marc Vef · PS73 uploaded 192d ago · NEW · open in Gerrit ↗

commit message

LU-12916 osd: use writeback for small writes in ldiskfs

This patch implements the writeback caching for small writes on
Lustre OSD (ldiskfs).
The writes are not synchronous any more on OST. It marks the page
as dirty and create buffer heads mapping pages to the physical
blocks. The kernel writeback mechanism will handle the actual I/O.

This patch should improve the performance for I/O cases such as
small writes from many files (i.e. mdtest-hard-write) or
unaligned I/O on a shared file (i.e. ior-hard-write).

Add a ldiskfs patch "ext4-writeback.patch" to not zero out
non-full page during writeback for a encrypted file. Without this
ldiskfs patch, it will fail sanity-sec/test_37.

According to the benchmark results, this patch can improve the
IO500 mdtest-hard performance and the performance of the real
application Nek5000 turbulent pipe with more than 10%.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I9b5aa568a49d5b39d04656fdacb94c3a6ec5a7ff

60957 master J ✓B ✓T ✓R -1/2−

LU-18756 tests: sanity-sec 75a sync resource IDs

mine Andreas Dilger voted -1

mtfl_p2 stalled 255d 1 unresolved S +6 −4

255d

Andreas Dilger voted -1

1 thread(s) waiting on others

no vote yet from Sebastien Buisson

reviews: Andreas Dilger -1 2025-08-29 17:28

no vote yet: Sebastien Buisson

patchsets: PS1 08-28 PS2 08-28 PS3 08-29

threads: 0 need your reply · 1 waiting on others · 0 own notes · 0 bot

owner Marc Vef · uploader Marc Vef · PS3 uploaded 430d ago · NEW · open in Gerrit ↗

commit message

LU-18756 tests: sanity-sec 75a sync resource IDs

After sanity-sec 75a setups the test namespace, it creates and writes
to files before changing the user to the correct tenant user which
carries out the commands. Later, when the ID check is enabled, it
could happen that the corresponding OST object still has the initial
user ID set and that the "chown" changes have not yet been propagated
from the MDT to the OSTs. In such a case, the test fails the ID check
because the OST object has the wrong IDs set.

This patch adds an additional step after namespace creation to
synchronize ID changes to and between the servers before continuing
with the test.

Fixes: 719f6763fe ("LU-18756 sec: add resource id check to oss and mds")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=75
Test-Parameters: testlist=sanity-sec env=ONLY=75a,ONLY_REPEAT=100
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I07d602375fd34334dc5047cfecea60f09f03bd42

Mentions

1 waiting on you · 1 open

Unresolved threads where someone @-mentioned you (username, email or full name) — the pings that drown in Gerrit's email. Red: you have not replied since the ping. Muted: you replied, but the thread is still open, so it stays on the list until someone resolves it. Both clear on the refresh after the thread is resolved.

69026 master J ✗1B ✓T ✗1R -2/2−

LU-17217 nodemap: add client version policy

cc Timothy Day · Andreas Dilger pinged you: “@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here? If this pa”

needs rebase 1 unique failure 12 unresolved L +805 −3

20d

1 ping waiting on you:

Gerrit AI review for Lustre PS1 · 2026-09-18 09:49

(minor) Adding the bit changes NODEMAP_RAISE_PRIV_NONE from 0xfffffe00 to 0xfffffc00, and that value is stored on disk rather than being a pure in-memory sentinel.

nodemap_idx_cluster_roles_modify() writes a NODEMAP_CLUSTER_ROLES record whenever rbac != NODEMAP_RBAC_ALL, and the record carries ncrr_privs = nmf_raise_privs, i.e. the old 0xfffffe00 for a nodemap whose raise_privs is 'none'. nodemap_cluster_roles_helper() then assigns it back verbatim with no masking.

So after an upgrade, a nodemap that merely had a non-default rbac comes back with bit 0x200 set, and

    if ((nodemap->nm_parent_nm->nmf_raise_privs & priv) && priv != NODEMAP_RAISE_PRIV_RBAC)
            return true;

short-circuits the new NODEMAP_RAISE_PRIV_VERSION check, so a dynamic child can override a parent policy that was meant to be locked. raise_privs would also read back as 'version_policy' for a nodemap the admin set to 'none'.

This encoding predates the patch (CAPS and DENY_MNT had the same effect when they were added) so it probably belongs in its own change rather than blocking this one - but is it worth masking the loaded value to the known bits, or is the intent that unknown bits stay raised?

Andreas Dilger PS1 · 2026-09-19 08:08

@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here?  If this patch is introducing the mismatch between the flags then it makes sense to fix it here as well instead of separate patch.

Sebastien Buisson PS1 · 2026-09-25 15:55

`NODEMAP_RAISE_PRIV_NONE` was made on the same model as `NODEMAP_RBAC_NONE`, but the sematic is quite the opposite actually. Rbac roles are all present by default, and we remove some of them to lower client capabilities. 'Raise privs' are all absent by default, and we add some of them to grant privileges. So I think we should set `ncrr_privs` to 0 when `nmf_raise_privs` is `NODEMAP_RAISE_PRIV_NONE`. That would avoid the problem mentioned here when we upgrade a system after new privs have been added, and that would be a simple fix. I could push a quick patch, so that it is fixed right now and does not wait for this bigger patch.

owner Timothy Day · uploader Timothy Day · PS1 uploaded 45d ago · NEW · open in Gerrit ↗

0 watched

Nothing watched. Add a change above — e.g. after rebasing someone's patch — and its build/test state shows up here.

107 open

▶

Failed / broken

(94)

68801 master J !B ✗T –R +1/2

LU-10026 ofd: fix punch vs BRW deadlock on compressed files

build failure (#131557)

csdc XS +8 −1

10d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #131557 FAILURE

reviews: Sebastien Buisson +1 2026-10-23 09:56

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: fix punch vs BRW deadlock on compressed files

ofd_object_punch_compr() converts a compressed chunk back into a
plain one when a truncate lands in the middle of it.  It asks the
OSD for the chunk pages first and takes the chunk range lock
afterwards, while ofd_preprw_read_compr() and
ofd_preprw_write_compr() take the range lock first and only then
ask the OSD for the pages.

A truncate and a BRW touching the same chunk deadlock on that: the
punch thread holds the page locks that osd_bufs_get() acquired and
sleeps in range_lock(), while the BRW thread holds the chunk range
lock and sleeps in osd_get_page() waiting for one of those pages.
Nothing breaks the cycle, so the OST_PUNCH never completes and the
client stays blocked in ftruncate().  sanity-compr test_fsx trips
this a minute or two after fsx starts and then eats the whole
session timeout.

Take the chunk range lock before mapping the chunks to OSD buffers
so that all three paths lock the range before the pages.

Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11952
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6095f9533c23a376cf0b5c84bb75c4d956922204

65737 master J –B ✓T ✗2R -1/2−

LU-10026 csdc: set compress component for file

needs rebase — checkpatch: cannot be cherry-picked

needs rebase csdc 14 unresolved XL +1163 −16

11d

needs rebase — checkpatch: cannot be cherry-picked

enforced failed: custom-1001, review-dne-part-7

14 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

failed enforced test	platform	detail
custom-1001	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-pfl.	session
review-dne-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-pcc.	session

build #132858 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Roman Bolshakov, Andreas Dilger, Artem Blagodarenko

patchsets:

+10 earlier

PS1 05-07 PS2 05-07 PS3 05-28 PS4 05-28 PS5 06-11 PS6 06-11 PS7 06-16 Artem Blagodarenko PS8 06-16 Artem Blagodarenko PS9 07-24 Artem Blagodarenko PS10 08-19 Artem Blagodarenko

PS11 08-19 Artem Blagodarenko PS12 08-20 Artem Blagodarenko PS13 08-20 Artem Blagodarenko PS14 08-21 Artem Blagodarenko PS15 10-05 Roman Bolshakov

threads: 14 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(minor) The body doesn't mention the client-side gate added by squashing Ib1e2123ff: the new lustre_check_lov_user_md() and lustre_check_lov_comp_md_v1() in ptlrpc/pack_generic.c, and their calls from ll_lov_setstripe_ea_info() and ll_xattr_set(). These reject a compress layout when `llite.*.enable_compression=0`, which is a user-visible behaviour change. Could a short paragraph describing them be added?

Andreas Dilger PS11 · 2026-08-19 23:52

@ablagodarenko@thelustrecollective.com it looks like the patches were rebased, but did not address any of the AI review comments?  While we can't go overboard changing every detail of the patches, I think it does make sense to fix the code where possible so that we don't carry technical debt later.

Artem Blagodarenko PS11 · 2026-08-20 12:36

Hi Andreas. Thanks for poining to this. I have reverted all fixed by Patchset 11 pushing wrong branch. I have reverted to Patchset 10 where I addressed AI bot comments and fixed build.

reply PS14 2026-10-05 17:51 Roman Bolshakov patchset comment — Squashed Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f into the change. I'll address a couple more comments in another push.

Roman Bolshakov PS14 · 2026-10-05 17:51

Squashed Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f into the change. I'll address a couple more comments in another push.

reply PS9 2026-08-19 23:52 Andreas Dilger lfs-getstripe.1:331 — Should be fixed if patch is refreshed.

Gerrit AI review for Lustre PS9 · 2026-07-28 09:23

(minor) There is no `.TP` before this entry, so `--compress-type|--compr-type` and its description get folded into the `-z, --extension-size` paragraph above instead of starting their own tagged item. The two entries below it do have theirs.

Oleg Drokin PS9 · 2026-08-12 02:29

weird but in my man output it looks ok.

Artem Blagodarenko PS10 · 2026-08-19 19:47

The entry was missing its .TP in the troff source. Some man implementations are lenient and reflow correctly, but others fold the entry into the previous paragraph. Added .TP for correctness.

Artem Blagodarenko PS10 · 2026-08-19 19:47

Done. Added the missing .TP before the --compress-type|--compr-type entry.

Andreas Dilger PS9 · 2026-08-19 23:52

Should be fixed if patch is refreshed.

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) Neither of the two new man3 pages has an EXAMPLES section. Lustre llapi pages are expected to carry a short usage example showing the required header and a realistic call.

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(minor) This, and the matching EINVAL entry under ERRORS, says a `stripe_size` that is not a multiple of `chunk_size` is an error. llapi_layout_compress_set() never fails for that: it decrements `log_bits` until the chunk divides the stripe and stores the smaller chunk (sanity-pfl test_100c expects 3*64KiB to become 128KiB). Similarly, gzip/lz4/lz4fast levels outside the listed ranges are clamped by their from_*_level() helpers, not rejected with EINVAL as the text above says (test_100c expects gzip:15 to read back as 9). If the patch is refreshed, could the page describe the rounding and clamping instead?

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) llapi_parse_compress_type() also returns -EINVAL - for a level that isn't a valid number or falls outside +/-COMPR_LEVEL_MAX - which neither RETURN VALUES nor ERRORS mentions. Mixing -1 with -Exxx returns in one function is also awkward for callers; would returning -EINVAL for the unknown-type case be more consistent?

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) The 9 used as the lz4 default collides with a real level. from_lz4_level(10) returns 9, and to_lz4_level(9) returns 9 because 9 == LZ4HC_DEFAULT_CLEVEL, so 'lfs setstripe -Z lz4:10' reports back as lz4:9 and level 10 can never be displayed - even though lfs-setstripe.1 documents 1-16 as the valid lz4 range.

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) This gate runs only for regular files: ll_lov_setstripe_ea_info(), and the `lov.` branch in ll_xattr_set(). A directory default goes through ll_setstripe_ea() -> ll_dir_setstripe() (and the LL_IOC_LOV_SETSTRIPE case in ll_dir_ioctl()), which never calls lustre_check_lov_user_md(). So on a client with `enable_compression=0`:

    lfs setstripe -E eof -Z lz4 $dir    # accepted
    touch $dir/f                        # MDS instantiates a compress layout

This still produces a file the client can't handle. lsm_lmm_verify_v1v3() rejects it with "file compression not enabled", and the PS14 report shows a later unlink or stripe change hitting the lov_io_layout_at() LBUG. Should ll_dir_setstripe() apply the same check? Or, if allowing compressed defaults on a disabled client is intended, should the client reject the layout cleanly instead of asserting?

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) On this `lov.` path nothing validates `size` before lustre_check_lov_user_md() walks the entries. lustre_check_lov_comp_md_v1() reads `lcm_entry_count` and then up to 65535 `lcm_entries[]`, but the buffer here is only the `size` bytes copied in by the VFS.

The `lustre.` xattr namespace needs only write permission on the inode, so any file owner can trigger it:

    setxattr(f, "lustre.lov.add", buf, 8, 0);  /* buf starts with LOV_USER_MAGIC_COMP_V1 */

That reads lcm_entry_count past the end of an 8-byte allocation and keeps reading for up to ~3MB. KASAN reports it as slab-out-of-bounds. Before this patch, little-endian clients only read `lmm_magic` here.

The next change in the series ("utils: add 'lfs find' support for compressed file") adds a `size < need` check just above this call. Since this patch introduces the read, should that check move here? Passing `size` into lustre_check_lov_user_md(), the way lustre_swab_lov_user_md() takes one, would also let it validate the buffer itself.

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) This applies the chunk adjustment to every component including extension ones, where llc_stripe_size holds the extension size rather than a real stripe size. lod_declare_layout_add() deliberately skips LCME_FL_EXTENSION components for exactly that reason, so the same input layout gets different chunk_lum_bits depending on whether it arrives via create or via --component-add. Should this site skip extension components as well?

While there: the comment added in lod_declare_layout_add() says chunk adjustment is deferred until instantiation, but nothing calls lod_adjust_compr_chunk_size() on the instantiation path - the only three callers are here, lod_declare_layout_add() and lod_striping_from_default().

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) -ENOTSUPP (524) is kernel-internal and has no userspace errno, so this reaches `lfs setstripe` through ioctl()/setxattr() as "Unknown error 524". The erasure-coding check just above it in ll_lov_setstripe_ea_info() returns -EOPNOTSUPP for the same kind of rejection. Should this be -EOPNOTSUPP too?

While here, a couple of nits: `rc` is never changed from 0, there's a double blank line after the EXPORT_SYMBOL(), and the default case in lustre_check_lov_user_md() logs "Invalid LOV magic" but still returns 0.

tags: csdc

owner Marc Vef · uploader Roman Bolshakov · PS15 uploaded 27d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: set compress component for file

Introduce per-component compression support for composite LOV layouts.
A new LCME_FL_COMPRESS component flag marks entries that carry
compress fields; the flag is set when the user supplies a compression
type via 'lfs setstripe -Z' and is propagated through the LOD and LOV
layers so that compressed components are distinguished from plain
ones.

LOD layer:
- lod_parse_striping(): read lcme_compr_type/lvl/chunk_lum_bits from
  the user-supplied lov_comp_md_v1 when LOV_PATTERN_COMPRESS is set.
- lod_use_defined_striping(): likewise for pre-defined layouts.
- lod_get_default_lov_striping(): carry compress fields into the LOD
  component when inheriting a default layout that includes
  compression.
- lod_generate_lovea(): serialise llc_compr_* fields into the on-wire
  lcm entry when LOV_PATTERN_COMPRESS is set.
- lod_adjust_compr_chunk_size(): new helper that rounds down the
  requested chunk-size exponent so the chunk fits evenly within one
  stripe and within one RPC (clamped to COMPR_CHUNK_MAX_BITS in the
  following changes).

API / utils layer:
- 'lfs setstripe' gains --compress|-Z <type>[:<level>] and
  --compress-chunk=<size> options to set the compression type, level,
  and chunk size on a new component.  Like -S and -c, -Z is inherited
  by subsequent -E components unless overridden by another -Z.

  Example:
	$ lfs setstripe -Eeof -Z lz4:5 --compress-chunk=512 <file>

- 'lfs getstripe' gains --compress-type, --compress-level, and
  --compress-chunk options to print individual compress parameters:

  Example:
	$ lfs getstripe <file>
    lcme_compr_type:     lz4
    lcme_compr_lvl:      5
    lcme_compr_chunk_kb: 512
      lmm_pattern:       raid0,compress

- llapi_layout_merge(): copy compress fields when merging layouts.
- Two new man3 pages: llapi_layout_compress_set.3 and
  llapi_parse_compress_type.3.

Misc:
- dump_lsm() in lov_ea.c prints compress fields for debugging.
- compr_name_from_type() helper added in lov_internal.h and wired up
  in dump_lsm() to print compression type names in debug output.

Test-Parameters: testlist=sanity-pfl serverversion=2.17
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Signed-off-by: Roman Bolshakov <rbolshakov@ddn.com>
Change-Id: Ife0382469cbc5099e0c6dc96534bb169ddeff61e
Was-Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f

68797 master J ✗1B ✗T –R +2/2

LU-10026 ofd: properly locate root object

build failure (#133377)

1 unique failure csdc 1 unresolved S +17 −8

22d

build failure (#133377)

janitor: 1 test failure unique to this patch — sanity1@ldiskfs+DNE:test_56od

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity1@ldiskfs+DNE:test_56od	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #133377 FAILURE

reviews: Andreas Dilger +1 2026-10-11 04:49 · Mikhail Pershin +1 2026-09-11 06:17

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

On this branch ofd_tunables_init() already ran before the compr_stats block, so the new failure paths jump to a new err_fini_tunables label (ofd_procfs_fini()) ahead of the existing teardown. When the compr_stats file cannot be obtained the stored error pointer is cleared so ofd_fini() does not try to save/put it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: properly locate root object

ofd_init0() gets root object by using dt_locate_at() with
an OSD device as top. As result lu_object can be incomplete
if it doesn't exist yet. Instead dt_locate() should be used.

Patch fixes that and adds error handling in related places.
When the compr_stats file cannot be obtained the reference is
cleared, so that it is neither saved nor released at fini.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-13567
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9f5eaaae9c4ea23225f574b8e7cde64b3c5f47b6

68391 master J !B ✗T –R 0/2

LU-10026 csdc: migrate to another compression level

build failure (#130363)

csdc 2 unresolved M +70 −0

47d

build failure (#130363)

1 unresolved thread(s) await your reply — last from Patrick Farrell

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130363 FAILURE

no vote yet: Sergey Cheremencev, Artem Blagodarenko, Alex Zhuravlev

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

Version gate converted to compression_supported.

reply PS1 2026-09-15 18:51 Patrick Farrell sanity-compr.sh:2033 — **[AI review]**

Patrick Farrell PS1 · 2026-09-15 18:51

**[AI review]**

test_1030 reads $tmp_hdf and $hdf, but neither is set here. Every other test that uses them declares them as locals in its own function (test_1007 at sanity-compr.sh:552-554, and five more sites), so they are empty by the time test_1030 runs.

So [[ ! -f $tmp_hdf ]] is always true, [[ -f $hdf.bz2 ]] tests the literal path ".bz2" and is always false, and the test always takes the else branch. test_1030 never executes, and the skip reason it prints is misleading.

Seen on a full sanity-compr run of this series on Rocky 9, where bunzip2, bzcat and lustre/tests/AMSR_E_L3_DailyOcean_V05_20111003.hdf.bz2 were all present, and 1007 and 1008 -- which do declare the locals -- had already run and passed:

  SKIP: sanity-compr test_1030 bunzip2 is not installed

Declaring the same locals here as the other tests do should be enough:

  local hdf=$LUSTRE/tests/AMSR_E_L3_DailyOcean_V05_20111003.hdf
  local tmp_hdf=$TMP/$tfile.hdf

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: migrate to another compression level

Here is a test there lfs migrate with changed
compression level.

tests: fix sanity-compr 1030

Set compress_check_bytes to 2 times larger
than uncompressed file size. Without that
file might be considered incompressible.

Add extra debugging to help understand test failures.

DDN-bug-id: EX-10512
Was-Change-Id: I559f94b24bd2abb821a07775198bb4e511793a72

tests: change algo in 1030

to compare apples with apples. skip (i.e. do no fail) if
less data were compressed in the 2nd approach as this may
happen due to external conditions we don't control well
(grants, etc).
take into account that higher compression level in lz4fast
means less efforts and larger result.

DDN-bug-id: EX-10512
Was-Change-Id: Ia4c655ec99c1b45ff9c273311e5d567b8201cd5a

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9889
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia67316a247fed99f6ad002c8b67053a3c2fa0cff

68411 master J !B ✗T –R 0/2

LU-10026 csdc: parameter to disable compression on HDD OSTs

build failure (#130383)

csdc 2 unresolved L +471 −11

48d

build failure (#130383)

1 unresolved thread(s) await your reply — last from Patrick Farrell

janitor run: 40 failing config(s), none unique to this patch — janitor results

build #130383 FAILURE

no vote yet: Yang Sheng, Sergey Cheremencev, Artem Blagodarenko

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- In lod_ost_alloc_rr the -EBADR early-out jumps to this tree's out_pool label (the original's 'out' label; identical cleanup).
- The HDD check in lod_ost_alloc_qos sits after the additional allocation filters this tree has in that loop.
- The sysfs attribute is inserted in the alphabetically sorted attr table (the original appended).
- The three sanity-compr 2020* version gates converted to compression_supported; the statistics-init helper keeps the unconditional obdfilter access.
- Subject shortened to the 62-column limit; the tunable paragraph in the message and man page rewrapped.

Patrick Farrell PS1 · 2026-09-14 22:19

test_2020a needs at least two OSTs and has no guard for it: it marks OST0001 as SSD with 'do_facet ost2' below, waits on an OST0001 line from 'lfs df -v', and calls subtest_2020a with '-i 1'. On a single-OST setup it fails rather than skipping.

Adding the existing idiom next to the compression_supported check would cover it:

	(( OSTCOUNT >= 2 )) || skip_env "needs >= 2 OSTs"

Confirmed by running it both ways: it fails with one OST and passes with two. test_2020b and test_2020c only use osts_nodes, so they are fine as they are.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: parameter to disable compression on HDD OSTs

The current CSDC compression does not perform well with HDD OST
storage due to fragmentation of the block allocation for a
compressed file. This is acceptable for NVMe OSTs because they
do not have seek latency, but can cause significant performance
impact for HDD OSTs, both for the compressed objects, as well as
uncompressed objects that are being written concurrently due to
IOPS starvation and fragmented free space.

This patch optimizes compression for systems with HDD OSTs.
By default, compression is not supported on HDD OSTs. Such an OST
is skipped and the next SSD-based OSD is chosen. This behaviour can
be adjusted using lod.*.enable_compr_rotational. Allowed values are:
 * -1 - return -EBADR error when trying to allocate compressed files
        on HDD OSTs
 * 1  - allow compression on HDD OSTs
 * 0  - skip an HDD OST

tests: fix sanity-compr/2020 for one-node setup

The check in sanity-compr test_2020() if an OST was flash or HDD
was unreliable if the MDT was also on a flash device, or if there
were other statfs flags on the target devices.

Improve this check to only look for the 'f' (flash/nonrotational)
status on the specific OSTs being used for the test.

DDN-bug-id: EX-10989
Was-Change-Id: If8ec5e6ce02ce4e5eee4ffb00bc71cda577d98e0

lod: dereference the device after check

Ensure the index is valid before dereferencing it.

DDN-bug-id: EX-10989
Was-Change-Id: Iefab5ae2036c09819016b2e569c870ba36174e38

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10989
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibc485aaed6fe719591f64e55b1203ddd2e646f09

68796 master J ✗1B ✗T –R 0/2

LU-19794 csdc: use acomp compression API

build failure (#131552)

1 unique failure csdc 2 unresolved M +199 −8

49d

build failure (#131552)

janitor: 1 test failure unique to this patch — sanity-quota@ldiskfs+DNE:test_1k

1 unresolved thread(s) await your reply — last from Nikos Papakonstantinou

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-quota@ldiskfs+DNE:test_1k	seen in 11 other reviews

build #131552 FAILURE

no vote yet: Nikos Papakonstantinou

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

reply PS1 2026-09-14 09:36 Nikos Papakonstantinou COMMIT_MSG:1 — I am not aware of this tbh. I tried to build this but I didn't got this.

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-09-10 20:19

ldlm/ldlm_lib.c:
 warn: client_connect_import():duplicate check 'rc' (previous on line 745)

Nikos Papakonstantinou PS1 · 2026-09-14 09:36

I am not aware of this tbh. I tried to build this but I didn't got this.
I am using a Ubuntu 25.10 vm with 6.17.0-14 kernel.
Of course I am building it with --disable-server option.

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- Only the series files are ported here (lustre_compr.h/.c, lustre_crypto.h, ofd_compress.c, osc_compress.c, configure probe); the hunks to the lz4/gzip/zstd modules, kcompr kunit and compr_private/scompress.h belong to the standalone module changes (57776/62408/66001) and are left for new patchsets there, together with the LC_HAVE_CRYPTO_STREAM_IN_SCOMP probe they need.
- The HAVE_CRYPTO_REQUEST_SET_DMA probe sits under the 6.15 section (kernel commit b67a0260037, v6.15-rc1).
- The acomp branch of ofd_decompress_chunk() was completed: the request/wait variables are declared, and a failed request allocation sets -ENOMEM and falls through to the existing error handling (the original referenced undeclared variables and a missing label there). acomp_request_alloc() returns NULL on failure, so the allocation checks test for NULL instead of IS_ERR().
- The request buffers are set with the non-DMA variants of the acomp setters (acomp_request_set_src/dst_nondma): every buffer handed in is a vm_map_ram() mapping or a large pool allocation, which the DMA variants' contract does not allow (inert for scomp-backed algorithms, wrong for a hardware acomp provider).
- ll_crypto_comp_set_level() is compiled out when the legacy comp API is absent; the acomp variant is unconditional on this branch.
- Caveat: no build available to the porter compiles the acomp side (el8 4.18; the janitor uses rocky 8/9); verified by reading against the v6.15 headers. Reviewers with a 6.15+ kernel, please build it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-19794 csdc: use acomp compression API

For kernels 6.15 and beyond use the acomp compression API with DMA
source and destination virtual addresses, instead of the legacy comp
API which was removed there.

The acomp request allocation returns NULL on failure, so the
allocation checks test for NULL and fail with -ENOMEM. The buffers
handed to the request are vm_map_ram() mappings or large pool
allocations, which are not usable for DMA, so they are set with the
non-DMA variants of the request setters.

Test-Parameters: forjanitoronly
Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I1fa60a7a933e6cdd2baa525d0520e7372def29df

68799 master J ✗1B ✗T –R +1/2

LU-10026 ofd: fix csdc optimizations

build failure (#131555)

1 unique failure csdc 1 unresolved S +17 −4

49d

build failure (#131555)

janitor: 1 test failure unique to this patch — sanity-slow@ldiskfs+DNE:test_255c

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-slow@ldiskfs+DNE:test_255c	seen in 5 other reviews

build #131555 FAILURE

reviews: Alex Zhuravlev +1 2026-09-14 08:25

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The fix Alex Zhuravlev asked to land together with 68366/68378; `llch.` member access follows the stack-copy form of the header introduced by the checksum change.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: fix csdc optimizations

we shouldn't ignore pre-existing chunks doing RMW if new data
don't cover full range (i.e. have gap(s))

Test-Parameters: forjanitoronly
DDN-bug-id: EX-14659
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I08e7d3bf9a263ed31ed5ea82a4f0800eeb0529c7

68786 master J !B ✗T –R 0/2

LU-10026 tests: move verify_comp_attr to test-framework

build failure (#131542)

csdc 1 unresolved M +79 −74

51d

janitor run: 9 failing config(s), none unique to this patch — janitor results

build #131542 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

Preparatory, new on master: verify_mirror_count()/verify_comp_attr() move from sanity-flr.sh to test-framework.sh (sanity-pfl needs them, including the already-present test_100k) and verify_comp_attr() learns compr-type/compr-level/compr-chunk/pattern/mirror-id. The string comparison is kept (the internal variant compares arithmetically, which is always true for non-numeric attributes).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: move verify_comp_attr to test-framework

Move verify_mirror_count() and verify_comp_attr() from sanity-flr.sh
to test-framework.sh so that other suites can use them, and teach
verify_comp_attr() the compression attributes (compr-type,
compr-level, compr-chunk) as well as pattern and mirror-id.

sanity-pfl test_100k already uses verify_comp_attr(), and the
following compression mirror tests need it too.

Test-Parameters: forjanitoronly
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0a679215a0848e5d348c4175b09d39fb125c7607

68792 master J !B ✗T –R 0/2

LU-10026 tests: FID retrieval from getstripe

build failure (#131548)

csdc 1 unresolved S +33 −6

51d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #131548 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The sanityn.sh hunk (a trash-can test) does not apply on this branch and was dropped; the rest as the original.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: FID retrieval from getstripe

Some tests rely on the getstripe output to obtain
an object's FID. However, for compressed files, the
getstripe output format differs significantly.

This patch introduces a function that handles both
output formats. All functions potentially affected
by this change have been updated accordingly.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8258
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia9ce932a023066c78aa819bdf502191c5ecc2044

68790 master J !B ✗T –R 0/2

LU-10026 csdc: compress_request memory optimisation

build failure (#131546)

csdc 1 unresolved M +52 −55

51d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #131546 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- merge_chunk() fills the page array with brw_folio_page() (this branch's brw_page carries a folio + page index instead of a page); otherwise as the original.
- Notes for reviewers (inherited, unchanged): the vm_map_ram() NULL path in compress_request() returns -ENOMEM without releasing the pool destination buffer; decompress_request() leaves the mapping in place on two error paths after it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: compress_request memory optimisation

Map brw pages with vm_map_ram instead of copying them to a
buffer allocated by sptlrpc_pool_get_pages. This should save
memory and some CPU cycles.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10553
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3a9280a70a7ac25e29b4e68f0810fe89a5ca69fc

68791 master J !B ✗T –R 0/2

LU-10026 lov: disable compression for DoM

build failure (#131547)

csdc 1 unresolved S +45 −0

51d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #131547 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

test_2003 relocated after test_2002 (numeric order); its version gate converted. Its fix "distinguish the -Z none parameter for DoM" follows three changes later (source landing order kept).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: disable compression for DoM

Disallow enabling compression on the DoM component until
the related feature is properly fixed.

Added a test to ensure this combination is prohibited,
while confirming that DoM continues to function normally.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8718
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I338b375ab386826d08b5610105b14d7496e33aa7

68794 master J ✗2B ✗T –R 0/2

LU-10026 csdc: distinguish the -Z none parameter for DoM

build failure (#131550)

2 unique failures csdc XS +3 −2

51d

build failure (#131550)

janitor: 2 test failures unique to this patch — recovery-small@ldiskfs+DNE:test_115b, sanity-slow@zfs:test_255c

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
recovery-small@ldiskfs+DNE:test_115b	seen in 7 other reviews
sanity-slow@zfs:test_255c	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #131550 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: distinguish the -Z none parameter for DoM

When "-Z none" is passed, LOV_PATTERN_COMPRESS is set
with LL_COMPR_TYPE_SPEC_NONE, this is how this case
can be distinguished for DoM with compression
prevention.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-13161
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0718af523efe2952b844e2d474acc1c02493303a

66001 master J !B ✗T –R 0/2

LU-10026 lustre: add lzstd kernel module

build failure

csdc 34 unresolved XL +33132 −11

51d

build failure

34 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 4 failing config(s), none unique to this patch — janitor results

no vote yet: Andreas Dilger, Sebastien Buisson, Artem Blagodarenko, Oleg Drokin

patchsets: PS1 05-19 PS2 05-19 PS3 09-10 Sebastien Buisson PS4 09-10 Sebastien Buisson

threads: 34 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) debug.c:24 — warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) debug.c:24 — warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) huf_compress.c:460 — warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) huf_compress.c:460 — warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_compress.c:4684 — warn: ZSTD_initCDict_internal():inconsistent indenting

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_initCDict_internal():inconsistent indenting

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) zstd_compress.c:4684 — warn: ZSTD_initCDict_internal():inconsistent indenting

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warn: ZSTD_initCDict_internal():inconsistent indenting

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1577 — warn: ZSTD_compressBlock_lazy_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1605 — parse ZSTD_compressBlock_lazy_generic():error: Function too hairy. Giving up. 3 seconds

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

parse ZSTD_compressBlock_lazy_generic():error: Function too hairy.  Giving up. 3 seconds
warn: ZSTD_compressBlock_lazy_generic():Function too hairy.  No more merges.

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1915 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1937 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1947 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1979 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

tags: csdc

owner Marc Vef · uploader Sebastien Buisson · PS4 uploaded 53d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lustre: add lzstd kernel module

lzstd kernel module implements compression according to the zstd
algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'zstd':
* zstd-lustre-generic of type compression
* zstd-lustre-scomp of type scomp
And 2 cipher drivers under the generic name 'zstdfast':
* zstdfast-lustre-generic of type compression
* zstdfast-lustre-scomp of type scomp

Note the 'zstd' name is identical to the in-kernel module, but lzstd
registers it with a slightly higher .cra_priority = 110, so that it is
preferred over the in-kernel module.
Our 'zstd' is also different in that it accepts a compression level
as explained below. 'zstdfast' is handling the 'negative' compression
levels, interpreted as an acceleration factor.

lzstd kernel module sources are copied from linux v6.9. This module
implements the Crypto API interface, and rely on the zstd kernel
library for compression implementation. It has been modified to grok
a compression level, as read from the top 4 bits of the crypto_tfm
flags, and pass it to the underlying library.
The zstd library sources are also copied from linux v6.9 and built
statically. Headers have also been copied from linux v6.9 for
consistency, and source files modified to include the copied headers
instead of the system headers.
In order to maximize compatibility with various distributions, we copy
xxhash sources from linux v6.9, and link these statically.
All aforementioned sources are located in the
lustre_compat/crypto/zstd directory.
The lzstd module is always built with Lustre.

This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'zstd' module.
It also tries to manually load the lzstd kernel module when a file
requires compression with the 'zstd' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.

EX-9678 csdc: defer memory allocation in zstd/zstdfast

Zstandard normally proceeds to memory allocation for its context
(workspace) while in the init phase, which corresponds to a call to
crypto_alloc_comp() when used via the Crypto API.
At this stage we cannot be aware of the compression level or the chunk
size. So we have to make zstd allocate a workspace that can fit any
level/chunk. And the higher the level is, the larger the workspace
needs to be. Problem is zstd is very memory eager, and by default for
the higher compression level it requires a workspace of 650 MiB. This
is insane compared to our usual compression chunk sizes.

So we defer workspace initialization to the first time zstd actually
compresses data. This corresponds to a call to crypto_comp_compress()
when used via the Crypto API. And by that time we know the compression
level and chunk size, which enables us to make zstd allocate a finer
grained workspace, that just fits our actual needs. This saves a lot
on memory allocation, as can be seen from these values (workspace size
expressed in bytes):

                      no src hint   64kB src
   level 1 cwksp size      582392    303864
   level 2 cwksp size      779000    402168
   level 3 cwksp size     1303288    664312
   level 4 cwksp size     2614008   1057528
   level 5 cwksp size     3662584   1057528
   level 6 cwksp size     3662584   1057528
   level 7 cwksp size     6808312   1057528
   level 8 cwksp size     6808312   1057528
   level 9 cwksp size    13099768   1057528
   level 11 cwksp size   25682680   1319672
   level 13 cwksp size   34071288   1468856
   level 15 cwksp size   67625720   1791095
   level 17 cwksp size   50997688   1791095
   level 19 cwksp size   85196598   1791095
   level 21 cwksp size  336854838   1791095
   level 22 cwksp size  680943414   1791095

And of course because we reduce contention on memory a lot, we improve
performance when zstd is used via the Crypto API, which is done in
osc_brw_prep_request() for ever RPC that involves a file compressed
with zstd.
Was-Change-Id: I470331ea0f2e550761ce7223e7be7e1373701bd2

Was-Change-Id: Idf7382093feec21bed5b6e62001a085a386736d2
Was-Change-Id: Ifa9d38d807d0ab55f85f6c6c8672c340778c2855

EX-bug-id: EX-9678
Test-Parameters: testlist=sanity env=ONLY=84
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I113345cd10f41197683bfe127d373b8628364f88

68798 master J !B ✗T –R 0/2

LU-10026 osc: do not expand if overflow compression

build failure (#131554)

csdc S +23 −0

51d

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #131554 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: do not expand if overflow compression

The extent expansion function osc_extent_expand() can
expand extents beyond the compression chunk limit, completely
bypassing the limit enforced in osc_extent_find() and
try_to_add_extent_for_io().

Let's add the check to the osc_extent_expand() to
make this situation impossible.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12849
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id0240c2078840a250d169a946e1894f4375fd0b9

68795 master J !B ✗T –R 0/2

LU-10026 utils: add compression type completion

build failure (#131551)

csdc 1 unresolved M +59 −0

52d

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #131551 FAILURE

no vote yet: Chakshu Kansal

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

Applied cleanly; test_440b's compression_enabled gate replaced by compression_supported (the test only reads osc.*.compress_types).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: add compression type completion

Add bash completion support for compression types in lfs setstripe
command. Also adds a test case to verify the functionality works
correctly.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12299
Signed-off-by: Chakshu Kansal <ckansal@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id19e27a3088023dca4dda03b19054e5fdf0a4385

68787 master J !B ✗T –R 0/2

LU-10026 lov: pack layout_gen and compr info into lcme

build failure (#131543)

csdc 1 unresolved M +122 −0

52d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #131543 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- Field renames to *_chunk_lum_bits; the two sanity-pfl gates converted to compression_supported; subject shortened.
- test_100n expects the pattern "raid0,compress" for the compressed mirror (the original compared "compress" arithmetically, which always passed; the string comparison used here needs the full pattern name, as in test_100k). Depends on the preceding helper commit (verify_comp_attr in test-framework.sh with the compression attributes).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: pack layout_gen and compr info into lcme

Creating a compressed mirror with "lfs mirror -N" did not produce a
compressed component. Even when the compression parameters were fully
specified, the new mirror was created without compression, and lfs
getstripe showed the mirror entry lacking compression fields.

lov_lsm_pack() did not populate the composite entry fields
lcme_layout_gen and lcme_compr_* when packing a lov_comp_md_v1. As a
result the MDS/client logic took the "no layout/compression in entry"
path and defaulted to an uncompressed layout for the new mirror.
llapi_layout_mirror_inherit() should set compression fields.

Set these fields for each composite entry and set during layout mirror
inherit to fix the problem.

sanity-pfl 100n and 100m were added.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12672
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I5bfcd2d942e9eeb853674763f8ea14d97e4a966c

68788 master J ✓B ✗T –R 0/2

LU-10026 tests: compression component via --component-add

build failure (#131544)

csdc 1 unresolved S +15 −0

52d

build #131544 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The lod fix this test was written for is already on master (90c093b7850, LU-17468); only the sanity-compr test_1a part was unlanded - ported under a new Change-Id with the original as Was-Change-Id.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: compression component via --component-add

Add sanity-compr test_1a: create a file with a plain first component,
append a compressed component with "lfs setstripe --component-add
-Z gzip" and verify the added component carries the compression
type. This covers the compression branch of the component-add path
in lod_declare_layout_add().

The lod fix this test was written for is already on master as
commit 90c093b7850 ("LU-17468 lod: component add missed pattern
info"); only the test part is new here.

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I7ad746a550f1afea54a6f5b68823a79a85a44082
Change-Id: I0624bf3f5f9c965f826e72656a8cd8198c5624fd

68793 master J –B ✗T –R 0/2

LU-10026 tests: enable CSDC layout in sanity-compr.sh

build failure (#131549)

csdc 1 unresolved M +146 −56

52d

build #131549 FAILURE

no vote yet: Sarah Liu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: enable CSDC layout in sanity-compr.sh

Enable CSDC layout on root mount in sanity-compr.sh and run the
sanity and sanityn suites with that layout. Subtests that do not
work with a compressed root layout yet are excepted when the layout
carries a compression component, and a few tests are adjusted to
either disable compression for the file under test or to account
for compressed data (block usage, statistics, FID parsing).

sanity test_127a skips its sum check only when the layout is
compressed, and sanityn test_93 reads the OST index from both the
plain and the composite getstripe output.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8258
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2e0e0c3e34d398156a56128925f02336d59ecfca

68372 master J –B ✗T –R -1/2−

LU-10026 tests: add sanity-compr test for compression

build failure (#130385)

csdc 1 unresolved L +309 −9

52d

build #130385 FAILURE

no vote yet: Artem Blagodarenko, Alex Zhuravlev

patchsets: PS1 08-26 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- ll_compression_scan moved out of the server-only install block (bin_SCRIPTS under UTILS), matching the patch intent against master's restructured scripts/Makefile.am.
- The report_error() changes were already on master; only error_ignore_no_dump() is new.
- The ubuntu always_except for test 1008 is removed as in the original (the test source dir change makes it unnecessary).
- test_1007/test_1008 gates converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: add sanity-compr test for compression

This patch adds a sanity-compr test to validate that
we get space usage reduction with compression.
The test uses ll_compression_scan tool to calculate
the compressed size of the source file and compares
it with the size of the Lustre CSDC compressed file.

tests: add sanity-compr test for dir compression

This patch adds a sanity-compr test to validate that
we get directory space usage reduction with compression.

DDN-bug-id: EX-7795
Was-Change-Id: I16f3a3f1e413e4884b3973829df36500667271ce

tests: skip sanity-compr 1007 and 1008

if needed tools are not installed

DDN-bug-id: EX-9871
Was-Change-Id: I0dc3d44c300708f3a25bfce06b81993cdd30c418

tests: change source dir for sanity-compr/1008

While the source directory contains almost all small files,
sanity-compr test_1008 will hit the
"failed estimates > 50% of total estimates" failure on SLES15.

The patch fixes the issue by changing the source dir to
contain large files.

DDN-bug-id: EX-9125
Was-Change-Id: Iad661750cba7c9d2204f2306e73169deb012ddf4

utils: fix ll_compression_scan overflow

Fix the total size estimate calculation to avoid overflow for
large total sizes.  Otherwise, the intermediate register used
for the calculation will overflow and produce an invalid estimte.

DDN-bug-id: EX-9125
Was-Change-Id: I12099cea1268abede17c92e0422b6c1c25ac66d5

tests: sanity-compr/1008 CentOS 7.9 more margin

For sanity-compr test_1008 allow a higher margin of error for
CentOS 7.9 clients because they consistently show slightly
worse compression ratios and fail the test.  It isn't clear
if this is because of worse/partial RPC formation or because
the compression code in this kernel is less optimal.

These tests should still be run to ensure the code is working, but
not fail so much, because fixing these old clients is not critical.

DDN-bug-id: EX-9125
Was-Change-Id: I6c1c6044daf5d0706cfb661b4f0f051769174ff9

tests: get stats_compr param in sanity-compr/1007

This patch adds getting stats_compr param in sanity-compr
test 1007 and 1008 to track data compression statistics.

DDN-bug-id: EX-9125
Was-Change-Id: I7df25d36c8c89ec5a568bf1e8d5694a97dd18ecc

tests: ignore sanity-compr/1008 failures

Temporarily ignore test failures for sanity-compr.sh test_1008 on
SLES15 and aarch64/ppc64le due to very high failure rates on those
systems.

Rename the *second* test_1008 to test_1080 so it can run, and allow
other new tests to start using disjoint numbers to avoid conflicts.

DDN-bug-id: EX-9125
Was-Change-Id: I199613b1e5b1ee8ea7ca4287a6dfe090257ed72f
Was-Change-Id: I3611d1a8b41267f5c38b8088e06eb8f0cad9dd48

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7795
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icf763331205a3e937b794f90444f756fc59f9050

68800 master J ✓B ✗T –R 0/2

LU-10026 tests: skip sanity-compr/1010 on EL10 clients

build failure (#131556)

csdc 1 unresolved XS +6 −0

52d

build #131556 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip sanity-compr/1010 on EL10 clients

sanity-compr test_1010 ("still can read the broken chunk?") fails when
clients run Rocky Linux 10.x. Skip the test until a fix is available
as it blocks testing.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12865
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id84d8bc4922f8fd37b9e42ae64234a0593d79f7f

68785 master J –B ✗T –R -1/2−

LU-10026 csdc: add compression header checksum

build failure (#131541)

csdc 1 unresolved L +241 −48

52d

build #131541 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- This is series entry 81 (ex-8688), which the original port skipped by an off-by-one in the entry numbering; ported now on top of the series. sanity-compr test_1080 (dropped from "tests: more tests for csdc" for that reason) returns with it.
- The header checksum is gated on the LLCH_FL_HDR_CSUM flag that this branch's ll_compr_hdr reserves for it (set by compress_chunk(), checked by is_chunk_start()) instead of the original's "csum != 0" heuristic. The pre-built sample lustre/tests/compressed.bin has the flag set and its header checksum recomputed accordingly; the checksum constant in test_1080's corrupted-size case was recomputed the same way (algorithm unchanged: crc32 over the header with the checksum field zeroed). test_1080's last case clears the flag (instead of zeroing the checksum) so that the no-checksum path is what gets exercised.
- is_chunk_start() takes the folio and gained the int *rc out-param; the four ofd call sites and the osc call site adapted. The ofd functions keep a stack copy of the header as in the original.
- The header-size sanity limit uses sizeof(*header) (the original compared against the size of a pointer).
- The decompress_rnb() prototype hunk does not apply (that helper was removed earlier in the series).
- The forced-decompression fail-loc (OBD_FAIL_OSC_FORCE_DECOMPR) keeps 0x420, which is free on this branch; the forced type is LL_COMPR_TYPE_LZ4.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: add compression header checksum

This commit adds functionality to verify and calculate the checksum
of the header in the `ll_compr_hdr` struct and some another sanity
check.

The client sets LLCH_FL_HDR_CSUM in llch_flags when it stores the
header checksum, and the checksum is only verified when that flag
is set. A chunk with a valid magic and checksum but inconsistent
header fields is treated as corrupted and the read fails with
-EUCLEAN; without a valid checksum such a chunk is passed through
as plain data.

Add sanity-compr test_1080, which exercises these checks with a
pre-built compressed sample (lustre/tests/compressed.bin).

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8688
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I24a9ab9cb7bea1208ada23aa6550127fe6a55017

68789 master J !B ✗T –R 0/2

LU-10026 osc: prevent oversized extents for compressed objects

build failure (#131545)

csdc 1 unresolved M +165 −1

52d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #131545 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- test_2030 anchored after test_2020c. The original flipped the mode of sanity-compr.sh to 0644; the port keeps it executable.
- Note for reviewers: `if (( -n "$brw_check" ))` in test_2030 is a bash arithmetic expression, not a string test (inherited).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: prevent oversized extents for compressed objects

When compressed objects have high compress_rpc_fragment_ratio values,
large extents could contain too many compression chunks to fit in a
single RPC, causing try_to_add_extent_for_io() to crash with
LASSERT(data->erd_page_count > 0) when the oversized extent was the
first one being considered.

This client-side bug also caused server-side buffer overflows where
the client would send thousands of compression chunks in a single
RPC, exceeding the server's PTLRPC_MAX_BRW_PAGES limit and
triggering 'too large request' errors.

Instead of splitting extents after they are created, this fix
prevents the problem at the source by limiting extent size during
extent creation in osc_extent_find(). For compressed objects, the
maximum extent size is now calculated based on compression chunk
limits to ensure extents never exceed the RPC fragment ratio.

This proactive approach prevents both client crashes and server
buffer overflows while maintaining optimal performance by avoiding
the overhead of extent splitting operations.

tests: improve sanity-compr test_2030

Some improvements:

- Use save_lustre_params/restore_lustre_params for brw_size params
- Use save_lustre_params/restore_lustre_params for client OSC params
- Add comprehensive parameter validation to ensure settings applied
- Improve brw_size verification to check all OSTs not just first
- Replace cat > /dev/null with MD5 checksum for data integrity

DDN-bug-id: EX-12849
Was-Change-Id: I3a8dfd393289f5be0c638e84e1097b18a4303240

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12849
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id161a17b4b45a6cad39ccd55e18da0f6d4f58957

68397 master J –B ✗T –R -1/2−

LU-10026 tests: more tests for csdc

build failure (#130369)

csdc 2 unresolved XL +1436 −534

52d

build #130369 FAILURE

no vote yet: Artem Blagodarenko, Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 2 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] correction to the earlier remark: test_1080 is not excluded because its prerequisite is outside the series - the header-checksum

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] correction to the earlier remark: test_1080 is not excluded because its prerequisite is outside the series - the header-checksum change IS series entry 81 and was skipped by an off-by-one in the original port. It is now ported as https://review.whamcloud.com/c/fs/lustre-release/+/68785 ("LU-10026 csdc: add compression header checksum"), which brings test_1080 back together with the compressed.bin sample and the fail-loc it needs.

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- This patch reorganizes sanity-compr.sh wholesale (renumbers 1000->1000a, fsx->1000b, adds 1000c, adds tests 1009a-1016, moves tests into numeric order, adds iozone/dbench/fsx wrappers, file-scope compression gates and DIRECTIO/RAMKB/THREADS/SIZE setup). The port reconstructed the patch's intended post-state and re-applied this tree's conversions on top:
- internal version gates -> compression_supported (or dropped where they guard functionality this series itself provides);
- test_2000 keeps the master client version (2.17.53) for the OOM fix;
- OBD_FAIL_OST_BRW_WRITE_BULK3 uses 0x258 on this tree (0x256 is taken); OBD_FAIL_OST_CSDC_WRITE_BAD stays 0x257;
- the local enable/disable_compression definitions removed by this patch were already provided by test-framework.sh here;
- test_1123 keeps the compression_supported gate added when it was introduced;
- the local RAMKB in subtest_2000 gave way to the new file-scope RAMKB.
- test_1080 is excluded: it depends on the header-checksum change (Was-Change-Id: I24a9ab9cb7bea1208ada23aa6550127fe6a55017), which is not part of this series; it should return together with a port of that change.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: more tests for csdc

- synthetic tests to cover all simple cases like full/partial
  write/overwrite/append/read which verifies the operations
  doing same IO to tmpfs
- a test simulating a bulk error
  to ensure all bufs/pages are released
- few tests simulating read/write/truncate to corrupted chunks
- a test checking how much IO different cases need
- add own fsx with compression enforced
- add own dbench with compression enforced
- add own iozone with compression enforced (slow)

tests: cleanup sanity-compr.sh

Fix style issues in sanity-compr (whitespace, variables).
Add check_and_set_extents_dense() helper in a few places.
Re-enable test_1012 now that server-side code is fixed.

DDN-bug-id: EX-10107
Was-Change-Id: I7f5e5b759a8081efed4700d1c963afde135e690f

tests: improve sanity-compr/1000c

the existing test doesn't take into account that allocations can be
done in few racing threads, thus allocation is not serialized. thus
it can be non-sequential, but still dense.
the test should sort filefrag's output by physical location first,
then count gaps.

DDN-bug-id: EX-10763
Was-Change-Id: Ie3e1122e549da0a181f8cf5dbb0aa4304d1c3157

tests: Use less data in csdc DIO iozone test

"EX-6266 llite: DIO fallback on compressed files" forces
to switch any DIO operation to the buffered one. This behaviour
breaks any iozone expectations and the test goes too slowly.
Having such an iozone test with DIO and compression is a good
idea, but let's reduce the testing file size so this test pass
for reasonable amount of time.

DDN-bug-id: EX-10445
Was-Change-Id: I8dee83398a4e5249ad254d061bafa89240feb2e2

tests: Use less data in iozone test

"EX-10445 tests: Use less data in csdc DIO iozone test" sets to
use 1/10 of available space for DIO iozone test. The same chage
should be done for buffered iozone test too, because the test it
too slow for large disks and slow VMs.

DDN-bug-id: EX-10445
Was-Change-Id: I40ca9783826448885cb4d8d3ca889d94df26f257

csdc: execute tests if compression is enabled

"csdc: Provide finer grained enable_compression control" patch
enables CSDC by default on all archs except aarch64 and ppc64le.
No need to execute enable_compression/disable_compression for
each test, so this test changes them to compression_enabled
check that skips test if compression is disabled for the arch.

This patch forces to skip compression tests for ppc64le and
aarch64 so they stop adding noise to the test results.

DDN-bug-id: EX-8981
Was-Change-Id: I2b5deba0672f3bad028f383cf1205e66c2b05529

tests: disable compressibility checks

some tests want to mix compressible and incompressible data for
verification. EX-8355 can disable compression for a period when
incompressible data met. disable this functionality in few
tests.

DDN-bug-id: EX-10653
Was-Change-Id: I1da67af9802e92b29651b8b532ab21c3cc75dff2

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9858
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icbc13ee239a68d165b77481678665e80690564b2

68408 master J ✓B ✗T –R 0/2

LU-10026 tests: disable sanity-compr/1011

build failure (#130380)

csdc 1 unresolved XS +1 −0

53d

build failure (#130380)

1 unresolved thread(s) await your reply — last from Timothy Day

build #130380 FAILURE

no vote yet: Alex Zhuravlev

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-09-10 14:36 Timothy Day patchset comment — What's the significance of these 'upstream remarks for reference'? Differences between your in-house change vs. the version in Gerrit?

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

Remainder of a later snapshot of the test_2001 change (same original Change-Id): a single always_except for test_1011, tag renumbered to LU-10026. New Change-Id, original kept as Was-Change-Id.

Timothy Day PS1 · 2026-09-10 14:36

What's the significance of these 'upstream remarks for reference'? Differences between your in-house change vs. the version in Gerrit?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: disable sanity-compr/1011

Add sanity-compr test_1011 (grants let a compressed file fill
OST) to the always-except list; the failure is tracked
separately.

This is the remaining delta of a later snapshot of the change
that added sanity-compr test_2001.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10142
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I6dd2643c93739fd6abaeb7add27f530a57fdcf64
Change-Id: I9570a6f71145f4f75c2bb2a09f8387936c88b114

68385 master J !B ✗T –R 0/2

LU-10026 utils: lfs setstripe adds -E with -Z

build failure (#130357)

csdc 1 unresolved S +22 −5

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130357 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- test_1014 is anchored after the 1013x tests (numeric order; the original anchored it after test_1002, but this series adds 1013a-1013e in between).
- Version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: lfs setstripe adds -E with -Z

When specifying a layout with "lfs setstripe -Z" it will ignore
this option if no PFL component is specified with "-E".

Instead, "lfs setstripe -Z" should automatically upgrade the file
layout to a PFL layout so the compression parameters are saved.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9708
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I29cc373fabd352d6f8b6781c238806b75cce7057

68389 master J !B ✗T –R 0/2

LU-10026 csdc: inherit CSDC params unless overriden

build failure (#130361)

csdc 1 unresolved L +258 −25

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130361 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- Rewritten onto the current naming: LL_COMPR_TYPE_SPEC_NONE = 9 added (MAX moves to 10); wire asserts extended accordingly (this tree carries LL_COMPR_TYPE_MAX asserts the original did not have).
- compr_type_table: the NONE row loses its name, SPEC_NONE row ("none") added before the MAX sentinel - the compress_type_2str()/parse helpers were already in the matching form.
- The inheritance-hint struct uses cd_compr_chunk_lum_bits with this series' lum-bits semantics (offset above COMPR_CHUNK_MIN_BITS - no +16 at use sites), and lod_adjust_compr_chunk_size() is called in its take-by-value/return-by-value form.
- lod_qos_parse_config: the convert_plain arm skips all lcme-derived assignments including fields this tree has that the original context did not (llc_mirror_link_id, llc_dstripe_count, llc_cstripe_count).
- SPEC_NONE clearing blocks in lod_parse_striping/lsm_unpackmd applied at the current field naming.
- test_1000d version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: inherit CSDC params unless overriden

If a parent directory has been set CSDC parameters, the file created
under it needs to inherit the CSDC parameters unless another CSDC
parameters or an empty CDSC has been specified for a component of the
file. The file create method includes touch, setstripe a plain layout
file, or setstripe a composite layout file.

As "-Z none" would be used to set compression type to 0 and server
would treat it as no CSDC parameters has been set, so "-Z none"
can not be used to stop inherit parent directory's CSDC inheritance,
so this patch introduces another compression type
LL_COMPR_TYPE_SPEC_NONE to associate with "none" compresstion type,
and it can be used to stop the CSDC inheritance from parent
directories, when user uses "-Z none" to reset compression type,
this patch would set the compression type to LL_COMPR_TYPE_SPEC_NONE.

If we use patched new client to create a file with "-Z none" upon old
servers, the old servers cannot recognize this new
LL_COMPR_TYPE_SPEC_NONE compression type, and it just records the
type as is in the layout.

On old clients "lfs getstripe" will show "unknown" to the component's
lcme_compr_type, and on new clients, "lfs getstripe" will show "none"
to the lcme_compr_type.

When we write data to this file, both clients work but with different
behavior. On old client, with "unknown" compression type, it chooses
a fallback algorithm to compress the data, twisting the original
intention not to compress the component; while on new client, it will
treat "none" component as uncompressed one as intended.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10201
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibbdb3f6489cf7e19bfebe075bf88580f39ed79e4

68392 master J ✗1B ✗T –R 0/2

LU-10026 lov: overstriping + compression is OK

build failure (#130364)

1 unique failure csdc 1 unresolved XS +6 −0

66d

build failure (#130364)

janitor: 1 test failure unique to this patch — sanity-slow@zfs:test_64b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-slow@zfs:test_64b	seen in 2 other reviews

build #130364 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The lov_pattern_supported() hunk re-applied against the current helper (which has a RAID0|PARITY row and no MDT|COMPRESS row).
- lov_pattern_available() postdates this patch's original base and needed the same OVERSTRIPING|COMPRESS row added - without it the pattern is "supported" but never "available" and setstripe fails with -EINVAL.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: overstriping + compression is OK

it should be OK to enable compression on overstriped files.

lov: extend supported normal striping

where compression + overstriping is OK

DDN-bug-id: EX-10854
Was-Change-Id: I2474329d5b1d8684353efe8423f225e7ee779da3

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10854
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I45d14b3fc5badcb04652df2b64524c143b45c528

68394 master J !B ✗T –R 0/2

LU-10026 csdc: Added client parameters for compression types.

build failure (#130438)

csdc 1 unresolved L +243 −50

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130438 FAILURE

no vote yet: Chakshu Kansal

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The original's common target-side helpers are seq_file/procfs-based; on this branch mdt.*.compress_types and obdfilter.*.compress_types are sysfs attributes, so the common helpers were adapted to a mask-based sysfs form (lprocfs_compress_types_mask_show/store in lprocfs_status.c) and the mdt/ofd show/store delegate to them (renamed with mdt_/ofd_ prefixes to coexist with the new common client compress_types_show).
- The client-side compress_types_show (osc/mdc, reads ocd_compress_types, "disabled" without the connect flag) is as in the original.
- Tests 1013d/1013e anchored after test_1013c. The MDC-specific connect-flag check in test_1013e is kept as-is: it probes the MDC import, and compression_supported() probes the OSC import, so it is not a substitute (MDT-side compression negotiation stays disabled).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: Added client parameters for compression types.

Added new parameters 'osc.*.compress_types' and 'mdc.*.compress_types'
on the client side to report the list of available compression
algorithms.

DDN-bug-id: EX-12289
Was-Change-Id: Iba832e5351fc159e49916df2c06b75ad5ad37638

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12289
Signed-off-by: Chakshu Kansal <ckansal@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I07cd0531e9c2934c28dd999b21310d18a988e2f7

68406 master J !B ✗T –R 0/2

LU-10026 llite: skip layout changes if update is held

build failure (#130378)

csdc S +15 −0

66d

janitor run: 9 failing config(s), none unique to this patch — janitor results

build #130378 FAILURE

no vote yet: Alex Zhuravlev

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: skip layout changes if update is held

skip minor layout changes (like incompressability) when the client
helds UPDATE lock as MDT grabs UPDATE lock for this kind of changes.

llite: do not update layout from fsync

fsync can be called from within write. in this case there would be one
active IO (from LOV's point of view). if we update the layout at this
point, then fsync would need to refresh the layout to proceed, but
given there is an active IO, LOV would block layout refresh waiting
for all IOs to finish - a dealock.

DDN-bug-id: EX-10979
Was-Change-Id: I6d1cd7787f4c0d9f75f01030ed1235279bb6d301

llite: do not update layout at umount

umount can race with ptlrpcd finishing IOs and locks.
at the same time, umount resets sb->s_root to NULL which
is needed to form new RPCs, so it's safer to skip minor
layout update.

DDN-bug-id: EX-11166
Was-Change-Id: Ie9df33e95317bf38452ddd3afdfa30b31932bdb4

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10857
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I52e31a9c15967dd92714e6102e6b16d22695f81b

68410 master J !B ✗T –R 0/2

LU-10026 llite: issue compressiblity change in workqueue

build failure (#130382)

csdc 3 unresolved M +118 −56

66d

build failure (#130382)

2 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130382 FAILURE

no vote yet: Artem Blagodarenko, Zhenyu Xu

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The test_1021 tweaks (sleeps, echo moves) re-applied against the unwrapped assertion blocks; ci_need_change_layout renamed as in the original.

reply PS1 2026-08-26 13:39 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:292 — warn: vvp_layout_minor_change():passing zero to 'ERR_CAST'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:39

warn: vvp_layout_minor_change():passing zero to 'ERR_CAST'

reply PS1 2026-08-26 13:39 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:294 — error: vvp_layout_minor_change():'vlcw' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:39

error: vvp_layout_minor_change():'vlcw' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: issue compressiblity change in workqueue

This patch uses kernel-global workqueue to handle the file's minor
layout change RPC equest. At vvp_io_fini() when it find that the
file's compressibility should be changed, it creates a work and
insert the request to the kernel workqueue.

This would prevent the layout minor change RPC issued from within an
IO, and it could avoid the possible double lock the LOV conf lock.

lov: replace switch compressibility

The switch compressibility flag results in weird behavior -
it tells us we are to flip what's there, but results in some
very hard to read code.

I *think* the existing code is probably correct in absence
of race conditions, but an instruction to "flip" rather than
set is very confusing and could easily do the wrong thing
if there's a race which changes the compressibility value.

Replace it with two explicit bit flags.

DDN-bug-id: EX-10857
Was-Change-Id: I3b1d9f8d635e0bdbe67d5806127727a7264d26be

Singed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10857
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic4345b1387288ab11ce9061f40b056b07cb976bf

68364 master J !B ✗T –R 0/2

LU-10026 ofd: avoid assertion on large read request

build failure (#130339)

csdc 5 unresolved S +24 −5

66d

build failure (#130339)

4 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130339 FAILURE

no vote yet: Artem Blagodarenko

threads: 4 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Commit message no longer references a specific downstream release version for the affected old clients.

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:500 — warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:504 — warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:684 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:688 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: avoid assertion on large read request

In case an old client using CSDC sends a
read IO that will overflow buffers on OST, OST should
return an error rather than crash.

A similar check was previously added to the write path,
but was not added to the read path.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10359
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I7dc4c1a7a19c71d757d0a505003c3012edbf987a

68366 master J !B ✗T –R +1/2

LU-10026 ofd: skip reading compressed tails

build failure (#130341)

csdc 2 unresolved M +37 −28

66d

build failure (#130341)

1 thread(s) waiting on others

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130341 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:02

threads: 0 need your reply · 1 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied on the integration patch with only the series' field naming (ol_compr_chunk_lum_bits) where the original said _log_bits.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: skip reading compressed tails

if a tail is being overwritten, then we don't need to read
and decompress old content.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10116
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icde88d9fbd0ba0bdf53497c0c14d1215fa9cad73

68363 master J !B ✗T –R +1/2

LU-10026 ofd: reject non-aligned compressed niobufs

build failure (#130338)

csdc 1 unresolved S +7 −3

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130338 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 10:24

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied on the integration patch with only the series' field naming (ol_compr_chunk_lum_bits) where the original said _log_bits.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: reject non-aligned compressed niobufs

a client can send non-aligned compressed niobuf by mistake,
OST should reject it with an error.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11092
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2a61f13b5be17ca22c3a92b7d5b7b8bfd739c1eb

68365 master J !B ✗T –R 0/2

LU-10026 csdc: finer enable_compression control

build failure (#130340)

csdc 1 unresolved S +20 −0

66d

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130340 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Note for reviewers: the #if (PAGE_SIZE > 4096) guards in userspace (liblustreapi_layout.c) rely on the build host's page size, not the runtime client's; probably wants a runtime sysconf(_SC_PAGESIZE) check. Kept as the original for now.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: finer enable_compression control

On all architectures other than aarch64 and ppc64le enable_compression
is now enabled by default. lfs warning message is gone.

To use CSDC on aarch64/ppc64le (on your own risk)
llite.*.enable_compression=1 should be set. lfs
set_stripe command still prints a warning message in this case.

Test-Parameters: forjanitoronly
Test-Parameters: trivial
DDN-bug-id: EX-9074
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic8edc5bbeb8f9a3cd34ad3fc4e8c78e59f4cc34f

68367 master J !B ✗T –R +1/2

LU-10026 ofd: don't write blocks after compressed size

build failure (#130342)

csdc S +11 −0

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130342 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:00

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: don't write blocks after compressed size

due to limits in ib/rdma/lnet, client has to send full pages.
if client's pagesize is bigger then server's pagesize, then
we want to skip writing useless bytes from those full pages
containing compressed data.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10712
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I61c6fe72af1d2e8679d9fde303a561e80a287746

68368 master J !B ✗T –R 0/2

LU-10026 csdc: tune allowed compression type on server

build failure (#130343)

csdc 1 unresolved L +258 −18

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130343 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- The patch re-created a libcfs_string.h/.c pair to host string-mask helpers; master already provides cfs_str2mask()/cfs_mask2str() with u64 masks (lnet/libcfs/debug.c), so both files were dropped and the new compress_types stores call the existing helper (~0ULL masks). A hunk that narrowed the existing capability-mask store from ~0ULL to ~0 was dropped for the same reason.
- mdt.*.compress_types and obdfilter.*.compress_types are implemented as sysfs attributes (the target-side procfs vars tables the original patch extended no longer exist on master).
- The obd_compress_types mask enum / COMPRESS_NAMES use the series' type naming (lz4 for the HC type) and already include zstd, which is a valid layout type at this point in the series.
- OST-side only, as in the original: OBD_CONNECT2_COMPRESS is advertised in OST_CONNECT_SUPPORTED2 and requested by the OSC; MDT enabling remains pending (test_1013b keeps its unconditional skip; its old-version negative-check block, meaningless on master, was dropped).
- compression_supported() in test-framework.sh now probes the OSC import for the compressed_file connect flag instead of comparing versions. (The OSC import, not the MDC one, because MDT support is not enabled.)
- test_1013a/b were placed before test_1015 (the tests they were anchored to arrive later in the series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: tune allowed compression type on server

Use lctl get_param {mdt|obdfiler}.*.compress_types to list supported
compression types on server.

Use lctl set_param {mdt|obdfilter}.*.compress_types="+gzip-lzo" to
add gzip to and delete lzo from existing compression types on server.

Server will negotiate supported compression types with client in
ocd_compress_types and client import stats could show the negotiated
supported compression types in "lctl get_param {mdc|odc}.*.import"
connect_data section:

import:
  ....
  connect_data:
     ....
     compress_types: [ fast,best,gzip,lz4fast,lz4hc,lzo ]

The OST support for the connect flags is enabled in this patch, but
MDT enabling is pending unaligned compression support for DoM files.

test: refine sanity-compr.sh/test_0a

Only change and verify compression types negotiation between client
and OST0000.

DDN-bug-id: EX-9802
Was-Change-Id: I307f3826973f48791b7fbf4cdfa121401b059b0e
Was-Change-Id: Ic22685d2b21e872c01d90f2c3062b5d0d9b92ec2

test: sanity-compr.sh/test_0a failure

* move sanity-compr.sh test_0a/test_0b to test_1013a/test_1013b
* explicitly specify client's hostname to get client's negotiated
  compression types, as test environtment could mount several clients

DDN-bug-id: EX-9802
Was-Change-Id: Ic091f55a59ad9cfdaf3a94a58d44b372070c7875

Test-Parameters: forjanitoronly
DDN-bug-id: EX-6269
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I18943352e25ed9d5fe82442df9f00a7ef388f242

68369 master J !B ✗T –R 0/2

LU-10026 llite: set STATX_ATTR_COMPRESSED in ll_iocontrol()

build failure (#130344)

csdc 1 unresolved S +10 −0

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130344 FAILURE

no vote yet: Jian Yu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

LLIF_COMPR_FL renumbered 6 -> 8 (bit 6 is LLIF_ACL_VALID on master). Subject shortened to fit the 62-column limit.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: set STATX_ATTR_COMPRESSED in ll_iocontrol()

This patch extracts the compression flag LUSTRE_COMPR_FL from
mbo_flags and set STATX_ATTR_COMPRESSED flag in ll_iocontrol()
to please lsattr and other e2fsprogs tools.

Test-Parameters: forjanitoronly
Test-Parameters: trivial testlist=sanity-compr
DDN-bug-id: EX-8669
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I14d2082a6719a1ca5708f7aef7a2fb0f085ca63c

68374 master J ✗2B ✗T –R 0/2

LU-10026 csdc: store compression info in FID EA

build failure (#130348)

2 unique failures csdc 1 unresolved M +37 −65

66d

build failure (#130348)

janitor: 2 test failures unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b, sanity-quota@ldiskfs+DNE:test_80

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 4 other reviews
sanity-quota@ldiskfs+DNE:test_80	seen in 73 other reviews

build #130348 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied cleanly. Note for reviewers: this changes where the PFID EA is stored for small inodes (no longer embedded in LMA for <=256-byte inodes) - needs interop/lfsck review on master.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: store compression info in FID EA

Store compression information in OST-object's FID EA, and lfsck could
use it to recover the MDT-object layout EA from orphan OST-object(s).

2.15 Lustre may embed PFID and layout stripe info in LMA EA, this
patch would clear them from LMA EA and store them with compression
info directly into FID EA thereafter.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8038
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iacac04601b73f85d9bc057b8dd34a5004248dac4

66612 master J !B ✗T –R 0/2

LU-10026 csdc: sending compression info to server

build failure (#130320)

csdc 4 unresolved M +100 −13

66d

build failure (#130320)

3 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 11 failing config(s), none unique to this patch — janitor results

build #130320 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 3 need your reply · 0 waiting on others · 1 own notes · 0 bot

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This is an on-wire change: the client now fills bytes in the obdo (formerly o_padding_4) that are sent to the OST/MDS. Test-Parameters: forjanitoronly doesn't exercise interoperability. Consider requesting old-server/new-client and new-server/old-client runs with serverjob/serverbuildno and clientjob/clientbuildno so the padding-reuse stays wire-compatible with unpatched peers.

note PS2 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS2 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase notes: kept the base's comment wording for the chunk-bits field
(2^(16+chunk_lum_bits)); the lov_lsm2layout() wrapper is added alongside
the base's compr_name_from_type() helper in lov_internal.h.

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This comment reword looks unrelated to sending compression info to the server; is it meant to be in this patch? Also the code computes the chunk size with COMPR_CHUNK_MIN_BITS (COMPR_GET_CHUNK_SIZE()), not LOV_MIN_STRIPE_BITS; they happen to both be 16, but referencing the constant the code actually uses would be clearer.

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

This isn't a bug, but after this patch lov_lsm2layout() has no remaining callers (both lov_io.c and lovsub_object.c now call lov_lsm2layout2()). Folding the olc parameter into lov_lsm2layout() and updating those two call sites would avoid the version-suffixed name and the now-dead wrapper.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: sending compression info to server

Client fills in layout compression info into obdo and passes it
to server.

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ieb5d7b3609da41f35f8622ed6116f19ce7567ddb

68371 master J ✗1B ✗T –R -1/2−

LU-10026 osc: Add counters for compressed data

build failure (#130346)

1 unique failure csdc 1 unresolved L +544 −265

66d

build failure (#130346)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 5 other reviews

build #130346 FAILURE

no vote yet: Sebastien Buisson, Artem Blagodarenko, Zhenyu Xu, Vitaliy Kuznetsov

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- The removed dead decompression helpers (decompress_chunk_in_lnb / decompress_rnb) were deleted against their current (folio/accessor) form.
- obt_compr_stats placed after the histogram member master added since.
- The osc stats_compr file and the compress_type_fast/best entries live in the debugfs vars table; the stats_compr fops therefore use LDEBUGFS_SEQ_FOPS (LPROC_SEQ_FOPS produces struct proc_ops on kernels >= 5.6, which the CI -Werror build rejects for debugfs entries). The ofd counter-init sits next to the reworked kernel-doc of ofd_stats_counter_init().
- New tests' old-version gates converted to compression_supported; the counter-reset calls are unconditional (the parameters always exist on a server with this series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: Add counters for compressed data

This patch is the first of two patches that add counters
to track client/server-side data compression statistics.
This patch add new compr_stats file in osc.*.compr_stats.

From added counters:
1. Size of compressed/uncompressed chunks written/read by
   client to compressed files, in chunk/bytes;
2. Compressed page counter;

ofd: Add counters for compressed data

This patch is the second of two patches that add counters
to track server-side data compression statistics.
From added counters:
1. Size of compressed/uncompressed chunks read by
   client to compressed files, in chunk/bytes;

Example:
	obdfilter.lustre-OST0000.stats_compr=
	snapshot_time             1715876501.602669648 secs.nsecs
	start_time                1715873670.819396144 secs.nsecs
	elapsed_time              2830.783273504 secs.nsecs
	compressed_objects        2 samples [reqs]
	uncompressed_objects      1 samples [reqs]
	compr_bytes_read          885 samples [bytes] 557 126974 52174403
	compr_bytes_read_raw      893 samples [bytes] 16384 131104 115885042
	incompr_bytes_read        8 samples [bytes] 16384 16384 131072
	compr_chunks_read         885 samples [reqs]
	incompr_chunks_read       8 samples [reqs]

DDN-bug-id: EX-7729
Was-Change-Id: I0998b1facaffd3e84b8d9429c5674d8d9ad60020

csdc: rename client-side stats

Some CSDC client-side stats can be confusing for the uninitiated:
- write_pages_compr is the number of pages from the 'raw' file that
  could actually be compressed;
- write_pages_uncompr is the number of pages from the 'raw' file that
  count not be compressed;
- write_chunks_compr is the number of chunks from the 'raw' file that
  could actually be compressed;
- write_chunks_no_compr is the number of chunks from the 'raw' file
  that count not be compressed;
- read_pages_compr is the number of pages from the 'raw' file that
  were actually compressed;
- read_chunks_compr is the number of chunks from the 'raw' file that
  were actually compressed;

So rename those parameters as follows:

    write_pages_compr -> write_pages_compressible
    write_pages_uncompr -> write_pages_incompressible
    write_chunks_compr -> write_chunks_compressible
    write_chunks_no_compr -> write_chunks_incompressible
    read_pages_compr -> read_pages_compressed
    read_chunks_compr -> read_chunks_compressed

And also change the corresponding field names in struct client_obd to
make them more explicit.

DDN-bug-id: EX-10914
Was-Change-Id: Ia876e0cde435095493e0b7457ce7f363ea86146d

test: sanity-compr.sh/test_1001 error

sanity-compr test_1001: operand expected (error token is "+ ")

Some compression stat counters could be 0 and not shows in the
stats_compr stats, so we need to assign 0 to the counter if
necessary.

DDN-bug-id: EX-10088
Was-Change-Id: I87948a61e591680d105a8a98a4cde4fba634e5fe

tests: fix check compr_chunk in test 1020

Skip test 1020 in sanity-compr, which depends on fixes
applied to the code that were not available in older
versions.

Temporarily add an additional tolerance of 1 unit when
checking the number of chunks in test 1020 in
sanity-compr.sh

DDN-bug-id: EX-10914
Was-Change-Id: Ia1c071d754217127decbf4d4397a58b512808843

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7729
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I091153480e53309c641d39f271bef536296dc09e

68378 master J ✗1B ✗T –R +1/2

LU-10026 ofd: skip decompression

build failure (#130352)

1 unique failure csdc 2 unresolved XS +5 −0

66d

build failure (#130352)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

1 thread(s) waiting on others

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 3 other reviews

build #130352 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 10:43

threads: 0 need your reply · 1 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

One context line updated to the folio form of is_chunk_start().

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: skip decompression

we can skip decompression if new data overwrite all raw bytes of
compressed chunk.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10103
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3511c6bff2a8c4c8d98532d025beead78c597a5b

68377 master J ✗1B ✗T –R 0/2

LU-10026 csdc: process all compressed chunks in rnb

build failure (#130351)

1 unique failure csdc S +22 −12

66d

build failure (#130351)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 2 other reviews

build #130351 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: process all compressed chunks in rnb

It is supposed, that every rnb contains only
continuous pages. There is always a gap after
a compressed chunk, so it is not expected to
have any other chunk in the same rnb after the
compressed chunk.

Compressing with chunk size >= than page size
is an exception to the rule above. After the
compression, data size is rounded up to page
size. So, it is possible that another compressed
chunk goes after the compressed one.

To support such a sutuation an aditional
loop was added to ofd_map_niobuf_to_chunks().

This patch is useful for ARM archs with 64k page.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibaac18e4d046e020a7ad60b8a436c789ba4c990b

68380 master J !B ✗T –R 0/2

LU-10026 csdc: exponential backoff of incompressiblity check

build failure (#130354)

csdc 1 unresolved M +144 −90

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130354 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:38 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:38

[Marc Bot] upstream remarks for reference:

Chunk-bits field naming follows the series (llc_compr_chunk_lum_bits).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: exponential backoff of incompressiblity check

If a file is incompressible and after a round of compressibility
check, if it remains incompressible, this patch would doubles the
skip bytes before running into another round of compressiblity
check.

This patch adds a state machine for update_compression_info(), the
states could be CS_INCOMPR, CS_COMPR, CS_CHECK, CS_CHECK_INC.

CS_INCOMPR    --  file is not compressible
CS_COMPR      --  file is  compressible
CS_CHECK      --  was in CS_COMPR, is now under check phase
CS_CHECK_INC  --  was in CS_INCOMPR, is now under check phase

state transition:
INCOMPR --> INCOMPR   : skip bytes has not been passed
INCOMPR --> CHECK_INC : skip bytes has been reached, in check phase
COMPR   --> CHECK     : in check phase, checked bytes has not been
                        reached
CHECK*  --> CHECK*    : in check phase, checked bytes has not been
                        reached
CHECK*/COMPR
          --> COMPR   : file is determined to be compressible
CHECK/COMPR
          --> INCOMPR : file is determined to be incompressible, reset
                        the skip bytes check length
CHECK_INC --> INCOMPR : file is determined to be incompressible,
                        double the skip bytes check length

====

Test with 4 OSTs system, target file layout is set as:
lfs setstripe -Eeof -c-1 -Z lz4 --compress-chunk=64 $tf

and src file is 336M bz2 file, using "dd if=$src of=$tf bs=4M" copy
this bz2 file to target file.

w/o the patch, copy uses about 7 seconds
** start 1724216088 **
84+1 records in
84+1 records out
352625400 bytes (353 MB, 336 MiB) copied, 3.76366 s, 93.7 MB/s
** end 1724216095 **
...7 seconds

w/ the patch, copy uses about 4 seconds
** start 1724215237 **
84+1 records in
84+1 records out
352625400 bytes (353 MB, 336 MiB) copied, 2.07946 s, 170 MB/s
** end 1724215241 **
...4 seconds

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I42105d3e137bf50897ff5c67654ca33552b23102

68379 master J !B ✗T –R 0/2

LU-10026 utils: return error on invalid compress level setting

build failure (#130353)

csdc 1 unresolved S +11 −4

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130353 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:38 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:38

[Marc Bot] upstream remarks for reference:

- The strtoul/end-pointer validation this patch adds was already carried (in stricter form) by the earlier level-handling patches of this series; those hunks were dropped.
- The parse-time rewrite of low lz4 levels to the lz4fast default was removed here (matching the original series' final behaviour): the stored level is kept and the compression engine falls back to lz4fast when allocating the compressor. This also makes the negotiated-type test lists in sanity-pfl roundtrip correctly.
- The new negative test was renamed test_1013a -> test_1013c: 1013a already exists (type negotiation test); the duplicate name would have silently shadowed it (latent issue in the original series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: return error on invalid compress level setting

lfs setstripe -Z <compress_type>:<compress_level> does not report
error on invalid <compress_level>.
An explicit low level for the lz4 type (below the lz4hc minimum) is
no longer silently rewritten to the lz4fast default at parse time;
the stored level is kept and the compression engine falls back to
lz4fast for such levels when allocating the compressor.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10186
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie794a6d4cf208a9c66d1d179c157fee756477aee

66279 master J !B ✗T –R +1/2

LU-10026 sec: disable compression for encrypted files

build failure (#130315)

csdc 2 unresolved S +10 −3

66d

build failure (#130315)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #130315 FAILURE

reviews: Sebastien Buisson +1 2026-08-26 09:47

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(style) The body explains why compression must be disabled for encrypted files, but that logic (compressed = 0; goto skip_compression) already lives in the parent compression patch. What this change actually does — throttling the existing CWARN to once per 24h and adding `always_except LU-10026 66` in sanity-sec.sh — isn't mentioned. Could the message describe the change it makes?

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) There is no test_66 in sanity-sec.sh (tests jump 65 -> 68), so this exclusion is a no-op in this tree. Which patch adds sanity-sec test_66, and should the always_except land together with it? Worth also confirming sanity-sec is the intended suite rather than sanity-compr.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 sec: disable compression for encrypted files

In case a read-modify-write io pattern is carried out on a compressed
file, it has to be handled on server side.
But because encryption cannot be done on server side for security
reasons, we are not able to handle that kind of io pattern if the file
is encrypted + compressed.
So just disable compression for all encrypted files.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I84881fb1235f015d022751d4cce2d43a7231c2b4

66280 master J !B ✗T –R 0/2

LU-10026 osc: disable CPU-access features for RDMA only pages

build failure (#130316)

csdc 1 unresolved S +27 −10

66d

build failure (#130316)

1 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130316 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) `req` was just allocated above (ptlrpc_request_alloc_pool for writes, ptlrpc_request_alloc for reads) and every other error path in this function frees it with ptlrpc_request_free(req) before returning. These two early returns leave without freeing it, and *reqp is only assigned on the success path, so the caller can't release it either:

    if (compressed && !page_access_allowed)
        RETURN(-EINVAL);
    if (inode && IS_ENCRYPTED(inode) && !page_access_allowed)
        RETURN(-EINVAL);

For OST_WRITE the request comes from osc_rq_pool, so a GPU-direct write to a compressed or encrypted file leaks a pooled request each time and can eventually drain the pool. Should both routes free req first?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: disable CPU-access features for RDMA only pages

Pages which cannot be accessed by the CPU are referred to
as RDMA only pages.  If pages cannot be accessed by the
CPU, it is impossible for us to do compression,
encryption, checksums, or short-io (data-in-RPC) on them.

This patch disables compression and encryption for these
pages and cleans up the code so checksums and short-io
are disabled by the same code.

The only user of RDMA only pages today is Nvidia's GPU
direct, so this patch disables compression and
encryption with GPU direct.

NB: We eventually intend to handle compression for
GPU direct with server side compress/decompress.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iad9311617cddf27d3ff75a17429499c573067ea0

66611 master J !B ✗T –R 0/2

LU-10026 lfs: migrate compressed file without stripe info

build failure (#130319)

csdc 2 unresolved M +76 −14

66d

build failure (#130319)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #130319 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) The diff also touches Documentation/man1/lfs-mirror-extend.1 (documenting compression options for SETSTRIPE_OPTIONS), which the message doesn't mention and which is unrelated to the migrate fix. Is that hunk meant to be part of this patch, or should it be split out / called out in the body?

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This prints `fd`, which is always -1 on this path, so the message reads "Cannot open file X: -1" with no useful detail. The error is in errno from open() – strerror(errno) would be the informative value.

Also note the caller (lfs_migrate/lfs.c) already emits its own llapi_error() when this returns < 0, so this adds a duplicate line. Using llapi_error(LLAPI_MSG_ERROR, errno, ...) here (as the ENOSPC path below does) would decode errno and respect the message level instead of writing to stderr unconditionally.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lfs: migrate compressed file without stripe info

lfs migrate file without specifying stripe info will get layout info
from the file as the target layout template, and
llapi_layout_get_by_xattr() tries to convert LOV_PATTERN_* values
to user scope LLAPI_LAYOUT_* values, while LOV_PATTERN_COMPRESS
is missed in this conversion.

This patch add a function llapi_pattern_from_lov() to handle this
conversion specifically.

This patch also add more error messages for llapi_layout_file_open().

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I49a43cc7761cd2baed7a5da7d4e7cff2152ff9bb

66615 master J ✗1B ✗T –R 0/2

LU-10026 csdc: prohibit set compression upon encrypted file

build failure (#130322)

1 unique failure csdc 1 unresolved L +247 −35

66d

build failure (#130322)

janitor: 1 test failure unique to this patch — conf-sanity4@zfs:test_123aj

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
conf-sanity4@zfs:test_123aj	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #130322 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS2 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS2 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: kept the base's newer compression-parameter assignment block
in lod_use_defined_striping() (gated on LOV_PATTERN_COMPRESS, with the
extension-component chunk-adjustment deferral); this patch's encryption
checks and attr threading apply on top unchanged.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: prohibit set compression upon encrypted file

Setting compression layout component upon encrypted file is not
allowed for now.

This patch add this check on MDS in creating file with layout,
adding/merging new mirror to existing file.

Test-Parameters: forjanitoronly
Test-Parameters: testlist=sanity-sec env=ONLY=67,PTLDEBUG=-1
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I60d9f4bfce3a498f1eb3994c6276afb9d89c99a7

68356 master J !B ✗T –R +1/2

LU-10026 ofd: remove unused csdc bits

build failure (#130335)

csdc 1 unresolved L +150 −594

66d

janitor run: 15 failing config(s), none unique to this patch — janitor results

build #130335 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:10

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- The dt_read_prep() signature revert (dropping the decompression flag) was
  applied to both the header prototype and the function body in
  obdclass/dt_object.c (the bodies moved there upstream, LU-13139); the
  osd-wbcfs dbo_read_prep implementation lost the parameter as well.
- In ofd_preprw_write(), this patch removes the separate err_put label; the
  early "BRW to missing obj" failure (which happens after
  tgt_grant_prepare_write()) now puts the object and jumps to err_commit so
  the prepared grants are still committed.
- A whitespace-only obd.h hunk (tab/space churn) was dropped.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: remove unused csdc bits

some bits are not used with the ofd csdc integration patch
implemented, we can remove
those now. also the patch tries to sync the code with master
branch a bit.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9996
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8edb25fe322f4fafaac1b9a832dd5b4b72e52a42

68346 master J –B ✗T –R -1/2−

LU-10026 tests: improve/skip sanity test_460a

build failure (#130326)

M +108 −27

66d

build #130326 FAILURE

patchsets: PS1 08-25 PS2 08-26

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: improve/skip sanity test_460a

Skip sanity test_460a for el9.2 clients, since they appear to be
failing that test regularly, but no other distro client is.
Improve the log messages to see what stage is currently running.
Limit the "cmp --verbose" messages to one chunk, otherwise it
may print the entire 14MB test file (about 80 MiB of ASCII).

Move enable_compression() and disable_compression() functions
into test-framework.sh so that they can be used for all tests.

Set LFS_SETSTRIPE_COMPR_OK=y in enable_compression() since we
already know this is a preview and don't need it printed.

Allow sanity-compr.sh to specify SANITY_ONLY and/or SANITYN_ONLY,
and skip the other test script run if only one of them is set.

Test-Parameters: forjanitoronly
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8cb2f67689824513335f3fa65e9ea7519e3ebbe5

66263 master J !B ✗T –R 0/2

LU-10026 osc: add compression to can_merge_pages

build failure (#130261)

csdc XS +5 −2

66d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #130261 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: add compression to can_merge_pages

Some BRW flags are OK to have on only some pages in a BRW,
others are not.  can_merge_pages has a whitelist of the
flags which are safe to have on only some pages in a BRW,
and prints a warning if other flags are seen.

Add compression to the white list, because while all pages
in an niobuf must be compressed, it is normal to have only
some pages in a BRW compressed.

Prior to this patch, this warning was printing during
normal usage of compression.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia262d4fc878e5328bd956865047e997aa77946f0

66266 master J !B ✗T –R 0/2

LU-10026 osc: clear oi_write_osclock in lock fini func

build failure (#130263)

csdc S +12 −4

66d

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130263 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: clear oi_write_osclock in lock fini func

Move osc_io::oi_write_osclock clearance in osc_lock_fini() as
it's set in osc_lock_init().

Compression IO could possibly expand lock region and
osc_lock_set_writer() could access a osc_io that is not accessed
in osc_io_iter_init(), so that osc_io_rw_iter_fini() miss clearing
osc_io's oi_writer_osclock.

This patch moves the oi_write_osclock clearance in lock fini function
to match its creation in osc_lock_init().

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ied42f5befc1abd76aa10a7666eadb9a58e1f1783

66267 master J !B ✗T –R +1/2

LU-10026 llite: getting stripe info optimization

build failure (#130264)

csdc 2 unresolved M +63 −0

66d

build failure (#130264)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130264 FAILURE

reviews: Sebastien Buisson +1 2026-08-25 21:12

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) These CDEBUG() pairs print start_idx/end_idx before and after the rounding with no descriptive text, on the D_SEC (security) mask which is unrelated to readahead. Same at lines 813 and 816. They look like leftover debugging scaffolding; if kept, D_READA would be the right mask.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) vui_ra_pages is initialized once per read syscall in vvp_io_read_start() and only reset at IO teardown, but ll_readahead() runs once per page (ll_readpage()->ll_io_read_page()->ll_readahead()). Won't this += accumulate across every not-uptodate page of a compressed read, progressively inflating the RA window enlarge at lines 806-808 and ria_end_idx_min at line 863?

Also, the delta is taken from end_idx before the EOF clamp just below (lines 819-828), so vui_ra_pages can grow past the file size. ria_end_idx_min then feeds ll_read_ahead_pages() at lines 478-479, which raises ria_end_idx without an ria_eof guard - so readahead can extend past eof_index, the opposite of the "restrict readahead to eof" goal in the commit message.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: getting stripe info optimization

ll_lov_getstripe_ea_info() is expensive call and should be avoided
if possible. Let's use cached chunk size rather than get it from
stripe info every time.

llite: restrict readahead to eof

Compressed file readahead rounding needs to come before
readahead is limited to EOF.

DDN-bug-id: EX-7601
Was-Change-Id: I4e9e7fe63301c08efcb05f170726735593a9431d

llite: fix chunk_bits usage in readahead

For the minimum compression chunk size, chunk bits is zero,
so we cannot use if (chunk_bits) to determine if we're
doing compression.

This also fixes two other things:
1. A rounding error when rounding to chunk
2. Move rounding of end_idx to before first usage of
end_idx, so calculation of number of pages is correct
Without this, when the user reads 1 page or less, readahead
will calculate the readahead page count as 0 and will exit
without reading the chunk.

DDN-bug-id: EX-6127
Was-Change-Id: I273506fd4f6ed5f0b8b5020357fd7caf0531e61c

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id08487ec782f797e242e3f673c4a4dd8d526c9cc

68375 master J –B ✗T –R -1/2−

LU-10026 tests: Skip sanity 430c for CSDC layout

build failure (#130349)

csdc 1 unresolved XS +1 −0

66d

build #130349 FAILURE

no vote yet: Sarah Liu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: Skip sanity 430c for CSDC layout

Skip sanity test_430c until SEEK_HOLE is implemented for CSDC

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7680
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I7359262de9e3d1644d2a45b5336328bd8253f91b

68390 master J !B ✗T –R 0/2

LU-10026 mmap: add mmap support for compression

build failure (#130362)

csdc 1 unresolved L +446 −78

66d

janitor run: 31 failing config(s), none unique to this patch — janitor results

build #130362 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- OBD_FAIL_LOV_SKIP_CHECK_COMPR renumbered to 0x143a: the original's 0x1436 collides with OBD_FAIL_LLITE_TRUNC_PAUSE on this branch, and the original's sanity.sh test actually set fail_loc=0x1429, which is OBD_FAIL_LLITE_FORCE_BIO_AS_DIO - both the define and the test now use 0x143a consistently.
- The osd_lseek() compressed-object rejection was merged into the current form of the function (osd_get_filp_for_inode(), not osd_alloc_file_pseudo()); osd_xattr_get() prototype added to osd_internal.h.
- mmap_copy_file added to the current THETESTS line in tests/Makefile.am.
- test_1003 anchored before test_1004; its gate and the sanity.sh 460g gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 mmap: add mmap support for compression

This removes the EOPNOTSUPP for compression with mmap and
adds an mmap sanity test for compression.  This patch
removes all the restrictions for mmap, but we actually only
have unaligned read support right now, so the test is
deliberately simplified to only test reads.

A more complicated version which also tests mmap writes
comes later in the series, once read-modify-write is
supported.

The test tests mmap by copying data at several different
block sizes with several different compression chunk sizes.

tests: add mmap write test

This improves the existing mmap test to test mmap writing
as well as mmap reading.

DDN-bug-id: EX-7601
Was-Change-Id: I81840c7bbbefbc5c3bae6b270c2d94297a254d19

csdc: server reject FIEMAP/SEEK_HOLE|DATA on compr obj

Server return -EOPNOTSUPP if they get the FIEMAP and
SEEK_HOLE/SEEK_DATA requests upon compressed file objects.

DDN-bug-id: EX-8927
Was-Change-Id: I9f04fbb13a22cc83402d9989daab63a59367ff33

csdc: check ocd flag for fiemap/lseek

Currently client will not sending fiemap/lseek request to OST if
the file is a compressed one. This patch will check the
OBD_CONNECT2_COMPRESS flags and send the request if OST supports
compression as server would do the fiemap/lseek check instead.

DDN-bug-id: EX-8927
Was-Change-Id: I478cbf161044165fa31d4caa2336e9949fc626fe

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7644
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4a37b106831a903d90e8a8871e9a93baac4e201e

68407 master J !B ✗T –R 0/2

LU-10026 tests: minor cleanups to sanity-compr

build failure (#130379)

csdc 1 unresolved XS +6 −0

66d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130379 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The test_1001 hunk re-anchored (its context had a version guard this tree already dropped). One sub-section sentence of the original message did not apply to this content and was dropped.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: minor cleanups to sanity-compr

add/fix few subtests to remove big temporary files

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11112
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id04692cbd2443721070a7152186b38829b075907

68403 master J –B ✗T –R -1/2−

LU-10026 tests: fix 1009e and 1122 for arm clients

build failure (#130375)

csdc 1 unresolved S +12 −12

66d

build #130375 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

Applied against the LU-10026-tagged aarch64 except block.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: fix 1009e and 1122 for arm clients

arm clients use 64k page size, this should be reflected in
the tests checking stats on different operations.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10755
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia84ceef06c7ce96ae464abef919bf368583a70e5

68404 master J !B ✗T –R 0/2

LU-10026 tests: drop redundant gate in sanity-compr/2002

build failure (#130376)

csdc 1 unresolved XS +0 −3

66d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130376 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The original is a later snapshot of the change that introduced test_2002 (same Change-Id) and re-added subtest_2002/test_2002 at the file end, duplicating the definitions (the shell silently uses the second copy, and run_test 2002 would execute twice). The port keeps a single copy - the later version - in place; the effective delta is dropping the per-test gate now covered by the suite-wide gates. New Change-Id, original kept as Was-Change-Id.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: drop redundant gate in sanity-compr/2002

sanity-compr now checks compression support and enablement at
file scope, so test_2002 does not need its own skip gate.

This is the remaining delta of a later snapshot of the change
that introduced test_2002; the rest of it is already applied.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11120
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I7daec2d28fa32646e607291a6fcdd0e303351f67
Change-Id: I6b3bd30b8009fa702f9ddae83fdaf005fd40fd09

68350 master J ✗8B ✗T –R 0/2

LU-10026 tests: memory check depends on RAM

build failure (#130330)

8 unique failures csdc 1 unresolved XS +2 −7

66d

build failure (#130330)

janitor: 8 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1, sanity1@ldiskfs+DNE:test_36f +5 more

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 2 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews
sanity1@ldiskfs+DNE:test_36f	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_36h	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_44b	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_56ra	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_56x	seen in 37 other reviews
sanity1@ldiskfs+DNE:test_56xa	seen in 38 other reviews

build #130330 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- The Fixes: reference was remapped to the master commit carrying the same
  change (14fc034e983e "LU-18086 obdclass: page_pools shrinker fix").
- The debug echos this patch removes were already absent from the ported
  subtest_2000; the local count declaration is kept (count is still used
  only inside the function).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: memory check depends on RAM

Each open file consumes about 4.7KB of RAM, 300000
files consume about 1.3GB. At systems with 2.6GB RAM
it might cause an OOM with SLOW=yes option.
1. Consume 40% of RAM to check that page pools would be
shrinked.
2. Change createmany with memhog.
3. Export MEMHOG in init_test_env.

Test-Parameters: forjanitoronly
Fixes: 14fc034e983e ("LU-18086 obdclass: page_pools shrinker fix")
DDN-bug-id: EX-10278
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I58ae0a201456ed084db5c528007fe8d7a3782315

66277 master J !B ✗T –R 0/2

LU-10026 llite: restrict overwrites during preview

build failure (#130314)

csdc 1 unresolved M +80 −2

67d

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130314 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: ci_size_extend_compression:1 is placed inside the cl_io
bitfield next to the new ci_parity_io:1 member that master added in the
meantime (the non-bitfield loff_t ci_parity_eof follows the bitfield).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: restrict overwrites during preview

EX-7601 is an issue where when modifying a compressed file
we do not correctly read-up and re-write existing
compressed data.

To avoid this, we can only allow writes which are not
aligned to compression chunk size when they are not
overwriting existing data, ie, when they are extending the
file.

This returns EINVAL for all writes to compressed files
which are not either chunk aligned or extending the file.
This should prevent users from hitting the data corruption
issue but still allows some basic usage.  This is intended
just for the preview period.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0eea01e2249866a074afd0d0642fe6dce9a49664

65767 master J !B ✗T –R 0/2

LU-10026 utils: update handling of compr level for lz4

build failure (#130258)

csdc 6 unresolved M +107 −49

67d

build failure (#130258)

5 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130258 FAILURE

patchsets:

+1 earlier

PS1 05-08

PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 08-25

threads: 5 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS6 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS6 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebased onto the current series base (65769). Notes on this patchset:

- The level-mapping helpers in lustreapi.h were reconciled with the current
  base's version of the file: the base had switched the numeric bound 0xf to
  the named LL_LZ4FAST_MAX_LEVEL constant; this patch keeps the named
  constant while applying the new +1 level-shift semantics.
- The crypto algorithm-name helper this patch previously extended no longer
  exists at this point in the series (the earlier patches carry a
  display-only compr_name_from_type() in lov instead). The level-dependent
  algorithm selection ("lz4" vs "lz4fast" by level) now lands together with
  the compression engine in 65770, which carries the final form of that
  helper in obdclass.
- In llapi_parse_compress_type(), the "no level given" branch now guards the
  '*level = -1' store with a NULL check: the function accepts a NULL level
  out-pointer and the unguarded store would dereference NULL.
- test_100d: merged this patch's "lz4 twice (std + HC)" coverage with the
  base's newer type list. Final list is "lz4 gzip lz4 lz4fast" plus
  no-level types "lzo zstd", and the compress-file creation passes -S4M so
  the doubling chunk sizes (64K..2M) stay below the stripe size (the chunk
  size is clamped to the stripe size, which would break the per-chunk find
  counts). File names carry _${lvl}_${cs} suffixes so the duplicate lz4
  entries do not collide.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(minor) The comment above still says the acceleration factor is mapped "keeping 1-9 as-is", but from/to now apply a +1/-1 offset, so the encode and decode boundaries no longer line up at 9. from_lz4fast_level(9) returns 10 (the `level < 10` path, level+1), but to_lz4fast_level(10) takes the `>= 10` path and returns 11. So `lfs setstripe -Z lz4fast:9` reads back as level 11 via `lfs getstripe --compr-level` (8 round-trips, 9 does not). Was the boundary meant to be `level <= 10` on decode (or `level < 9` on encode)?

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(style) These crypto headers are #include'd in the middle of the file, interleaved between the new static inline definitions, rather than grouped with the other includes near the top. If the patch is refreshed, consider moving them up. (There's also a stray double blank line at the end of the block.)

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre lustre_crypto.h:200 — (typo) "Cyrpto" -> "Crypto" here and in the ll_crypto_acomp_set_level() comment below.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(typo) "Cyrpto" -> "Crypto" here and in the ll_crypto_acomp_set_level() comment below.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) ll_crypto_comp_set_level() and ll_crypto_acomp_set_level() (and COMPR_LEVEL_MASK / the *_MOD_NAME macros) don't appear to be called or referenced anywhere in the tree. If nothing uses them yet, this looks like it belongs in the kernel patch that consumes them rather than in a utils level-mapping change.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) This branch is entered when `level` is NULL (`!level`), but then dereferences it. The pre-patch code returned early for exactly the `!level` case, so a caller of this exported liblustreapi function that passes NULL for `level` (only interested in the type) now crashes. Both in-tree callers happen to pass non-NULL, but the guard still contradicts the body. Something like:

    if (!level)
        return 0;
    if (!ptr) {
        *level = -1;
        return 0;
    }

tags: csdc

owner Marc Vef · uploader Marc Vef · PS6 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: update handling of compr level for lz4

The way lz4 compression level or acceleration factor is handled needs
to be adapted in order to match what is provided by the lz4 userspace
tool:
- any level between 0 and 2 is interpreted as the default lz4
  acceleration factor of 1;
- any level from 3 and up to 16 is interpreted as a compression level
  for internal lz4hc. Increasing the compression level trades CPU
  time for improved compression ratio;
- acceleration factor can be specified for the lz4fast compression
  type, from 1 to 26. This acceleration factor trades compression
  ratio for faster speed.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4711217c1a6601f29f78d262567da5998f657fc9

66282 master J !B ✗T –R 0/2

LU-10026 llite: skip hole/data lseek() on compressed file

build failure (#130317)

csdc M +48 −5

67d

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #130317 FAILURE

no vote yet: Mikhail Pershin

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: skip hole/data lseek() on compressed file

Doesn't execute real lseek() on compressed file with
SEEK_HOLE/SEEK_DATA origin but consider file always
non-sparsed and do generic lseek() only

test: get size before fallback to generic

If ll_file_seek() falls back to generic_file_llseek_size()
then make sure inode size is reliable

DDN-bug-id: EX-7786
Was-Change-Id: Ieb8c90e66fb19675ba41f0147c0d9cdaf29ea20a

Test-Parameters: forjanitoronly
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2cd587cc0205e85758e06bbaafafe0e2959e0ade

68351 master J ✗2B ✗T –R 0/2

LU-10026 llite: don't queue async IO page for inode in freeing

build failure (#130329)

2 unique failures csdc 1 unresolved S +19 −0

67d

build failure (#130329)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 5 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130329 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- Registered as .cio_commit_async with the current signature (the
  enum cl_io_priority argument was added upstream), next to master's
  renamed .cio_read_ahead_prep hook.
- Reviewer note: master meanwhile carries LU-17364 ("discard un-flushable
  pages on evicted inode teardown"), which addresses the same teardown
  crash from another angle; this patch is kept as an additional guard and
  may be droppable after evaluation.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: don't queue async IO page for inode in freeing

When inode is being freed, do not try to queue async IO pages as
there is a race window between ll_delete_inode() and
ll_truncate_inode_pages_final(), and the queued async IO pages
cannot be teardown.

(osc_page.c:182:osc_page_delete()) Trying to teardown failed: -16
(osc_page.c:184:osc_page_delete()) ASSERTION( 0 ) failed:
(osc_page.c:184:osc_page_delete()) LBUG

    lbug_with_loc+0x64/0xb0 [libcfs]
    osc_page_delete+0x290/0x574 [osc]
    cl_page_delete0+0xb8/0x404 [obdclass]
    cl_page_delete+0x64/0x22c [obdclass]
    ll_invalidate_folio+0x114/0x2a0 [lustre]
    truncate_cleanup_folio+0x6c/0xf0
    truncate_inode_pages_range+0xf8/0x4e0
    truncate_inode_pages_final+0x54/0x80
    ll_truncate_inode_pages_final+0x40/0x520 [lustre]
    ll_delete_inode+0x68/0x25c [lustre]
    evict+0xb8/0x1a0

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11006
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: If655a89faf6f979167c05c91c7b52a6c49a7884b

68352 master J ✗2B ✗T –R +1/2

LU-10026 ofd: space is always granted to server

build failure (#130332)

2 unique failures csdc XS +1 −1

67d

build failure (#130332)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 8 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130332 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 09:47

patchsets: PS1 08-25 PS2 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: space is always granted to server

when it's converting a compressed chunk to uncompressed one for
partial truncate.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10391
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic636525e76c544b55dede13133f423cc0aed534f

68354 master J ✗2B ✗T –R 0/2

LU-10026 sec: set page sync flags to avoid ENOMEM

build failure (#130334)

2 unique failures csdc 1 unresolved S +24 −6

67d

build failure (#130334)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 6 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130334 FAILURE

no vote yet: Artem Blagodarenko

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port note: fill_cpga() debug output adapted to the folio-based brw_page
(bp_folio).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 sec: set page sync flags to avoid ENOMEM

Set OBD_BRW_SOFT_SYNC page flag when appropriate page_pool
is at least 75% full. This would cause a "soft sync" update
at OST side. When pool is at least 81% full clear OBD_BRW_ASYNC
flag to trigger a sync commit. This needs to not stop
compression at client side when page pool becomes empty,
i.e. no free elements.

There is sanity-compr_2001 to check that all chunks from written
1GB data have been successfully compressed, but it will be
landed later,
in a separate patch, after the statistics patches.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10481
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I715d2fb95a459b47c003dad3339f8b6f775245a8

68353 master J ✗2B ✗T –R 0/2

LU-10026 scripts: Compression estimate script

build failure (#130331)

2 unique failures csdc 1 unresolved L +472 −1

67d

build failure (#130331)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 7 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130331 FAILURE

no vote yet: Raphael Druon, Jian Yu

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- always_except tag renumbered to LU-10026.
- ll_compression_scan added to EXTRA_DIST so it survives make dist (the
  tarball would otherwise fail to build with the script listed in
  bin_SCRIPTS but absent).
- Reviewer note: the script is installed under the server-only block
  although it can run on any client; left as the original patch had it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 scripts: Compression estimate script

ll_compression_scan is a simple tool which can be run on any
Linux system to estimate the space usage reduction from the
Lustre Client Side Data Compression (CSDC) feature with
particular compression settings (algorithm, chunk size,
and compression level).

When run on one or more directories, it will recursively
examine a percentage of files under that directory, sampling
data in those files to estimate how the files will compress.

This tools samples data throughout the file, so it should
avoid problems with poor estimates for files with headers
which differ from the bulk data in the file.

However, if the directory tree is particularly imbalanced,
with a few large uncompressible files in one directory, and
many small files in other directories, then scanning a small
percentage of files may give a misleading compression estimate.
Sampling a larger percentage of files will improve this.

This tool requires the lz4, lzop, and gzip utilities to
be installed in order to test those compression types.
(lzop is the command line utility for lzo compression)

scripts: add whole file to compression scan

Add a mode where the compression scan script compresses the
entire file, which in theory should 100% match the
compression results from using CSDC and allow a test to
calculate the exact space usage reduction expected by
using CSDC.

This is intended to be used mostly for testing.

Change help documentation slightly to make clear this can
also accept a path to a single file.

DDN-bug-id: EX-7795
Was-Change-Id: I606a33d686d87dd631bf5b33dc85ee8c24fe9f67

scripts: Improve estimated ratio

ll_compression_scan does not take in account the size of the
sampled files, this might lead to uncorrect estimated ratio for non
homogeneous file.

This patch apply the compression ratio estimated with the sampled data
and applies it to the entire file size, assuming the file will have
the same compression ratio across it.

DDN-bug-id: EX-8362
Was-Change-Id: Ic4a26460e17c666b9edf4c0d8d450a06fad5920f

scripts: improve ll_compression_scan functionality

Improve ll_compression_scan script functionality without
changing the compression estimates.
- add a version string to the output to allow tracking
- handle pathnames with spaces in them
- handle the lz4fast compression type
- allow running on MacOS for testing

DDN-bug-id: EX-8362
Was-Change-Id: I0b8442a2590fdb9c718b1404cba1d73c26cff03c

scripts: improve ll_compression_scan estimate

Improve ll_compression_scan script to give a better estimate of
actual compression ratios.
- add a '-d' debug option for verbose output during testing
- log and report incompressible small files < 4096
- log and report incompressible file count and size
- include small/incompressible/large files in compression estimate
- add a correction factor to calculations for safety margin

DDN-bug-id: EX-8362
Was-Change-Id: If561b0273e38e4821de228c81291859c7bb1a0d2

tests: exclude sanity-compr/1008 on Ubuntu

This subtest is failing consistently on Ubuntu.  Disable until it
can be fixed.

DDN-bug-id: EX-9125
Was-Change-Id: Ie71462e7f033be914523ca96b22478a53b81b882

utils: always print ll_compression_scan summary

Print the summary block for ll_compression_scan even if it is
killed with CTRL-C before finishing the scan.  Otherwise, only
the running total compression ratio estimate will be printed.

DDN-bug-id: EX-9125
Was-Change-Id: I64e135fa6729767d279a52df767252103a2540e5

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7681
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Raphael Druon <rdruon@ddn.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I092f9608553eba10bacfcc3c4a3fafc9a454c287

68399 master J –B ✗T –R -1/2−

LU-10026 tests: sanity-compr/1000d failure for ARM client

build failure (#130371)

csdc S +9 −7

67d

build #130371 FAILURE

no vote yet: Zhenyu Xu

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: sanity-compr/1000d failure for ARM client

aarch64 has 64K page size and compression chunk size could be adjusted
upward if it's original setting value is equal to page size, so
the test should be refined to get the actual chunk size be set.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ide2f91e255ba886f9c1e18d1e1fe89c589f3bca1

68348 master J ✗2B ✗T –R 0/2

LU-10026 csdc: do not mix compressed and plain in niobuf

build failure (#130328)

2 unique failures csdc 1 unresolved XS +1 −1

67d

build failure (#130328)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 3 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130328 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port note: applied as a hand edit - the mask line context had been
reflowed by the earlier can_merge_pages() changes. Content unchanged
(removes OBD_BRW_COMPRESSED from the merge-safe mask).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: do not mix compressed and plain in niobuf

It is important to prevent placing
compresed and uncompressed chunks to the
same rnb, because rnb has only one set flags
for all chunks contained.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: If3fbe160a9f32af2d51f4ba694f444563feb82f6

68401 master J !B ✗T –R 0/2

LU-10026 tests: Add compression statistics output

build failure (#130373)

csdc 1 unresolved S +25 −0

67d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #130373 FAILURE

no vote yet: Sergey Cheremencev, Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

The two OST-version guards around the obdfilter stats access were dropped - compression_enabled() already implies the server supports compression on this tree. The interop paragraph of the message was reworded accordingly.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: Add compression statistics output

sanity-comr executes sanity, sanityn large set of its tests

Currently, it is not even easy to understand that data was
compressed/decompressed. Adding compression statists gives a sign
that compression is enabled and works, showing that parts
(client/server, any features etc.) were executed during the test.

tests: skip CSDC stats errors in interop

This also avoids trying to clear or fetch compression statistics
from servers that do not support compression, always returning
success in that case.
Let's avoid trying to clear or fetch these stats from servers that
do not have them and always return success when clear/read stats.

DDN-bug-id: EX-10124
Was-Change-Id: Ide3e69dfd0268041591c3445aff39532ff00f019

tests: osc compr stats for all OSTs

Show osc compression statistic for all OSTs in
the end of the test session.

DDN-bug-id: EX-11007
Was-Change-Id: Id6ae864438931d9ce50300911490760bf24dd88a

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10124
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9856bad32b0f421ce8cb8ff45a171e229f7c8f02

68370 master J ✗1B ✗T –R -1/2−

LU-10026 csdc: enable zstd/zstdfast as compr type in layout

build failure (#130345)

1 unique failure csdc 1 unresolved M +107 −31

67d

build failure (#130345)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 1 other review

build #130345 FAILURE

no vote yet: Sebastien Buisson

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- Rewritten onto the current naming: only LL_COMPR_TYPE_ZSTDFAST=8 is new (zstd=7 already existed); the existing zstd table row gained its level converters (renamed to from_/to_zstd_level, from_/to_zstdfast_level); zstdfast row added before the MAX sentinel; wire asserts extended (ZSTDFAST==8, MAX 8->9 in both wiretest copies + wirecheck).
- The duplicate COMPR_LEVEL_MAX definition was removed while raising it to 100 (previously the header defined it twice; the values would have diverged).
- Man page: zstdfast added to the type list and level-range sentence (including correcting the lz4fast range to 1-26, matching the current mapping); zstd/zstdfast bullets added after lzo.
- sanity-pfl test_100b/test_100d switched to deriving the compression type lists from the negotiated compress_types in the OSC import, as in the original; adapted to the parametric test structure used by this series.
- The old-version gate around the zstd line in sanity.sh compress_content() is meaningless on master and was dropped (line unconditional).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: enable zstd/zstdfast as compr type in layout

The Zstandard compression algorithm shows good promise for both higher
compression than gzip, while also being faster.
Available compression levels for zstd are 1 through 22. In addition,
faster levels supported by zstdfast are 1-10, 20-100 (in increments of
10).

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9678
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Idc320a9039d2d974683339b8033b23736fe2a20f

68402 master J !B ✗T –R 0/2

LU-10026 csdc: enable compression on aarch64

build failure (#130374)

csdc 1 unresolved S +13 −19

67d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #130374 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- Removes the PAGE_SIZE-conditional default (llite_enable_compression now unconditionally 1) and the userspace PAGE_SIZE preview-warning blocks (which were dead in userspace, since PAGE_SIZE is not a preprocessor macro there).
- aarch64 always_except tags renumbered to LU-10026; one comment reworded to drop an internal change reference.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: enable compression on aarch64

This patch enables CSDC by default on aarch64 client
with >4k PAGE_SIZE.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6172e569c0ae5eb0b823f703c640847479909bdd

66274 master J !B ✗T –R 0/2

LU-10026 osc: don't do decomp. with compression disabled

build failure (#130267)

csdc 3 unresolved XS +3 −1

67d

build failure (#130267)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130267 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-05-28 20:13 Marc Vef COMMIT_MSG:15 — this is not helpful and needs to be updated

Marc Vef PS1 · 2026-05-28 20:13

this is not helpful and needs to be updated

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) The body says decompress_request() has "a significant performance cost for checking if data is compressed", but it already returns immediately when the first page's cp_compr_type is LL_COMPR_TYPE_NONE. For uncompressed files that check is cheap; the scan cost only applies to compressed data, which is the case that still needs decompressing. Could the justification be tightened, and the read/write asymmetry this creates (writes still compress off the layout while reads stop decompressing) be noted?

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre osc_request.c:2537 — (defect) This gates decompression on the global llite_enable_compression instead of the data's own cp_compr_type.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) This gates decompression on the global llite_enable_compression instead of the data's own cp_compr_type.

decompress_request() already returns early when pga[0]'s cp_compr_type is LL_COMPR_TYPE_NONE:

    type = clpage->cp_compr_type;
    /* no compression */
    if (type == LL_COMPR_TYPE_NONE)
        RETURN(0);

so uncompressed reads were never paying for a full scan.

The concern is a genuinely compressed read reaching here with the tunable off. lsm_lmm_verify_v1v3() only rejects a fresh compressed-layout unpack when compression is disabled; a cached lsm survives a runtime toggle. On read, set_page_compression()->__set_page_compression() still fills cp_compr_type from that cached lsme, and the write path still compresses off cp_compr_type regardless of the global.

After echo 0 > ...enable_compression on a client that already holds the compressed file's layout, does a cache-cold read here hand back the still-compressed bytes undecompressed?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: don't do decomp. with compression disabled

decompress_request likely has a significant performance
cost for checking if data is compressed, so we should not
call it when compression is disabled.

This is a stop-gap solution for the preview until we can
have the server tell the client if data is compressed, as
described in EX-7818.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iba8feba3ab0fe620d0594f59c2c6ddea25faeb4f

66265 master J !B ✗T –R 0/2

LU-10026 llite: allow mmap reads of compressed files

build failure (#130262)

csdc 4 unresolved M +107 −3

67d

build failure (#130262)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130262 FAILURE

no vote yet: Artem Blagodarenko

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(minor) `off` is computed at the call site as vma->vm_pgoff<<PAGE_SHIFT and passed in, but never used in the body. Drop the parameter, or use it.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(suggestion) The block is keyed on the file open mode, so an fd opened O_RDWR that only creates a PROT_READ mapping is rejected with -EOPNOTSUPP even though it never writes the file (a common open-O_RDWR-then-mmap-read pattern). Since read mmap of compressed files is supported, would keying on the mapping's write intent (vma->vm_flags VM_WRITE/VM_SHARED) rather than the open mode let those read-only mappings through while still avoiding the write SIGBUS?

reply PS3 2026-07-15 00:56 Gerrit AI review for Lustre llite_mmap.c:613 — (minor) The message names cl_io_rw_init(), but this path calls cl_io_init().

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(minor) The message names cl_io_rw_init(), but this path calls cl_io_init().

reply PS3 2026-08-26 11:52 Alex Zhuravlev multiop.c:585 — what do we do with this?

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(defect) These treat O_RDONLY/O_WRONLY/O_RDWR as bitmask flags, but the access mode is a 2-bit field. O_RDONLY is 0, so `fd_flags & O_RDONLY` is always 0, and for an O_RDONLY fd `fd_flags & O_RDWR` is also 0, so mmap_mode stays 0 (PROT_NONE) instead of PROT_READ.

Mask and compare instead:

    switch (fd_flags & O_ACCMODE) {
    case O_RDONLY: mmap_mode = PROT_READ; break;
    case O_WRONLY: mmap_mode = PROT_WRITE; break;
    case O_RDWR:   mmap_mode = PROT_READ | PROT_WRITE; break;
    }

Because of this, the `oSMc` case in test_460e maps PROT_NONE, so "readonly mmap should work" passes without ever exercising a readable mapping.

Alex Zhuravlev PS3 · 2026-08-26 11:52

what do we do with this?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: allow mmap reads of compressed files

mmap reads of compressed files work, so we should only
block writes.

We cannot block the actual fault operations because that
will cause the application to get a SIGBUS, so we check
the file open mode when we go to create the memory mapping.

llite: disable mmap on compressed files

disable mmap(2) on compressed files until well tested.

DDN-bug-id: EX-6265
Was-Change-Id: I3464a03b16708edcd0692bc9db337eb8473ea047

llite: use CIT_MISC in check_compression

ll_mmap_check_compression uses a CIT_FAULT io type and
initializes it with cl_io_rw_init.  This doesn't
initialize the IO correctly and this sometimes results
in the following crash during io_fini, because the ft_page
pointer is initialized to something else by cl_io_rw_init:

BUG: unable to handle kernel NULL pointer dereference
at 000000000000004a
RIP: 0010:cl_pagevec_put+0x9f/0x3a0 [obdclass]
 vvp_io_fault_fini+0x21/0x40 [lustre]
 cl_io_fini+0x7a/0x230 [obdclass]
 ll_mmap_check_compression+0x403/0x540 [lustre]

DDN-bug-id: EX-7812
Was-Change-Id: Ib1f270f071370d6c045bb9d799ab5b7b41a6c6be

Test-Parameters: forjanitoronly
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I636f398fd247ddcd153f94bf8116440540e8469c

66268 master J !B ✗T –R 0/2

LU-10026 utils: error for invalid compression level

build failure (#130265)

csdc 3 unresolved M +78 −45

67d

build failure (#130265)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130265 FAILURE

no vote yet: Sebastien Buisson

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

The message (and the sanity-pfl/sanity test updates) cover only lzo, but the new error path also fires for zstd, since it likewise has no level mapping. Worth naming zstd here and adding coverage for it, or clarifying why zstd is excluded from the documented 1-21 level range.

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase notes:

- The base meanwhile added zstd to the compression type table (no level
  converter). Since this patch makes an explicit level on a no-converter
  type an error, zstd was added to test_100d's type_nolvl list next to lzo,
  and the compress-file creation now passes -S4M so the six doubling chunk
  sizes (64K..2M) stay below the stripe size (the chunk size is clamped to
  the stripe size, which would break the per-chunk find counts).
- test_100d's chunk-verification loop now iterates all types with a
  doubling chunk size (the previous patchset's loop only ran one
  iteration).

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) This whitelists only fast/best to carry a level; every other type with a NULL ctn_from_compr_level falls through to the error. That is lzo (intended) but also zstd. So `lfs setstripe -Z zstd:5` now fails, whereas pre-patch it stored the level. lfs-setstripe.1:232 (added in this same LU-10026 series) says zstd takes a level 1-21. Is the zstd rejection intended? If zstd should not take a level, the man page needs updating too; if it should, zstd wants a from_compr_level mapping rather than the error path.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: error for invalid compression level

Not all compression types accept a compression level (e.g. lzo in its
kernel implementation). For those, return an error if a compression
level is provided on 'lfs setstripe' command-line.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6c367c4bfd76cc81c890a89ba9f994f4fd9f4f80

68349 master J ✗2B ✗T –R +0/2−

LU-10026 osc: limit compression chunks in RPC

build failure (#130327)

2 unique failures csdc 2 unresolved L +283 −22

67d

build failure (#130327)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

1 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 10 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 10 other reviews

build #130327 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:31

no vote yet: Artem Blagodarenko, Zhenyu Xu

patchsets: PS1 08-25 PS2 08-26

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- try_to_add_extent_for_io()/get_write_extents() gained their env parameter
  on top of master's reworked extent-RPC code: the erd_max_compr_chunks
  member was added to the write_rpc_def()/read_rpc_def() initializer macros
  (which master introduced since), with the macros now taking the osc
  object.
- Page indices in the chunk-counting loops use osc_index() - struct
  brw_page no longer has a bp_page member on master (folio-based).
- The RPC-fragment limiting in the submit path is applied in
  osc_io_submit() only; master has since split direct-IO submission into
  osc_dio_submit(), which keeps its existing behaviour (DIO on compressed
  files is restricted to chunk-aligned IO anyway).
- cp_compr_chunk_log_bits -> cp_compr_chunk_lum_bits (series field naming).
- Fixed a short memcpy in the chunk-ledger growth helper
  (expand_osc_compr_chunk_array copied the element count as bytes,
  losing 3/4 of the saved entries on expansion).
- The new sanity-compr test_1016 gained a compression_supported skip guard
  (it had none).

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) osc_io.c:138 — error: osc_io_submit():'osc' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

error: osc_io_submit():'osc' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: limit compression chunks in RPC

Limit the number of partial compressed chunks that can be sent to
the server in a single RPC, so that it doesn't need to allocate
buffers for read-modify-write on a huge number of chunks at once.
It is better to handle that in separate server threads.

In the worst case, each extent in the BRW RPC belongs to a different
compression chunk, so the worst_case_chunks would expand on the OSS
to worst_case_chunks * compression_chunk_size bytes IO on the server.
That might be hundreds of MB in size per thread and LASSERT or OOM.

Add tunable osc.*.compress_rpc_fragment_ratio to reduce the maximum
compressed chunks by the given factor, its value ranges in [1, 32].

osc: scan extents to find compression chunks

as we can't trust oe_start and oe_end, we have to scan all the
pages covered by extent and so find all compression chunks to
estimate and limit potential read-modify-write IO on OST.

osc_submit_io() should not create too big extents we can't send
due to limits for compressed chunks.

DDN-bug-id: EX-10141
Was-Change-Id: I00b104ded6cdb8e66414eb2f759f6dca1b7b4d11

osc: make oti_compr_chunks ledger expandable

This patch makes osc_thread_info::oti_compr_chunks an expandable
array.

DDN-bug-id: EX-10361
Was-Change-Id: Ib2ad5407815c733687eadd4a46014d0a694dbf41

osc: expand_osc_compr_chunks_array() use atomic alloc

This can be called in "spinlock" context and generate a stack dump:

  BUG: sleeping function called from invalid context
  in_atomic(): 1, irqs_disabled(): 0, name: ptlrpcd_00_02
  Call Trace:
    ___might_sleep.cold.23+0x104/0x118
    kmem_cache_alloc_trace+0x1cd/0x2a0
    expand_osc_compr_chunk_array+0xc6/0x5d0 [osc]
    try_to_add_extent_for_io.isra.10+0x11f6/0x16b0 [osc]
    osc_io_unplug0+0x6a4/0x1b50 [osc]
    brw_queue_work+0x29/0xb0 [osc]
    work_interpreter+0x2b/0x100 [ptlrpc]
    ptlrpc_check_set+0x583/0x2fe0 [ptlrpc]
    ptlrpcd_check+0x511/0x570 [ptlrpc]
    ptlrpcd+0x1b5/0x4b0 [ptlrpc]

DDN-bug-id: EX-10361
Was-Change-Id: I72f30b5185be0417c1bb7170ac278f48b04e7d0b

osc: revert to correct cl_max_extent_pages

otherwise grant overhead calculation breaks.

DDN-bug-id: EX-10267
Was-Change-Id: I4e74524b17c7834c432a708bb917f291fd773023

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10047
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4d0b4bdaaa91f46147d2ca1c52741117533f8f7a

68355 master J ✗2B ✗T –R +1/2

LU-10026 ofd: make oti_chunk_bufs expandable

build failure (#130333)

2 unique failures csdc M +60 −70

67d

build failure (#130333)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 4 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130333 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 09:47

patchsets: PS1 08-25 PS2 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: make oti_chunk_bufs expandable

to be able to handle very large IO with lots of chunks.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10394
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8ad24acd61037cf2804612940cdb74662c0c5303

68373 master J –B ✗T –R +0/2−

LU-10026 csdc: stop compressing incompressible file data

build failure (#130347)

csdc 2 unresolved L +413 −17

67d

build failure (#130347)

1 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

build #130347 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 12:21

no vote yet: Zhenyu Xu

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

ci_incompressible placed in the relocated cl_io bitfield next to ci_size_extend_compression; test_1021 gate converted to compression_supported.

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) osc_io.c:566 — error: osc_io_iter_init():'osc' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

error: osc_io_iter_init():'osc' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: stop compressing incompressible file data

The reduced_ratio (original_size/compress_reduced_size) represents
the minimum fraction of pages that are compressed out of each chunk,
namely the compressed chunk needs to shrink by at least
1/reduced_ratio blocks for it to be "compressible".

Let size compression_ratio be defined as
original_size/after_compression_size, so

reduced_ratio = compression_ratio / (compression_ratio - 1)

and we set its default value to 16, equivalent to 1.07 of compression
ratio (i.e. needs to shrink at least one 4KB block out of each 64KB
chunk).

After every compress_check_bytes of data being compressed, file's
compressibility would be re-calculated based on average
compress_reduced and average compress_orig data size.

Stop compressing file data if it is deemed to be incompressible, and
after compress_skip_bytes data have been written uncompressed , retry
the file compressibility check.

compress_reduced_ratio, compress_check_bytes, compress_skip_bytes
are tunable parameters:
	osc.*.compress_reduced_ratio
	osc.*.compress_check_bytes
	osc.*.compress_skip_bytes
their default values are 16, 1M and 32M respectively.

tests: use /dev/zero as compressible source

in test 1021 as hdf file has few incompressible chunks and this
breaks the test infrequently.

DDN-bug-id: EX-10857
Was-Change-Id: Iba29d9e654a245d3c0cbb14a8b7dbb9f847f52d5

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4ce3d752c67f18ba7b100c72a2bb61a91258c6e8

68409 master J –B ✗T –R -1/2−

LU-10026 csdc: tunable to disable compression on fallocate

build failure (#130381)

csdc 1 unresolved L +238 −12

67d

build #130381 FAILURE

no vote yet: Sebastien Buisson

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- The connect-time obd_set_info_async(KEY_FALLOCATE_COMPR) block was merged into the current client_common_fill_super() flow (placed before cl_sb_init; the error path uses this function's current label and puts the pending request like its sibling paths).
- test_1031's version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: tunable to disable compression on fallocate

fallocate is denied for compressed files because preallocating space
for a compressed file makes the compression useless. But sometimes it
could be preferable to deactivate compression and honor fallocate
instead.
So add a tunable for disabling compression on fallocate, 0 by default:
llite.*.prefer_fallocate_over_compr
Note disabling compression is possible only if none of the components
have been already initialized.

Add sanity-compr test_1031 to exercise this.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10053
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3f272f39166287cd459ebcfbb9bd61ed0fefd30f

68405 master J !B ✗T –R 0/2

LU-10026 cdsc: update layout compressibility

build failure (#130377)

csdc 4 unresolved L +335 −54

67d

build failure (#130377)

3 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130377 FAILURE

no vote yet: Zhenyu Xu, Alex Zhuravlev

threads: 3 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- OBD_CONNECT2_UPDATE_LAYOUT was already reserved on master at the same bit; only the MDT_CONNECT_SUPPORTED2 and client connect lists needed the addition.
- ci_need_change_layout added to the current cl_io bitfield; CAT_COMPRESSIBLE handling merged into the current vvp_attr_update().
- mdd_layout_update_pccro was renamed/extended per the patch on the current (retry-aware) master form.
- FORMATTED_BUF_LEN (used by the new getstripe formatting) is defined locally - the source branch carries it from an unrelated change.
- test_1021 layout-flag assertions applied unconditionally (the original wrapped them in version gates).

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:297 — error: vvp_layout_minor_change():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_layout_minor_change():'vob' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:429 — error: vvp_io_fini():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_io_fini():'vob' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:1973 — error: vvp_io_init():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_io_init():'vob' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 cdsc: update layout compressibility

When compressing write changes the file's compressibility, it updates
the file layout flags so that its compressibility persists.

lod: reset la_attr at declare

to avoid unplanned local attributes update during minor layout change.

DDN-bug-id: EX-10786
Was-Change-Id: I093efeb9cf110adfe6a7590091e60b1e9f1eb9ed

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia2964817d1b5fee2d84ef15e9f1036882d70adb2

66284 master J –B ✗T –R -1/2−

LU-10026 csdc: handle data compression on MDT

build failure (#130318)

csdc 4 unresolved S +39 −1

67d

build failure (#130318)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

build #130318 FAILURE

no vote yet: Zhenyu Xu

patchsets:

+2 earlier

PS1 05-28 PS2 06-11

PS3 06-11 PS4 06-11 PS5 06-11 PS6 08-25 PS7 08-26

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(minor) The body explains the DoM+compress block, but two hunks pull the other way and aren't mentioned: lov_pattern_available() gains LOV_PATTERN_MDT | LOV_PATTERN_COMPRESS, and llapi_pattern_to_lov() gains MDT|COMPRESS and RAID0|COMPRESS cases. Both make the combination more acceptable, not less. Could the message cover why they belong in a patch whose stated purpose is to disallow it?

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(defect) This makes the server accept an MDT|COMPRESS component (lod_lov.c / lov_ea.c call lov_pattern_available()), yet lov_pattern_supported() still excludes it, so client IO marks such an entry invalid (lov_object.c sets lle_valid = 0). The only guard against creating it lives in lfs.c; a liblustreapi caller doing llapi_layout_pattern_set(MDT) + llapi_layout_compress_set() bypasses that and produces a layout the server stores but IO cannot use. If DoM+compress is unsupported, should the block live in the library/server rather than only the CLI?

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(defect) errno is set to a negative value here; every other error path in comp_args_to_layout() uses errno = EINVAL. A negative errno breaks strerror(errno)/perror on this -1 return. Should be errno = EINVAL.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(minor) These two cases look unreachable. The COMPRESS bit never reaches llc_pattern: llapi_layout_pattern_set() rejects any pattern carrying it, and llapi_layout_from_lum() stores compression in llc_compr_type, not the pattern. The bit is OR'd into lmm_pattern separately in llapi_layout_to_lum() (blob->lmm_pattern |= LOV_PATTERN_COMPRESS). Was a real caller intended, or can these be dropped?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS7 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: handle data compression on MDT

Do not support setting data compression component on DoM until
data compression on MDT implemented.

DDN-bug-id: EX-7806
Was-Change-Id: I3794460140f08a073377c418dd56e7dda907d96d

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8b5c4d5fd18033e796b1c7555c37251e4f7f9a49

68400 master J –B ✗T –R -1/2−

LU-10026 tests: skip 1000d if not enough OSTs

build failure (#130372)

csdc 1 unresolved XS +1 −0

67d

build #130372 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

OSTCOUNT gate added after the converted compression_supported gate.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip 1000d if not enough OSTs

sanity-compr/1000d needs 2 OSTs at least

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10785
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I93d7a2c0178549980348ab41c2fcb62669571f6a

68398 master J !B ✗T –R 0/2

LU-10026 tests: one more compressed truncate

build failure (#130370)

csdc 1 unresolved S +14 −0

67d

janitor run: 21 failing config(s), none unique to this patch — janitor results

build #130370 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

The version guard inside the new truncate case (guarding a fix contained in this series) was dropped; its bug-reference comment renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: one more compressed truncate

check one more scenario of compressed file's truncation.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10391
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I186b92f36dd93afe943ea8f87a6e334f73fd9d26

68396 master J !B ✗T –R 0/2

LU-10026 osc: mem leak in compress_request

build failure (#130368)

csdc 1 unresolved M +70 −13

67d

janitor run: 25 failing config(s), none unique to this patch — janitor results

build #130368 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

Applied cleanly; test_2002's client-version gate converted to compression_supported (dropped again later in the series once the suite gained file-scope gates). The commit message's review pointer was generalized.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: mem leak in compress_request

1. Fix memory leak in compress_request.
2. In case of ENOMEM compress_request should return 0
if it has been able successfully compress at least
one chunk.

osc: fix OBD_FAIL_OSC_COMPR_ENOMEM

Set cfs_fail_loc to OBD_FAIL_OSC_COMPR_ENOMEM to
always fail with ENOMEM after skipping fail_val times.

Several cosmetic changes in sanity-compr test_2002 to address
review comments on an earlier version.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11120
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I745e17978d885d40218efe13e68e7e41b4b33d31

68395 master J !B ✗T –R 0/2

LU-10026 tests: check that all chunks are compressed

build failure (#130367)

csdc 1 unresolved S +35 −0

67d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130367 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The osc half of the original change ("osc: improve can_merge_pages()") is already on master as 366fe06822, so this ports only the test remainder under a new Change-Id, with the original kept as Was-Change-Id.
- Adds test_2001; its client-version gate (a fix contained in this series) converted to compression_supported. test_1030 is left in place: the series' final state keeps both tests.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: check that all chunks are compressed

Add sanity-compr test_2001, which streams a large sequential
write to a compressed file and verifies from the client-side
compression stats that (nearly) every chunk was written
compressed, i.e. that the OSC forms full compressible chunks.

This exercises the can_merge_pages() behaviour that landed on
master in commit 366fe06822 ("LU-10026 osc: improve
can_merge_pages()"): pages with different OBD_BRW_ASYNC /
OBD_BRW_SOFT_SYNC hints must still merge into full chunks.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11063
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I6dd2643c93739fd6abaeb7add27f530a57fdcf64
Change-Id: Ice73943d4a2c297c1a95e0754b75b1f157df6c6f

68393 master J ✓B ✗T –R 0/2

LU-10026 tests: disable 1081 in sanity-compr

build failure (#130365)

csdc 1 unresolved XS +2 −0

67d

build #130365 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: disable 1081 in sanity-compr

causing too many failures.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10077
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2b389c2ae2c733c3e6b8b100cdd0ca84df303246

68388 master J –B ✗T –R -1/2−

LU-10026 ofd: don't increment csdc counters

build failure (#130360)

csdc M +64 −21

67d

build #130360 FAILURE

no vote yet: Alex Zhuravlev

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: don't increment csdc counters

if corresponding event doesn't happen on the server.

add few more tests for compression counters.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10647
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id9badab250a3337d6a968680ce37e41e74b9c269

68387 master J !B ✗T –R 0/2

LU-10026 ofd: persistent compression counters

build failure (#130359)

csdc 10 unresolved L +333 −1

67d

build failure (#130359)

9 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130359 FAILURE

no vote yet: Alex Zhuravlev

threads: 9 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- Context-only adaptation: struct ofd_device gained members on master (new fields appended at the end), and the include list and module-init surroundings in ofd_dev.c differ.
- test_1123 gained a compression_supported gate (the original had none; without it the test would run and fail on servers without compression).
- lprocfs_load_stats/lprocfs_save_stats land in lprocfs_status_server.c as in the original; both EXPORT_SYMBOL'd (used from ofd.ko).

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1618 — warn: lprocfs_save_stats():passing zero to 'ERR_CAST'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():passing zero to 'ERR_CAST'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1629 — error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1630 — warn: lprocfs_save_stats():possible memory leak of 'buf'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():possible memory leak of 'buf'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1641 — error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1672 — warn: lprocfs_save_stats():possible memory leak of 'buf'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():possible memory leak of 'buf'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1716 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1723 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1737 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1750 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: persistent compression counters

ofd to save compression stats every hour and at umount,
then to load the saved stats at mount time so compression
stats can be found persistent and repsresenting overall
compression usage.
the file is called "compr_stats", stored in the root dir
of underlaying fs and uses YAML format:
- { name: uncompressed_objects, count:0, min:92, max:0, sum:0 }
- { name: compressed_objects, count:1, min:92233, max:0, sum:0 }
- { name: compr_chunks_read_disk, count:0, min:9223, max:0, sum:0 }
- { name: compr_bytes_read_disk, count:0, min:92233, max:0, sum:0 }
- { name: compr_bytes_read_user, count:0, min:92233, max:0, sum:0 }
- { name: compr_chunks_read_decompress, count:0, min:9, max:0, sum:0 }
- { name: compr_bytes_read_decompresse, count:0, min:9, max:0, sum:0 }
- { name: compr_chunks_write_disk, count:4, min:1, max:8, sum:11 }
- { name: compr_bytes_write_disk, count:4, min:8192, max:1310, sum:20}
- { name: compr_bytes_write_user, count:4, min:8192, max:10486, sum:1}

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10130
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I59016c473607fe7e4aa2583c28dfc223bc3d9b7c

68386 master J !B ✗T –R 0/2

LU-10026 ofd: support for compression counters

build failure (#130358)

csdc 1 unresolved L +257 −38

67d

janitor run: 21 failing config(s), none unique to this patch — janitor results

build #130358 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The two ofd_compress.c hunks around is_chunk_start() re-applied against the folio form of the call sites.
- test_1122 anchored after test_1121; its version gate converted to compression_supported.
- Commit message reworded to drop downstream release-version references in the two test-skip sub-sections.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: support for compression counters

the patch adds additional compression related counters:
compr_bytes_read_disk -  bytes were read from disk
compr_chunks_read_disk -  chunks were read from disk
compr_bytes_read_user -  bytes were sent to clients
compr_bytes_read_decompressed -  bytes were sent after decompression
compr_chunks_read_decompressed -  chunks were decompressed
compr_bytes_write_disk -  bytes were sent to disk
compr_chunks_write_disk -  chunks were sent to disk
compr_bytes_write_user -  bytes clients send to server
compr_chunks_rmw -  chunks were involved in read-modify-write
compr_bytes_rmw -  disk bytes were involved in read-modify-write
compr_bytes_rmw_user -  bytes clients sent causing read-modify-write
decompression_time - time in usec spent in decompression

tests: skip sanity-compr/1122 with old OST

skip sanity-compr test_1122 when the OST does not provide the
server-side compression counters.

DDN-bug-id: EX-10586
Was-Change-Id: I2f5608e93880ad6569d68e83110144bfcf791f80

tests: adjust the sanity-compr/1122 version check

adjust the version check for the server-side counters.

DDN-bug-id: EX-10890
Was-Change-Id: I4f58cac84a7c7dcbec667292528ae4667396f1f0

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10128
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie974cbd15451ae790310ec8e3d5c729a849c882a

68376 master J ✗1B ✗T –R -1/2−

LU-10026 tests: skip sanity-pfl tests if no compression

build failure (#130350)

1 unique failure csdc 1 unresolved S +17 −29

67d

build failure (#130350)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #130350 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Re-derived by hand - every hunk's context had version gates this series already replaced with compression_supported. Each test_100* gained 'compression_enabled || skip' and lost its per-test llite.*.enable_compression save/set/restore block (100a keeps a bare enable, 100e keeps its disable + re-enable trap), relying on the default-on behaviour from the finer-grained enable_compression control patch.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip sanity-pfl tests if no compression

Skip sanity-pfl test_100* for servers that do not understand CSDC.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9482
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ib3fcfd77e9e7ffb122ed6ade9015b02d42ea8319

66276 master J !B ✗T –R 0/2

LU-10026 lov: fallocate is not allowed for compressed files

build failure (#130268)

csdc 4 unresolved S +30 −0

68d

build failure (#130268)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130268 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

The commit message only describes disabling fallocate, but the diff also adds a `rsync --sparse` decompression round-trip (the `decomp2` / "fiomap case" block) to compress_type() in sanity.sh. That change is about the sparse-copy/FIEMAP read path, not fallocate, and isn't mentioned here. Is it meant to be part of this patch, or should it be split into its own change with its own Change-Id?

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(suggestion) This only checks lsm_entries[0]. lsme_compr_type is set per component (lov_ea.c copies lcme_compr_type for each LOV_PATTERN_COMPRESS entry), so a composite/PFL file could have an uncompressed component 0 and a compressed later component. Would fallocate over such a file still be allowed even though it targets a compressed component? (The ll_fiemap() guard has the same entry[0]-only limitation.)

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) `lio` and `loo` here just re-derive values already in scope: the outer `lio` (declared at the top of the function) is the same `cl2lov_io(env, ios)`, and `loo` is `lio->lis_object`. The inner `struct lov_io *lio` also shadows the outer one. This could just use the existing `lio->lis_object` without the extra declarations.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) fallocate(2) is directly user-triggerable, so a CWARN here lets an unprivileged caller flood the console by repeatedly calling fallocate on a compressed file. The equivalent compressed-file guard in ll_fiemap() (lov_object.c around line 2021) returns -EOPNOTSUPP silently. Consider CDEBUG(D_VFSTRACE, ...) or a rate-limited message instead.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: fallocate is not allowed for compressed files

Client Side Data Compression allocates blocks after a compression.
It is impossible to preallocate blocks for the whole file, so
fallocate should be disabled in case of compression.

Test-Parameters: forjanitoronly
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie834ace183fdcec0d7d6f747237e0964c3c4120b

66264 master J –B ✗T –R 0/2

LU-10026 man: document supported compression types

build failure (#130260)

csdc 3 unresolved S +16 −0

68d

build failure (#130260)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

build #130260 FAILURE

no vote yet: Sebastien Buisson

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: the lfs-setstripe.1 sentence this patch anchors on was
reformatted on the base (.B "lfs getstripe" roff style); the
compression-type list is inserted after the reformatted sentence, content
unchanged.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(suggestion) This new list of currently supported types omits zstd (and fast/best), but the untouched line just above still advertises "1-21 for zstd" as a valid level range. If zstd is not currently supported, consider reconciling the two so the level-range hint doesn't contradict this list.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(defect) Is the lz4 default really 1? When no level is given, llapi_parse_compress_type() passes level = -1, and from_lz4_level(-1) returns LZ4HC_DEFAULT_CLEVEL - 1 = 8, which lfs getstripe maps back via to_lz4_level() to 9. So the effective default looks like 9 (highest ratio / slowest), not 1. The LZ4_ACCELERATION_DEFAULT = 1 case only applies when an explicit level < 3 is given, not for the unspecified default. Compare gzip just above, where the documented default of 6 does match from_gzip_level(-1).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 man: document supported compression types

Update lfs-setstripe to mention the currently supported compression
types: gzip, lz4, lz4fast and lzo.

Test-Parameters: forjanitoronly
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9b587516d27b882bec5855da4948f489e5a0041f

65762 master J ✗1B ✓T ✗1R 0/2

LU-10026 csdc: prefer uncompressed mirror for read and write

janitor: 1 test failure unique to this patch — sanity3@zfs:test_907

1 unique failure csdc 4 unresolved L +213 −47

72d

janitor: 1 test failure unique to this patch — sanity3@zfs:test_907

enforced failed: review-dne-selinux-ssk-part-2

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity3@zfs:test_907	seen in 57 other reviews

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #129912 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Andreas Dilger, Artem Blagodarenko, Zhenyu Xu

patchsets:

+7 earlier

PS1 05-08 PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 07-27 Artem Blagodarenko PS7 07-28 Artem Blagodarenko

PS8 08-19 Artem Blagodarenko PS9 08-19 Artem Blagodarenko PS10 08-20 Artem Blagodarenko PS11 08-20 Artem Blagodarenko PS12 08-21 Artem Blagodarenko

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

This bonus is added once per allocated stripe, which is exactly the weight the OS_STATFS_NONROT bonus a few lines above uses, so the two can cancel out.

Two mirrors, no LCME_FL_PREF_WR anywhere, one stripe each:

    mirror 0: compressed,   stripe on a NONROT OST -> pref = 1
    mirror 1: uncompressed, stripe on a rotating OST -> pref = 1

lme_preference ties, so the `!found_preferred` fallback below keeps the lower index (`>` is strict) and sets lme_prefer on mirror 0. lod_primary_pick() then classifies mirror 0 as case B (preferred + compressed) and mirror 1 as case C, and B outranks C, so the write goes to the compressed mirror.

A wider compressed mirror wins outright rather than tying: 4 NONROT stripes compressed = 4 vs 1 NONROT stripe uncompressed = 2.

Mixed flash/disk mirrors are the case the non-rotational preference was added for, so this is not an exotic layout. Should the uncompressed bonus be on a different scale than the per-stripe NONROT count (or applied once per mirror) so it can't be traded away against OST speed?

The comment also says "mirror", but this sits in the per-stripe loop of one component.

reply PS12 2026-08-22 12:56 Gerrit AI review for Lustre lod_object.c:8060 — (style) Not a bug, but this comment has a UTF-8 em dash in it; the rest of the tree is plain ASCII. A plain '-' would do.

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

(style) Not a bug, but this comment has a UTF-8 em dash in it; the rest of the tree is plain ASCII. A plain '-' would do.

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

Ranking B (preferred + compressed) above C (uncompressed) reads as "an explicit `lfs setstripe --flags=prefer` beats the compression heuristic". But lme_prefer is not only user intent: when no component carries LCME_FL_PREF_WR, lod_fill_mirrors() picks the highest lme_preference non-stale mirror and sets lme_prefer = 1 on it.

So tier B is reachable on a file the user never flagged, and the heuristic ends up overriding its own compression preference. Should case B be restricted to mirrors that actually carry LCME_FL_PREF_WR, or should the synthesized preference be tracked separately from the user flag?

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

(minor) ci_ndelay_tried > 0 means "not the first lov_io_mirror_init() for this env", which is broader than "this is an FLR error retry".

ll_file_io_generic() restarts the io for reasons unrelated to mirror failure and preserves the counter:

    partial_io -> io->ci_need_restart = 1   (file.c)
    retried = io->ci_ndelay_tried; goto restart;

A single buffered read larger than max_io_bytes therefore takes the uncompressed mirror for the first chunk only; from the second chunk on ci_ndelay_tried is non-zero, lis_mirror_index has been advanced by one, and the first in-range mirror is accepted even when it is compressed. The layout-change restart in vvp_io_iter_init() has the same shape.

Would gating on something that only the -EAGAIN mirror-failover path sets (io->ci_tried_all_mirrors, or a dedicated flag set where cl_io_loop() retries) keep the failover fix without giving up the preference on ordinary io continuation?

tags: csdc

owner Marc Vef · uploader Artem Blagodarenko · PS12 uploaded 73d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: prefer uncompressed mirror for read and write

When accessing a mirrored file with both compressed and uncompressed
components, choose uncompressed components for read (lov_io.c).

When writing to mirrored files with both compressed and uncompressed
mirrors, prefer the uncompressed components to write; this is better
for performance, more compatible with older clients, and better fits
the model of compressing files after initial write. The write-side
preference is implemented via lod_fill_mirrors() and lod_primary_pick()
using a new lme_compressed flag.

Note: the changes to lod_fill_mirrors() and lod_primary_pick() affect
all FLR files, not only those with compressed mirrors. Plain FLR writes
will also use the updated preferred-mirror selection logic.

DDN-bug-id: EX-6510
Was-Change-Id: I62a117d5cc3d34e2c0c96d1a9ade8eef0a2d1291

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@gmail.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I043b27bd891c039901075a08c76630f8f0f9f182

65769 master J !B ✓T ✗1R 0/2

LU-10026 csdc: add ll_compr_hdr structure

enforced failed: review-dne-selinux-ssk-part-2

csdc hidden 4 unresolved L +270 −58

72d

enforced failed: review-dne-selinux-ssk-part-2

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #129913 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Andreas Dilger, Artem Blagodarenko

patchsets:

+9 earlier

PS1 05-08 PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 07-28 Artem Blagodarenko PS7 07-28 Artem Blagodarenko PS8 07-31 Andreas Dilger PS9 07-31 Andreas Dilger

PS10 08-19 Artem Blagodarenko PS11 08-19 Artem Blagodarenko PS12 08-20 Artem Blagodarenko PS13 08-20 Artem Blagodarenko PS14 08-21 Artem Blagodarenko

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS14 2026-08-22 12:39 Gerrit AI review for Lustre COMMIT_MSG:21 — (minor) Two hunks aren't accounted for by the body.

Gerrit AI review for Lustre PS14 · 2026-08-22 12:39

(minor) Two hunks aren't accounted for by the body.

COMPR_GET_CHUNK_SIZE() in lustre_user.h gains argument parentheses and a 1U base. That is a real hardening of the macro, but nothing in the message mentions it.

llapi_layout_compress_set() gains a type range check that makes the function return -1/EINVAL for inputs it previously accepted. That is a behaviour change to a public liblustreapi entry point and deserves a line.

Are both meant to ride along in this patch?

reply PS14 2026-08-22 12:39 Gerrit AI review for Lustre lustre_idl.h:4134 — (minor) This isn't a bug today, but the two arguments are treated asymmetrically: `lvl` is masked with 0x0f, `bits` isn't. The matching acce

Gerrit AI review for Lustre PS14 · 2026-08-22 12:39

(minor) This isn't a bug today, but the two arguments are treated asymmetrically: `lvl` is masked with 0x0f, `bits` isn't. The matching accessor LLCH_CHUNK_LUM_BITS() does mask on the way out.

    LLCH_LEVEL_AND_CHUNK(3, 16) -> (16 << 4) | 3 == 259

The result is an int, so the overflow only disappears when it's stored into the __u8 llch_level_and_chunk, and the header then decodes as chunk_lum_bits 0 (64 KiB) rather than anything detectable. Valid chunk_lum_bits is 0..10 so no current caller can hit it, but masking `bits` the same way as `lvl` would keep the macro self-contained.

reply PS14 2026-08-22 12:39 Gerrit AI review for Lustre liblustreapi_layout.c:1721 — (minor) Documentation/man3/llapi_layout_compress_set.3 doesn't seem to be updated to match. Its ERRORS section lists EINVAL only for the str

Gerrit AI review for Lustre PS14 · 2026-08-22 12:39

(minor) Documentation/man3/llapi_layout_compress_set.3 doesn't seem to be updated to match. Its ERRORS section lists EINVAL only for the stripe_size/chunk_size cases, and DESCRIPTION enumerates the valid `type` values without saying that an out-of-range type or the new LL_COMPR_TYPE_UNCHANGED sentinel is now rejected. Should the man page change land in the same patch?

reply PS14 2026-08-22 12:39 Gerrit AI review for Lustre liblustreapi_layout.c:1736 — (style) Not a bug, but the first test can never decide the branch on its own: LL_COMPR_TYPE_UNCHANGED is 255 and LL_COMPR_TYPE_MAX is 8, so

Gerrit AI review for Lustre PS14 · 2026-08-22 12:39

(style) Not a bug, but the first test can never decide the branch on its own: LL_COMPR_TYPE_UNCHANGED is 255 and LL_COMPR_TYPE_MAX is 8, so `type >= LL_COMPR_TYPE_MAX` already covers the sentinel. Keeping it reads as documentation, but a reader has to work that out.

tags: csdc

owner Marc Vef · uploader Artem Blagodarenko · PS14 uploaded 73d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: add ll_compr_hdr structure

It is an important structure for CSDC feature. Will be used
in the next patches.

Also adds:
- LL_COMPR_TYPE_UNCHANGED enum value (used to signal that the
  compression type should not be changed on update).
- LL_LZ4FAST_MAX_LEVEL define (15: the largest value the 4-bit
  stored level field can hold for the LZ4FAST algorithm).
- LLCH_FL_COMPR_CSUM / LLCH_FL_HDR_CSUM flags in llch_flags to
  signal checksum presence explicitly (a crc32 of 0 is ambiguous
  otherwise).
- wirecheck: pin COMPR_CHUNK_MIN_BITS, COMPR_CHUNK_MAX_BITS, and
  LL_LZ4FAST_MAX_LEVEL values.

Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I26f2fc7c0212f1a4cad36805c75aa766d6aed548

65761 master J !B ✓T ✗1R 0/2

LU-10026 utils: add 'lfs find' support for compressed file

enforced failed: review-dne-selinux-ssk-part-2

csdc 9 unresolved L +715 −67

72d

enforced failed: review-dne-selinux-ssk-part-2

9 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 4 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #129911 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Andreas Dilger, Artem Blagodarenko, Zhenyu Xu

patchsets:

+12 earlier

PS13 08-19 Artem Blagodarenko PS14 08-19 Artem Blagodarenko PS15 08-20 Artem Blagodarenko PS16 08-20 Artem Blagodarenko PS17 08-21 Artem Blagodarenko

threads: 9 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(minor) The first three bullets read as new functionality, but `--comp-flags=[^]compress` and `--comp-flags=[^]nocompr` already work (LCME_FL_COMPRESS/LCME_FL_NOCOMPR are in comp_flags_table[]), and so does `-L compress` (LOV_PATTERN_COMPRESS is in lov_pattern_names[]). This patch documents them and adds test coverage rather than adding them. Could the wording say "document" for those three, so the message matches the diff?

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) COMPR_LEVEL_MAX is already defined with the same value earlier in this header, just after `enum ll_compr_type`, with a comment explaining it. Two definitions can drift apart later; can this one be dropped?

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre sanity-pfl.sh:3078 — (style) extra blank line. Also z1..z4 here, and `expect` in test_100d, are not declared `local`.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) extra blank line. Also z1..z4 here, and `expect` in test_100d, are not declared `local`.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) not a bug, but `compress-chunk` already appears in this long_opts[] further down, so it is listed twice now. getopt_long takes the first match so it still works; `compr-chunk` looks like the only new alias actually needed here.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(typo) this comment uses a UTF-8 em dash; the tree is otherwise plain ASCII. Same in the `%LZ` directory-branch comment in liblustreapi_pfind.c.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) atoi() cannot report a parse failure, so `--compr-level=abc` silently becomes level 0, and `--compr-level=+abc` becomes "level > 0", i.e. every compressed component. The unsigned range check above cannot catch it.

`-Z gzip:abc` is rejected by llapi_parse_compress_type(), and the setstripe `--compress-level` path uses strtol() with an end-pointer check, so the three spellings of the same thing disagree. strtol() plus `*end != '\0'` here would line them up.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(minor) This helper backs every numeric `lfs find` predicate, and the new exact-match branch only changes behaviour when `margin == 0`. `lfs find -z N` with a plain number is such a caller: lfs.c does `fp_ext_size_units /= SEL_UNIT_SIZE`, so 1/1024 becomes 0, and an exact `-z` match that previously reported "no match" now matches. That may well be desirable, but it is a silent change to an unrelated option.

The new compression predicates are the only callers passing margin 0; the rest pass 1 or a unit size, and with margin 1 this function already returns 1 on an exact match. Passing 1 there would avoid touching the shared helper, and would also make `--compr-level=-N` / `--compr-chunk=-N` mean "< N" like `-c -N`, `--size -N` and friends, rather than "<= N".

The table above the function (rows 5 and 6, and the note about the interval `(limit - margin, limit]`) no longer describes the code either.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) Negation is handled differently for level/chunk than for type. `fp_exclude_compr_lvl` is passed into find_value_cmp() as `negopt`, so `found_lvl` ends up meaning "some component does not match" instead of "no component matches", and the aggregation here then ignores the exclude flag entirely - unlike the fp_check_compr_type block just above.

For a file with two compressed components, level 3 and level 5:

    lfs find ! --compr-level=3 dir

entry(lvl 5) makes find_value_cmp() return -1, negopt flips it to 1, found_lvl is set, and the file is reported - even though it does have a level-3 component. The equivalent `! --compr-type=gzip` on a gzip+lz4 file correctly does not match.

Should the level and chunk comparisons drop the negopt argument and be inverted here the same way type is?

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) `fp_lum_size` is the size common_param_init() allocated for the lmd buffer (>= XATTR_SIZE_MAX, 64KiB), not the length of the layout stored in it.

For a non-composite lum, llapi_layout_get_by_xattr() never narrows that value, so llapi_layout_objects_in_lum() computes (65536 - 32) / 24 = 2729 objects and __llapi_comp_alloc() rejects it with EINVAL because 2729 > LOV_MAX_STRIPE_COUNT (2000).

So this call fails for every plain V1/V3 file and always falls through to llapi_layout_get_by_path(); only composite layouts reach the new path, because the entry loop resets the size to `ent->lcme_size`. Passing the actual layout length instead would make the fast path work for non-PFL files too.

tags: csdc

owner Marc Vef · uploader Artem Blagodarenko · PS17 uploaded 73d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: add 'lfs find' support for compressed file

* Add "--comp-flags=[^]compress" to locate file with/without
  compressed components.
* Add "--comp-flags=[^]nocompr" to locate file with/without setting
  component compress preference.
* Add "[!] --layout=compress" to locate file with/without compressed
  components.
* Add "[!] --compress-type=<compress-type>" to locate compressed file
  with/without specified compress algorithm.
* Add "[!] --compress-level=[+-]<compress-level>" to locate compressed
  file with/without specified compress level.
* Add "[!] --compress-chunk=[+-]<compress-chunk>" to locate compressed
  file with/without specified compress chunk in KiB.

utils: add -Z option for 'lfs getstripe/find'

Add support for "lfs getstripe -Z" to get the last instantiated
component compression information.

Add support for "lfs find -Z <type>[:[+-]<level>]" to keep consistent
options with "lfs setstripe -Z".
DDN-bug-id: EX-6856
Was-Change-Id: Ia5e2a2ea6937dc2b46e224cec808504a196b974c

utils: logical AND for 'lfs find' compression exprs

All search expressions provided to 'lfs find' must be combined as a
logical AND. Fix newly added options for compression support, so that
they comply with this logical AND.
DDN-bug-id: EX-7714
Was-Change-Id: I3b28cd87c1d304df6d04753b413d46f5abcfe16e

csdc: don't set compression layout when disabled

When llite_enable_compression is disabled
(lfs set_param llite.*.enable_compression=0), we should check
it before sending it to MDS lest we get a file with compressed
component which we cannot handle.
DDN-bug-id: EX-7593
Was-Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f

utils: support 'lfs find --printf %LZ'

Add support for "lfs find --printf %LZ" to print the compression
type:level of the last instantiated component of a file.
DDN-bug-id: EX-6856
Was-Change-Id: Iaf1b6c031b06c70e7b5be51354697aa6bdcc9850

csdc: right error code in case compression is disabled

Currently, if compression is disabled on a client the error
message is confusing "Cannot set layout EA: Unknown error 524"

Let's replace the error code from  ENOTSUPP to EOPNOTSUPP,
so error messages become more informative

ctl get_param -n llite.*.enable_compression
0
lfs setstripe -i 0 -c 1 -E -1 -Z lz4 /mnt/lustre/foo.txt
Cannot set layout EA: Operation not supported
lfs setstripe: cannot create composite file
'/mnt/lustre/foo.txt': Inappropriate ioctl for device

This message could be even more informative, but this version
is already much better.
DDN-bug-id: EX-10577
Was-Change-Id: I7fd67d5d3d20bc7de169f35f6fa63b53f2bfe718

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic36946738c3463fd862aeca4ee2e2c2ed85eff84

▶

Review feedback — your turn

(1)

60957 master J ✓B ✓T ✓R -1/2−

LU-18756 tests: sanity-sec 75a sync resource IDs

Andreas Dilger voted -1

mtfl_p2 stalled 255d 1 unresolved S +6 −4

255d

Andreas Dilger voted -1

1 thread(s) waiting on others

no vote yet from Sebastien Buisson

reviews: Andreas Dilger -1 2025-08-29 17:28

no vote yet: Sebastien Buisson

patchsets: PS1 08-28 PS2 08-28 PS3 08-29

threads: 0 need your reply · 1 waiting on others · 0 own notes · 0 bot

owner Marc Vef · uploader Marc Vef · PS3 uploaded 430d ago · NEW · open in Gerrit ↗

commit message

LU-18756 tests: sanity-sec 75a sync resource IDs

After sanity-sec 75a setups the test namespace, it creates and writes
to files before changing the user to the correct tenant user which
carries out the commands. Later, when the ID check is enabled, it
could happen that the corresponding OST object still has the initial
user ID set and that the "chown" changes have not yet been propagated
from the MDT to the OSTs. In such a case, the test fails the ID check
because the OST object has the wrong IDs set.

This patch adds an additional step after namespace creation to
synchronize ID changes to and between the servers before continuing
with the test.

Fixes: 719f6763fe ("LU-18756 sec: add resource id check to oss and mds")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=75
Test-Parameters: testlist=sanity-sec env=ONLY=75a,ONLY_REPEAT=100
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I07d602375fd34334dc5047cfecea60f09f03bd42

▶

In CI

(0)

nothing here right now

▶

Needs reviewers

(1)

70236 master J –B ✓T ✓R 0/2

LU-0000 tests: private test patch, do not land

CI green, no reviews yet — add reviewers

XS +1 −0

build #134893 SUCCESS · tests all Maloo sessions

patchsets: PS1 10-30 PS2 10-30

owner Marc Vef · uploader Marc Vef · PS2 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-0000 tests: private test patch, do not land

Trivial comment change for a private Gerrit test change.

Test-Parameters: trivial
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I78dd5f283fdd3399a22e222bb60d2ab284cecb9e

▶

Waiting on reviewers

(1)

70000 master J –B ✓T ✓R +1/2

LU-20723 tests: regroup subtests skipped in 1-client sessions

no vote yet from Andreas Dilger

L +251 −264

build #134772 SUCCESS · tests all Maloo sessions

reviews: Sebastien Buisson +1 2026-10-30 09:15

no vote yet: Andreas Dilger

patchsets: PS1 10-23 PS2 10-23 PS3 10-26 PS4 10-27 PS5 10-29

owner Marc Vef · uploader Marc Vef · PS5 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-20723 tests: regroup subtests skipped in 1-client sessions

Most review test sessions are moving to a single client VM, with
the tests that need a second client grouped into a few two-client
sessions.  Five subtests in the single-client sessions skip without
a second client.

Move the three that need a second node to suites that keep two
clients:
- replay-single test_70a to replay-dual test_200
- sanity test_160v to replay-dual test_201
- replay-single test_200 to recovery-small test_200, and update the
  "DO NOT MODIFY" note in ptlrpc_expire_one_request()

Run the other two on one node:
- sanityn test_33b runs createmany on $CLIENT1 only
- replay-vbr test_10b uses $MOUNT3 for its third client

Test-Parameters: trivial
Test-Parameters: clientcount=1 mdscount=2 mdtcount=4 testlist=sanityn env=ONLY=33b
Test-Parameters: clientcount=1 mdscount=2 mdtcount=4 testlist=replay-vbr env=ONLY=10b
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic1db87ab68ff45e81af569a57592e0cdb298cb71

▶

Ready to land

(1)

68837 master J –B ✓T ✓R +2/2

LU-930 doc: fix outdated and dead URLs

all green — queued for landing (master-next)

master-next 2 unresolved S +17 −21

all green — queued for landing (master-next)

2 reviewer comment(s) to address if a respin is needed — last from Andreas Dilger

build #134645 SUCCESS · tests all Maloo sessions

reviews: Arshad Hussain +1 2026-10-29 03:44 · Andreas Dilger +1 2026-10-28 21:30

no vote yet: Timothy Day

patchsets:

+1 earlier

PS1 09-11

PS2 09-11 PS3 10-27 PS4 10-28 PS5 10-28 PS6 10-28

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS6 2026-10-28 21:30 Andreas Dilger lbuild:1200 — This patch already landed long ago.

Andreas Dilger PS6 · 2026-10-28 21:30

This patch already landed long ago.

reply PS6 2026-10-28 21:30 Andreas Dilger lbuild:1202 — This PR is closed. Maybe this can now be simplified?

Andreas Dilger PS6 · 2026-10-28 21:30

This PR is closed.  Maybe this can now be simplified?

owner Marc Vef · uploader Marc Vef · PS6 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-930 doc: fix outdated and dead URLs

Switch the remaining http:// lustre.org links in README to https://.

Point links whose hosts have moved or gone away at their current
location: OpenZFS and DKMS on GitHub, the CRUSH paper on ceph.io,
and Oracle's copy of the Sun GSS-API guide.

Drop the links to the FID interoperability design page, which no
longer exists; the relevant parts are in the lustre_fid.h comment.

Test-Parameters: trivial
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I87fdf6c67c631675dbe00acce8de873fb6eae33e

▶

Parked (WIP / self-blocked)

(9)

68347 master J –B ✗T –R ✗

LU-10026 ofd: csdc final integration patch

parked by your own -1

CR -1 csdc 5 unresolved XL +3772 −421

build #130325 FAILURE

reviews: Marc Vef -1 2026-08-27 20:12

no vote yet: Sergey Cheremencev, Sebastien Buisson, Zhenyu Xu, Alex Zhuravlev, Hongchao Zhang, Li Dongyang

patchsets: PS1 08-25 PS2 08-26

threads: 3 need your reply · 0 waiting on others · 2 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes for this upstream submission (large squash; adapted to current
master):

- The build-file hunk was retargeted: the tree no longer has per-directory
  autotools files (LU-18876), so ofd_compress.o is added to the kbuild
  lustre/ofd/Makefile; the EXTRA_DIST hunk has no equivalent and was
  dropped.
- LU-13139 moved the dt_bufs_get()/dt_read_prep()/dt_write_prep() bodies
  out of dt_object.h into obdclass/dt_object.c; this patch's signature
  changes (offset/len args, decompression/read flags) were applied to the
  relocated bodies, with the prototypes and the new dt_unlock_pages()
  wrapper in the header.
- OBD_FAIL_OST_BRW_WRITE_BULK3 renumbered 0x256 -> 0x258: 0x256 is already
  OBD_FAIL_OST_GRANT_PREPARE on master.
- An m4 call to LC_FORCE_UACCESS_BEGIN was dropped: the macro is not
  defined anywhere (neither in this patch nor in the tree). The functional
  m4 content (LC_VM_MAP_RAM_PROT_ARG probe for the 4-arg vm_map_ram()) is
  kept.
- The compression level-mapping helpers and compr_type_table were moved
  from lustreapi.h into the lustre_user.h UAPI header (as this patch
  intends, so server code can share them), based on the current series'
  versions of those helpers; the table gained the LL_COMPR_TYPE_MAX
  sentinel and iteration skips NULL-name entries. compress_type_2str()/
  mapback_compress_level() are static inlines there now; the lfs find
  loops live in liblustreapi_pfind.c and were converted there.
- Adapted to the reworked struct ll_compr_hdr (65769): split magic words
  and the packed level_and_chunk byte with its accessors, in ofd_compress.c
  and the shared compression library. is_chunk_start() takes a folio in
  this series; the OFD call sites pass page_folio(lnb_page).
- The chunk-bits fields follow the series' *_lum_bits naming and offset
  semantics; llc_compr_chunk_lum_bits already stores the offset above
  COMPR_CHUNK_MIN_BITS, so the filter-fid assignment in lod_object.c is
  direct, without re-subtracting COMPR_CHUNK_MIN_BITS.
- ptlrpc_connection_is_local() does not exist upstream - replaced with the
  LNetIsPeerLocal(&exp->exp_connection->c_peer.nid) form used by the
  surrounding OFD code.
- osd_write_prep(): the partial-page zeroing block had been converted to
  kmap_local_page()/folio_index_page() upstream; the patch's changes
  (PageUptodate skip, read flag, whole-page memset) were applied on the
  current form.
- The preview-period client-side restriction on non-chunk-aligned
  overwrites (vvp_io_write_start) is removed as in the original patch.
- unmerge_chunk(): the new tail-clearing loop indexed lnb[i] while its loop
  condition and debug message used first + i; the stores now use
  lnb[first + i] consistently.
- Old version gates carried by the squashed test additions (sanity-compr
  1004-1006, sanity-lfsck 18i, test_460e area) are meaningless on master
  and were converted to the compression_supported() helper.
- A Fixes: line pointing at one of this patch's own squashed sub-patches
  was dropped; a file for an external repository (lipe) was dropped.

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:663 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'rc + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'rc + maxlnb' 'unit_page + unit_array_size'

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:667 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'rc + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'rc + maxlnb' 'unit_page + unit_array_size'

note PS2 2026-08-27 20:12 Marc Vef ofd_compress.c:975 — (defect) range lock must be taken before `ofd_map_chunks_to_lnb_write()`

Marc Vef PS2 · 2026-08-27 20:12

(defect) range lock must be taken before `ofd_map_chunks_to_lnb_write()`

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:1048 — warn: ofd_object_punch_compr():duplicate check 'orig_lnb' (previous on line 957)

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

warn: ofd_object_punch_compr():duplicate check 'orig_lnb' (previous on line 957)

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: csdc final integration patch

Add calls to the csdc functions from the regular read/write paths.
This also brings OFD range locking back for compressed objects,
reverting commit a21ce928aa64 ("LU-15894 ofd: revert range locking
in ofd") for this use case.

ofd: rename 'local' in thread_big_cache

It's not a big deal since it's only used a few times, but
let's give this variable a descriptive name.

DDN-bug-id: EX-7601
Was-Change-Id: Ide136cd42e885d59f1a2e4ce22a2e7449faca3f9

ofd: rename map_remote_to_local

osd_map_remote_to_local implies some complex role, but in
fact what this does is initialize the fields of the
local niobuf structs to represent the requested range.

This *may* be the same as a remote niobuf, but it also
isn't in some cases.  Name it osd_init_lnbs instead.

DDN-bug-id: EX-7601
Was-Change-Id: I0d3b5f24e42ee8dc962437daea7cf9347ccb9059

ofd: series description and reorder declarations

Reorder declarations in tgt_brw_read prior to adding things.

This trivial patch is a good place to put the description
of this series, which handles unaligned reads to compressed
files.

-----------------------
These patches handle compression chunk unaligned reads on
the server.

When using compression, the client attempts to send chunk
aligned reads, but sometimes it can't, and the client will
send a read to the server which is not chunk aligned.

In this case, the server must read the full chunk,
decompress it, and provide the requested data to the client.

Here's how we do this.

The server receives a set of remote niobufs describing IO
from the client.  Each remote niobuf (rnb) describes a range
of data the client wants to do IO to.

These are translated to a set of local niobufs on the
server, which we then use to do the read.  For compression,
the server has to read complete chunks on unalinged reads.

So we walk these remote niobufs and identify unaligned read
requests (in ofd_preprw_read), then round them to chunk
size. The server then reads the chunk rounded read request
from storage.

The local niobufs now contain a set of complete compressed
chunks, ie, the raw data from disk.  We need to decompress
the chunks where the client is doing an unaligned read, but
leave the other chunks compressed (because the client will
uncompress them).

So, in obd_decompress_read, we use the remote niobuf to
identify unaligned reads from the client.  We then walk the
local niobufs, identify the chunks which match the unaligned
reads from the client, and decompress them 'in place'.
The decompression uses temporary buffers, but the
decompressed data is placed back in the local niobuf.
(If the data is uncompressed on disk, we of course do not
decompress it.  This happens for incompressible data.)

Now the local niobuf contains some raw chunks and some
chunks which have been decompressed.  This is *more* data
than the client asked for.  Normally, the server local
niobuf contains exactly what the client asked for, so the
server checksums and sends the entire local niobuf.  But
because we read complete chunks, the local niobuf contains
more data than the client requested.

This means we need to identify the subset of the local
niobuf which the client actually wants to read and present
that to the client.

In order to do that, we walk the local niobuf and use the
remote niobufs (the description of the pages the client
needs) and create a special tx niobuf which points to only
the pages the client wants (io_lnb_to_tx_lnb).  Then we use
this tx niobuf for checksum and transfer to the client.

DDN-bug-id: EX-7601
Was-Change-Id: Ic89dcef7e169879725caa6cdef4619b9a76b2b37

ofd: identify reads to round

If the beginning or end of a client read is unaligned, we
must round the locking.  This patch identifies reads where
this is required, the next patch will do the locking.

Print a debug message when such an IO is found, but don't
do anything different - yet.

DDN-bug-id: EX-7601
Was-Change-Id: Ibdab35b733225b4b1349ef457f66ca37dcb2d9bf

ofd: round read lock to chunk

For unaligned reads, we need to round the read locking to
cover the any leading or trailing chunks.  We do this by
creating a local 'remote niobuf' to describe the rounded
range and doing the locking against that niobuf.

DDN-bug-id: EX-7601
Was-Change-Id: I8818522c188aca3c5c5eb564da2a8ba8aef18a4b

ofd: convert dt_bufs_get to offset, len

dt_bufs_get takes a remote niobuf, but just uses the
offset and length for getting pages.

Compression requires rounding the local IO to include the
full compression chunk, which means the local IO does not
match the remote niobuf any more.

So we modify dt_bufs_get to take an offset and length
rather than a remote niobuf, so we can ask for the pages we
need.

DDN-bug-id: EX-7601
Was-Change-Id: I4beaf8207fa00d802c0a339df3de2a3c71154fc7

ofd: add chunk_size to preprw_read

preprw_read needs chunk size for rounding.  Add this in a
separate patch to keep things trivial, it will be used in
a subsequent patch.

Also use this to add a check in DOM to ensure it doesn't
attempt to do compression.  This should already be
prevented by setstripe, so this is just an extra safety
check.

DDN-bug-id: EX-7601
Was-Change-Id: I9dc4d1559e5c8be315268a593466571b54c90a96

ofd: add remote_pages

When we round a read to get all of the compressed chunks,
the number of local and the number of remote pages will
differ.  We need to make sure we do the checksum and data
transfer using the number of remote pages, not the number of
local pages.

This patch calculates the number of remote pages and uses it
accordingly.  This doesn't do anything yet, but when we
round the local read to include the whole compressed chunk,
this will be needed.

DDN-bug-id: EX-7601
Was-Change-Id: I4875b02016570d227b3b926efd117f0a7cda41b4

tgt: add second tbc lnb for RDMA

Compression requires the server to do local IO which differs
from the IO requested by the client.  This means we cannot
directly use the IO niobufs for doing the transfer to the
client.

So we add a second set of lnb pointers, which are used to
point at a specific subset of the pages in the main
per-thread cache.  This subset will be used for doing the
transfer to the client.

DDN-bug-id: EX-7601
Was-Change-Id: I53aa46045aaf335da20a311900ac0bf425823b22

tgt: add io_lnb_to_tx_lnb

With compression, the lnbs used for the disk IO on the
server can contain more data than the client requested,
due to reading up whole chunks for decompression.

This means we need to transfer only a subset of the lnbs.
We do this by creating a second set of lnbs, and pointing
them at the pages in the local io lnb which need to be
transferred to the client.

This code doesn't do anything for now, but it will kick in
with the next patch when we start rounding chunks for read.

DDN-bug-id: EX-7601
Was-Change-Id: I0fe690718a3484578b139eaaec52c0c3b265da6a

ofd: add chunk rounding to read

We need to round all niobufs to chunk size in the read
process, so we read in the full chunk.

dt_bufs_get sets up the local niobuf for the read, so we
round before calling it.

This patch is a partial implementation of unaligned read
support, and breaks compression testing until the next few
patches are landed.  So this patch temporarily adds the
compression tests to ALWAYS_EXCEPT.

DDN-bug-id: EX-7601
Was-Change-Id: I587a519db4dae983db5db1d690e63e15bc010b7e

ofd: add chunk_size to preprw_write

preprw_write needs chunk size for rounding.  Add this in a
separate patch to keep things trivial, it will be used in
a subsequent patch.

This patch is really trivial on the write side, since the
read side already did most of this.  But it's being kept
separate for symmetry.

DDN-bug-id: EX-7601
Was-Change-Id: Id25957dbc185b6e61b7f208cee8cf5f897f03944

ofd: do not use page cache for compressed files

It is challenging for the server to safely use the page
cache with compressed files, because if data is
decompressed in to the page cache, the data in cache now
differs from the data on disk.

This is a problem if *part* of the page cache is ever
evicted, because we can end up with a situation where a read
will be partially satisfied from cache and partially from
disk, but the data on disk is compressed and the data in
cache is not.

It is possible to deal with this by carefully ensuring the
page cache is not used just for decompressed data, but this
makes getting the buffers/lnbs for compressed files fairly
complicated.  Instead, we can just entirely block using the
server page cache for compressed files.

This must be done for both read and write, and only works
for ldiskfs - ZFS cannot easily be forced to not use its
page cache.  But that's OK because we do not support CSDC
with ZFS.

DDN-bug-id: EX-7601
Was-Change-Id: Iee73abb29ad5631bb2203c2133756d7ebf5b686d

ofd: add ofd_decompress_read implementation

ofd_decompress_read is responsible for walking the
remote niobufs (rnbs) in the RPC and identifying if they
are chunk unaligned.  It then passes them on to the rnb
decompression code (not implemented yet, see next patch).

It also allocates the bounce buffers for decompression so
they can be reused for each remote niobuf.

DDN-bug-id: EX-7601
Was-Change-Id: I1f2f86ce3fc036ac5d79b060a5e44f6564e123aa

ofd: lock pages from read to decompression

When using the page cache on the server, for pages which
will be decompressed, we can't unlock them until they've
been decompressed.

Rather than only waiting to unlock the pages which will be
decompressed, we keep all of the read pages locked.  This
simplifies the code, at the cost of delaying other reads to
the aligned portion of an unaligned read.  ie, shouldn't be
important in practice.

DDN-bug-id: EX-7601
Was-Change-Id: Ie98920327979a5c9600e8c9e8627816461ea1a34

ofd: (ofd part)add obd level compression lib

Some compression functions will be used by several areas of
of Lustre, so they need to be in obdclass.

This moves merge_chunk and unmerge_chunk there and adds the
ability for them to merge lnbs.  This is used in a future
patch.

DDN-bug-id: EX-7601
Was-Change-Id: If4a318119bb7685e41adb9f3b31a66074031e6ac

ofd: make compress_chunk take chunk_bits

Chunk bits is used everywhere, have compress_chunk convert
to log bits rather than have the callers do it.

Was-Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>

ofd: add decompress_rnb implementation

This implements decompress_rnb, which is the core code for
handling unaligned reads from the client.

Decompress rnb takes an unaligned remote niobuf and
identifies the unaligned portion(s) of the IO, then finds
the corresponding local niobufs (pages read from disk),
and passes them on for decompression in place.

decompress_chunk_in_lnb decompresses the data in a set of
lnbs and copies it back to the same location, replacing the
raw data from disk with decompressed data.  (If the chunk
was not compressed, it does nothing.)

With this patch, the implementation of unaligned reads is
complete and we can add the compression sanity tests back
safely.

DDN-bug-id: EX-7601
Was-Change-Id: Ifd1d9b03d5d004bec3f5e456da359b8d10e005f9

ofd: improve decompress_rnb debug

Since we're very close on landing the unaligned read
patches, this minor debug improvement is being placed later
in the series.

DDN-bug-id: EX-7601
Was-Change-Id: I36ad243bd1f7025e358f9593f1008f0b851cc1bb

tgt: reorder tgt_brw_write decls

Reorder the declarations in tgt_brw_write.

This patch also serves as the series head for implementing
read-modify-write support for compressed chunks.

The process for read-modify-write is similar to that used
for unaligned reads.

At a high level, read-modify-write means we must read up,
decompress, then recompress and write back the data.  This
only applies when we're actually doing read-modify-write.

To know when to do this, we rely partly on the client.  If
the client is able to compress a chunk, either because it is
a complete chunk, or because the start is chunk aligned and
the write is past EOF, we know there is no read-modify-write
required.  Either there is no existing data (write past EOF)
or the data will be fully replaced.

So, when we see a write which is not fully chunk aligned and
not already compressed, we will do a read-modify-write.

For this, we round the IO lnbs and associated locking to
cover complete chunks, then we do a read of the unaligned
chunks.

ie, if we have a write which goes from 63 KiB to 257 KiB
with a chunk size of 64 KiB, we will read 0-64 KiB and
256-320 KiB, and decompress those chunks in to the buffer.
64 KiB to 256 KiB is *NOT* read, because those are complete
chunks.

We then set up a transfer mapping - identical to the process
for unaligned reads - so the client data is written in to
the correct lnbs.

Now we have a set of chunk aligned lnbs which contain data
updated with the client write.  In the initial version, we
write these to disk uncompressed.  This is sufficient for
correct operation, but it does mean read-modify-write will
decompress those chunks.

There is code for recompression, but it is not working 100%
yet, and there are some complexities around managing holes
and EOF which still need to be resolved.

TBD if this will make our initial release - I am hopeful but
not sure yet.

DDN-bug-id: EX-7601
Was-Change-Id: Ia24583d4221f498928e99afa8c289b70e4d25f5b

tests: unaligned read tests

This adds testing for handling unaligned reads and partial
chunk reads from compressed files.

Testing for writes and multi-client and racing tests will
be added later, but we put the checking function in
test-framework now so it's easy to use later.

DDN-bug-id: EX-7601
Was-Change-Id: I06217c8aeba75016aa4168f329026842dff1d979

obd: add error check in merge_chunk

If the lnbs we're trying to merge have an error recorded in
them, then they're not going to be valid input for
decompression, so return an error.

DDN-bug-id: EX-7601
Was-Change-Id: I1bf17131cb65106087eb5e72e2700db30c0cc975

obd: do not decompress empty lnbs

For reads which cross EOF, we may get lnbs with no data in
them (similarly for writes which cross EOF).

For these cases, it's important to only copy from the lnbs
where there is data, and only do decompression on the lnbs
if there's actually data in them.

Modify merge chunk to do this.

DDN-bug-id: EX-7601
Was-Change-Id: I83fefcfa6d1396dcd97fad994334bf29438bb4bf

ofd: clear pages in decompression

Handling writes to compressed files requires a
read-modify-write cycle, which has implications for how we
handle reads.

Consider the case of a file with an 8 KiB write at offset 0,
which is compressed to 4 KiB.  Then there is another 4 KiB
write at offset 16 KiB.

Updating this correctly requires reading the first chunk,
then decompressing it.  However, this read will go past
EOF, because the write has not occurred yet.  The OSD read
code does not fill in these pages, because read past EOF is
not returned to the client (client gets a short read and
does not actually use the pages).

In our case, however, we must use these pages (from 8 KiB
16 KiB).  In the naive version without recompression, we
simply write out 0 - 16 KiB, so we must have zeroes in
those pages, and once we have recompression, we must
compress those pages so we need zeroes in that case too.

So we note if a page has data in it after decompression,
then if it does not, we clear the page.  Note we do NOT set
lnb_rc to 0 when we clear a page, because lnb_rc = 0 is
used to indicate EOF rather than a gap in the file.

DDN-bug-id: EX-7601
Was-Change-Id: If1d1360185eb087e821167a08e49c9427e29ffc4

tgt: identify writes to round

If the beginning or end of a client write is unaligned, we
must round the locking.  This patch identifies writes where
this is required, the next patch will do the locking.

DDN-bug-id: EX-7601
Was-Change-Id: Iec140c24423a0da478f6d42ff6fc620d7ad3ba4a

tgt: round write lock to chunk

For unaligned writes, we need to round the write locking to
cover the any leading or trailing chunks.  We do this by
creating a local 'remote niobuf' to describe the rounded
range and doing the locking against that niobuf.

DDN-bug-id: EX-7601
Was-Change-Id: I2bdea620386ad229375647a0e2cc6180c9bd7aa6

tgt: add remote_pages for writes

When we round a write to get all of the compressed chunks,
the number of local and the number of remote pages will
differ.  We need to make sure we do the checksum and data
transfer using the number of remote pages, not the number of
local pages.

This patch calculates the number of remote pages and uses it
accordingly.

Note that just like on the read side, this patch doesn't do
anything until we're actually rounding the chunks for IO in
a later patch.

DDN-bug-id: EX-7601
Was-Change-Id: I38256070d68246613ce67b0bfe328f6443a95533

csdc: fix ofd_preprw_write() for sanity 819b test

Sanity 819b asserts:
tgt_brw_write()) ASSERTION( npages_local == npages_remote )

The test triggers fault inject in ofd_preprw_write():
if (OBD_FAIL_CHECK(OBD_FAIL_OST_2BIG_NIOBUF))
     rnb[i].rnb_len += PAGE_SIZE;

ofd_preprw_write() calculates npages_local taking in account
additional len from fault inject, BUT npages_remote is calulated
BEFORE the fault inject. So npages_remote was not adjusted.

To solve the problem it is enough to move range_to_page_count() call.

DDN-bug-id: EX-9523
Was-Change-Id: Ifd659985a78c7630049a17622aff2eb7f4525fb1

tgt: add tx lnb for writes

With compression, the lnbs used for the disk IO on the
server can contain more data than the client requested,
due to reading up whole chunks for decompression.

This means the client is only going to write data in to
a subset of the lnbs used for io to storage.

We handle this the same way we do for reads:
We create a second set of lnbs just for the transfer, and
point these lnbs at the pages which will actually receive
data from the client.

DDN-bug-id: EX-7601
Was-Change-Id: I5b668547537698309792daf309842866be79f0b6

ofd: switch preprw to chunk_bits

The compression/decompression code requires chunk_bits
rather than chunk size.  Since we need to call this code
from ofd_preprw_write, we need chunk_bits there.

This modifies the functions so chunk_bits is available
there.

DDN-bug-id: EX-7601
Was-Change-Id: Id98e6d6364eeaaa7753a8aba059387e3e659d2a2

vvp: remove unaligned write restriction

This series will resolve the unaligned write issue for
compressed files, so we need to remove the restriction on
unaligned writes in order to test it.

This does not mean unaligned writes are working yet, but we
need to make this change so the subsequent patches can be
tested.

DDN-bug-id: EX-7601
Was-Change-Id: I4abcbdcd18b00718099483c8dfdb9a7aa41c3ce7

ofd: add chunk rounding to write

For compressed files, we need to round all niobufs to
chunk size in the write process, so we have buffers for
reading in and rewriting the complete chunks.

dt_bufs_get sets up the local niobuf for the write, so we
round before calling it.

Note this breaks writing to compressed files, which is not
fixed until a few patches later.  For this reason, we
disable the compression tests.  They will be reenabled
shortly - similar to how we handled the read series.

DDN-bug-id: EX-7601
Was-Change-Id: I413aaba9866dd7d6c4463fa620eadf1423379ba1

ofd: add read to write process

This adds a very simple read to the write process, which
just reads up the entire chunk-rounded write range.

This is a first step - the read will eventually be modified
to only read the unaligned portions which must be
decompressed for read-modify-write.  We will create a
special lnb mapping which contains only the pages which must
be read for decompression (similar to the tx lnb mapping).

For now, this read allows us to test decompression without
handling the mapping.

DDN-bug-id: EX-7601
Was-Change-Id: I169ddc2e161094aebdad1a60ec62e9c1d75cd6d8

ofd: add decompress_read to ofd_preprw_write

We have read up the compressed data from disk, now we must
decompress it so we can rewrite it successfully.

This code still works on the whole lnbs rather than just on
the portion of it which is unaligned.  This is temporary
and will be resolved by a future patch.

With this patch, we have basic read-modify-write support,
so we can re-enable testing.  The next patch adds tests
for read-modify-write.

DDN-bug-id: EX-7601
Was-Change-Id: Ib6503c15e9fb3d425a7bc295bcc61b41c089a1f0

ofd: add read lnb to ofd_preprw_write

The read phase of read-modify-write for compressed files
needs to read only a subset of the pages which will be
written, so it needs a separate set of lnb pointers for
tracking this subset.

This patch passes around the necessary argument but does
not set up or use the lnb yet.

DDN-bug-id: EX-7601
Was-Change-Id: I7ec7101e65e73a6c9e67cea3c58d8cace38e70e0

ofd: distinguish nr_write and nr_read

We will have two counts of pages in lnbs, distinguish
between them.

Not actually used yet - will be calculated when the read
lnb mapping is created.

DDN-bug-id: EX-7601
Was-Change-Id: I709b8fd299163d348a196184152bb0294fcb650b

ofd: create read mapping for read-modify-write

When we need to do a read-modify-write for unaligned writes
to a compressed file, it's important we read only the
portion of the file which is receiving unaligned IO.

This patch identifies these chunks in preprw_write and
creates a read lnb mapping from a subset of the pages for
write.  These pages we read up are then decompressed.

Note one issue this patch does not address is reading of
data past EOF.  If the final chunk is unaligned, we will
round the write to cover it.  This results in extending the
file inappropriately, writing zeroes where they aren't
needed.  The read side gives us the info to address this,
which we will do in a future patch.

DDN-bug-id: EX-7601
Was-Change-Id: Iede43f12127cbb93e73c22a915192aa2f814a927

osd: add assert for prepare partial page

In the write prep code, we read up any partial pages (pages
which are not completely overwritten by the write) to
prepare them for write.  But for compressed files, we will
have already done this to prepare for decompression.

Add an assert to make sure we catch if this is ever wrong.

DDN-bug-id: EX-7601
Was-Change-Id: I1366b1f5b191a4d581448d692933d562198c3a1f

ofd: minor debug improvements

A smattering of minor debug improvements across several
patches, placed at the end because they're all minor and
some of them would disturb early parts of the series.

DDN-bug-id: EX-7601
Was-Change-Id: I2071911eb09f5c7fad28203db05396bb31ccda59

ofd: set compressed file size for fake writes

When using the fake writes fail_loc, file size setting is
done at the ofd layer, since the osd layer isn't used.  So
we must also handle the compressed file size for this case.

This fixes sanity test 399a with compression.

DDN-bug-id: EX-8826
Was-Change-Id: Icda612405908166d043e1e568d0d8bd9cd0c5156

tgt: objcount in RPC must be 1

Much of the BRW write code assumes objcount is one, but
there is some provision for multiple objects.

Since the code will break if we send it multiple objects,
add errors to make sure anyone changing it will notice.

This isn't strictly compression related, but compression
adds even more code which assumes this, so this protection
will be useful.

DDN-bug-id: EX-7601
Was-Change-Id: Idcbf33fd14d4b1bd179c9516bed07cca907008bc

ofd: multiple reads in same chunk

When doing DIO or if we get unusual cache behavior on the
client, multiple reads can hit the same chunk.

This only shows up in racing tests, but it's important to
handle.  We do this by making sure we start searching the
lnbs for decompression at the start of the last chunk we
decompressed.

DDN-bug-id: EX-7601
Was-Change-Id: I81fbbba79b16066e6d4519c66030cc58e03d2de7

ofd: don't read for writes past eof

There's no data past EOF, so there's no need to do
read-modify-writes when the entire write is past the chunk
at EOF.  So in that case, don't read up data and don't
attempt decompression.

There's no explicit test for this, but this shows up
immediately in the random-offset copy tests, because they
seek and write various sizes to offsets past current EOF.

We also need this functionality for reads, because in some
cases the client will do reads past EOF (this is unusual,
but can still happen sometimes).  This is added in a
separate patch because it requires some code reorganization.

DDN-bug-id: EX-7601
Was-Change-Id: Ia2b598165d5645c5a44c3d58bea69c7e42f10e41

ofd: same-ify preprw_read and preprw_write

preprw_read and preprw_write have some sections which are
functionally the same but which have diverged slightly.
(These can't easily be shared between the functions.)

This is a short patch to make them more similar before
adding eof checking to reads.

DDN-bug-id: EX-7601
Was-Change-Id: I7bce912e99e61a4eec4060d6b49d4917894b44c4

ofd: put decompress_read in read prep

ofd_decompress_read is called from ofd_write_prep for
writes, but from tgt_brw_read for reads.  This makes the
code a little harder to follow and makes it difficult to
check read side decompression against EOF.

Instead, we move the decompression call to ofd_preprw_read.
This makes no change to the real operations here, but makes
for better code (and more similar code between read and
write).

DDN-bug-id: EX-7601
Was-Change-Id: Ibefd0a48ad08e83725f2df64618db60ba61c5ce0

ofd: add past eof check for reads

The client does not normally generate reads past EOF, but
this can occur during some racing situations.  We need to
check for that case and not attempt decompression, since
there's no data to decompress if we're reading past EOF.

This covers a failure which shows up occasionally in the
racing parts of the test suite, but it's challenging to
write an explicit test for this.

We also add handling for complete reads of the last chunk,
even if that chunk is partial, because we can send that to
the client for decompression.

This allows us to remove the slightly funky eof handling
in decompress_rnb, since we'll just not call that code in
this case now.  Note we'll still call decompress_rnb, etc,
for writes if they start before EOF and finish after EOF
(and are unaligned).  This is fine - this case should be
rare and if we hit it, we'll notice there's nothing to
decompress and proceed accordingly.

DDN-bug-id: EX-7601
Was-Change-Id: I50295f2803af611de5069d094c0a5d1b0a4a9c2d

ofd: add checks to io_lnb_to_tx_lnb

We should always be able to find the remote niobuf in the
local io range, if we can't, there's a bug.  So assert on
this.

We should also never have page level overlapping remote IOs,
at least until we have unaligned DIO.  (We can remove this
check when we combine the features.)

DDN-bug-id: EX-7601
Was-Change-Id: I325d4a37d25c116e42621964e90b225b71fd8f1f

tests: tests for read-modify-write

This patch adds tests for the read-modify-write case for
EX-7601.  There's still some additional tests to be added
here, but this is a good start.

DDN-bug-id: EX-7601
Was-Change-Id: I5dd9e566b8274ece99283c8962e0d34225089cc0

tests: use alternative data source for rewriting

Using the same file as input has disadvantages. It is not
possible to understand that data was not rewritten at all.
Alternative data source should be used.

Let's shift source file data and use it as a source.
To check rewriting result the same operarion is performed
on the destination file copy stored outside the Lustre FS.

DDN-bug-id: EX-8598
Was-Change-Id: I6ef400520359bfe9156c3f47e757064863bdf4e0

tests: add multi-mount compression test

This adds a multi-mount correctness test for compression.
This races IO from two mountpoints at varying sizes to
stress test compression.

DDN-bug-id: EX-7601
Was-Change-Id: If49cbd6d171068faa802835146f273d835b39bc3

obd: add 'lvl' for best and fast

'best' and 'fast' compression types must also set a level,
because not all levels are supported by all algorithms.

Rather than trying to be clever, just use simple universally
supported values, except for lz4fast, where we special case
this, because otherwise '0' is the slowest setting (and
lz4fast is likely to remain our default fastest).

DDN-bug-id: EX-6269
Was-Change-Id: I7c29659d4f027af2e44285ae38e4c9d91e35509a

ofd: use niocount consistently

'niocount' refers to the number of remote niobufs, ie, the
number of separate IOs from the client.  Except for a few
places, where it's used to refer to the number of pages in
the entire RPC.  Eek.

Replace this usage with 'npages', making niocount usage
consistent.

DDN-bug-id: EX-8993
Was-Change-Id: I266087ad8dccadb54c054b0a11fb03dc9868a725

csdc: allow specify 'fast'/'best' compression type

Use lctl set_param osc.*.compress_type_{fast|best}=<type>:<level>
to specify the compression_type:level for LL_COMPR_TYPE_FAST/
LL_COMPR_TYPE_BEST.

lctl get_param osc.*.compress_type_{fast|best} will list these
values.

DDN-bug-id: EX-8311
Was-Change-Id: Ifeff7f25e30fc0884f0c770a3b6d0798937b3c35

lfs: not iterate compr_type_table using ARRAY_SIZE

patch modifies compr_type_table to contain NULL fields in the
array, so iterate over the array should not use ARRAY_SIZE, but skip
those elements with NULL compression type name.

DDN-bug-id: EX-8311
Was-Change-Id: I8e4988fd3a63c1cb66f75510d190c2ebc4f8f9be

ofd: handle 'missing object' reads

When the read code (eg, mdt_preprw_read) finds there is no
object, it will return a read with 0 pages, but not fail the
read.  The assert for local and remote pages needs to
recognize this case.

DDN-bug-id: EX-8996
Was-Change-Id: Idc6ff70f71abc100f750a63eca73a754a56f6435

ofd: do not write 'hole' pages on compression

When doing unaligned read-modify-write to a compressed file,
we must round the IO lnb used for write in order to read up
the compressed data for modification.

In some cases, this creates a situation where there are
pages in the write lnb which have no data in them.  It is
important not to write out these pages, because if we do,
this wastes space and can cause incorrect file size.

In most cases, the file size is covered by the client
sending the file size, but if the client does not compress
a particular write, it does not send the size and the server
does not use it.  We could resolve this by having the client
always send size info and have the server always use it, but
it's better to make server writes 'hole' aware, since this
improves space usage.  (And this will be required for the
server to do recompression on read-modify-write, otherwise
no space is gained.)

DDN-bug-id: EX-8993
Was-Change-Id: I66169e205fe4691ed03b2c9b3005ffc4ecd3213d

obdclass: disable T10PI guard from disk when decompressing

When server handles chunk unaligned read, we need to read
the chunk from disk and decompress the raw data.
This means we should not use the guard tags from the storage,
as they only match the raw data.
Disable the guard from disk and the T10PI checksum will be
recalculated later using decompressed data.

DDN-bug-id: EX-9108
Was-Change-Id: I49964e1412e9d02769797c64aab290b17851e26f

ofd: take local chunk-aligned lock

On OST side to prevent racing read-modify-write against same
compressed chunk from the same client.

DDN-bug-id: EX-9192
Was-Change-Id: Iffaf2d2856e276cb2f9becce2506154314217e3c

csdc: Do not print error if a chunk is not compressed

is_chunk_start() can decide that a chunk can not be decompressed in
two cases: 1) chunk has not been compressed 2) chunk is corrupted

The error message should be printed only in case 2)

DDN-bug-id: EX-9784
Was-Change-Id: I85f4850f989ba0fc8f00653f8f6b0f1b4837d625

osd: add @read param to dt_write_prep()

the purpose is to skip full page reading when OSD is aked for
pages to be written later. in CSDC we need all data as the
pages can be compressed and we may need to decompress them
first.

DDN-bug-id: EX-9858
Was-Change-Id: I2ea9888a1dab419275dbee97e9acc661101b5262

ofd: new structures and data for csdc in ofd

just structures and few helper function to manipulate
chunks and buffers.

DDN-bug-id: EX-9858
Was-Change-Id: I25931f8360a30689a027eaf05ad6c43b4f1746f9

ofd: chunk decompress function

ofd_decompress_chunk() is to literally decompress chunks.
it uses kernel's vmap infrastructure to create a virtual
linear buffer from the pages provided by OSD.

memcpy vs vm_map_ramp()+vm_unmap_ram() benchmark as it's
a replacement for memcpy, each operation repeats 1M times
in each thread, then cost in usec is calculated:

memcpy 32 pages 1 threads - 2404282998, 2404 usec/1
memcpy 32 pages 2 threads - 4830830436, 2415 usec/1
memcpy 32 pages 4 threads - 10098563354, 2524 usec/1

vmap 32 pages 1 threads - 1088538713, 1088 usec/1
vmap 32 pages 2 threads - 2207667980, 1103 usec/1
vmap 32 pages 4 threads - 4613110111, 1153 usec/1

memcpy 64 pages 1 threads - 5564630189, 5564 usec/1
memcpy 64 pages 2 threads - 15904163026, 7952 usec/1
memcpy 64 pages 4 threads - 30345722986, 7586 usec/1

vmap 64 pages 1 threads - 1587480151, 1587 usec/1
vmap 64 pages 2 threads - 3190874847, 1595 usec/1
vmap 64 pages 4 threads - 6887315343, 1721 usec/1

the benchmark was done in qemu (6 cores enabled).

DDN-bug-id: EX-9858
Was-Change-Id: Ia46406401d3c8d720065cc8000b6a52d1a437349

ofd: vm_map_ram prot argument

From kernel 5.8 commit d4efd79a81 "mm: remove the prot argument from
vm_map_ram" the vm_map_ram() function does not accept a prot argument
anymore.

DDN-bug-id: EX-9858
Was-Change-Id: Ib878ba6c6a214ebc0b0716392ce9b7e91bb74453

ofd: csdc read helpers

few functions to implement read path in ofd.

DDN-bug-id: EX-9895
Was-Change-Id: Idca3b095471dbbf177b4026853aa716414280757

ofd: csdc write helpers

few functions to implement write path in ofd.

DDN-bug-id: EX-9895
Was-Change-Id: I0f6802b0911bd358bdb69e0ddb5e8581878cb9dc

ofd: csdc truncate support

to truncate a compressed file correctly we may need to
convert a compressed chunk into a regular one first.

DDN-bug-id: EX-9895
Was-Change-Id: Ia8a3b12eda81874e209046eef1afd33e165561b0

ofd: ofd_object_punch_compr

Fix potential memory leak in ofd_object_punch_compr.
It might not release orig_lnb in case of error from
ofd_trans_create or ofd_map_niobuf_to_chunks.

DDN-bug-id: EX-11092
Was-Change-Id: I45bafdbbf06036add5b5fa9993cbac9e06c856e9

tests: couple new FAIL codes

and trivial way to break a compressed chunk on a disk.

DDN-bug-id: EX-9858
Was-Change-Id: I5516ab3d7add4eb5d76292cf15ac220db08a7150

csdc: expand filter_fid

Expand filter_fid to include compression information, for
compatibility reason, if the file is an uncompressed file, still
store the old filter_fid with no compression info in FID EA.

WasChange-Id: I388500c03604749d05849aeed3c9141974540e4a

lfsck: support compression in filter_fid

The OST object compression info has been stored in filter_fid
by EX-8038, the LFSCK also needs to handle the compression info
during processing the filter_fid.

DDN-bug-id: EX-6247
Was-Change-Id: Id050214ba0057776a05c194dc9222117c5d7fd8b

ofd: add info in overflow error message

Such a message should help to inverstigate which
client triggered the overflow.

DDN-bug-id: EX-9895
Was-Change-Id: I701d8c8fb2a37684ecb0cd5cd272321c715db864

ofd: reject incomplete compressed chunks

client can send a compressed tail, but this RPC can race with another
newer RPC containing bits (uncompressed) from the same compression
chunk as the first RPC. OST can't handle such a compressed partial
chunk properly (that would need to decompress into a temp buffer,
then merge back to disk buffers). but we can ask the client to resend
this RPC uncompressed.

DDN-bug-id: EX-10402
Was-Change-Id: I5e854be4300d50bc8368911782018fb6fa6d38fa

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I7b95416530b5d0f1994900ee4ae1ceb392046e32

62537 master J –B ✗T –R 0/2

LU-18756 ofd: repair missing parent FID on OST read

work in progress

WIP L +351 −38

12d

build #134162 ABORTED

no vote yet: Andreas Dilger

patchsets: PS1 11-14 PS2 10-21 PS3 10-21

owner Marc Vef · uploader Marc Vef · PS3 uploaded 12d ago · NEW · open in Gerrit ↗

commit message

LU-18756 ofd: repair missing parent FID on OST read

An OST object gets its parent FID (trusted.fid) only when a client
writes to it, truncates it or changes its ownership. Objects that
never received such an RPC, e.g. the unwritten stripes of a sparse
file, have no parent FID, so scanning an OST cannot map them back to
the MDT file that references them, e.g. to migrate them off the OST.

On OST_READ, queue a repair of a missing parent FID to the resource ID
repair thread, which writes it in the same transaction as any missing
UID/GID/PROJID. The parent FID, stripe index and OST layout from the
obdo are stored as a write stores them, so LFSCK finds the component
ID of PFL files. Like the ID repair, it is disabled by
enable_resource_id_repair=0. It is queued whether or not
enable_resource_id_check is set, but not when that check denies
access.

Only a missing parent FID, or one that fails fid_is_sane(), is
repaired. A valid parent FID that differs from the obdo may come from
a second layout referencing the object, which is left to LFSCK. The
parent FID sent by the client is trusted like the UID/GID repaired
from the same RPC, also from clients of read-only exports; a write
stores it unconditionally.

filter_fid::ff_parent::f_ver holds the stripe index, so only f_seq and
f_oid of the parent FID are checked with fid_is_sane().

Stripes that are never read, e.g. beyond EOF, still have no parent
FID.

Add sanity-sec test_75c: unwritten stripes 0 and 1 of PFL files get
their parent FID and layout on read, with the ID check disabled and
enabled; a read with another parent FID keeps a valid one; and an
object missing both IDs and parent FID gets both repaired.

Test-Parameters: testlist=sanity-sec env=ONLY=75b,75c
Test-Parameters: testlist=sanity-lfsck
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iafa97b1204a96ef3349009a340fd812528ae43ef

68697 master J !B ✓T ✗1R ✗

LU-19548 mdd: refuse to split a parity mirror to a victim

parked by your own -1

CR -1 pt_ecro S +28 −5

53d

janitor run: 3 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-part-2	RHEL 9.7 / x86_64	ran 11 tests. 1 tests failed: sanity-lfsck.	session

build #131271 SUCCESS · tests all Maloo sessions

reviews: Marc Vef -1 2026-09-08 20:12

patchsets: PS1 09-07 PS2 09-07

tags: pt_ecro

owner Marc Vef · uploader Marc Vef · PS2 uploaded 55d ago · NEW · open in Gerrit ↗

commit message

LU-19548 mdd: refuse to split a parity mirror to a victim

A file whose layout holds only an EC parity mirror cannot be read:
parity is not file data, and there is no data mirror left to serve
reads. lfs already refuses to split a parity mirror into a file of
its own; the MDS should not rely on the client for that. Pass whether
a victim object exists (mrd_obj != NULL) into mdd_split_ea() and
return -EINVAL when the target is a parity mirror. Destroying a parity
mirror with 'lfs mirror delete' is unaffected.

lfs no longer takes the old-MDS volatile fallback for a parity
target. An MDS that knows what a parity mirror is also has the
fd == fdv purge path (LU-14521), and with the preceding LU-18466 fix
that path no longer fails with -EINVAL, so the fallback could only
build a volatile victim that the MDS now refuses. A client old enough
to always route 'lfs mirror delete' through a volatile file cannot
delete a parity mirror against this MDS.

Assisted-by: ClaudeCode:Fable-5.1 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id41a40b21f3d31635904a1233653b91ba6d55a6c

68696 master J ✗1B ✗T –R ✗

LU-18466 mdd: declare layout split before purge

parked by your own -1

CR -1 1 unique failure S +10 −7

54d

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-quota@ldiskfs+DNE:test_58	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #131270 FAILURE

reviews: Marc Vef -1 2026-09-08 20:12

patchsets: PS1 09-07 PS2 09-07

owner Marc Vef · uploader Marc Vef · PS2 uploaded 55d ago · NEW · open in Gerrit ↗

commit message

LU-18466 mdd: declare layout split before purge

mdd_xattr_split() declares the mirror purge before the layout split
when a mirror is deleted in place (fd == fdv, LU-14521). Declaring
the split bumps the object's layout generation, so the second LOV
declaration on the same object trips lod_save_layout_gen_intrans()
and the whole operation fails with -EINVAL:

  lod_save_layout_gen_intrans() obj ... gen changed from 4 to 5 in
  trans declaration

lfs treats that -EINVAL as "old MDS rejects fd == fdv" and silently
retries through a volatile victim file, so every 'lfs mirror delete',
'lfs mirror split -d', 'lfs migrate -L mdt' and 'lfs pcc delete' has
taken the volatile-file path instead of the purge path since the
generation check was added, printing a stray "cannot get UNLOCK lease
...: Invalid argument (22)" on the way. Declare in the order the
changes are applied, split first, so the recorded generation still
matches.

Fixes: d5ab62af2416 ("LU-17048 mdd: protect layout change in MDD layer")
Assisted-by: ClaudeCode:Fable-5.1 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I624f537059c398facfb4693754731222e7e55e77

65770 master J –B ✗T –R ✗

LU-10026 osc: osc brw request compression

parked by your own -1

CR -1 csdc 15 unresolved XL +1774 −50

62d

build #130259 FAILURE

reviews: Marc Vef -1 2026-09-01 09:48

patchsets:

+1 earlier

PS1 05-08

PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 08-25

threads: 12 need your reply · 0 waiting on others · 3 own notes · 0 bot

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre COMMIT_MSG — (defect) The message describes a "sec: page_pools shrinker fix" (element_size / ppp_idle_idx / IDLE_IDX_MAX changes) and adds sanity-compr t

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) The message describes a "sec: page_pools shrinker fix" (element_size / ppp_idle_idx / IDLE_IDX_MAX changes) and adds sanity-compr test_2000 gated on client 2.17.53 "for compression OOM fix", but there is no page_pools.c change anywhere in the diff, so test_2000 can't pass on this commit. A few hunks also aren't accounted for by the message: the ll_lov_getstripe_ea_info() switch to md_getattr() when filename is NULL in llite/file.c, and the LINVRNT->LASSERT change in cl_io_rw_init(). Are these meant to be part of this patch? Also, since this touches the OST_WRITE/OST_READ wire path (OBD_BRW_COMPRESSED, o_size/OBD_MD_FLSIZE), a Test-Parameters: line requesting client/server interop testing would be worthwhile.

reply PS5 2026-07-27 15:10 Andreas Dilger COMMIT_MSG:806 — (defect) this needs to add:

Andreas Dilger PS5 · 2026-07-27 15:10

(defect) this needs to add:
```
Test-Parameters: testlist=sanity-compr
```
otherwise that test session is not being run on master.

note PS6 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS6 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebased onto the current series base (65769). Notes on this patchset:

- OBD_FAIL_OSC_WRONG_COMP_ALG renumbered 0x418 -> 0x41b: 0x418 is already
  OBD_FAIL_OSC_FIEMAP on master (the previous patchset defined a duplicate
  value). sanity test_460b's fail_loc values updated to match
  (0x8000041b / 0x1000041b).
- Adapted to the reworked struct ll_compr_hdr from 65769: split magic
  (llch_magic_lo/llch_magic_hi) and the packed llch_level_and_chunk byte
  with the LLCH_COMPR_LEVEL()/LLCH_CHUNK_LUM_BITS()/LLCH_LEVEL_AND_CHUNK()
  accessors replacing the old level/chunk bitfields, in compress_chunk(),
  is_chunk_start() and osc_decompress().
- compress_chunk() now also zeroes llch_extra_flags - previously the field
  was left uninitialized in the pool-allocated buffer and written to
  storage as garbage.
- Dropped the check_ll_compr_type() call from wirecheck main(): the current
  base's check_ll_compr_hdr() already emits the identical
  CHECK_VALUE(LL_COMPR_TYPE_*) set; the separate function no longer exists.
- lov_io_slice_init()/lov_io_mirror_init() gained a const struct lu_env *env
  first argument on master; the compressed-file detection block inserted
  there was adapted to the new signatures.
- Fixed a misindented closing brace in lov_io_submit() (came in with the
  original patch).

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre obd_support.h:440 — (defect) 0x418 is already used by OBD_FAIL_OSC_FIEMAP two lines above, so this fault injection point collides with it (triggering one fires

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) 0x418 is already used by OBD_FAIL_OSC_FIEMAP two lines above, so this fault injection point collides with it (triggering one fires the other). 0x41a or 0x420 appear free.

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre lustre_compr.c:99 — (defect) On the decompress path *type is llch_compr_type read from storage/the wire, so a corrupt header carrying LL_COMPR_TYPE_BEST or LL_C

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) On the decompress path *type is llch_compr_type read from storage/the wire, so a corrupt header carrying LL_COMPR_TYPE_BEST or LL_COMPR_TYPE_FAST would LBUG the client here. The sibling change that replaced an assert with EIO ("we shouldn't assert on values read from storage") applies - return -EIO instead of asserting?

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre osc_compress.c:285 — (minor) If fill_cpga() fails here, `dst` (obtained just above from obd_pool_get_objects) has not yet been stored in bp_cmp_chunk, and the ou

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(minor) If fill_cpga() fails here, `dst` (obtained just above from obd_pool_get_objects) has not yet been stored in bp_cmp_chunk, and the out: path only releases `src`, so the chunk buffer looks leaked on this error return.

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre osc_compress.c:451 — (defect) compressed_pages comes from llch.llch_compr_size, which is read straight out of the server reply by is_chunk_start() (that only che

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) compressed_pages comes from llch.llch_compr_size, which is read straight out of the server reply by is_chunk_start() (that only checks llch_magic). It is bounded above only by page_count - i, not by pages_per_chunk. merge_chunk() then copies compressed_pages * PAGE_SIZE bytes into `src`, a single chunk_size buffer from obd_pool_get_objects(&src, buf_bits). A corrupt or hostile compr_size larger than one chunk overflows `src`. The LASSERT(src_size <= chunk_size) on the next line runs only after the copy (and asserting on wire data is itself unsafe). Can compressed_pages be bounded against pages_per_chunk before merging, returning -EUCLEAN otherwise?

note PS6 2026-09-01 09:33 Marc Vef osc_compress.c:101 — (defect) `pg->bp_pgno` is not set. This causes silent data corruption on native-folio kernels: the compressed output buffer can be a multi-p

Marc Vef PS6 · 2026-09-01 09:33

(defect) `pg->bp_pgno` is not set. This causes silent data corruption on native-folio kernels: the compressed output buffer can be a multi-page compound allocation, so every one of its pages maps to the same head folio. Fix below as prepared by Patrick:

```
diff --git a/lustre/osc/osc_compress.c b/lustre/osc/osc_compress.c
index a0e569c89b..b0bb2142e2 100644
--- a/lustre/osc/osc_compress.c
+++ b/lustre/osc/osc_compress.c
@@ -52,10 +52,24 @@ void free_cpga(struct brw_page **cpga, u32 page_count)
  OBD_FREE(cpga, page_count * sizeof(*cpga));
}

+/*
+ * Index of @page within @folio.  Mirrors cl_folio_pgno(), which takes a
+ * cl_page; here the page comes from the compressed output buffer instead.
+ */
+static inline int compr_folio_pgno(struct folio *folio, struct page *page)
+{
+#ifdef HAVE___FILEMAP_GET_FOLIO
+  return folio_page_idx(folio, page);
+#else
+  return 0;
+#endif
+}
+
static int fill_cpga(struct brw_page **cpga, struct brw_page **pga, char *dst,
         int src_from, int dst_from, size_t dst_size, int order)
{
  struct brw_page *pg;
+  struct page *cpage;
  int chunk_offset;
  int dst_page;
  int src_page;
@@ -98,7 +112,18 @@ static int fill_cpga(struct brw_page **cpga, struct brw_page **pga, char *dst,
     * patch.
     */
    pg->bp_count = PAGE_SIZE;
-    pg->bp_folio = page_folio(mem_to_page(dst + chunk_offset));
+    cpage = mem_to_page(dst + chunk_offset);
+    pg->bp_folio = page_folio(cpage);
+    /*
+     * The compressed output buffer can be a multi-page (compound)
+     * allocation, in which case every one of its pages maps to the
+     * same head folio.  bp_pgno selects the page inside that
+     * folio, so it must be set here: otherwise brw_folio_page()
+     * returns the folio's first page for every brw_page and a
+     * compressed chunk spanning more than one page is sent as
+     * repeated copies of its first page.
+     */
+    pg->bp_pgno = compr_folio_pgno(pg->bp_folio, cpage);
    /* we get flags from the first page in the chunk and
     * add COMPRESSED
     */
 
```

reply PS6 2026-08-26 08:52 Misc Code Checks Robot (Gatekeeper helper) osc_compress.c:216 — warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-08-26 08:52

warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

reply PS6 2026-08-26 08:52 Misc Code Checks Robot (Gatekeeper helper) osc_compress.c:233 — warn: compress_request():missing conversion: 'pga_i + chunk_len' 'unit_array_size + unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-08-26 08:52

warn: compress_request():missing conversion: 'pga_i + chunk_len' 'unit_array_size + unit_page'

reply PS6 2026-08-26 08:52 Misc Code Checks Robot (Gatekeeper helper) osc_compress.c:242 — warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-08-26 08:52

warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

reply PS6 2026-08-26 08:52 Misc Code Checks Robot (Gatekeeper helper) osc_compress.c:271 — warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-08-26 08:52

warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

tags: csdc

owner Marc Vef · uploader Marc Vef · PS6 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: osc brw request compression

This patch adds client-side compression/decompression.

The client-side data compression project (CSDC) reduces storage and
network
utilization by leveraging the more plentiful memory and CPU resources
on the
local client. Data is sent compressed over the network, saved
directly to
storage on the server side, and decompressed back on the client side.
Uncompressed data is kept in client page cache, all while being
functionally
transparent to the end user and application.

As an example, a test file is compressed and decompressed.
The resulting file is compared with the original one.

The test case shows 2.5x compression ratio:
356K	/mnt/lustre/d460.sanity/sanity.sh
884K	/tmp/cmp-46ofie/decompressed_sanity.sh

Compression should read whole chunk even if offset
and size differ.

Let's modify readahead to force reading data from the
offset and size multiple to the chunk size.

utils: fix cp_comp_type size

cp_comp_type should be 8 bits, as llch_compr_type and all associated
variables are declared as u8.
So remove useless cp_comp_enabled and fix code to test for compressed
component with cp_comp_type against LL_COMPR_TYPE_NONE.
And update LL_COMPR_TYPE_MAX value to 255 to avoid conflicts with
future compression types.

DDN-bug-id: EX-7775
Was-Change-Id: Ia15868ac0ac003b62942540a57f782226ae8c141

tests: test compression without bzip2/HDF5

Run as much of sanity test_460a compression tests as possible,
even if bunzip2 or HDF5 file are unavailable.

Print a clear message in test_84 if bunzip2 unavailable.

DDN-bug-id: EX-6127
Was-Change-Id: I36251834f636600eb9b0194ccd14c8b203da32e5

lov: refactor lov_io_lsme_at

lov_io_lsme_at needs some minor changes to be called from
lov_io_slice_init().

DDN-bug-id: EX-7601
Was-Change-Id: I0611d66052e22d349932eb26257369e07b9b8167

osc: don't check for start inside the chunk

Chunk size is the same for the whole request and every
chunk offset is multiple to a chunk size.

No need to search for compression header in every page.
It is enough to check every with offset multiple to a chunk
size.

DDN-bug-id: EX-7818
Was-Change-Id: Ie2ef645130656279e152ea1f7e6db01cb33836ca

osc: minor compression cleanups

This cleans up some style and argument issues I found made
the code a little harder to follow.

DDN-bug-id: EX-8270
Was-Change-Id: Ia3492ae79acf6c83d724cc91b0201c7872325853

osc: move common CSDC code to the library

CSDC repacks a chunk on the server side in case of the
partial rewrite. There are routines that can be shared
between client and server.

This patch moves common compression code to the
libcfs.

DDN-bug-id: EX-7601
Was-Change-Id: I824211a3435b0479f7a3b8f08598a5b567b67d3c

osc: use correct count

Using the number of bytes in the compressed page creates
gaps in the RDMA, which IB memory registration cannot
accept.

Fix this by always setting count to PAGE_SIZE for
compressed pages and otherwise using the count from the
original source page.  Setting PAGE_SIZE for compressed
pages is valid because client only does compression for
aligned IO, except for the trailing chunk.  For the
trailing chunk, the file size is set on the server, so any
trailing bytes are ignored.

DDN-bug-id: EX-8245
Was-Change-Id: Ied89d3ac328fb6020079392f5a8812ad5637b4a4

csdc: remove holes from struct ll_compr_hdr

This patch reorganizes struct ll_compr_hdr to remove
alignment holes.

DDN-bug-id: EX-8353
Was-Change-Id: I59800b00e3a17972d621bae21ba06509a39b1036

osc: apply compressed flag to dst page

The existing code to apply brw flags to compressed pages
has two issues:
1. The dst_page is NOT an osc async page, it is a bare BRW
page.  This means the brw_page2oap macro isn't right,
because there is no oap page.
Because oap_brw_flags is actually oap_brw_page.flag, we
don't ever access the memory pointed at by OAP, just use it
to find an offset back in to the brw page.

This means the flags are set correctly, but we still
shouldn't use this macro.
2. However, the function then overwrites these flags by
copying from a page in the source, so OBD_BRW_COMPRESSED is
lost.

Add OBD_BRW_COMPRESSED when we set flags.  This ensures the
flag is actually sent to the server on compressed IO.

This was not causing any problems because the server does
not actually use the OBD_BRW_COMPRESSED flag yet.
(EX-7601 uses this flag)

DDN-bug-id: EX-7601
Was-Change-Id: Ia94cdc803868ce16a0b66fd58578ec8b2d00cbae

osc: remove unused 'wrkmem'

compress_chunk() takes a wrkmem buffer, which it does not
use.

Remove this and its allocation in compress_request.

DDN-bug-id: EX-7601
Was-Change-Id: I6f236f018f5b79c57cc8725ca0f95125810a4064

osc: walk chunk unaligned RPC correctly

For decompression, the client must start looking for
compressed chunks at a chunk aligned offset.

Implement this in decompress_request.

DDN-bug-id: EX-7601
Was-Change-Id: I3273135990ddf51e8b3c651734e19350e91f659c

ofd: add obd level compression lib

Some compression functions will be used by several areas of
of Lustre, so they need to be in obdclass.

This moves merge_chunk and unmerge_chunk there and adds the
ability for them to merge lnbs.  This is used in a future
patch.

DDN-bug-id: EX-7601
Was-Change-Id: If4a318119bb7685e41adb9f3b31a66074031e6ac

osc: remove cpga fill bits

cpga fill bits are not needed now that we don't support
compression and encryption.

DDN-bug-id: EX-7601
Was-Change-Id: I13c2278e085e9b288bd896585947e28e2ea505ca

osc: only set compressed flag on compressed pages

The code accidentally sets the compressed flag on all
pages processed through fill_cpga, even if they're not
compressed.  Oops.

Also stop setting pg->index on the pages in the compressed
pga, this is only used by encryption and that's no longer
supported with compression.

DDN-bug-id: EX-7601
Was-Change-Id: I313fd943a18b71cd52493852a6884f30d187e52f

osc: use pages_left in unmerge_chunk

Since we have compressed chunks < chunk_size (if they're
after EOF), we must use pages_left in unmgerge_chunk or it
will go off the end of the page array.

This also lets us remove the workaround where unmerge_chunk
would skip pages that were not present.  unmerge_chunk
always works with a known and complete set of pages, so this
check is unneeded.

We should also check that our count of bytes is correct
when we finish.

DDN-bug-id: EX-7600
Was-Change-Id: I88896307990ff839514e54e9a7e18390a457e5d8

osc: rename 'done'

Rename the ambiguous 'done' and remove it where not used.

DDN-bug-id: EX-7601
Was-Change-Id: I8fb88b7a91fcc7dbd5ce2d29a61c18330fc0cda3

osc: cleanup compression variables

Make usage of the compression variables more readable.

DDN-bug-id: EX-7601
Was-Change-Id: I6daff56b56877c8f36e02303cc0579ba7faa731b

ofd: make compress_chunk take chunk_bits

Chunk bits is used everywhere, have compress_chunk convert
to log bits rather than have the callers do it.

DDN-bug-id: EX-7601
Was-Change-Id: Ic01bb749425cb95d9c5717965d692a18138ceeb7

obd: move module load to function

This is a trivial code change to make alloc_compr a bit
shorter.

DDN-bug-id: EX-7601
Was-Change-Id: I0a790afe7afebde1d223420d9a578529da6ff7e5

osc: variable cleanup in decompress_req

Use type and lvl variables in decompress_request.

Remove an unused variable and an assert which can never
fire.

DDN-bug-id: EX-7601
Was-Change-Id: Ieff57411a2a41215fd368d731614801bd0f43e38

osc: replace assert with error

We shouldn't assert on values read from storage, instead if
they are incorrect, we should give EIO.

DDN-bug-id: EX-7601
Was-Change-Id: Icda213e3c5a90a848c9b008788e92ee49e2efcb1

osc: rearrange compress_request

A trivial rearrangement of compress_request to make it
more readable before redoing the core logic.

DDN-bug-id: EX-7601
Was-Change-Id: I1d34cd2a2a6d84bc30cc7dae8eb07586c4837f7d

osc: give compress_request explicit success

Compress_request has explicit failure handling, but the
success handling just follows the failure handling.  This is
confusing - on failure, we do:
page_count = *pcount
then immediately do:
*pcount = page_count

It also sets *orig_pga = pga on success OR failure, which
is wrong because compress_request may have modified pga and
then failed.

DDN-bug-id: EX-7601
Was-Change-Id: I121ec71cfe35babc4a572951e93f7581887ade80

osc: remove &pga usage in compress_request

The usage of 'pga' and '&pga' in compress_request is
confusing, but also, compress_request modifies &pga by
allocating a new compressed page array.  Except if we fail
in compress_request, we free that new page array.

This means failing in compress_request replaces 'pga' with
a pointer to freed memory.  Instead, create an explicit
cpga pointer in the caller and use that.  This allows
compress_request to fail safely.

DDN-bug-id: EX-7601
Was-Change-Id: Idaf592103c57b0e9ce76ab520a69b819d4f37be9

obd: move type switching to alloc_compr callers

The code is much cleaner if we can eliminated applied type
and handle that issue once per compression or decompression
rather than for every chunk.  This requires moving the type
switching inside alloc_compr.  (Also improve some error
messages - alloc_compr can fail with ENOMEM as well.)

The compression code currently allocates a transform for
every chunk on the client.  This is relatively cheap, but
it also complicates the code by repeatedly checking if a
particular compression type is supported (this is the
"applied type" code).

Moving alloc_compr to compress/decompress request makes the
code much simpler.

DDN-bug-id: EX-7601
Was-Change-Id: I162e81577db721a9715d57b3f262fcabbcbf308a

osc: rename pages_in_chunk

Chunks can have variable numbers of pages in them.

DDN-bug-id: EX-7601
Was-Change-Id: If199d777367569e62c21305f6e4b9f3e4cce6d06

osc: allow multiple chunks in read

It's rare, but reads can sometimes have multiple
discontiguous chunks.  Update decompress_request to
handle this case.

DDN-bug-id: EX-7601
Was-Change-Id: I880af95db285dce76db3610e8140a0f54baa401b

ofd: do not overwrite rc in unmerge_chunk

unmerge_chunk should not be responsible for setting the
lnb rc, because this overwrites the result of any previous
activity on the lnb.  Plus, unmerge_chunk can't fail.

DDN-bug-id: EX-7601
Was-Change-Id: Id1ce590c7f1da3ab7faddbd685d264a33c08d639

osc: calculate compressed size reduction accurately

Compression reduces space used if it results in allocating
at least one fewer block on disk.  Modify the checks in
compress_chunk to reflect this, rather than using the
simpler "reduce size by at least 4K" calculation.

Also do not attempt to compress chunks if they are less
than 4K in size, since they can't possibly get a space
benefit.

This improved my measured ratio on a version of the Linux
kernel source data set from 1.24 to 1.56, so this is
significant for datasets with many small files.  (This
version of the source had large incompressible files
removed, to focus on smaller files.  The unmodified data set
would not improve as much.)

Note this is still short of our estimates, so either the
estimate or Lustre still needs adjustment.  TBD.

DDN-bug-id: EX-7601
Was-Change-Id: I815706914b88de4f532a674d773769aa3a64d218

osc: rewrite compress_request

The existing version of compress_request can't handle
discontiguous RPCs.  Rewrite the logic to handle this
case properly.

This also implements kms handling.

If a write chunks ends at the known minimum size, we know
this write is after all other data in the file and so
there is no compressed data under it.  This means we can
compress this chunk.

DDN-bug-id: EX-7601
Was-Change-Id: I8a912d9e279d04c8ff07de39e63a1ec9b490d921

osc: handle partial chunks in decompress_request

Now that we have compression for incomplete chunks at the
end of files, decompress_request needs to handle these
chunks.  This patch modifies it to understand compressed
chunks which are less than chunk_size pages.

DDN-bug-id: EX-7601
Was-Change-Id: I877550fa0d418def406e0308392a5336ec9f3ab6

osc: add check to decompress_request

decompress_request should check to see if there's room in
the RPC for the decompressed data, since this can occur if
there's a bug or data corruption, and otherwise we will
go past the end of the RPC during decompression.

DDN-bug-id: EX-7601
Was-Change-Id: Ib1bf19bf39701b72f0f5a61b2aaff2f2fdad1897

osc: debug fix in decompress_request

Debug message had an incorrect subtraction.

DDN-bug-id: EX-7601
Was-Change-Id: I5daf360766ca77b98dc5af3d72c42ac38f5782bc

obd: add 'lvl' for best and fast

'best' and 'fast' compression types must also set a level,
because not all levels are supported by all algorithms.

Rather than trying to be clever, just use simple universally
supported values, except for lz4fast, where we special case
this, because otherwise '0' is the slowest setting (and
lz4fast is likely to remain our default fastest).

DDN-bug-id: EX-6269
Was-Change-Id: I7c29659d4f027af2e44285ae38e4c9d91e35509a

osc: decompress with algorithm from server

Data may not be compressed with the compression type and
level from the layout, so we must use the compression type
and level from storage for decompression.

DDN-bug-id: EX-6269
Was-Change-Id: Ib4cdccf294ef631a25147413d7f5c1a847c9504e

osc: handle different compression types

Allow the client to handle different compression types in a single
component.  This shouldn't happen normally, but it may happen in
the future if there is dynamic compression algorithm selection for
"fast" or "best" types (e.g. compress based on available CPU and
network bandwidth or RPC backlog).

DDN-bug-id: EX-6269
Was-Change-Id: Ide2731c60a68584e7cbb474bee88a17e9a7b8fec

osc: add COMPR_GAP check to compress_request

Currently, compress_request will build the compression
buffer (calling merge_chunk()) for requests which are less
than the minimum compression gap.  This is noticed in the
compression code when it checks if there's enough data to
attempt compression, but we can do a trivial check in
compress_request() to save that work.

Also fix a few minor style things.

This is not an important fix, but I discovered it while
investigating another issue and it's trivial to resolve.

DDN-bug-id: EX-7601
Was-Change-Id: Ieb32e6297e10d229f23c58e2ef4d933ce3dda4f2

lustre: add uncompressed size to compression header

It's useful to have the uncompressed size of the data in the
compression header.  Also, we have three checksum fields -
compressed, uncompressed, and header, but in practice,
checksumming the compressed data including the header is
enough to cover all of these.

This patch cleans up all of this at the same time.

DDN-bug-id: EX-8851
Was-Change-Id: Ie82e0dbe9c862ddc88999b109cea1f27577dbbff

csdc: rename "cp_comp_*" to "cp_compr_*"

This patch renames "cp_comp_type", "cp_comp_level",
and "cp_chunk_log_bits" to use "compr" in the name
to be consistent with other variable names.

DDN-bug-id: EX-8353
Was-Change-Id: I428ff3a789b33da02832dee02f316b02d97137e2

osc: Do not iterate over chunk pages

If osc_decompress() knows chunk size, no need to iterate
other all pages in chunk. Thay can be skipped.

DDN-bug-id: EX-7729
Was-Change-Id: Ib84b060075c55c97eba9f74ef017c0a956e85b12

csdc: Fix the upper mergeable chunk pointer

If the full chunk is followed by un mergeable page, the upper
mergeable chunk pointer is occasionally set to this unmanageable page.
The chunk size is calculated wrongly then and the next condition
suggest not to compress this chunk, because its size is not equal to
the expected size.

The pointer should be moved to the first instruction after the
can_merge_pages().

DDN-bug-id: EX-9192
Was-Change-Id: I09fedc770c8bbcac4864b32372a941da5e0c7ac3

obdclass: reset bits after decompression

as uncompressed data can be less than chunk/page, but still be
visiable to userspace as a part of a sparse file.

DDN-bug-id: EX-9873
Was-Change-Id: I4114b0704fb685013f4e03cf2d80ccde2cc8c87f

osc: preserve compressed pages for OST_WRITE replay

it's incorrect to release compressed pages right after reply
as we may resend them during OST_WRITE replay.

DDN-bug-id: EX-9895
Was-Change-Id: I3edc16d6556ddd60735d2f14fe879fc0f45231d7

csdc: Fix the next available algorithm selection

Currently, if a chosen algorithm is not available, then
next preferable is chosen, but an error code is not cleared
so __alloc_compr() returns this wrong code. Data if
written uncompressed while it can be compressed with
next appropriate algorithm.

This patch adds this error code clearing. Test is
provided.

DDN-bug-id: EX-9889
Was-Change-Id: I59f65058a0fe9b108de3d4ba7cf5950f18e32204

ptlrpc: reset refcount reusing pages for new req

when a set of compressed pages are reused for a redo RPC, we have
to reset refcounter.

DDN-bug-id: EX-10061
Was-Change-Id: I16c46d857d60cf4dd0f6317190e7c99b2667e135

ptlrpc: drop extra reference to compressed pages

if the request is not replayable, so brw_commit() won't be called.

DDN-bug-id: EX-10184
Was-Change-Id: Ib29ab19b69fd15ef339cc18c5dfef17217a32cb1

osc: remove noisy warning

if we can't compress data on the client side for a reason
(e.g. too random),
we just send data uncompressed, makes sense to hide the
error message confusing customers:
kernel: LustreError: 936336:0:(lustre_compr.c:345:compress_chunk())
	exafs-OST0007-osc-ff49193c293f3800:
	Compression failed, type 5, lvl 5, -22

DDN-bug-id: EX-10593
Was-Change-Id: Icbc8be38b0372f8f67690e7d94384a16ab02cbe4

osc: fix osc_compress ENOMEM handling

Send already compressed chunks in compress_request
even if can't allocate memory for the next chunks.
This patch also fixes statistic in case of ENOMEM.
Without that write_chunks_incompressible hasn't been
incremented in case of ENOMEM.

Add sanity-compr_2002 to check write_chunks_compressible
and write_chunks_incompressible in case of ENOMEM

DDN-bug-id: EX-11035
Was-Change-Id: I3fc58ece6cf22582166ea77e4a217bc7a9efd77d

csdc: remove buffer double sizes

Pools now (POOL_ADD_PAGE is defined) allocate
additional page for memory misuse debug, so
no need to allocate double-sized buffers to
prevent lz4 decompress problem.

DDN-bug-id: EX-8276
Was-Change-Id: Iea04943ae0f95c8983c5ba7ec9a2c63003afe351

utils: fix LL_COMPR_TYPE wire checks

Fix LL_COMPR_TYPE* wire checks to avoid duplication.
The value of LL_COMPR_TYPE_UNCHANGED is also declared as 255 so that
is does not conflict with other potential real compression types in
the future.

DDN-bug-id: EX-7775
Was-Change-Id: Iab9830f09f0778e1e1f3b1ea4c9878ce1017de8d

sec: page_pools shrinker fix

page_pool shrinker should consider ppp_free_pages
as a number of memory regions with the size of
2^pool_order pages. Otherwhise it startis shrinking
only when becomes too large, for example 2GB instead
of 64MB. It often might be the reason of OOM especially
when compressed files with different chunk-sizes
have been written. In a such case several pools
created for different chunk sizes might hold gygabytes
of memory without any chance to release it.

Fix element_size to return correct element size instead
of number of pages. Because of that elements in
page_pools had below sizes:

  order 0 = 4096
  order 1 = 2
  order 2 = 4
  ....
  order 17 = 131072

This made possible to have elements with sizes 2,4,8,...
bytes that have been allocated with OBD_VMALLOC.
Now all sizes are miltiple of PAGE_SIZE:

  order 0 = 4096
  order 1 = 8192
  ...
  order 5 = 131072

Change the logic around ppp_idle_idx.

1. Recalculate ppp_idle_idx in __sptlrpc_pool_put_pages the
   same way as it done in __sptlrpc_pool_get_pages. It is
   possible that ppp_idle_idx might become 0 or very small
   (less than 10). If there is no new allocations it would
   be impossible to free anything from the pool during 40
   seconds(CACHE_QUIESCENT_PERIOD) despite the large amount
   of free elements ready to shrinking.
2. In pool_shrink_count when ppp_idle_idx == IDLE_IDX_MAX,
   it means no one accessed pool for CACHE_QUIESCENT_PERIOD
   (40 seconds), i.e. we may release as much as possible
   pages. Earlier in a such case pool_shrink_count always
   returned 0. Another words it was impossible to shrink
   anything from the pool if there was no allocations for 40s.

Add sanity-compr 2000 that reproduces the OOM problem on
a client without this patch.

DDN-bug-id: EX-10094
Was-Change-Id: I6b55ba67b0d21cdffdb57034e8e66063745f796e

osc: Add BRW_COMPRESSED flag to reads

We need to add the BRW_COMPRESSED flag to reads so servers
can know if the client is able to decompress data.

This lets servers decide if a client can be sent compressed
data and the result won't be nonsense/corruption.  This is
important for future support of GPU direct, where the
server will need to do the decompression.

DDN-bug-id: EX-6269
Was-Change-Id: I36b5b73f983ce8f2e5297c3e9dc778a5eca54e6a

osc: don't discard decompress_request error

The error handling for decompress_request is unusual - non-zero
returns are just discarded. And rc2 is just discarded.
The read() doesn't fail or get a short read.

Fix this so if decompression fails with an error.

DDN-bug-id: EX-7807
Was-Change-Id: Idd01947c7375c9586a64f064dd6ee0ac2308ea86

osc: save compressed object size

CSDC uses a sparse file feature. A client writes compressed data
chunks to the original offsets so the same data is expected to be
read from the same offsets.

There are no writes after the last compressed chunk, so no "hole"
after the last compressed chunk.

Compressed file size (based on OST objects size) is smaller than
the original on the "original last chunk size - compressed last
chunk size" delta.

Object size should be set to uncompressed size. This size is used to
calculate file size and to remove the workaround of not compressing
the last chunk in the file.

DDN-bug-id: EX-7600
Was-Change-Id: I387c282e1cf788c3b8f6230ef555d73ffffe49c1

osd: save compressed object size on zfs

"osc: save compressed object size" added means to transfer
object size to the osd and added ldiskfs support.

This patch adds saving objec size to the ZFS backend.
Currently this fix submitted as separete patch, for
testing purpouse, but can be marged to the main patch latter.

DDN-bug-id: EX-7600
Was-Change-Id: I99e29e3f756a070b5f3cece12c4ca58f668a2ecf

ldiskfs: fix detection of compressed extent

The code in ldiskfs_map_inode_pages which detects a
compressed extent checks the first lnb for that extent, but
it's possible for some lnbs and not others to be compressed
in a given extent, so we must check all of them.  This
occurs when multiple writes have been combined in to one
RPC.

If we don't detect compression correctly, we won't set the
file size correctly and we'll get data corruption.

DDN-bug-id: EX-7601
Was-Change-Id: I11d50bdc45c40d93bb1b829fcd930165b7626432

llite: DIO fallback on compressed files

Fully supporting direct I/O on compressed files is tricky
because we cannot pull the full chunk in to the page cache
(because there is no page cache for DIO).

So instead we fall back to buffered I/O for DIO on
compressed files.

This patch adds the check and a test for this.

llite: allow aligned DIO with compression

DDN-bug-id: EX-6127
DDN-bug-id: EX-7601
Was-Change-Id: I8224ef9b8ad1d912d8a11eccad37d3dff8dd8498

If a DIO is fully aligned to compression chunk boundaries,
it is safe to do on a compressed file, so allow it.

Was-Change-Id: If8fa3397c9424254538738f4d77f9f50d1c21129

lov: set IO property check in lov_io_init_composite

Move compression IO check down to lov_io_init_composite().

DDN-bug-id: EX-9853
Was-Change-Id: I6d4ce3388d8eef788d31dabf4b3a5dd5eff5091f

llite: DIO fallback on compressed files

Direct I/O on compressed files is still not perfect for now,
so we'd fall back to buffered I/O for DIO on compressed files.

DDN-bug-id: EX-6266
Was-Change-Id: I2255638e4fbde766abf061989d07b37ad9f369a1

llite: Compute compression chunk ranges

Determine the edges of any leading and trailing compression
chunks touched by this IO and store them in the cl_io
struct.

The functionality in this patch also allows us to adjust
the lock and read rounding to do them more intelligently,
this will be done in a future patch.

DDN-bug-id: EX-7601
Was-Change-Id: I526563ea347fb0246f97f3532b823c4345c3fa27

llite: round LDLM lock requests to chunk

When we do IO with compression, we may need to 'fill' the
compression chunk, reading up pages which have already been
written to storage, so we can compress the whole chunk.

Doing this safely requires that any dlmlock we're using
always covers the full chunk.

The easiest way to do this is to round the entire locking
process to include leading or trailing compression chunks.

DDN-bug-id: EX-7601
Was-Change-Id: I3c365844561d0da909e6290f4b58ef2211c2d255

csdc: is_chunk_start should return header copy

In is_chunk_start()

*ret_header = header;
...
kunmap_atomic(header);

ret_header is used after is_chunk_start(). The header
copy should be returned from is_chunk_start() for safe work.

DDN-bug-id: EX-9878
Was-Change-Id: Ib5e828d6b61e90dcd70c28589931a4490cf19c22

Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9b41ab815db3df9ad7bdea5fca4c093cbda8814b

66616 master J !B ✗T –R ✗

LU-10026 osd: osd_bufs_put does not always handle all pages

parked by your own -1

CR -1 csdc XS +0 −2

66d

janitor run: 16 failing config(s), none unique to this patch — janitor results

build #130323 FAILURE

reviews: Marc Vef -1 2026-08-26 09:47

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

your previous vote: -1 on PS1 (2026-06-15 10:19) — now at PS3

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osd: osd_bufs_put does not always handle all pages

osd_bufs_put asserts that the dio pages used after are
always zero, but there's no reason for this to be true and
compression specifically violates this by using 1 page at
a time.

Without this patch, we hit this assert and crash when
nonrotational = 1.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: If6bdb11f254c260e2da4cabe11a82693a468e6fb

66275 master J !B ✗T –R ✗

LU-10026 osc: do not compress resends

parked by your own -1

CR -1 csdc 2 unresolved S +30 −2

66d

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #130266 FAILURE

reviews: Marc Vef -1 2026-08-25 21:12

no vote yet: Artem Blagodarenko

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

your previous vote: -1 on PS3 (2026-06-12 10:19) — now at PS4

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS3 2026-06-12 10:19 Marc Vef COMMIT_MSG:15 — the reason is now known and should be included here. Note for me to update later

Marc Vef PS3 · 2026-06-12 10:19

the reason is now known and should be included here. Note for me to update later

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre osc_request.c:2604 — (defect) The resend is rebuilt from the uncompressed arrays, so besides aa_requested_nob the niobuf/page counts also change (compression yie

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) The resend is rebuilt from the uncompressed arrays, so besides aa_requested_nob the niobuf/page counts also change (compression yields fewer niobufs), yet only aa_requested_nob is propagated back into `aa` before `new_req->rq_async_args = request->rq_async_args` overwrites new_aa. After the resend new_aa->aa_nio_count still holds the compressed count, while the reply's RMF_RCS vector is sized from the uncompressed `niocount` in osc_brw_prep_request().

    check_write_rcs(req, aa->aa_requested_nob, aa->aa_nio_count, ...)

check_write_rcs() then only inspects the first aa_nio_count per-niobuf rcs, so a server-side write error on a niobuf past the compressed count would be skipped. aa_page_count is similarly stale. Should aa_nio_count (and aa_page_count) be updated here the same way as aa_requested_nob?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: do not compress resends

There's some issue with doing compression on resent
requests, so this patch works around it with two things:
1. Use the uncompressed page array for resend
(this was always necessary unless we modified resend to
know it already had compressed pages as input)
2. Disable compression on resend (not clear why 1. wasn't
enough)

csdc: Update async_args after resend

It is decided to send an uncompressed request on redo.
osc_brw_prep_request() processes uncompressed data and prepares
a request, so some parts of the old request are outdated.

Let's update the old request with information from the new one.

DDN-bug-id: EX-8814
Was-Change-Id: Idb1c6ee9db64cb1f2ea1c1562b1c5aae443263e3

Test-Parameters: forjanitoronly
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I5fbbdc2771f8c2c7b5c28f0b70d89b8b6015147f

66613 master J ✓B ✗T –R ✗

LU-10026 tests: change sanity-compr 2000

parked by your own -1

CR -1 csdc 2 unresolved XS +8 −1

67d

build #130321 FAILURE

reviews: Marc Vef -1 2026-08-26 09:47

no vote yet: Sergey Cheremencev

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

your previous vote: -1 on PS1 (2026-06-12 09:25) — now at PS3

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-07-15 00:58 Gerrit AI review for Lustre COMMIT_MSG:15 — (minor) The Fixes: commit only touched lustre/obdclass/page_pools.c and never added this test. subtest_2000() and the dd count=5000 that thi

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) The Fixes: commit only touched lustre/obdclass/page_pools.c and never added this test. subtest_2000() and the dd count=5000 that this patch tunes down were both introduced by 452599da95 ("LU-10026 osc: osc brw request compression", Gerrit 65770). If the patch stays standalone, should Fixes: point at 452599da95 instead? (This also lines up with the existing suggestion to fold it into 65770, which would drop the tag entirely.)

reply PS1 2026-07-15 00:58 Gerrit AI review for Lustre sanity-compr.sh:232 — (minor) Both echoes print the same "count $count" label, one before and one after the clamp, so a log reader can't tell the raw 70%-of-RAM v

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) Both echoes print the same "count $count" label, one before and one after the clamp, so a log reader can't tell the raw 70%-of-RAM value from the clamped one. Consider labelling them differently or dropping the first.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: change sanity-compr 2000

Make dd write 70% of RAM but less than 5GB.
The test case is too agressive and sometimes
causes OOM as client have to preserve
compressed pages for OST_WRITE replay(see
EX-9895 for details).

Fixes: 14fc034e983e ("LU-18086 obdclass: page_pools shrinker fix")
Test-Parameters: forjanitoronly
Test-Parameters: trivial testlist=sanity-compr env=ONLY=2000,ONLY_REPEAT=10
Test-Parameters: trivial env=ONLY=2000,SLOW=yes testlist=sanity-compr
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I594b963ca358c5adbf077bcd59378dd13506ccad

66614 master J ✓B ✗T –R ✗

LU-10026 tests: skip sanity-compr for old servers

parked by your own -1

CR -1 csdc XS +2 −0

67d

build #130324 FAILURE

reviews: Marc Vef -1 2026-08-26 09:47

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

your previous vote: -1 on PS1 (2026-06-12 09:24) — now at PS3

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip sanity-compr for old servers

Skip for old servers.

Test-Parameters: forjanitoronly
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia42f1f32fe166bae84a7c38c02d6a218303cc51e

▶

csdc

(99)

68347 master J –B ✗T –R ✗

LU-10026 ofd: csdc final integration patch

parked by your own -1

CR -1 csdc 5 unresolved XL +3772 −421

build #130325 FAILURE

reviews: Marc Vef -1 2026-08-27 20:12

no vote yet: Sergey Cheremencev, Sebastien Buisson, Zhenyu Xu, Alex Zhuravlev, Hongchao Zhang, Li Dongyang

patchsets: PS1 08-25 PS2 08-26

threads: 3 need your reply · 0 waiting on others · 2 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes for this upstream submission (large squash; adapted to current
master):

- The build-file hunk was retargeted: the tree no longer has per-directory
  autotools files (LU-18876), so ofd_compress.o is added to the kbuild
  lustre/ofd/Makefile; the EXTRA_DIST hunk has no equivalent and was
  dropped.
- LU-13139 moved the dt_bufs_get()/dt_read_prep()/dt_write_prep() bodies
  out of dt_object.h into obdclass/dt_object.c; this patch's signature
  changes (offset/len args, decompression/read flags) were applied to the
  relocated bodies, with the prototypes and the new dt_unlock_pages()
  wrapper in the header.
- OBD_FAIL_OST_BRW_WRITE_BULK3 renumbered 0x256 -> 0x258: 0x256 is already
  OBD_FAIL_OST_GRANT_PREPARE on master.
- An m4 call to LC_FORCE_UACCESS_BEGIN was dropped: the macro is not
  defined anywhere (neither in this patch nor in the tree). The functional
  m4 content (LC_VM_MAP_RAM_PROT_ARG probe for the 4-arg vm_map_ram()) is
  kept.
- The compression level-mapping helpers and compr_type_table were moved
  from lustreapi.h into the lustre_user.h UAPI header (as this patch
  intends, so server code can share them), based on the current series'
  versions of those helpers; the table gained the LL_COMPR_TYPE_MAX
  sentinel and iteration skips NULL-name entries. compress_type_2str()/
  mapback_compress_level() are static inlines there now; the lfs find
  loops live in liblustreapi_pfind.c and were converted there.
- Adapted to the reworked struct ll_compr_hdr (65769): split magic words
  and the packed level_and_chunk byte with its accessors, in ofd_compress.c
  and the shared compression library. is_chunk_start() takes a folio in
  this series; the OFD call sites pass page_folio(lnb_page).
- The chunk-bits fields follow the series' *_lum_bits naming and offset
  semantics; llc_compr_chunk_lum_bits already stores the offset above
  COMPR_CHUNK_MIN_BITS, so the filter-fid assignment in lod_object.c is
  direct, without re-subtracting COMPR_CHUNK_MIN_BITS.
- ptlrpc_connection_is_local() does not exist upstream - replaced with the
  LNetIsPeerLocal(&exp->exp_connection->c_peer.nid) form used by the
  surrounding OFD code.
- osd_write_prep(): the partial-page zeroing block had been converted to
  kmap_local_page()/folio_index_page() upstream; the patch's changes
  (PageUptodate skip, read flag, whole-page memset) were applied on the
  current form.
- The preview-period client-side restriction on non-chunk-aligned
  overwrites (vvp_io_write_start) is removed as in the original patch.
- unmerge_chunk(): the new tail-clearing loop indexed lnb[i] while its loop
  condition and debug message used first + i; the stores now use
  lnb[first + i] consistently.
- Old version gates carried by the squashed test additions (sanity-compr
  1004-1006, sanity-lfsck 18i, test_460e area) are meaningless on master
  and were converted to the compression_supported() helper.
- A Fixes: line pointing at one of this patch's own squashed sub-patches
  was dropped; a file for an external repository (lipe) was dropped.

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:663 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'rc + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'rc + maxlnb' 'unit_page + unit_array_size'

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:667 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'rc + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'rc + maxlnb' 'unit_page + unit_array_size'

note PS2 2026-08-27 20:12 Marc Vef ofd_compress.c:975 — (defect) range lock must be taken before `ofd_map_chunks_to_lnb_write()`

Marc Vef PS2 · 2026-08-27 20:12

(defect) range lock must be taken before `ofd_map_chunks_to_lnb_write()`

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:1048 — warn: ofd_object_punch_compr():duplicate check 'orig_lnb' (previous on line 957)

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

warn: ofd_object_punch_compr():duplicate check 'orig_lnb' (previous on line 957)

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: csdc final integration patch

Add calls to the csdc functions from the regular read/write paths.
This also brings OFD range locking back for compressed objects,
reverting commit a21ce928aa64 ("LU-15894 ofd: revert range locking
in ofd") for this use case.

ofd: rename 'local' in thread_big_cache

It's not a big deal since it's only used a few times, but
let's give this variable a descriptive name.

DDN-bug-id: EX-7601
Was-Change-Id: Ide136cd42e885d59f1a2e4ce22a2e7449faca3f9

ofd: rename map_remote_to_local

osd_map_remote_to_local implies some complex role, but in
fact what this does is initialize the fields of the
local niobuf structs to represent the requested range.

This *may* be the same as a remote niobuf, but it also
isn't in some cases.  Name it osd_init_lnbs instead.

DDN-bug-id: EX-7601
Was-Change-Id: I0d3b5f24e42ee8dc962437daea7cf9347ccb9059

ofd: series description and reorder declarations

Reorder declarations in tgt_brw_read prior to adding things.

This trivial patch is a good place to put the description
of this series, which handles unaligned reads to compressed
files.

-----------------------
These patches handle compression chunk unaligned reads on
the server.

When using compression, the client attempts to send chunk
aligned reads, but sometimes it can't, and the client will
send a read to the server which is not chunk aligned.

In this case, the server must read the full chunk,
decompress it, and provide the requested data to the client.

Here's how we do this.

The server receives a set of remote niobufs describing IO
from the client.  Each remote niobuf (rnb) describes a range
of data the client wants to do IO to.

These are translated to a set of local niobufs on the
server, which we then use to do the read.  For compression,
the server has to read complete chunks on unalinged reads.

So we walk these remote niobufs and identify unaligned read
requests (in ofd_preprw_read), then round them to chunk
size. The server then reads the chunk rounded read request
from storage.

The local niobufs now contain a set of complete compressed
chunks, ie, the raw data from disk.  We need to decompress
the chunks where the client is doing an unaligned read, but
leave the other chunks compressed (because the client will
uncompress them).

So, in obd_decompress_read, we use the remote niobuf to
identify unaligned reads from the client.  We then walk the
local niobufs, identify the chunks which match the unaligned
reads from the client, and decompress them 'in place'.
The decompression uses temporary buffers, but the
decompressed data is placed back in the local niobuf.
(If the data is uncompressed on disk, we of course do not
decompress it.  This happens for incompressible data.)

Now the local niobuf contains some raw chunks and some
chunks which have been decompressed.  This is *more* data
than the client asked for.  Normally, the server local
niobuf contains exactly what the client asked for, so the
server checksums and sends the entire local niobuf.  But
because we read complete chunks, the local niobuf contains
more data than the client requested.

This means we need to identify the subset of the local
niobuf which the client actually wants to read and present
that to the client.

In order to do that, we walk the local niobuf and use the
remote niobufs (the description of the pages the client
needs) and create a special tx niobuf which points to only
the pages the client wants (io_lnb_to_tx_lnb).  Then we use
this tx niobuf for checksum and transfer to the client.

DDN-bug-id: EX-7601
Was-Change-Id: Ic89dcef7e169879725caa6cdef4619b9a76b2b37

ofd: identify reads to round

If the beginning or end of a client read is unaligned, we
must round the locking.  This patch identifies reads where
this is required, the next patch will do the locking.

Print a debug message when such an IO is found, but don't
do anything different - yet.

DDN-bug-id: EX-7601
Was-Change-Id: Ibdab35b733225b4b1349ef457f66ca37dcb2d9bf

ofd: round read lock to chunk

For unaligned reads, we need to round the read locking to
cover the any leading or trailing chunks.  We do this by
creating a local 'remote niobuf' to describe the rounded
range and doing the locking against that niobuf.

DDN-bug-id: EX-7601
Was-Change-Id: I8818522c188aca3c5c5eb564da2a8ba8aef18a4b

ofd: convert dt_bufs_get to offset, len

dt_bufs_get takes a remote niobuf, but just uses the
offset and length for getting pages.

Compression requires rounding the local IO to include the
full compression chunk, which means the local IO does not
match the remote niobuf any more.

So we modify dt_bufs_get to take an offset and length
rather than a remote niobuf, so we can ask for the pages we
need.

DDN-bug-id: EX-7601
Was-Change-Id: I4beaf8207fa00d802c0a339df3de2a3c71154fc7

ofd: add chunk_size to preprw_read

preprw_read needs chunk size for rounding.  Add this in a
separate patch to keep things trivial, it will be used in
a subsequent patch.

Also use this to add a check in DOM to ensure it doesn't
attempt to do compression.  This should already be
prevented by setstripe, so this is just an extra safety
check.

DDN-bug-id: EX-7601
Was-Change-Id: I9dc4d1559e5c8be315268a593466571b54c90a96

ofd: add remote_pages

When we round a read to get all of the compressed chunks,
the number of local and the number of remote pages will
differ.  We need to make sure we do the checksum and data
transfer using the number of remote pages, not the number of
local pages.

This patch calculates the number of remote pages and uses it
accordingly.  This doesn't do anything yet, but when we
round the local read to include the whole compressed chunk,
this will be needed.

DDN-bug-id: EX-7601
Was-Change-Id: I4875b02016570d227b3b926efd117f0a7cda41b4

tgt: add second tbc lnb for RDMA

Compression requires the server to do local IO which differs
from the IO requested by the client.  This means we cannot
directly use the IO niobufs for doing the transfer to the
client.

So we add a second set of lnb pointers, which are used to
point at a specific subset of the pages in the main
per-thread cache.  This subset will be used for doing the
transfer to the client.

DDN-bug-id: EX-7601
Was-Change-Id: I53aa46045aaf335da20a311900ac0bf425823b22

tgt: add io_lnb_to_tx_lnb

With compression, the lnbs used for the disk IO on the
server can contain more data than the client requested,
due to reading up whole chunks for decompression.

This means we need to transfer only a subset of the lnbs.
We do this by creating a second set of lnbs, and pointing
them at the pages in the local io lnb which need to be
transferred to the client.

This code doesn't do anything for now, but it will kick in
with the next patch when we start rounding chunks for read.

DDN-bug-id: EX-7601
Was-Change-Id: I0fe690718a3484578b139eaaec52c0c3b265da6a

ofd: add chunk rounding to read

We need to round all niobufs to chunk size in the read
process, so we read in the full chunk.

dt_bufs_get sets up the local niobuf for the read, so we
round before calling it.

This patch is a partial implementation of unaligned read
support, and breaks compression testing until the next few
patches are landed.  So this patch temporarily adds the
compression tests to ALWAYS_EXCEPT.

DDN-bug-id: EX-7601
Was-Change-Id: I587a519db4dae983db5db1d690e63e15bc010b7e

ofd: add chunk_size to preprw_write

preprw_write needs chunk size for rounding.  Add this in a
separate patch to keep things trivial, it will be used in
a subsequent patch.

This patch is really trivial on the write side, since the
read side already did most of this.  But it's being kept
separate for symmetry.

DDN-bug-id: EX-7601
Was-Change-Id: Id25957dbc185b6e61b7f208cee8cf5f897f03944

ofd: do not use page cache for compressed files

It is challenging for the server to safely use the page
cache with compressed files, because if data is
decompressed in to the page cache, the data in cache now
differs from the data on disk.

This is a problem if *part* of the page cache is ever
evicted, because we can end up with a situation where a read
will be partially satisfied from cache and partially from
disk, but the data on disk is compressed and the data in
cache is not.

It is possible to deal with this by carefully ensuring the
page cache is not used just for decompressed data, but this
makes getting the buffers/lnbs for compressed files fairly
complicated.  Instead, we can just entirely block using the
server page cache for compressed files.

This must be done for both read and write, and only works
for ldiskfs - ZFS cannot easily be forced to not use its
page cache.  But that's OK because we do not support CSDC
with ZFS.

DDN-bug-id: EX-7601
Was-Change-Id: Iee73abb29ad5631bb2203c2133756d7ebf5b686d

ofd: add ofd_decompress_read implementation

ofd_decompress_read is responsible for walking the
remote niobufs (rnbs) in the RPC and identifying if they
are chunk unaligned.  It then passes them on to the rnb
decompression code (not implemented yet, see next patch).

It also allocates the bounce buffers for decompression so
they can be reused for each remote niobuf.

DDN-bug-id: EX-7601
Was-Change-Id: I1f2f86ce3fc036ac5d79b060a5e44f6564e123aa

ofd: lock pages from read to decompression

When using the page cache on the server, for pages which
will be decompressed, we can't unlock them until they've
been decompressed.

Rather than only waiting to unlock the pages which will be
decompressed, we keep all of the read pages locked.  This
simplifies the code, at the cost of delaying other reads to
the aligned portion of an unaligned read.  ie, shouldn't be
important in practice.

DDN-bug-id: EX-7601
Was-Change-Id: Ie98920327979a5c9600e8c9e8627816461ea1a34

ofd: (ofd part)add obd level compression lib

Some compression functions will be used by several areas of
of Lustre, so they need to be in obdclass.

This moves merge_chunk and unmerge_chunk there and adds the
ability for them to merge lnbs.  This is used in a future
patch.

DDN-bug-id: EX-7601
Was-Change-Id: If4a318119bb7685e41adb9f3b31a66074031e6ac

ofd: make compress_chunk take chunk_bits

Chunk bits is used everywhere, have compress_chunk convert
to log bits rather than have the callers do it.

Was-Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>

ofd: add decompress_rnb implementation

This implements decompress_rnb, which is the core code for
handling unaligned reads from the client.

Decompress rnb takes an unaligned remote niobuf and
identifies the unaligned portion(s) of the IO, then finds
the corresponding local niobufs (pages read from disk),
and passes them on for decompression in place.

decompress_chunk_in_lnb decompresses the data in a set of
lnbs and copies it back to the same location, replacing the
raw data from disk with decompressed data.  (If the chunk
was not compressed, it does nothing.)

With this patch, the implementation of unaligned reads is
complete and we can add the compression sanity tests back
safely.

DDN-bug-id: EX-7601
Was-Change-Id: Ifd1d9b03d5d004bec3f5e456da359b8d10e005f9

ofd: improve decompress_rnb debug

Since we're very close on landing the unaligned read
patches, this minor debug improvement is being placed later
in the series.

DDN-bug-id: EX-7601
Was-Change-Id: I36ad243bd1f7025e358f9593f1008f0b851cc1bb

tgt: reorder tgt_brw_write decls

Reorder the declarations in tgt_brw_write.

This patch also serves as the series head for implementing
read-modify-write support for compressed chunks.

The process for read-modify-write is similar to that used
for unaligned reads.

At a high level, read-modify-write means we must read up,
decompress, then recompress and write back the data.  This
only applies when we're actually doing read-modify-write.

To know when to do this, we rely partly on the client.  If
the client is able to compress a chunk, either because it is
a complete chunk, or because the start is chunk aligned and
the write is past EOF, we know there is no read-modify-write
required.  Either there is no existing data (write past EOF)
or the data will be fully replaced.

So, when we see a write which is not fully chunk aligned and
not already compressed, we will do a read-modify-write.

For this, we round the IO lnbs and associated locking to
cover complete chunks, then we do a read of the unaligned
chunks.

ie, if we have a write which goes from 63 KiB to 257 KiB
with a chunk size of 64 KiB, we will read 0-64 KiB and
256-320 KiB, and decompress those chunks in to the buffer.
64 KiB to 256 KiB is *NOT* read, because those are complete
chunks.

We then set up a transfer mapping - identical to the process
for unaligned reads - so the client data is written in to
the correct lnbs.

Now we have a set of chunk aligned lnbs which contain data
updated with the client write.  In the initial version, we
write these to disk uncompressed.  This is sufficient for
correct operation, but it does mean read-modify-write will
decompress those chunks.

There is code for recompression, but it is not working 100%
yet, and there are some complexities around managing holes
and EOF which still need to be resolved.

TBD if this will make our initial release - I am hopeful but
not sure yet.

DDN-bug-id: EX-7601
Was-Change-Id: Ia24583d4221f498928e99afa8c289b70e4d25f5b

tests: unaligned read tests

This adds testing for handling unaligned reads and partial
chunk reads from compressed files.

Testing for writes and multi-client and racing tests will
be added later, but we put the checking function in
test-framework now so it's easy to use later.

DDN-bug-id: EX-7601
Was-Change-Id: I06217c8aeba75016aa4168f329026842dff1d979

obd: add error check in merge_chunk

If the lnbs we're trying to merge have an error recorded in
them, then they're not going to be valid input for
decompression, so return an error.

DDN-bug-id: EX-7601
Was-Change-Id: I1bf17131cb65106087eb5e72e2700db30c0cc975

obd: do not decompress empty lnbs

For reads which cross EOF, we may get lnbs with no data in
them (similarly for writes which cross EOF).

For these cases, it's important to only copy from the lnbs
where there is data, and only do decompression on the lnbs
if there's actually data in them.

Modify merge chunk to do this.

DDN-bug-id: EX-7601
Was-Change-Id: I83fefcfa6d1396dcd97fad994334bf29438bb4bf

ofd: clear pages in decompression

Handling writes to compressed files requires a
read-modify-write cycle, which has implications for how we
handle reads.

Consider the case of a file with an 8 KiB write at offset 0,
which is compressed to 4 KiB.  Then there is another 4 KiB
write at offset 16 KiB.

Updating this correctly requires reading the first chunk,
then decompressing it.  However, this read will go past
EOF, because the write has not occurred yet.  The OSD read
code does not fill in these pages, because read past EOF is
not returned to the client (client gets a short read and
does not actually use the pages).

In our case, however, we must use these pages (from 8 KiB
16 KiB).  In the naive version without recompression, we
simply write out 0 - 16 KiB, so we must have zeroes in
those pages, and once we have recompression, we must
compress those pages so we need zeroes in that case too.

So we note if a page has data in it after decompression,
then if it does not, we clear the page.  Note we do NOT set
lnb_rc to 0 when we clear a page, because lnb_rc = 0 is
used to indicate EOF rather than a gap in the file.

DDN-bug-id: EX-7601
Was-Change-Id: If1d1360185eb087e821167a08e49c9427e29ffc4

tgt: identify writes to round

If the beginning or end of a client write is unaligned, we
must round the locking.  This patch identifies writes where
this is required, the next patch will do the locking.

DDN-bug-id: EX-7601
Was-Change-Id: Iec140c24423a0da478f6d42ff6fc620d7ad3ba4a

tgt: round write lock to chunk

For unaligned writes, we need to round the write locking to
cover the any leading or trailing chunks.  We do this by
creating a local 'remote niobuf' to describe the rounded
range and doing the locking against that niobuf.

DDN-bug-id: EX-7601
Was-Change-Id: I2bdea620386ad229375647a0e2cc6180c9bd7aa6

tgt: add remote_pages for writes

When we round a write to get all of the compressed chunks,
the number of local and the number of remote pages will
differ.  We need to make sure we do the checksum and data
transfer using the number of remote pages, not the number of
local pages.

This patch calculates the number of remote pages and uses it
accordingly.

Note that just like on the read side, this patch doesn't do
anything until we're actually rounding the chunks for IO in
a later patch.

DDN-bug-id: EX-7601
Was-Change-Id: I38256070d68246613ce67b0bfe328f6443a95533

csdc: fix ofd_preprw_write() for sanity 819b test

Sanity 819b asserts:
tgt_brw_write()) ASSERTION( npages_local == npages_remote )

The test triggers fault inject in ofd_preprw_write():
if (OBD_FAIL_CHECK(OBD_FAIL_OST_2BIG_NIOBUF))
     rnb[i].rnb_len += PAGE_SIZE;

ofd_preprw_write() calculates npages_local taking in account
additional len from fault inject, BUT npages_remote is calulated
BEFORE the fault inject. So npages_remote was not adjusted.

To solve the problem it is enough to move range_to_page_count() call.

DDN-bug-id: EX-9523
Was-Change-Id: Ifd659985a78c7630049a17622aff2eb7f4525fb1

tgt: add tx lnb for writes

With compression, the lnbs used for the disk IO on the
server can contain more data than the client requested,
due to reading up whole chunks for decompression.

This means the client is only going to write data in to
a subset of the lnbs used for io to storage.

We handle this the same way we do for reads:
We create a second set of lnbs just for the transfer, and
point these lnbs at the pages which will actually receive
data from the client.

DDN-bug-id: EX-7601
Was-Change-Id: I5b668547537698309792daf309842866be79f0b6

ofd: switch preprw to chunk_bits

The compression/decompression code requires chunk_bits
rather than chunk size.  Since we need to call this code
from ofd_preprw_write, we need chunk_bits there.

This modifies the functions so chunk_bits is available
there.

DDN-bug-id: EX-7601
Was-Change-Id: Id98e6d6364eeaaa7753a8aba059387e3e659d2a2

vvp: remove unaligned write restriction

This series will resolve the unaligned write issue for
compressed files, so we need to remove the restriction on
unaligned writes in order to test it.

This does not mean unaligned writes are working yet, but we
need to make this change so the subsequent patches can be
tested.

DDN-bug-id: EX-7601
Was-Change-Id: I4abcbdcd18b00718099483c8dfdb9a7aa41c3ce7

ofd: add chunk rounding to write

For compressed files, we need to round all niobufs to
chunk size in the write process, so we have buffers for
reading in and rewriting the complete chunks.

dt_bufs_get sets up the local niobuf for the write, so we
round before calling it.

Note this breaks writing to compressed files, which is not
fixed until a few patches later.  For this reason, we
disable the compression tests.  They will be reenabled
shortly - similar to how we handled the read series.

DDN-bug-id: EX-7601
Was-Change-Id: I413aaba9866dd7d6c4463fa620eadf1423379ba1

ofd: add read to write process

This adds a very simple read to the write process, which
just reads up the entire chunk-rounded write range.

This is a first step - the read will eventually be modified
to only read the unaligned portions which must be
decompressed for read-modify-write.  We will create a
special lnb mapping which contains only the pages which must
be read for decompression (similar to the tx lnb mapping).

For now, this read allows us to test decompression without
handling the mapping.

DDN-bug-id: EX-7601
Was-Change-Id: I169ddc2e161094aebdad1a60ec62e9c1d75cd6d8

ofd: add decompress_read to ofd_preprw_write

We have read up the compressed data from disk, now we must
decompress it so we can rewrite it successfully.

This code still works on the whole lnbs rather than just on
the portion of it which is unaligned.  This is temporary
and will be resolved by a future patch.

With this patch, we have basic read-modify-write support,
so we can re-enable testing.  The next patch adds tests
for read-modify-write.

DDN-bug-id: EX-7601
Was-Change-Id: Ib6503c15e9fb3d425a7bc295bcc61b41c089a1f0

ofd: add read lnb to ofd_preprw_write

The read phase of read-modify-write for compressed files
needs to read only a subset of the pages which will be
written, so it needs a separate set of lnb pointers for
tracking this subset.

This patch passes around the necessary argument but does
not set up or use the lnb yet.

DDN-bug-id: EX-7601
Was-Change-Id: I7ec7101e65e73a6c9e67cea3c58d8cace38e70e0

ofd: distinguish nr_write and nr_read

We will have two counts of pages in lnbs, distinguish
between them.

Not actually used yet - will be calculated when the read
lnb mapping is created.

DDN-bug-id: EX-7601
Was-Change-Id: I709b8fd299163d348a196184152bb0294fcb650b

ofd: create read mapping for read-modify-write

When we need to do a read-modify-write for unaligned writes
to a compressed file, it's important we read only the
portion of the file which is receiving unaligned IO.

This patch identifies these chunks in preprw_write and
creates a read lnb mapping from a subset of the pages for
write.  These pages we read up are then decompressed.

Note one issue this patch does not address is reading of
data past EOF.  If the final chunk is unaligned, we will
round the write to cover it.  This results in extending the
file inappropriately, writing zeroes where they aren't
needed.  The read side gives us the info to address this,
which we will do in a future patch.

DDN-bug-id: EX-7601
Was-Change-Id: Iede43f12127cbb93e73c22a915192aa2f814a927

osd: add assert for prepare partial page

In the write prep code, we read up any partial pages (pages
which are not completely overwritten by the write) to
prepare them for write.  But for compressed files, we will
have already done this to prepare for decompression.

Add an assert to make sure we catch if this is ever wrong.

DDN-bug-id: EX-7601
Was-Change-Id: I1366b1f5b191a4d581448d692933d562198c3a1f

ofd: minor debug improvements

A smattering of minor debug improvements across several
patches, placed at the end because they're all minor and
some of them would disturb early parts of the series.

DDN-bug-id: EX-7601
Was-Change-Id: I2071911eb09f5c7fad28203db05396bb31ccda59

ofd: set compressed file size for fake writes

When using the fake writes fail_loc, file size setting is
done at the ofd layer, since the osd layer isn't used.  So
we must also handle the compressed file size for this case.

This fixes sanity test 399a with compression.

DDN-bug-id: EX-8826
Was-Change-Id: Icda612405908166d043e1e568d0d8bd9cd0c5156

tgt: objcount in RPC must be 1

Much of the BRW write code assumes objcount is one, but
there is some provision for multiple objects.

Since the code will break if we send it multiple objects,
add errors to make sure anyone changing it will notice.

This isn't strictly compression related, but compression
adds even more code which assumes this, so this protection
will be useful.

DDN-bug-id: EX-7601
Was-Change-Id: Idcbf33fd14d4b1bd179c9516bed07cca907008bc

ofd: multiple reads in same chunk

When doing DIO or if we get unusual cache behavior on the
client, multiple reads can hit the same chunk.

This only shows up in racing tests, but it's important to
handle.  We do this by making sure we start searching the
lnbs for decompression at the start of the last chunk we
decompressed.

DDN-bug-id: EX-7601
Was-Change-Id: I81fbbba79b16066e6d4519c66030cc58e03d2de7

ofd: don't read for writes past eof

There's no data past EOF, so there's no need to do
read-modify-writes when the entire write is past the chunk
at EOF.  So in that case, don't read up data and don't
attempt decompression.

There's no explicit test for this, but this shows up
immediately in the random-offset copy tests, because they
seek and write various sizes to offsets past current EOF.

We also need this functionality for reads, because in some
cases the client will do reads past EOF (this is unusual,
but can still happen sometimes).  This is added in a
separate patch because it requires some code reorganization.

DDN-bug-id: EX-7601
Was-Change-Id: Ia2b598165d5645c5a44c3d58bea69c7e42f10e41

ofd: same-ify preprw_read and preprw_write

preprw_read and preprw_write have some sections which are
functionally the same but which have diverged slightly.
(These can't easily be shared between the functions.)

This is a short patch to make them more similar before
adding eof checking to reads.

DDN-bug-id: EX-7601
Was-Change-Id: I7bce912e99e61a4eec4060d6b49d4917894b44c4

ofd: put decompress_read in read prep

ofd_decompress_read is called from ofd_write_prep for
writes, but from tgt_brw_read for reads.  This makes the
code a little harder to follow and makes it difficult to
check read side decompression against EOF.

Instead, we move the decompression call to ofd_preprw_read.
This makes no change to the real operations here, but makes
for better code (and more similar code between read and
write).

DDN-bug-id: EX-7601
Was-Change-Id: Ibefd0a48ad08e83725f2df64618db60ba61c5ce0

ofd: add past eof check for reads

The client does not normally generate reads past EOF, but
this can occur during some racing situations.  We need to
check for that case and not attempt decompression, since
there's no data to decompress if we're reading past EOF.

This covers a failure which shows up occasionally in the
racing parts of the test suite, but it's challenging to
write an explicit test for this.

We also add handling for complete reads of the last chunk,
even if that chunk is partial, because we can send that to
the client for decompression.

This allows us to remove the slightly funky eof handling
in decompress_rnb, since we'll just not call that code in
this case now.  Note we'll still call decompress_rnb, etc,
for writes if they start before EOF and finish after EOF
(and are unaligned).  This is fine - this case should be
rare and if we hit it, we'll notice there's nothing to
decompress and proceed accordingly.

DDN-bug-id: EX-7601
Was-Change-Id: I50295f2803af611de5069d094c0a5d1b0a4a9c2d

ofd: add checks to io_lnb_to_tx_lnb

We should always be able to find the remote niobuf in the
local io range, if we can't, there's a bug.  So assert on
this.

We should also never have page level overlapping remote IOs,
at least until we have unaligned DIO.  (We can remove this
check when we combine the features.)

DDN-bug-id: EX-7601
Was-Change-Id: I325d4a37d25c116e42621964e90b225b71fd8f1f

tests: tests for read-modify-write

This patch adds tests for the read-modify-write case for
EX-7601.  There's still some additional tests to be added
here, but this is a good start.

DDN-bug-id: EX-7601
Was-Change-Id: I5dd9e566b8274ece99283c8962e0d34225089cc0

tests: use alternative data source for rewriting

Using the same file as input has disadvantages. It is not
possible to understand that data was not rewritten at all.
Alternative data source should be used.

Let's shift source file data and use it as a source.
To check rewriting result the same operarion is performed
on the destination file copy stored outside the Lustre FS.

DDN-bug-id: EX-8598
Was-Change-Id: I6ef400520359bfe9156c3f47e757064863bdf4e0

tests: add multi-mount compression test

This adds a multi-mount correctness test for compression.
This races IO from two mountpoints at varying sizes to
stress test compression.

DDN-bug-id: EX-7601
Was-Change-Id: If49cbd6d171068faa802835146f273d835b39bc3

obd: add 'lvl' for best and fast

'best' and 'fast' compression types must also set a level,
because not all levels are supported by all algorithms.

Rather than trying to be clever, just use simple universally
supported values, except for lz4fast, where we special case
this, because otherwise '0' is the slowest setting (and
lz4fast is likely to remain our default fastest).

DDN-bug-id: EX-6269
Was-Change-Id: I7c29659d4f027af2e44285ae38e4c9d91e35509a

ofd: use niocount consistently

'niocount' refers to the number of remote niobufs, ie, the
number of separate IOs from the client.  Except for a few
places, where it's used to refer to the number of pages in
the entire RPC.  Eek.

Replace this usage with 'npages', making niocount usage
consistent.

DDN-bug-id: EX-8993
Was-Change-Id: I266087ad8dccadb54c054b0a11fb03dc9868a725

csdc: allow specify 'fast'/'best' compression type

Use lctl set_param osc.*.compress_type_{fast|best}=<type>:<level>
to specify the compression_type:level for LL_COMPR_TYPE_FAST/
LL_COMPR_TYPE_BEST.

lctl get_param osc.*.compress_type_{fast|best} will list these
values.

DDN-bug-id: EX-8311
Was-Change-Id: Ifeff7f25e30fc0884f0c770a3b6d0798937b3c35

lfs: not iterate compr_type_table using ARRAY_SIZE

patch modifies compr_type_table to contain NULL fields in the
array, so iterate over the array should not use ARRAY_SIZE, but skip
those elements with NULL compression type name.

DDN-bug-id: EX-8311
Was-Change-Id: I8e4988fd3a63c1cb66f75510d190c2ebc4f8f9be

ofd: handle 'missing object' reads

When the read code (eg, mdt_preprw_read) finds there is no
object, it will return a read with 0 pages, but not fail the
read.  The assert for local and remote pages needs to
recognize this case.

DDN-bug-id: EX-8996
Was-Change-Id: Idc6ff70f71abc100f750a63eca73a754a56f6435

ofd: do not write 'hole' pages on compression

When doing unaligned read-modify-write to a compressed file,
we must round the IO lnb used for write in order to read up
the compressed data for modification.

In some cases, this creates a situation where there are
pages in the write lnb which have no data in them.  It is
important not to write out these pages, because if we do,
this wastes space and can cause incorrect file size.

In most cases, the file size is covered by the client
sending the file size, but if the client does not compress
a particular write, it does not send the size and the server
does not use it.  We could resolve this by having the client
always send size info and have the server always use it, but
it's better to make server writes 'hole' aware, since this
improves space usage.  (And this will be required for the
server to do recompression on read-modify-write, otherwise
no space is gained.)

DDN-bug-id: EX-8993
Was-Change-Id: I66169e205fe4691ed03b2c9b3005ffc4ecd3213d

obdclass: disable T10PI guard from disk when decompressing

When server handles chunk unaligned read, we need to read
the chunk from disk and decompress the raw data.
This means we should not use the guard tags from the storage,
as they only match the raw data.
Disable the guard from disk and the T10PI checksum will be
recalculated later using decompressed data.

DDN-bug-id: EX-9108
Was-Change-Id: I49964e1412e9d02769797c64aab290b17851e26f

ofd: take local chunk-aligned lock

On OST side to prevent racing read-modify-write against same
compressed chunk from the same client.

DDN-bug-id: EX-9192
Was-Change-Id: Iffaf2d2856e276cb2f9becce2506154314217e3c

csdc: Do not print error if a chunk is not compressed

is_chunk_start() can decide that a chunk can not be decompressed in
two cases: 1) chunk has not been compressed 2) chunk is corrupted

The error message should be printed only in case 2)

DDN-bug-id: EX-9784
Was-Change-Id: I85f4850f989ba0fc8f00653f8f6b0f1b4837d625

osd: add @read param to dt_write_prep()

the purpose is to skip full page reading when OSD is aked for
pages to be written later. in CSDC we need all data as the
pages can be compressed and we may need to decompress them
first.

DDN-bug-id: EX-9858
Was-Change-Id: I2ea9888a1dab419275dbee97e9acc661101b5262

ofd: new structures and data for csdc in ofd

just structures and few helper function to manipulate
chunks and buffers.

DDN-bug-id: EX-9858
Was-Change-Id: I25931f8360a30689a027eaf05ad6c43b4f1746f9

ofd: chunk decompress function

ofd_decompress_chunk() is to literally decompress chunks.
it uses kernel's vmap infrastructure to create a virtual
linear buffer from the pages provided by OSD.

memcpy vs vm_map_ramp()+vm_unmap_ram() benchmark as it's
a replacement for memcpy, each operation repeats 1M times
in each thread, then cost in usec is calculated:

memcpy 32 pages 1 threads - 2404282998, 2404 usec/1
memcpy 32 pages 2 threads - 4830830436, 2415 usec/1
memcpy 32 pages 4 threads - 10098563354, 2524 usec/1

vmap 32 pages 1 threads - 1088538713, 1088 usec/1
vmap 32 pages 2 threads - 2207667980, 1103 usec/1
vmap 32 pages 4 threads - 4613110111, 1153 usec/1

memcpy 64 pages 1 threads - 5564630189, 5564 usec/1
memcpy 64 pages 2 threads - 15904163026, 7952 usec/1
memcpy 64 pages 4 threads - 30345722986, 7586 usec/1

vmap 64 pages 1 threads - 1587480151, 1587 usec/1
vmap 64 pages 2 threads - 3190874847, 1595 usec/1
vmap 64 pages 4 threads - 6887315343, 1721 usec/1

the benchmark was done in qemu (6 cores enabled).

DDN-bug-id: EX-9858
Was-Change-Id: Ia46406401d3c8d720065cc8000b6a52d1a437349

ofd: vm_map_ram prot argument

From kernel 5.8 commit d4efd79a81 "mm: remove the prot argument from
vm_map_ram" the vm_map_ram() function does not accept a prot argument
anymore.

DDN-bug-id: EX-9858
Was-Change-Id: Ib878ba6c6a214ebc0b0716392ce9b7e91bb74453

ofd: csdc read helpers

few functions to implement read path in ofd.

DDN-bug-id: EX-9895
Was-Change-Id: Idca3b095471dbbf177b4026853aa716414280757

ofd: csdc write helpers

few functions to implement write path in ofd.

DDN-bug-id: EX-9895
Was-Change-Id: I0f6802b0911bd358bdb69e0ddb5e8581878cb9dc

ofd: csdc truncate support

to truncate a compressed file correctly we may need to
convert a compressed chunk into a regular one first.

DDN-bug-id: EX-9895
Was-Change-Id: Ia8a3b12eda81874e209046eef1afd33e165561b0

ofd: ofd_object_punch_compr

Fix potential memory leak in ofd_object_punch_compr.
It might not release orig_lnb in case of error from
ofd_trans_create or ofd_map_niobuf_to_chunks.

DDN-bug-id: EX-11092
Was-Change-Id: I45bafdbbf06036add5b5fa9993cbac9e06c856e9

tests: couple new FAIL codes

and trivial way to break a compressed chunk on a disk.

DDN-bug-id: EX-9858
Was-Change-Id: I5516ab3d7add4eb5d76292cf15ac220db08a7150

csdc: expand filter_fid

Expand filter_fid to include compression information, for
compatibility reason, if the file is an uncompressed file, still
store the old filter_fid with no compression info in FID EA.

WasChange-Id: I388500c03604749d05849aeed3c9141974540e4a

lfsck: support compression in filter_fid

The OST object compression info has been stored in filter_fid
by EX-8038, the LFSCK also needs to handle the compression info
during processing the filter_fid.

DDN-bug-id: EX-6247
Was-Change-Id: Id050214ba0057776a05c194dc9222117c5d7fd8b

ofd: add info in overflow error message

Such a message should help to inverstigate which
client triggered the overflow.

DDN-bug-id: EX-9895
Was-Change-Id: I701d8c8fb2a37684ecb0cd5cd272321c715db864

ofd: reject incomplete compressed chunks

client can send a compressed tail, but this RPC can race with another
newer RPC containing bits (uncompressed) from the same compression
chunk as the first RPC. OST can't handle such a compressed partial
chunk properly (that would need to decompress into a temp buffer,
then merge back to disk buffers). but we can ask the client to resend
this RPC uncompressed.

DDN-bug-id: EX-10402
Was-Change-Id: I5e854be4300d50bc8368911782018fb6fa6d38fa

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I7b95416530b5d0f1994900ee4ae1ceb392046e32

68801 master J !B ✗T –R +1/2

LU-10026 ofd: fix punch vs BRW deadlock on compressed files

build failure (#131557)

csdc XS +8 −1

10d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #131557 FAILURE

reviews: Sebastien Buisson +1 2026-10-23 09:56

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: fix punch vs BRW deadlock on compressed files

ofd_object_punch_compr() converts a compressed chunk back into a
plain one when a truncate lands in the middle of it.  It asks the
OSD for the chunk pages first and takes the chunk range lock
afterwards, while ofd_preprw_read_compr() and
ofd_preprw_write_compr() take the range lock first and only then
ask the OSD for the pages.

A truncate and a BRW touching the same chunk deadlock on that: the
punch thread holds the page locks that osd_bufs_get() acquired and
sleeps in range_lock(), while the BRW thread holds the chunk range
lock and sleeps in osd_get_page() waiting for one of those pages.
Nothing breaks the cycle, so the OST_PUNCH never completes and the
client stays blocked in ftruncate().  sanity-compr test_fsx trips
this a minute or two after fsx starts and then eats the whole
session timeout.

Take the chunk range lock before mapping the chunks to OSD buffers
so that all three paths lock the range before the pages.

Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11952
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6095f9533c23a376cf0b5c84bb75c4d956922204

65737 master J –B ✓T ✗2R -1/2−

LU-10026 csdc: set compress component for file

needs rebase — checkpatch: cannot be cherry-picked

needs rebase csdc 14 unresolved XL +1163 −16

11d

needs rebase — checkpatch: cannot be cherry-picked

enforced failed: custom-1001, review-dne-part-7

14 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

failed enforced test	platform	detail
custom-1001	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-pfl.	session
review-dne-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-pcc.	session

build #132858 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Roman Bolshakov, Andreas Dilger, Artem Blagodarenko

patchsets:

+10 earlier

PS1 05-07 PS2 05-07 PS3 05-28 PS4 05-28 PS5 06-11 PS6 06-11 PS7 06-16 Artem Blagodarenko PS8 06-16 Artem Blagodarenko PS9 07-24 Artem Blagodarenko PS10 08-19 Artem Blagodarenko

PS11 08-19 Artem Blagodarenko PS12 08-20 Artem Blagodarenko PS13 08-20 Artem Blagodarenko PS14 08-21 Artem Blagodarenko PS15 10-05 Roman Bolshakov

threads: 14 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(minor) The body doesn't mention the client-side gate added by squashing Ib1e2123ff: the new lustre_check_lov_user_md() and lustre_check_lov_comp_md_v1() in ptlrpc/pack_generic.c, and their calls from ll_lov_setstripe_ea_info() and ll_xattr_set(). These reject a compress layout when `llite.*.enable_compression=0`, which is a user-visible behaviour change. Could a short paragraph describing them be added?

Andreas Dilger PS11 · 2026-08-19 23:52

@ablagodarenko@thelustrecollective.com it looks like the patches were rebased, but did not address any of the AI review comments?  While we can't go overboard changing every detail of the patches, I think it does make sense to fix the code where possible so that we don't carry technical debt later.

Artem Blagodarenko PS11 · 2026-08-20 12:36

Hi Andreas. Thanks for poining to this. I have reverted all fixed by Patchset 11 pushing wrong branch. I have reverted to Patchset 10 where I addressed AI bot comments and fixed build.

reply PS14 2026-10-05 17:51 Roman Bolshakov patchset comment — Squashed Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f into the change. I'll address a couple more comments in another push.

Roman Bolshakov PS14 · 2026-10-05 17:51

Squashed Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f into the change. I'll address a couple more comments in another push.

reply PS9 2026-08-19 23:52 Andreas Dilger lfs-getstripe.1:331 — Should be fixed if patch is refreshed.

Gerrit AI review for Lustre PS9 · 2026-07-28 09:23

(minor) There is no `.TP` before this entry, so `--compress-type|--compr-type` and its description get folded into the `-z, --extension-size` paragraph above instead of starting their own tagged item. The two entries below it do have theirs.

Oleg Drokin PS9 · 2026-08-12 02:29

weird but in my man output it looks ok.

Artem Blagodarenko PS10 · 2026-08-19 19:47

The entry was missing its .TP in the troff source. Some man implementations are lenient and reflow correctly, but others fold the entry into the previous paragraph. Added .TP for correctness.

Artem Blagodarenko PS10 · 2026-08-19 19:47

Done. Added the missing .TP before the --compress-type|--compr-type entry.

Andreas Dilger PS9 · 2026-08-19 23:52

Should be fixed if patch is refreshed.

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) Neither of the two new man3 pages has an EXAMPLES section. Lustre llapi pages are expected to carry a short usage example showing the required header and a realistic call.

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(minor) This, and the matching EINVAL entry under ERRORS, says a `stripe_size` that is not a multiple of `chunk_size` is an error. llapi_layout_compress_set() never fails for that: it decrements `log_bits` until the chunk divides the stripe and stores the smaller chunk (sanity-pfl test_100c expects 3*64KiB to become 128KiB). Similarly, gzip/lz4/lz4fast levels outside the listed ranges are clamped by their from_*_level() helpers, not rejected with EINVAL as the text above says (test_100c expects gzip:15 to read back as 9). If the patch is refreshed, could the page describe the rounding and clamping instead?

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) llapi_parse_compress_type() also returns -EINVAL - for a level that isn't a valid number or falls outside +/-COMPR_LEVEL_MAX - which neither RETURN VALUES nor ERRORS mentions. Mixing -1 with -Exxx returns in one function is also awkward for callers; would returning -EINVAL for the unknown-type case be more consistent?

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) The 9 used as the lz4 default collides with a real level. from_lz4_level(10) returns 9, and to_lz4_level(9) returns 9 because 9 == LZ4HC_DEFAULT_CLEVEL, so 'lfs setstripe -Z lz4:10' reports back as lz4:9 and level 10 can never be displayed - even though lfs-setstripe.1 documents 1-16 as the valid lz4 range.

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) This gate runs only for regular files: ll_lov_setstripe_ea_info(), and the `lov.` branch in ll_xattr_set(). A directory default goes through ll_setstripe_ea() -> ll_dir_setstripe() (and the LL_IOC_LOV_SETSTRIPE case in ll_dir_ioctl()), which never calls lustre_check_lov_user_md(). So on a client with `enable_compression=0`:

    lfs setstripe -E eof -Z lz4 $dir    # accepted
    touch $dir/f                        # MDS instantiates a compress layout

This still produces a file the client can't handle. lsm_lmm_verify_v1v3() rejects it with "file compression not enabled", and the PS14 report shows a later unlink or stripe change hitting the lov_io_layout_at() LBUG. Should ll_dir_setstripe() apply the same check? Or, if allowing compressed defaults on a disabled client is intended, should the client reject the layout cleanly instead of asserting?

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) On this `lov.` path nothing validates `size` before lustre_check_lov_user_md() walks the entries. lustre_check_lov_comp_md_v1() reads `lcm_entry_count` and then up to 65535 `lcm_entries[]`, but the buffer here is only the `size` bytes copied in by the VFS.

The `lustre.` xattr namespace needs only write permission on the inode, so any file owner can trigger it:

    setxattr(f, "lustre.lov.add", buf, 8, 0);  /* buf starts with LOV_USER_MAGIC_COMP_V1 */

That reads lcm_entry_count past the end of an 8-byte allocation and keeps reading for up to ~3MB. KASAN reports it as slab-out-of-bounds. Before this patch, little-endian clients only read `lmm_magic` here.

The next change in the series ("utils: add 'lfs find' support for compressed file") adds a `size < need` check just above this call. Since this patch introduces the read, should that check move here? Passing `size` into lustre_check_lov_user_md(), the way lustre_swab_lov_user_md() takes one, would also let it validate the buffer itself.

Gerrit AI review for Lustre PS14 · 2026-08-22 12:47

(minor) This applies the chunk adjustment to every component including extension ones, where llc_stripe_size holds the extension size rather than a real stripe size. lod_declare_layout_add() deliberately skips LCME_FL_EXTENSION components for exactly that reason, so the same input layout gets different chunk_lum_bits depending on whether it arrives via create or via --component-add. Should this site skip extension components as well?

While there: the comment added in lod_declare_layout_add() says chunk adjustment is deferred until instantiation, but nothing calls lod_adjust_compr_chunk_size() on the instantiation path - the only three callers are here, lod_declare_layout_add() and lod_striping_from_default().

Gerrit AI review for Lustre PS15 · 2026-10-06 08:14

(defect) -ENOTSUPP (524) is kernel-internal and has no userspace errno, so this reaches `lfs setstripe` through ioctl()/setxattr() as "Unknown error 524". The erasure-coding check just above it in ll_lov_setstripe_ea_info() returns -EOPNOTSUPP for the same kind of rejection. Should this be -EOPNOTSUPP too?

While here, a couple of nits: `rc` is never changed from 0, there's a double blank line after the EXPORT_SYMBOL(), and the default case in lustre_check_lov_user_md() logs "Invalid LOV magic" but still returns 0.

tags: csdc

owner Marc Vef · uploader Roman Bolshakov · PS15 uploaded 27d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: set compress component for file

Introduce per-component compression support for composite LOV layouts.
A new LCME_FL_COMPRESS component flag marks entries that carry
compress fields; the flag is set when the user supplies a compression
type via 'lfs setstripe -Z' and is propagated through the LOD and LOV
layers so that compressed components are distinguished from plain
ones.

LOD layer:
- lod_parse_striping(): read lcme_compr_type/lvl/chunk_lum_bits from
  the user-supplied lov_comp_md_v1 when LOV_PATTERN_COMPRESS is set.
- lod_use_defined_striping(): likewise for pre-defined layouts.
- lod_get_default_lov_striping(): carry compress fields into the LOD
  component when inheriting a default layout that includes
  compression.
- lod_generate_lovea(): serialise llc_compr_* fields into the on-wire
  lcm entry when LOV_PATTERN_COMPRESS is set.
- lod_adjust_compr_chunk_size(): new helper that rounds down the
  requested chunk-size exponent so the chunk fits evenly within one
  stripe and within one RPC (clamped to COMPR_CHUNK_MAX_BITS in the
  following changes).

API / utils layer:
- 'lfs setstripe' gains --compress|-Z <type>[:<level>] and
  --compress-chunk=<size> options to set the compression type, level,
  and chunk size on a new component.  Like -S and -c, -Z is inherited
  by subsequent -E components unless overridden by another -Z.

  Example:
	$ lfs setstripe -Eeof -Z lz4:5 --compress-chunk=512 <file>

- 'lfs getstripe' gains --compress-type, --compress-level, and
  --compress-chunk options to print individual compress parameters:

  Example:
	$ lfs getstripe <file>
    lcme_compr_type:     lz4
    lcme_compr_lvl:      5
    lcme_compr_chunk_kb: 512
      lmm_pattern:       raid0,compress

- llapi_layout_merge(): copy compress fields when merging layouts.
- Two new man3 pages: llapi_layout_compress_set.3 and
  llapi_parse_compress_type.3.

Misc:
- dump_lsm() in lov_ea.c prints compress fields for debugging.
- compr_name_from_type() helper added in lov_internal.h and wired up
  in dump_lsm() to print compression type names in debug output.

Test-Parameters: testlist=sanity-pfl serverversion=2.17
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Signed-off-by: Roman Bolshakov <rbolshakov@ddn.com>
Change-Id: Ife0382469cbc5099e0c6dc96534bb169ddeff61e
Was-Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f

68797 master J ✗1B ✗T –R +2/2

LU-10026 ofd: properly locate root object

build failure (#133377)

1 unique failure csdc 1 unresolved S +17 −8

22d

build failure (#133377)

janitor: 1 test failure unique to this patch — sanity1@ldiskfs+DNE:test_56od

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity1@ldiskfs+DNE:test_56od	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #133377 FAILURE

reviews: Andreas Dilger +1 2026-10-11 04:49 · Mikhail Pershin +1 2026-09-11 06:17

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

On this branch ofd_tunables_init() already ran before the compr_stats block, so the new failure paths jump to a new err_fini_tunables label (ofd_procfs_fini()) ahead of the existing teardown. When the compr_stats file cannot be obtained the stored error pointer is cleared so ofd_fini() does not try to save/put it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: properly locate root object

ofd_init0() gets root object by using dt_locate_at() with
an OSD device as top. As result lu_object can be incomplete
if it doesn't exist yet. Instead dt_locate() should be used.

Patch fixes that and adds error handling in related places.
When the compr_stats file cannot be obtained the reference is
cleared, so that it is neither saved nor released at fini.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-13567
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9f5eaaae9c4ea23225f574b8e7cde64b3c5f47b6

68391 master J !B ✗T –R 0/2

LU-10026 csdc: migrate to another compression level

build failure (#130363)

csdc 2 unresolved M +70 −0

47d

build failure (#130363)

1 unresolved thread(s) await your reply — last from Patrick Farrell

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130363 FAILURE

no vote yet: Sergey Cheremencev, Artem Blagodarenko, Alex Zhuravlev

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

Version gate converted to compression_supported.

reply PS1 2026-09-15 18:51 Patrick Farrell sanity-compr.sh:2033 — **[AI review]**

Patrick Farrell PS1 · 2026-09-15 18:51

**[AI review]**

test_1030 reads $tmp_hdf and $hdf, but neither is set here. Every other test that uses them declares them as locals in its own function (test_1007 at sanity-compr.sh:552-554, and five more sites), so they are empty by the time test_1030 runs.

So [[ ! -f $tmp_hdf ]] is always true, [[ -f $hdf.bz2 ]] tests the literal path ".bz2" and is always false, and the test always takes the else branch. test_1030 never executes, and the skip reason it prints is misleading.

Seen on a full sanity-compr run of this series on Rocky 9, where bunzip2, bzcat and lustre/tests/AMSR_E_L3_DailyOcean_V05_20111003.hdf.bz2 were all present, and 1007 and 1008 -- which do declare the locals -- had already run and passed:

  SKIP: sanity-compr test_1030 bunzip2 is not installed

Declaring the same locals here as the other tests do should be enough:

  local hdf=$LUSTRE/tests/AMSR_E_L3_DailyOcean_V05_20111003.hdf
  local tmp_hdf=$TMP/$tfile.hdf

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: migrate to another compression level

Here is a test there lfs migrate with changed
compression level.

tests: fix sanity-compr 1030

Set compress_check_bytes to 2 times larger
than uncompressed file size. Without that
file might be considered incompressible.

Add extra debugging to help understand test failures.

DDN-bug-id: EX-10512
Was-Change-Id: I559f94b24bd2abb821a07775198bb4e511793a72

tests: change algo in 1030

to compare apples with apples. skip (i.e. do no fail) if
less data were compressed in the 2nd approach as this may
happen due to external conditions we don't control well
(grants, etc).
take into account that higher compression level in lz4fast
means less efforts and larger result.

DDN-bug-id: EX-10512
Was-Change-Id: Ia4c655ec99c1b45ff9c273311e5d567b8201cd5a

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9889
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia67316a247fed99f6ad002c8b67053a3c2fa0cff

68411 master J !B ✗T –R 0/2

LU-10026 csdc: parameter to disable compression on HDD OSTs

build failure (#130383)

csdc 2 unresolved L +471 −11

48d

build failure (#130383)

1 unresolved thread(s) await your reply — last from Patrick Farrell

janitor run: 40 failing config(s), none unique to this patch — janitor results

build #130383 FAILURE

no vote yet: Yang Sheng, Sergey Cheremencev, Artem Blagodarenko

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- In lod_ost_alloc_rr the -EBADR early-out jumps to this tree's out_pool label (the original's 'out' label; identical cleanup).
- The HDD check in lod_ost_alloc_qos sits after the additional allocation filters this tree has in that loop.
- The sysfs attribute is inserted in the alphabetically sorted attr table (the original appended).
- The three sanity-compr 2020* version gates converted to compression_supported; the statistics-init helper keeps the unconditional obdfilter access.
- Subject shortened to the 62-column limit; the tunable paragraph in the message and man page rewrapped.

Patrick Farrell PS1 · 2026-09-14 22:19

test_2020a needs at least two OSTs and has no guard for it: it marks OST0001 as SSD with 'do_facet ost2' below, waits on an OST0001 line from 'lfs df -v', and calls subtest_2020a with '-i 1'. On a single-OST setup it fails rather than skipping.

Adding the existing idiom next to the compression_supported check would cover it:

	(( OSTCOUNT >= 2 )) || skip_env "needs >= 2 OSTs"

Confirmed by running it both ways: it fails with one OST and passes with two. test_2020b and test_2020c only use osts_nodes, so they are fine as they are.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: parameter to disable compression on HDD OSTs

The current CSDC compression does not perform well with HDD OST
storage due to fragmentation of the block allocation for a
compressed file. This is acceptable for NVMe OSTs because they
do not have seek latency, but can cause significant performance
impact for HDD OSTs, both for the compressed objects, as well as
uncompressed objects that are being written concurrently due to
IOPS starvation and fragmented free space.

This patch optimizes compression for systems with HDD OSTs.
By default, compression is not supported on HDD OSTs. Such an OST
is skipped and the next SSD-based OSD is chosen. This behaviour can
be adjusted using lod.*.enable_compr_rotational. Allowed values are:
 * -1 - return -EBADR error when trying to allocate compressed files
        on HDD OSTs
 * 1  - allow compression on HDD OSTs
 * 0  - skip an HDD OST

tests: fix sanity-compr/2020 for one-node setup

The check in sanity-compr test_2020() if an OST was flash or HDD
was unreliable if the MDT was also on a flash device, or if there
were other statfs flags on the target devices.

Improve this check to only look for the 'f' (flash/nonrotational)
status on the specific OSTs being used for the test.

DDN-bug-id: EX-10989
Was-Change-Id: If8ec5e6ce02ce4e5eee4ffb00bc71cda577d98e0

lod: dereference the device after check

Ensure the index is valid before dereferencing it.

DDN-bug-id: EX-10989
Was-Change-Id: Iefab5ae2036c09819016b2e569c870ba36174e38

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10989
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibc485aaed6fe719591f64e55b1203ddd2e646f09

68796 master J ✗1B ✗T –R 0/2

LU-19794 csdc: use acomp compression API

build failure (#131552)

1 unique failure csdc 2 unresolved M +199 −8

49d

build failure (#131552)

janitor: 1 test failure unique to this patch — sanity-quota@ldiskfs+DNE:test_1k

1 unresolved thread(s) await your reply — last from Nikos Papakonstantinou

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-quota@ldiskfs+DNE:test_1k	seen in 11 other reviews

build #131552 FAILURE

no vote yet: Nikos Papakonstantinou

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

reply PS1 2026-09-14 09:36 Nikos Papakonstantinou COMMIT_MSG:1 — I am not aware of this tbh. I tried to build this but I didn't got this.

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-09-10 20:19

ldlm/ldlm_lib.c:
 warn: client_connect_import():duplicate check 'rc' (previous on line 745)

Nikos Papakonstantinou PS1 · 2026-09-14 09:36

I am not aware of this tbh. I tried to build this but I didn't got this.
I am using a Ubuntu 25.10 vm with 6.17.0-14 kernel.
Of course I am building it with --disable-server option.

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- Only the series files are ported here (lustre_compr.h/.c, lustre_crypto.h, ofd_compress.c, osc_compress.c, configure probe); the hunks to the lz4/gzip/zstd modules, kcompr kunit and compr_private/scompress.h belong to the standalone module changes (57776/62408/66001) and are left for new patchsets there, together with the LC_HAVE_CRYPTO_STREAM_IN_SCOMP probe they need.
- The HAVE_CRYPTO_REQUEST_SET_DMA probe sits under the 6.15 section (kernel commit b67a0260037, v6.15-rc1).
- The acomp branch of ofd_decompress_chunk() was completed: the request/wait variables are declared, and a failed request allocation sets -ENOMEM and falls through to the existing error handling (the original referenced undeclared variables and a missing label there). acomp_request_alloc() returns NULL on failure, so the allocation checks test for NULL instead of IS_ERR().
- The request buffers are set with the non-DMA variants of the acomp setters (acomp_request_set_src/dst_nondma): every buffer handed in is a vm_map_ram() mapping or a large pool allocation, which the DMA variants' contract does not allow (inert for scomp-backed algorithms, wrong for a hardware acomp provider).
- ll_crypto_comp_set_level() is compiled out when the legacy comp API is absent; the acomp variant is unconditional on this branch.
- Caveat: no build available to the porter compiles the acomp side (el8 4.18; the janitor uses rocky 8/9); verified by reading against the v6.15 headers. Reviewers with a 6.15+ kernel, please build it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-19794 csdc: use acomp compression API

For kernels 6.15 and beyond use the acomp compression API with DMA
source and destination virtual addresses, instead of the legacy comp
API which was removed there.

The acomp request allocation returns NULL on failure, so the
allocation checks test for NULL and fail with -ENOMEM. The buffers
handed to the request are vm_map_ram() mappings or large pool
allocations, which are not usable for DMA, so they are set with the
non-DMA variants of the request setters.

Test-Parameters: forjanitoronly
Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I1fa60a7a933e6cdd2baa525d0520e7372def29df

68799 master J ✗1B ✗T –R +1/2

LU-10026 ofd: fix csdc optimizations

build failure (#131555)

1 unique failure csdc 1 unresolved S +17 −4

49d

build failure (#131555)

janitor: 1 test failure unique to this patch — sanity-slow@ldiskfs+DNE:test_255c

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-slow@ldiskfs+DNE:test_255c	seen in 5 other reviews

build #131555 FAILURE

reviews: Alex Zhuravlev +1 2026-09-14 08:25

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The fix Alex Zhuravlev asked to land together with 68366/68378; `llch.` member access follows the stack-copy form of the header introduced by the checksum change.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: fix csdc optimizations

we shouldn't ignore pre-existing chunks doing RMW if new data
don't cover full range (i.e. have gap(s))

Test-Parameters: forjanitoronly
DDN-bug-id: EX-14659
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I08e7d3bf9a263ed31ed5ea82a4f0800eeb0529c7

68786 master J !B ✗T –R 0/2

LU-10026 tests: move verify_comp_attr to test-framework

build failure (#131542)

csdc 1 unresolved M +79 −74

51d

janitor run: 9 failing config(s), none unique to this patch — janitor results

build #131542 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

Preparatory, new on master: verify_mirror_count()/verify_comp_attr() move from sanity-flr.sh to test-framework.sh (sanity-pfl needs them, including the already-present test_100k) and verify_comp_attr() learns compr-type/compr-level/compr-chunk/pattern/mirror-id. The string comparison is kept (the internal variant compares arithmetically, which is always true for non-numeric attributes).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: move verify_comp_attr to test-framework

Move verify_mirror_count() and verify_comp_attr() from sanity-flr.sh
to test-framework.sh so that other suites can use them, and teach
verify_comp_attr() the compression attributes (compr-type,
compr-level, compr-chunk) as well as pattern and mirror-id.

sanity-pfl test_100k already uses verify_comp_attr(), and the
following compression mirror tests need it too.

Test-Parameters: forjanitoronly
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0a679215a0848e5d348c4175b09d39fb125c7607

68792 master J !B ✗T –R 0/2

LU-10026 tests: FID retrieval from getstripe

build failure (#131548)

csdc 1 unresolved S +33 −6

51d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #131548 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The sanityn.sh hunk (a trash-can test) does not apply on this branch and was dropped; the rest as the original.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: FID retrieval from getstripe

Some tests rely on the getstripe output to obtain
an object's FID. However, for compressed files, the
getstripe output format differs significantly.

This patch introduces a function that handles both
output formats. All functions potentially affected
by this change have been updated accordingly.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8258
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia9ce932a023066c78aa819bdf502191c5ecc2044

68790 master J !B ✗T –R 0/2

LU-10026 csdc: compress_request memory optimisation

build failure (#131546)

csdc 1 unresolved M +52 −55

51d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #131546 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- merge_chunk() fills the page array with brw_folio_page() (this branch's brw_page carries a folio + page index instead of a page); otherwise as the original.
- Notes for reviewers (inherited, unchanged): the vm_map_ram() NULL path in compress_request() returns -ENOMEM without releasing the pool destination buffer; decompress_request() leaves the mapping in place on two error paths after it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: compress_request memory optimisation

Map brw pages with vm_map_ram instead of copying them to a
buffer allocated by sptlrpc_pool_get_pages. This should save
memory and some CPU cycles.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10553
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3a9280a70a7ac25e29b4e68f0810fe89a5ca69fc

68791 master J !B ✗T –R 0/2

LU-10026 lov: disable compression for DoM

build failure (#131547)

csdc 1 unresolved S +45 −0

51d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #131547 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

test_2003 relocated after test_2002 (numeric order); its version gate converted. Its fix "distinguish the -Z none parameter for DoM" follows three changes later (source landing order kept).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: disable compression for DoM

Disallow enabling compression on the DoM component until
the related feature is properly fixed.

Added a test to ensure this combination is prohibited,
while confirming that DoM continues to function normally.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8718
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I338b375ab386826d08b5610105b14d7496e33aa7

68794 master J ✗2B ✗T –R 0/2

LU-10026 csdc: distinguish the -Z none parameter for DoM

build failure (#131550)

2 unique failures csdc XS +3 −2

51d

build failure (#131550)

janitor: 2 test failures unique to this patch — recovery-small@ldiskfs+DNE:test_115b, sanity-slow@zfs:test_255c

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
recovery-small@ldiskfs+DNE:test_115b	seen in 7 other reviews
sanity-slow@zfs:test_255c	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #131550 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: distinguish the -Z none parameter for DoM

When "-Z none" is passed, LOV_PATTERN_COMPRESS is set
with LL_COMPR_TYPE_SPEC_NONE, this is how this case
can be distinguished for DoM with compression
prevention.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-13161
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0718af523efe2952b844e2d474acc1c02493303a

66001 master J !B ✗T –R 0/2

LU-10026 lustre: add lzstd kernel module

build failure

csdc 34 unresolved XL +33132 −11

51d

build failure

34 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 4 failing config(s), none unique to this patch — janitor results

no vote yet: Andreas Dilger, Sebastien Buisson, Artem Blagodarenko, Oleg Drokin

patchsets: PS1 05-19 PS2 05-19 PS3 09-10 Sebastien Buisson PS4 09-10 Sebastien Buisson

threads: 34 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) debug.c:24 — warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) debug.c:24 — warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warning: sparse:symbol 'g_debuglevel' was not declared. Should it be static?

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) huf_compress.c:460 — warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) huf_compress.c:460 — warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warn: HUF_getIndex():the 'RANK_POSITION_LOG_BUCKETS_BEGIN' macro might need parens

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_compress.c:4684 — warn: ZSTD_initCDict_internal():inconsistent indenting

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_initCDict_internal():inconsistent indenting

reply PS4 2026-09-10 16:10 Misc Code Checks Robot (Gatekeeper helper) zstd_compress.c:4684 — warn: ZSTD_initCDict_internal():inconsistent indenting

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-09-10 16:10

warn: ZSTD_initCDict_internal():inconsistent indenting

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1577 — warn: ZSTD_compressBlock_lazy_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1605 — parse ZSTD_compressBlock_lazy_generic():error: Function too hairy. Giving up. 3 seconds

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

parse ZSTD_compressBlock_lazy_generic():error: Function too hairy.  Giving up. 3 seconds
warn: ZSTD_compressBlock_lazy_generic():Function too hairy.  No more merges.

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1915 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1937 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1947 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

reply PS3 2026-09-10 16:07 Misc Code Checks Robot (Gatekeeper helper) zstd_lazy.c:1979 — warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-09-10 16:07

warn: ZSTD_compressBlock_lazy_extDict_generic():if statement not indented

tags: csdc

owner Marc Vef · uploader Sebastien Buisson · PS4 uploaded 53d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lustre: add lzstd kernel module

lzstd kernel module implements compression according to the zstd
algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'zstd':
* zstd-lustre-generic of type compression
* zstd-lustre-scomp of type scomp
And 2 cipher drivers under the generic name 'zstdfast':
* zstdfast-lustre-generic of type compression
* zstdfast-lustre-scomp of type scomp

Note the 'zstd' name is identical to the in-kernel module, but lzstd
registers it with a slightly higher .cra_priority = 110, so that it is
preferred over the in-kernel module.
Our 'zstd' is also different in that it accepts a compression level
as explained below. 'zstdfast' is handling the 'negative' compression
levels, interpreted as an acceleration factor.

lzstd kernel module sources are copied from linux v6.9. This module
implements the Crypto API interface, and rely on the zstd kernel
library for compression implementation. It has been modified to grok
a compression level, as read from the top 4 bits of the crypto_tfm
flags, and pass it to the underlying library.
The zstd library sources are also copied from linux v6.9 and built
statically. Headers have also been copied from linux v6.9 for
consistency, and source files modified to include the copied headers
instead of the system headers.
In order to maximize compatibility with various distributions, we copy
xxhash sources from linux v6.9, and link these statically.
All aforementioned sources are located in the
lustre_compat/crypto/zstd directory.
The lzstd module is always built with Lustre.

This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'zstd' module.
It also tries to manually load the lzstd kernel module when a file
requires compression with the 'zstd' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.

EX-9678 csdc: defer memory allocation in zstd/zstdfast

Zstandard normally proceeds to memory allocation for its context
(workspace) while in the init phase, which corresponds to a call to
crypto_alloc_comp() when used via the Crypto API.
At this stage we cannot be aware of the compression level or the chunk
size. So we have to make zstd allocate a workspace that can fit any
level/chunk. And the higher the level is, the larger the workspace
needs to be. Problem is zstd is very memory eager, and by default for
the higher compression level it requires a workspace of 650 MiB. This
is insane compared to our usual compression chunk sizes.

So we defer workspace initialization to the first time zstd actually
compresses data. This corresponds to a call to crypto_comp_compress()
when used via the Crypto API. And by that time we know the compression
level and chunk size, which enables us to make zstd allocate a finer
grained workspace, that just fits our actual needs. This saves a lot
on memory allocation, as can be seen from these values (workspace size
expressed in bytes):

                      no src hint   64kB src
   level 1 cwksp size      582392    303864
   level 2 cwksp size      779000    402168
   level 3 cwksp size     1303288    664312
   level 4 cwksp size     2614008   1057528
   level 5 cwksp size     3662584   1057528
   level 6 cwksp size     3662584   1057528
   level 7 cwksp size     6808312   1057528
   level 8 cwksp size     6808312   1057528
   level 9 cwksp size    13099768   1057528
   level 11 cwksp size   25682680   1319672
   level 13 cwksp size   34071288   1468856
   level 15 cwksp size   67625720   1791095
   level 17 cwksp size   50997688   1791095
   level 19 cwksp size   85196598   1791095
   level 21 cwksp size  336854838   1791095
   level 22 cwksp size  680943414   1791095

And of course because we reduce contention on memory a lot, we improve
performance when zstd is used via the Crypto API, which is done in
osc_brw_prep_request() for ever RPC that involves a file compressed
with zstd.
Was-Change-Id: I470331ea0f2e550761ce7223e7be7e1373701bd2

Was-Change-Id: Idf7382093feec21bed5b6e62001a085a386736d2
Was-Change-Id: Ifa9d38d807d0ab55f85f6c6c8672c340778c2855

EX-bug-id: EX-9678
Test-Parameters: testlist=sanity env=ONLY=84
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I113345cd10f41197683bfe127d373b8628364f88

68798 master J !B ✗T –R 0/2

LU-10026 osc: do not expand if overflow compression

build failure (#131554)

csdc S +23 −0

51d

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #131554 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: do not expand if overflow compression

The extent expansion function osc_extent_expand() can
expand extents beyond the compression chunk limit, completely
bypassing the limit enforced in osc_extent_find() and
try_to_add_extent_for_io().

Let's add the check to the osc_extent_expand() to
make this situation impossible.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12849
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id0240c2078840a250d169a946e1894f4375fd0b9

68795 master J !B ✗T –R 0/2

LU-10026 utils: add compression type completion

build failure (#131551)

csdc 1 unresolved M +59 −0

52d

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #131551 FAILURE

no vote yet: Chakshu Kansal

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

Applied cleanly; test_440b's compression_enabled gate replaced by compression_supported (the test only reads osc.*.compress_types).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: add compression type completion

Add bash completion support for compression types in lfs setstripe
command. Also adds a test case to verify the functionality works
correctly.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12299
Signed-off-by: Chakshu Kansal <ckansal@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id19e27a3088023dca4dda03b19054e5fdf0a4385

68787 master J !B ✗T –R 0/2

LU-10026 lov: pack layout_gen and compr info into lcme

build failure (#131543)

csdc 1 unresolved M +122 −0

52d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #131543 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- Field renames to *_chunk_lum_bits; the two sanity-pfl gates converted to compression_supported; subject shortened.
- test_100n expects the pattern "raid0,compress" for the compressed mirror (the original compared "compress" arithmetically, which always passed; the string comparison used here needs the full pattern name, as in test_100k). Depends on the preceding helper commit (verify_comp_attr in test-framework.sh with the compression attributes).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: pack layout_gen and compr info into lcme

Creating a compressed mirror with "lfs mirror -N" did not produce a
compressed component. Even when the compression parameters were fully
specified, the new mirror was created without compression, and lfs
getstripe showed the mirror entry lacking compression fields.

lov_lsm_pack() did not populate the composite entry fields
lcme_layout_gen and lcme_compr_* when packing a lov_comp_md_v1. As a
result the MDS/client logic took the "no layout/compression in entry"
path and defaulted to an uncompressed layout for the new mirror.
llapi_layout_mirror_inherit() should set compression fields.

Set these fields for each composite entry and set during layout mirror
inherit to fix the problem.

sanity-pfl 100n and 100m were added.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12672
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I5bfcd2d942e9eeb853674763f8ea14d97e4a966c

68788 master J ✓B ✗T –R 0/2

LU-10026 tests: compression component via --component-add

build failure (#131544)

csdc 1 unresolved S +15 −0

52d

build #131544 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

The lod fix this test was written for is already on master (90c093b7850, LU-17468); only the sanity-compr test_1a part was unlanded - ported under a new Change-Id with the original as Was-Change-Id.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: compression component via --component-add

Add sanity-compr test_1a: create a file with a plain first component,
append a compressed component with "lfs setstripe --component-add
-Z gzip" and verify the added component carries the compression
type. This covers the compression branch of the component-add path
in lod_declare_layout_add().

The lod fix this test was written for is already on master as
commit 90c093b7850 ("LU-17468 lod: component add missed pattern
info"); only the test part is new here.

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I7ad746a550f1afea54a6f5b68823a79a85a44082
Change-Id: I0624bf3f5f9c965f826e72656a8cd8198c5624fd

68793 master J –B ✗T –R 0/2

LU-10026 tests: enable CSDC layout in sanity-compr.sh

build failure (#131549)

csdc 1 unresolved M +146 −56

52d

build #131549 FAILURE

no vote yet: Sarah Liu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: enable CSDC layout in sanity-compr.sh

Enable CSDC layout on root mount in sanity-compr.sh and run the
sanity and sanityn suites with that layout. Subtests that do not
work with a compressed root layout yet are excepted when the layout
carries a compression component, and a few tests are adjusted to
either disable compression for the file under test or to account
for compressed data (block usage, statistics, FID parsing).

sanity test_127a skips its sum check only when the layout is
compressed, and sanityn test_93 reads the OST index from both the
plain and the composite getstripe output.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8258
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2e0e0c3e34d398156a56128925f02336d59ecfca

68372 master J –B ✗T –R -1/2−

LU-10026 tests: add sanity-compr test for compression

build failure (#130385)

csdc 1 unresolved L +309 −9

52d

build #130385 FAILURE

no vote yet: Artem Blagodarenko, Alex Zhuravlev

patchsets: PS1 08-26 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- ll_compression_scan moved out of the server-only install block (bin_SCRIPTS under UTILS), matching the patch intent against master's restructured scripts/Makefile.am.
- The report_error() changes were already on master; only error_ignore_no_dump() is new.
- The ubuntu always_except for test 1008 is removed as in the original (the test source dir change makes it unnecessary).
- test_1007/test_1008 gates converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: add sanity-compr test for compression

This patch adds a sanity-compr test to validate that
we get space usage reduction with compression.
The test uses ll_compression_scan tool to calculate
the compressed size of the source file and compares
it with the size of the Lustre CSDC compressed file.

tests: add sanity-compr test for dir compression

This patch adds a sanity-compr test to validate that
we get directory space usage reduction with compression.

DDN-bug-id: EX-7795
Was-Change-Id: I16f3a3f1e413e4884b3973829df36500667271ce

tests: skip sanity-compr 1007 and 1008

if needed tools are not installed

DDN-bug-id: EX-9871
Was-Change-Id: I0dc3d44c300708f3a25bfce06b81993cdd30c418

tests: change source dir for sanity-compr/1008

While the source directory contains almost all small files,
sanity-compr test_1008 will hit the
"failed estimates > 50% of total estimates" failure on SLES15.

The patch fixes the issue by changing the source dir to
contain large files.

DDN-bug-id: EX-9125
Was-Change-Id: Iad661750cba7c9d2204f2306e73169deb012ddf4

utils: fix ll_compression_scan overflow

Fix the total size estimate calculation to avoid overflow for
large total sizes.  Otherwise, the intermediate register used
for the calculation will overflow and produce an invalid estimte.

DDN-bug-id: EX-9125
Was-Change-Id: I12099cea1268abede17c92e0422b6c1c25ac66d5

tests: sanity-compr/1008 CentOS 7.9 more margin

For sanity-compr test_1008 allow a higher margin of error for
CentOS 7.9 clients because they consistently show slightly
worse compression ratios and fail the test.  It isn't clear
if this is because of worse/partial RPC formation or because
the compression code in this kernel is less optimal.

These tests should still be run to ensure the code is working, but
not fail so much, because fixing these old clients is not critical.

DDN-bug-id: EX-9125
Was-Change-Id: I6c1c6044daf5d0706cfb661b4f0f051769174ff9

tests: get stats_compr param in sanity-compr/1007

This patch adds getting stats_compr param in sanity-compr
test 1007 and 1008 to track data compression statistics.

DDN-bug-id: EX-9125
Was-Change-Id: I7df25d36c8c89ec5a568bf1e8d5694a97dd18ecc

tests: ignore sanity-compr/1008 failures

Temporarily ignore test failures for sanity-compr.sh test_1008 on
SLES15 and aarch64/ppc64le due to very high failure rates on those
systems.

Rename the *second* test_1008 to test_1080 so it can run, and allow
other new tests to start using disjoint numbers to avoid conflicts.

DDN-bug-id: EX-9125
Was-Change-Id: I199613b1e5b1ee8ea7ca4287a6dfe090257ed72f
Was-Change-Id: I3611d1a8b41267f5c38b8088e06eb8f0cad9dd48

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7795
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icf763331205a3e937b794f90444f756fc59f9050

68800 master J ✓B ✗T –R 0/2

LU-10026 tests: skip sanity-compr/1010 on EL10 clients

build failure (#131556)

csdc 1 unresolved XS +6 −0

52d

build #131556 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip sanity-compr/1010 on EL10 clients

sanity-compr test_1010 ("still can read the broken chunk?") fails when
clients run Rocky Linux 10.x. Skip the test until a fix is available
as it blocks testing.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12865
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id84d8bc4922f8fd37b9e42ae64234a0593d79f7f

68785 master J –B ✗T –R -1/2−

LU-10026 csdc: add compression header checksum

build failure (#131541)

csdc 1 unresolved L +241 −48

52d

build #131541 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- This is series entry 81 (ex-8688), which the original port skipped by an off-by-one in the entry numbering; ported now on top of the series. sanity-compr test_1080 (dropped from "tests: more tests for csdc" for that reason) returns with it.
- The header checksum is gated on the LLCH_FL_HDR_CSUM flag that this branch's ll_compr_hdr reserves for it (set by compress_chunk(), checked by is_chunk_start()) instead of the original's "csum != 0" heuristic. The pre-built sample lustre/tests/compressed.bin has the flag set and its header checksum recomputed accordingly; the checksum constant in test_1080's corrupted-size case was recomputed the same way (algorithm unchanged: crc32 over the header with the checksum field zeroed). test_1080's last case clears the flag (instead of zeroing the checksum) so that the no-checksum path is what gets exercised.
- is_chunk_start() takes the folio and gained the int *rc out-param; the four ofd call sites and the osc call site adapted. The ofd functions keep a stack copy of the header as in the original.
- The header-size sanity limit uses sizeof(*header) (the original compared against the size of a pointer).
- The decompress_rnb() prototype hunk does not apply (that helper was removed earlier in the series).
- The forced-decompression fail-loc (OBD_FAIL_OSC_FORCE_DECOMPR) keeps 0x420, which is free on this branch; the forced type is LL_COMPR_TYPE_LZ4.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: add compression header checksum

This commit adds functionality to verify and calculate the checksum
of the header in the `ll_compr_hdr` struct and some another sanity
check.

The client sets LLCH_FL_HDR_CSUM in llch_flags when it stores the
header checksum, and the checksum is only verified when that flag
is set. A chunk with a valid magic and checksum but inconsistent
header fields is treated as corrupted and the read fails with
-EUCLEAN; without a valid checksum such a chunk is passed through
as plain data.

Add sanity-compr test_1080, which exercises these checks with a
pre-built compressed sample (lustre/tests/compressed.bin).

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8688
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I24a9ab9cb7bea1208ada23aa6550127fe6a55017

68789 master J !B ✗T –R 0/2

LU-10026 osc: prevent oversized extents for compressed objects

build failure (#131545)

csdc 1 unresolved M +165 −1

52d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #131545 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] upstream remarks for reference:

- test_2030 anchored after test_2020c. The original flipped the mode of sanity-compr.sh to 0644; the port keeps it executable.
- Note for reviewers: `if (( -n "$brw_check" ))` in test_2030 is a bash arithmetic expression, not a string test (inherited).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 52d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: prevent oversized extents for compressed objects

When compressed objects have high compress_rpc_fragment_ratio values,
large extents could contain too many compression chunks to fit in a
single RPC, causing try_to_add_extent_for_io() to crash with
LASSERT(data->erd_page_count > 0) when the oversized extent was the
first one being considered.

This client-side bug also caused server-side buffer overflows where
the client would send thousands of compression chunks in a single
RPC, exceeding the server's PTLRPC_MAX_BRW_PAGES limit and
triggering 'too large request' errors.

Instead of splitting extents after they are created, this fix
prevents the problem at the source by limiting extent size during
extent creation in osc_extent_find(). For compressed objects, the
maximum extent size is now calculated based on compression chunk
limits to ensure extents never exceed the RPC fragment ratio.

This proactive approach prevents both client crashes and server
buffer overflows while maintaining optimal performance by avoiding
the overhead of extent splitting operations.

tests: improve sanity-compr test_2030

Some improvements:

- Use save_lustre_params/restore_lustre_params for brw_size params
- Use save_lustre_params/restore_lustre_params for client OSC params
- Add comprehensive parameter validation to ensure settings applied
- Improve brw_size verification to check all OSTs not just first
- Replace cat > /dev/null with MD5 checksum for data integrity

DDN-bug-id: EX-12849
Was-Change-Id: I3a8dfd393289f5be0c638e84e1097b18a4303240

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12849
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id161a17b4b45a6cad39ccd55e18da0f6d4f58957

68397 master J –B ✗T –R -1/2−

LU-10026 tests: more tests for csdc

build failure (#130369)

csdc 2 unresolved XL +1436 −534

52d

build #130369 FAILURE

no vote yet: Artem Blagodarenko, Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 2 own notes · 0 bot

note PS1 2026-09-10 19:52 Marc Vef patchset comment — [Marc Bot] correction to the earlier remark: test_1080 is not excluded because its prerequisite is outside the series - the header-checksum

Marc Vef PS1 · 2026-09-10 19:52

[Marc Bot] correction to the earlier remark: test_1080 is not excluded because its prerequisite is outside the series - the header-checksum change IS series entry 81 and was skipped by an off-by-one in the original port. It is now ported as https://review.whamcloud.com/c/fs/lustre-release/+/68785 ("LU-10026 csdc: add compression header checksum"), which brings test_1080 back together with the compressed.bin sample and the fail-loc it needs.

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- This patch reorganizes sanity-compr.sh wholesale (renumbers 1000->1000a, fsx->1000b, adds 1000c, adds tests 1009a-1016, moves tests into numeric order, adds iozone/dbench/fsx wrappers, file-scope compression gates and DIRECTIO/RAMKB/THREADS/SIZE setup). The port reconstructed the patch's intended post-state and re-applied this tree's conversions on top:
- internal version gates -> compression_supported (or dropped where they guard functionality this series itself provides);
- test_2000 keeps the master client version (2.17.53) for the OOM fix;
- OBD_FAIL_OST_BRW_WRITE_BULK3 uses 0x258 on this tree (0x256 is taken); OBD_FAIL_OST_CSDC_WRITE_BAD stays 0x257;
- the local enable/disable_compression definitions removed by this patch were already provided by test-framework.sh here;
- test_1123 keeps the compression_supported gate added when it was introduced;
- the local RAMKB in subtest_2000 gave way to the new file-scope RAMKB.
- test_1080 is excluded: it depends on the header-checksum change (Was-Change-Id: I24a9ab9cb7bea1208ada23aa6550127fe6a55017), which is not part of this series; it should return together with a port of that change.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: more tests for csdc

- synthetic tests to cover all simple cases like full/partial
  write/overwrite/append/read which verifies the operations
  doing same IO to tmpfs
- a test simulating a bulk error
  to ensure all bufs/pages are released
- few tests simulating read/write/truncate to corrupted chunks
- a test checking how much IO different cases need
- add own fsx with compression enforced
- add own dbench with compression enforced
- add own iozone with compression enforced (slow)

tests: cleanup sanity-compr.sh

Fix style issues in sanity-compr (whitespace, variables).
Add check_and_set_extents_dense() helper in a few places.
Re-enable test_1012 now that server-side code is fixed.

DDN-bug-id: EX-10107
Was-Change-Id: I7f5e5b759a8081efed4700d1c963afde135e690f

tests: improve sanity-compr/1000c

the existing test doesn't take into account that allocations can be
done in few racing threads, thus allocation is not serialized. thus
it can be non-sequential, but still dense.
the test should sort filefrag's output by physical location first,
then count gaps.

DDN-bug-id: EX-10763
Was-Change-Id: Ie3e1122e549da0a181f8cf5dbb0aa4304d1c3157

tests: Use less data in csdc DIO iozone test

"EX-6266 llite: DIO fallback on compressed files" forces
to switch any DIO operation to the buffered one. This behaviour
breaks any iozone expectations and the test goes too slowly.
Having such an iozone test with DIO and compression is a good
idea, but let's reduce the testing file size so this test pass
for reasonable amount of time.

DDN-bug-id: EX-10445
Was-Change-Id: I8dee83398a4e5249ad254d061bafa89240feb2e2

tests: Use less data in iozone test

"EX-10445 tests: Use less data in csdc DIO iozone test" sets to
use 1/10 of available space for DIO iozone test. The same chage
should be done for buffered iozone test too, because the test it
too slow for large disks and slow VMs.

DDN-bug-id: EX-10445
Was-Change-Id: I40ca9783826448885cb4d8d3ca889d94df26f257

csdc: execute tests if compression is enabled

"csdc: Provide finer grained enable_compression control" patch
enables CSDC by default on all archs except aarch64 and ppc64le.
No need to execute enable_compression/disable_compression for
each test, so this test changes them to compression_enabled
check that skips test if compression is disabled for the arch.

This patch forces to skip compression tests for ppc64le and
aarch64 so they stop adding noise to the test results.

DDN-bug-id: EX-8981
Was-Change-Id: I2b5deba0672f3bad028f383cf1205e66c2b05529

tests: disable compressibility checks

some tests want to mix compressible and incompressible data for
verification. EX-8355 can disable compression for a period when
incompressible data met. disable this functionality in few
tests.

DDN-bug-id: EX-10653
Was-Change-Id: I1da67af9802e92b29651b8b532ab21c3cc75dff2

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9858
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icbc13ee239a68d165b77481678665e80690564b2

68408 master J ✓B ✗T –R 0/2

LU-10026 tests: disable sanity-compr/1011

build failure (#130380)

csdc 1 unresolved XS +1 −0

53d

build failure (#130380)

1 unresolved thread(s) await your reply — last from Timothy Day

build #130380 FAILURE

no vote yet: Alex Zhuravlev

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-09-10 14:36 Timothy Day patchset comment — What's the significance of these 'upstream remarks for reference'? Differences between your in-house change vs. the version in Gerrit?

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

Remainder of a later snapshot of the test_2001 change (same original Change-Id): a single always_except for test_1011, tag renumbered to LU-10026. New Change-Id, original kept as Was-Change-Id.

Timothy Day PS1 · 2026-09-10 14:36

What's the significance of these 'upstream remarks for reference'? Differences between your in-house change vs. the version in Gerrit?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: disable sanity-compr/1011

Add sanity-compr test_1011 (grants let a compressed file fill
OST) to the always-except list; the failure is tracked
separately.

This is the remaining delta of a later snapshot of the change
that added sanity-compr test_2001.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10142
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I6dd2643c93739fd6abaeb7add27f530a57fdcf64
Change-Id: I9570a6f71145f4f75c2bb2a09f8387936c88b114

65770 master J –B ✗T –R ✗

LU-10026 osc: osc brw request compression

parked by your own -1

CR -1 csdc 15 unresolved XL +1774 −50

62d

build #130259 FAILURE

reviews: Marc Vef -1 2026-09-01 09:48

patchsets:

+1 earlier

PS1 05-08

PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 08-25

threads: 12 need your reply · 0 waiting on others · 3 own notes · 0 bot

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) The message describes a "sec: page_pools shrinker fix" (element_size / ppp_idle_idx / IDLE_IDX_MAX changes) and adds sanity-compr test_2000 gated on client 2.17.53 "for compression OOM fix", but there is no page_pools.c change anywhere in the diff, so test_2000 can't pass on this commit. A few hunks also aren't accounted for by the message: the ll_lov_getstripe_ea_info() switch to md_getattr() when filename is NULL in llite/file.c, and the LINVRNT->LASSERT change in cl_io_rw_init(). Are these meant to be part of this patch? Also, since this touches the OST_WRITE/OST_READ wire path (OBD_BRW_COMPRESSED, o_size/OBD_MD_FLSIZE), a Test-Parameters: line requesting client/server interop testing would be worthwhile.

reply PS5 2026-07-27 15:10 Andreas Dilger COMMIT_MSG:806 — (defect) this needs to add:

Andreas Dilger PS5 · 2026-07-27 15:10

(defect) this needs to add:
```
Test-Parameters: testlist=sanity-compr
```
otherwise that test session is not being run on master.

note PS6 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS6 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebased onto the current series base (65769). Notes on this patchset:

- OBD_FAIL_OSC_WRONG_COMP_ALG renumbered 0x418 -> 0x41b: 0x418 is already
  OBD_FAIL_OSC_FIEMAP on master (the previous patchset defined a duplicate
  value). sanity test_460b's fail_loc values updated to match
  (0x8000041b / 0x1000041b).
- Adapted to the reworked struct ll_compr_hdr from 65769: split magic
  (llch_magic_lo/llch_magic_hi) and the packed llch_level_and_chunk byte
  with the LLCH_COMPR_LEVEL()/LLCH_CHUNK_LUM_BITS()/LLCH_LEVEL_AND_CHUNK()
  accessors replacing the old level/chunk bitfields, in compress_chunk(),
  is_chunk_start() and osc_decompress().
- compress_chunk() now also zeroes llch_extra_flags - previously the field
  was left uninitialized in the pool-allocated buffer and written to
  storage as garbage.
- Dropped the check_ll_compr_type() call from wirecheck main(): the current
  base's check_ll_compr_hdr() already emits the identical
  CHECK_VALUE(LL_COMPR_TYPE_*) set; the separate function no longer exists.
- lov_io_slice_init()/lov_io_mirror_init() gained a const struct lu_env *env
  first argument on master; the compressed-file detection block inserted
  there was adapted to the new signatures.
- Fixed a misindented closing brace in lov_io_submit() (came in with the
  original patch).

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) 0x418 is already used by OBD_FAIL_OSC_FIEMAP two lines above, so this fault injection point collides with it (triggering one fires the other). 0x41a or 0x420 appear free.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) On the decompress path *type is llch_compr_type read from storage/the wire, so a corrupt header carrying LL_COMPR_TYPE_BEST or LL_COMPR_TYPE_FAST would LBUG the client here. The sibling change that replaced an assert with EIO ("we shouldn't assert on values read from storage") applies - return -EIO instead of asserting?

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(minor) If fill_cpga() fails here, `dst` (obtained just above from obd_pool_get_objects) has not yet been stored in bp_cmp_chunk, and the out: path only releases `src`, so the chunk buffer looks leaked on this error return.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) compressed_pages comes from llch.llch_compr_size, which is read straight out of the server reply by is_chunk_start() (that only checks llch_magic). It is bounded above only by page_count - i, not by pages_per_chunk. merge_chunk() then copies compressed_pages * PAGE_SIZE bytes into `src`, a single chunk_size buffer from obd_pool_get_objects(&src, buf_bits). A corrupt or hostile compr_size larger than one chunk overflows `src`. The LASSERT(src_size <= chunk_size) on the next line runs only after the copy (and asserting on wire data is itself unsafe). Can compressed_pages be bounded against pages_per_chunk before merging, returning -EUCLEAN otherwise?

note PS6 2026-09-01 09:33 Marc Vef osc_compress.c:101 — (defect) `pg->bp_pgno` is not set. This causes silent data corruption on native-folio kernels: the compressed output buffer can be a multi-p

Marc Vef PS6 · 2026-09-01 09:33

(defect) `pg->bp_pgno` is not set. This causes silent data corruption on native-folio kernels: the compressed output buffer can be a multi-page compound allocation, so every one of its pages maps to the same head folio. Fix below as prepared by Patrick:

```
diff --git a/lustre/osc/osc_compress.c b/lustre/osc/osc_compress.c
index a0e569c89b..b0bb2142e2 100644
--- a/lustre/osc/osc_compress.c
+++ b/lustre/osc/osc_compress.c
@@ -52,10 +52,24 @@ void free_cpga(struct brw_page **cpga, u32 page_count)
  OBD_FREE(cpga, page_count * sizeof(*cpga));
}

+/*
+ * Index of @page within @folio.  Mirrors cl_folio_pgno(), which takes a
+ * cl_page; here the page comes from the compressed output buffer instead.
+ */
+static inline int compr_folio_pgno(struct folio *folio, struct page *page)
+{
+#ifdef HAVE___FILEMAP_GET_FOLIO
+  return folio_page_idx(folio, page);
+#else
+  return 0;
+#endif
+}
+
static int fill_cpga(struct brw_page **cpga, struct brw_page **pga, char *dst,
         int src_from, int dst_from, size_t dst_size, int order)
{
  struct brw_page *pg;
+  struct page *cpage;
  int chunk_offset;
  int dst_page;
  int src_page;
@@ -98,7 +112,18 @@ static int fill_cpga(struct brw_page **cpga, struct brw_page **pga, char *dst,
     * patch.
     */
    pg->bp_count = PAGE_SIZE;
-    pg->bp_folio = page_folio(mem_to_page(dst + chunk_offset));
+    cpage = mem_to_page(dst + chunk_offset);
+    pg->bp_folio = page_folio(cpage);
+    /*
+     * The compressed output buffer can be a multi-page (compound)
+     * allocation, in which case every one of its pages maps to the
+     * same head folio.  bp_pgno selects the page inside that
+     * folio, so it must be set here: otherwise brw_folio_page()
+     * returns the folio's first page for every brw_page and a
+     * compressed chunk spanning more than one page is sent as
+     * repeated copies of its first page.
+     */
+    pg->bp_pgno = compr_folio_pgno(pg->bp_folio, cpage);
    /* we get flags from the first page in the chunk and
     * add COMPRESSED
     */
 
```

reply PS6 2026-08-26 08:52 Misc Code Checks Robot (Gatekeeper helper) osc_compress.c:216 — warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-08-26 08:52

warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

reply PS6 2026-08-26 08:52 Misc Code Checks Robot (Gatekeeper helper) osc_compress.c:233 — warn: compress_request():missing conversion: 'pga_i + chunk_len' 'unit_array_size + unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-08-26 08:52

warn: compress_request():missing conversion: 'pga_i + chunk_len' 'unit_array_size + unit_page'

reply PS6 2026-08-26 08:52 Misc Code Checks Robot (Gatekeeper helper) osc_compress.c:242 — warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-08-26 08:52

warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

reply PS6 2026-08-26 08:52 Misc Code Checks Robot (Gatekeeper helper) osc_compress.c:271 — warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-08-26 08:52

warn: compress_request():missing conversion: 'chunk_start + chunk_len' 'unit_array_size + unit_page'

tags: csdc

owner Marc Vef · uploader Marc Vef · PS6 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: osc brw request compression

This patch adds client-side compression/decompression.

The client-side data compression project (CSDC) reduces storage and
network
utilization by leveraging the more plentiful memory and CPU resources
on the
local client. Data is sent compressed over the network, saved
directly to
storage on the server side, and decompressed back on the client side.
Uncompressed data is kept in client page cache, all while being
functionally
transparent to the end user and application.

As an example, a test file is compressed and decompressed.
The resulting file is compared with the original one.

The test case shows 2.5x compression ratio:
356K	/mnt/lustre/d460.sanity/sanity.sh
884K	/tmp/cmp-46ofie/decompressed_sanity.sh

Compression should read whole chunk even if offset
and size differ.

Let's modify readahead to force reading data from the
offset and size multiple to the chunk size.

utils: fix cp_comp_type size

cp_comp_type should be 8 bits, as llch_compr_type and all associated
variables are declared as u8.
So remove useless cp_comp_enabled and fix code to test for compressed
component with cp_comp_type against LL_COMPR_TYPE_NONE.
And update LL_COMPR_TYPE_MAX value to 255 to avoid conflicts with
future compression types.

DDN-bug-id: EX-7775
Was-Change-Id: Ia15868ac0ac003b62942540a57f782226ae8c141

tests: test compression without bzip2/HDF5

Run as much of sanity test_460a compression tests as possible,
even if bunzip2 or HDF5 file are unavailable.

Print a clear message in test_84 if bunzip2 unavailable.

DDN-bug-id: EX-6127
Was-Change-Id: I36251834f636600eb9b0194ccd14c8b203da32e5

lov: refactor lov_io_lsme_at

lov_io_lsme_at needs some minor changes to be called from
lov_io_slice_init().

DDN-bug-id: EX-7601
Was-Change-Id: I0611d66052e22d349932eb26257369e07b9b8167

osc: don't check for start inside the chunk

Chunk size is the same for the whole request and every
chunk offset is multiple to a chunk size.

No need to search for compression header in every page.
It is enough to check every with offset multiple to a chunk
size.

DDN-bug-id: EX-7818
Was-Change-Id: Ie2ef645130656279e152ea1f7e6db01cb33836ca

osc: minor compression cleanups

This cleans up some style and argument issues I found made
the code a little harder to follow.

DDN-bug-id: EX-8270
Was-Change-Id: Ia3492ae79acf6c83d724cc91b0201c7872325853

osc: move common CSDC code to the library

CSDC repacks a chunk on the server side in case of the
partial rewrite. There are routines that can be shared
between client and server.

This patch moves common compression code to the
libcfs.

DDN-bug-id: EX-7601
Was-Change-Id: I824211a3435b0479f7a3b8f08598a5b567b67d3c

osc: use correct count

Using the number of bytes in the compressed page creates
gaps in the RDMA, which IB memory registration cannot
accept.

Fix this by always setting count to PAGE_SIZE for
compressed pages and otherwise using the count from the
original source page.  Setting PAGE_SIZE for compressed
pages is valid because client only does compression for
aligned IO, except for the trailing chunk.  For the
trailing chunk, the file size is set on the server, so any
trailing bytes are ignored.

DDN-bug-id: EX-8245
Was-Change-Id: Ied89d3ac328fb6020079392f5a8812ad5637b4a4

csdc: remove holes from struct ll_compr_hdr

This patch reorganizes struct ll_compr_hdr to remove
alignment holes.

DDN-bug-id: EX-8353
Was-Change-Id: I59800b00e3a17972d621bae21ba06509a39b1036

osc: apply compressed flag to dst page

The existing code to apply brw flags to compressed pages
has two issues:
1. The dst_page is NOT an osc async page, it is a bare BRW
page.  This means the brw_page2oap macro isn't right,
because there is no oap page.
Because oap_brw_flags is actually oap_brw_page.flag, we
don't ever access the memory pointed at by OAP, just use it
to find an offset back in to the brw page.

This means the flags are set correctly, but we still
shouldn't use this macro.
2. However, the function then overwrites these flags by
copying from a page in the source, so OBD_BRW_COMPRESSED is
lost.

Add OBD_BRW_COMPRESSED when we set flags.  This ensures the
flag is actually sent to the server on compressed IO.

This was not causing any problems because the server does
not actually use the OBD_BRW_COMPRESSED flag yet.
(EX-7601 uses this flag)

DDN-bug-id: EX-7601
Was-Change-Id: Ia94cdc803868ce16a0b66fd58578ec8b2d00cbae

osc: remove unused 'wrkmem'

compress_chunk() takes a wrkmem buffer, which it does not
use.

Remove this and its allocation in compress_request.

DDN-bug-id: EX-7601
Was-Change-Id: I6f236f018f5b79c57cc8725ca0f95125810a4064

osc: walk chunk unaligned RPC correctly

For decompression, the client must start looking for
compressed chunks at a chunk aligned offset.

Implement this in decompress_request.

DDN-bug-id: EX-7601
Was-Change-Id: I3273135990ddf51e8b3c651734e19350e91f659c

ofd: add obd level compression lib

Some compression functions will be used by several areas of
of Lustre, so they need to be in obdclass.

This moves merge_chunk and unmerge_chunk there and adds the
ability for them to merge lnbs.  This is used in a future
patch.

DDN-bug-id: EX-7601
Was-Change-Id: If4a318119bb7685e41adb9f3b31a66074031e6ac

osc: remove cpga fill bits

cpga fill bits are not needed now that we don't support
compression and encryption.

DDN-bug-id: EX-7601
Was-Change-Id: I13c2278e085e9b288bd896585947e28e2ea505ca

osc: only set compressed flag on compressed pages

The code accidentally sets the compressed flag on all
pages processed through fill_cpga, even if they're not
compressed.  Oops.

Also stop setting pg->index on the pages in the compressed
pga, this is only used by encryption and that's no longer
supported with compression.

DDN-bug-id: EX-7601
Was-Change-Id: I313fd943a18b71cd52493852a6884f30d187e52f

osc: use pages_left in unmerge_chunk

Since we have compressed chunks < chunk_size (if they're
after EOF), we must use pages_left in unmgerge_chunk or it
will go off the end of the page array.

This also lets us remove the workaround where unmerge_chunk
would skip pages that were not present.  unmerge_chunk
always works with a known and complete set of pages, so this
check is unneeded.

We should also check that our count of bytes is correct
when we finish.

DDN-bug-id: EX-7600
Was-Change-Id: I88896307990ff839514e54e9a7e18390a457e5d8

osc: rename 'done'

Rename the ambiguous 'done' and remove it where not used.

DDN-bug-id: EX-7601
Was-Change-Id: I8fb88b7a91fcc7dbd5ce2d29a61c18330fc0cda3

osc: cleanup compression variables

Make usage of the compression variables more readable.

DDN-bug-id: EX-7601
Was-Change-Id: I6daff56b56877c8f36e02303cc0579ba7faa731b

ofd: make compress_chunk take chunk_bits

Chunk bits is used everywhere, have compress_chunk convert
to log bits rather than have the callers do it.

DDN-bug-id: EX-7601
Was-Change-Id: Ic01bb749425cb95d9c5717965d692a18138ceeb7

obd: move module load to function

This is a trivial code change to make alloc_compr a bit
shorter.

DDN-bug-id: EX-7601
Was-Change-Id: I0a790afe7afebde1d223420d9a578529da6ff7e5

osc: variable cleanup in decompress_req

Use type and lvl variables in decompress_request.

Remove an unused variable and an assert which can never
fire.

DDN-bug-id: EX-7601
Was-Change-Id: Ieff57411a2a41215fd368d731614801bd0f43e38

osc: replace assert with error

We shouldn't assert on values read from storage, instead if
they are incorrect, we should give EIO.

DDN-bug-id: EX-7601
Was-Change-Id: Icda213e3c5a90a848c9b008788e92ee49e2efcb1

osc: rearrange compress_request

A trivial rearrangement of compress_request to make it
more readable before redoing the core logic.

DDN-bug-id: EX-7601
Was-Change-Id: I1d34cd2a2a6d84bc30cc7dae8eb07586c4837f7d

osc: give compress_request explicit success

Compress_request has explicit failure handling, but the
success handling just follows the failure handling.  This is
confusing - on failure, we do:
page_count = *pcount
then immediately do:
*pcount = page_count

It also sets *orig_pga = pga on success OR failure, which
is wrong because compress_request may have modified pga and
then failed.

DDN-bug-id: EX-7601
Was-Change-Id: I121ec71cfe35babc4a572951e93f7581887ade80

osc: remove &pga usage in compress_request

The usage of 'pga' and '&pga' in compress_request is
confusing, but also, compress_request modifies &pga by
allocating a new compressed page array.  Except if we fail
in compress_request, we free that new page array.

This means failing in compress_request replaces 'pga' with
a pointer to freed memory.  Instead, create an explicit
cpga pointer in the caller and use that.  This allows
compress_request to fail safely.

DDN-bug-id: EX-7601
Was-Change-Id: Idaf592103c57b0e9ce76ab520a69b819d4f37be9

obd: move type switching to alloc_compr callers

The code is much cleaner if we can eliminated applied type
and handle that issue once per compression or decompression
rather than for every chunk.  This requires moving the type
switching inside alloc_compr.  (Also improve some error
messages - alloc_compr can fail with ENOMEM as well.)

The compression code currently allocates a transform for
every chunk on the client.  This is relatively cheap, but
it also complicates the code by repeatedly checking if a
particular compression type is supported (this is the
"applied type" code).

Moving alloc_compr to compress/decompress request makes the
code much simpler.

DDN-bug-id: EX-7601
Was-Change-Id: I162e81577db721a9715d57b3f262fcabbcbf308a

osc: rename pages_in_chunk

Chunks can have variable numbers of pages in them.

DDN-bug-id: EX-7601
Was-Change-Id: If199d777367569e62c21305f6e4b9f3e4cce6d06

osc: allow multiple chunks in read

It's rare, but reads can sometimes have multiple
discontiguous chunks.  Update decompress_request to
handle this case.

DDN-bug-id: EX-7601
Was-Change-Id: I880af95db285dce76db3610e8140a0f54baa401b

ofd: do not overwrite rc in unmerge_chunk

unmerge_chunk should not be responsible for setting the
lnb rc, because this overwrites the result of any previous
activity on the lnb.  Plus, unmerge_chunk can't fail.

DDN-bug-id: EX-7601
Was-Change-Id: Id1ce590c7f1da3ab7faddbd685d264a33c08d639

osc: calculate compressed size reduction accurately

Compression reduces space used if it results in allocating
at least one fewer block on disk.  Modify the checks in
compress_chunk to reflect this, rather than using the
simpler "reduce size by at least 4K" calculation.

Also do not attempt to compress chunks if they are less
than 4K in size, since they can't possibly get a space
benefit.

This improved my measured ratio on a version of the Linux
kernel source data set from 1.24 to 1.56, so this is
significant for datasets with many small files.  (This
version of the source had large incompressible files
removed, to focus on smaller files.  The unmodified data set
would not improve as much.)

Note this is still short of our estimates, so either the
estimate or Lustre still needs adjustment.  TBD.

DDN-bug-id: EX-7601
Was-Change-Id: I815706914b88de4f532a674d773769aa3a64d218

osc: rewrite compress_request

The existing version of compress_request can't handle
discontiguous RPCs.  Rewrite the logic to handle this
case properly.

This also implements kms handling.

If a write chunks ends at the known minimum size, we know
this write is after all other data in the file and so
there is no compressed data under it.  This means we can
compress this chunk.

DDN-bug-id: EX-7601
Was-Change-Id: I8a912d9e279d04c8ff07de39e63a1ec9b490d921

osc: handle partial chunks in decompress_request

Now that we have compression for incomplete chunks at the
end of files, decompress_request needs to handle these
chunks.  This patch modifies it to understand compressed
chunks which are less than chunk_size pages.

DDN-bug-id: EX-7601
Was-Change-Id: I877550fa0d418def406e0308392a5336ec9f3ab6

osc: add check to decompress_request

decompress_request should check to see if there's room in
the RPC for the decompressed data, since this can occur if
there's a bug or data corruption, and otherwise we will
go past the end of the RPC during decompression.

DDN-bug-id: EX-7601
Was-Change-Id: Ib1bf19bf39701b72f0f5a61b2aaff2f2fdad1897

osc: debug fix in decompress_request

Debug message had an incorrect subtraction.

DDN-bug-id: EX-7601
Was-Change-Id: I5daf360766ca77b98dc5af3d72c42ac38f5782bc

obd: add 'lvl' for best and fast

'best' and 'fast' compression types must also set a level,
because not all levels are supported by all algorithms.

Rather than trying to be clever, just use simple universally
supported values, except for lz4fast, where we special case
this, because otherwise '0' is the slowest setting (and
lz4fast is likely to remain our default fastest).

DDN-bug-id: EX-6269
Was-Change-Id: I7c29659d4f027af2e44285ae38e4c9d91e35509a

osc: decompress with algorithm from server

Data may not be compressed with the compression type and
level from the layout, so we must use the compression type
and level from storage for decompression.

DDN-bug-id: EX-6269
Was-Change-Id: Ib4cdccf294ef631a25147413d7f5c1a847c9504e

osc: handle different compression types

Allow the client to handle different compression types in a single
component.  This shouldn't happen normally, but it may happen in
the future if there is dynamic compression algorithm selection for
"fast" or "best" types (e.g. compress based on available CPU and
network bandwidth or RPC backlog).

DDN-bug-id: EX-6269
Was-Change-Id: Ide2731c60a68584e7cbb474bee88a17e9a7b8fec

osc: add COMPR_GAP check to compress_request

Currently, compress_request will build the compression
buffer (calling merge_chunk()) for requests which are less
than the minimum compression gap.  This is noticed in the
compression code when it checks if there's enough data to
attempt compression, but we can do a trivial check in
compress_request() to save that work.

Also fix a few minor style things.

This is not an important fix, but I discovered it while
investigating another issue and it's trivial to resolve.

DDN-bug-id: EX-7601
Was-Change-Id: Ieb32e6297e10d229f23c58e2ef4d933ce3dda4f2

lustre: add uncompressed size to compression header

It's useful to have the uncompressed size of the data in the
compression header.  Also, we have three checksum fields -
compressed, uncompressed, and header, but in practice,
checksumming the compressed data including the header is
enough to cover all of these.

This patch cleans up all of this at the same time.

DDN-bug-id: EX-8851
Was-Change-Id: Ie82e0dbe9c862ddc88999b109cea1f27577dbbff

csdc: rename "cp_comp_*" to "cp_compr_*"

This patch renames "cp_comp_type", "cp_comp_level",
and "cp_chunk_log_bits" to use "compr" in the name
to be consistent with other variable names.

DDN-bug-id: EX-8353
Was-Change-Id: I428ff3a789b33da02832dee02f316b02d97137e2

osc: Do not iterate over chunk pages

If osc_decompress() knows chunk size, no need to iterate
other all pages in chunk. Thay can be skipped.

DDN-bug-id: EX-7729
Was-Change-Id: Ib84b060075c55c97eba9f74ef017c0a956e85b12

csdc: Fix the upper mergeable chunk pointer

If the full chunk is followed by un mergeable page, the upper
mergeable chunk pointer is occasionally set to this unmanageable page.
The chunk size is calculated wrongly then and the next condition
suggest not to compress this chunk, because its size is not equal to
the expected size.

The pointer should be moved to the first instruction after the
can_merge_pages().

DDN-bug-id: EX-9192
Was-Change-Id: I09fedc770c8bbcac4864b32372a941da5e0c7ac3

obdclass: reset bits after decompression

as uncompressed data can be less than chunk/page, but still be
visiable to userspace as a part of a sparse file.

DDN-bug-id: EX-9873
Was-Change-Id: I4114b0704fb685013f4e03cf2d80ccde2cc8c87f

osc: preserve compressed pages for OST_WRITE replay

it's incorrect to release compressed pages right after reply
as we may resend them during OST_WRITE replay.

DDN-bug-id: EX-9895
Was-Change-Id: I3edc16d6556ddd60735d2f14fe879fc0f45231d7

csdc: Fix the next available algorithm selection

Currently, if a chosen algorithm is not available, then
next preferable is chosen, but an error code is not cleared
so __alloc_compr() returns this wrong code. Data if
written uncompressed while it can be compressed with
next appropriate algorithm.

This patch adds this error code clearing. Test is
provided.

DDN-bug-id: EX-9889
Was-Change-Id: I59f65058a0fe9b108de3d4ba7cf5950f18e32204

ptlrpc: reset refcount reusing pages for new req

when a set of compressed pages are reused for a redo RPC, we have
to reset refcounter.

DDN-bug-id: EX-10061
Was-Change-Id: I16c46d857d60cf4dd0f6317190e7c99b2667e135

ptlrpc: drop extra reference to compressed pages

if the request is not replayable, so brw_commit() won't be called.

DDN-bug-id: EX-10184
Was-Change-Id: Ib29ab19b69fd15ef339cc18c5dfef17217a32cb1

osc: remove noisy warning

if we can't compress data on the client side for a reason
(e.g. too random),
we just send data uncompressed, makes sense to hide the
error message confusing customers:
kernel: LustreError: 936336:0:(lustre_compr.c:345:compress_chunk())
	exafs-OST0007-osc-ff49193c293f3800:
	Compression failed, type 5, lvl 5, -22

DDN-bug-id: EX-10593
Was-Change-Id: Icbc8be38b0372f8f67690e7d94384a16ab02cbe4

osc: fix osc_compress ENOMEM handling

Send already compressed chunks in compress_request
even if can't allocate memory for the next chunks.
This patch also fixes statistic in case of ENOMEM.
Without that write_chunks_incompressible hasn't been
incremented in case of ENOMEM.

Add sanity-compr_2002 to check write_chunks_compressible
and write_chunks_incompressible in case of ENOMEM

DDN-bug-id: EX-11035
Was-Change-Id: I3fc58ece6cf22582166ea77e4a217bc7a9efd77d

csdc: remove buffer double sizes

Pools now (POOL_ADD_PAGE is defined) allocate
additional page for memory misuse debug, so
no need to allocate double-sized buffers to
prevent lz4 decompress problem.

DDN-bug-id: EX-8276
Was-Change-Id: Iea04943ae0f95c8983c5ba7ec9a2c63003afe351

utils: fix LL_COMPR_TYPE wire checks

Fix LL_COMPR_TYPE* wire checks to avoid duplication.
The value of LL_COMPR_TYPE_UNCHANGED is also declared as 255 so that
is does not conflict with other potential real compression types in
the future.

DDN-bug-id: EX-7775
Was-Change-Id: Iab9830f09f0778e1e1f3b1ea4c9878ce1017de8d

sec: page_pools shrinker fix

page_pool shrinker should consider ppp_free_pages
as a number of memory regions with the size of
2^pool_order pages. Otherwhise it startis shrinking
only when becomes too large, for example 2GB instead
of 64MB. It often might be the reason of OOM especially
when compressed files with different chunk-sizes
have been written. In a such case several pools
created for different chunk sizes might hold gygabytes
of memory without any chance to release it.

Fix element_size to return correct element size instead
of number of pages. Because of that elements in
page_pools had below sizes:

  order 0 = 4096
  order 1 = 2
  order 2 = 4
  ....
  order 17 = 131072

This made possible to have elements with sizes 2,4,8,...
bytes that have been allocated with OBD_VMALLOC.
Now all sizes are miltiple of PAGE_SIZE:

  order 0 = 4096
  order 1 = 8192
  ...
  order 5 = 131072

Change the logic around ppp_idle_idx.

1. Recalculate ppp_idle_idx in __sptlrpc_pool_put_pages the
   same way as it done in __sptlrpc_pool_get_pages. It is
   possible that ppp_idle_idx might become 0 or very small
   (less than 10). If there is no new allocations it would
   be impossible to free anything from the pool during 40
   seconds(CACHE_QUIESCENT_PERIOD) despite the large amount
   of free elements ready to shrinking.
2. In pool_shrink_count when ppp_idle_idx == IDLE_IDX_MAX,
   it means no one accessed pool for CACHE_QUIESCENT_PERIOD
   (40 seconds), i.e. we may release as much as possible
   pages. Earlier in a such case pool_shrink_count always
   returned 0. Another words it was impossible to shrink
   anything from the pool if there was no allocations for 40s.

Add sanity-compr 2000 that reproduces the OOM problem on
a client without this patch.

DDN-bug-id: EX-10094
Was-Change-Id: I6b55ba67b0d21cdffdb57034e8e66063745f796e

osc: Add BRW_COMPRESSED flag to reads

We need to add the BRW_COMPRESSED flag to reads so servers
can know if the client is able to decompress data.

This lets servers decide if a client can be sent compressed
data and the result won't be nonsense/corruption.  This is
important for future support of GPU direct, where the
server will need to do the decompression.

DDN-bug-id: EX-6269
Was-Change-Id: I36b5b73f983ce8f2e5297c3e9dc778a5eca54e6a

osc: don't discard decompress_request error

The error handling for decompress_request is unusual - non-zero
returns are just discarded. And rc2 is just discarded.
The read() doesn't fail or get a short read.

Fix this so if decompression fails with an error.

DDN-bug-id: EX-7807
Was-Change-Id: Idd01947c7375c9586a64f064dd6ee0ac2308ea86

osc: save compressed object size

CSDC uses a sparse file feature. A client writes compressed data
chunks to the original offsets so the same data is expected to be
read from the same offsets.

There are no writes after the last compressed chunk, so no "hole"
after the last compressed chunk.

Compressed file size (based on OST objects size) is smaller than
the original on the "original last chunk size - compressed last
chunk size" delta.

Object size should be set to uncompressed size. This size is used to
calculate file size and to remove the workaround of not compressing
the last chunk in the file.

DDN-bug-id: EX-7600
Was-Change-Id: I387c282e1cf788c3b8f6230ef555d73ffffe49c1

osd: save compressed object size on zfs

"osc: save compressed object size" added means to transfer
object size to the osd and added ldiskfs support.

This patch adds saving objec size to the ZFS backend.
Currently this fix submitted as separete patch, for
testing purpouse, but can be marged to the main patch latter.

DDN-bug-id: EX-7600
Was-Change-Id: I99e29e3f756a070b5f3cece12c4ca58f668a2ecf

ldiskfs: fix detection of compressed extent

The code in ldiskfs_map_inode_pages which detects a
compressed extent checks the first lnb for that extent, but
it's possible for some lnbs and not others to be compressed
in a given extent, so we must check all of them.  This
occurs when multiple writes have been combined in to one
RPC.

If we don't detect compression correctly, we won't set the
file size correctly and we'll get data corruption.

DDN-bug-id: EX-7601
Was-Change-Id: I11d50bdc45c40d93bb1b829fcd930165b7626432

llite: DIO fallback on compressed files

Fully supporting direct I/O on compressed files is tricky
because we cannot pull the full chunk in to the page cache
(because there is no page cache for DIO).

So instead we fall back to buffered I/O for DIO on
compressed files.

This patch adds the check and a test for this.

llite: allow aligned DIO with compression

DDN-bug-id: EX-6127
DDN-bug-id: EX-7601
Was-Change-Id: I8224ef9b8ad1d912d8a11eccad37d3dff8dd8498

If a DIO is fully aligned to compression chunk boundaries,
it is safe to do on a compressed file, so allow it.

Was-Change-Id: If8fa3397c9424254538738f4d77f9f50d1c21129

lov: set IO property check in lov_io_init_composite

Move compression IO check down to lov_io_init_composite().

DDN-bug-id: EX-9853
Was-Change-Id: I6d4ce3388d8eef788d31dabf4b3a5dd5eff5091f

llite: DIO fallback on compressed files

Direct I/O on compressed files is still not perfect for now,
so we'd fall back to buffered I/O for DIO on compressed files.

DDN-bug-id: EX-6266
Was-Change-Id: I2255638e4fbde766abf061989d07b37ad9f369a1

llite: Compute compression chunk ranges

Determine the edges of any leading and trailing compression
chunks touched by this IO and store them in the cl_io
struct.

The functionality in this patch also allows us to adjust
the lock and read rounding to do them more intelligently,
this will be done in a future patch.

DDN-bug-id: EX-7601
Was-Change-Id: I526563ea347fb0246f97f3532b823c4345c3fa27

llite: round LDLM lock requests to chunk

When we do IO with compression, we may need to 'fill' the
compression chunk, reading up pages which have already been
written to storage, so we can compress the whole chunk.

Doing this safely requires that any dlmlock we're using
always covers the full chunk.

The easiest way to do this is to round the entire locking
process to include leading or trailing compression chunks.

DDN-bug-id: EX-7601
Was-Change-Id: I3c365844561d0da909e6290f4b58ef2211c2d255

csdc: is_chunk_start should return header copy

In is_chunk_start()

*ret_header = header;
...
kunmap_atomic(header);

ret_header is used after is_chunk_start(). The header
copy should be returned from is_chunk_start() for safe work.

DDN-bug-id: EX-9878
Was-Change-Id: Ib5e828d6b61e90dcd70c28589931a4490cf19c22

Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9b41ab815db3df9ad7bdea5fca4c093cbda8814b

68385 master J !B ✗T –R 0/2

LU-10026 utils: lfs setstripe adds -E with -Z

build failure (#130357)

csdc 1 unresolved S +22 −5

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130357 FAILURE

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- test_1014 is anchored after the 1013x tests (numeric order; the original anchored it after test_1002, but this series adds 1013a-1013e in between).
- Version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: lfs setstripe adds -E with -Z

When specifying a layout with "lfs setstripe -Z" it will ignore
this option if no PFL component is specified with "-E".

Instead, "lfs setstripe -Z" should automatically upgrade the file
layout to a PFL layout so the compression parameters are saved.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9708
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I29cc373fabd352d6f8b6781c238806b75cce7057

68389 master J !B ✗T –R 0/2

LU-10026 csdc: inherit CSDC params unless overriden

build failure (#130361)

csdc 1 unresolved L +258 −25

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130361 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- Rewritten onto the current naming: LL_COMPR_TYPE_SPEC_NONE = 9 added (MAX moves to 10); wire asserts extended accordingly (this tree carries LL_COMPR_TYPE_MAX asserts the original did not have).
- compr_type_table: the NONE row loses its name, SPEC_NONE row ("none") added before the MAX sentinel - the compress_type_2str()/parse helpers were already in the matching form.
- The inheritance-hint struct uses cd_compr_chunk_lum_bits with this series' lum-bits semantics (offset above COMPR_CHUNK_MIN_BITS - no +16 at use sites), and lod_adjust_compr_chunk_size() is called in its take-by-value/return-by-value form.
- lod_qos_parse_config: the convert_plain arm skips all lcme-derived assignments including fields this tree has that the original context did not (llc_mirror_link_id, llc_dstripe_count, llc_cstripe_count).
- SPEC_NONE clearing blocks in lod_parse_striping/lsm_unpackmd applied at the current field naming.
- test_1000d version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: inherit CSDC params unless overriden

If a parent directory has been set CSDC parameters, the file created
under it needs to inherit the CSDC parameters unless another CSDC
parameters or an empty CDSC has been specified for a component of the
file. The file create method includes touch, setstripe a plain layout
file, or setstripe a composite layout file.

As "-Z none" would be used to set compression type to 0 and server
would treat it as no CSDC parameters has been set, so "-Z none"
can not be used to stop inherit parent directory's CSDC inheritance,
so this patch introduces another compression type
LL_COMPR_TYPE_SPEC_NONE to associate with "none" compresstion type,
and it can be used to stop the CSDC inheritance from parent
directories, when user uses "-Z none" to reset compression type,
this patch would set the compression type to LL_COMPR_TYPE_SPEC_NONE.

If we use patched new client to create a file with "-Z none" upon old
servers, the old servers cannot recognize this new
LL_COMPR_TYPE_SPEC_NONE compression type, and it just records the
type as is in the layout.

On old clients "lfs getstripe" will show "unknown" to the component's
lcme_compr_type, and on new clients, "lfs getstripe" will show "none"
to the lcme_compr_type.

When we write data to this file, both clients work but with different
behavior. On old client, with "unknown" compression type, it chooses
a fallback algorithm to compress the data, twisting the original
intention not to compress the component; while on new client, it will
treat "none" component as uncompressed one as intended.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10201
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibbdb3f6489cf7e19bfebe075bf88580f39ed79e4

68392 master J ✗1B ✗T –R 0/2

LU-10026 lov: overstriping + compression is OK

build failure (#130364)

1 unique failure csdc 1 unresolved XS +6 −0

66d

build failure (#130364)

janitor: 1 test failure unique to this patch — sanity-slow@zfs:test_64b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-slow@zfs:test_64b	seen in 2 other reviews

build #130364 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The lov_pattern_supported() hunk re-applied against the current helper (which has a RAID0|PARITY row and no MDT|COMPRESS row).
- lov_pattern_available() postdates this patch's original base and needed the same OVERSTRIPING|COMPRESS row added - without it the pattern is "supported" but never "available" and setstripe fails with -EINVAL.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: overstriping + compression is OK

it should be OK to enable compression on overstriped files.

lov: extend supported normal striping

where compression + overstriping is OK

DDN-bug-id: EX-10854
Was-Change-Id: I2474329d5b1d8684353efe8423f225e7ee779da3

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10854
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I45d14b3fc5badcb04652df2b64524c143b45c528

68394 master J !B ✗T –R 0/2

LU-10026 csdc: Added client parameters for compression types.

build failure (#130438)

csdc 1 unresolved L +243 −50

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130438 FAILURE

no vote yet: Chakshu Kansal

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The original's common target-side helpers are seq_file/procfs-based; on this branch mdt.*.compress_types and obdfilter.*.compress_types are sysfs attributes, so the common helpers were adapted to a mask-based sysfs form (lprocfs_compress_types_mask_show/store in lprocfs_status.c) and the mdt/ofd show/store delegate to them (renamed with mdt_/ofd_ prefixes to coexist with the new common client compress_types_show).
- The client-side compress_types_show (osc/mdc, reads ocd_compress_types, "disabled" without the connect flag) is as in the original.
- Tests 1013d/1013e anchored after test_1013c. The MDC-specific connect-flag check in test_1013e is kept as-is: it probes the MDC import, and compression_supported() probes the OSC import, so it is not a substitute (MDT-side compression negotiation stays disabled).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: Added client parameters for compression types.

Added new parameters 'osc.*.compress_types' and 'mdc.*.compress_types'
on the client side to report the list of available compression
algorithms.

DDN-bug-id: EX-12289
Was-Change-Id: Iba832e5351fc159e49916df2c06b75ad5ad37638

Test-Parameters: forjanitoronly
DDN-bug-id: EX-12289
Signed-off-by: Chakshu Kansal <ckansal@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I07cd0531e9c2934c28dd999b21310d18a988e2f7

68406 master J !B ✗T –R 0/2

LU-10026 llite: skip layout changes if update is held

build failure (#130378)

csdc S +15 −0

66d

janitor run: 9 failing config(s), none unique to this patch — janitor results

build #130378 FAILURE

no vote yet: Alex Zhuravlev

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: skip layout changes if update is held

skip minor layout changes (like incompressability) when the client
helds UPDATE lock as MDT grabs UPDATE lock for this kind of changes.

llite: do not update layout from fsync

fsync can be called from within write. in this case there would be one
active IO (from LOV's point of view). if we update the layout at this
point, then fsync would need to refresh the layout to proceed, but
given there is an active IO, LOV would block layout refresh waiting
for all IOs to finish - a dealock.

DDN-bug-id: EX-10979
Was-Change-Id: I6d1cd7787f4c0d9f75f01030ed1235279bb6d301

llite: do not update layout at umount

umount can race with ptlrpcd finishing IOs and locks.
at the same time, umount resets sb->s_root to NULL which
is needed to form new RPCs, so it's safer to skip minor
layout update.

DDN-bug-id: EX-11166
Was-Change-Id: Ie9df33e95317bf38452ddd3afdfa30b31932bdb4

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10857
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I52e31a9c15967dd92714e6102e6b16d22695f81b

68410 master J !B ✗T –R 0/2

LU-10026 llite: issue compressiblity change in workqueue

build failure (#130382)

csdc 3 unresolved M +118 −56

66d

build failure (#130382)

2 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130382 FAILURE

no vote yet: Artem Blagodarenko, Zhenyu Xu

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The test_1021 tweaks (sleeps, echo moves) re-applied against the unwrapped assertion blocks; ci_need_change_layout renamed as in the original.

reply PS1 2026-08-26 13:39 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:292 — warn: vvp_layout_minor_change():passing zero to 'ERR_CAST'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:39

warn: vvp_layout_minor_change():passing zero to 'ERR_CAST'

reply PS1 2026-08-26 13:39 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:294 — error: vvp_layout_minor_change():'vlcw' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:39

error: vvp_layout_minor_change():'vlcw' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: issue compressiblity change in workqueue

This patch uses kernel-global workqueue to handle the file's minor
layout change RPC equest. At vvp_io_fini() when it find that the
file's compressibility should be changed, it creates a work and
insert the request to the kernel workqueue.

This would prevent the layout minor change RPC issued from within an
IO, and it could avoid the possible double lock the LOV conf lock.

lov: replace switch compressibility

The switch compressibility flag results in weird behavior -
it tells us we are to flip what's there, but results in some
very hard to read code.

I *think* the existing code is probably correct in absence
of race conditions, but an instruction to "flip" rather than
set is very confusing and could easily do the wrong thing
if there's a race which changes the compressibility value.

Replace it with two explicit bit flags.

DDN-bug-id: EX-10857
Was-Change-Id: I3b1d9f8d635e0bdbe67d5806127727a7264d26be

Singed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10857
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic4345b1387288ab11ce9061f40b056b07cb976bf

68364 master J !B ✗T –R 0/2

LU-10026 ofd: avoid assertion on large read request

build failure (#130339)

csdc 5 unresolved S +24 −5

66d

build failure (#130339)

4 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130339 FAILURE

no vote yet: Artem Blagodarenko

threads: 4 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Commit message no longer references a specific downstream release version for the affected old clients.

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:500 — warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:504 — warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_read():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:684 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) ofd_compress.c:688 — warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

warn: ofd_map_chunks_to_lnb_write():missing conversion: 'pages + maxlnb' 'unit_page + unit_array_size'

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: avoid assertion on large read request

In case an old client using CSDC sends a
read IO that will overflow buffers on OST, OST should
return an error rather than crash.

A similar check was previously added to the write path,
but was not added to the read path.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10359
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I7dc4c1a7a19c71d757d0a505003c3012edbf987a

68366 master J !B ✗T –R +1/2

LU-10026 ofd: skip reading compressed tails

build failure (#130341)

csdc 2 unresolved M +37 −28

66d

build failure (#130341)

1 thread(s) waiting on others

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130341 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:02

threads: 0 need your reply · 1 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied on the integration patch with only the series' field naming (ol_compr_chunk_lum_bits) where the original said _log_bits.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: skip reading compressed tails

if a tail is being overwritten, then we don't need to read
and decompress old content.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10116
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Icde88d9fbd0ba0bdf53497c0c14d1215fa9cad73

68363 master J !B ✗T –R +1/2

LU-10026 ofd: reject non-aligned compressed niobufs

build failure (#130338)

csdc 1 unresolved S +7 −3

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130338 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 10:24

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied on the integration patch with only the series' field naming (ol_compr_chunk_lum_bits) where the original said _log_bits.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: reject non-aligned compressed niobufs

a client can send non-aligned compressed niobuf by mistake,
OST should reject it with an error.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11092
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2a61f13b5be17ca22c3a92b7d5b7b8bfd739c1eb

68365 master J !B ✗T –R 0/2

LU-10026 csdc: finer enable_compression control

build failure (#130340)

csdc 1 unresolved S +20 −0

66d

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130340 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Note for reviewers: the #if (PAGE_SIZE > 4096) guards in userspace (liblustreapi_layout.c) rely on the build host's page size, not the runtime client's; probably wants a runtime sysconf(_SC_PAGESIZE) check. Kept as the original for now.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: finer enable_compression control

On all architectures other than aarch64 and ppc64le enable_compression
is now enabled by default. lfs warning message is gone.

To use CSDC on aarch64/ppc64le (on your own risk)
llite.*.enable_compression=1 should be set. lfs
set_stripe command still prints a warning message in this case.

Test-Parameters: forjanitoronly
Test-Parameters: trivial
DDN-bug-id: EX-9074
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic8edc5bbeb8f9a3cd34ad3fc4e8c78e59f4cc34f

68367 master J !B ✗T –R +1/2

LU-10026 ofd: don't write blocks after compressed size

build failure (#130342)

csdc S +11 −0

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130342 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:00

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: don't write blocks after compressed size

due to limits in ib/rdma/lnet, client has to send full pages.
if client's pagesize is bigger then server's pagesize, then
we want to skip writing useless bytes from those full pages
containing compressed data.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10712
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I61c6fe72af1d2e8679d9fde303a561e80a287746

68368 master J !B ✗T –R 0/2

LU-10026 csdc: tune allowed compression type on server

build failure (#130343)

csdc 1 unresolved L +258 −18

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130343 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- The patch re-created a libcfs_string.h/.c pair to host string-mask helpers; master already provides cfs_str2mask()/cfs_mask2str() with u64 masks (lnet/libcfs/debug.c), so both files were dropped and the new compress_types stores call the existing helper (~0ULL masks). A hunk that narrowed the existing capability-mask store from ~0ULL to ~0 was dropped for the same reason.
- mdt.*.compress_types and obdfilter.*.compress_types are implemented as sysfs attributes (the target-side procfs vars tables the original patch extended no longer exist on master).
- The obd_compress_types mask enum / COMPRESS_NAMES use the series' type naming (lz4 for the HC type) and already include zstd, which is a valid layout type at this point in the series.
- OST-side only, as in the original: OBD_CONNECT2_COMPRESS is advertised in OST_CONNECT_SUPPORTED2 and requested by the OSC; MDT enabling remains pending (test_1013b keeps its unconditional skip; its old-version negative-check block, meaningless on master, was dropped).
- compression_supported() in test-framework.sh now probes the OSC import for the compressed_file connect flag instead of comparing versions. (The OSC import, not the MDC one, because MDT support is not enabled.)
- test_1013a/b were placed before test_1015 (the tests they were anchored to arrive later in the series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: tune allowed compression type on server

Use lctl get_param {mdt|obdfiler}.*.compress_types to list supported
compression types on server.

Use lctl set_param {mdt|obdfilter}.*.compress_types="+gzip-lzo" to
add gzip to and delete lzo from existing compression types on server.

Server will negotiate supported compression types with client in
ocd_compress_types and client import stats could show the negotiated
supported compression types in "lctl get_param {mdc|odc}.*.import"
connect_data section:

import:
  ....
  connect_data:
     ....
     compress_types: [ fast,best,gzip,lz4fast,lz4hc,lzo ]

The OST support for the connect flags is enabled in this patch, but
MDT enabling is pending unaligned compression support for DoM files.

test: refine sanity-compr.sh/test_0a

Only change and verify compression types negotiation between client
and OST0000.

DDN-bug-id: EX-9802
Was-Change-Id: I307f3826973f48791b7fbf4cdfa121401b059b0e
Was-Change-Id: Ic22685d2b21e872c01d90f2c3062b5d0d9b92ec2

test: sanity-compr.sh/test_0a failure

* move sanity-compr.sh test_0a/test_0b to test_1013a/test_1013b
* explicitly specify client's hostname to get client's negotiated
  compression types, as test environtment could mount several clients

DDN-bug-id: EX-9802
Was-Change-Id: Ic091f55a59ad9cfdaf3a94a58d44b372070c7875

Test-Parameters: forjanitoronly
DDN-bug-id: EX-6269
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I18943352e25ed9d5fe82442df9f00a7ef388f242

68369 master J !B ✗T –R 0/2

LU-10026 llite: set STATX_ATTR_COMPRESSED in ll_iocontrol()

build failure (#130344)

csdc 1 unresolved S +10 −0

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130344 FAILURE

no vote yet: Jian Yu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

LLIF_COMPR_FL renumbered 6 -> 8 (bit 6 is LLIF_ACL_VALID on master). Subject shortened to fit the 62-column limit.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: set STATX_ATTR_COMPRESSED in ll_iocontrol()

This patch extracts the compression flag LUSTRE_COMPR_FL from
mbo_flags and set STATX_ATTR_COMPRESSED flag in ll_iocontrol()
to please lsattr and other e2fsprogs tools.

Test-Parameters: forjanitoronly
Test-Parameters: trivial testlist=sanity-compr
DDN-bug-id: EX-8669
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I14d2082a6719a1ca5708f7aef7a2fb0f085ca63c

68374 master J ✗2B ✗T –R 0/2

LU-10026 csdc: store compression info in FID EA

build failure (#130348)

2 unique failures csdc 1 unresolved M +37 −65

66d

build failure (#130348)

janitor: 2 test failures unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b, sanity-quota@ldiskfs+DNE:test_80

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 4 other reviews
sanity-quota@ldiskfs+DNE:test_80	seen in 73 other reviews

build #130348 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Applied cleanly. Note for reviewers: this changes where the PFID EA is stored for small inodes (no longer embedded in LMA for <=256-byte inodes) - needs interop/lfsck review on master.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: store compression info in FID EA

Store compression information in OST-object's FID EA, and lfsck could
use it to recover the MDT-object layout EA from orphan OST-object(s).

2.15 Lustre may embed PFID and layout stripe info in LMA EA, this
patch would clear them from LMA EA and store them with compression
info directly into FID EA thereafter.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8038
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iacac04601b73f85d9bc057b8dd34a5004248dac4

66612 master J !B ✗T –R 0/2

LU-10026 csdc: sending compression info to server

build failure (#130320)

csdc 4 unresolved M +100 −13

66d

build failure (#130320)

3 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 11 failing config(s), none unique to this patch — janitor results

build #130320 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 3 need your reply · 0 waiting on others · 1 own notes · 0 bot

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This is an on-wire change: the client now fills bytes in the obdo (formerly o_padding_4) that are sent to the OST/MDS. Test-Parameters: forjanitoronly doesn't exercise interoperability. Consider requesting old-server/new-client and new-server/old-client runs with serverjob/serverbuildno and clientjob/clientbuildno so the padding-reuse stays wire-compatible with unpatched peers.

note PS2 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS2 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase notes: kept the base's comment wording for the chunk-bits field
(2^(16+chunk_lum_bits)); the lov_lsm2layout() wrapper is added alongside
the base's compr_name_from_type() helper in lov_internal.h.

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This comment reword looks unrelated to sending compression info to the server; is it meant to be in this patch? Also the code computes the chunk size with COMPR_CHUNK_MIN_BITS (COMPR_GET_CHUNK_SIZE()), not LOV_MIN_STRIPE_BITS; they happen to both be 16, but referencing the constant the code actually uses would be clearer.

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

This isn't a bug, but after this patch lov_lsm2layout() has no remaining callers (both lov_io.c and lovsub_object.c now call lov_lsm2layout2()). Folding the olc parameter into lov_lsm2layout() and updating those two call sites would avoid the version-suffixed name and the now-dead wrapper.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: sending compression info to server

Client fills in layout compression info into obdo and passes it
to server.

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ieb5d7b3609da41f35f8622ed6116f19ce7567ddb

68371 master J ✗1B ✗T –R -1/2−

LU-10026 osc: Add counters for compressed data

build failure (#130346)

1 unique failure csdc 1 unresolved L +544 −265

66d

build failure (#130346)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 5 other reviews

build #130346 FAILURE

no vote yet: Sebastien Buisson, Artem Blagodarenko, Zhenyu Xu, Vitaliy Kuznetsov

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- The removed dead decompression helpers (decompress_chunk_in_lnb / decompress_rnb) were deleted against their current (folio/accessor) form.
- obt_compr_stats placed after the histogram member master added since.
- The osc stats_compr file and the compress_type_fast/best entries live in the debugfs vars table; the stats_compr fops therefore use LDEBUGFS_SEQ_FOPS (LPROC_SEQ_FOPS produces struct proc_ops on kernels >= 5.6, which the CI -Werror build rejects for debugfs entries). The ofd counter-init sits next to the reworked kernel-doc of ofd_stats_counter_init().
- New tests' old-version gates converted to compression_supported; the counter-reset calls are unconditional (the parameters always exist on a server with this series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: Add counters for compressed data

This patch is the first of two patches that add counters
to track client/server-side data compression statistics.
This patch add new compr_stats file in osc.*.compr_stats.

From added counters:
1. Size of compressed/uncompressed chunks written/read by
   client to compressed files, in chunk/bytes;
2. Compressed page counter;

ofd: Add counters for compressed data

This patch is the second of two patches that add counters
to track server-side data compression statistics.
From added counters:
1. Size of compressed/uncompressed chunks read by
   client to compressed files, in chunk/bytes;

Example:
	obdfilter.lustre-OST0000.stats_compr=
	snapshot_time             1715876501.602669648 secs.nsecs
	start_time                1715873670.819396144 secs.nsecs
	elapsed_time              2830.783273504 secs.nsecs
	compressed_objects        2 samples [reqs]
	uncompressed_objects      1 samples [reqs]
	compr_bytes_read          885 samples [bytes] 557 126974 52174403
	compr_bytes_read_raw      893 samples [bytes] 16384 131104 115885042
	incompr_bytes_read        8 samples [bytes] 16384 16384 131072
	compr_chunks_read         885 samples [reqs]
	incompr_chunks_read       8 samples [reqs]

DDN-bug-id: EX-7729
Was-Change-Id: I0998b1facaffd3e84b8d9429c5674d8d9ad60020

csdc: rename client-side stats

Some CSDC client-side stats can be confusing for the uninitiated:
- write_pages_compr is the number of pages from the 'raw' file that
  could actually be compressed;
- write_pages_uncompr is the number of pages from the 'raw' file that
  count not be compressed;
- write_chunks_compr is the number of chunks from the 'raw' file that
  could actually be compressed;
- write_chunks_no_compr is the number of chunks from the 'raw' file
  that count not be compressed;
- read_pages_compr is the number of pages from the 'raw' file that
  were actually compressed;
- read_chunks_compr is the number of chunks from the 'raw' file that
  were actually compressed;

So rename those parameters as follows:

    write_pages_compr -> write_pages_compressible
    write_pages_uncompr -> write_pages_incompressible
    write_chunks_compr -> write_chunks_compressible
    write_chunks_no_compr -> write_chunks_incompressible
    read_pages_compr -> read_pages_compressed
    read_chunks_compr -> read_chunks_compressed

And also change the corresponding field names in struct client_obd to
make them more explicit.

DDN-bug-id: EX-10914
Was-Change-Id: Ia876e0cde435095493e0b7457ce7f363ea86146d

test: sanity-compr.sh/test_1001 error

sanity-compr test_1001: operand expected (error token is "+ ")

Some compression stat counters could be 0 and not shows in the
stats_compr stats, so we need to assign 0 to the counter if
necessary.

DDN-bug-id: EX-10088
Was-Change-Id: I87948a61e591680d105a8a98a4cde4fba634e5fe

tests: fix check compr_chunk in test 1020

Skip test 1020 in sanity-compr, which depends on fixes
applied to the code that were not available in older
versions.

Temporarily add an additional tolerance of 1 unit when
checking the number of chunks in test 1020 in
sanity-compr.sh

DDN-bug-id: EX-10914
Was-Change-Id: Ia1c071d754217127decbf4d4397a58b512808843

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7729
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I091153480e53309c641d39f271bef536296dc09e

68378 master J ✗1B ✗T –R +1/2

LU-10026 ofd: skip decompression

build failure (#130352)

1 unique failure csdc 2 unresolved XS +5 −0

66d

build failure (#130352)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

1 thread(s) waiting on others

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 3 other reviews

build #130352 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 10:43

threads: 0 need your reply · 1 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

One context line updated to the folio form of is_chunk_start().

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: skip decompression

we can skip decompression if new data overwrite all raw bytes of
compressed chunk.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10103
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3511c6bff2a8c4c8d98532d025beead78c597a5b

68377 master J ✗1B ✗T –R 0/2

LU-10026 csdc: process all compressed chunks in rnb

build failure (#130351)

1 unique failure csdc S +22 −12

66d

build failure (#130351)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 2 other reviews

build #130351 FAILURE

no vote yet: Artem Blagodarenko

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: process all compressed chunks in rnb

It is supposed, that every rnb contains only
continuous pages. There is always a gap after
a compressed chunk, so it is not expected to
have any other chunk in the same rnb after the
compressed chunk.

Compressing with chunk size >= than page size
is an exception to the rule above. After the
compression, data size is rounded up to page
size. So, it is possible that another compressed
chunk goes after the compressed one.

To support such a sutuation an aditional
loop was added to ofd_map_niobuf_to_chunks().

This patch is useful for ARM archs with 64k page.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ibaac18e4d046e020a7ad60b8a436c789ba4c990b

68380 master J !B ✗T –R 0/2

LU-10026 csdc: exponential backoff of incompressiblity check

build failure (#130354)

csdc 1 unresolved M +144 −90

66d

janitor run: 8 failing config(s), none unique to this patch — janitor results

build #130354 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:38 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:38

[Marc Bot] upstream remarks for reference:

Chunk-bits field naming follows the series (llc_compr_chunk_lum_bits).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: exponential backoff of incompressiblity check

If a file is incompressible and after a round of compressibility
check, if it remains incompressible, this patch would doubles the
skip bytes before running into another round of compressiblity
check.

This patch adds a state machine for update_compression_info(), the
states could be CS_INCOMPR, CS_COMPR, CS_CHECK, CS_CHECK_INC.

CS_INCOMPR    --  file is not compressible
CS_COMPR      --  file is  compressible
CS_CHECK      --  was in CS_COMPR, is now under check phase
CS_CHECK_INC  --  was in CS_INCOMPR, is now under check phase

state transition:
INCOMPR --> INCOMPR   : skip bytes has not been passed
INCOMPR --> CHECK_INC : skip bytes has been reached, in check phase
COMPR   --> CHECK     : in check phase, checked bytes has not been
                        reached
CHECK*  --> CHECK*    : in check phase, checked bytes has not been
                        reached
CHECK*/COMPR
          --> COMPR   : file is determined to be compressible
CHECK/COMPR
          --> INCOMPR : file is determined to be incompressible, reset
                        the skip bytes check length
CHECK_INC --> INCOMPR : file is determined to be incompressible,
                        double the skip bytes check length

====

Test with 4 OSTs system, target file layout is set as:
lfs setstripe -Eeof -c-1 -Z lz4 --compress-chunk=64 $tf

and src file is 336M bz2 file, using "dd if=$src of=$tf bs=4M" copy
this bz2 file to target file.

w/o the patch, copy uses about 7 seconds
** start 1724216088 **
84+1 records in
84+1 records out
352625400 bytes (353 MB, 336 MiB) copied, 3.76366 s, 93.7 MB/s
** end 1724216095 **
...7 seconds

w/ the patch, copy uses about 4 seconds
** start 1724215237 **
84+1 records in
84+1 records out
352625400 bytes (353 MB, 336 MiB) copied, 2.07946 s, 170 MB/s
** end 1724215241 **
...4 seconds

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I42105d3e137bf50897ff5c67654ca33552b23102

68379 master J !B ✗T –R 0/2

LU-10026 utils: return error on invalid compress level setting

build failure (#130353)

csdc 1 unresolved S +11 −4

66d

janitor run: 7 failing config(s), none unique to this patch — janitor results

build #130353 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:38 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:38

[Marc Bot] upstream remarks for reference:

- The strtoul/end-pointer validation this patch adds was already carried (in stricter form) by the earlier level-handling patches of this series; those hunks were dropped.
- The parse-time rewrite of low lz4 levels to the lz4fast default was removed here (matching the original series' final behaviour): the stored level is kept and the compression engine falls back to lz4fast when allocating the compressor. This also makes the negotiated-type test lists in sanity-pfl roundtrip correctly.
- The new negative test was renamed test_1013a -> test_1013c: 1013a already exists (type negotiation test); the duplicate name would have silently shadowed it (latent issue in the original series).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: return error on invalid compress level setting

lfs setstripe -Z <compress_type>:<compress_level> does not report
error on invalid <compress_level>.
An explicit low level for the lz4 type (below the lz4hc minimum) is
no longer silently rewritten to the lz4fast default at parse time;
the stored level is kept and the compression engine falls back to
lz4fast for such levels when allocating the compressor.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10186
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie794a6d4cf208a9c66d1d179c157fee756477aee

66279 master J !B ✗T –R +1/2

LU-10026 sec: disable compression for encrypted files

build failure (#130315)

csdc 2 unresolved S +10 −3

66d

build failure (#130315)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #130315 FAILURE

reviews: Sebastien Buisson +1 2026-08-26 09:47

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(style) The body explains why compression must be disabled for encrypted files, but that logic (compressed = 0; goto skip_compression) already lives in the parent compression patch. What this change actually does — throttling the existing CWARN to once per 24h and adding `always_except LU-10026 66` in sanity-sec.sh — isn't mentioned. Could the message describe the change it makes?

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) There is no test_66 in sanity-sec.sh (tests jump 65 -> 68), so this exclusion is a no-op in this tree. Which patch adds sanity-sec test_66, and should the always_except land together with it? Worth also confirming sanity-sec is the intended suite rather than sanity-compr.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 sec: disable compression for encrypted files

In case a read-modify-write io pattern is carried out on a compressed
file, it has to be handled on server side.
But because encryption cannot be done on server side for security
reasons, we are not able to handle that kind of io pattern if the file
is encrypted + compressed.
So just disable compression for all encrypted files.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I84881fb1235f015d022751d4cce2d43a7231c2b4

66280 master J !B ✗T –R 0/2

LU-10026 osc: disable CPU-access features for RDMA only pages

build failure (#130316)

csdc 1 unresolved S +27 −10

66d

build failure (#130316)

1 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130316 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) `req` was just allocated above (ptlrpc_request_alloc_pool for writes, ptlrpc_request_alloc for reads) and every other error path in this function frees it with ptlrpc_request_free(req) before returning. These two early returns leave without freeing it, and *reqp is only assigned on the success path, so the caller can't release it either:

    if (compressed && !page_access_allowed)
        RETURN(-EINVAL);
    if (inode && IS_ENCRYPTED(inode) && !page_access_allowed)
        RETURN(-EINVAL);

For OST_WRITE the request comes from osc_rq_pool, so a GPU-direct write to a compressed or encrypted file leaks a pooled request each time and can eventually drain the pool. Should both routes free req first?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: disable CPU-access features for RDMA only pages

Pages which cannot be accessed by the CPU are referred to
as RDMA only pages.  If pages cannot be accessed by the
CPU, it is impossible for us to do compression,
encryption, checksums, or short-io (data-in-RPC) on them.

This patch disables compression and encryption for these
pages and cleans up the code so checksums and short-io
are disabled by the same code.

The only user of RDMA only pages today is Nvidia's GPU
direct, so this patch disables compression and
encryption with GPU direct.

NB: We eventually intend to handle compression for
GPU direct with server side compress/decompress.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iad9311617cddf27d3ff75a17429499c573067ea0

66611 master J !B ✗T –R 0/2

LU-10026 lfs: migrate compressed file without stripe info

build failure (#130319)

csdc 2 unresolved M +76 −14

66d

build failure (#130319)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #130319 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) The diff also touches Documentation/man1/lfs-mirror-extend.1 (documenting compression options for SETSTRIPE_OPTIONS), which the message doesn't mention and which is unrelated to the migrate fix. Is that hunk meant to be part of this patch, or should it be split out / called out in the body?

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) This prints `fd`, which is always -1 on this path, so the message reads "Cannot open file X: -1" with no useful detail. The error is in errno from open() – strerror(errno) would be the informative value.

Also note the caller (lfs_migrate/lfs.c) already emits its own llapi_error() when this returns < 0, so this adds a duplicate line. Using llapi_error(LLAPI_MSG_ERROR, errno, ...) here (as the ENOSPC path below does) would decode errno and respect the message level instead of writing to stderr unconditionally.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lfs: migrate compressed file without stripe info

lfs migrate file without specifying stripe info will get layout info
from the file as the target layout template, and
llapi_layout_get_by_xattr() tries to convert LOV_PATTERN_* values
to user scope LLAPI_LAYOUT_* values, while LOV_PATTERN_COMPRESS
is missed in this conversion.

This patch add a function llapi_pattern_from_lov() to handle this
conversion specifically.

This patch also add more error messages for llapi_layout_file_open().

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I49a43cc7761cd2baed7a5da7d4e7cff2152ff9bb

66615 master J ✗1B ✗T –R 0/2

LU-10026 csdc: prohibit set compression upon encrypted file

build failure (#130322)

1 unique failure csdc 1 unresolved L +247 −35

66d

build failure (#130322)

janitor: 1 test failure unique to this patch — conf-sanity4@zfs:test_123aj

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
conf-sanity4@zfs:test_123aj	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #130322 FAILURE

no vote yet: Zhenyu Xu

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS2 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS2 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: kept the base's newer compression-parameter assignment block
in lod_use_defined_striping() (gated on LOV_PATTERN_COMPRESS, with the
extension-component chunk-adjustment deferral); this patch's encryption
checks and attr threading apply on top unchanged.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: prohibit set compression upon encrypted file

Setting compression layout component upon encrypted file is not
allowed for now.

This patch add this check on MDS in creating file with layout,
adding/merging new mirror to existing file.

Test-Parameters: forjanitoronly
Test-Parameters: testlist=sanity-sec env=ONLY=67,PTLDEBUG=-1
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I60d9f4bfce3a498f1eb3994c6276afb9d89c99a7

66616 master J !B ✗T –R ✗

LU-10026 osd: osd_bufs_put does not always handle all pages

parked by your own -1

CR -1 csdc XS +0 −2

66d

janitor run: 16 failing config(s), none unique to this patch — janitor results

build #130323 FAILURE

reviews: Marc Vef -1 2026-08-26 09:47

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

your previous vote: -1 on PS1 (2026-06-15 10:19) — now at PS3

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osd: osd_bufs_put does not always handle all pages

osd_bufs_put asserts that the dio pages used after are
always zero, but there's no reason for this to be true and
compression specifically violates this by using 1 page at
a time.

Without this patch, we hit this assert and crash when
nonrotational = 1.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: If6bdb11f254c260e2da4cabe11a82693a468e6fb

68356 master J !B ✗T –R +1/2

LU-10026 ofd: remove unused csdc bits

build failure (#130335)

csdc 1 unresolved L +150 −594

66d

janitor run: 15 failing config(s), none unique to this patch — janitor results

build #130335 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:10

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- The dt_read_prep() signature revert (dropping the decompression flag) was
  applied to both the header prototype and the function body in
  obdclass/dt_object.c (the bodies moved there upstream, LU-13139); the
  osd-wbcfs dbo_read_prep implementation lost the parameter as well.
- In ofd_preprw_write(), this patch removes the separate err_put label; the
  early "BRW to missing obj" failure (which happens after
  tgt_grant_prepare_write()) now puts the object and jumps to err_commit so
  the prepared grants are still committed.
- A whitespace-only obd.h hunk (tab/space churn) was dropped.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: remove unused csdc bits

some bits are not used with the ofd csdc integration patch
implemented, we can remove
those now. also the patch tries to sync the code with master
branch a bit.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9996
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8edb25fe322f4fafaac1b9a832dd5b4b72e52a42

66263 master J !B ✗T –R 0/2

LU-10026 osc: add compression to can_merge_pages

build failure (#130261)

csdc XS +5 −2

66d

janitor run: 5 failing config(s), none unique to this patch — janitor results

build #130261 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: add compression to can_merge_pages

Some BRW flags are OK to have on only some pages in a BRW,
others are not.  can_merge_pages has a whitelist of the
flags which are safe to have on only some pages in a BRW,
and prints a warning if other flags are seen.

Add compression to the white list, because while all pages
in an niobuf must be compressed, it is normal to have only
some pages in a BRW compressed.

Prior to this patch, this warning was printing during
normal usage of compression.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia262d4fc878e5328bd956865047e997aa77946f0

66266 master J !B ✗T –R 0/2

LU-10026 osc: clear oi_write_osclock in lock fini func

build failure (#130263)

csdc S +12 −4

66d

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130263 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: clear oi_write_osclock in lock fini func

Move osc_io::oi_write_osclock clearance in osc_lock_fini() as
it's set in osc_lock_init().

Compression IO could possibly expand lock region and
osc_lock_set_writer() could access a osc_io that is not accessed
in osc_io_iter_init(), so that osc_io_rw_iter_fini() miss clearing
osc_io's oi_writer_osclock.

This patch moves the oi_write_osclock clearance in lock fini function
to match its creation in osc_lock_init().

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ied42f5befc1abd76aa10a7666eadb9a58e1f1783

66267 master J !B ✗T –R +1/2

LU-10026 llite: getting stripe info optimization

build failure (#130264)

csdc 2 unresolved M +63 −0

66d

build failure (#130264)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 6 failing config(s), none unique to this patch — janitor results

build #130264 FAILURE

reviews: Sebastien Buisson +1 2026-08-25 21:12

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) These CDEBUG() pairs print start_idx/end_idx before and after the rounding with no descriptive text, on the D_SEC (security) mask which is unrelated to readahead. Same at lines 813 and 816. They look like leftover debugging scaffolding; if kept, D_READA would be the right mask.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) vui_ra_pages is initialized once per read syscall in vvp_io_read_start() and only reset at IO teardown, but ll_readahead() runs once per page (ll_readpage()->ll_io_read_page()->ll_readahead()). Won't this += accumulate across every not-uptodate page of a compressed read, progressively inflating the RA window enlarge at lines 806-808 and ria_end_idx_min at line 863?

Also, the delta is taken from end_idx before the EOF clamp just below (lines 819-828), so vui_ra_pages can grow past the file size. ria_end_idx_min then feeds ll_read_ahead_pages() at lines 478-479, which raises ria_end_idx without an ria_eof guard - so readahead can extend past eof_index, the opposite of the "restrict readahead to eof" goal in the commit message.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: getting stripe info optimization

ll_lov_getstripe_ea_info() is expensive call and should be avoided
if possible. Let's use cached chunk size rather than get it from
stripe info every time.

llite: restrict readahead to eof

Compressed file readahead rounding needs to come before
readahead is limited to EOF.

DDN-bug-id: EX-7601
Was-Change-Id: I4e9e7fe63301c08efcb05f170726735593a9431d

llite: fix chunk_bits usage in readahead

For the minimum compression chunk size, chunk bits is zero,
so we cannot use if (chunk_bits) to determine if we're
doing compression.

This also fixes two other things:
1. A rounding error when rounding to chunk
2. Move rounding of end_idx to before first usage of
end_idx, so calculation of number of pages is correct
Without this, when the user reads 1 page or less, readahead
will calculate the readahead page count as 0 and will exit
without reading the chunk.

DDN-bug-id: EX-6127
Was-Change-Id: I273506fd4f6ed5f0b8b5020357fd7caf0531e61c

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id08487ec782f797e242e3f673c4a4dd8d526c9cc

66275 master J !B ✗T –R ✗

LU-10026 osc: do not compress resends

parked by your own -1

CR -1 csdc 2 unresolved S +30 −2

66d

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #130266 FAILURE

reviews: Marc Vef -1 2026-08-25 21:12

no vote yet: Artem Blagodarenko

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

your previous vote: -1 on PS3 (2026-06-12 10:19) — now at PS4

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS3 2026-06-12 10:19 Marc Vef COMMIT_MSG:15 — the reason is now known and should be included here. Note for me to update later

Marc Vef PS3 · 2026-06-12 10:19

the reason is now known and should be included here. Note for me to update later

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) The resend is rebuilt from the uncompressed arrays, so besides aa_requested_nob the niobuf/page counts also change (compression yields fewer niobufs), yet only aa_requested_nob is propagated back into `aa` before `new_req->rq_async_args = request->rq_async_args` overwrites new_aa. After the resend new_aa->aa_nio_count still holds the compressed count, while the reply's RMF_RCS vector is sized from the uncompressed `niocount` in osc_brw_prep_request().

    check_write_rcs(req, aa->aa_requested_nob, aa->aa_nio_count, ...)

check_write_rcs() then only inspects the first aa_nio_count per-niobuf rcs, so a server-side write error on a niobuf past the compressed count would be skipped. aa_page_count is similarly stale. Should aa_nio_count (and aa_page_count) be updated here the same way as aa_requested_nob?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: do not compress resends

There's some issue with doing compression on resent
requests, so this patch works around it with two things:
1. Use the uncompressed page array for resend
(this was always necessary unless we modified resend to
know it already had compressed pages as input)
2. Disable compression on resend (not clear why 1. wasn't
enough)

csdc: Update async_args after resend

It is decided to send an uncompressed request on redo.
osc_brw_prep_request() processes uncompressed data and prepares
a request, so some parts of the old request are outdated.

Let's update the old request with information from the new one.

DDN-bug-id: EX-8814
Was-Change-Id: Idb1c6ee9db64cb1f2ea1c1562b1c5aae443263e3

Test-Parameters: forjanitoronly
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I5fbbdc2771f8c2c7b5c28f0b70d89b8b6015147f

68375 master J –B ✗T –R -1/2−

LU-10026 tests: Skip sanity 430c for CSDC layout

build failure (#130349)

csdc 1 unresolved XS +1 −0

66d

build #130349 FAILURE

no vote yet: Sarah Liu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: Skip sanity 430c for CSDC layout

Skip sanity test_430c until SEEK_HOLE is implemented for CSDC

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7680
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I7359262de9e3d1644d2a45b5336328bd8253f91b

68390 master J !B ✗T –R 0/2

LU-10026 mmap: add mmap support for compression

build failure (#130362)

csdc 1 unresolved L +446 −78

66d

janitor run: 31 failing config(s), none unique to this patch — janitor results

build #130362 FAILURE

no vote yet: Zhenyu Xu

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- OBD_FAIL_LOV_SKIP_CHECK_COMPR renumbered to 0x143a: the original's 0x1436 collides with OBD_FAIL_LLITE_TRUNC_PAUSE on this branch, and the original's sanity.sh test actually set fail_loc=0x1429, which is OBD_FAIL_LLITE_FORCE_BIO_AS_DIO - both the define and the test now use 0x143a consistently.
- The osd_lseek() compressed-object rejection was merged into the current form of the function (osd_get_filp_for_inode(), not osd_alloc_file_pseudo()); osd_xattr_get() prototype added to osd_internal.h.
- mmap_copy_file added to the current THETESTS line in tests/Makefile.am.
- test_1003 anchored before test_1004; its gate and the sanity.sh 460g gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 mmap: add mmap support for compression

This removes the EOPNOTSUPP for compression with mmap and
adds an mmap sanity test for compression.  This patch
removes all the restrictions for mmap, but we actually only
have unaligned read support right now, so the test is
deliberately simplified to only test reads.

A more complicated version which also tests mmap writes
comes later in the series, once read-modify-write is
supported.

The test tests mmap by copying data at several different
block sizes with several different compression chunk sizes.

tests: add mmap write test

This improves the existing mmap test to test mmap writing
as well as mmap reading.

DDN-bug-id: EX-7601
Was-Change-Id: I81840c7bbbefbc5c3bae6b270c2d94297a254d19

csdc: server reject FIEMAP/SEEK_HOLE|DATA on compr obj

Server return -EOPNOTSUPP if they get the FIEMAP and
SEEK_HOLE/SEEK_DATA requests upon compressed file objects.

DDN-bug-id: EX-8927
Was-Change-Id: I9f04fbb13a22cc83402d9989daab63a59367ff33

csdc: check ocd flag for fiemap/lseek

Currently client will not sending fiemap/lseek request to OST if
the file is a compressed one. This patch will check the
OBD_CONNECT2_COMPRESS flags and send the request if OST supports
compression as server would do the fiemap/lseek check instead.

DDN-bug-id: EX-8927
Was-Change-Id: I478cbf161044165fa31d4caa2336e9949fc626fe

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7644
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4a37b106831a903d90e8a8871e9a93baac4e201e

68407 master J !B ✗T –R 0/2

LU-10026 tests: minor cleanups to sanity-compr

build failure (#130379)

csdc 1 unresolved XS +6 −0

66d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130379 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The test_1001 hunk re-anchored (its context had a version guard this tree already dropped). One sub-section sentence of the original message did not apply to this content and was dropped.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: minor cleanups to sanity-compr

add/fix few subtests to remove big temporary files

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11112
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id04692cbd2443721070a7152186b38829b075907

68403 master J –B ✗T –R -1/2−

LU-10026 tests: fix 1009e and 1122 for arm clients

build failure (#130375)

csdc 1 unresolved S +12 −12

66d

build #130375 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

Applied against the LU-10026-tagged aarch64 except block.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: fix 1009e and 1122 for arm clients

arm clients use 64k page size, this should be reflected in
the tests checking stats on different operations.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10755
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia84ceef06c7ce96ae464abef919bf368583a70e5

68404 master J !B ✗T –R 0/2

LU-10026 tests: drop redundant gate in sanity-compr/2002

build failure (#130376)

csdc 1 unresolved XS +0 −3

66d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130376 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

The original is a later snapshot of the change that introduced test_2002 (same Change-Id) and re-added subtest_2002/test_2002 at the file end, duplicating the definitions (the shell silently uses the second copy, and run_test 2002 would execute twice). The port keeps a single copy - the later version - in place; the effective delta is dropping the per-test gate now covered by the suite-wide gates. New Change-Id, original kept as Was-Change-Id.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: drop redundant gate in sanity-compr/2002

sanity-compr now checks compression support and enablement at
file scope, so test_2002 does not need its own skip gate.

This is the remaining delta of a later snapshot of the change
that introduced test_2002; the rest of it is already applied.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11120
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I7daec2d28fa32646e607291a6fcdd0e303351f67
Change-Id: I6b3bd30b8009fa702f9ddae83fdaf005fd40fd09

68350 master J ✗8B ✗T –R 0/2

LU-10026 tests: memory check depends on RAM

build failure (#130330)

8 unique failures csdc 1 unresolved XS +2 −7

66d

build failure (#130330)

janitor: 8 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1, sanity1@ldiskfs+DNE:test_36f +5 more

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 2 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews
sanity1@ldiskfs+DNE:test_36f	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_36h	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_44b	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_56ra	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@ldiskfs+DNE:test_56x	seen in 37 other reviews
sanity1@ldiskfs+DNE:test_56xa	seen in 38 other reviews

build #130330 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- The Fixes: reference was remapped to the master commit carrying the same
  change (14fc034e983e "LU-18086 obdclass: page_pools shrinker fix").
- The debug echos this patch removes were already absent from the ported
  subtest_2000; the local count declaration is kept (count is still used
  only inside the function).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: memory check depends on RAM

Each open file consumes about 4.7KB of RAM, 300000
files consume about 1.3GB. At systems with 2.6GB RAM
it might cause an OOM with SLOW=yes option.
1. Consume 40% of RAM to check that page pools would be
shrinked.
2. Change createmany with memhog.
3. Export MEMHOG in init_test_env.

Test-Parameters: forjanitoronly
Fixes: 14fc034e983e ("LU-18086 obdclass: page_pools shrinker fix")
DDN-bug-id: EX-10278
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I58ae0a201456ed084db5c528007fe8d7a3782315

66277 master J !B ✗T –R 0/2

LU-10026 llite: restrict overwrites during preview

build failure (#130314)

csdc 1 unresolved M +80 −2

67d

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130314 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: ci_size_extend_compression:1 is placed inside the cl_io
bitfield next to the new ci_parity_io:1 member that master added in the
meantime (the non-bitfield loff_t ci_parity_eof follows the bitfield).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: restrict overwrites during preview

EX-7601 is an issue where when modifying a compressed file
we do not correctly read-up and re-write existing
compressed data.

To avoid this, we can only allow writes which are not
aligned to compression chunk size when they are not
overwriting existing data, ie, when they are extending the
file.

This returns EINVAL for all writes to compressed files
which are not either chunk aligned or extending the file.
This should prevent users from hitting the data corruption
issue but still allows some basic usage.  This is intended
just for the preview period.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I0eea01e2249866a074afd0d0642fe6dce9a49664

65767 master J !B ✗T –R 0/2

LU-10026 utils: update handling of compr level for lz4

build failure (#130258)

csdc 6 unresolved M +107 −49

67d

build failure (#130258)

5 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130258 FAILURE

patchsets:

+1 earlier

PS1 05-08

PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 08-25

threads: 5 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS6 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS6 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebased onto the current series base (65769). Notes on this patchset:

- The level-mapping helpers in lustreapi.h were reconciled with the current
  base's version of the file: the base had switched the numeric bound 0xf to
  the named LL_LZ4FAST_MAX_LEVEL constant; this patch keeps the named
  constant while applying the new +1 level-shift semantics.
- The crypto algorithm-name helper this patch previously extended no longer
  exists at this point in the series (the earlier patches carry a
  display-only compr_name_from_type() in lov instead). The level-dependent
  algorithm selection ("lz4" vs "lz4fast" by level) now lands together with
  the compression engine in 65770, which carries the final form of that
  helper in obdclass.
- In llapi_parse_compress_type(), the "no level given" branch now guards the
  '*level = -1' store with a NULL check: the function accepts a NULL level
  out-pointer and the unguarded store would dereference NULL.
- test_100d: merged this patch's "lz4 twice (std + HC)" coverage with the
  base's newer type list. Final list is "lz4 gzip lz4 lz4fast" plus
  no-level types "lzo zstd", and the compress-file creation passes -S4M so
  the doubling chunk sizes (64K..2M) stay below the stripe size (the chunk
  size is clamped to the stripe size, which would break the per-chunk find
  counts). File names carry _${lvl}_${cs} suffixes so the duplicate lz4
  entries do not collide.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(minor) The comment above still says the acceleration factor is mapped "keeping 1-9 as-is", but from/to now apply a +1/-1 offset, so the encode and decode boundaries no longer line up at 9. from_lz4fast_level(9) returns 10 (the `level < 10` path, level+1), but to_lz4fast_level(10) takes the `>= 10` path and returns 11. So `lfs setstripe -Z lz4fast:9` reads back as level 11 via `lfs getstripe --compr-level` (8 round-trips, 9 does not). Was the boundary meant to be `level <= 10` on decode (or `level < 9` on encode)?

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(style) These crypto headers are #include'd in the middle of the file, interleaved between the new static inline definitions, rather than grouped with the other includes near the top. If the patch is refreshed, consider moving them up. (There's also a stray double blank line at the end of the block.)

reply PS5 2026-07-15 00:56 Gerrit AI review for Lustre lustre_crypto.h:200 — (typo) "Cyrpto" -> "Crypto" here and in the ll_crypto_acomp_set_level() comment below.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(typo) "Cyrpto" -> "Crypto" here and in the ll_crypto_acomp_set_level() comment below.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) ll_crypto_comp_set_level() and ll_crypto_acomp_set_level() (and COMPR_LEVEL_MASK / the *_MOD_NAME macros) don't appear to be called or referenced anywhere in the tree. If nothing uses them yet, this looks like it belongs in the kernel patch that consumes them rather than in a utils level-mapping change.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:56

(defect) This branch is entered when `level` is NULL (`!level`), but then dereferences it. The pre-patch code returned early for exactly the `!level` case, so a caller of this exported liblustreapi function that passes NULL for `level` (only interested in the type) now crashes. Both in-tree callers happen to pass non-NULL, but the guard still contradicts the body. Something like:

    if (!level)
        return 0;
    if (!ptr) {
        *level = -1;
        return 0;
    }

tags: csdc

owner Marc Vef · uploader Marc Vef · PS6 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: update handling of compr level for lz4

The way lz4 compression level or acceleration factor is handled needs
to be adapted in order to match what is provided by the lz4 userspace
tool:
- any level between 0 and 2 is interpreted as the default lz4
  acceleration factor of 1;
- any level from 3 and up to 16 is interpreted as a compression level
  for internal lz4hc. Increasing the compression level trades CPU
  time for improved compression ratio;
- acceleration factor can be specified for the lz4fast compression
  type, from 1 to 26. This acceleration factor trades compression
  ratio for faster speed.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4711217c1a6601f29f78d262567da5998f657fc9

66282 master J !B ✗T –R 0/2

LU-10026 llite: skip hole/data lseek() on compressed file

build failure (#130317)

csdc M +48 −5

67d

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #130317 FAILURE

no vote yet: Mikhail Pershin

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25 PS5 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS5 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: skip hole/data lseek() on compressed file

Doesn't execute real lseek() on compressed file with
SEEK_HOLE/SEEK_DATA origin but consider file always
non-sparsed and do generic lseek() only

test: get size before fallback to generic

If ll_file_seek() falls back to generic_file_llseek_size()
then make sure inode size is reliable

DDN-bug-id: EX-7786
Was-Change-Id: Ieb8c90e66fb19675ba41f0147c0d9cdaf29ea20a

Test-Parameters: forjanitoronly
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2cd587cc0205e85758e06bbaafafe0e2959e0ade

68351 master J ✗2B ✗T –R 0/2

LU-10026 llite: don't queue async IO page for inode in freeing

build failure (#130329)

2 unique failures csdc 1 unresolved S +19 −0

67d

build failure (#130329)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 5 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130329 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- Registered as .cio_commit_async with the current signature (the
  enum cl_io_priority argument was added upstream), next to master's
  renamed .cio_read_ahead_prep hook.
- Reviewer note: master meanwhile carries LU-17364 ("discard un-flushable
  pages on evicted inode teardown"), which addresses the same teardown
  crash from another angle; this patch is kept as an additional guard and
  may be droppable after evaluation.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: don't queue async IO page for inode in freeing

When inode is being freed, do not try to queue async IO pages as
there is a race window between ll_delete_inode() and
ll_truncate_inode_pages_final(), and the queued async IO pages
cannot be teardown.

(osc_page.c:182:osc_page_delete()) Trying to teardown failed: -16
(osc_page.c:184:osc_page_delete()) ASSERTION( 0 ) failed:
(osc_page.c:184:osc_page_delete()) LBUG

    lbug_with_loc+0x64/0xb0 [libcfs]
    osc_page_delete+0x290/0x574 [osc]
    cl_page_delete0+0xb8/0x404 [obdclass]
    cl_page_delete+0x64/0x22c [obdclass]
    ll_invalidate_folio+0x114/0x2a0 [lustre]
    truncate_cleanup_folio+0x6c/0xf0
    truncate_inode_pages_range+0xf8/0x4e0
    truncate_inode_pages_final+0x54/0x80
    ll_truncate_inode_pages_final+0x40/0x520 [lustre]
    ll_delete_inode+0x68/0x25c [lustre]
    evict+0xb8/0x1a0

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11006
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: If655a89faf6f979167c05c91c7b52a6c49a7884b

68352 master J ✗2B ✗T –R +1/2

LU-10026 ofd: space is always granted to server

build failure (#130332)

2 unique failures csdc XS +1 −1

67d

build failure (#130332)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 8 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130332 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 09:47

patchsets: PS1 08-25 PS2 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: space is always granted to server

when it's converting a compressed chunk to uncompressed one for
partial truncate.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10391
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic636525e76c544b55dede13133f423cc0aed534f

68354 master J ✗2B ✗T –R 0/2

LU-10026 sec: set page sync flags to avoid ENOMEM

build failure (#130334)

2 unique failures csdc 1 unresolved S +24 −6

67d

build failure (#130334)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 6 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130334 FAILURE

no vote yet: Artem Blagodarenko

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port note: fill_cpga() debug output adapted to the folio-based brw_page
(bp_folio).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 sec: set page sync flags to avoid ENOMEM

Set OBD_BRW_SOFT_SYNC page flag when appropriate page_pool
is at least 75% full. This would cause a "soft sync" update
at OST side. When pool is at least 81% full clear OBD_BRW_ASYNC
flag to trigger a sync commit. This needs to not stop
compression at client side when page pool becomes empty,
i.e. no free elements.

There is sanity-compr_2001 to check that all chunks from written
1GB data have been successfully compressed, but it will be
landed later,
in a separate patch, after the statistics patches.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10481
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I715d2fb95a459b47c003dad3339f8b6f775245a8

68353 master J ✗2B ✗T –R 0/2

LU-10026 scripts: Compression estimate script

build failure (#130331)

2 unique failures csdc 1 unresolved L +472 −1

67d

build failure (#130331)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 7 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130331 FAILURE

no vote yet: Raphael Druon, Jian Yu

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- always_except tag renumbered to LU-10026.
- ll_compression_scan added to EXTRA_DIST so it survives make dist (the
  tarball would otherwise fail to build with the script listed in
  bin_SCRIPTS but absent).
- Reviewer note: the script is installed under the server-only block
  although it can run on any client; left as the original patch had it.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 scripts: Compression estimate script

ll_compression_scan is a simple tool which can be run on any
Linux system to estimate the space usage reduction from the
Lustre Client Side Data Compression (CSDC) feature with
particular compression settings (algorithm, chunk size,
and compression level).

When run on one or more directories, it will recursively
examine a percentage of files under that directory, sampling
data in those files to estimate how the files will compress.

This tools samples data throughout the file, so it should
avoid problems with poor estimates for files with headers
which differ from the bulk data in the file.

However, if the directory tree is particularly imbalanced,
with a few large uncompressible files in one directory, and
many small files in other directories, then scanning a small
percentage of files may give a misleading compression estimate.
Sampling a larger percentage of files will improve this.

This tool requires the lz4, lzop, and gzip utilities to
be installed in order to test those compression types.
(lzop is the command line utility for lzo compression)

scripts: add whole file to compression scan

Add a mode where the compression scan script compresses the
entire file, which in theory should 100% match the
compression results from using CSDC and allow a test to
calculate the exact space usage reduction expected by
using CSDC.

This is intended to be used mostly for testing.

Change help documentation slightly to make clear this can
also accept a path to a single file.

DDN-bug-id: EX-7795
Was-Change-Id: I606a33d686d87dd631bf5b33dc85ee8c24fe9f67

scripts: Improve estimated ratio

ll_compression_scan does not take in account the size of the
sampled files, this might lead to uncorrect estimated ratio for non
homogeneous file.

This patch apply the compression ratio estimated with the sampled data
and applies it to the entire file size, assuming the file will have
the same compression ratio across it.

DDN-bug-id: EX-8362
Was-Change-Id: Ic4a26460e17c666b9edf4c0d8d450a06fad5920f

scripts: improve ll_compression_scan functionality

Improve ll_compression_scan script functionality without
changing the compression estimates.
- add a version string to the output to allow tracking
- handle pathnames with spaces in them
- handle the lz4fast compression type
- allow running on MacOS for testing

DDN-bug-id: EX-8362
Was-Change-Id: I0b8442a2590fdb9c718b1404cba1d73c26cff03c

scripts: improve ll_compression_scan estimate

Improve ll_compression_scan script to give a better estimate of
actual compression ratios.
- add a '-d' debug option for verbose output during testing
- log and report incompressible small files < 4096
- log and report incompressible file count and size
- include small/incompressible/large files in compression estimate
- add a correction factor to calculations for safety margin

DDN-bug-id: EX-8362
Was-Change-Id: If561b0273e38e4821de228c81291859c7bb1a0d2

tests: exclude sanity-compr/1008 on Ubuntu

This subtest is failing consistently on Ubuntu.  Disable until it
can be fixed.

DDN-bug-id: EX-9125
Was-Change-Id: Ie71462e7f033be914523ca96b22478a53b81b882

utils: always print ll_compression_scan summary

Print the summary block for ll_compression_scan even if it is
killed with CTRL-C before finishing the scan.  Otherwise, only
the running total compression ratio estimate will be printed.

DDN-bug-id: EX-9125
Was-Change-Id: I64e135fa6729767d279a52df767252103a2540e5

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7681
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Raphael Druon <rdruon@ddn.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I092f9608553eba10bacfcc3c4a3fafc9a454c287

68399 master J –B ✗T –R -1/2−

LU-10026 tests: sanity-compr/1000d failure for ARM client

build failure (#130371)

csdc S +9 −7

67d

build #130371 FAILURE

no vote yet: Zhenyu Xu

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: sanity-compr/1000d failure for ARM client

aarch64 has 64K page size and compression chunk size could be adjusted
upward if it's original setting value is equal to page size, so
the test should be refined to get the actual chunk size be set.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ide2f91e255ba886f9c1e18d1e1fe89c589f3bca1

68348 master J ✗2B ✗T –R 0/2

LU-10026 csdc: do not mix compressed and plain in niobuf

build failure (#130328)

2 unique failures csdc 1 unresolved XS +1 −1

67d

build failure (#130328)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 3 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130328 FAILURE

patchsets: PS1 08-25 PS2 08-26

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port note: applied as a hand edit - the mask line context had been
reflowed by the earlier can_merge_pages() changes. Content unchanged
(removes OBD_BRW_COMPRESSED from the merge-safe mask).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: do not mix compressed and plain in niobuf

It is important to prevent placing
compresed and uncompressed chunks to the
same rnb, because rnb has only one set flags
for all chunks contained.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: If3fbe160a9f32af2d51f4ba694f444563feb82f6

68401 master J !B ✗T –R 0/2

LU-10026 tests: Add compression statistics output

build failure (#130373)

csdc 1 unresolved S +25 −0

67d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #130373 FAILURE

no vote yet: Sergey Cheremencev, Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

The two OST-version guards around the obdfilter stats access were dropped - compression_enabled() already implies the server supports compression on this tree. The interop paragraph of the message was reworded accordingly.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: Add compression statistics output

sanity-comr executes sanity, sanityn large set of its tests

Currently, it is not even easy to understand that data was
compressed/decompressed. Adding compression statists gives a sign
that compression is enabled and works, showing that parts
(client/server, any features etc.) were executed during the test.

tests: skip CSDC stats errors in interop

This also avoids trying to clear or fetch compression statistics
from servers that do not support compression, always returning
success in that case.
Let's avoid trying to clear or fetch these stats from servers that
do not have them and always return success when clear/read stats.

DDN-bug-id: EX-10124
Was-Change-Id: Ide3e69dfd0268041591c3445aff39532ff00f019

tests: osc compr stats for all OSTs

Show osc compression statistic for all OSTs in
the end of the test session.

DDN-bug-id: EX-11007
Was-Change-Id: Id6ae864438931d9ce50300911490760bf24dd88a

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10124
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9856bad32b0f421ce8cb8ff45a171e229f7c8f02

66613 master J ✓B ✗T –R ✗

LU-10026 tests: change sanity-compr 2000

parked by your own -1

CR -1 csdc 2 unresolved XS +8 −1

67d

build #130321 FAILURE

reviews: Marc Vef -1 2026-08-26 09:47

no vote yet: Sergey Cheremencev

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

your previous vote: -1 on PS1 (2026-06-12 09:25) — now at PS3

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) The Fixes: commit only touched lustre/obdclass/page_pools.c and never added this test. subtest_2000() and the dd count=5000 that this patch tunes down were both introduced by 452599da95 ("LU-10026 osc: osc brw request compression", Gerrit 65770). If the patch stays standalone, should Fixes: point at 452599da95 instead? (This also lines up with the existing suggestion to fold it into 65770, which would drop the tag entirely.)

Gerrit AI review for Lustre PS1 · 2026-07-15 00:58

(minor) Both echoes print the same "count $count" label, one before and one after the clamp, so a log reader can't tell the raw 70%-of-RAM value from the clamped one. Consider labelling them differently or dropping the first.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: change sanity-compr 2000

Make dd write 70% of RAM but less than 5GB.
The test case is too agressive and sometimes
causes OOM as client have to preserve
compressed pages for OST_WRITE replay(see
EX-9895 for details).

Fixes: 14fc034e983e ("LU-18086 obdclass: page_pools shrinker fix")
Test-Parameters: forjanitoronly
Test-Parameters: trivial testlist=sanity-compr env=ONLY=2000,ONLY_REPEAT=10
Test-Parameters: trivial env=ONLY=2000,SLOW=yes testlist=sanity-compr
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I594b963ca358c5adbf077bcd59378dd13506ccad

68370 master J ✗1B ✗T –R -1/2−

LU-10026 csdc: enable zstd/zstdfast as compr type in layout

build failure (#130345)

1 unique failure csdc 1 unresolved M +107 −31

67d

build failure (#130345)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	seen in 1 other review

build #130345 FAILURE

no vote yet: Sebastien Buisson

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

- Rewritten onto the current naming: only LL_COMPR_TYPE_ZSTDFAST=8 is new (zstd=7 already existed); the existing zstd table row gained its level converters (renamed to from_/to_zstd_level, from_/to_zstdfast_level); zstdfast row added before the MAX sentinel; wire asserts extended (ZSTDFAST==8, MAX 8->9 in both wiretest copies + wirecheck).
- The duplicate COMPR_LEVEL_MAX definition was removed while raising it to 100 (previously the header defined it twice; the values would have diverged).
- Man page: zstdfast added to the type list and level-range sentence (including correcting the lz4fast range to 1-26, matching the current mapping); zstd/zstdfast bullets added after lzo.
- sanity-pfl test_100b/test_100d switched to deriving the compression type lists from the negotiated compress_types in the OSC import, as in the original; adapted to the parametric test structure used by this series.
- The old-version gate around the zstd line in sanity.sh compress_content() is meaningless on master and was dropped (line unconditional).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: enable zstd/zstdfast as compr type in layout

The Zstandard compression algorithm shows good promise for both higher
compression than gzip, while also being faster.
Available compression levels for zstd are 1 through 22. In addition,
faster levels supported by zstdfast are 1-10, 20-100 (in increments of
10).

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9678
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Idc320a9039d2d974683339b8033b23736fe2a20f

68402 master J !B ✗T –R 0/2

LU-10026 csdc: enable compression on aarch64

build failure (#130374)

csdc 1 unresolved S +13 −19

67d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #130374 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- Removes the PAGE_SIZE-conditional default (llite_enable_compression now unconditionally 1) and the userspace PAGE_SIZE preview-warning blocks (which were dead in userspace, since PAGE_SIZE is not a preprocessor macro there).
- aarch64 always_except tags renumbered to LU-10026; one comment reworded to drop an internal change reference.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: enable compression on aarch64

This patch enables CSDC by default on aarch64 client
with >4k PAGE_SIZE.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-7536
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6172e569c0ae5eb0b823f703c640847479909bdd

66274 master J !B ✗T –R 0/2

LU-10026 osc: don't do decomp. with compression disabled

build failure (#130267)

csdc 3 unresolved XS +3 −1

67d

build failure (#130267)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130267 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-05-28 20:13 Marc Vef COMMIT_MSG:15 — this is not helpful and needs to be updated

Marc Vef PS1 · 2026-05-28 20:13

this is not helpful and needs to be updated

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) The body says decompress_request() has "a significant performance cost for checking if data is compressed", but it already returns immediately when the first page's cp_compr_type is LL_COMPR_TYPE_NONE. For uncompressed files that check is cheap; the scan cost only applies to compressed data, which is the case that still needs decompressing. Could the justification be tightened, and the read/write asymmetry this creates (writes still compress off the layout while reads stop decompressing) be noted?

reply PS3 2026-07-15 00:57 Gerrit AI review for Lustre osc_request.c:2537 — (defect) This gates decompression on the global llite_enable_compression instead of the data's own cp_compr_type.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) This gates decompression on the global llite_enable_compression instead of the data's own cp_compr_type.

decompress_request() already returns early when pga[0]'s cp_compr_type is LL_COMPR_TYPE_NONE:

    type = clpage->cp_compr_type;
    /* no compression */
    if (type == LL_COMPR_TYPE_NONE)
        RETURN(0);

so uncompressed reads were never paying for a full scan.

The concern is a genuinely compressed read reaching here with the tunable off. lsm_lmm_verify_v1v3() only rejects a fresh compressed-layout unpack when compression is disabled; a cached lsm survives a runtime toggle. On read, set_page_compression()->__set_page_compression() still fills cp_compr_type from that cached lsme, and the write path still compresses off cp_compr_type regardless of the global.

After echo 0 > ...enable_compression on a client that already holds the compressed file's layout, does a cache-cold read here hand back the still-compressed bytes undecompressed?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: don't do decomp. with compression disabled

decompress_request likely has a significant performance
cost for checking if data is compressed, so we should not
call it when compression is disabled.

This is a stop-gap solution for the preview until we can
have the server tell the client if data is compressed, as
described in EX-7818.

Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iba8feba3ab0fe620d0594f59c2c6ddea25faeb4f

66265 master J !B ✗T –R 0/2

LU-10026 llite: allow mmap reads of compressed files

build failure (#130262)

csdc 4 unresolved M +107 −3

67d

build failure (#130262)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130262 FAILURE

no vote yet: Artem Blagodarenko

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(minor) `off` is computed at the call site as vma->vm_pgoff<<PAGE_SHIFT and passed in, but never used in the body. Drop the parameter, or use it.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(suggestion) The block is keyed on the file open mode, so an fd opened O_RDWR that only creates a PROT_READ mapping is rejected with -EOPNOTSUPP even though it never writes the file (a common open-O_RDWR-then-mmap-read pattern). Since read mmap of compressed files is supported, would keying on the mapping's write intent (vma->vm_flags VM_WRITE/VM_SHARED) rather than the open mode let those read-only mappings through while still avoiding the write SIGBUS?

reply PS3 2026-07-15 00:56 Gerrit AI review for Lustre llite_mmap.c:613 — (minor) The message names cl_io_rw_init(), but this path calls cl_io_init().

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(minor) The message names cl_io_rw_init(), but this path calls cl_io_init().

reply PS3 2026-08-26 11:52 Alex Zhuravlev multiop.c:585 — what do we do with this?

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(defect) These treat O_RDONLY/O_WRONLY/O_RDWR as bitmask flags, but the access mode is a 2-bit field. O_RDONLY is 0, so `fd_flags & O_RDONLY` is always 0, and for an O_RDONLY fd `fd_flags & O_RDWR` is also 0, so mmap_mode stays 0 (PROT_NONE) instead of PROT_READ.

Mask and compare instead:

    switch (fd_flags & O_ACCMODE) {
    case O_RDONLY: mmap_mode = PROT_READ; break;
    case O_WRONLY: mmap_mode = PROT_WRITE; break;
    case O_RDWR:   mmap_mode = PROT_READ | PROT_WRITE; break;
    }

Because of this, the `oSMc` case in test_460e maps PROT_NONE, so "readonly mmap should work" passes without ever exercising a readable mapping.

Alex Zhuravlev PS3 · 2026-08-26 11:52

what do we do with this?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 llite: allow mmap reads of compressed files

mmap reads of compressed files work, so we should only
block writes.

We cannot block the actual fault operations because that
will cause the application to get a SIGBUS, so we check
the file open mode when we go to create the memory mapping.

llite: disable mmap on compressed files

disable mmap(2) on compressed files until well tested.

DDN-bug-id: EX-6265
Was-Change-Id: I3464a03b16708edcd0692bc9db337eb8473ea047

llite: use CIT_MISC in check_compression

ll_mmap_check_compression uses a CIT_FAULT io type and
initializes it with cl_io_rw_init.  This doesn't
initialize the IO correctly and this sometimes results
in the following crash during io_fini, because the ft_page
pointer is initialized to something else by cl_io_rw_init:

BUG: unable to handle kernel NULL pointer dereference
at 000000000000004a
RIP: 0010:cl_pagevec_put+0x9f/0x3a0 [obdclass]
 vvp_io_fault_fini+0x21/0x40 [lustre]
 cl_io_fini+0x7a/0x230 [obdclass]
 ll_mmap_check_compression+0x403/0x540 [lustre]

DDN-bug-id: EX-7812
Was-Change-Id: Ib1f270f071370d6c045bb9d799ab5b7b41a6c6be

Test-Parameters: forjanitoronly
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I636f398fd247ddcd153f94bf8116440540e8469c

66268 master J !B ✗T –R 0/2

LU-10026 utils: error for invalid compression level

build failure (#130265)

csdc 3 unresolved M +78 −45

67d

build failure (#130265)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130265 FAILURE

no vote yet: Sebastien Buisson

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

The message (and the sanity-pfl/sanity test updates) cover only lzo, but the new error path also fires for zstd, since it likewise has no level mapping. Worth naming zstd here and adding coverage for it, or clarifying why zstd is excluded from the documented 1-21 level range.

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase notes:

- The base meanwhile added zstd to the compression type table (no level
  converter). Since this patch makes an explicit level on a no-converter
  type an error, zstd was added to test_100d's type_nolvl list next to lzo,
  and the compress-file creation now passes -S4M so the six doubling chunk
  sizes (64K..2M) stay below the stripe size (the chunk size is clamped to
  the stripe size, which would break the per-chunk find counts).
- test_100d's chunk-verification loop now iterates all types with a
  doubling chunk size (the previous patchset's loop only ran one
  iteration).

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(defect) This whitelists only fast/best to carry a level; every other type with a NULL ctn_from_compr_level falls through to the error. That is lzo (intended) but also zstd. So `lfs setstripe -Z zstd:5` now fails, whereas pre-patch it stored the level. lfs-setstripe.1:232 (added in this same LU-10026 series) says zstd takes a level 1-21. Is the zstd rejection intended? If zstd should not take a level, the man page needs updating too; if it should, zstd wants a from_compr_level mapping rather than the error path.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: error for invalid compression level

Not all compression types accept a compression level (e.g. lzo in its
kernel implementation). For those, return an error if a compression
level is provided on 'lfs setstripe' command-line.

Test-Parameters: forjanitoronly
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I6c367c4bfd76cc81c890a89ba9f994f4fd9f4f80

68349 master J ✗2B ✗T –R +0/2−

LU-10026 osc: limit compression chunks in RPC

build failure (#130327)

2 unique failures csdc 2 unresolved L +283 −22

67d

build failure (#130327)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

1 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 10 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 10 other reviews

build #130327 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 11:31

no vote yet: Artem Blagodarenko, Zhenyu Xu

patchsets: PS1 08-25 PS2 08-26

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-25 21:24 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-25 21:24

[Marc Bot] upstream remarks for reference:

Port notes:

- try_to_add_extent_for_io()/get_write_extents() gained their env parameter
  on top of master's reworked extent-RPC code: the erd_max_compr_chunks
  member was added to the write_rpc_def()/read_rpc_def() initializer macros
  (which master introduced since), with the macros now taking the osc
  object.
- Page indices in the chunk-counting loops use osc_index() - struct
  brw_page no longer has a bp_page member on master (folio-based).
- The RPC-fragment limiting in the submit path is applied in
  osc_io_submit() only; master has since split direct-IO submission into
  osc_dio_submit(), which keeps its existing behaviour (DIO on compressed
  files is restricted to chunk-aligned IO anyway).
- cp_compr_chunk_log_bits -> cp_compr_chunk_lum_bits (series field naming).
- Fixed a short memcpy in the chunk-ledger growth helper
  (expand_osc_compr_chunk_array copied the element count as bytes,
  losing 3/4 of the saved entries on expansion).
- The new sanity-compr test_1016 gained a compression_supported skip guard
  (it had none).

reply PS2 2026-08-26 10:30 Misc Code Checks Robot (Gatekeeper helper) osc_io.c:138 — error: osc_io_submit():'osc' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-08-26 10:30

error: osc_io_submit():'osc' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: limit compression chunks in RPC

Limit the number of partial compressed chunks that can be sent to
the server in a single RPC, so that it doesn't need to allocate
buffers for read-modify-write on a huge number of chunks at once.
It is better to handle that in separate server threads.

In the worst case, each extent in the BRW RPC belongs to a different
compression chunk, so the worst_case_chunks would expand on the OSS
to worst_case_chunks * compression_chunk_size bytes IO on the server.
That might be hundreds of MB in size per thread and LASSERT or OOM.

Add tunable osc.*.compress_rpc_fragment_ratio to reduce the maximum
compressed chunks by the given factor, its value ranges in [1, 32].

osc: scan extents to find compression chunks

as we can't trust oe_start and oe_end, we have to scan all the
pages covered by extent and so find all compression chunks to
estimate and limit potential read-modify-write IO on OST.

osc_submit_io() should not create too big extents we can't send
due to limits for compressed chunks.

DDN-bug-id: EX-10141
Was-Change-Id: I00b104ded6cdb8e66414eb2f759f6dca1b7b4d11

osc: make oti_compr_chunks ledger expandable

This patch makes osc_thread_info::oti_compr_chunks an expandable
array.

DDN-bug-id: EX-10361
Was-Change-Id: Ib2ad5407815c733687eadd4a46014d0a694dbf41

osc: expand_osc_compr_chunks_array() use atomic alloc

This can be called in "spinlock" context and generate a stack dump:

  BUG: sleeping function called from invalid context
  in_atomic(): 1, irqs_disabled(): 0, name: ptlrpcd_00_02
  Call Trace:
    ___might_sleep.cold.23+0x104/0x118
    kmem_cache_alloc_trace+0x1cd/0x2a0
    expand_osc_compr_chunk_array+0xc6/0x5d0 [osc]
    try_to_add_extent_for_io.isra.10+0x11f6/0x16b0 [osc]
    osc_io_unplug0+0x6a4/0x1b50 [osc]
    brw_queue_work+0x29/0xb0 [osc]
    work_interpreter+0x2b/0x100 [ptlrpc]
    ptlrpc_check_set+0x583/0x2fe0 [ptlrpc]
    ptlrpcd_check+0x511/0x570 [ptlrpc]
    ptlrpcd+0x1b5/0x4b0 [ptlrpc]

DDN-bug-id: EX-10361
Was-Change-Id: I72f30b5185be0417c1bb7170ac278f48b04e7d0b

osc: revert to correct cl_max_extent_pages

otherwise grant overhead calculation breaks.

DDN-bug-id: EX-10267
Was-Change-Id: I4e74524b17c7834c432a708bb917f291fd773023

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10047
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4d0b4bdaaa91f46147d2ca1c52741117533f8f7a

66614 master J ✓B ✗T –R ✗

LU-10026 tests: skip sanity-compr for old servers

parked by your own -1

CR -1 csdc XS +2 −0

67d

build #130324 FAILURE

reviews: Marc Vef -1 2026-08-26 09:47

patchsets: PS1 06-11 PS2 08-25 PS3 08-26

your previous vote: -1 on PS1 (2026-06-12 09:24) — now at PS3

tags: csdc

owner Marc Vef · uploader Marc Vef · PS3 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip sanity-compr for old servers

Skip for old servers.

Test-Parameters: forjanitoronly
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia42f1f32fe166bae84a7c38c02d6a218303cc51e

68355 master J ✗2B ✗T –R +1/2

LU-10026 ofd: make oti_chunk_bufs expandable

build failure (#130333)

2 unique failures csdc M +60 −70

67d

build failure (#130333)

janitor: 2 test failures unique to this patch — runtests@ldiskfs+DNE:test_1, runtests-ssk@ldiskfs+SharedKey:test_1

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
runtests@ldiskfs+DNE:test_1	seen in 4 other reviews
runtests-ssk@ldiskfs+SharedKey:test_1	seen in 8 other reviews

build #130333 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 09:47

patchsets: PS1 08-25 PS2 08-26

tags: csdc

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: make oti_chunk_bufs expandable

to be able to handle very large IO with lots of chunks.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10394
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8ad24acd61037cf2804612940cdb74662c0c5303

68373 master J –B ✗T –R +0/2−

LU-10026 csdc: stop compressing incompressible file data

build failure (#130347)

csdc 2 unresolved L +413 −17

67d

build failure (#130347)

1 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

build #130347 FAILURE

reviews: Alex Zhuravlev +1 2026-08-26 12:21

no vote yet: Zhenyu Xu

threads: 1 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

ci_incompressible placed in the relocated cl_io bitfield next to ci_size_extend_compression; test_1021 gate converted to compression_supported.

reply PS1 2026-08-26 11:09 Misc Code Checks Robot (Gatekeeper helper) osc_io.c:566 — error: osc_io_iter_init():'osc' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 11:09

error: osc_io_iter_init():'osc' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: stop compressing incompressible file data

The reduced_ratio (original_size/compress_reduced_size) represents
the minimum fraction of pages that are compressed out of each chunk,
namely the compressed chunk needs to shrink by at least
1/reduced_ratio blocks for it to be "compressible".

Let size compression_ratio be defined as
original_size/after_compression_size, so

reduced_ratio = compression_ratio / (compression_ratio - 1)

and we set its default value to 16, equivalent to 1.07 of compression
ratio (i.e. needs to shrink at least one 4KB block out of each 64KB
chunk).

After every compress_check_bytes of data being compressed, file's
compressibility would be re-calculated based on average
compress_reduced and average compress_orig data size.

Stop compressing file data if it is deemed to be incompressible, and
after compress_skip_bytes data have been written uncompressed , retry
the file compressibility check.

compress_reduced_ratio, compress_check_bytes, compress_skip_bytes
are tunable parameters:
	osc.*.compress_reduced_ratio
	osc.*.compress_check_bytes
	osc.*.compress_skip_bytes
their default values are 16, 1M and 32M respectively.

tests: use /dev/zero as compressible source

in test 1021 as hdf file has few incompressible chunks and this
breaks the test infrequently.

DDN-bug-id: EX-10857
Was-Change-Id: Iba29d9e654a245d3c0cbb14a8b7dbb9f847f52d5

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I4ce3d752c67f18ba7b100c72a2bb61a91258c6e8

68409 master J –B ✗T –R -1/2−

LU-10026 csdc: tunable to disable compression on fallocate

build failure (#130381)

csdc 1 unresolved L +238 −12

67d

build #130381 FAILURE

no vote yet: Sebastien Buisson

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- The connect-time obd_set_info_async(KEY_FALLOCATE_COMPR) block was merged into the current client_common_fill_super() flow (placed before cl_sb_init; the error path uses this function's current label and puts the pending request like its sibling paths).
- test_1031's version gate converted to compression_supported.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: tunable to disable compression on fallocate

fallocate is denied for compressed files because preallocating space
for a compressed file makes the compression useless. But sometimes it
could be preferable to deactivate compression and honor fallocate
instead.
So add a tunable for disabling compression on fallocate, 0 by default:
llite.*.prefer_fallocate_over_compr
Note disabling compression is possible only if none of the components
have been already initialized.

Add sanity-compr test_1031 to exercise this.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10053
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I3f272f39166287cd459ebcfbb9bd61ed0fefd30f

68405 master J !B ✗T –R 0/2

LU-10026 cdsc: update layout compressibility

build failure (#130377)

csdc 4 unresolved L +335 −54

67d

build failure (#130377)

3 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #130377 FAILURE

no vote yet: Zhenyu Xu, Alex Zhuravlev

threads: 3 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:51 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:51

[Marc Bot] upstream remarks for reference:

- OBD_CONNECT2_UPDATE_LAYOUT was already reserved on master at the same bit; only the MDT_CONNECT_SUPPORTED2 and client connect lists needed the addition.
- ci_need_change_layout added to the current cl_io bitfield; CAT_COMPRESSIBLE handling merged into the current vvp_attr_update().
- mdd_layout_update_pccro was renamed/extended per the patch on the current (retry-aware) master form.
- FORMATTED_BUF_LEN (used by the new getstripe formatting) is defined locally - the source branch carries it from an unrelated change.
- test_1021 layout-flag assertions applied unconditionally (the original wrapped them in version gates).

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:297 — error: vvp_layout_minor_change():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_layout_minor_change():'vob' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:429 — error: vvp_io_fini():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_io_fini():'vob' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:38 Misc Code Checks Robot (Gatekeeper helper) vvp_io.c:1973 — error: vvp_io_init():'vob' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:38

error: vvp_io_init():'vob' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 cdsc: update layout compressibility

When compressing write changes the file's compressibility, it updates
the file layout flags so that its compressibility persists.

lod: reset la_attr at declare

to avoid unplanned local attributes update during minor layout change.

DDN-bug-id: EX-10786
Was-Change-Id: I093efeb9cf110adfe6a7590091e60b1e9f1eb9ed

Test-Parameters: forjanitoronly
DDN-bug-id: EX-8355
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia2964817d1b5fee2d84ef15e9f1036882d70adb2

66284 master J –B ✗T –R -1/2−

LU-10026 csdc: handle data compression on MDT

build failure (#130318)

csdc 4 unresolved S +39 −1

67d

build failure (#130318)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

build #130318 FAILURE

no vote yet: Zhenyu Xu

patchsets:

+2 earlier

PS1 05-28 PS2 06-11

PS3 06-11 PS4 06-11 PS5 06-11 PS6 08-25 PS7 08-26

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(minor) The body explains the DoM+compress block, but two hunks pull the other way and aren't mentioned: lov_pattern_available() gains LOV_PATTERN_MDT | LOV_PATTERN_COMPRESS, and llapi_pattern_to_lov() gains MDT|COMPRESS and RAID0|COMPRESS cases. Both make the combination more acceptable, not less. Could the message cover why they belong in a patch whose stated purpose is to disallow it?

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(defect) This makes the server accept an MDT|COMPRESS component (lod_lov.c / lov_ea.c call lov_pattern_available()), yet lov_pattern_supported() still excludes it, so client IO marks such an entry invalid (lov_object.c sets lle_valid = 0). The only guard against creating it lives in lfs.c; a liblustreapi caller doing llapi_layout_pattern_set(MDT) + llapi_layout_compress_set() bypasses that and produces a layout the server stores but IO cannot use. If DoM+compress is unsupported, should the block live in the library/server rather than only the CLI?

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(defect) errno is set to a negative value here; every other error path in comp_args_to_layout() uses errno = EINVAL. A negative errno breaks strerror(errno)/perror on this -1 return. Should be errno = EINVAL.

Gerrit AI review for Lustre PS5 · 2026-07-15 00:57

(minor) These two cases look unreachable. The COMPRESS bit never reaches llc_pattern: llapi_layout_pattern_set() rejects any pattern carrying it, and llapi_layout_from_lum() stores compression in llc_compr_type, not the pattern. The bit is OR'd into lmm_pattern separately in llapi_layout_to_lum() (blob->lmm_pattern |= LOV_PATTERN_COMPRESS). Was a real caller intended, or can these be dropped?

tags: csdc

owner Marc Vef · uploader Marc Vef · PS7 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: handle data compression on MDT

Do not support setting data compression component on DoM until
data compression on MDT implemented.

DDN-bug-id: EX-7806
Was-Change-Id: I3794460140f08a073377c418dd56e7dda907d96d

Test-Parameters: forjanitoronly
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8b5c4d5fd18033e796b1c7555c37251e4f7f9a49

68400 master J –B ✗T –R -1/2−

LU-10026 tests: skip 1000d if not enough OSTs

build failure (#130372)

csdc 1 unresolved XS +1 −0

67d

build #130372 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

OSTCOUNT gate added after the converted compression_supported gate.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip 1000d if not enough OSTs

sanity-compr/1000d needs 2 OSTs at least

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10785
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I93d7a2c0178549980348ab41c2fcb62669571f6a

68398 master J !B ✗T –R 0/2

LU-10026 tests: one more compressed truncate

build failure (#130370)

csdc 1 unresolved S +14 −0

67d

janitor run: 21 failing config(s), none unique to this patch — janitor results

build #130370 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

The version guard inside the new truncate case (guarding a fix contained in this series) was dropped; its bug-reference comment renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: one more compressed truncate

check one more scenario of compressed file's truncation.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10391
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I186b92f36dd93afe943ea8f87a6e334f73fd9d26

68396 master J !B ✗T –R 0/2

LU-10026 osc: mem leak in compress_request

build failure (#130368)

csdc 1 unresolved M +70 −13

67d

janitor run: 25 failing config(s), none unique to this patch — janitor results

build #130368 FAILURE

no vote yet: Sergey Cheremencev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

Applied cleanly; test_2002's client-version gate converted to compression_supported (dropped again later in the series once the suite gained file-scope gates). The commit message's review pointer was generalized.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 osc: mem leak in compress_request

1. Fix memory leak in compress_request.
2. In case of ENOMEM compress_request should return 0
if it has been able successfully compress at least
one chunk.

osc: fix OBD_FAIL_OSC_COMPR_ENOMEM

Set cfs_fail_loc to OBD_FAIL_OSC_COMPR_ENOMEM to
always fail with ENOMEM after skipping fail_val times.

Several cosmetic changes in sanity-compr test_2002 to address
review comments on an earlier version.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11120
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I745e17978d885d40218efe13e68e7e41b4b33d31

68395 master J !B ✗T –R 0/2

LU-10026 tests: check that all chunks are compressed

build failure (#130367)

csdc 1 unresolved S +35 −0

67d

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130367 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The osc half of the original change ("osc: improve can_merge_pages()") is already on master as 366fe06822, so this ports only the test remainder under a new Change-Id, with the original kept as Was-Change-Id.
- Adds test_2001; its client-version gate (a fix contained in this series) converted to compression_supported. test_1030 is left in place: the series' final state keeps both tests.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: check that all chunks are compressed

Add sanity-compr test_2001, which streams a large sequential
write to a compressed file and verifies from the client-side
compression stats that (nearly) every chunk was written
compressed, i.e. that the OSC forms full compressible chunks.

This exercises the can_merge_pages() behaviour that landed on
master in commit 366fe06822 ("LU-10026 osc: improve
can_merge_pages()"): pages with different OBD_BRW_ASYNC /
OBD_BRW_SOFT_SYNC hints must still merge into full chunks.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-11063
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Was-Change-Id: I6dd2643c93739fd6abaeb7add27f530a57fdcf64
Change-Id: Ice73943d4a2c297c1a95e0754b75b1f157df6c6f

68393 master J ✓B ✗T –R 0/2

LU-10026 tests: disable 1081 in sanity-compr

build failure (#130365)

csdc 1 unresolved XS +2 −0

67d

build #130365 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

always_except tag renumbered to LU-10026.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: disable 1081 in sanity-compr

causing too many failures.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10077
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I2b389c2ae2c733c3e6b8b100cdd0ca84df303246

68388 master J –B ✗T –R -1/2−

LU-10026 ofd: don't increment csdc counters

build failure (#130360)

csdc M +64 −21

67d

build #130360 FAILURE

no vote yet: Alex Zhuravlev

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: don't increment csdc counters

if corresponding event doesn't happen on the server.

add few more tests for compression counters.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10647
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id9badab250a3337d6a968680ce37e41e74b9c269

68387 master J !B ✗T –R 0/2

LU-10026 ofd: persistent compression counters

build failure (#130359)

csdc 10 unresolved L +333 −1

67d

build failure (#130359)

9 unresolved thread(s) await your reply — last from Misc Code Checks Robot (Gatekeeper helper)

janitor run: 20 failing config(s), none unique to this patch — janitor results

build #130359 FAILURE

no vote yet: Alex Zhuravlev

threads: 9 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- Context-only adaptation: struct ofd_device gained members on master (new fields appended at the end), and the include list and module-init surroundings in ofd_dev.c differ.
- test_1123 gained a compression_supported gate (the original had none; without it the test would run and fail on servers without compression).
- lprocfs_load_stats/lprocfs_save_stats land in lprocfs_status_server.c as in the original; both EXPORT_SYMBOL'd (used from ofd.ko).

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1618 — warn: lprocfs_save_stats():passing zero to 'ERR_CAST'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():passing zero to 'ERR_CAST'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1629 — error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1630 — warn: lprocfs_save_stats():possible memory leak of 'buf'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():possible memory leak of 'buf'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1641 — error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_save_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1672 — warn: lprocfs_save_stats():possible memory leak of 'buf'

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

warn: lprocfs_save_stats():possible memory leak of 'buf'

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1716 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1723 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1737 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

reply PS1 2026-08-26 13:37 Misc Code Checks Robot (Gatekeeper helper) lprocfs_status_server.c:1750 — error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-08-26 13:37

error: lprocfs_load_stats():'dt' dereferencing possible ERR_PTR()

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: persistent compression counters

ofd to save compression stats every hour and at umount,
then to load the saved stats at mount time so compression
stats can be found persistent and repsresenting overall
compression usage.
the file is called "compr_stats", stored in the root dir
of underlaying fs and uses YAML format:
- { name: uncompressed_objects, count:0, min:92, max:0, sum:0 }
- { name: compressed_objects, count:1, min:92233, max:0, sum:0 }
- { name: compr_chunks_read_disk, count:0, min:9223, max:0, sum:0 }
- { name: compr_bytes_read_disk, count:0, min:92233, max:0, sum:0 }
- { name: compr_bytes_read_user, count:0, min:92233, max:0, sum:0 }
- { name: compr_chunks_read_decompress, count:0, min:9, max:0, sum:0 }
- { name: compr_bytes_read_decompresse, count:0, min:9, max:0, sum:0 }
- { name: compr_chunks_write_disk, count:4, min:1, max:8, sum:11 }
- { name: compr_bytes_write_disk, count:4, min:8192, max:1310, sum:20}
- { name: compr_bytes_write_user, count:4, min:8192, max:10486, sum:1}

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10130
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I59016c473607fe7e4aa2583c28dfc223bc3d9b7c

68386 master J !B ✗T –R 0/2

LU-10026 ofd: support for compression counters

build failure (#130358)

csdc 1 unresolved L +257 −38

67d

janitor run: 21 failing config(s), none unique to this patch — janitor results

build #130358 FAILURE

no vote yet: Alex Zhuravlev

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 12:50 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 12:50

[Marc Bot] upstream remarks for reference:

- The two ofd_compress.c hunks around is_chunk_start() re-applied against the folio form of the call sites.
- test_1122 anchored after test_1121; its version gate converted to compression_supported.
- Commit message reworded to drop downstream release-version references in the two test-skip sub-sections.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 ofd: support for compression counters

the patch adds additional compression related counters:
compr_bytes_read_disk -  bytes were read from disk
compr_chunks_read_disk -  chunks were read from disk
compr_bytes_read_user -  bytes were sent to clients
compr_bytes_read_decompressed -  bytes were sent after decompression
compr_chunks_read_decompressed -  chunks were decompressed
compr_bytes_write_disk -  bytes were sent to disk
compr_chunks_write_disk -  chunks were sent to disk
compr_bytes_write_user -  bytes clients send to server
compr_chunks_rmw -  chunks were involved in read-modify-write
compr_bytes_rmw -  disk bytes were involved in read-modify-write
compr_bytes_rmw_user -  bytes clients sent causing read-modify-write
decompression_time - time in usec spent in decompression

tests: skip sanity-compr/1122 with old OST

skip sanity-compr test_1122 when the OST does not provide the
server-side compression counters.

DDN-bug-id: EX-10586
Was-Change-Id: I2f5608e93880ad6569d68e83110144bfcf791f80

tests: adjust the sanity-compr/1122 version check

adjust the version check for the server-side counters.

DDN-bug-id: EX-10890
Was-Change-Id: I4f58cac84a7c7dcbec667292528ae4667396f1f0

Test-Parameters: forjanitoronly
DDN-bug-id: EX-10128
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie974cbd15451ae790310ec8e3d5c729a849c882a

68376 master J ✗1B ✗T –R -1/2−

LU-10026 tests: skip sanity-pfl tests if no compression

build failure (#130350)

1 unique failure csdc 1 unresolved S +17 −29

67d

build failure (#130350)

janitor: 1 test failure unique to this patch — sanity-pfl@ldiskfs+DNE:test_100b

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-pfl@ldiskfs+DNE:test_100b	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #130350 FAILURE

no vote yet: Artem Blagodarenko

threads: 0 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS1 2026-08-26 10:37 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS1 · 2026-08-26 10:37

[Marc Bot] upstream remarks for reference:

Re-derived by hand - every hunk's context had version gates this series already replaced with compression_supported. Each test_100* gained 'compression_enabled || skip' and lost its per-test llite.*.enable_compression save/set/restore block (100a keeps a bare enable, 100e keeps its disable + re-enable trap), relying on the default-on behaviour from the finer-grained enable_compression control patch.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS1 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: skip sanity-pfl tests if no compression

Skip sanity-pfl test_100* for servers that do not understand CSDC.

Test-Parameters: forjanitoronly
DDN-bug-id: EX-9482
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ib3fcfd77e9e7ffb122ed6ade9015b02d42ea8319

66276 master J !B ✗T –R 0/2

LU-10026 lov: fallocate is not allowed for compressed files

build failure (#130268)

csdc 4 unresolved S +30 −0

68d

build failure (#130268)

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 2 failing config(s), none unique to this patch — janitor results

build #130268 FAILURE

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

The commit message only describes disabling fallocate, but the diff also adds a `rsync --sparse` decompression round-trip (the `decomp2` / "fiomap case" block) to compress_type() in sanity.sh. That change is about the sparse-copy/FIEMAP read path, not fallocate, and isn't mentioned here. Is it meant to be part of this patch, or should it be split into its own change with its own Change-Id?

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(suggestion) This only checks lsm_entries[0]. lsme_compr_type is set per component (lov_ea.c copies lcme_compr_type for each LOV_PATTERN_COMPRESS entry), so a composite/PFL file could have an uncompressed component 0 and a compressed later component. Would fallocate over such a file still be allowed even though it targets a compressed component? (The ll_fiemap() guard has the same entry[0]-only limitation.)

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) `lio` and `loo` here just re-derive values already in scope: the outer `lio` (declared at the top of the function) is the same `cl2lov_io(env, ios)`, and `loo` is `lio->lis_object`. The inner `struct lov_io *lio` also shadows the outer one. This could just use the existing `lio->lis_object` without the extra declarations.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:57

(minor) fallocate(2) is directly user-triggerable, so a CWARN here lets an unprivileged caller flood the console by repeatedly calling fallocate on a compressed file. The equivalent compressed-file guard in ll_fiemap() (lov_object.c around line 2021) returns -EOPNOTSUPP silently. Consider CDEBUG(D_VFSTRACE, ...) or a rate-limited message instead.

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lov: fallocate is not allowed for compressed files

Client Side Data Compression allocates blocks after a compression.
It is impossible to preallocate blocks for the whole file, so
fallocate should be disabled in case of compression.

Test-Parameters: forjanitoronly
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ie834ace183fdcec0d7d6f747237e0964c3c4120b

66264 master J –B ✗T –R 0/2

LU-10026 man: document supported compression types

build failure (#130260)

csdc 3 unresolved S +16 −0

68d

build failure (#130260)

2 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

build #130260 FAILURE

no vote yet: Sebastien Buisson

patchsets: PS1 05-28 PS2 06-11 PS3 06-11 PS4 08-25

threads: 2 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS4 2026-08-25 21:23 Marc Vef patchset comment — [Marc Bot] upstream remarks for reference:

Marc Vef PS4 · 2026-08-25 21:23

[Marc Bot] upstream remarks for reference:

Rebase note: the lfs-setstripe.1 sentence this patch anchors on was
reformatted on the base (.B "lfs getstripe" roff style); the
compression-type list is inserted after the reformatted sentence, content
unchanged.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(suggestion) This new list of currently supported types omits zstd (and fast/best), but the untouched line just above still advertises "1-21 for zstd" as a valid level range. If zstd is not currently supported, consider reconciling the two so the level-range hint doesn't contradict this list.

Gerrit AI review for Lustre PS3 · 2026-07-15 00:56

(defect) Is the lz4 default really 1? When no level is given, llapi_parse_compress_type() passes level = -1, and from_lz4_level(-1) returns LZ4HC_DEFAULT_CLEVEL - 1 = 8, which lfs getstripe maps back via to_lz4_level() to 9. So the effective default looks like 9 (highest ratio / slowest), not 1. The LZ4_ACCELERATION_DEFAULT = 1 case only applies when an explicit level < 3 is given, not for the unspecified default. Compare gzip just above, where the documented default of 6 does match from_gzip_level(-1).

tags: csdc

owner Marc Vef · uploader Marc Vef · PS4 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 man: document supported compression types

Update lfs-setstripe to mention the currently supported compression
types: gzip, lz4, lz4fast and lzo.

Test-Parameters: forjanitoronly
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I9b587516d27b882bec5855da4948f489e5a0041f

65762 master J ✗1B ✓T ✗1R 0/2

LU-10026 csdc: prefer uncompressed mirror for read and write

janitor: 1 test failure unique to this patch — sanity3@zfs:test_907

1 unique failure csdc 4 unresolved L +213 −47

72d

janitor: 1 test failure unique to this patch — sanity3@zfs:test_907

enforced failed: review-dne-selinux-ssk-part-2

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity3@zfs:test_907	seen in 57 other reviews

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #129912 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Andreas Dilger, Artem Blagodarenko, Zhenyu Xu

patchsets:

+7 earlier

PS1 05-08 PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 07-27 Artem Blagodarenko PS7 07-28 Artem Blagodarenko

PS8 08-19 Artem Blagodarenko PS9 08-19 Artem Blagodarenko PS10 08-20 Artem Blagodarenko PS11 08-20 Artem Blagodarenko PS12 08-21 Artem Blagodarenko

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

This bonus is added once per allocated stripe, which is exactly the weight the OS_STATFS_NONROT bonus a few lines above uses, so the two can cancel out.

Two mirrors, no LCME_FL_PREF_WR anywhere, one stripe each:

    mirror 0: compressed,   stripe on a NONROT OST -> pref = 1
    mirror 1: uncompressed, stripe on a rotating OST -> pref = 1

lme_preference ties, so the `!found_preferred` fallback below keeps the lower index (`>` is strict) and sets lme_prefer on mirror 0. lod_primary_pick() then classifies mirror 0 as case B (preferred + compressed) and mirror 1 as case C, and B outranks C, so the write goes to the compressed mirror.

A wider compressed mirror wins outright rather than tying: 4 NONROT stripes compressed = 4 vs 1 NONROT stripe uncompressed = 2.

Mixed flash/disk mirrors are the case the non-rotational preference was added for, so this is not an exotic layout. Should the uncompressed bonus be on a different scale than the per-stripe NONROT count (or applied once per mirror) so it can't be traded away against OST speed?

The comment also says "mirror", but this sits in the per-stripe loop of one component.

reply PS12 2026-08-22 12:56 Gerrit AI review for Lustre lod_object.c:8060 — (style) Not a bug, but this comment has a UTF-8 em dash in it; the rest of the tree is plain ASCII. A plain '-' would do.

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

(style) Not a bug, but this comment has a UTF-8 em dash in it; the rest of the tree is plain ASCII. A plain '-' would do.

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

Ranking B (preferred + compressed) above C (uncompressed) reads as "an explicit `lfs setstripe --flags=prefer` beats the compression heuristic". But lme_prefer is not only user intent: when no component carries LCME_FL_PREF_WR, lod_fill_mirrors() picks the highest lme_preference non-stale mirror and sets lme_prefer = 1 on it.

So tier B is reachable on a file the user never flagged, and the heuristic ends up overriding its own compression preference. Should case B be restricted to mirrors that actually carry LCME_FL_PREF_WR, or should the synthesized preference be tracked separately from the user flag?

Gerrit AI review for Lustre PS12 · 2026-08-22 12:56

(minor) ci_ndelay_tried > 0 means "not the first lov_io_mirror_init() for this env", which is broader than "this is an FLR error retry".

ll_file_io_generic() restarts the io for reasons unrelated to mirror failure and preserves the counter:

    partial_io -> io->ci_need_restart = 1   (file.c)
    retried = io->ci_ndelay_tried; goto restart;

A single buffered read larger than max_io_bytes therefore takes the uncompressed mirror for the first chunk only; from the second chunk on ci_ndelay_tried is non-zero, lis_mirror_index has been advanced by one, and the first in-range mirror is accepted even when it is compressed. The layout-change restart in vvp_io_iter_init() has the same shape.

Would gating on something that only the -EAGAIN mirror-failover path sets (io->ci_tried_all_mirrors, or a dedicated flag set where cl_io_loop() retries) keep the failover fix without giving up the preference on ordinary io continuation?

tags: csdc

owner Marc Vef · uploader Artem Blagodarenko · PS12 uploaded 73d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: prefer uncompressed mirror for read and write

When accessing a mirrored file with both compressed and uncompressed
components, choose uncompressed components for read (lov_io.c).

When writing to mirrored files with both compressed and uncompressed
mirrors, prefer the uncompressed components to write; this is better
for performance, more compatible with older clients, and better fits
the model of compressing files after initial write. The write-side
preference is implemented via lod_fill_mirrors() and lod_primary_pick()
using a new lme_compressed flag.

Note: the changes to lod_fill_mirrors() and lod_primary_pick() affect
all FLR files, not only those with compressed mirrors. Plain FLR writes
will also use the updated preferred-mirror selection logic.

DDN-bug-id: EX-6510
Was-Change-Id: I62a117d5cc3d34e2c0c96d1a9ade8eef0a2d1291

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@gmail.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I043b27bd891c039901075a08c76630f8f0f9f182

65769 master J !B ✓T ✗1R 0/2

LU-10026 csdc: add ll_compr_hdr structure

enforced failed: review-dne-selinux-ssk-part-2

csdc hidden 4 unresolved L +270 −58

72d

enforced failed: review-dne-selinux-ssk-part-2

4 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 3 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #129913 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Andreas Dilger, Artem Blagodarenko

patchsets:

+9 earlier

PS1 05-08 PS2 05-28 PS3 05-28 PS4 06-11 PS5 06-11 PS6 07-28 Artem Blagodarenko PS7 07-28 Artem Blagodarenko PS8 07-31 Andreas Dilger PS9 07-31 Andreas Dilger

PS10 08-19 Artem Blagodarenko PS11 08-19 Artem Blagodarenko PS12 08-20 Artem Blagodarenko PS13 08-20 Artem Blagodarenko PS14 08-21 Artem Blagodarenko

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS14 2026-08-22 12:39 Gerrit AI review for Lustre COMMIT_MSG:21 — (minor) Two hunks aren't accounted for by the body.

Gerrit AI review for Lustre PS14 · 2026-08-22 12:39

(minor) Two hunks aren't accounted for by the body.

COMPR_GET_CHUNK_SIZE() in lustre_user.h gains argument parentheses and a 1U base. That is a real hardening of the macro, but nothing in the message mentions it.

llapi_layout_compress_set() gains a type range check that makes the function return -1/EINVAL for inputs it previously accepted. That is a behaviour change to a public liblustreapi entry point and deserves a line.

Are both meant to ride along in this patch?

Gerrit AI review for Lustre PS14 · 2026-08-22 12:39

(minor) This isn't a bug today, but the two arguments are treated asymmetrically: `lvl` is masked with 0x0f, `bits` isn't. The matching accessor LLCH_CHUNK_LUM_BITS() does mask on the way out.

    LLCH_LEVEL_AND_CHUNK(3, 16) -> (16 << 4) | 3 == 259

The result is an int, so the overflow only disappears when it's stored into the __u8 llch_level_and_chunk, and the header then decodes as chunk_lum_bits 0 (64 KiB) rather than anything detectable. Valid chunk_lum_bits is 0..10 so no current caller can hit it, but masking `bits` the same way as `lvl` would keep the macro self-contained.

Gerrit AI review for Lustre PS14 · 2026-08-22 12:39

(minor) Documentation/man3/llapi_layout_compress_set.3 doesn't seem to be updated to match. Its ERRORS section lists EINVAL only for the stripe_size/chunk_size cases, and DESCRIPTION enumerates the valid `type` values without saying that an out-of-range type or the new LL_COMPR_TYPE_UNCHANGED sentinel is now rejected. Should the man page change land in the same patch?

Gerrit AI review for Lustre PS14 · 2026-08-22 12:39

(style) Not a bug, but the first test can never decide the branch on its own: LL_COMPR_TYPE_UNCHANGED is 255 and LL_COMPR_TYPE_MAX is 8, so `type >= LL_COMPR_TYPE_MAX` already covers the sentinel. Keeping it reads as documentation, but a reader has to work that out.

tags: csdc

owner Marc Vef · uploader Artem Blagodarenko · PS14 uploaded 73d ago · NEW · open in Gerrit ↗

commit message

LU-10026 csdc: add ll_compr_hdr structure

It is an important structure for CSDC feature. Will be used
in the next patches.

Also adds:
- LL_COMPR_TYPE_UNCHANGED enum value (used to signal that the
  compression type should not be changed on update).
- LL_LZ4FAST_MAX_LEVEL define (15: the largest value the 4-bit
  stored level field can hold for the LZ4FAST algorithm).
- LLCH_FL_COMPR_CSUM / LLCH_FL_HDR_CSUM flags in llch_flags to
  signal checksum presence explicitly (a crc32 of 0 is ambiguous
  otherwise).
- wirecheck: pin COMPR_CHUNK_MIN_BITS, COMPR_CHUNK_MAX_BITS, and
  LL_LZ4FAST_MAX_LEVEL values.

Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I26f2fc7c0212f1a4cad36805c75aa766d6aed548

65761 master J !B ✓T ✗1R 0/2

LU-10026 utils: add 'lfs find' support for compressed file

enforced failed: review-dne-selinux-ssk-part-2

csdc 9 unresolved L +715 −67

72d

enforced failed: review-dne-selinux-ssk-part-2

9 unresolved thread(s) await your reply — last from Gerrit AI review for Lustre

janitor run: 4 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #129911 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Andreas Dilger, Artem Blagodarenko, Zhenyu Xu

patchsets:

+12 earlier

PS13 08-19 Artem Blagodarenko PS14 08-19 Artem Blagodarenko PS15 08-20 Artem Blagodarenko PS16 08-20 Artem Blagodarenko PS17 08-21 Artem Blagodarenko

threads: 9 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(minor) The first three bullets read as new functionality, but `--comp-flags=[^]compress` and `--comp-flags=[^]nocompr` already work (LCME_FL_COMPRESS/LCME_FL_NOCOMPR are in comp_flags_table[]), and so does `-L compress` (LOV_PATTERN_COMPRESS is in lov_pattern_names[]). This patch documents them and adds test coverage rather than adding them. Could the wording say "document" for those three, so the message matches the diff?

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) COMPR_LEVEL_MAX is already defined with the same value earlier in this header, just after `enum ll_compr_type`, with a comment explaining it. Two definitions can drift apart later; can this one be dropped?

reply PS17 2026-08-22 12:35 Gerrit AI review for Lustre sanity-pfl.sh:3078 — (style) extra blank line. Also z1..z4 here, and `expect` in test_100d, are not declared `local`.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) extra blank line. Also z1..z4 here, and `expect` in test_100d, are not declared `local`.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(style) not a bug, but `compress-chunk` already appears in this long_opts[] further down, so it is listed twice now. getopt_long takes the first match so it still works; `compr-chunk` looks like the only new alias actually needed here.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(typo) this comment uses a UTF-8 em dash; the tree is otherwise plain ASCII. Same in the `%LZ` directory-branch comment in liblustreapi_pfind.c.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) atoi() cannot report a parse failure, so `--compr-level=abc` silently becomes level 0, and `--compr-level=+abc` becomes "level > 0", i.e. every compressed component. The unsigned range check above cannot catch it.

`-Z gzip:abc` is rejected by llapi_parse_compress_type(), and the setstripe `--compress-level` path uses strtol() with an end-pointer check, so the three spellings of the same thing disagree. strtol() plus `*end != '\0'` here would line them up.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(minor) This helper backs every numeric `lfs find` predicate, and the new exact-match branch only changes behaviour when `margin == 0`. `lfs find -z N` with a plain number is such a caller: lfs.c does `fp_ext_size_units /= SEL_UNIT_SIZE`, so 1/1024 becomes 0, and an exact `-z` match that previously reported "no match" now matches. That may well be desirable, but it is a silent change to an unrelated option.

The new compression predicates are the only callers passing margin 0; the rest pass 1 or a unit size, and with margin 1 this function already returns 1 on an exact match. Passing 1 there would avoid touching the shared helper, and would also make `--compr-level=-N` / `--compr-chunk=-N` mean "< N" like `-c -N`, `--size -N` and friends, rather than "<= N".

The table above the function (rows 5 and 6, and the note about the interval `(limit - margin, limit]`) no longer describes the code either.

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) Negation is handled differently for level/chunk than for type. `fp_exclude_compr_lvl` is passed into find_value_cmp() as `negopt`, so `found_lvl` ends up meaning "some component does not match" instead of "no component matches", and the aggregation here then ignores the exclude flag entirely - unlike the fp_check_compr_type block just above.

For a file with two compressed components, level 3 and level 5:

    lfs find ! --compr-level=3 dir

entry(lvl 5) makes find_value_cmp() return -1, negopt flips it to 1, found_lvl is set, and the file is reported - even though it does have a level-3 component. The equivalent `! --compr-type=gzip` on a gzip+lz4 file correctly does not match.

Should the level and chunk comparisons drop the negopt argument and be inverted here the same way type is?

Gerrit AI review for Lustre PS17 · 2026-08-22 12:35

(defect) `fp_lum_size` is the size common_param_init() allocated for the lmd buffer (>= XATTR_SIZE_MAX, 64KiB), not the length of the layout stored in it.

For a non-composite lum, llapi_layout_get_by_xattr() never narrows that value, so llapi_layout_objects_in_lum() computes (65536 - 32) / 24 = 2729 objects and __llapi_comp_alloc() rejects it with EINVAL because 2729 > LOV_MAX_STRIPE_COUNT (2000).

So this call fails for every plain V1/V3 file and always falls through to llapi_layout_get_by_path(); only composite layouts reach the new path, because the entry loop resets the size to `ent->lcme_size`. Passing the actual layout length instead would make the fast path work for non-PFL files too.

tags: csdc

owner Marc Vef · uploader Artem Blagodarenko · PS17 uploaded 73d ago · NEW · open in Gerrit ↗

commit message

LU-10026 utils: add 'lfs find' support for compressed file

* Add "--comp-flags=[^]compress" to locate file with/without
  compressed components.
* Add "--comp-flags=[^]nocompr" to locate file with/without setting
  component compress preference.
* Add "[!] --layout=compress" to locate file with/without compressed
  components.
* Add "[!] --compress-type=<compress-type>" to locate compressed file
  with/without specified compress algorithm.
* Add "[!] --compress-level=[+-]<compress-level>" to locate compressed
  file with/without specified compress level.
* Add "[!] --compress-chunk=[+-]<compress-chunk>" to locate compressed
  file with/without specified compress chunk in KiB.

utils: add -Z option for 'lfs getstripe/find'

Add support for "lfs getstripe -Z" to get the last instantiated
component compression information.

Add support for "lfs find -Z <type>[:[+-]<level>]" to keep consistent
options with "lfs setstripe -Z".
DDN-bug-id: EX-6856
Was-Change-Id: Ia5e2a2ea6937dc2b46e224cec808504a196b974c

utils: logical AND for 'lfs find' compression exprs

All search expressions provided to 'lfs find' must be combined as a
logical AND. Fix newly added options for compression support, so that
they comply with this logical AND.
DDN-bug-id: EX-7714
Was-Change-Id: I3b28cd87c1d304df6d04753b413d46f5abcfe16e

csdc: don't set compression layout when disabled

When llite_enable_compression is disabled
(lfs set_param llite.*.enable_compression=0), we should check
it before sending it to MDS lest we get a file with compressed
component which we cannot handle.
DDN-bug-id: EX-7593
Was-Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f

utils: support 'lfs find --printf %LZ'

Add support for "lfs find --printf %LZ" to print the compression
type:level of the last instantiated component of a file.
DDN-bug-id: EX-6856
Was-Change-Id: Iaf1b6c031b06c70e7b5be51354697aa6bdcc9850

csdc: right error code in case compression is disabled

Currently, if compression is disabled on a client the error
message is confusing "Cannot set layout EA: Unknown error 524"

Let's replace the error code from  ENOTSUPP to EOPNOTSUPP,
so error messages become more informative

ctl get_param -n llite.*.enable_compression
0
lfs setstripe -i 0 -c 1 -E -1 -Z lz4 /mnt/lustre/foo.txt
Cannot set layout EA: Operation not supported
lfs setstripe: cannot create composite file
'/mnt/lustre/foo.txt': Inappropriate ioctl for device

This message could be even more informative, but this version
is already much better.
DDN-bug-id: EX-10577
Was-Change-Id: I7fd67d5d3d20bc7de169f35f6fa63b53f2bfe718

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic36946738c3463fd862aeca4ee2e2c2ed85eff84

▶

pt_ecro

(1)

68697 master J !B ✓T ✗1R ✗

LU-19548 mdd: refuse to split a parity mirror to a victim

parked by your own -1

CR -1 pt_ecro S +28 −5

53d

janitor run: 3 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-part-2	RHEL 9.7 / x86_64	ran 11 tests. 1 tests failed: sanity-lfsck.	session

build #131271 SUCCESS · tests all Maloo sessions

reviews: Marc Vef -1 2026-09-08 20:12

patchsets: PS1 09-07 PS2 09-07

tags: pt_ecro

owner Marc Vef · uploader Marc Vef · PS2 uploaded 55d ago · NEW · open in Gerrit ↗

commit message

LU-19548 mdd: refuse to split a parity mirror to a victim

A file whose layout holds only an EC parity mirror cannot be read:
parity is not file data, and there is no data mirror left to serve
reads. lfs already refuses to split a parity mirror into a file of
its own; the MDS should not rely on the client for that. Pass whether
a victim object exists (mrd_obj != NULL) into mdd_split_ea() and
return -EINVAL when the target is a parity mirror. Destroying a parity
mirror with 'lfs mirror delete' is unaffected.

lfs no longer takes the old-MDS volatile fallback for a parity
target. An MDS that knows what a parity mirror is also has the
fd == fdv purge path (LU-14521), and with the preceding LU-18466 fix
that path no longer fails with -EINVAL, so the fallback could only
build a volatile victim that the MDS now refuses. A client old enough
to always route 'lfs mirror delete' through a volatile file cannot
delete a parity mirror against this MDS.

Assisted-by: ClaudeCode:Fable-5.1 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Id41a40b21f3d31635904a1233653b91ba6d55a6c

▶

(untagged)

(7)

70236 master J –B ✓T ✓R 0/2

LU-0000 tests: private test patch, do not land

CI green, no reviews yet — add reviewers

XS +1 −0

build #134893 SUCCESS · tests all Maloo sessions

patchsets: PS1 10-30 PS2 10-30

owner Marc Vef · uploader Marc Vef · PS2 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-0000 tests: private test patch, do not land

Trivial comment change for a private Gerrit test change.

Test-Parameters: trivial
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I78dd5f283fdd3399a22e222bb60d2ab284cecb9e

70000 master J –B ✓T ✓R +1/2

LU-20723 tests: regroup subtests skipped in 1-client sessions

no vote yet from Andreas Dilger

L +251 −264

build #134772 SUCCESS · tests all Maloo sessions

reviews: Sebastien Buisson +1 2026-10-30 09:15

no vote yet: Andreas Dilger

patchsets: PS1 10-23 PS2 10-23 PS3 10-26 PS4 10-27 PS5 10-29

owner Marc Vef · uploader Marc Vef · PS5 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-20723 tests: regroup subtests skipped in 1-client sessions

Most review test sessions are moving to a single client VM, with
the tests that need a second client grouped into a few two-client
sessions.  Five subtests in the single-client sessions skip without
a second client.

Move the three that need a second node to suites that keep two
clients:
- replay-single test_70a to replay-dual test_200
- sanity test_160v to replay-dual test_201
- replay-single test_200 to recovery-small test_200, and update the
  "DO NOT MODIFY" note in ptlrpc_expire_one_request()

Run the other two on one node:
- sanityn test_33b runs createmany on $CLIENT1 only
- replay-vbr test_10b uses $MOUNT3 for its third client

Test-Parameters: trivial
Test-Parameters: clientcount=1 mdscount=2 mdtcount=4 testlist=sanityn env=ONLY=33b
Test-Parameters: clientcount=1 mdscount=2 mdtcount=4 testlist=replay-vbr env=ONLY=10b
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ic1db87ab68ff45e81af569a57592e0cdb298cb71

68837 master J –B ✓T ✓R +2/2

LU-930 doc: fix outdated and dead URLs

all green — queued for landing (master-next)

master-next 2 unresolved S +17 −21

all green — queued for landing (master-next)

2 reviewer comment(s) to address if a respin is needed — last from Andreas Dilger

build #134645 SUCCESS · tests all Maloo sessions

reviews: Arshad Hussain +1 2026-10-29 03:44 · Andreas Dilger +1 2026-10-28 21:30

no vote yet: Timothy Day

patchsets:

+1 earlier

PS1 09-11

PS2 09-11 PS3 10-27 PS4 10-28 PS5 10-28 PS6 10-28

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS6 2026-10-28 21:30 Andreas Dilger lbuild:1200 — This patch already landed long ago.

Andreas Dilger PS6 · 2026-10-28 21:30

This patch already landed long ago.

reply PS6 2026-10-28 21:30 Andreas Dilger lbuild:1202 — This PR is closed. Maybe this can now be simplified?

Andreas Dilger PS6 · 2026-10-28 21:30

This PR is closed.  Maybe this can now be simplified?

owner Marc Vef · uploader Marc Vef · PS6 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-930 doc: fix outdated and dead URLs

Switch the remaining http:// lustre.org links in README to https://.

Point links whose hosts have moved or gone away at their current
location: OpenZFS and DKMS on GitHub, the CRUSH paper on ceph.io,
and Oracle's copy of the Sun GSS-API guide.

Drop the links to the FID interoperability design page, which no
longer exists; the relevant parts are in the lustre_fid.h comment.

Test-Parameters: trivial
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I87fdf6c67c631675dbe00acce8de873fb6eae33e

62537 master J –B ✗T –R 0/2

LU-18756 ofd: repair missing parent FID on OST read

work in progress

WIP L +351 −38

12d

build #134162 ABORTED

no vote yet: Andreas Dilger

patchsets: PS1 11-14 PS2 10-21 PS3 10-21

owner Marc Vef · uploader Marc Vef · PS3 uploaded 12d ago · NEW · open in Gerrit ↗

commit message

LU-18756 ofd: repair missing parent FID on OST read

An OST object gets its parent FID (trusted.fid) only when a client
writes to it, truncates it or changes its ownership. Objects that
never received such an RPC, e.g. the unwritten stripes of a sparse
file, have no parent FID, so scanning an OST cannot map them back to
the MDT file that references them, e.g. to migrate them off the OST.

On OST_READ, queue a repair of a missing parent FID to the resource ID
repair thread, which writes it in the same transaction as any missing
UID/GID/PROJID. The parent FID, stripe index and OST layout from the
obdo are stored as a write stores them, so LFSCK finds the component
ID of PFL files. Like the ID repair, it is disabled by
enable_resource_id_repair=0. It is queued whether or not
enable_resource_id_check is set, but not when that check denies
access.

Only a missing parent FID, or one that fails fid_is_sane(), is
repaired. A valid parent FID that differs from the obdo may come from
a second layout referencing the object, which is left to LFSCK. The
parent FID sent by the client is trusted like the UID/GID repaired
from the same RPC, also from clients of read-only exports; a write
stores it unconditionally.

filter_fid::ff_parent::f_ver holds the stripe index, so only f_seq and
f_oid of the parent FID are checked with fid_is_sane().

Stripes that are never read, e.g. beyond EOF, still have no parent
FID.

Add sanity-sec test_75c: unwritten stripes 0 and 1 of PFL files get
their parent FID and layout on read, with the ID check disabled and
enabled; a read with another parent FID keeps a valid one; and an
object missing both IDs and parent FID gets both repaired.

Test-Parameters: testlist=sanity-sec env=ONLY=75b,75c
Test-Parameters: testlist=sanity-lfsck
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Iafa97b1204a96ef3349009a340fd812528ae43ef

68696 master J ✗1B ✗T –R ✗

LU-18466 mdd: declare layout split before purge

parked by your own -1

CR -1 1 unique failure S +10 −7

54d

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-quota@ldiskfs+DNE:test_58	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #131270 FAILURE

reviews: Marc Vef -1 2026-09-08 20:12

patchsets: PS1 09-07 PS2 09-07

owner Marc Vef · uploader Marc Vef · PS2 uploaded 55d ago · NEW · open in Gerrit ↗

commit message

LU-18466 mdd: declare layout split before purge

mdd_xattr_split() declares the mirror purge before the layout split
when a mirror is deleted in place (fd == fdv, LU-14521). Declaring
the split bumps the object's layout generation, so the second LOV
declaration on the same object trips lod_save_layout_gen_intrans()
and the whole operation fails with -EINVAL:

  lod_save_layout_gen_intrans() obj ... gen changed from 4 to 5 in
  trans declaration

lfs treats that -EINVAL as "old MDS rejects fd == fdv" and silently
retries through a volatile victim file, so every 'lfs mirror delete',
'lfs mirror split -d', 'lfs migrate -L mdt' and 'lfs pcc delete' has
taken the volatile-file path instead of the purge path since the
generation check was added, printing a stray "cannot get UNLOCK lease
...: Invalid argument (22)" on the way. Declare in the order the
changes are applied, split first, so the recorded generation still
matches.

Fixes: d5ab62af2416 ("LU-17048 mdd: protect layout change in MDD layer")
Assisted-by: ClaudeCode:Fable-5.1 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I624f537059c398facfb4693754731222e7e55e77

68346 master J –B ✗T –R -1/2−

LU-10026 tests: improve/skip sanity test_460a

build failure (#130326)

M +108 −27

66d

build #130326 FAILURE

patchsets: PS1 08-25 PS2 08-26

owner Marc Vef · uploader Marc Vef · PS2 uploaded 68d ago · NEW · open in Gerrit ↗

commit message

LU-10026 tests: improve/skip sanity test_460a

Skip sanity test_460a for el9.2 clients, since they appear to be
failing that test regularly, but no other distro client is.
Improve the log messages to see what stage is currently running.
Limit the "cmp --verbose" messages to one chunk, otherwise it
may print the entire 14MB test file (about 80 MiB of ASCII).

Move enable_compression() and disable_compression() functions
into test-framework.sh so that they can be used for all tests.

Set LFS_SETSTRIPE_COMPR_OK=y in enable_compression() since we
already know this is a preview and don't need it printed.

Allow sanity-compr.sh to specify SANITY_ONLY and/or SANITYN_ONLY,
and skip the other test script run if only one of them is set.

Test-Parameters: forjanitoronly
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8cb2f67689824513335f3fa65e9ea7519e3ebbe5

60957 master J ✓B ✓T ✓R -1/2−

LU-18756 tests: sanity-sec 75a sync resource IDs

Andreas Dilger voted -1

mtfl_p2 stalled 255d 1 unresolved S +6 −4

255d

Andreas Dilger voted -1

1 thread(s) waiting on others

no vote yet from Sebastien Buisson

reviews: Andreas Dilger -1 2025-08-29 17:28

no vote yet: Sebastien Buisson

patchsets: PS1 08-28 PS2 08-28 PS3 08-29

threads: 0 need your reply · 1 waiting on others · 0 own notes · 0 bot

owner Marc Vef · uploader Marc Vef · PS3 uploaded 430d ago · NEW · open in Gerrit ↗

commit message

LU-18756 tests: sanity-sec 75a sync resource IDs

After sanity-sec 75a setups the test namespace, it creates and writes
to files before changing the user to the correct tenant user which
carries out the commands. Later, when the ID check is enabled, it
could happen that the corresponding OST object still has the initial
user ID set and that the "chown" changes have not yet been propagated
from the MDT to the OSTs. In such a case, the test fails the ID check
because the OST object has the wrong IDs set.

This patch adds an additional step after namespace creation to
synchronize ID changes to and between the servers before continuing
with the test.

Fixes: 719f6763fe ("LU-18756 sec: add resource id check to oss and mds")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=75
Test-Parameters: testlist=sanity-sec env=ONLY=75a,ONLY_REPEAT=100
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I07d602375fd34334dc5047cfecea60f09f03bd42

Carrying — uploaded by me, owned by others

8 open

▶

Failed / broken

(4)

62859 master J ✓B ✓T ✗12R -1/2−

LU-12668 ec: Add tests for computing the parity coverage

carrying Ronnie Sahlberg · needs rebase — checkpatch: cannot be cherry-picked

needs rebase ec2 pt_ecro L +460 −0

44d

needs rebase — checkpatch: cannot be cherry-picked

enforced failed: custom-1001, custom-1002, review-dne-subtest-change, review-dne-zfs-subtest-change, review-ldiskfs-dne-arm

failed enforced test	platform	detail
custom-1001	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
custom-1002	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-ldiskfs-dne-arm	RHEL 8.10 / x86_64, Rocky 9.5 / aarch64	ran 5 tests. 1 tests failed: sanity.	session

build #127014 SUCCESS · tests all Maloo sessions

no vote yet: Keguang Xu, Qian Yingjin, Maximilian Dilger

patchsets:

+42 earlier

PS43 05-06 PS44 05-14 Marc Vef PS45 05-20 Marc Vef PS46 06-22 Marc Vef PS47 06-26 Marc Vef

your previous vote: -1 on PS45 (2026-05-21 08:46) — now at PS47

tags: pt_ecro

owner Ronnie Sahlberg · uploader Marc Vef · PS47 uploaded 128d ago · NEW · open in Gerrit ↗

commit message

LU-12668 ec: Add tests for computing the parity coverage

When we write or verify the parities we no longer
use the whole stripe, instead we compute the coverage
of what ranges of parity is important and what can be
ignored.
This is based on SEEK_DATA/SEEK_HOLE and EOF.

Add tests that we compute these ranges correctly.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: Iaee9d0ccb0875f6d515a8adc322fa3cd37bfe0f6

57776 master J ✗2B ✓T ✓R +1/2−

LU-10026 lustre: add lz4 and lz4hc kernel modules

carrying Artem Blagodarenko · needs rebase — checkpatch: cannot be cherry-picked

needs rebase csdc 5 unresolved XL +4422 −2

46d

needs rebase — checkpatch: cannot be cherry-picked

Timothy Day voted -1

3 unresolved thread(s) await your reply — last from Timothy Day

2 thread(s) waiting on others

no vote yet from Sebastien Buisson, Oleg Drokin

janitor flagged test failures unique to this patch — superseded (tests have since passed) — full janitor results

unique failing test	history
sanity-lfsck@zfs:test_18c	seen in 25 other reviews
sanity-scrub@ldiskfs+DNE:test_9	seen in 5 other reviews

build #125233 SUCCESS · tests all Maloo sessions

reviews: James Simmons +1 2026-05-29 01:12 · Marc Vef +1 2026-05-19 09:37 · Andreas Dilger +1 2026-05-19 20:23 · Timothy Day -1 2026-06-05 20:49

no vote yet: Sebastien Buisson, Oleg Drokin

patchsets:

+27 earlier

PS28 05-11 Marc Vef PS29 05-11 Marc Vef PS30 05-11 Marc Vef PS31 05-13 Marc Vef PS32 05-18 Marc Vef

your previous vote: -1 on PS27 (2026-05-07 21:28) — now at PS32

threads: 3 need your reply · 2 waiting on others · 0 own notes · 0 bot

reply PS32 2026-05-29 17:08 Timothy Day COMMIT_MSG:36 — This is false.

Timothy Day PS32 · 2026-05-29 17:08

This is false.

reply PS32 2026-05-29 17:08 Timothy Day COMMIT_MSG:60 — These modules do not compile on newer kernels (6.18, at least). I think it should be possible to build without these.

Timothy Day PS32 · 2026-05-29 17:08

These modules do not compile on newer kernels (6.18, at least). I think it should be possible to build without these.

reply PS32 2026-05-29 21:34 Oleg Drokin kcompr.c:86 — just as the smatch implies this should really be IS_ERR, I don't think filp_open can return NULL?

Oleg Drokin PS32 · 2026-05-29 21:34

just as the smatch implies this should really be IS_ERR, I don't think filp_open can return NULL?

tags: csdc

owner Artem Blagodarenko · uploader Marc Vef · PS32 uploaded 168d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lustre: add lz4 and lz4hc kernel modules

lz4 and lz4hc kernel modules implement compression according to the
lz4 and lz4hc algorithms respectively, through the kernel Crypto API.
lz4 module provides 2 cipher drivers under the generic name 'lz4':
* lz4-lustre-generic of type compression
* lz4-lustre-scomp of type scomp
lz4hc module provides 2 cipher drivers under the generic name 'lz4hc':
* lz4hc-lustre-generic of type compression
* lz4hc-lustre-scomp of type scomp

lz4 and lz4hc kernel module sources are copied from linux v6.1-rc5,
and renamed to llz4.c and llz4hc.c respectively to avoid name
collisions. Use of vmalloc has been changed to kvmalloc since it is
faster in most cases. They implement the Crypto API interface, and
rely on the lz4/lz4hc kernel library for compression implementation.
They have been modified to grok a compression acceleration/level, as
read from the top 4 bits of the crypto_tfm flags, and pass it to the
underlying library.
The lz4/lz4hc library sources are also copied from linux v6.1-rc5 and
built statically, so lz4_compress, lz4_decompress and lz4hc_compress
sources have been "de-modulified", and EXPORT_SYMBOLs removed. Headers
have also been copied from linux v6.1-rc5 for consistency, and source
files modified to include the copied headers instead of the system
headers.

All aforementioned sources are located in the lustre_compat/crypto/lz4
directory.

The lz4/lz4hc modules are built only if the kernel does not provide
them. This is verified by checking if the CONFIG_CRYPTO_LZ4 and
CONFIG_CRYPTO_LZ4HC kernel config options are defined.

This patch provides unit testing of several compression modules, such
as lz4, lz4hc, lzo and gzip. This is done via a new test kernel module
kcompr.ko, and new sanity test_84. File lustre/tests/kernel/kcompr.c
contains examples of how to call the compression/decompression
routines.

EX-7998 lustre: Fix build with crypto_tfm_ctx on kernel 6.2

In recent kernels, commit e634ac4
"crypto: api - Add crypto_tfm_ctx_dma"
moved crypto_tfm_ctx into algapi.h. So this file must be included
from lustre_crypto.h.
Was-Change-Id: If10f721355da3e58ca541c17615e978334e8d718

EX-7683 utils: always try to use our own lz4/lz4hc

lz4/lz4hc provided by the kernel do not grok a compression level.
The built-in lz4/lz4hc do, so always build them as dedicated kernel
modules llz4.ko and llz4hc.ko, with the same .cra_name but with a
slightly higher .cra_priority = 110, so that they are preferred over
the in-kernel modules if any.

And try to manually load the llz4/llz4hc kernel modules when a file
requires compression with the corresponding alg. This is a "one-shot"
try that allows us to prefer our modules that has level support, but
continues to at least compress/decompress files even if our own
modules are not available.
Was-Change-Id: I0bdf267f998e21df81e460250a653aed34e3215d

EX-bug-id: EX-6275
Test-Parameters: testlist=sanity env=ONLY=84
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I74ce95ff18194c6052d291588d7e8c79596a7f23

52776 master J !B ✓T ✓R -1/2−

LU-13802 llite: add file nonrotational check

carrying Patrick Farrell · needs rebase — checkpatch: cannot be cherry-picked

needs rebase hio stalled 133d 1 unresolved S +17 −2

133d

needs rebase — checkpatch: cannot be cherry-picked

1 unresolved thread(s) await your reply — last from Andreas Dilger

CI green — no vote yet from Shaun Tancheff, Andreas Dilger

janitor run: 13 failing config(s), none unique to this patch — janitor results

6 optional test failure(s) — informational

build #124324 SUCCESS · tests all Maloo sessions

no vote yet: Shaun Tancheff, Andreas Dilger

patchsets:

+16 earlier

PS1 10-20 PS2 10-20 PS3 10-20 PS4 10-24 PS5 10-24 PS6 01-16 PS7 01-16 PS8 02-10 PS9 02-12 PS10 02-12 PS11 02-12 PS12 02-13 PS13 02-13 PS14 03-13 PS15 03-13 PS16 03-21

PS17 03-21 PS18 03-27 PS19 04-01 PS20 04-23 Marc Vef PS21 04-23 Marc Vef

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

Andreas Dilger PS19 · 2024-04-11 23:20

We aggregate attributes from multiple objects for other reasons (e.g. maxbytes, timestamps, size, etc.).

The current implementation looks "mostly OK" and will work for basic linear IO patterns, where the file transitions from non-rotational to rotational.  However, any other kind of IO pattern (e.g. HDF5 where some writes are at the beginning of the file and others at the end) might have a serious issue?

IMHO, it would be better to aggregate this information once from the object (maybe from the OSC's the object is allocated on) and then be done with it?  Storing a threshold when the file transitions from non-rotational to rotational would be best, as this could be computed once based on the layout.  It is fairly unlikely that a file would have HDD stripes in the middle and flash at the beginning and end.

Alternately (probably better) is to store the nonrot state in each layout component (which is a perfect 1:1 mapping), and then this can be checked at IO submission time to see what type of storage it is covering.

Patrick Farrell PS19 · 2024-04-11 23:58

Hm, OK.  I don't think we can do this as easily as we'd like - the issue is we have to have this information very early.  I can consider this, though, your points are good ones.

The good thing is this isn't essential to merging the core feature, particularly since we're leaving it off by default for 2.16.

Andreas Dilger PS19 · 2026-04-26 02:56

Actually, an OST is (generally) only going to be rotational or non-rotational, and this is already returned to the client via statfs, so the clients should have full knowledge whether any IO is on flash or disk.

owner Patrick Farrell · uploader Marc Vef · PS21 uploaded 192d ago · NEW · open in Gerrit ↗

commit message

LU-13802 llite: add file nonrotational check

This patch adds the ability to note whether or not the
last IO to a file hit a rotational or a non-rotational
device.

This will be used in selecting the cutover thresholds for
hybrid IO, since DIO is synchronous and much slower on
spinning disk.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I75a9970f91b1776ed6f04ac0d000a9ba576df75a

50687 master J ✗1B ✓T ✗12R -2/2−

LU-12916 osd: use writeback for small writes in ldiskfs

carrying Qian Yingjin · needs rebase — checkpatch: cannot be cherry-picked

needs rebase 1 unique failure stalled 186d 1 unresolved L +639 −14

186d

needs rebase — checkpatch: cannot be cherry-picked

janitor: 1 test failure unique to this patch — sanity2@ldiskfs+DNE:test_127f

enforced failed: review-dne-part-1, review-dne-part-2, review-dne-part-3, review-dne-part-5, review-dne-selinux-ssk-part-1, review-dne-subtest-change, review-ldiskfs, review-ldiskfs-ubuntu

1 unresolved thread(s) await your reply — last from Qian Yingjin

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity2@ldiskfs+DNE:test_127f	seen in 1 other review

failed enforced test	platform	detail
review-dne-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-part-1	RHEL 9.5 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-part-2	RHEL 8.10 / x86_64	ran 11 tests. 1 tests failed: sanity-sec.	session
review-dne-part-3	RHEL 9.5 / x86_64	ran 3 tests. 1 tests failed: conf-sanity.	session
review-dne-part-5 crashed	RHEL 9.5 / x86_64	ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %%	session
review-dne-part-5	RHEL 8.10 / x86_64	ran 5 tests. 1 tests failed: sanityn.	session
review-dne-selinux-ssk-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanityn.	session
review-ldiskfs	RHEL 8.10 / x86_64, RHEL 9.5 / x86_64	ran 5 tests. 1 tests failed: sanity.	session
review-ldiskfs	RHEL 8.10 / x86_64, SLES 15.5 / x86_64	ran 5 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64	ran 6 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64	ran 6 tests. 1 tests failed: sanity.	session

12 optional test failure(s) — informational

build #124337 SUCCESS · tests all Maloo sessions

no vote yet: Sohei Koyama, Shaun Tancheff, Shuichi Ihara, Andreas Dilger, Alex Zhuravlev, Patrick Farrell

patchsets:

+68 earlier

PS69 10-02 Sohei Koyama PS70 10-29 Sohei Koyama PS71 10-29 Janitor Bot PS72 04-23 Marc Vef PS73 04-24 Marc Vef

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS72 2026-04-24 08:41 Qian Yingjin COMMIT_MSG:19 — Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext

Andreas Dilger PS72 · 2026-04-24 08:29

(typo) ".patch"

Writes for fscrypt files *should* always be full-block writes?  Why/how would writes to fscrypt files be partial blocks?  The client should be sending a full encrypted 4KiB block, but then set the size on the inode without truncating it.

Qian Yingjin PS72 · 2026-04-24 08:41

Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext4 for fscypt with write through (direct I/O) mode.

Although writes from clients are always full-block writes, but the size calls via lnb offset + len may be not full-block aligned, we use this size for the actual size of the file?

owner Qian Yingjin · uploader Marc Vef · PS73 uploaded 192d ago · NEW · open in Gerrit ↗

commit message

LU-12916 osd: use writeback for small writes in ldiskfs

This patch implements the writeback caching for small writes on
Lustre OSD (ldiskfs).
The writes are not synchronous any more on OST. It marks the page
as dirty and create buffer heads mapping pages to the physical
blocks. The kernel writeback mechanism will handle the actual I/O.

This patch should improve the performance for I/O cases such as
small writes from many files (i.e. mdtest-hard-write) or
unaligned I/O on a shared file (i.e. ior-hard-write).

Add a ldiskfs patch "ext4-writeback.patch" to not zero out
non-full page during writeback for a encrypted file. Without this
ldiskfs patch, it will fail sanity-sec/test_37.

According to the benchmark results, this patch can improve the
IO500 mdtest-hard performance and the performance of the real
application Nek5000 turbulent pipe with more than 10%.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I9b5aa568a49d5b39d04656fdacb94c3a6ec5a7ff

▶

Review feedback — your turn

(0)

nothing here right now

▶

In CI

(1)

64441 master J –B ✓T …R 0/2

LU-12187 lov: MDS layout pattern negotiation

carrying Maximilian Dilger · build OK — testing in progress (typically 6–14h)

pt_ecro XL +1273 −53

build #135207 SUCCESS · tests all Maloo sessions · test queue

no vote yet: Andreas Dilger, Zhenyu Xu, Oleg Drokin

patchsets:

+29 earlier

PS1 03-09 PS2 03-12 PS3 03-12 PS4 03-12 PS5 03-15 PS6 03-26 PS7 04-10 PS8 04-23 PS9 04-23 Andreas Dilger PS10 04-24 Andreas Dilger PS11 05-04 Andreas Dilger PS12 06-03 PS13 06-05 PS14 06-08 PS15 06-24 PS16 06-25 PS17 07-09 PS18 07-10 PS19 08-13 PS20 08-21 PS21 08-25 PS22 08-25 PS23 09-02 PS24 09-03 PS25 09-08 PS26 09-09 PS27 09-11 PS28 09-16 PS29 10-13

PS30 10-13 PS31 10-21 PS32 10-24 PS33 11-02 Marc Vef PS34 11-02 Marc Vef

your previous vote: -1 on PS17 (2026-07-10 11:39) — now at PS34

tags: pt_ecro

owner Maximilian Dilger · uploader Marc Vef · PS34 uploaded 5h ago · NEW · open in Gerrit ↗

commit message

LU-12187 lov: MDS layout pattern negotiation

Add obd_connect_data::ocd_layout_pattern_types so a client and the MDS
negotiate the layout patterns and LOV_PATTERN_F_* flags both support.
The client sends what it knows, the MDS replies with the intersection
when it grants OBD_CONNECT2_FLR_EC, and the client falls back to the
pre-2.18 baseline LOV_LAYOUT_PATTERN_TYPES_217 otherwise.  FLR-EC uses
this for LOV_PATTERN_PARITY.  The mask comes from MDT0000 through the
LMV and is refreshed on every MDC reconnect; the client's own request
is restored for each reconnect like the connect flags, so a pattern
the MDS re-enables comes back without a remount.  OSTs are not
involved.

A component whose pattern or flag was not negotiated is left
uninitialized by lov_init_composite() like an unknown pattern, so IO
reads another mirror instead; writes still stale it so that a later
resync rebuilds it.  The client refuses to create such a component
with EOPNOTSUPP on both the setstripe ioctl and lustre.lov xattr
paths.  Parsing stays pattern-agnostic so lsme alloc/free remain
symmetric.

Add mdt.*.layout_pattern_types to change the advertised types at
runtime (mdt_enable_flr_ec only seeds it at MDT start), read-only
lov.*.layout_pattern_types on the client, and man pages for both.
The value is filesystem-wide, so set it on all MDTs.

Rename llite_enable_flr_ec to enable_flr_ec, reached as
lustre.enable_flr_ec via 'lctl {get,set}_param --module'.  Since
mdt_enable_flr_ec only seeds an MDT at startup, set_mdt_flr_ec() in
the test framework also enables parity on the MDT it has just
mounted.  MODOPTS_MDT never reaches the MDS when mount loads the
server modules (LOAD_MODULES_REMOTE=false, as in autotest), so
sanity-ec and sanity test_27D negotiate parity explicitly, and
test_27D skips the EC subtests that create parity layouts when the
client or MDS cannot.

Add conf-sanity 163a-163d: the negotiated mask in the import, the
tunable and reconnect renegotiation, write staling from a client
without EC, and an un-negotiated layout flag injected with
OBD_FAIL_LOV_COMP_FLAG.

Assisted-by: ClaudeCode:Fable-5.1 llm_code_and_review_tools
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=conf-sanity env=ONLY="163"
Test-Parameters: testlist=sanity serverversion=2.17
Test-Parameters: testlist=sanity-pfl clientversion=2.17
Test-Parameters: testlist=sanity-pfl serverversion=2.17 env=SANITY_PFL_EXCEPT="18 28"
Test-Parameters: testlist=sanity-flr clientversion=2.17
Test-Parameters: testlist=sanity-flr serverversion=2.17 env=SANITY_FLR_EXCEPT="0a"
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Change-Id: Iaef716e28014be5b91dd50b117dd0881f5c37f37

▶

Needs reviewers

(1)

68030 master J !B ✓T ✓R 0/2

LU-20597 utils: handle sparse target indices in quota

carrying Sergey N · CI green — no vote yet from Sergey Cheremencev, Andreas Dilger, Feng Lei

L +329 −57

10d

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #134120 SUCCESS · tests all Maloo sessions

no vote yet: Sergey Cheremencev, Andreas Dilger, Feng Lei

patchsets:

+7 earlier

PS1 08-16 PS2 08-16 PS3 08-20 PS4 08-20 PS5 08-21 PS6 08-23 PS7 08-24

PS8 08-25 PS9 08-26 PS10 08-27 PS11 08-28 PS12 09-03 Marc Vef

owner Sergey N · uploader Marc Vef · PS12 uploaded 60d ago · NEW · open in Gerrit ↗

commit message

LU-20597 utils: handle sparse target indices in quota

llapi_get_obd_count() returns different values for MDTs and OSTs.
For MDTs, it returns the target array size, which covers sparse
indices but also includes holes. For OSTs, it returns the number of
configured targets.

Therefore, a filesystem with eight OSTs at indices 256 through 263 is
queried at indices 0 through 7. print_obd_quota() omits the existing
targets, and lfs_reset_quota() does not verify that their quota grants
were reset.

Obtain the actual MDT and OST indices with llapi_get_target_uuids()
and use them for quota queries. Keep a UUID array to enable capacity
checking when target_obd can be read from the parameter file. Allocate
an extra zeroed index so the netlink parser's post-loop range check
stays within bounds.

Build both target lists before issuing LUSTRE_Q_RESETQID so an
allocation or enumeration failure aborts before quota limits are
cleared.

Reset wait_index when changing from MDTs to OSTs so the OST phase
starts at its first target. Reset wait_count so each phase has its own
30-second retry period. Return -ETIMEDOUT when the retry budget is
exhausted and report the target type and index in the diagnostic.

Clear rc when -ENODEV or -ENODATA is skipped. Otherwise, if the last
target in the final phase is inactive, lfs_reset_quota() returns that
error even though the reset completed.

Extend sanity-quota test 16b to verify quota output with sparse MDT
and OST indices. Add test 16d to verify quota grant reset on sparse
targets. Use observable quota grant state instead of a fixed delay.

Gate both tests with skip_env before rebuilding the filesystem when
the required MDT or OST count is unavailable. Since fail_loc is
node-global, also skip test 16d when OST0064 shares its host with an
earlier OST or any MDT.

Fixes: a2fd4d3aee97 ("LU-15880 quota: fix insane grant quota")
Test-Parameters: testlist=sanity-quota mdtcount=3 mdscount=3 ostcount=2 osscount=2 env=ONLY=16b+16d
Test-Parameters: testlist=sanity-quota mdtcount=3 mdscount=1 ostcount=1 osscount=2 env=ONLY=16d,ONLY_REPEAT=10
Signed-off-by: Sergey Noskov <sergey.noskov.sn@gmail.com>
Change-Id: I8a921053b4a8eb71118fd523edc1fe5a5e73879d

▶

Waiting on reviewers

(0)

nothing here right now

▶

Ready to land

(1)

62408 master J !B ✓T ✓R +2/2

LU-10026 lustre: add lgzip kernel module

carrying Artem Blagodarenko · all green — ready to land (not in master-next yet)

csdc XL +5580 −10

52d

all green — ready to land (not in master-next yet)

Timothy Day voted -1 — the review total is met; address it if a respin is needed

janitor run: 5 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-part-2	RHEL 8.10 / x86_64	ran 11 tests. 1 tests failed: sanity-sec.	session

build #125322 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-05-19 13:49 · Sebastien Buisson +1 2026-09-11 13:44 · Andreas Dilger +1 2026-06-05 22:45 · Timothy Day -1 2026-06-05 20:51

patchsets:

+1 earlier

PS1 11-06

PS2 05-07 Marc Vef PS3 05-07 Marc Vef PS4 05-07 Marc Vef PS5 05-13 Marc Vef PS6 05-19 Marc Vef

your previous vote: +1 on PS5 (2026-05-18 08:01) — now at PS6

tags: csdc

owner Artem Blagodarenko · uploader Marc Vef · PS6 uploaded 167d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lustre: add lgzip kernel module

lgzip kernel module implements compression according to the
deflate/zlib algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'deflate':
* deflate-lustre-generic of type compression
* deflate-lustre-scomp of type scomp

Note the 'deflate' name is identical to the in-kernel module, but
lgzip registers it with a slightly higher .cra_priority = 110, so that
it is preferred over the in-kernel module.
Our 'deflate' is also different in that it accepts a compression level
as explained below.

lgzip kernel module sources are copied from linux v6.2-rc5 and renamed
to gzip.c to avoid name collisions. It implements the Crypto API
interface, and rely on the deflate/zlib kernel library for compression
implementation. It has been modified to grok a compression level, as
read from the top 4 bits of the crypto_tfm flags, and pass it to the
underlying library.
The deflate/zlib library sources are also copied from linux v6.2-rc5
and built statically. Headers have also been copied from linux
v6.2-rc5 for consistency, and source files modified to include the
copied headers instead of the system headers.
All aforementioned sources are located in the
lustre_compat/crypto/gzip directory.
The lgzip module is always built with Lustre.

This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'deflate' module.
It also tries to manually load the lgzip kernel module when a file
requires compression with the 'delfate' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.

EX-bug-id: EX-6206
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I10b0ec03cf7d0e4fe3dd2898e77f357667209755

▶

Parked (WIP / self-blocked)

(1)

62868 master J –B ✓T ✗1R -2/2−

LU-12187 llite: enable FLR EC support unconditionally

carrying Patrick Farrell · parked by your own -1

needs rebase ec2 pt_ecro218_review_priority 4 unresolved M +9 −67

failed enforced test	platform	detail
review-ldiskfs-dne	RHEL 9.8 / x86_64	ran 5 tests. 1 tests failed: sanity.	session

build #134456 SUCCESS · tests all Maloo sessions

reviews: Marc Vef -1 2026-10-26 15:54

patchsets: PS1 12-03 PS2 09-23 Marc Vef PS3 09-23 Marc Vef PS4 10-26 Marc Vef

your previous vote: -1 on PS3 (2026-09-23 17:11) — now at PS4

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS4 2026-10-26 22:47 Gerrit AI review for Lustre COMMIT_MSG:7 — (style) This isn't a bug, but the change spans llite, mdt, utils (lfs) and tests, and turns on a whole feature rather than changing llite be

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(style) This isn't a bug, but the change spans llite, mdt, utils (lfs) and tests, and turns on a whole feature rather than changing llite behaviour. Would a feature component such as "ec:" (already used elsewhere in this topic) describe it better than "llite:"?

reply PS4 2026-10-26 22:47 Gerrit AI review for Lustre COMMIT_MSG:10 — (style) If the patch is refreshed: "This patch is intended to be applied after the guarded development phase is complete" is a landing note

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(style) If the patch is refreshed: "This patch is intended to be applied after the guarded development phase is complete" is a landing note that will mean nothing in git log. The user-visible effect is more useful there: `lfs setstripe --ec` works without LFS_EC_OK, `llite.*.enable_erasure_coding` now defaults to 1, and the `mdt_enable_flr_ec`/`llite_enable_flr_ec` module options are gone (any modprobe.d entries that set them become "unknown parameter"). The "Changes:" list mostly repeats the diff and could be folded into one sentence.

reply PS4 2026-10-27 07:34 Andreas Dilger llite_lib.c:228 — should be fixed when patch is refreshed.

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(minor) This flips the default of `llite.*.enable_erasure_coding`, which is now the only remaining off switch for EC creation, but there is no man4 page for that tunable (Documentation/man4 has llite.enable_setstripe_gid.4 etc., nothing for enable_erasure_coding). If the patch is refreshed, could a llite.enable_erasure_coding.4 be added describing the new default of 1 and that it only blocks creating parity layouts (setstripe/setxattr), not reading existing EC files?

Andreas Dilger PS4 · 2026-10-27 07:34

should be fixed when patch is refreshed.

reply PS4 2026-10-26 22:47 Gerrit AI review for Lustre llite_lib.c:496 — (suggestion) Following up on the patch set 1 request for a client-side setting to disable EC data rebuild if it misbehaves in production: ll

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(suggestion) Following up on the patch set 1 request for a client-side setting to disable EC data rebuild if it misbehaves in production: llite_enable_flr_ec never controlled that. It only decided whether OBD_CONNECT2_FLR_EC was requested, and the only client consumer of the flag is the LL_LEASE_ALLOW_STALE merge check in ll_file_unlock_lease(). Degraded reads switch to parity recovery with no check at all:

    lov_io_mirror_init()
      if (io->ci_type == CIT_READ && io->ci_cross_ec)
          io->ci_switch_ec_io = 1;   /* -> CIT_EC_RD */

enable_erasure_coding only gates ll_lov_setstripe_ea_info() and ll_adjust_lum(). So both before and after this change, there is no way to fall back to waiting for the OST instead of rebuilding from parity. Is that switch still planned before EC is enabled by default, or is it no longer wanted?

tags: pt_ecro 218_review_priority

owner Patrick Farrell · uploader Marc Vef · PS4 uploaded 7d ago · NEW · open in Gerrit ↗

commit message

LU-12187 llite: enable FLR EC support unconditionally

Remove module parameter guards and enable FLR EC support
unconditionally. This patch is intended to be applied
after the guarded development phase is complete.

Changes:
- Remove mdt_enable_flr_ec module parameter and add
  OBD_CONNECT2_FLR_EC to MDT_CONNECT_SUPPORTED2
- Remove llite_enable_flr_ec module parameter
- Add OBD_CONNECT2_FLR_EC unconditionally to client
  connect flags
- Change ll_enable_erasure_coding default from 0 to 1
- Remove the LFS_EC_OK check from lfs setstripe
- Remove the FLR EC module options and set_mdt_flr_ec()
  from test framework

Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I431d309f85c8964e13a11eaaa729d944e9228c29

▶

218_review_priority

(1)

62868 master J –B ✓T ✗1R -2/2−

LU-12187 llite: enable FLR EC support unconditionally

carrying Patrick Farrell · parked by your own -1

needs rebase ec2 pt_ecro218_review_priority 4 unresolved M +9 −67

failed enforced test	platform	detail
review-ldiskfs-dne	RHEL 9.8 / x86_64	ran 5 tests. 1 tests failed: sanity.	session

build #134456 SUCCESS · tests all Maloo sessions

reviews: Marc Vef -1 2026-10-26 15:54

patchsets: PS1 12-03 PS2 09-23 Marc Vef PS3 09-23 Marc Vef PS4 10-26 Marc Vef

your previous vote: -1 on PS3 (2026-09-23 17:11) — now at PS4

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(style) This isn't a bug, but the change spans llite, mdt, utils (lfs) and tests, and turns on a whole feature rather than changing llite behaviour. Would a feature component such as "ec:" (already used elsewhere in this topic) describe it better than "llite:"?

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(style) If the patch is refreshed: "This patch is intended to be applied after the guarded development phase is complete" is a landing note that will mean nothing in git log. The user-visible effect is more useful there: `lfs setstripe --ec` works without LFS_EC_OK, `llite.*.enable_erasure_coding` now defaults to 1, and the `mdt_enable_flr_ec`/`llite_enable_flr_ec` module options are gone (any modprobe.d entries that set them become "unknown parameter"). The "Changes:" list mostly repeats the diff and could be folded into one sentence.

reply PS4 2026-10-27 07:34 Andreas Dilger llite_lib.c:228 — should be fixed when patch is refreshed.

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(minor) This flips the default of `llite.*.enable_erasure_coding`, which is now the only remaining off switch for EC creation, but there is no man4 page for that tunable (Documentation/man4 has llite.enable_setstripe_gid.4 etc., nothing for enable_erasure_coding). If the patch is refreshed, could a llite.enable_erasure_coding.4 be added describing the new default of 1 and that it only blocks creating parity layouts (setstripe/setxattr), not reading existing EC files?

Andreas Dilger PS4 · 2026-10-27 07:34

should be fixed when patch is refreshed.

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(suggestion) Following up on the patch set 1 request for a client-side setting to disable EC data rebuild if it misbehaves in production: llite_enable_flr_ec never controlled that. It only decided whether OBD_CONNECT2_FLR_EC was requested, and the only client consumer of the flag is the LL_LEASE_ALLOW_STALE merge check in ll_file_unlock_lease(). Degraded reads switch to parity recovery with no check at all:

    lov_io_mirror_init()
      if (io->ci_type == CIT_READ && io->ci_cross_ec)
          io->ci_switch_ec_io = 1;   /* -> CIT_EC_RD */

enable_erasure_coding only gates ll_lov_setstripe_ea_info() and ll_adjust_lum(). So both before and after this change, there is no way to fall back to waiting for the OST instead of rebuilding from parity. Is that switch still planned before EC is enabled by default, or is it no longer wanted?

tags: pt_ecro 218_review_priority

owner Patrick Farrell · uploader Marc Vef · PS4 uploaded 7d ago · NEW · open in Gerrit ↗

commit message

LU-12187 llite: enable FLR EC support unconditionally

Remove module parameter guards and enable FLR EC support
unconditionally. This patch is intended to be applied
after the guarded development phase is complete.

Changes:
- Remove mdt_enable_flr_ec module parameter and add
  OBD_CONNECT2_FLR_EC to MDT_CONNECT_SUPPORTED2
- Remove llite_enable_flr_ec module parameter
- Add OBD_CONNECT2_FLR_EC unconditionally to client
  connect flags
- Change ll_enable_erasure_coding default from 0 to 1
- Remove the LFS_EC_OK check from lfs setstripe
- Remove the FLR EC module options and set_mdt_flr_ec()
  from test framework

Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I431d309f85c8964e13a11eaaa729d944e9228c29

▶

csdc

(2)

57776 master J ✗2B ✓T ✓R +1/2−

LU-10026 lustre: add lz4 and lz4hc kernel modules

carrying Artem Blagodarenko · needs rebase — checkpatch: cannot be cherry-picked

needs rebase csdc 5 unresolved XL +4422 −2

46d

needs rebase — checkpatch: cannot be cherry-picked

Timothy Day voted -1

3 unresolved thread(s) await your reply — last from Timothy Day

2 thread(s) waiting on others

no vote yet from Sebastien Buisson, Oleg Drokin

janitor flagged test failures unique to this patch — superseded (tests have since passed) — full janitor results

unique failing test	history
sanity-lfsck@zfs:test_18c	seen in 25 other reviews
sanity-scrub@ldiskfs+DNE:test_9	seen in 5 other reviews

build #125233 SUCCESS · tests all Maloo sessions

reviews: James Simmons +1 2026-05-29 01:12 · Marc Vef +1 2026-05-19 09:37 · Andreas Dilger +1 2026-05-19 20:23 · Timothy Day -1 2026-06-05 20:49

no vote yet: Sebastien Buisson, Oleg Drokin

patchsets:

+27 earlier

PS28 05-11 Marc Vef PS29 05-11 Marc Vef PS30 05-11 Marc Vef PS31 05-13 Marc Vef PS32 05-18 Marc Vef

your previous vote: -1 on PS27 (2026-05-07 21:28) — now at PS32

threads: 3 need your reply · 2 waiting on others · 0 own notes · 0 bot

reply PS32 2026-05-29 17:08 Timothy Day COMMIT_MSG:36 — This is false.

Timothy Day PS32 · 2026-05-29 17:08

This is false.

reply PS32 2026-05-29 17:08 Timothy Day COMMIT_MSG:60 — These modules do not compile on newer kernels (6.18, at least). I think it should be possible to build without these.

Timothy Day PS32 · 2026-05-29 17:08

These modules do not compile on newer kernels (6.18, at least). I think it should be possible to build without these.

reply PS32 2026-05-29 21:34 Oleg Drokin kcompr.c:86 — just as the smatch implies this should really be IS_ERR, I don't think filp_open can return NULL?

Oleg Drokin PS32 · 2026-05-29 21:34

just as the smatch implies this should really be IS_ERR, I don't think filp_open can return NULL?

tags: csdc

owner Artem Blagodarenko · uploader Marc Vef · PS32 uploaded 168d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lustre: add lz4 and lz4hc kernel modules

lz4 and lz4hc kernel modules implement compression according to the
lz4 and lz4hc algorithms respectively, through the kernel Crypto API.
lz4 module provides 2 cipher drivers under the generic name 'lz4':
* lz4-lustre-generic of type compression
* lz4-lustre-scomp of type scomp
lz4hc module provides 2 cipher drivers under the generic name 'lz4hc':
* lz4hc-lustre-generic of type compression
* lz4hc-lustre-scomp of type scomp

lz4 and lz4hc kernel module sources are copied from linux v6.1-rc5,
and renamed to llz4.c and llz4hc.c respectively to avoid name
collisions. Use of vmalloc has been changed to kvmalloc since it is
faster in most cases. They implement the Crypto API interface, and
rely on the lz4/lz4hc kernel library for compression implementation.
They have been modified to grok a compression acceleration/level, as
read from the top 4 bits of the crypto_tfm flags, and pass it to the
underlying library.
The lz4/lz4hc library sources are also copied from linux v6.1-rc5 and
built statically, so lz4_compress, lz4_decompress and lz4hc_compress
sources have been "de-modulified", and EXPORT_SYMBOLs removed. Headers
have also been copied from linux v6.1-rc5 for consistency, and source
files modified to include the copied headers instead of the system
headers.

All aforementioned sources are located in the lustre_compat/crypto/lz4
directory.

The lz4/lz4hc modules are built only if the kernel does not provide
them. This is verified by checking if the CONFIG_CRYPTO_LZ4 and
CONFIG_CRYPTO_LZ4HC kernel config options are defined.

This patch provides unit testing of several compression modules, such
as lz4, lz4hc, lzo and gzip. This is done via a new test kernel module
kcompr.ko, and new sanity test_84. File lustre/tests/kernel/kcompr.c
contains examples of how to call the compression/decompression
routines.

EX-7998 lustre: Fix build with crypto_tfm_ctx on kernel 6.2

In recent kernels, commit e634ac4
"crypto: api - Add crypto_tfm_ctx_dma"
moved crypto_tfm_ctx into algapi.h. So this file must be included
from lustre_crypto.h.
Was-Change-Id: If10f721355da3e58ca541c17615e978334e8d718

EX-7683 utils: always try to use our own lz4/lz4hc

lz4/lz4hc provided by the kernel do not grok a compression level.
The built-in lz4/lz4hc do, so always build them as dedicated kernel
modules llz4.ko and llz4hc.ko, with the same .cra_name but with a
slightly higher .cra_priority = 110, so that they are preferred over
the in-kernel modules if any.

And try to manually load the llz4/llz4hc kernel modules when a file
requires compression with the corresponding alg. This is a "one-shot"
try that allows us to prefer our modules that has level support, but
continues to at least compress/decompress files even if our own
modules are not available.
Was-Change-Id: I0bdf267f998e21df81e460250a653aed34e3215d

EX-bug-id: EX-6275
Test-Parameters: testlist=sanity env=ONLY=84
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I74ce95ff18194c6052d291588d7e8c79596a7f23

62408 master J !B ✓T ✓R +2/2

LU-10026 lustre: add lgzip kernel module

carrying Artem Blagodarenko · all green — ready to land (not in master-next yet)

csdc XL +5580 −10

52d

all green — ready to land (not in master-next yet)

Timothy Day voted -1 — the review total is met; address it if a respin is needed

janitor run: 5 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-part-2	RHEL 8.10 / x86_64	ran 11 tests. 1 tests failed: sanity-sec.	session

build #125322 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-05-19 13:49 · Sebastien Buisson +1 2026-09-11 13:44 · Andreas Dilger +1 2026-06-05 22:45 · Timothy Day -1 2026-06-05 20:51

patchsets:

+1 earlier

PS1 11-06

PS2 05-07 Marc Vef PS3 05-07 Marc Vef PS4 05-07 Marc Vef PS5 05-13 Marc Vef PS6 05-19 Marc Vef

your previous vote: +1 on PS5 (2026-05-18 08:01) — now at PS6

tags: csdc

owner Artem Blagodarenko · uploader Marc Vef · PS6 uploaded 167d ago · NEW · open in Gerrit ↗

commit message

LU-10026 lustre: add lgzip kernel module

lgzip kernel module implements compression according to the
deflate/zlib algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'deflate':
* deflate-lustre-generic of type compression
* deflate-lustre-scomp of type scomp

Note the 'deflate' name is identical to the in-kernel module, but
lgzip registers it with a slightly higher .cra_priority = 110, so that
it is preferred over the in-kernel module.
Our 'deflate' is also different in that it accepts a compression level
as explained below.

lgzip kernel module sources are copied from linux v6.2-rc5 and renamed
to gzip.c to avoid name collisions. It implements the Crypto API
interface, and rely on the deflate/zlib kernel library for compression
implementation. It has been modified to grok a compression level, as
read from the top 4 bits of the crypto_tfm flags, and pass it to the
underlying library.
The deflate/zlib library sources are also copied from linux v6.2-rc5
and built statically. Headers have also been copied from linux
v6.2-rc5 for consistency, and source files modified to include the
copied headers instead of the system headers.
All aforementioned sources are located in the
lustre_compat/crypto/gzip directory.
The lgzip module is always built with Lustre.

This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'deflate' module.
It also tries to manually load the lgzip kernel module when a file
requires compression with the 'delfate' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.

EX-bug-id: EX-6206
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I10b0ec03cf7d0e4fe3dd2898e77f357667209755

▶

pt_ecro

(3)

64441 master J –B ✓T …R 0/2

LU-12187 lov: MDS layout pattern negotiation

carrying Maximilian Dilger · build OK — testing in progress (typically 6–14h)

pt_ecro XL +1273 −53

build #135207 SUCCESS · tests all Maloo sessions · test queue

no vote yet: Andreas Dilger, Zhenyu Xu, Oleg Drokin

patchsets:

+29 earlier

PS30 10-13 PS31 10-21 PS32 10-24 PS33 11-02 Marc Vef PS34 11-02 Marc Vef

your previous vote: -1 on PS17 (2026-07-10 11:39) — now at PS34

tags: pt_ecro

owner Maximilian Dilger · uploader Marc Vef · PS34 uploaded 5h ago · NEW · open in Gerrit ↗

commit message

LU-12187 lov: MDS layout pattern negotiation

Add obd_connect_data::ocd_layout_pattern_types so a client and the MDS
negotiate the layout patterns and LOV_PATTERN_F_* flags both support.
The client sends what it knows, the MDS replies with the intersection
when it grants OBD_CONNECT2_FLR_EC, and the client falls back to the
pre-2.18 baseline LOV_LAYOUT_PATTERN_TYPES_217 otherwise.  FLR-EC uses
this for LOV_PATTERN_PARITY.  The mask comes from MDT0000 through the
LMV and is refreshed on every MDC reconnect; the client's own request
is restored for each reconnect like the connect flags, so a pattern
the MDS re-enables comes back without a remount.  OSTs are not
involved.

A component whose pattern or flag was not negotiated is left
uninitialized by lov_init_composite() like an unknown pattern, so IO
reads another mirror instead; writes still stale it so that a later
resync rebuilds it.  The client refuses to create such a component
with EOPNOTSUPP on both the setstripe ioctl and lustre.lov xattr
paths.  Parsing stays pattern-agnostic so lsme alloc/free remain
symmetric.

Add mdt.*.layout_pattern_types to change the advertised types at
runtime (mdt_enable_flr_ec only seeds it at MDT start), read-only
lov.*.layout_pattern_types on the client, and man pages for both.
The value is filesystem-wide, so set it on all MDTs.

Rename llite_enable_flr_ec to enable_flr_ec, reached as
lustre.enable_flr_ec via 'lctl {get,set}_param --module'.  Since
mdt_enable_flr_ec only seeds an MDT at startup, set_mdt_flr_ec() in
the test framework also enables parity on the MDT it has just
mounted.  MODOPTS_MDT never reaches the MDS when mount loads the
server modules (LOAD_MODULES_REMOTE=false, as in autotest), so
sanity-ec and sanity test_27D negotiate parity explicitly, and
test_27D skips the EC subtests that create parity layouts when the
client or MDS cannot.

Add conf-sanity 163a-163d: the negotiated mask in the import, the
tunable and reconnect renegotiation, write staling from a client
without EC, and an un-negotiated layout flag injected with
OBD_FAIL_LOV_COMP_FLAG.

Assisted-by: ClaudeCode:Fable-5.1 llm_code_and_review_tools
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=conf-sanity env=ONLY="163"
Test-Parameters: testlist=sanity serverversion=2.17
Test-Parameters: testlist=sanity-pfl clientversion=2.17
Test-Parameters: testlist=sanity-pfl serverversion=2.17 env=SANITY_PFL_EXCEPT="18 28"
Test-Parameters: testlist=sanity-flr clientversion=2.17
Test-Parameters: testlist=sanity-flr serverversion=2.17 env=SANITY_FLR_EXCEPT="0a"
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Change-Id: Iaef716e28014be5b91dd50b117dd0881f5c37f37

62868 master J –B ✓T ✗1R -2/2−

LU-12187 llite: enable FLR EC support unconditionally

carrying Patrick Farrell · parked by your own -1

needs rebase ec2 pt_ecro218_review_priority 4 unresolved M +9 −67

failed enforced test	platform	detail
review-ldiskfs-dne	RHEL 9.8 / x86_64	ran 5 tests. 1 tests failed: sanity.	session

build #134456 SUCCESS · tests all Maloo sessions

reviews: Marc Vef -1 2026-10-26 15:54

patchsets: PS1 12-03 PS2 09-23 Marc Vef PS3 09-23 Marc Vef PS4 10-26 Marc Vef

your previous vote: -1 on PS3 (2026-09-23 17:11) — now at PS4

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(style) This isn't a bug, but the change spans llite, mdt, utils (lfs) and tests, and turns on a whole feature rather than changing llite behaviour. Would a feature component such as "ec:" (already used elsewhere in this topic) describe it better than "llite:"?

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(style) If the patch is refreshed: "This patch is intended to be applied after the guarded development phase is complete" is a landing note that will mean nothing in git log. The user-visible effect is more useful there: `lfs setstripe --ec` works without LFS_EC_OK, `llite.*.enable_erasure_coding` now defaults to 1, and the `mdt_enable_flr_ec`/`llite_enable_flr_ec` module options are gone (any modprobe.d entries that set them become "unknown parameter"). The "Changes:" list mostly repeats the diff and could be folded into one sentence.

reply PS4 2026-10-27 07:34 Andreas Dilger llite_lib.c:228 — should be fixed when patch is refreshed.

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(minor) This flips the default of `llite.*.enable_erasure_coding`, which is now the only remaining off switch for EC creation, but there is no man4 page for that tunable (Documentation/man4 has llite.enable_setstripe_gid.4 etc., nothing for enable_erasure_coding). If the patch is refreshed, could a llite.enable_erasure_coding.4 be added describing the new default of 1 and that it only blocks creating parity layouts (setstripe/setxattr), not reading existing EC files?

Andreas Dilger PS4 · 2026-10-27 07:34

should be fixed when patch is refreshed.

Gerrit AI review for Lustre PS4 · 2026-10-26 22:47

(suggestion) Following up on the patch set 1 request for a client-side setting to disable EC data rebuild if it misbehaves in production: llite_enable_flr_ec never controlled that. It only decided whether OBD_CONNECT2_FLR_EC was requested, and the only client consumer of the flag is the LL_LEASE_ALLOW_STALE merge check in ll_file_unlock_lease(). Degraded reads switch to parity recovery with no check at all:

    lov_io_mirror_init()
      if (io->ci_type == CIT_READ && io->ci_cross_ec)
          io->ci_switch_ec_io = 1;   /* -> CIT_EC_RD */

enable_erasure_coding only gates ll_lov_setstripe_ea_info() and ll_adjust_lum(). So both before and after this change, there is no way to fall back to waiting for the OST instead of rebuilding from parity. Is that switch still planned before EC is enabled by default, or is it no longer wanted?

tags: pt_ecro 218_review_priority

owner Patrick Farrell · uploader Marc Vef · PS4 uploaded 7d ago · NEW · open in Gerrit ↗

commit message

LU-12187 llite: enable FLR EC support unconditionally

Remove module parameter guards and enable FLR EC support
unconditionally. This patch is intended to be applied
after the guarded development phase is complete.

Changes:
- Remove mdt_enable_flr_ec module parameter and add
  OBD_CONNECT2_FLR_EC to MDT_CONNECT_SUPPORTED2
- Remove llite_enable_flr_ec module parameter
- Add OBD_CONNECT2_FLR_EC unconditionally to client
  connect flags
- Change ll_enable_erasure_coding default from 0 to 1
- Remove the LFS_EC_OK check from lfs setstripe
- Remove the FLR EC module options and set_mdt_flr_ec()
  from test framework

Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I431d309f85c8964e13a11eaaa729d944e9228c29

62859 master J ✓B ✓T ✗12R -1/2−

LU-12668 ec: Add tests for computing the parity coverage

carrying Ronnie Sahlberg · needs rebase — checkpatch: cannot be cherry-picked

needs rebase ec2 pt_ecro L +460 −0

44d

needs rebase — checkpatch: cannot be cherry-picked

enforced failed: custom-1001, custom-1002, review-dne-subtest-change, review-dne-zfs-subtest-change, review-ldiskfs-dne-arm

failed enforced test	platform	detail
custom-1001	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
custom-1002	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-ldiskfs-dne-arm	RHEL 8.10 / x86_64, Rocky 9.5 / aarch64	ran 5 tests. 1 tests failed: sanity.	session

build #127014 SUCCESS · tests all Maloo sessions

no vote yet: Keguang Xu, Qian Yingjin, Maximilian Dilger

patchsets:

+42 earlier

PS43 05-06 PS44 05-14 Marc Vef PS45 05-20 Marc Vef PS46 06-22 Marc Vef PS47 06-26 Marc Vef

your previous vote: -1 on PS45 (2026-05-21 08:46) — now at PS47

tags: pt_ecro

owner Ronnie Sahlberg · uploader Marc Vef · PS47 uploaded 128d ago · NEW · open in Gerrit ↗

commit message

LU-12668 ec: Add tests for computing the parity coverage

When we write or verify the parities we no longer
use the whole stripe, instead we compute the coverage
of what ranges of parity is important and what can be
ignored.
This is based on SEEK_DATA/SEEK_HOLE and EOF.

Add tests that we compute these ranges correctly.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: Iaee9d0ccb0875f6d515a8adc322fa3cd37bfe0f6

▶

(untagged)

(3)

68030 master J !B ✓T ✓R 0/2

LU-20597 utils: handle sparse target indices in quota

carrying Sergey N · CI green — no vote yet from Sergey Cheremencev, Andreas Dilger, Feng Lei

L +329 −57

10d

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #134120 SUCCESS · tests all Maloo sessions

no vote yet: Sergey Cheremencev, Andreas Dilger, Feng Lei

patchsets:

+7 earlier

PS1 08-16 PS2 08-16 PS3 08-20 PS4 08-20 PS5 08-21 PS6 08-23 PS7 08-24

PS8 08-25 PS9 08-26 PS10 08-27 PS11 08-28 PS12 09-03 Marc Vef

owner Sergey N · uploader Marc Vef · PS12 uploaded 60d ago · NEW · open in Gerrit ↗

commit message

LU-20597 utils: handle sparse target indices in quota

Build both target lists before issuing LUSTRE_Q_RESETQID so an
allocation or enumeration failure aborts before quota limits are
cleared.

Clear rc when -ENODEV or -ENODATA is skipped. Otherwise, if the last
target in the final phase is inactive, lfs_reset_quota() returns that
error even though the reset completed.

52776 master J !B ✓T ✓R -1/2−

LU-13802 llite: add file nonrotational check

carrying Patrick Farrell · needs rebase — checkpatch: cannot be cherry-picked

needs rebase hio stalled 133d 1 unresolved S +17 −2

133d

needs rebase — checkpatch: cannot be cherry-picked

1 unresolved thread(s) await your reply — last from Andreas Dilger

CI green — no vote yet from Shaun Tancheff, Andreas Dilger

janitor run: 13 failing config(s), none unique to this patch — janitor results

6 optional test failure(s) — informational

build #124324 SUCCESS · tests all Maloo sessions

no vote yet: Shaun Tancheff, Andreas Dilger

patchsets:

+16 earlier

PS1 10-20 PS2 10-20 PS3 10-20 PS4 10-24 PS5 10-24 PS6 01-16 PS7 01-16 PS8 02-10 PS9 02-12 PS10 02-12 PS11 02-12 PS12 02-13 PS13 02-13 PS14 03-13 PS15 03-13 PS16 03-21

PS17 03-21 PS18 03-27 PS19 04-01 PS20 04-23 Marc Vef PS21 04-23 Marc Vef

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

Andreas Dilger PS19 · 2024-04-11 23:20

We aggregate attributes from multiple objects for other reasons (e.g. maxbytes, timestamps, size, etc.).

The current implementation looks "mostly OK" and will work for basic linear IO patterns, where the file transitions from non-rotational to rotational.  However, any other kind of IO pattern (e.g. HDF5 where some writes are at the beginning of the file and others at the end) might have a serious issue?

IMHO, it would be better to aggregate this information once from the object (maybe from the OSC's the object is allocated on) and then be done with it?  Storing a threshold when the file transitions from non-rotational to rotational would be best, as this could be computed once based on the layout.  It is fairly unlikely that a file would have HDD stripes in the middle and flash at the beginning and end.

Alternately (probably better) is to store the nonrot state in each layout component (which is a perfect 1:1 mapping), and then this can be checked at IO submission time to see what type of storage it is covering.

Patrick Farrell PS19 · 2024-04-11 23:58

Hm, OK.  I don't think we can do this as easily as we'd like - the issue is we have to have this information very early.  I can consider this, though, your points are good ones.

The good thing is this isn't essential to merging the core feature, particularly since we're leaving it off by default for 2.16.

Andreas Dilger PS19 · 2026-04-26 02:56

Actually, an OST is (generally) only going to be rotational or non-rotational, and this is already returned to the client via statfs, so the clients should have full knowledge whether any IO is on flash or disk.

owner Patrick Farrell · uploader Marc Vef · PS21 uploaded 192d ago · NEW · open in Gerrit ↗

commit message

LU-13802 llite: add file nonrotational check

This patch adds the ability to note whether or not the
last IO to a file hit a rotational or a non-rotational
device.

This will be used in selecting the cutover thresholds for
hybrid IO, since DIO is synchronous and much slower on
spinning disk.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I75a9970f91b1776ed6f04ac0d000a9ba576df75a

50687 master J ✗1B ✓T ✗12R -2/2−

LU-12916 osd: use writeback for small writes in ldiskfs

carrying Qian Yingjin · needs rebase — checkpatch: cannot be cherry-picked

needs rebase 1 unique failure stalled 186d 1 unresolved L +639 −14

186d

needs rebase — checkpatch: cannot be cherry-picked

janitor: 1 test failure unique to this patch — sanity2@ldiskfs+DNE:test_127f

enforced failed: review-dne-part-1, review-dne-part-2, review-dne-part-3, review-dne-part-5, review-dne-selinux-ssk-part-1, review-dne-subtest-change, review-ldiskfs, review-ldiskfs-ubuntu

1 unresolved thread(s) await your reply — last from Qian Yingjin

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity2@ldiskfs+DNE:test_127f	seen in 1 other review

failed enforced test	platform	detail
review-dne-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-part-1	RHEL 9.5 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-part-2	RHEL 8.10 / x86_64	ran 11 tests. 1 tests failed: sanity-sec.	session
review-dne-part-3	RHEL 9.5 / x86_64	ran 3 tests. 1 tests failed: conf-sanity.	session
review-dne-part-5 crashed	RHEL 9.5 / x86_64	ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %%	session
review-dne-part-5	RHEL 8.10 / x86_64	ran 5 tests. 1 tests failed: sanityn.	session
review-dne-selinux-ssk-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-subtest-change	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanityn.	session
review-ldiskfs	RHEL 8.10 / x86_64, RHEL 9.5 / x86_64	ran 5 tests. 1 tests failed: sanity.	session
review-ldiskfs	RHEL 8.10 / x86_64, SLES 15.5 / x86_64	ran 5 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64	ran 6 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64	ran 6 tests. 1 tests failed: sanity.	session

12 optional test failure(s) — informational

build #124337 SUCCESS · tests all Maloo sessions

no vote yet: Sohei Koyama, Shaun Tancheff, Shuichi Ihara, Andreas Dilger, Alex Zhuravlev, Patrick Farrell

patchsets:

+68 earlier

PS69 10-02 Sohei Koyama PS70 10-29 Sohei Koyama PS71 10-29 Janitor Bot PS72 04-23 Marc Vef PS73 04-24 Marc Vef

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS72 2026-04-24 08:41 Qian Yingjin COMMIT_MSG:19 — Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext

Andreas Dilger PS72 · 2026-04-24 08:29

(typo) ".patch"

Writes for fscrypt files *should* always be full-block writes?  Why/how would writes to fscrypt files be partial blocks?  The client should be sending a full encrypted 4KiB block, but then set the size on the inode without truncating it.

Qian Yingjin PS72 · 2026-04-24 08:41

Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext4 for fscypt with write through (direct I/O) mode.

Although writes from clients are always full-block writes, but the size calls via lnb offset + len may be not full-block aligned, we use this size for the actual size of the file?

owner Qian Yingjin · uploader Marc Vef · PS73 uploaded 192d ago · NEW · open in Gerrit ↗

commit message

LU-12916 osd: use writeback for small writes in ldiskfs

This patch implements the writeback caching for small writes on
Lustre OSD (ldiskfs).
The writes are not synchronous any more on OST. It marks the page
as dirty and create buffer heads mapping pages to the physical
blocks. The kernel writeback mechanism will handle the actual I/O.

This patch should improve the performance for I/O cases such as
small writes from many files (i.e. mdtest-hard-write) or
unaligned I/O on a shared file (i.e. ior-hard-write).

Add a ldiskfs patch "ext4-writeback.patch" to not zero out
non-full page during writeback for a encrypted file. Without this
ldiskfs patch, it will fail sanity-sec/test_37.

According to the benchmark results, this patch can improve the
IO500 mdtest-hard performance and the performance of the real
application Nek5000 turbulent pipe with more than 10%.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I9b5aa568a49d5b39d04656fdacb94c3a6ec5a7ff

87 open

▶

Re-review needed — new patchset since your vote

(10)

64320 master J –B ✓T ✓R 0/2

LU-19921 target: verify GSSIAM tokens on the target

reviewing wangdi · PS16→PS97 since your -1 — your objection was dropped

4 unresolved XL +1829 −35

build #135119 SUCCESS · tests all Maloo sessions

no vote yet: Jinshan Xiong, Sebastien Buisson, Bruno Faccini, Andreas Dilger, Lai Siyao

patchsets:

+92 earlier

PS93 08-26 PS94 08-27 PS95 09-07 PS96 10-23 PS97 11-01

your previous vote: -1 on PS16 (2026-04-15 09:27) — now at PS97

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS97 · 2026-11-02 10:53

(minor) The helper does not return a Project ID. The downcall it writes is `<key> <mode>:<expire>:<principal>`, which tgt_gssiam_parse_downcall() parses as deny|ro|rw|expired, a relative expiry in seconds and the principal. The projid is resolved by MDT0 from the namespace (mdt_gssiam_projid_cb()). The DESCRIPTION paragraph says the same thing ("associated Project ID").

The downcall format itself is not documented anywhere on the page. That matters because a production site is expected to supply its own helper, and tgt_gssiam_parse_downcall() rejects ro/rw results that do not include a principal. Could the page describe the result format, including the required principal, and drop the Project ID wording?

Gerrit AI review for Lustre PS97 · 2026-11-02 10:53

(minor) On success gssiam_verify_target() leaves major = GSS_S_COMPLETE. Every later failure in this function then goes through `out:` with that value: rsc_entry_get(), the gctx allocation, gssiam_get_nodemap() (for example a nodemap_fileset_add() error other than -EPERM or -EEXIST), the gsc_nm_name allocation and lustre_pack_reply_v2(). The result is

    gss_pack_err_notify(req, GSS_S_COMPLETE, 0)

so the client gets a PROC_ERR reply with a success major and logs "GSSIAM init failed: 0/0". Before this patch, verify always failed, so these paths were never reached. Should major be reset to GSS_S_FAILURE after the verify call, or set at each later error?

Gerrit AI review for Lustre PS97 · 2026-11-02 10:53

(minor) If this OBD_STRDUP() fails, the entry continues with gd_subdir == NULL. alloc_entry() only drops an entry when init_entry marked it UC_CACHE_SET_INVALID(), which the client-side gssiam_entry_init() does on its own strdup failure.

Here a request for "/foo" is then handled as if the subdir were empty:

    tgt_gssiam_do_upcall()->mdt_gssiam_projid_cb(path = NULL)
      ->nodemap_fileset_get_root(NULL) -> "" (or the primary fileset)
      ->mdt_lookup_fileset("") -> ROOT projid
    l_gssiam_auth is asked about subdir ""
    tgt_gssiam_info_alloc() -> lid_subdir "" -> nodemap_fileset_clear()

An allocation failure therefore quietly changes the authorization scope and the projid rather than failing the request. Could this, and the gd_token allocation above, set UC_CACHE_SET_INVALID(entry) instead?

Gerrit AI review for Lustre PS97 · 2026-11-02 10:53

(minor) The refcount test runs without uc_lock, so a concurrent lookup can take a reference between this check and the state change:

    detach: gd_export_count -> 0, ue_refcount == 1
                         ctx init, same token:
                         upcall_cache_get_entry() -> ref 2, entry valid
    detach: upcall_cache_update_entry(UC_CACHE_INVALID)
                         tgt_gssiam_verify(): !UC_CACHE_IS_VALID -> -EACCES

The new mount then gets GSS_S_DEFECTIVE_CREDENTIAL, and the client fails with -EPERM rather than retrying. This can happen when the last mount for a token goes away while another mount with the same token connects. The UC_CACHE_EXPIRED path in tgt_gssiam_attach_cache_entry() has the same window. Would it be simpler not to invalidate here, and let the entry age out or be unlinked by check_unlink_entry() once nobody holds it?

owner wangdi · uploader wangdi · PS97 uploaded 20h ago · NEW · open in Gerrit ↗

commit message

LU-19921 target: verify GSSIAM tokens on the target

When the target receives a SEC_CTX_INIT RPC, it calls
tgt_gssiam_verify() to authenticate the token with the external
GSSIAM server via the l_gssiam_auth upcall. Once authenticated,
the token and its resolved identity/permissions are cached and
attached to the export.

This patch includes:
- Target GSSIAM verification framework (tgt_gssiam.c) and upcall
  cache interactions to validate tokens and manage the gssiam_server
  cache.
- l_gssiam_auth utility and man page to perform external
  authentication and downcall results to the kernel.
- Remote authentication forwarding via MDS_GET_INFO RPCs
  (tgt_gssiam_rpc_authenticate, mdt_get_info_gssiam_authenticate,
  ofd_gssiam_authenticate_remote) delegating verification to MDT0.
- Dynamic per-identity nodemap creation and property configuration
  (gssiam_get_nodemap) derived from the authenticated principal.
- Wire format definition (struct gssiam_auth_rep) in lustre_idl.h
  along with corresponding wiretest entries.
- Move DEFAULT_NODEMAP definition from nodemap_internal.h to
  lustre_nodemap.h as LUSTRE_DEFAULT_NODEMAP for cross-subsystem
  access.
- Rework tgt_mod_init() error-unwind paths in tgt_main.c to ensure
  complete rollback on initialization failures, including
  tgt_gssiam_init().

Signed-off-by: Di Wang <ddiwang@google.com>
Change-Id: I9fbd92e7e5dcdbc0039f90f547ce3841b005615e

66481 master J –B ✓T ✓R ✗

LU-12669 ec: fix EC recovery read for encrypted files

reviewing Zhenyu Xu · PS21→PS24 since your -1 — your objection was dropped

CR -1 ec2 pt_ecro218_review_blocker 5 unresolved L +309 −10

PS21→PS24 since your -1 — your objection was dropped

blocked on the owner: Zhenyu Xu voted -1

build #134661 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-11-02 00:14 · Zhenyu Xu -1 2026-11-02 10:01

no vote yet: Keguang Xu

patchsets:

+19 earlier

PS20 08-11 PS21 08-19 PS22 08-20 PS23 10-08 Andreas Dilger PS24 10-28 Andreas Dilger

your previous vote: -1 on PS21 (2026-08-19 20:14) — now at PS24

threads: 5 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS24 · 2026-10-30 11:22

(suggestion) This isn't a bug, but this loop is a copy of the page-cache branch of the decrypt loop in osc_brw_fini_request(): it stops at the first all-zero unit, clears private_2 when the zero unit is at offset 0, and calls llcrypt_decrypt_pagecache_blocks() per LUSTRE_ENCRYPTION_UNIT_SIZE. If the patch is refreshed, could both callers share one helper? Then a later change to the hole handling can't update only one of them.

reply PS24 2026-11-02 10:01 Zhenyu Xu lov_io.c:1994 — (defect) for 64K page size client (arm64), 4K empty does not mean a empty page.

Zhenyu Xu PS24 · 2026-11-02 10:01

(defect) for 64K page size client (arm64), 4K empty does not mean a empty page.
```
bool empty = true;

for (..) {
      if (is_empty_folio(folio, pgno, offs, unit))
                  continue;
      empty = false;
      ...
}
if (empty)
      folio_clear_private_2(folio);```

reply PS24 2026-11-02 10:11 Zhenyu Xu lov_io.c:2232 — (suggestion) as this function is also used to read raw data from data stripe, could change its name to lov_ec_read_raw_stripe(), and lov_ec_

Zhenyu Xu PS24 · 2026-11-02 10:11

(suggestion) as this function is also used to read raw data from data stripe, could change its name to lov_ec_read_raw_stripe(), and lov_ec_read_partiy_stripe() can be routed to it.

Gerrit AI review for Lustre PS24 · 2026-10-30 11:22

(style) This isn't a bug, but lov_ec_read_parity_stripe() now also reads data stripes. Its name and its parameters (pcomp_i, parity_idx, parity_foff) still say parity, so this call reads as if it fetched parity stripe k. If the patch is refreshed, consider a neutral name such as lov_ec_read_raw_stripe() with comp_i/stripe_i/rg_off parameters.

Gerrit AI review for Lustre PS24 · 2026-10-30 11:22

(suggestion) This fails some reads with -EIO even though the data could be rebuilt. Take 4+2 with data stripes A and B both down. A's page at position j is still cached from before the failure, and B's page at j is EC_DPG_ERROR:

- err_nr == 2, so two parity stripes are read
- lov_ec_read_raw_cached() skips A because lov_ec_readable_sub() returns NULL, so raw_folios[A] stays NULL
- at j, r_j == 1 and no_raw is set, so this returns -EIO and the whole read fails

The plaintext of A can't be used here. But A's page at j could be counted as one more erasure: its recov ptr would go to a scratch buffer, and the cached page would be left alone. Then r_j == err_nr and the shared recover_ctx already covers {A, B}. Is that worth handling here or in a follow-up? Before this patch the same case returned garbage, so -EIO is still an improvement.

tags: pt_ecro 218_review_blocker

owner Zhenyu Xu · uploader Andreas Dilger · PS24 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-12669 ec: fix EC recovery read for encrypted files

The parity of an encrypted file is computed from its data as stored
on the OSTs, but an EC recovery read rebuilds a stripe from data
pages that OSC has already decrypted, together with the parity as
stored, and returns garbage.

Rebuild the stripe from the data as stored instead: have OSC leave
the data pages read for the recovery encrypted and not uptodate,
rebuild the missing pages from them and the parity, then decrypt the
pages read and rebuilt, leaving an all-zero block (a hole) as OSC
does.  A data page already in the page cache holds the plaintext, so
read the stripe again from the OST for the positions being rebuilt,
rather than encrypting the plaintext, which cannot recreate the
stored data of a hole.  Without the key, leave the pages encrypted,
as a normal read does.  Keep these pages out of extents that OSC
decrypts, and send the encryption flag with the parity reads.

Add sanity-ec test_56a to recover an encrypted file with a hole in
a surviving stripe, both from the OSTs and with that stripe cached.

Test-Parameters: testlist=sanity-ec
Fixes: 783c13bb0b8f ("LU-12669 ec: recover data from parity")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: Ida4722d05d79bcbee4e680e75e94b005579adcac

69762 master J –B ✓T ✓R +2/2

LU-20287 lov: don't block setattr on unavailable OST

reviewing Keguang Xu · PS5→PS7 since your -1 — your objection was dropped

pt_ecro218_review_priority master-next 1 unresolved L +319 −25

build #134595 SUCCESS · tests all Maloo sessions

reviews: Ronnie Sahlberg +1 2026-10-29 03:43 · Andreas Dilger +1 2026-11-02 00:58

patchsets:

+2 earlier

PS1 10-17 PS2 10-18

PS3 10-22 PS4 10-25 PS5 10-27 PS6 10-27 Marc Vef PS7 10-28

your previous vote: -1 on PS5 (2026-10-27 09:34) — now at PS7

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS7 2026-11-02 00:58 Andreas Dilger cl_object.h:2567 — seems reasonable if patch is refreshed

Gerrit AI review for Lustre PS7 · 2026-10-28 16:35

(minor) The new mtime and the new atime are both compared with one shared baseline, max(ctime, sa_cur_time). sa_cur_time is the max of the old mtime and atime, the cached OST mtime and atime, and the inode mtime and atime that vvp_attr_get() reads at iter_init. ll_md_setattr() has already written the new values into the inode by then. ll_merge_attr() merges each field on its own (atime with atime, mtime with mtime), so a newer value in the *other* field can never win the max() for this one.

With the shared baseline:

    touch -m -d '+1 hour' f   # mtime ahead, no lock (new >= ctime)
    touch -a f                # atime = now < base (mtime +1h)

The second command now counts as backward. vvp_io_setattr_lock() takes a CLM_WRITE [0, EOF] lock, which revokes other clients' extent locks even when every OST is up. If an OST is down, the command also blocks. Before this patch it took no lock. touch -m behaves the same way when the atime is ahead. So does utimensat() with atime < mtime when both are later than ctime, because the new mtime is already in the inode.

Would it be safer to keep separate mtime and atime baselines (sa_cur_mtime / sa_cur_atime), so that each new time is compared only with its own field? The comments on sa_cur_time, in cl_setattr_ost() and in vvp_io_setattr_iter_init() would then also match what is actually collected.

Andreas Dilger PS7 · 2026-11-02 00:58

seems reasonable if patch is refreshed

tags: pt_ecro 218_review_priority

owner Keguang Xu · uploader Keguang Xu · PS7 uploaded 5d ago · NEW · open in Gerrit ↗

commit message

LU-20287 lov: don't block setattr on unavailable OST

chown/chgrp/chmod/touch wait in cl_setattr_ost() when a stripe OST
is down: VFS sets ATTR_CTIME and the client sends OST_SETATTR to
every stripe. That RPC only carries times/flags; UID/GID/PROJID are
authoritative on the MDT and are pushed by MDS osp_sync (persistent
llog, retried until the OST returns). Skipping the client RPC
therefore does not leave stale ownership on OST objects.

For non-truncate/non-fallocate setattr on any layout, mark LOV
sub-IOs ci_ndelay so OSC sends OST_SETATTR with rq_no_delay and
rq_no_resend (plus a 5s delay limit for idle import reconnect),
and ignore -EAGAIN/-ENOTCONN/-ESHUTDOWN/-ETIMEDOUT for such sub-IOs
in osc_io_setattr_end(). Ndelay applies only when the new mtime/atime
are >= max(ctime, sa_cur_time), where sa_cur_time is the known prior
inode/cached OST times: ll_merge_attr() takes max(MDT, OST), so
skipping an OST that already holds a newer time would leave that
value visible. Missed updates are not repaired later, and on a slow
but connected OST the client update may also be dropped.

The CLM_WRITE [0, EOF] lock in vvp_io_setattr_lock() uses that same
baseline, not ctime alone. Plain touch/utime(NULL) therefore takes
the lock when sa_cur_time is ahead of the client clock (writer clock
skew, or a future timestamp) and revokes other clients' cached
extent locks; that path does not set ndelay. That is intentional:
without it ndelay would skip the newer OST and max() would keep the
wrong time.

Still block when:
- mtime/atime moves earlier than that baseline;
- OP_XVALID_FLAGS is set (chattr, encryption flag): MDS never
  pushes flags to OSTs.

This only unblocks the setattr RPC itself (including chmod(2)).
stat/glimpse can still hang when no usable stripe/mirror is
reachable: e.g. coreutils chmod stats via FTS before chmod(2),
so the command may block on plain files even though the syscall
would not. That is unchanged here (see LU-11962 for MDT SOM).

Also re-push UID/GID/PROJID from the MDT inode on FLR RESYNC_DONE
in lod_layout_change() with the layout version update. osp_sync
normally retries ownership pushes; this is belt-and-suspenders if
an MDS->OST ownership setattr was skipped (e.g. exercised with
OBD_FAIL_LFSCK_BAD_OWNER; layout LFSCK can also repair that).

Add sanity 39v (plain file) and sanity-flr 100a/100b.

Test-Parameters: testlist=sanity env=ONLY=39v
Test-Parameters: testlist=sanity-flr
Assisted-by: Cursor:Grok-4.6/Opus5.5 llm_code_and_review_tools
Signed-off-by: Keguang Xu <kxu@ddn.com>
Change-Id: I5005451e10f293279c441c1fc6d012fb5337d95d

69573 master J –B ✓T ✓R 0/2

LU-19932 llite: Treat direct write as sync write again

reviewing Xiao Yang · PS8→PS10 since your +1

S +35 −2

20h

build #135046 SUCCESS · tests all Maloo sessions

no vote yet: Qian Yingjin, Andreas Dilger, Zhenyu Xu, Patrick Farrell, Timothy Day

patchsets:

+5 earlier

PS1 10-11 PS2 10-12 PS3 10-12 PS4 10-14 PS5 10-15

PS6 10-15 PS7 10-18 PS8 10-29 PS9 10-29 PS10 11-01

your previous vote: +1 on PS8 (2026-10-29 10:31) — now at PS10

owner Xiao Yang · uploader Xiao Yang · PS10 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-19932 llite: Treat direct write as sync write again

LU-16695 dropped O_DIRECT from wr_sync when switching to ki_flags.
When a DIO write falls back to a buffered write (e.g. unaligned DIO to
OSTs without unaligned DIO support, or -EBUSY from invalidating cached
pages) and pages are committed to OSC once the buffered write reaches
64MiB (PTLRPC_MAX_BRW_SIZE), the last extent committed by the write
cl_io stays active until its cl_io_end. Before that, the same thread
runs direct_write_fallback(), whose nested fsync cl_io waits for that
extent, so the thread deadlocks with itself.

Include IOCB_DIRECT in wr_sync again so the extent is released at
commit in cl_io_start instead of at cl_io_end.

Test-Parameters: serverversion=2.15 testlist=sanity env=ONLY=119
Fixes: dad7079dfd9d ("LU-16695 llite: switch to ki_flags from f_flags")
Signed-off-by: Xiao Yang <xyang@ddn.com>
Change-Id: I1dce660ee3a29341e1c484dc7c466f05a176a228

65219 master J –B ✓T ✓R +1/2

LU-12669 llite: make AIO reads on EC file synchronous

reviewing Patrick Farrell · PS38→PS39 since your +1

pw_ownpt_ecro M +127 −14

build #134978 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-10-31 07:23

no vote yet: Ronnie Sahlberg, Shaun Tancheff, Zhenyu Xu

patchsets:

+34 earlier

PS1 04-10 PS2 04-10 PS3 04-10 PS4 04-10 PS5 04-10 PS6 04-15 PS7 04-15 PS8 04-25 PS9 04-29 PS10 04-29 PS11 04-29 PS12 04-29 PS13 05-05 PS14 05-05 PS15 05-05 PS16 05-06 Ronnie Sahlberg PS17 05-20 Marc Vef PS18 06-04 Ronnie Sahlberg PS19 07-13 Marc Vef PS20 07-13 Marc Vef PS21 07-13 Marc Vef PS22 07-15 Marc Vef PS23 07-26 Marc Vef PS24 07-26 Marc Vef PS25 08-17 Marc Vef PS26 08-17 Marc Vef PS27 08-17 Marc Vef PS28 08-17 Marc Vef PS29 08-17 Marc Vef PS30 08-19 Maximilian Dilger PS31 10-16 patrick bot PS32 10-18 patrick bot PS33 10-19 patrick bot PS34 10-19 patrick bot

PS35 10-20 patrick bot PS36 10-28 patrick bot PS37 10-28 patrick bot PS38 10-29 patrick bot PS39 10-31 patrick bot

your previous vote: +1 on PS38 (2026-10-30 17:46) — now at PS39

tags: pw_own pt_ecro

owner Patrick Farrell · uploader patrick bot · PS39 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-12669 llite: make AIO reads on EC file synchronous

For sync DIO, EC recovery on read failure runs inline in
ll_file_io_generic() after cl_sync_io_wait_recycle().  AIO
has the same recovery requirement but cannot run recovery
from the sub-DIO completion path: ll_file_io_generic()
returns -EIOCBQUEUED to the VFS before the BRW completion
fires, so the syscall context recovery needs (the lu_env,
the user's iov_iter, the range lock) is gone by the time the
read error is known.

Rather than building a workqueue-based async recovery path,
make AIO reads on EC files behave as sync DIO: after
cl_io_loop(), for an AIO read on a layout with a parity
mirror, set cda_no_aio_complete and cda_creator_free to
disarm the async ki_complete path, then drop into the same
wait + restart code sync DIO uses.  The VFS calls
ki_complete itself when we return.

The check is on the whole layout, not on the component at
the read offset, so a read starting in a plain component is
drained on its first pass too.  Otherwise an error from that
pass surfaces in a later EC pass, which reconstructs only
its own range and reports the earlier bytes as read.  For
the same reason, an AIO read whose layout gains a parity
mirror after an earlier async pass has moved bytes stays
async, and reports a read error as it would on a plain file.

The flag flip is race-free because the submission ref on
cda_sync is still held: end_io cannot fire until we drop it
in cl_sync_io_wait_recycle().

This disables async semantics for every AIO read on an EC
file, not only those that need recovery. We cannot tell at
submission time whether reconstruction will be needed, so
the sync conversion drains queued IO even when submission
requests a restart without returning -EIOCBQUEUED.  As a
side effect, the range_lock taken by ll_file_io_generic() is
now held until all sub-DIOs have drained, instead of being
dropped while they are still in flight.

Do not retry -EINVAL or -EFAULT as a reconstruction read,
for sync DIO or AIO: they reject the request itself, so an
unaligned O_DIRECT read fails on an EC file as it does on a
plain file.

Re-enable the existing degraded AIO regression tests now
that these reads use the synchronous recovery path. Cover
an inactive later stripe in a single multi-stripe AIO, and
a failed plain component ahead of an EC one.

Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Assisted-by: Opus:4.8 llm_code_and_review_tools
Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I394a3610e33b29ead8f5adb52dfa21db6b721944

64086 master J –B ✓T ✓R 0/2

LU-20958 lov: handle unavailable EC stripes in lseek

reviewing Patrick Farrell · PS47→PS48 since your -1 — your objection was dropped

ec2 pt_ecro218_review_priority 5 unresolved L +453 −12

build #134718 SUCCESS · tests all Maloo sessions

no vote yet: Keguang Xu, Andreas Dilger, Zhenyu Xu

patchsets:

+43 earlier

PS44 08-19 Maximilian Dilger PS45 10-14 Andreas Dilger PS46 10-16 Andreas Dilger PS47 10-21 Andreas Dilger PS48 10-29 Andreas Dilger

your previous vote: -1 on PS47 (2026-10-27 10:04) — now at PS48

threads: 4 need your reply · 0 waiting on others · 1 own notes · 0 bot

note PS47 2026-10-27 10:04 Marc Vef patchset comment — Highlighting a conflict with 69111. This patch (and the parent) needs to be refreshed due to 68361 since the code is no longer compatible. (

Marc Vef PS47 · 2026-10-27 10:04

Highlighting a conflict with 69111. This patch (and the parent) needs to be refreshed due to 68361 since the code is no longer compatible. (-1)

Does this patch need the current parent patches or can it be rebased and updatd for the RAID10 patch now?

Gerrit AI review for Lustre PS48 · 2026-10-31 02:29

(defect) The code here depends on lli_attr_valid losing OBD_MD_FLSIZE when the UPDATE lock is cancelled. That clearing is added by 69828 ("LU-20958 llite: drop strict SOM size with the UPDATE lock"), but 69828 is not an ancestor of this patch set. The parent is 8bef60849e (an older patch set of 69694), which sits directly on 40b80bc048 on master. In this tree, lli_attr_valid is written only in ll_update_inode(), and nothing clears it in ll_md_blocking_ast().

Without 69828, this sequence goes wrong:

    client A: open/stat while the file is in sync -> OBD_MD_FLSIZE, i_size = S
    client B: writes the file up to T > S (MDS revokes LAYOUT|UPDATE, marks the SOM stale)
    an OST of a data stripe becomes unavailable
    client A: lseek(SEEK_DATA, S) -> ci_ec_unavail, but FLSIZE is still set
      -> no revalidate; vvp_io_lseek_start() skips ll_merge_attr()
      -> eof = S, so the result is -ENXIO although [S, T) has data

Because ll_merge_attr() is skipped, this also misses B's data on the stripes that are available. Should this patch be rebased onto 69828 so that the two cannot land in the wrong order?

reply PS47 2026-10-29 01:23 Andreas Dilger lov_io.c:3833 — Not yet in PS48: 68361 is still at PS17 and its stack does not apply to master. This will be done when 64086 is rebased onto the refreshed

Andreas Dilger PS47 · 2026-10-27 03:43

This needs to be updated for the LU-20690 parity layout in https://review.whamcloud.com/68361 ("LU-20690 ec: like-RAID10 parity layout"), which removes RAID0|PARITY: parity components become LOV_PATTERN_PARITY with the p parity objects of a raid set sharing one offset.  Once that lands, this check returns -EOPNOTSUPP for every EC file and the parity SEEK_DATA/SEEK_HOLE path is never used.  Please check for the parity flag/pattern instead and use the new mapping below.

Andreas Dilger PS47 · 2026-10-29 01:23

Not yet in PS48: 68361 is still at PS17 and its stack does not apply to master.  This will be done when 64086 is rebased onto the refreshed 68361.

reply PS47 2026-10-29 01:23 Andreas Dilger lov_io.c:3862 — Not yet in PS48, for the same reason. The slot helpers and absolute-offset row/set mapping will be used once 68361 is refreshed.

Andreas Dilger PS47 · 2026-10-27 03:43

The parity offset needs to change with 68361 in two ways:
1. RAID10 layout: one stripe_size per raid set per row, and the parity index selects the object rather than the offset, i.e. poff = e_start + (row * R + set) * ss + intra, pstripe = (stripe of that slot) + pi with R = esc_n0 + esc_n1, via the 69516/68361 slot helpers (lsme_parity_slots_per_raidset()) rather than lov_stripe_number() on the concatenated offset.
2. The row/set must come from the absolute data offset, not offset - e_start (see my lov_io.c comment on 68361 PS17): row = off / (c * ss), offset_i = (off / ss) % c, and poff relative to row0 = e_start / (c * ss).  Otherwise when e_start is not a multiple of the stripe width this looks at the parity of a different raid set than the one holding the data.
Ideally use the shared helper that 68361 adds for this mapping so the kernel recovery, llapi resync, hedge (64919) and lseek all agree.  Rebase onto that series once it is updated.

Andreas Dilger PS47 · 2026-10-29 01:23

Not yet in PS48, for the same reason.  The slot helpers and absolute-offset row/set mapping will be used once 68361 is refreshed.

Gerrit AI review for Lustre PS48 · 2026-10-31 02:29

(minor) lov_parity_mirror_index_from_data() looks up the parity link from the data component at lio->lis_pos, which is the lseek start, not from dsub's component dindex. If the seek starts in an earlier component of the mirror that has no parity link (e.g. a non-EC first PFL component), pmirror is < 0, and the stripe is reported as all data even though in-sync parity exists. The result is still correct, only less precise. If this is changed to use lsm_entries[dindex]->lsme_mirror_link_id, note that the LASSERTF in lov_io_sub_init() does the same lis_pos lookup, so it would need to stay in step for lov_sub_get() on the parity stripe.

tags: pt_ecro 218_review_priority

owner Patrick Farrell · uploader Andreas Dilger · PS48 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-20958 lov: handle unavailable EC stripes in lseek

SEEK_DATA/SEEK_HOLE on an EC file with a data stripe on an OST that
was deactivated would fail with -ESHUTDOWN, and with an OST that was
down, e.g. during an OSS failover, the lseek would wait for the OST to
return, although reads of the file are rebuilt from parity at once.

Do the lseek with no-delay lock enqueues, as reads do.  When the
sub-lock enqueue of a stripe in an EC data component fails, mark it in
this IO's lov_io_sub::sub_ec_err and do not start its sub-IO.  Retry
with blocking enqueues once every mirror has been tried.  The
unavailable stripe leaves its size out of the merged stripe
attributes, so take EOF from the strict SOM size from the MDS instead,
fetched again as stat() does if the UPDATE lock was dropped.  Without
one, wait for the OST, or return -EIO if it stays unavailable, rather
than miss data past a smaller EOF.  A stripe whose extent lock is
still cached is locked locally, so its OST_SEEK still waits for the
OST.

Resync writes parity only where a data stripe of its raid set has
data, page aligned, and punches the rest, so a hole in an in-sync
parity stripe is a hole in every data stripe of the raid set.  Seek an
available parity stripe of the unavailable stripe's raid set from the
matching offset, and map the result back, for the RAID0|PARITY parity
layout.  Each row of a raid set holds one stripe_size chunk in every
data and parity object, so offsets in the two objects advance
together.  The result can report data where only another stripe of the
raid set has data, but never a hole where the stripe may have data.
With no parity stripe that can answer, report the stripe as data from
where the IO starts in it, with no hole before its end, which is
always correct: a seek must not skip data, but may report a hole as
data.  Since such a stripe has no hole at its end, report the hole at
EOF for SEEK_HOLE if nothing before it is found, and -ENXIO for
SEEK_DATA at EOF.

The parity object is not covered by the lseek lock, so seek it with a
server lock, and do not trust the LVB size of an unlocked object in
osc_io_lseek_start().  Seek parity without delay, failing at once if
its OST is disconnected; an idle import still reconnects for the seek.

Add sanity-ec test_79a, which stops the OST of a stripe, and
test_79b, which checks holes found from parity, and that the stripe
is reported as data when the parity OSTs are stopped too.

Test-Parameters: testlist=sanity-ec ostcount=8
Fixes: 783c13bb0b8f ("LU-12669 ec: recover data from parity")
Signed-off-by: Patrick Farrell <patrick@thelustrecollective.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I5cce4e0ea51c68b0c6fda1d83b694af19cad57bd

62389 master J –B ✓T ✓R 0/2

LU-19566 lustre: add layout flags to lfsck

reviewing Patrick Farrell · PS14→PS20 since your -1 — your objection was dropped

ec2 pw_ownpt_ecro L +761 −149

build #133811 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger, Lai Siyao, Alex Zhuravlev, Hongchao Zhang, Timothy Day

patchsets:

+15 earlier

PS1 11-05 PS2 11-05 PS3 11-06 PS4 11-07 PS5 11-07 PS6 11-07 PS7 11-08 PS8 11-08 PS9 11-09 PS10 11-11 PS11 11-11 PS12 11-26 PS13 03-30 Marc Vef PS14 03-30 Marc Vef PS15 10-07

PS16 10-08 PS17 10-08 PS18 10-09 patrick bot PS19 10-10 patrick bot PS20 10-16 patrick bot

your previous vote: -1 on PS14 (2026-04-02 10:21) — now at PS20

tags: pw_own pt_ecro

owner Patrick Farrell · uploader patrick bot · PS20 uploaded 17d ago · NEW · open in Gerrit ↗

commit message

LU-19566 lustre: add layout flags to lfsck

EC parity mirror components hold parity, not file data, so
LFSCK must restore them as parity components when it
rebuilds a layout from orphan OST objects. It restored each
component with only LCME_FL_INIT, losing the parity flag,
the EC geometry and the link between the data and parity
mirrors, so a rebuilt EC file could not use its parity.

For EC components, store the long-lived component flags
(prefer, nosync, parity, nocompr), the EC data and code
stripe counts and the mirror link id at the end of
filter_fid. The client sends them on writes and setattr in
the obdo fields that were o_uid_h and o_gid_h, never used,
flagged by the never-used 0x00200000 valid bit (formerly
OBD_MD_FLQOS), now OBD_MD_FLCOMPFLAGS, and the OST passes
them to the MDT in the orphan record, so LFSCK restores
them. The objects only get them from a write or setattr, so
a change of flags is stored once the file is next written. A
rebuilt parity component also gets the parity pattern, which
clients use to keep reads off it, and LFSCK accepts that
pattern when it adds further orphans to such a file. When
LFSCK adds a component to an EC file its parity is marked
stale, since the objects do not show whether the parity was
current, so it is only used again after a resync. LFSCK
scans still skip EC files: repairing a missing parity object
there would need the parity regenerated or marked stale.

STALE is not stored: resync writes an object while its
mirror is still stale, so a stored STALE is backward and can
rebuild a layout whose mirrors are all stale, which LBUGs
the MDS when it loads that layout.

Other components keep the 52-byte filter_fid, so that
servers before this change and older tools still read it,
and only EC files, which need the new servers anyway, get
the 60-byte form. Servers without this change do not handle
it: their LFSCK and MDT migration get -ERANGE reading it,
and an OST keeping the parent FID in LMA (256-byte inodes)
refuses to have it written, unless they have "lfsck: read a
filter_fid longer than ours", which this patch is based on.
An OST keeping the parent FID in LMA has no room for these
fields, and does not store them.

sanity-lfsck test_47 and test_48 cover mirrors after resync
and EC files; test_48 checks the rebuilt EC geometry and,
after a resync, reads the file with a data OST failing.
test_45 now waits for LFSCK to finish on every target, not
only on the OST it checks, so that the next test can start
LFSCK.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Assisted-by: ClaudeCode:claude-opus-5-5
Change-Id: I716dbf24db48cc50385a2fa9bcc446d291380814

66526 master J –B ✓T ✓R 0/2

LU-19079 krb: mount-point specific principal

reviewing Sebastien Buisson · PS8→PS26 since your +1

L +698 −118

build #133487 SUCCESS · tests all Maloo sessions

no vote yet: Aurelien Degremont, Bruno Faccini, Andreas Dilger, Li Xi

patchsets:

+21 earlier

PS1 06-08 PS2 06-08 PS3 06-08 PS4 06-09 PS5 06-09 PS6 06-23 PS7 07-09 PS8 07-09 PS9 08-17 PS10 08-18 PS11 08-19 PS12 08-20 PS13 08-21 PS14 08-24 PS15 08-25 PS16 08-26 PS17 08-28 PS18 08-28 PS19 08-31 PS20 09-01 PS21 09-02

PS22 09-03 PS23 09-04 PS24 09-23 PS25 09-24 PS26 10-13

your previous vote: +1 on PS8 (2026-07-10 08:06) — now at PS26

owner Sebastien Buisson · uploader Sebastien Buisson · PS26 uploaded 20d ago · NEW · open in Gerrit ↗

commit message

LU-19079 krb: mount-point specific principal

By default, the principal used by the Lustre client to authenticate
against the KDC is the first valid one found in the system keytab
(usually /etc/krb5.keytab). This means we cannot choose which
principal to use for Lustre.
However, there is a need to support multiple Kerberos principals at
the same time on the client, for instance in the context of GSS based
identification to allow different mount points on the same client to
be considered part of different nodemaps.
To address this need, this patch introduces a new client mount option
'krb5princ', to allow specifying the full Service Principal Name (SPN)
to use for Kerberos authentication of this specific client mount.
To pass this information to the kernel, we resort to a kernel keyring
key whose descriptor is in the form "lustre:<fsname>:<client uuid>",
and whose payload is the principal name to use.
Having the client UUID in the key desc allows request_key() to find
this client-specific key by putting the UUID into the callout info for
context negotiation that happens in userspace. There, the SPN
contained in the key payload allows to filter out the principals
fetched by the krb5 primitives.
Note that we have to make use of a temporary kernel keyring key, as
the UUID is computed at mount in kernel space, whereas the krb5princ
option is handled in userspace by the mount command before filling the
superblock.
Lastly, Kerberos credentials cache (krb5cc) on the client must be
created for each nodemap-specific principal being used, to support
multiple principals on the same node.

sanity-krb5 test_12 is added to exercise the new krb5princ client
mount option.

Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ic0e059bfadbd37c11fcdfec3884eacf4c028337f
Reviewed-by: Marc Vef <mvef@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>

52166 master J ✓B ✓T ✓R -1/2−

LU-13814 osc: add osc_dio_page_submit

reviewing Patrick Farrell · PS34→PS36 since your -1 — your objection was dropped

needs rebase M +42 −16

59d

PS34→PS36 since your -1 — your objection was dropped

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked

failed enforced test	platform	detail
review-dne-part-3	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: conf-sanity.	session
review-dne-part-5	RHEL 8.10 / x86_64	ran 5 tests. 1 tests failed: sanityn.	session
review-dne-part-8	RHEL 9.5 / x86_64	ran 4 tests. 1 tests failed: replay-dual.	session
review-dne-zfs-part-1	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-zfs-part-2	RHEL 8.10 / x86_64	ran 11 tests. 1 tests failed: sanity-lnet.	session
review-dne-zfs-part-7	RHEL 8.10 / x86_64	ran 5 tests. 1 tests failed: sanity-scrub.	session
review-ldiskfs	RHEL 8.10 / x86_64, RHEL 9.5 / x86_64	ran 5 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64	ran 8 tests. 1 tests failed: sanity.	session
review-zfs	RHEL 8.10 / x86_64	ran 10 tests. 1 tests failed: replay-single.	session

8 optional test failure(s) — informational

build #119017 SUCCESS · tests all Maloo sessions

no vote yet: Qian Yingjin, Andreas Dilger, Zhenyu Xu

patchsets:

+31 earlier

PS32 07-04 PS33 07-14 PS34 11-06 PS35 11-09 PS36 11-16

your previous vote: -1 on PS34 (2025-11-07 08:23) — now at PS36

owner Patrick Farrell · uploader Patrick Farrell · PS36 uploaded 350d ago · NEW · open in Gerrit ↗

commit message

LU-13814 osc: add osc_dio_page_submit

This patch adds the osc_dio_page_submit and specializes
osc_page_submit to take only BIO pages.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I048bbcd25ffee28fbbd8fee26017796b5e3586bd

68384 b2_15 B ✓T ✓R -1/1−

LU-20554 quota: preserve partial OST usage

reviewing Sergey N · PS2→PS7 since your +1

M +127 −6

59d

build #130942 SUCCESS · tests all Maloo sessions

patchsets:

+2 earlier

PS1 08-26 PS2 08-26

PS3 08-26 PS4 08-30 PS5 08-30 PS6 08-31 PS7 09-03

your previous vote: +1 on PS2 (2026-08-26 13:07) — now at PS7

owner Sergey N · uploader Sergey N · PS7 uploaded 60d ago · NEW · backport · open in Gerrit ↗

commit message

LU-20554 quota: preserve partial OST usage

lov_quotactl() accumulates quota usage from reachable OSTs and stores
the subtotal in dqb_curspace even when querying another target fails.
quotactl_ioctl() currently ignores this subtotal for errors that are
not explicitly handled, so lfs quota may report only MDT usage.

Always copy dqb_curspace returned by lov_quotactl(), but set QIF_SPACE
only when all expected OST quota queries succeed. This reports usage
from reachable OSTs and MDTs while retaining brackets to mark
incomplete results.
For example, with an unavailable OST, lfs quota now reports:

Filesystem     used  bquota  blimit  bgrace ...
/lusfs/ [105.3G]    200G    200G       - ...

Some errors happened when getting quota info. Some devices may be
not working or deactivated. The data in "[]" is inaccurate.

The brackets make it clear that the displayed usage is incomplete,
while 105.3G includes usage reported by the reachable OSTs.

Remove the obsolete special handling for -EREMOTEIO. Commit
001b8dbfacb7 ("LU-3963 cleanup: C89 and build cleanups") changed
lov_quotactl() to return -ENETDOWN when an expected OST is inactive,
making the special case unreachable.

Add sanity-quota test_16c to verify that usage from a reachable OST is
preserved and marked incomplete while another OST is unavailable.
The b2_15 backport uses getquota because lfs quota --space is not
available in 2.15.8. During cleanup, the test attempts to reactivate
the OSC on every MDT and verifies the final import states because
activation can race with recovery and return -EALREADY.

Lustre-change: https://review.whamcloud.com/67911
Lustre-commit: TBD (from ebb8c1421fcc0642c4fda3c1879929eb928a262c)

Fixes: 001b8dbfacb7 ("LU-3963 cleanup: C89 and build cleanups")
Test-Parameters: testlist=sanity-quota env=ONLY=16c
Signed-off-by: Sergey Noskov <sergey.noskov.sn@gmail.com>
Change-Id: I17252ef34ff32dbdd0561c1570ce66f31cacfb87

▶

Awaiting your review — CI green

(52)

69746 master J –B ✓T ✓R 0/2

LU-20974 gss: drop unused multi-round context negotiation

reviewing Sebastien Buisson · CI green — awaiting your review

pw_shepherd S +19 −20

build #134097 SUCCESS · tests all Maloo sessions

no vote yet: Arshad Hussain, Bruno Faccini, Andreas Dilger, Timothy Day

patchsets: PS1 10-16 PS2 10-20

tags: pw_shepherd

owner Sebastien Buisson · uploader Sebastien Buisson · PS2 uploaded 13d ago · NEW · open in Gerrit ↗

commit message

LU-20974 gss: drop unused multi-round context negotiation

Lustre clients always initiate GSS context negotiation with
PTLRPC_GSS_PROC_INIT and an empty handle, and complete it in a
single round. PTLRPC_GSS_PROC_CONTINUE_INIT is never sent, so the
server-side support for multi-round negotiation is never used.

Remove it:
- in gss_svc_accept(), stop routing PTLRPC_GSS_PROC_CONTINUE_INIT
  to gss_svc_handle_init(), so it is dropped as an unknown proc;
- in gss_svc_handle_init(), refuse any init request carrying a
  non-empty handle;
- in lsvcgssd, refuse requests with a non-empty input handle, and
  treat GSS_S_CONTINUE_NEEDED from gss_accept_sec_context() as a
  negotiation failure, releasing the context.

Test-Parameters: trivial testlist=sanity
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Assisted-by: Claude:claude-opus-5-5
Change-Id: If72606991a3f49ae7f8e53a4e46a819a0e561bcb

64321 master J –B ✓T ✓R +3/2

LU-19921 ptlrpc: register gssiam policy

reviewing wangdi · CI green — awaiting your review

S +16 −2

build #135122 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-11-01 19:45 · Sebastien Buisson +1 2026-11-01 19:45 · Alex Zhuravlev +1 2026-11-01 19:45

no vote yet: Jinshan Xiong

patchsets:

+102 earlier

PS1 03-03 PS2 03-03 PS3 03-10 PS4 03-12 PS5 03-12 PS6 03-12 PS7 03-28 PS8 03-29 PS9 03-29 PS10 03-29 PS11 03-30 PS12 03-30 PS13 03-30 PS14 04-03 PS15 04-03 PS16 04-04 PS17 04-07 PS18 04-10 PS19 04-10 PS20 04-12 PS21 04-17 PS22 04-20 PS23 04-23 PS24 04-24 PS25 04-25 PS26 04-27 PS27 05-13 PS28 05-14 PS29 05-17 PS30 05-17 PS31 05-17 PS32 05-19 PS33 05-20 PS34 05-21 PS35 05-21 PS36 05-25 PS37 05-29 PS38 05-30 PS39 06-02 PS40 06-03 PS41 06-03 PS42 06-04 PS43 06-04 PS44 06-06 PS45 06-08 PS46 06-10 PS47 06-10 PS48 06-11 PS49 06-12 PS50 06-13 PS51 06-15 PS52 06-15 PS53 06-18 PS54 06-19 PS55 06-22 PS56 07-10 PS57 07-12 PS58 07-20 PS59 07-24 PS60 07-27 PS61 07-29 PS62 07-30 PS63 08-02 PS64 08-02 PS65 08-02 PS66 08-02 PS67 08-02 PS68 08-04 PS69 08-05 PS70 08-05 PS71 08-06 PS72 08-06 PS73 08-07 PS74 08-07 PS75 08-07 PS76 08-07 PS77 08-08 PS78 08-09 PS79 08-09 PS80 08-10 PS81 08-11 PS82 08-13 PS83 08-13 PS84 08-13 PS85 08-13 PS86 08-14 PS87 08-15 PS88 08-16 PS89 08-17 PS90 08-18 PS91 08-18 PS92 08-18 PS93 08-19 PS94 08-20 PS95 08-21 PS96 08-22 PS97 08-22 PS98 08-22 PS99 08-23 PS100 08-24 PS101 08-25 PS102 08-25

PS103 08-26 PS104 08-27 PS105 09-07 PS106 10-23 PS107 11-01

owner wangdi · uploader wangdi · PS107 uploaded 20h ago · NEW · open in Gerrit ↗

commit message

LU-19921 ptlrpc: register gssiam policy

This is the final patch in the GSSIAM (Identity and Access
Management) security flavor series. It calls
sptlrpc_register_policy(&sptlrpc_gssiam_policy) in
sptlrpc_gssiam_init() and sptlrpc_unregister_policy() in
sptlrpc_gssiam_exit() to enable the GSSIAM policy for client/server
authentication and authorization.

It also improves error-unwind handling in sptlrpc_gssiam_init()
by introducing out-of-line tunables_cleanup and cache_cleanup
labels after policy registration.

Signed-off-by: Di Wang <ddiwang@google.com>
Change-Id: Ic5e1bc4f7d0db18abcc7f52525d4a8e0efdae7c3

70268 master J –B ✓T ✓R +1/2

LU-20076 tests: read all servers at once in wait_nm_sync

reviewing patrick bot · CI green — awaiting your review

pw_own S +22 −18

build #134964 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-10-31 00:53

no vote yet: Patrick Farrell

patchsets: PS1 10-30 PS2 10-30 PS3 10-31

tags: pw_own

owner patrick bot · uploader patrick bot · PS3 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-20076 tests: read all servers at once in wait_nm_sync

wait_nm_sync took about 9 remote commands per call on three
server nodes: a NID lookup and a get_param per server, each its
own ssh, plus the MGS lookups. It runs about 500 times per
sanity-sec run, and is a third of that suite's time.

Assuming time scales with remote-command count, reducing 9 to 3
projects a 2/9 (22%) saving for sanity-sec. Rainshadow records
1,546 review-dne-part-2 runs averaging 7,148 seconds over
October 1-30, 2026: about 26 minutes per run, or 680 suite-hours
in that window. This is an overhead-based estimate using the
one-third profile share above, not a measured before/after
saving. Other configurations and callers in sanity, sanityn,
sanity-quota and sanity-hsm are not included in that estimate.

Compare host names rather than NIDs, and read all the servers
with one command per pass, which leaves 2 or 3 per call.

A pass is now much shorter, so 10 passes would no longer allow
the time they took on CI, where each cost seconds of ssh. Wait
up to 40 seconds rather than 10 passes, and report the seconds
waited. A real sync failure is reported after those 40 seconds.

Test-Parameters: trivial testlist=sanity-sec
Test-Parameters: mdscount=2 mdtcount=4 testlist=sanity-sec
Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Patrick Farrell <patrick@thelustrecollective.com>
Change-Id: Iea781dace01af90058f95ad17cc6429fe45fb547

65798 master J –B ✓T ✓R 0/2

LU-20238 ec: GFNI-accelerated EC primitives

reviewing Hiroshi Nishida · CI green — awaiting your review

pt_ecropw_smallfix XL +3332 −38

build #134847 SUCCESS · tests all Maloo sessions

no vote yet: patrick bot, Keguang Xu, Ronnie Sahlberg, Patrick Farrell, Maximilian Dilger, Andreas Dilger, Zhenyu Xu, Oleg Drokin

patchsets:

+18 earlier

PS1 05-11 PS2 05-11 PS3 05-11 PS4 05-12 PS5 05-12 PS6 05-13 PS7 05-13 PS8 05-17 PS9 06-08 PS10 06-17 PS11 07-10 PS12 07-10 PS13 08-05 PS14 08-06 PS15 08-07 PS16 10-01 patrick bot PS17 10-02 PS18 10-05 patrick bot

PS19 10-13 PS20 10-21 patrick bot PS21 10-21 PS22 10-24 patrick bot PS23 10-30 patrick bot

tags: pt_ecro pw_smallfix

owner Hiroshi Nishida · uploader patrick bot · PS23 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20238 ec: GFNI-accelerated EC primitives

Add a runtime-dispatched GFNI fast path to libec.a (LU-19465).
On x86_64 with GFNI, ec_encode_data(), ec_encode_data_update(),
ec_init_tables(), gf_vect_dot_prod() and gf_vect_mad() route to
AVX-512 or AVX2 GFNI kernels; everywhere else the existing
ec_*_base() path is used unchanged.

The kernels are C intrinsics (_mm{256,512}_gf2p8affine_epi64_epi8)
gated per-function with __attribute__((target(...))), so the build
needs only GCC >= 10 / clang >= 8 and no NASM.  They use only the
GF(2^8) affine instruction and XOR/load/store, not the PSHUFB
nibble-table lookup the ISA-L AVX2/SSE kernels rely on.

The multi-output AVX-512 dot_prod kernels take the coefficient as
an EVEX {1to8} embedded broadcast to keep it off Port5; compilers
will not emit {1to8} from intrinsics, so a short inline-asm macro
is used.  The mad kernels use _mm512_set1_epi64(), the coefficient
being loop-invariant there.
Against ISA-L's NASM kernels this reaches parity on AVX-512 (+1%
to -5%) and matches or beats NASM on 4 of 5 AVX2 benchmarks, the
one slower case by 6%.

A configure probe (EC_GFNI_SUPPORT) compile-tests the intrinsic
and defines LUSTRE_EC_GFNI for the libec.a build only.  It rejects
GCC 8 and 9, which mark GF2P8AFFINEQB commutative and can swap its
data and matrix operands, giving wrong parity.  Every
dispatch block is guarded by the pair defined(LUSTRE_EC_GFNI) &&
!defined(__KERNEL__), so no shipped consumer reaches a GFNI kernel
yet: ec.ko and liblustreapi both take the base path, and the
numbers above compare kernels directly rather than describing a
user-visible speedup.  Follow-on patches add the Kbuild plumbing
and the liblustreapi wiring.

gfni_ec_test checks that the GFNI and base paths produce identical
output, and exits 77 without GFNI so a caller can skip instead of
recording a base-against-base pass.  It installs with the test
suite; sanity-ec.sh test_0 sweeps geometries and lengths through
it.  -i picks the kernel family directly, because the runtime
dispatch always prefers AVX-512 and would otherwise leave the
AVX2 kernels untested on any CPU that has both.

The vendored gf_*_gfni.c kernels are reflowed for lustre
checkpatch; see lustre/utils/erasurecode/GFNI_VENDORED.md.

Test-Parameters: trivial testlist=sanity-ec
Signed-off-by: Hiroshi Nishida <hnishida@thelustrecollective.com>
Assisted-by: Claude:Opus-5 claude-code
Change-Id: Ia40a743c8e47c018250728668090215c7bd8365b
Signed-off-by: Patrickbot <patrick-bot@mulberrytree.us>

69350 master J –B ✓T ✓R +2/2

LU-12669 ec: create lock placeholder for reconstructed pages

reviewing Andreas Dilger · CI green — awaiting your review

pt_ecro218_review_priority master-next 4 unresolved L +532 −68

build #134472 SUCCESS · tests all Maloo sessions

reviews: Keguang Xu +1 2026-10-29 00:03 · Zhenyu Xu +1 2026-10-29 00:41

no vote yet: Patrick Farrell

patchsets:

+2 earlier

PS1 10-01 PS2 10-01

PS3 10-13 PS4 10-13 PS5 10-14 PS6 10-14 PS7 10-26

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS6 2026-10-17 13:31 Gerrit AI review for Lustre ldlm_request.c:583 — (minor) If the patch is refreshed: this says matchers must skip LDLM_FL_PLACEHOLDER unless they only test page coverage, but osc_lock_to_pla

Gerrit AI review for Lustre PS6 · 2026-10-17 13:31

(minor) If the patch is refreshed: this says matchers must skip LDLM_FL_PLACEHOLDER unless they only test page coverage, but osc_lock_to_placeholder() matches with skip_flags 0 and without LDLM_FL_TEST_LOCK, taking a reader reference so it can reuse an existing placeholder:

    mode = ldlm_lock_match_with_skip(ns, LDLM_FL_BLOCK_GRANTED, 0, ...
                                     LCK_PR, LDLM_MATCH_AST, &lockh);

The same "only matched by page coverage tests" wording appears in the osc_lock_to_placeholder() comment and in the commit message. Maybe mention placeholder reuse as the second allowed match, so a future caller doesn't treat the reuse match as a contract violation, or copy it as a precedent for a generic match?

reply PS2 2026-10-05 06:31 Gerrit AI review for Lustre vvp_io.c:494 — (minor) If the patch is refreshed: this comment says the LOV layer uses a placeholder for these mmap-buffer locks, but vvp_mode_from_vma() r

Gerrit AI review for Lustre PS2 · 2026-10-05 06:31

(minor) If the patch is refreshed: this comment says the LOV layer uses a placeholder for these mmap-buffer locks, but vvp_mode_from_vma() returns CLM_WRITE for a shared writable mapping, and osc_lock_to_placeholder() refuses anything but LCK_PR. For such a buffer the stripe is only tolerated, and the reconstructed pages are dropped after the read through ec_unprotected_recovery. Maybe say "a placeholder for a read-mode lock"?

reply PS7 2026-10-28 23:59 Andreas Dilger vvp_io.c:1583 — Right, the kept pages never leave cl_lru_busy. This is fixed by https://review.whamcloud.com/69908 ("LU-12669 ec: fix degraded read page le

Gerrit AI review for Lustre PS7 · 2026-10-27 13:16

(defect) Do the pages kept here ever reach the OSC LRU list?

A reconstructed page is created by cl_page_find(CPT_CACHEABLE) in lov_ec_read_stripe_pages(), so osc_page_init()->osc_lru_alloc() reserves a cache slot for it (ops_in_lru = 1, cl_lru_busy++). A page only moves from "busy" onto cl_lru_list in osc_extent_finish()->osc_lru_add_batch(), after a BRW. A rebuilt page never goes through a BRW, so it stays counted in cl_lru_busy and takes a ccc_lru_left slot. osc_lru_shrink() only scans cl_lru_list, so it cannot reclaim the page.

Before this change, cl_page_delete() here always returned the slot through osc_page_delete()->osc_lru_del(). Now the slot is held until the placeholder (or the granted lock) is cancelled, or until kernel memory reclaim calls ll_release_folio().

A large degraded read can pin max_cached_mb this way, e.g. one dead stripe of a 4+2 file several times larger than the cache. osc_lru_alloc() callers for any file on the mount then sleep in l_wait_event_abortable(osc_lru_waitq, cl_lru_left > 0), and readahead gets -EBUSY. This lasts until the placeholder ages out of the ldlm LRU.

Should the kept pages be added to the client LRU when they are marked Uptodate, like osc_lru_add_batch() does after a read?

Andreas Dilger PS7 · 2026-10-28 23:59

Right, the kept pages never leave cl_lru_busy.  This is fixed by https://review.whamcloud.com/69908 ("LU-12669 ec: fix degraded read page leak and short read"), which adds the kept pages to the client LRU with cl_page_lru_add().  That fix will be merged into this patch the next time it is refreshed.

reply PS2 2026-10-05 06:31 Gerrit AI review for Lustre osc_lock.c:1268 — (suggestion) This isn't a bug, but exporting ldlm_lock_create() and ldlm_lock_enqueue() means osc now builds an ldlm_lock by hand: it sets l

Gerrit AI review for Lustre PS2 · 2026-10-05 06:31

(suggestion) This isn't a bug, but exporting ldlm_lock_create() and ldlm_lock_enqueue() means osc now builds an ldlm_lock by hand: it sets l_conn_export, l_flags, l_policy_data and l_req_extent, and takes the addref before the grant. ldlm_cli_enqueue_local() does the same steps but asserts on a client namespace. Would a small exported client-side helper in ldlm (or a relaxed ldlm_cli_enqueue_local()) keep these lock-construction details inside ldlm, instead of making the two internals public API?

tags: pt_ecro 218_review_priority

owner Andreas Dilger · uploader Andreas Dilger · PS7 uploaded 6d ago · NEW · open in Gerrit ↗

commit message

LU-12669 ec: create lock placeholder for reconstructed pages

Pages that CIT_EC_RD reconstructs from parity for a stripe on an
unavailable OST are not covered by a DLM lock, so they are dropped
after each read, and every later read rebuilds them from parity
again.

Grant a client-only LDLM_FL_PLACEHOLDER ldlm_lock over the whole
stripe object locally, without an RPC, and keep the reconstructed
pages Uptodate under it.  A stripe gets a placeholder when its target
is inactive, or when its sub-lock enqueue fails.  If the conversion
fails, drop the pages after the read as before.  Add
cl_lock_operations::clo_to_placeholder() and
ldlm_cli_enqueue_placeholder() for this.

The pages stay valid while the parity they were rebuilt from is not
STALE.  Any write or truncate of the file must stale the parity with
a layout write intent first, which revokes the layout lock on every
client.  When a client loses the layout lock, cancel the placeholder
locks of the file with LCF_PLACEHOLDER, which discards their pages
under the page lock as a server lock cancel does, and cancel one that
a read creates meanwhile once that read is done.  Fast reads from
the page cache already wait for the next IO to refresh the layout,
so make the mmap fast fault do the same.  So the placeholder needs no
server state: it is neither replayed nor dropped before a lock
replay, is only matched by page coverage tests, does not hold up the
KMS, and otherwise leaves the cache through the LRU like any unused
lock.  Only PR locks are converted, since a PW placeholder could
satisfy a later write without a server lock.

Also cancel initialized sub-locks on the lov_lock_sub_init() error
path, and in lov_lock_cancel() when the sub-lock env is not found, so
a placeholder reference is not leaked, and free the lov_lock with the
size it was allocated with.  Drop lov_lock_sub::sub_initialized, as
every sub-lock is now initialized.

Add sanity-ec test_49f to check that a second degraded read sends no
read RPCs, that the placeholder survives OST recovery, and that the
cached pages do not hide a later write from another client from
either read() or an existing mmap.

Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: Augment Agent:DeepSeek V4.1 Flash
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: Idbad18031562d5cef02ab16aadcd4818ec3ebbe5

67069 master J –B ✓T ✓R +2/2

LU-20451 llapi: keep existing file on restricted setstripe

reviewing Sohei Koyama · CI green — awaiting your review

218_review_prioritypw_shepherd218_review_blocker CI@PS9 master-next 11 unresolved L +263 −96

build #134604 SUCCESS · tests all Maloo sessions

reviews: Robert Read +1 2026-11-02 04:46 · Andreas Dilger +1 2026-11-02 04:46

no vote yet: Xiao Yang, Maximilian Dilger

patchsets:

+5 earlier

PS1 07-06 PS2 08-03 PS3 08-19 PS4 09-29 PS5 09-29

PS6 10-15 PS7 10-21 patrick bot PS8 10-28 PS9 10-28 PS10 11-02

threads: 11 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:3952 — (typo) this looks like it is creating a *non-existent* file?

Andreas Dilger PS9 · 2026-10-30 02:43

(typo) this looks like it is creating a *non-existent* file?

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:3958 — (style) prefer `[[...]]` for string comparisons in bash

Andreas Dilger PS9 · 2026-10-30 02:43

(style) prefer `[[...]]` for string comparisons in bash

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:3960 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:3970 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:3984 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:3996 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:3998 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:4006 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:4023 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:4044 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

reply PS9 2026-10-30 02:43 Andreas Dilger sanity.sh:4061 — ...

Andreas Dilger PS9 · 2026-10-30 02:43

...

tags: 218_review_priority pw_shepherd 218_review_blocker

build/test results are from PS9 — PS10…PS10 changed only the commit message (votes copied, CI not re-run)

owner Sohei Koyama · uploader Sohei Koyama · PS10 uploaded 11h ago · NEW · open in Gerrit ↗

commit message

LU-20451 llapi: keep existing file on restricted setstripe

Previously, when setstripe was restricted, the LL_IOC_LOV_SETSTRIPE
ioctl returned -EACCES and llapi_file_open_param() and
llapi_file_create_foreign() used to remove the file and retry with
the default layout, potentially deleting an existing file.

The restriction is checked in the client ioctl and the remove and
retry are done by liblustreapi, so the fix is in llapi.

Create with O_EXCL and only remove a file that was newly created by
the command itself.  Leave an existing file untouched and return
-EEXIST, as unrestricted setstripe does.  For a dangling symlink,
remove the created target only while it has no layout.

Restricted 'lfs migrate' now uses the default layout instead of
failing with "cannot create volatile file": remove() ignores ENOENT
and the warning omits the volatile name.  Share this handling
between both functions in one helper.

Add sanity test_27W.

Assisted-by: ClaudeCode:Opus-5.5
Fixes: 5b99b881c412 ("LU-17628 lfs: add lfs_setstripe admin restrict")
Signed-off-by: Sohei Koyama <skoyama@ddn.com>
Change-Id: I3f9b20d27b4cae1b714136e38a525e7d81c4df24

69493 master J –B ✓T ✓R +1/2

LU-19823 lod: unique failure domains per EC raid set

reviewing Andreas Dilger · CI green — awaiting your review

pt_ecro218_review_blocker218_review_priority L +464 −119

10h

build #134652 SUCCESS · tests all Maloo sessions

reviews: Keguang Xu +1 2026-10-29 05:45

no vote yet: Ronnie Sahlberg, kg.xu, Oleg Drokin

patchsets: PS1 10-07 PS2 10-13 PS3 10-24 PS4 10-27 PS5 10-28

tags: pt_ecro 218_review_blocker 218_review_priority

owner Andreas Dilger · uploader Andreas Dilger · PS5 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-19823 lod: unique failure domains per EC raid set

With "lfs setstripe -c N --ec K+M" the allocator required every data
and parity stripe of the component to be in a different failure
domain, while with -C this was only needed within each raid set.
A wide file such as "-c 40 --ec 8+2" on 40 OSTs needs 10 domains
per raid set but 50 for the component, so it failed with ENOSPC
unless the file was overstriped.

Each raid set is protected on its own, so require different failure
domains only within each raid set, for both -c and -C.  With -c the
data stripes are still on different OSTs, so the file can use the
bandwidth of all OSTs, and parity shares OSTs with the data of other
raid sets.  A -c component short of OSTs is split into raid sets
again after its stripes are placed, so keep each run of consecutive
data stripes as long as the largest possible raid set in different
domains.  QoS no longer trims the data stripes of a -c component with
several raid sets to leave separate OSTs for its parity.

Each raid set still needs its data and parity stripes in different
failure domains, but the data stripes of a -c component need fewer, so
with too few domains the data could be placed and the RR allocator
then accepted a parity component with too few stripes.  Check the
domains before placing the data, and use raid sets of fewer data
stripes, down to K - K/4 as in lod_stripe_count_min(), or fail with
ENOSPC.  A single raid set gets fewer data stripes, so "-c 3 --ec 2+1"
with 3 domains becomes "-c 2 --ec 2+1".  Also fail a parity component
that RR could not place completely, as QoS does, since other mirrors
can use up domains that this check does not see.

An EC file is also two mirrors, so the FLR check required different
domains for all stripes of the data component too.  Leave the domains
within an EC component to the EC check.

Also fix lod_comp_find_linked() matching a plain mirror component to
the parity of an EC mirror, which let the plain mirror share a failure
domain with that parity, and restore the count of OSTs available to
mirrors when QoS gives up, so the RR fallback can place EC stripes.

Rename lod_fd_check_*() to lod_fdom_check_*() so "fd" is not read as a
file descriptor.  Describe the mirror rules in
obdfilter.failure_domain.4 and reference it from lfs-setstripe.1,
lfs-mirror-create.1 and lfs-mirror-extend.1.

Fixes: 026de09ead3d ("LU-19823 lod: Make the stripe allocators failure domain aware.")
Fixes: c53abca9f9ee ("LU-19852 lod: raidset aware stripe allocator")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: Ic197df8fba1a9e0a313442cc435d1959763ebbe5

69536 master J –B ✓T ✓R +1/2

LU-19566 lfsck: read a filter_fid longer than ours

reviewing Patrick Farrell · CI green — awaiting your review

pt_ecropw_own M +108 −17

14h

build #135060 SUCCESS · tests all Maloo sessions

reviews: Alex Zhuravlev +1 2026-11-01 05:37

no vote yet: Lai Siyao, Andreas Dilger, Hongchao Zhang

patchsets: PS1 10-08 PS2 10-09 patrick bot PS3 10-09 patrick bot PS4 10-10 patrick bot PS5 11-01 patrick bot

tags: pt_ecro pw_own

owner Patrick Farrell · uploader patrick bot · PS5 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-19566 lfsck: read a filter_fid longer than ours

A newer server will store a longer filter_fid on OST
objects. LFSCK and LOD read it into a buffer of exactly
sizeof(struct filter_fid) and fail with -ERANGE, so after a
downgrade, or with an MDT older than its OSTs, layout LFSCK
fails on those objects and MDT migration of those files
fails. The OST-side orphan iterator compared that error with
a size_t, so it took -ERANGE for a valid length and rebuilt
the orphan under a new FID, as if it had no parent. An OST
that keeps the parent FID in LMA refused to have a longer
filter_fid written to it.

Read filter_fid into a buffer FILTER_FID_EXTRA_SIZE (32)
bytes larger, as OFD already does after -ERANGE, compare the
iterator's length as signed, write back only the filter_fid
this version knows when LOD replaces the parent FID, and
keep the start of a longer filter_fid in LMA. sanity-lfsck
test_46 grows two OST objects' filter_fid and checks that
LFSCK neither fails on them nor loses an orphan's FID,
and MDT migration preserves the file data.

This is for the maintenance branches, ahead of the
filter_fid growth in https://review.whamcloud.com/62389.

Fixes: db5661e8aad3 ("LU-8998 pfl: enhance PFID EA for PFL")
Signed-off-by: Patrick Farrell <patrick@thelustrecollective.com>
Assisted-by: Codex:gpt-6.1-sol
Change-Id: Iebd099c0307239c1f8af5477c35c3a8521eb16db

69697 master J –B ✓T ✓R +1/2

LU-20967 osd-ldiskfs: name the failure_domain local file

reviewing Andreas Dilger · CI green — awaiting your review

218_review_blocker218_review_prioritypt_ecro 3 unresolved L +320 −5

15h

build #134528 SUCCESS · tests all Maloo sessions

reviews: Chris Walker +1 2026-10-29 11:48

no vote yet: Ronnie Sahlberg, Alex Zhuravlev

patchsets:

+2 earlier

PS1 10-15 PS2 10-21 Marc Vef

PS3 10-22 PS4 10-26 PS5 10-27 PS6 10-27 PS7 10-27

threads: 3 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS7 2026-10-27 23:17 Gerrit AI review for Lustre COMMIT_MSG:41 — (style) This isn't a bug, but contrib/git-hooks/commit-msg rejects this trailer ("commit message invalid signoff section line"), because Co-

Gerrit AI review for Lustre PS7 · 2026-10-27 23:17

(style) This isn't a bug, but contrib/git-hooks/commit-msg rejects this trailer ("commit message invalid signoff section line"), because Co-authored-by is not in its accepted tag list (Acked-by, Tested-by, Reported-by, Reviewed-by, Suggested-by, CC). A second Signed-off-by: line for the co-author would pass the hook, if the patch is refreshed.

reply PS1 2026-10-26 22:39 Andreas Dilger osd_scrub.c:2530 — The relink should check the mode on the inode and insert the dirent with the correct type.

Gerrit AI review for Lustre PS1 · 2026-10-17 15:14

(defect) This loop runs for every osd_lf_maps[] entry with a non-zero FID, not only OLF_SHOW_NAME regular files as the osd_lf_relink() comment says. Two side effects:

- For entries without OLF_SHOW_NAME (lost+found/OSD_LPF_OID, index_backup/INDEX_BACKUP_OID), osd_lf_fid2name() returns "". osd_obj_spec_insert() then returns 0 without inserting anything, so osd_lf_relink() returns true. The stale-mapping deletion is silently skipped, and a journal handle is started for nothing.
- update_log_dir is OLF_SHOW_NAME, and on MDT0000 its FID from lu_update_log_dir_fid(fid, 0) equals the map FID {FID_SEQ_UPDATE_LOG_DIR, 0, 0}. If its name is lost, it gets relinked through osd_obj_add_entry(), which hardcodes `i_mode = S_IFREG`. That produces a regular-file dirent for a directory. If e2fsck pass 3 already reconnected it to /lost+found, it also produces a hard link to a directory.

Should the relink be restricted to `(map->olm_flags & OLF_SHOW_NAME) && !(map->olm_flags & (OLF_SCAN_SUBITEMS | OLF_IDX_IN_FID))`, or should the inode be checked for S_ISREG before relinking?

Andreas Dilger PS1 · 2026-10-26 22:39

The relink should check the mode on the inode and insert the dirent with the correct type.

reply PS3 2026-10-26 22:39 Andreas Dilger osd_scrub.c:2476 — Seems reasonable to fix.

Gerrit AI review for Lustre PS3 · 2026-10-26 13:23

(minor) The kernel-doc says %false means "no such object", but osd_lf_relink() also returns false when the object does exist and the relink just failed:

    osd_oi_lookup() -> -EIO / -ENOMEM
    osd_iget() -> -ENOMEM / -EIO (not only -ENOENT/-ESTALE)
    osd_journal_start_sb() -> error
    osd_obj_spec_insert() -> -ENOSPC, etc.

In each case osd_initial_OI_scrub() then calls osd_scrub_refresh_mapping(DTO_INDEX_DELETE) and drops the OI mapping of an inode that still exists. That leaves last_rcvd, failure_domain, etc. orphaned, which is the outcome this function is meant to prevent. The old code dropped the mapping in these cases too, so this isn't new breakage.

If the patch is refreshed, consider returning an int and dropping the mapping only for -ENOENT/-ESTALE, or at least correcting the Return: description.

Andreas Dilger PS3 · 2026-10-26 22:39

Seems reasonable to fix.

tags: 218_review_blocker 218_review_priority pt_ecro

owner Andreas Dilger · uploader Andreas Dilger · PS7 uploaded 6d ago · NEW · open in Gerrit ↗

commit message

LU-20967 osd-ldiskfs: name the failure_domain local file

The failure_domain file added in LU-19066 has no entry in the
osd_lf_maps[] table in osd_scrub.c, so osd_lf_fid2name() returns no
name for its FID and osd_obj_spec_insert() creates no directory entry
for it in the OST device root.  e2fsck then treats the inode as
disconnected and wants to reconnect it into lost+found:

  Pass 4: Checking reference counts
  Unattached inode 88
  Connect to /lost+found? no
  myfs-OST0003: ***** WARNING: Filesystem still has errors *****

Add a FAILURE_DOMAIN entry to osd_lf_maps[] with OLF_SHOW_NAME, like
health_check, so the file is linked into the device root under its own
name and e2fsck no longer reports an error.

Existing targets already have the unnamed inode, and the same can
happen to any named local file whose name is lost to directory
corruption that e2fsck repaired.  The initial OI scrub already walks
osd_lf_maps[] to drop stale OI mappings for files that no longer
exist; make it instead relink a file that still has a valid OI mapping
and inode but no name in the root, so such files are recovered on
mount rather than left unattached.  The relink is restricted to plain
named regular-file entries and keeps the OI mapping on a transient
failure, so an inode that still exists is never orphaned.

When e2fsck has already reconnected such a file into lost+found under
a numeric name, the OI scrub now renames it back out of lost+found to
its proper name during the lost+found scan, moving the single name
rather than adding a second one, so the link count stays correct.

Fixes: 612732b59482 ("LU-19066 obdfilter: Create a failure_domain file")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Co-authored-by: Chris Walker <chris.walker@netapp.com>
Assisted-by: ClaudeCode:claude-opus-4.8 ltvm,llm_code_and_review_tools
Change-Id: Idf1c415213059670f4fbac1b708526acc73ebbe5

69426 master J –B ✓T ✓R +1/2

LU-19823 lod: spread failover groups over the RR array

reviewing Andreas Dilger · CI green — awaiting your review

pt_ecro218_review_priority 5 unresolved M +203 −11

19h

build #135076 SUCCESS · tests all Maloo sessions

reviews: Ronnie Sahlberg +1 2026-11-01 07:57

no vote yet: Keguang Xu

patchsets:

+1 earlier

PS1 10-05

PS2 10-06 PS3 10-07 PS4 10-10 PS5 10-13 PS6 11-01

threads: 5 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-10-08 12:07 Gerrit AI review for Lustre lu_tgt_alloc.h:26 — (nit) ltd_rr_failover_order() stores lu_rr_group_order() + 1 here, so 0 still means "not set". Maybe "failure domain, or failover group orde

Gerrit AI review for Lustre PS3 · 2026-10-08 12:07

(nit) ltd_rr_failover_order() stores lu_rr_group_order() + 1 here, so 0 still means "not set". Maybe "failure domain, or failover group order + 1, 0 if not set"?

reply PS3 2026-10-08 12:07 Gerrit AI review for Lustre lod_qos.c:294 — (minor) If the patch is refreshed, could this kernel-doc also say that, without failure domains, the targets are ordered by failover group (

Gerrit AI review for Lustre PS3 · 2026-10-08 12:07

(minor) If the patch is refreshed, could this kernel-doc also say that, without failure domains, the targets are ordered by failover group (ltd_rr_failover_order())? It would also help to note that a change to the import's connection list doesn't trigger a resort and only takes effect at the next rebuild for another reason.

reply PS3 2026-10-24 20:10 Andreas Dilger lod_qos.c:440 — This should be a very uncommon config. I don't want to make the code increasingly complex to handle theoretical corner cases. 99% of systems

Gerrit AI review for Lustre PS3 · 2026-10-08 12:07

(minor) With failover groups set as lrt_fd, lu_rr_fill() takes the by_fd path. In that path a server's targets are only spaced within their group, not across the whole array. When the OSSs of a pair have different OST counts, this can put two OSTs of the same OSS next to each other where the old by-server order did not.

I ran lu_rr_fill() in userspace (the lod_rr_test harness) with 6 OSSs in 3 pairs (oss/2) and OST counts 2,3 | 2,4 | 3,4 (18 OSTs). OSS sequence of the array:

    old: 3 5 1 4 0 2 3 5 1 4 3 5 0 2 1 4 3 5
    new: 5 3 1 4 2 0 5 3 1 4 3 5 0 2 4 1 3 5

The old order has no adjacent same-OSS slots and one same-pair slot. The new order has one adjacent pair of slots, and it is on a single OSS (5 at the end wraps to 5 at the start). Over 4-OSS configs with 1-8 OSTs per OSS, 972 of 4096 have more same-OSS adjacencies than before. The smallest is 2,4 | 4,3, where the old order had none. Equal pairs, and pairs that differ by one OST (e.g. one inactive OST), are fine.

Is that trade-off intended? If not, one option is to use the failover order only when it doesn't increase same-server adjacency, or to keep server spacing as the primary key and use the group only to break ties. Either way, it would help to mention it in the commit message.

Andreas Dilger PS3 · 2026-10-24 20:10

This should be a very uncommon config. I don't want to make the code increasingly complex to handle theoretical corner cases. 99% of systems will have totally uniform OSS nodes. A fraction will have maybe 2-3 different OSS configs, likely in different pools that would not be in the same allocation.

reply PS4 2026-10-24 20:10 Andreas Dilger lod_qos.c:443 — I don't think this is _worse_ than before the patch. There are more patches in this series, and it could be further improved in a later pat

Gerrit AI review for Lustre PS4 · 2026-10-11 01:49

(defect) When any OST has ltq_failure_domain set, fdoms is allocated, and lu_rr_fdoms_init() now makes one lu_rr_fdom per inferred failover group instead of one per server. For FLR and EC layouts (fdom_rr in lod_ost_alloc_rr()), each slot of such a group now hands out the targets of both OSSs of the pair in turn from the shared lrf_next/lrf_next0 counters, and the lod_oss_avoided() retries skip through the pair's targets. Is that intended? The commit message says this "only orders the array, and does not limit where the stripes of a layout or its mirrors may go", but it also changes which OST each slot gives.

I checked this with lod_rr_test changed so that -u leaves the first half of the OSSs without a domain (pairs 0/1, 2/3, ... as with -F), and compared runs with and without -F:

    lod_rr_test -o 10 -s 4 -d 4 -u -c 2 -M -F
      OST1 has 560 objects, OST2 has 81
      OST0 has 300 #0 objects, OST2 has 1
    (without -F: 467 vs 366, and 200 #0 objects on each OST)

Over -o 8-64 -s 4-16 -d 2-8 -u -c 1-4:
- with -M, failed combinations go from 360 to 980 of 9399, and layouts with two stripes on one OSS from 1233 to 65633
- with -e 2, failed combinations go from 305 to 969 of 10467, and layouts with two stripes on one OSS from 100 to 116724

One option is to keep a separate lu_rr_fdom per server for the inferred groups and use the group only to order the array. Another is to skip ltd_rr_failover_order() when the array has explicit failure domains.

Andreas Dilger PS4 · 2026-10-24 20:10

I don't think this is _worse_ than before the patch.  There are more patches in this series, and it could be further improved in a later patch.

reply PS4 2026-10-24 20:10 Andreas Dilger sanity.sh:7000 — Can be done in a later patch. Mixed fdom and non-fdom configs _should_ be rare, but it should try to do a reasonable job if that happens. We

Gerrit AI review for Lustre PS4 · 2026-10-11 01:49

(suggestion) With -u, only OSS 0 has no domain, so its inferred group holds just that one OSS. This case doesn't cover an inferred group of two OSSs next to explicit domains, which is the case lrrt_inferred exists for. The -U line below covers it only with -c 1, without mirrors or parity. Could a case with an HA pair that has no domain, along with -M and -e, be added? That would also cover the lu_rr_fdom_tgt() imbalance described in lod_qos.c.

Andreas Dilger PS4 · 2026-10-24 20:10

Can be done in a later patch. Mixed fdom and non-fdom configs _should_ be rare, but it should try to do a reasonable job if that happens. We can refine this further, but it is not the common case.

tags: pt_ecro 218_review_priority

owner Andreas Dilger · uploader Andreas Dilger · PS6 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-19823 lod: spread failover groups over the RR array

Without failure domains, the round-robin array only keeps the targets
of each server apart.  The targets of the two servers of an HA pair
can still be next to each other in the array, so consecutive stripes
of a file are on servers that can fail together, or be unavailable
together while one of them fails over to the other.

For the targets without a failure domain, ltd_rr_failover_order()
groups them by the lowest NID of their primary and failover
connections, which is the same for all the targets that can fail over
to each other, and orders the array by group as for failure domains.
lrrt_inferred keeps these groups apart from explicit failure domains
with the same number, also when only some targets have a domain.
All the targets of a server go in the group of its lowest NID, since
the array is ordered only by server if a server is in several groups.
The groups are ranked by NID, first by network and then by address,
and lu_rr_group_order() orders them by the reversed bits of their
rank, so groups with close NIDs, likely in the same network or rack,
are mostly far apart in the array.  Unlike failure domains, this does
not limit where the stripes of a layout or its mirrors may go, but
when other OSTs have failure domains, a slot of a failover group gives
the OSTs of both servers of the group in turn to EC and FLR layouts.
The MDT round-robin array, used for striped directories, is ordered
the same way.  The failover NIDs are read from the import at each
rebuild of the array, so add_conn and del_conn take effect at the next
rebuild.

Add a -F option to lod_rr_test to put the OSSs without a failure
domain in HA pairs, and check that consecutive OSTs in the array are
on different pairs, and a -U option to leave every other OST without a
failure domain.  Without the ordering, "lod_rr_test -f -o 4-64 -s 2-16
-F" finds OSTs of one HA pair next to each other in 1852 of 30512
combinations, and in none with it.

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: I39eae90881b61feee2e9b2aa20978a38e93ebbe5

69768 master J –B ✓T ✓R 0/2

LU-11962 flr: fix the handling of foreign mirrors

reviewing Andreas Dilger · CI green — awaiting your review

pccropt_ecro 7 unresolved M +161 −13

21h

build #135063 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Qian Yingjin, Bruno Faccini

patchsets: PS2 10-18 PS3 10-19 PS4 10-21 Marc Vef PS5 10-31 PS6 11-01

threads: 7 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS5 2026-11-01 03:22 Gerrit AI review for Lustre COMMIT_MSG:7 — (minor) LU-11962 is "File LSOM updates to store proper size via FLR for regular stat() usage" (an Improvement, currently Reopened). This pat

Gerrit AI review for Lustre PS5 · 2026-11-01 03:22

(minor) LU-11962 is "File LSOM updates to store proper size via FLR for regular stat() usage" (an Improvement, currently Reopened). This patch fixes foreign/HSM mirror handling for PCC-RO, FLR and EC. Is this the intended ticket, or would a dedicated LU ticket for the foreign mirror bugs make them easier to find and backport?

reply PS5 2026-11-01 03:22 Gerrit AI review for Lustre COMMIT_MSG:27 — (minor) The hsm_flags2str() change isn't described here. It changes `lfs getstripe -v` output: before, the full value was always printed aft

Gerrit AI review for Lustre PS5 · 2026-11-01 03:22

(minor) The hsm_flags2str() change isn't described here. It changes `lfs getstripe -v` output: before, the full value was always printed after the names (e.g. "exists,archived,pccro,0x109"), and now only the bits without a name are printed. If the patch is refreshed, could this get an "Also fix ..." sentence?

reply PS2 2026-10-19 03:00 Gerrit AI review for Lustre lfs.c:2688 — (minor) If the patch is refreshed, this description could mention foreign mirrors too. data_mirror_remains() now also skips them, but the co

Gerrit AI review for Lustre PS2 · 2026-10-19 03:00

(minor) If the patch is refreshed, this description could mention foreign mirrors too. data_mirror_remains() now also skips them, but the comment still says only parity mirrors are excluded. The same applies to the "the parity mirrors left behind cannot serve reads" comment above the caller in mirror_split().

reply PS2 2026-10-19 03:00 Gerrit AI review for Lustre lfs.c:2911 — (minor) This refusal is user-visible, but Documentation/man1/lfs-mirror-split.1 still says a data mirror split is refused only because "a fi

Gerrit AI review for Lustre PS2 · 2026-10-19 03:00

(minor) This refusal is user-visible, but Documentation/man1/lfs-mirror-split.1 still says a data mirror split is refused only because "a file left holding parity mirrors alone cannot be read". If the patch is refreshed, could the man page mention foreign (e.g. PCC-RO/HSM) mirrors as well?

reply PS4 2026-10-24 04:22 Andreas Dilger lfs.c:3448 — This is a bit of a grey area. HSM itself is useful only when the local mirror is released and the HSM copy is the only one. However, we don'

Gerrit AI review for Lustre PS4 · 2026-10-23 03:29

(suggestion) last_non_stale_mirror() has the same problem that data_mirror_remains() is fixed for here. A non-stale foreign HSM mirror has no LCME_FL_STALE component, so llapi_mirror_find_stale() returns 0 for it and this function returns false. lfs then skips the mandatory resync before mirror_split().

One way to get there: mirrors A and B, a write leaves B stale (WRITE_PENDING), and then `lfs pcc attach -r`. lod_declare_pccro_set() does not check the FLR state and adds a non-stale HSM mirror. After that, `lfs mirror split -d --mirror-id A` passes data_mirror_remains() because B remains. The resync is skipped because of the HSM mirror, and the only in-sync copy of the data is destroyed.

This predates the patch, but it is the same "foreign mirror counted as data" case. Should foreign mirrors be skipped here too (`is_parity_mirror() || is_foreign_mirror()`), either in this patch or in a follow-up?

Andreas Dilger PS4 · 2026-10-24 04:22

This is a bit of a grey area. HSM itself is useful only when the local mirror is released and the HSM copy is the only one. However, we don't want that to happen by accident either, so it shouldn't be possible to split the last good mirror if there is a stale non-HSM mirror, since that is likely an operational error and would require restoring from HSM.  Probably the last-good mirror removal should only happen with `lfs hsm release` instead of `lfs mirror split` accidentally matching a good mirror.

reply PS5 2026-11-01 03:22 Gerrit AI review for Lustre lfs.c:18051 — (suggestion) This stops the chunk comparison from reading the foreign mirror, but two other `lfs mirror verify` paths still call llapi_mirro

Gerrit AI review for Lustre PS5 · 2026-11-01 03:22

(suggestion) This stops the chunk comparison from reading the foreign mirror, but two other `lfs mirror verify` paths still call llapi_mirror_read() on it:

- print_stale_mirrors() reads every LCME_FL_STALE component's mirror. A write to a file that still has the HSM mirror (it stays after `lfs pcc detach`, since lod_declare_pccro_clear() only clears HS_PCCRO) goes through lod_declare_update_rdonly()->lod_stale_components(), which sets LCME_FL_STALE and HS_DIRTY on the HSM component. Plain `lfs mirror verify` would then print "Mirror N: (stale - read error)" for it.
- With `--stale`, auto_mark_stale_mirrors()->compare_mirror_checksums() reads every mirror returned by get_mirror_properties(), foreign mirrors included. The new -EINVAL from lov_io_mirror_init() makes the auto-mark fail with "error reading from mirror N".

Both paths crashed the client on the LASSERT before this patch, so they are pre-existing and now fail cleanly. Should they skip foreign mirrors too, here or in a follow-up?

reply PS4 2026-10-24 04:22 Andreas Dilger liblustreapi.c:2110 — Probably yes. While foreign/hsm mirrors will exist as part of the layout, they will have high cost to access and should not be considered "n

Gerrit AI review for Lustre PS4 · 2026-10-23 03:29

(suggestion) This fixes `-N`, but the HSM entry is still the one picked for the other single-field queries. print_last_init_comp() selects the last LCME_FL_INIT component. lod_striped_create() sets INIT on foreign components, so on a PCC-RO file that is the HSM entry:

- `lfs getstripe -c/-S/-i/-p` now print nothing (before, they printed the whole lhm_* block). Neither shows the data mirror's value.
- `lfs getstripe -L` still prints every lhm_* field, not just the pattern.

This is pre-existing and doesn't block the patch. Would it be better to skip foreign entries when lov_dump_comp_v1() picks the last initialized component, instead of (or as well as) this verbose check?

Andreas Dilger PS4 · 2026-10-24 04:22

Probably yes. While foreign/hsm mirrors will exist as part of the layout, they will have high cost to access and should not be considered "normal" mirrors in most cases.

tags: pccro pt_ecro

owner Andreas Dilger · uploader Andreas Dilger · PS6 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-11962 flr: fix the handling of foreign mirrors

Several places mishandle a mirror made of a foreign HSM component, as
PCC-RO adds:
- the client copies the EC stripe counts into every layout entry, but
  for a foreign entry they share a union with lsme_foreign_flags, so
  getxattr("lustre.lov") and llapi_layout_get_by_fd() report its HSM
  flags as 0;
- "lfs getstripe" prints all of its lhm_* fields whatever is asked,
  so "lfs getstripe -N" prints them after the mirror count, and it is
  the last initialized component that "-c", "-S" or "-i" show;
- lod_primary_pick() may pick it as the primary mirror for a write
  when the OSTs of the data mirror are unavailable;
- mdd_split_ea(), and data_mirror_remains() and
  last_non_stale_mirror() of lfs, count it as data, so "lfs mirror
  split" can remove the only data mirror, or the only in-sync one;
- a designated read of it, e.g. "lfs mirror read -N", which any user
  with read access can do, hits the LASSERT in
  lov_io_layout_at_mirror(), and "lfs mirror verify" reads from it.

Move the EC stripe counts out of the union, print an HSM component
only for the verbose and layout pattern output and never as the last
initialized one, never pick an HSM mirror in lod_primary_pick(), and
do not count foreign mirrors as data when splitting or verifying.
Refuse designated I/O to a foreign mirror in lov_io_mirror_init().

Add sanity-pcc test_51 to check the HSM flags, "lfs getstripe -N" and
"-c", the split of data mirrors, and the read and verify of a file
attached to PCC-RO.

Test-Parameters: testlist=sanity-pcc,sanity-flr
Fixes: 472e0f27d411 ("LU-19545 ec: tight binding parity mirror to data mirror")
Fixes: 22e9e7e8b264 ("LU-12187 llapi: EC layout creation support")
Fixes: ce98bfe5f726 ("LU-10499 pcc: add readonly mode for PCC")
Fixes: aebb405e32e4 ("LU-10499 pcc: use foreign layout for PCCRO on server side")
Fixes: 2e591389646c ("LU-19548 lfs: update mirror split for EC support")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: Idb5f277a3117a352a99cecdaefd6ee7ee89b5397

58253 master J –B ✓T ✓R 0/2

LU-17531 lfs: add 'mirror extend --no-resync' option

reviewing Zhenyu Xu · CI green — awaiting your review

pw_smallfixpt_ecro M +190 −5

22h

build #135058 SUCCESS · tests all Maloo sessions

no vote yet: Ronnie Sahlberg, Andreas Dilger, Jian Yu, Patrick Farrell

patchsets:

+19 earlier

PS1 02-27 PS2 02-27 PS3 02-28 PS4 03-01 PS5 03-03 PS6 03-03 PS7 03-04 PS8 03-05 PS9 03-06 PS10 04-17 PS11 05-05 Andreas Dilger PS12 05-05 Andreas Dilger PS13 05-05 Andreas Dilger PS14 05-05 Andreas Dilger PS15 06-01 PS16 08-12 PS17 10-14 patrick bot PS18 10-16 patrick bot PS19 10-27 patrick bot

PS20 10-28 patrick bot PS21 10-28 patrick bot PS22 10-30 patrick bot PS23 10-31 patrick bot PS24 11-01 patrick bot

tags: pw_smallfix pt_ecro

owner Zhenyu Xu · uploader patrick bot · PS24 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-17531 lfs: add 'mirror extend --no-resync' option

Allow adding a mirror without syncing its content
beforehand, so that it can be resynced later. The new
mirrors are marked stale and stay unusable until
"lfs mirror resync" is run. Merging a stale mirror needs FLR EC
enabled on both the client and the MDS (llite_enable_flr_ec and
mdt_enable_flr_ec, negotiated at mount), otherwise the extend is
refused.

--no-resync cannot be used with -f VICTIM_FILE or with --ec,
since the parity mirror is computed from up-to-date data.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I1b43fb0299c2aaea72587eff9168e4cdee075624

70273 master J –B ✓T ✓R 0/2

LU-20565 tests: use EC_* layouts in sanity-ec extend tests

reviewing Andreas Dilger · CI green — awaiting your review

testfix M +57 −54

build #134937 SUCCESS · tests all Maloo sessions

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: use EC_* layouts in sanity-ec extend tests

sanity-ec subtests 8a-8c and 8e-8l (lfs mirror extend --ec) hard-coded
an EC 4+2 layout on 4-stripe data mirrors with a gate of 6 OSTs,
although each tests a mirror extend rule, not the geometry.  They
would be skipped on the 4 OST test sessions.

Use the EC_ARG, EC_DSTRIPE and EC_CSTRIPE defaults that enable_ec()
scales to OSTCOUNT, and EC_DSTRIPE data stripes, so they run with 3
OSTs.  Put each PFL boundary in front of an EC component on a
multiple of the row width.

Test-Parameters: trivial testlist=sanity-ec
Test-Parameters: testlist=sanity-ec ostcount=4
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I48317fe17e7e538210970c657ffc55ba2f3ebbe5

70228 master J –B ✓T ✓R 0/2

LU-20923 mdt: validate metadata request fields before use

reviewing Andreas Dilger · CI green — awaiting your review

5 unresolved M +68 −2

build #134917 SUCCESS · tests all Maloo sessions

threads: 5 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 21:17 Gerrit AI review for Lustre COMMIT_MSG:27 — (minor) The dt_mode_to_dft() reasoning doesn't quite match the code. mdd_create_sanity_check() already returns -EINVAL for an unmappable S_I

Gerrit AI review for Lustre PS1 · 2026-10-31 21:17

(minor) The dt_mode_to_dft() reasoning doesn't quite match the code. mdd_create_sanity_check() already returns -EINVAL for an unmappable S_IFMT before dt_mode_to_dft() runs. For REINT_CREATE, mdt_reint_create() already rejects unsupported types, so that part is not a new check. What forcing S_IFREG in mdt_open_unpack() really prevents is an O_CREAT open creating a directory, device node or symlink (for S_IFLNK, sp_symname would alias the sp_ea eadata in the md_op_spec union). It would help to describe that instead.

reply PS1 2026-10-31 21:17 Gerrit AI review for Lustre COMMIT_MSG:35 — (minor) "previously left held" understates the result. A held mlh_rreg_lh in any mti_lh[] slot hits LASSERT(!lustre_handle_is_used(&lh->mlh_

Gerrit AI review for Lustre PS1 · 2026-10-31 21:17

(minor) "previously left held" understates the result. A held mlh_rreg_lh in any mti_lh[] slot hits LASSERT(!lustre_handle_is_used(&lh->mlh_rreg_lh)) in mdt_lock_handle_assert(), called from mdt_thread_info_fini() at the end of mdt_intent_policy(). So the old behaviour was an MDS assertion, not a lock leak. That is reachable through mdt_intent_lock_replace(), e.g. an intent getxattr or cross-ref getattr whose FID names an object on another MDT.

reply PS1 2026-10-31 21:17 Gerrit AI review for Lustre mdt_handler.c:5227 — (minor) Can mlh_rreg_lh actually be held on this exit? In mdt_getattr_name_lock(), a remote child in the by-name path gets a local LOOKUP lo

Gerrit AI review for Lustre PS1 · 2026-10-31 21:17

(minor) Can mlh_rreg_lh actually be held on this exit? In mdt_getattr_name_lock(), a remote child in the by-name path gets a local LOOKUP lock via mdt_object_lookup_lock(info, NULL, ...), held in mlh_reg_lh. A batched request returns -EREMOTE before locking. Only the mti_cross_ref branch can take a remote lock (when mti_object is remote), and it unlocks on every error and returns 0 with DISP_LOOKUP_POS on success. So this condition never seems to be true here. The new remote-lock release in mdt_intent_lock_replace() does cover the reachable case (cross-ref/getxattr on a remote FID, which has no reg lock). If this is only defensive, the comment should say so rather than claim the lock can be held here.

reply PS1 2026-10-31 21:17 Gerrit AI review for Lustre mdt_lib.c:1550 — (minor) Isn't this already covered? mdt_reint_create() has the same S_IFMT switch and returns err_serious(-EOPNOTSUPP) for any other type. B

Gerrit AI review for Lustre PS1 · 2026-10-31 21:17

(minor) Isn't this already covered? mdt_reint_create() has the same S_IFMT switch and returns err_serious(-EOPNOTSUPP) for any other type. Both the MDS_REINT and the intent-create paths reach it before mdt_create(). mdd_create_sanity_check() also rejects unknown types with -EINVAL before mdd_object_make_hint()->dt_mode_to_dft(). The only effects of this copy are the error code changing from -EOPNOTSUPP to -EINVAL and the generic "Can't unpack reint" console message in place of "Unsupported mode". If the patch is refreshed, consider dropping this hunk, or removing the check in mdt_reint_create() so only one remains.

reply PS1 2026-10-31 21:17 Gerrit AI review for Lustre mdt_open.c:1490 — (minor) This catches MDS_REINT/REINT_OPEN (format RQF_MDS_REINT_OPEN, via mdt_reint()) only when the request is handled normally. If the sam

Gerrit AI review for Lustre PS1 · 2026-10-31 21:17

(minor) This catches MDS_REINT/REINT_OPEN (format RQF_MDS_REINT_OPEN, via mdt_reint()) only when the request is handled normally. If the same request carries MSG_RESENT and its xid matches saved reply data, mdt_reint_internal() calls

    mdt_check_resent()->mdt_reconstruct()->mdt_reconstruct_open()

before mdt_reint_rec() runs. mdt_reconstruct_open() still has the identical LASSERT(pill->rc_fmt == &RQF_LDLM_INTENT_OPEN) and the RMF_DLM_REP fetch, so a client can still trigger the assertion. The assertion was there before this patch, but this hunk is meant to close exactly that hole.

No client sends REINT_OPEN through MDS_REINT (RQF_MDS_REINT_OPEN is only referenced by the mdt_reint() table and layout.c). Would it be simpler to reject REINT_OPEN in mdt_reint(), e.g. by dropping it from reint_fmts[] so mdt_reint_opcode() fails? That covers both paths at the entry point.

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20923 mdt: validate metadata request fields before use

Several MDT request-handling paths assumed the request format and the
operation encoded in the request agreed, and read format-specific
fields or mapped values without first checking that assumption.  Make
each path validate its inputs:

 - mdt_getxattr_pack_reply() serves both the intent getxattr format
   (no RMF_NAME, sized into RMF_EAVALS) and the plain MDS_GETXATTR
   format (RMF_NAME, sized into RMF_EADATA), selecting the operation
   from mbo_valid.  Require that the requested operation matches the
   fields the format provides - a name is fetched only when RMF_NAME
   is present, and OBD_MD_FLXATTRALL is requested exactly when the
   reply format carries RMF_EAVALS - and return -EPROTO otherwise.

 - mdt_reint_open() serves only the open intent.  Confirm the request
   uses RQF_LDLM_INTENT_OPEN, returning -EPROTO before fetching
   RMF_DLM_REP, rather than relying on the format.

 - mdt_create_unpack() and mdt_open_unpack() take the create mode from
   the request, and object creation maps its file type with
   dt_mode_to_dft().  Force S_IFREG for an O_CREAT open (which only
   ever creates a regular file, keeping the permission bits), and
   reject a REINT_CREATE whose mode does not name a supported file
   type with -EINVAL.

 - mdt_intent_getattr() and mdt_intent_lock_replace() hold a
   remote-object lock in mlh_rreg_lh.  Release that lock on the
   no-positive-disposition and no-local-lock exits, where it was
   previously left held, so it is dropped on those paths too.

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-4-8 ltvm,llm_code_and_review_tools
Change-Id: I6aa2f0306dfb2ef8739d353805d0653e023ebbe5

67651 master J –B ✓T ✓R -2/2−

LU-20541 lfs: Implement migrate_stripe command and add test

reviewing Ronnie Sahlberg · CI green — awaiting your review (needs rebase first)

needs rebase ec2 pt_ecro 11 unresolved XL +2004 −1

CI green — awaiting your review (needs rebase first)

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked

build #134057 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger -1 2026-10-20 07:01

no vote yet: Zhenyu Xu

patchsets:

+15 earlier

PS1 07-31 PS2 07-31 PS3 07-31 PS4 08-01 PS5 08-03 PS6 08-03 PS7 08-04 PS8 08-05 PS9 08-05 PS10 08-06 PS11 08-26 PS12 08-26 PS13 08-27 PS14 08-27 PS15 08-31

PS16 09-03 PS17 09-04 PS18 10-12 PS19 10-13 PS20 10-20

threads: 11 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS19 2026-10-16 10:50 Gerrit AI review for Lustre lfs-migrate_stripe.1:71 — (minor) lfs_migrate_stripe() also accepts `--src-ost-index` and `--dst-ost-index`, but the man page, the SYNOPSIS and the help text don't li

Gerrit AI review for Lustre PS19 · 2026-10-16 10:50

(minor) lfs_migrate_stripe() also accepts `--src-ost-index` and `--dst-ost-index`, but the man page, the SYNOPSIS and the help text don't list them. If the patch is refreshed, could they either be documented or dropped?

reply PS20 2026-10-20 07:01 Andreas Dilger lfs-migrate_stripe.1:9 — The -i and -o options are generally used interchangeably for regular lfs migrate, lfs find, lfs setstripe, so it _might_ be confusing here.

Andreas Dilger PS20 · 2026-10-20 07:01

The -i and -o options are generally used interchangeably for regular lfs migrate, lfs find, lfs setstripe, so it _might_ be confusing here.  However, I do see the benefit that they could be named `--input` and `--output` to make their distinction more clear.

I think requiring users to select the output OST index will not result in good choices, as that would be different for each raidset, each file, and a good choice will depend on the failure domain of every other stripe in the raidset.

reply PS20 2026-10-20 07:01 Andreas Dilger lfs-migrate_stripe.1:12 — This is "--block" (or "--non-block") for regular lfs migrate.

Andreas Dilger PS20 · 2026-10-20 07:01

This is "--block" (or "--non-block") for regular lfs migrate.

reply PS20 2026-10-20 07:01 Andreas Dilger lfs-migrate_stripe.1:54 — im not sure what this means?

Andreas Dilger PS20 · 2026-10-20 07:01

im not sure what this means?

reply PS20 2026-10-20 07:01 Andreas Dilger lfs-migrate_stripe.1:78 — In addition to the complexity of specifying the correct target OST for every file, another complexity here is that it is unlikely that a tar

Andreas Dilger PS20 · 2026-10-20 07:01

In addition to the complexity of specifying the correct target OST for every file, another complexity here is that it is unlikely that a target OST has enough space to hold the full capacity of an OST being evacuated, unless it is a brand-new replacement.  Otherwise, the single target OST will be full or severely imbalanced compared to the other OSTs.

Also, there would only be the write bandwidth of the single target rather than using the distributed write bandwidth and capacity of many OSTs (at least the peers in a target failure domain, or many failure domains if there are many OSTs in the filesystem (which is often true).

reply PS20 2026-10-20 07:01 Andreas Dilger sanity-pfl.sh:26 — (style) the preference for files that are changed is to use the "new" `always_except` helper to manage the list, so that the LU ticket numbe

Andreas Dilger PS20 · 2026-10-20 07:01

(style) the preference for files that are changed is to use the "new" `always_except` helper to manage the list, so that the LU ticket numbers are better associated with the excepted subtest, and there is no error in manipulating the string, like there was in this case.

(defect) excepted tests need to be associated with an LU ticket that describes the specific issue to be fixed, so that the next person working on this code (maybe you in a year) does not need to reverse engineer the reason for it.

reply PS20 2026-10-20 07:01 Andreas Dilger lfs.c:1667 — I don't think all of this code belongs in lfs.c. I _think_ there is code in llapi to calculate the object offsets, though we've tried to avo

Andreas Dilger PS20 · 2026-10-20 07:01

I don't think all of this code belongs in lfs.c. I _think_ there is code in llapi to calculate the object offsets, though we've tried to avoid externalizing that due to the burden of keeping it in sync for complex layouts.  If not, then this should live in a separate file to avoid adding complexity here (which is aspirationally a "user" of llapi and not a net provider of new/critical functionality, though it is imperfectly so).

reply PS20 2026-10-20 23:14 Gerrit AI review for Lustre lfs.c:1773 — (minor) Does this still match the MDS side? With the parent change, mdd_layout_swap_allowed() lets a CAP_FOWNER caller through a uid/gid mis

Gerrit AI review for Lustre PS20 · 2026-10-20 23:14

(minor) Does this still match the MDS side? With the parent change, mdd_layout_swap_allowed() lets a CAP_FOWNER caller through a uid/gid mismatch when SWAP_LAYOUTS_SINGLE_STRIPE is set:

    if (!(flags & SWAP_LAYOUTS_SINGLE_STRIPE))
        RETURN(-EPERM);
    uc = lu_ucred_check(env);
    if (uc == NULL || !cap_raised(uc->uc_cap, CAP_FOWNER))
        RETURN(-EPERM);

llapi_fswap_layout_fids() always sets that flag, so root would not get -EPERM here even without the fchown(). The fchown() is still reasonable (the volatile is charged to the right owner while it exists), but the stated reason doesn't hold. If the patch is refreshed, the same wording in lfs-migrate_stripe.1 (NOTES, "so mdd_swap_layouts accepts the exchange") and in the comment above sanity-pfl test_36 ("needs fchown on volatile") could be updated to match.

reply PS20 2026-10-20 23:14 Gerrit AI review for Lustre lfs.c:1920 — (style) This isn't a bug, but this kernel-doc block describes migrate_stripe_copy_data(), while it sits directly above migrate_stripe_copy_r

Gerrit AI review for Lustre PS20 · 2026-10-20 23:14

(style) This isn't a bug, but this kernel-doc block describes migrate_stripe_copy_data(), while it sits directly above migrate_stripe_copy_region(), which has its own comment block right after it. The same thing happens further down: the "Resolve OST object FID for stripe @stripe_index ..." block sits above struct migrate_stripe_info and is followed by a second description of migrate_stripe_get_ost_fid(). If the patch is refreshed, could each block be moved to (or merged into) the function it describes?

reply PS20 2026-10-20 23:14 Gerrit AI review for Lustre lfs.c:2288 — (minor) Can LOV_USER_MAGIC_SPECIFIC actually come back from the lustre.lov xattr of a regular file? lod only emits LOV_MAGIC_SPECIFIC for di

Gerrit AI review for Lustre PS20 · 2026-10-20 23:14

(minor) Can LOV_USER_MAGIC_SPECIFIC actually come back from the lustre.lov xattr of a regular file? lod only emits LOV_MAGIC_SPECIFIC for directory default layouts (lod_lov.c: `if (is_dir && lod_comp->llc_ostlist.op_count) magic = LOV_MAGIC_SPECIFIC;`), and lov_lsm_pack() writes V1/V3 for file components. If it were ever reached, it would also be parsed wrongly: SPECIFIC uses the lov_user_md_v3 shape (with lmm_pool_name), but the code takes `v1->lmm_objects` for it. The same applies to the plain-layout branch below. If the patch is refreshed, consider dropping SPECIFIC from both checks.

reply PS20 2026-10-20 23:14 Gerrit AI review for Lustre lfs.c:2933 — (suggestion) If the MDS does not negotiate OBD_CONNECT2_ONE_OBJ_SWAP, the -EOPNOTSUPP only appears at the end, in ll_swap_layouts(). By then

Gerrit AI review for Lustre PS20 · 2026-10-20 23:14

(suggestion) If the MDS does not negotiate OBD_CONNECT2_ONE_OBJ_SWAP, the -EOPNOTSUPP only appears at the end, in ll_swap_layouts(). By then the volatile has been created on the target OST and the whole stripe has been copied. This loop then repeats that full copy for every remaining FILE operand. For the many-file OST drain that the man page describes, could the loop stop after the first -EOPNOTSUPP, or could support be checked once before any copying starts?

tags: pt_ecro

owner Ronnie Sahlberg · uploader Ronnie Sahlberg · PS20 uploaded 13d ago · NEW · open in Gerrit ↗

commit message

LU-20541 lfs: Implement migrate_stripe command and add test

Add lfs migrate_stripe to move OST objects of a file onto another
OST using a single-stripe volatile partner and llapi_fswap_layout_fids().
Accepts multiple FILE operands, migrating each one in turn with the
same options, to make draining many files off an OST convenient.

-i selects a source OST index rather than a stripe index: with
multiple FILE operands the stripe holding data for a given source
OST differs from file to file, so identifying it by index would not
make sense. Instead every stripe currently on the source OST is
migrated to the target OST, including repeats of the same OST within
one component or across components, unless -I restricts the scan to
a single component.

Copy follows absolute LOV stripe numbering and object offsets, keeps
holes sparse, and ftruncates the volatile so trailing holes still
carry file size without SOM. Match source uid/gid on the volatile for
mdd_swap_layouts(). Default path is optimistic (DV check at swap);
--lock-copy holds a group lock across copy and swap.

Clamp the copy end from st_size after reading dv_src (under the group
lock for --lock-copy) so a concurrent append cannot leave the new
stripe short while CHECK_DV1 still matches.

llapi_layout_comp_flags_get() always reports LCME_FL_INIT clear for
a plain (non-composite) layout, so skip that check unless the layout
is actually composite; otherwise every stripe of a plain-layout file
would be silently skipped as "uninstantiated".

Document in lfs-migrate_stripe.1. sanity-pfl 30-39 cover content,
holes, PFL offsets, size, absolute striping, root on user files,
past-EOF components, an OST reused multiple times in a component,
and migrating multiple FILE operands in one invocation. Test 39
covers migrate_stripe on a plain (non-PFL) layout.

Test 32 is skipped on ZFS as we do not yet have FIEMAP support. LU-1941

Test-Parameters: trivial
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I48be7e597544c6856fdcf8edbd181b9f8d3db263

70280 master J –B ✓T ✓R 0/2

LU-20869 tests: cut sanity-sec 31 tunefs and MMP waits

reviewing patrick bot · CI green — awaiting your review

pw_own M +70 −40

build #135015 SUCCESS · tests all Maloo sessions

no vote yet: Sebastien Buisson, Andreas Dilger

patchsets: PS1 10-30 PS2 10-31

tags: pw_own

owner patrick bot · uploader patrick bot · PS2 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-20869 tests: cut sanity-sec 31 tunefs and MMP waits

test_31 and cleanup_31 run about 40 serial tunefs calls per pass,
three or more per target, each with its own ssh and a stop and
writeconf round trip. The servicenode tunefs on mds1 also turns on
ldiskfs MMP, after which every tunefs and mount of mds1 waits 11s
for the MMP check, six tunefs calls in cleanup alone.

Do two tunefs per target, one to erase the old mgsnode and one to
set the new one with writeconf, with the mds1 service node options
in the same calls, on all targets in parallel. The two must stay
separate: on ZFS a key erased and set in one call is stored with a
leading ':'. The parallel calls use parallel_start() with a new
TUNEFS operation, so PARALLEL_OPS=0 or PARALLEL_TUNEFS=0 serializes
them. When mds1 had no failover node before the test, pass
--skipmmp so MMP is not enabled, on servers new enough to have it.

cleanup_31 restored an existing mds1 failover node with
--servicenode, though the framework formats with --failnode, so
mds1 was left with only its partner as a service node and could
not mount. Restore it with the option it was set with.

On a 2 MDS (2 MDT each), 1 OSS (4 OST) ldiskfs VM cluster, test 31
took 415, 407 and 402s before and 312, 291 and 291s after. CI
runs it 6972 times per 30 days at a mean of 548s; at about 100s
saved on ldiskfs and less on ZFS, that is roughly 150 hours.

Test-Parameters: trivial testlist=sanity-sec env=ONLY=31
Test-Parameters: trivial fstype=zfs testlist=sanity-sec env=ONLY=31
Fixes: f70821fc14 ("LU-16775 tests: cleanup target after sanity-sec test_31")
Assisted-by: ClaudeCode:claude-sonnet-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Patrick Farrell <patrick@thelustrecollective.com>
Change-Id: Iaf833bb42bf91925e76f33775405fe8d753eee89

70234 master J –B ✓T ✓R 0/2

LU-20923 mdd: reject a PCC create without a layout

reviewing Andreas Dilger · CI green — awaiting your review

pccro 4 unresolved XS +6 −2

build #134922 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 22:30 Gerrit AI review for Lustre COMMIT_MSG:15 — (minor) Two points here understate what the patch fixes:

Gerrit AI review for Lustre PS1 · 2026-10-31 22:30

(minor) Two points here understate what the patch fixes:

1. "rather than proceeding": the old code did not proceed. It hit LASSERT() on a cr_flags value taken straight from the client, so any client could LBUG the MDS by sending MDS_OPEN_PCC without MDS_OPEN_HAS_EA. That is the main reason for the change and is worth stating.

2. The uninitialized hsm_buf free is not limited to the new reject path. Several existing paths already reached out_free with MDS_OPEN_PCC set before hsm_buf was memset, and then called lu_buf_free() on stack garbage: the OBD_FAIL_MDS_DQACQ_NET goto, a mdd_trans_create() failure, the -ENOMEM ACL buffer gotos, and errors from mdd_acl_init() and mdd_pin_init(). Initializing hsm_buf at its declaration fixes those as well.

reply PS1 2026-10-31 22:30 Gerrit AI review for Lustre COMMIT_MSG:16 — (minor) The patch fixes two bugs from the original PCC change: the LASSERT() on client-supplied flags and the uninitialized hsm_buf free on

Gerrit AI review for Lustre PS1 · 2026-10-31 22:30

(minor) The patch fixes two bugs from the original PCC change: the LASSERT() on client-supplied flags and the uninitialized hsm_buf free on the early error paths of mdd_create(). Both came in with that commit, so please add:

Fixes: f172b11688 ("LU-10092 llite: Add persistent cache on client")

reply PS1 2026-10-31 22:30 Gerrit AI review for Lustre mdd_dir.c:3339 — (suggestion) This isn't a bug, but the check only looks at spec flags, so it could run right after mdd_create_sanity_check() and use RETURN(

Gerrit AI review for Lustre PS1 · 2026-10-31 22:30

(suggestion) This isn't a bug, but the check only looks at spec flags, so it could run right after mdd_create_sanity_check() and use RETURN(-EINVAL), like the other request-validation failures. A malformed request then never creates a transaction or allocates ACL, pin and linkEA buffers that are thrown away straight after. hsm_buf still has to be initialized for the other early error paths, so that part of the change stays.

reply PS1 2026-10-31 22:30 Gerrit AI review for Lustre mdd_dir.c:3342 — (minor) Now that hsm_buf is initialized at its declaration, this memset() does nothing. The MDS_OPEN_PCC test around lu_buf_free(&hsm_buf) a

Gerrit AI review for Lustre PS1 · 2026-10-31 22:30

(minor) Now that hsm_buf is initialized at its declaration, this memset() does nothing. The MDS_OPEN_PCC test around lu_buf_free(&hsm_buf) at out_free is also no longer needed, because lu_buf_free() ignores a NULL lb_buf. If the patch is refreshed, consider dropping both so the code matches how pin_buf is handled.

tags: pccro

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20923 mdd: reject a PCC create without a layout

A PCC create (MDS_OPEN_PCC in cr_flags) must carry a layout
(MDS_OPEN_HAS_EA).  In mdd_create(), return -EINVAL for MDS_OPEN_PCC
without MDS_OPEN_HAS_EA rather than proceeding.  The reject path jumps
to the out_stop/out_free cleanup, which frees hsm_buf when
MDS_OPEN_PCC is set, so also initialize hsm_buf at its declaration
(like pin_buf); otherwise that path, which runs before hsm_buf is
memset, would free an uninitialized buffer.

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-4-8 ltvm,llm_code_and_review_tools
Change-Id: I038b27bd1b99c869084e7d5c2fe79bf4073ebbe5

70233 master J –B ✓T ✓R 0/2

LU-20923 target: reply -EPROTO when a handler packs no reply

reviewing Andreas Dilger · CI green — awaiting your review

3 unresolved S +8 −2

build #134921 SUCCESS · tests all Maloo sessions

threads: 3 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 22:14 Gerrit AI review for Lustre COMMIT_MSG:12 — (minor) Is "early success path" accurate for MDS_GETATTR and MDS_GETXATTR? The paths that reach this check return a plain, non-serious error

Gerrit AI review for Lustre PS1 · 2026-10-31 22:14

(minor) Is "early success path" accurate for MDS_GETATTR and MDS_GETXATTR? The paths that reach this check return a plain, non-serious error, not success. If the client clears OBD_MD_FLID in mbo_valid, mdt_body_unpack() returns 0 and leaves mti_object NULL. Then mdt_getattr() and mdt_tgt_getxattr() return -EPROTO without err_serious():

    if (unlikely(info->mti_object == NULL))
        RETURN(-EPROTO);

The test in tgt_handle_request0() is !is_serious(rc), not rc == 0, so any non-serious return with no reply reaches it, not only success. The same applies to "returned success with no reply" earlier in the paragraph.

reply PS1 2026-10-31 22:14 Gerrit AI review for Lustre COMMIT_MSG:17 — (style) The remotely reachable LBUG() came from returning a non-serious -EPROTO with no reply packed for a NULL mti_object. That return was

Gerrit AI review for Lustre PS1 · 2026-10-31 22:14

(style) The remotely reachable LBUG() came from returning a non-serious -EPROTO with no reply packed for a NULL mti_object. That return was added to mdt_getattr() and mdt_tgt_getxattr() by:

Fixes: e5e0bdb7a5 ("LU-12615 mdt: check mdt_object")

Before that change, these paths hit LASSERT(obj != NULL) or a NULL dereference. After it, they hit this LBUG() instead. Should the tag be added?

reply PS1 2026-10-31 22:14 Gerrit AI review for Lustre tgt_handler.c:499 — (minor) Every non-serious rc is replaced with -EPROTO here, including negative errors. Would it be better to keep the handler's errno and us

Gerrit AI review for Lustre PS1 · 2026-10-31 22:14

(minor) Every non-serious rc is replaced with -EPROTO here, including negative errors. Would it be better to keep the handler's errno and use -EPROTO only when the handler returned rc >= 0?

    rc = err_serious(rc < 0 ? rc : -EPROTO);

The current NULL mti_object paths return -EPROTO anyway, so the client sees no difference for them. But a handler that returns, say, -ENOENT without packing a reply would now get -EPROTO, which hides the real cause.

If the patch is refreshed, the comment above ("Returning success with no reply") and the message text ("returned no error") also don't match the condition, which also fires for non-serious errors. Printing the original rc in the DEBUG_REQ() would help identify the handler path.

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20923 target: reply -EPROTO when a handler packs no reply

tgt_handle_request0() requires each request handler to either pack a
reply or return an error, and treated a handler that returned success
with no reply as a coding bug.  A request can also drive a handler
(for example MDS_GETATTR or MDS_GETXATTR) onto an early success path
that packs no reply.  Send the client -EPROTO and log the handler at
D_ERROR in that case, so the request is failed and reported rather
than treated as a bug.  This is the common backstop reached by both
the MDC and OUT request paths.

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-4-8 ltvm,llm_code_and_review_tools
Change-Id: I470b72c8f9a9499b901ef62155b109620b3ebbe5

70232 master J –B ✓T ✓R 0/2

LU-20923 lod: reject a remote striped create with no layout

reviewing Andreas Dilger · CI green — awaiting your review

3 unresolved S +10 −1

build #134920 SUCCESS · tests all Maloo sessions

threads: 3 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 22:04 Gerrit AI review for Lustre COMMIT_MSG:13 — (minor) "instead of proceeding without one" understates the old behaviour. Before this patch, LASSERT(lovea != NULL) LBUGs the MDS. In a bui

Gerrit AI review for Lustre PS1 · 2026-10-31 22:04

(minor) "instead of proceeding without one" understates the old behaviour. Before this patch, LASSERT(lovea != NULL) LBUGs the MDS. In a build without LIBCFS_DEBUG the LASSERT compiles away, and `info->lti_buf = *lovea` dereferences NULL instead. Either way it is a server crash.

It would also help to say how this is reached. A client sends open(O_CREAT) with write intent, so md_should_create() is true and dof_reg.striped != 0. The new FID (rr_fid2) is in another MDT's sequence, which mdt_new_fid_check() accepts. The path is then mdt_reint_open()->mdo_create()->mdd_declare_create_object_internal()->lod_declare_create()->lod_declare_striped_create(..., NULL, ...). Migrate never comes through here: it sets MDS_OPEN_DELAY_CREATE and passes the layout through lod_declare_xattr_set().

reply PS1 2026-10-31 22:04 Gerrit AI review for Lustre COMMIT_MSG:14 — (minor) This fixes a crash from the assertion added with the migrate code. Consider adding:

Gerrit AI review for Lustre PS1 · 2026-10-31 22:04

(minor) This fixes a crash from the assertion added with the migrate code. Consider adding:

Fixes: 0209add4a509 ("LU-2430 mdd: add lfs mv to migrate inode.")

reply PS1 2026-10-31 22:04 Gerrit AI review for Lustre lod_object.c:6293 — (style) This isn't a bug, but if the patch is refreshed, consider setting `rc = -EINVAL;` before the CERROR() and printing `rc`, then `GOTO(

Gerrit AI review for Lustre PS1 · 2026-10-31 22:04

(style) This isn't a bug, but if the patch is refreshed, consider setting `rc = -EINVAL;` before the CERROR() and printing `rc`, then `GOTO(out, rc)`. That way the message and the return value can't drift apart.

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20923 lod: reject a remote striped create with no layout

lod_declare_striped_create() cannot select OST objects for a remote
object, so the caller must supply the stripe layout (lovea), which
normally happens only during migrate (mdd_migrate_create).  Return
-EINVAL when the layout is absent, so a remote striped create with no
layout fails cleanly instead of proceeding without one.

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-4-8 ltvm,llm_code_and_review_tools
Change-Id: Ib758bd0670ec7c4c3f82cee5e613c55cda3ebbe5

70231 master J –B ✓T ✓R 0/2

LU-20923 mdc: keep existing inode on a lock FID collision

reviewing Andreas Dilger · CI green — awaiting your review

6 unresolved S +17 −7

build #134919 SUCCESS · tests all Maloo sessions

threads: 6 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 21:56 Gerrit AI review for Lustre COMMIT_MSG:11 — (minor) The body doesn't say how this happens or what the user saw: which reply/caller produces the collision, and that the result was the L

Gerrit AI review for Lustre PS1 · 2026-10-31 21:56

(minor) The body doesn't say how this happens or what the user saw: which reply/caller produces the collision, and that the result was the LASSERTF() crash in mdc_set_lock_data(). Since LU-20923 is about the server checking bad requests, a sentence on how a client gets to this state (fuzzed/corrupt reply, a buggy server, a client race?) would make the patch easier to judge and to find later.

reply PS1 2026-10-31 21:56 Gerrit AI review for Lustre COMMIT_MSG:13 — Is "a later access re-enqueues the lock" true in every case? It depends on the FID of the new inode:

Gerrit AI review for Lustre PS1 · 2026-10-31 21:56

Is "a later access re-enqueues the lock" true in every case? It depends on the FID of the new inode:

- new inode FID != resource FID: llite matches locks on the new inode's own FID, so it re-enqueues. Fine.
- new inode FID == resource FID (what the code comment describes, "a reply named a FID the client already holds"): the lock isn't cancelled. ll_intent_drop_lock() only decrefs it into the LRU, so ll_have_md_lock()->md_lock_match() on that FID matches it and nothing is re-enqueued. The new inode then caches attributes/layout under a lock whose blocking AST only invalidates the old inode (ll_lock_cancel_bits()->ll_inode_from_resource_lock()).

Which case is the patch handling?

reply PS1 2026-10-31 21:56 Gerrit AI review for Lustre mdc_locks.c:110 — How can there be two different live inodes with the same FID? ll_iget() uses iget5_locked() with ll_test_inode() comparing lli_fid, and mdc_

Gerrit AI review for Lustre PS1 · 2026-10-31 21:56

How can there be two different live inodes with the same FID? ll_iget() uses iget5_locked() with ll_test_inode() comparing lli_fid, and mdc_finish_intent_lock() asserts that the lock resource matches mdt_body::mbo_fid1. So it looks like the new inode must have a different FID than the resource, not "a FID the client already holds".

If the FID really can be the same, there's a second problem: when the new inode is evicted, ll_clear_inode()->md_null_inode(ll_inode2fid(inode)) sets lr_lvb_inode to NULL on this resource. The old inode, which is still live, then loses its attachment and stops getting cache invalidation from blocking ASTs.

reply PS1 2026-10-31 21:56 Gerrit AI review for Lustre mdc_locks.c:115 — (suggestion) The invariant that matters later is that the attached inode's FID equals lr_name. ll_inode_from_resource_lock() LBUG()s on "dat

Gerrit AI review for Lustre PS1 · 2026-10-31 21:56

(suggestion) The invariant that matters later is that the attached inode's FID equals lr_name. ll_inode_from_resource_lock() LBUG()s on "data mismatch with object" when it doesn't. This check only covers the case where a live inode is already attached:

- If lr_lvb_inode is NULL, or the old inode is I_FREEING, an inode with a mismatched FID is still attached, and the next blocking AST crashes in ll_inode_from_resource_lock().
- If the old inode is the one with the wrong FID, keeping it just moves that LBUG to cancel time.

Would it be more robust to check the new inode's FID against the resource name (via an llite callback, to stay within the layering rules) and refuse the attach whenever they differ?

reply PS1 2026-10-31 21:56 Gerrit AI review for Lustre mdc_locks.c:116 — (minor) This used to be in an LASSERTF(), which crashes anyway. Now it's a recoverable console message that a server reply can trigger repea

Gerrit AI review for Lustre PS1 · 2026-10-31 21:56

(minor) This used to be in an LASSERTF(), which crashes anyway. Now it's a recoverable console message that a server reply can trigger repeatedly, so %px puts raw kernel addresses in dmesg and gets around kptr_restrict. Since this is no longer fatal, would %p (hashed) be enough? i_ino/i_generation already identify the inodes. Printing the resource FID (PLDLMRES(lock->l_resource)) would also be more useful to an admin than the pointers.

reply PS1 2026-10-31 21:56 Gerrit AI review for Lustre mdc_locks.c:121 — (style) This isn't a bug, but the message prints "rc = -17" while the function returns 0, so the log reports an error code that is never ret

Gerrit AI review for Lustre PS1 · 2026-10-31 21:56

(style) This isn't a bug, but the message prints "rc = -17" while the function returns 0, so the log reports an error code that is never returned. Either set rc = -EEXIST and use it for both the message and the return (callers ignore the return value), or drop the ": rc = %d" so the log doesn't suggest a failure was returned.

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20923 mdc: keep existing inode on a lock FID collision

An MDC lock resource is keyed by the object FID, and
mdc_set_lock_data() attaches the inode being set to that resource.
When the resource already references a different inode that is still
live, keep the existing inode attached and do not attach the new one;
a later access re-enqueues the lock.  This replaces the previous
requirement that a resource could only ever reference an inode on its
way out (I_FREEING).

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-4-8 ltvm,llm_code_and_review_tools
Change-Id: I4da7da3f4846a9f38d09ec09c7d51c994d3ebbe5

70274 master J –B ✓T ✓R 0/2

LU-20565 tests: use EC_* layouts in sanity-ec 27a-44a

reviewing Andreas Dilger · CI green — awaiting your review

testfix M +56 −59

build #134938 SUCCESS · tests all Maloo sessions

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: use EC_* layouts in sanity-ec 27a-44a

sanity-ec subtests 27a, 30b, 31a and the degraded-read subtests
41e-44a hard-coded an EC 4+2, 4+1 or 2+1 layout with a gate of 4 to
8 OSTs, although the geometry did not matter to what they tested.
Most would be skipped on the 4 OST test sessions.

Use the EC_ARG, EC_DSTRIPE and EC_CSTRIPE defaults that enable_ec()
scales to OSTCOUNT and drop the gates, so they run with 3 OSTs.
Scale sizes and offsets tied to a 4-stripe row to EC_DSTRIPE, and
put each PFL boundary in front of an EC component on a multiple of
the row width, with 1M stripes in 27a.  41g keeps its 2+1 layout,
which only needs the 3 OSTs that enable_ec() requires.

Test-Parameters: trivial testlist=sanity-ec
Test-Parameters: testlist=sanity-ec ostcount=4
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I8fb9da8c65de7ba367f0d70b7744d57c353ebbe5

70229 master J –B ✓T ✓R 0/2

LU-20923 llite: handle unexpected MDS reply contents

reviewing Andreas Dilger · CI green — awaiting your review

3 unresolved M +46 −15

build #134918 SUCCESS · tests all Maloo sessions

threads: 3 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 21:34 Gerrit AI review for Lustre file.c:1545 — (minor) The comment says the reply "can clear" the open-ref disposition, and the commit message says the reply must "carry" DISP_ENQ_OPEN_RE

Gerrit AI review for Lustre PS1 · 2026-10-31 21:34

(minor) The comment says the reply "can clear" the open-ref disposition, and the commit message says the reply must "carry" DISP_ENQ_OPEN_REF. The server never sends that flag. mdc_finish_enqueue() sets it on the client, and it does so exactly when DISP_OPEN_OPEN is set and it_open_error(DISP_OPEN_OPEN) is 0. Since the it_open_error() check above has passed, the only way to get here is a reply with no DISP_OPEN_OPEN, meaning the MDS didn't perform the open. If the patch is refreshed, consider saying that directly, e.g. "the MDS replied without executing the open". That would also make clear why there's no MDS open handle to close here.

reply PS1 2026-10-31 21:34 Gerrit AI review for Lustre llite_lib.c:3029 — (minor) This return comes before lli_replace_acl(), which normally takes over md->posix_acl. md_put_lustre_md() doesn't release the ACL (onl

Gerrit AI review for Lustre PS1 · 2026-10-31 21:34

(minor) This return comes before lli_replace_acl(), which normally takes over md->posix_acl. md_put_lustre_md() doesn't release the ACL (only lmd_clear_acl() does), so when the reply has OBD_MD_FLACL set, callers that don't clear it leak the posix_acl:

    ll_prep_inode(*inode != NULL)->ll_update_inode() -> GOTO(out) -> md_put_lustre_md()
    ll_md_setattr()->ll_update_inode() -> md_put_lustre_md()

The ll_iget() path is fine because ll_prep_inode() calls lmd_clear_acl() when ll_iget() fails. The existing cl_file_inode_init() and ll_update_lsm_md() error returns already leak the same way, so calling lmd_clear_acl() on the error path in ll_prep_inode() and ll_md_setattr() would fix all of them. That isn't urgent, since this only happens when the MDS sends a bad reply.

reply PS1 2026-10-31 21:34 Gerrit AI review for Lustre llite_lib.c:3029 — (style) This isn't a bug, but Lustre style is to set rc = -EPROTO once and use rc in both the CERROR() and the return, so the two can't drif

Gerrit AI review for Lustre PS1 · 2026-10-31 21:34

(style) This isn't a bug, but Lustre style is to set rc = -EPROTO once and use rc in both the CERROR() and the return, so the two can't drift apart. rc is already declared in this function.

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20923 llite: handle unexpected MDS reply contents

Several llite paths relied on invariants that an MDS reply or granted
lock is expected to satisfy, and handled only the expected case.  Make
each path handle the unexpected contents gracefully:

 - ll_update_inode() copies the FID from the reply body into the
   inode.  Check the reply FID against the inode FID before applying
   any attribute, and return -EPROTO on a mismatch, so a stale or
   foreign reply is dropped rather than partly applied; the FID is
   still populated the first time, when the inode FID is not yet set.

 - ll_inode_from_resource_lock() returns the inode cached on a lock
   resource, which is keyed by FID.  When the cached inode's FID does
   not match the resource name, log it and return no inode (as for the
   bogus-magic case) so the caller skips it; the grabbed inode is
   released with iput() after the resource lock is dropped, as iput()
   may sleep.

 - ll_lock_cancel_bits() maps an OPEN-bit lock's mode to the open mode
   to close.  For a lock granted in any other mode, log it and skip
   the open-close, leaving the OPEN bit cleared.

 - ll_lease_open() requires the open intent reply to carry the
   DISP_ENQ_OPEN_REF disposition.  Return -EPROTO when it does not; no
   open handle has been filled at that point, so only the intent is
   released.

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-4-8 ltvm,llm_code_and_review_tools
Change-Id: I0515212561d3943f88561736616bfd09f43ebbe5

70276 master J –B ✓T ✓R 0/2

LU-20565 tests: use EC_* layouts in sanity-ec 58b-65d

reviewing Andreas Dilger · CI green — awaiting your review

testfix 4 unresolved M +102 −74

build #134940 SUCCESS · tests all Maloo sessions

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 21:24 Gerrit AI review for Lustre COMMIT_MSG:9 — (minor) The body says 58b-65d hard-coded 4+2 behind a 6/8 OST gate and that the gates are dropped. However, 63d and 65b in that range still

Gerrit AI review for Lustre PS1 · 2026-10-31 21:24

(minor) The body says 58b-65d hard-coded 4+2 behind a 6/8 OST gate and that the gates are dropped. However, 63d and 65b in that range still have `(( OSTCOUNT >= 8 ))` and the fixed 4+2 layouts. The next patches in the chain (67a-71d, 73a-75e) don't pick them up either.

63d is the same two-component 4+2 PFL shape as 62c and 59a, which were converted here. 65b could keep its mixed 4+2/2+1 geometry with the gate lowered to 6 OSTs, the way 68c and 71b are handled in the 67a-71d patch. Was leaving both at 8 OSTs intentional? If so, a sentence saying why would help.

reply PS1 2026-10-31 21:24 Gerrit AI review for Lustre COMMIT_MSG:15 — (minor) "so they run with 3 OSTs" doesn't hold for 63a. With 3 OSTs enable_ec() picks 2+1, so the file has only 2 data stripes and `safe_ost

Gerrit AI review for Lustre PS1 · 2026-10-31 21:24

(minor) "so they run with 3 OSTs" doesn't hold for 63a. With 3 OSTs enable_ec() picks 2+1, so the file has only 2 data stripes and `safe_osts` has at most 2 entries. The unchanged `(( ${#safe_osts[@]} >= 3 )) || skip_env` check then always skips. Either relax that check (cycling through 2 OSTs still tests the same thing), or mention 63a with the other tests that skip.

reply PS1 2026-10-31 21:24 Gerrit AI review for Lustre sanity-ec.sh:8420 — (minor) At OSTCOUNT=3 the default layout is 2+1, so this can never see 3 safe data OSTs and 63a is still always skipped there. Would `>= 2`

Gerrit AI review for Lustre PS1 · 2026-10-31 21:24

(minor) At OSTCOUNT=3 the default layout is 2+1, so this can never see 3 safe data OSTs and 63a is still always skipped there. Would `>= 2` (or `>= EC_DSTRIPE` capped at 3) keep the test's intent and let it run on 3 OSTs?

reply PS1 2026-10-31 21:24 Gerrit AI review for Lustre sanity-ec.sh:8537 — (style) This isn't a bug, but removing the gate leaves a blank line right after the opening brace. The same happens in test_64d. If the patc

Gerrit AI review for Lustre PS1 · 2026-10-31 21:24

(style) This isn't a bug, but removing the gate leaves a blank line right after the opening brace. The same happens in test_64d. If the patch is refreshed, it could be dropped.

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: use EC_* layouts in sanity-ec 58b-65d

sanity-ec degraded-read subtests 58b-65d hard-coded an EC 4+2
layout with a gate of 6 or 8 OSTs, although the geometry did not
matter to what they tested.  They would be skipped on the 4 and 7
OST test sessions.

Use the EC_ARG, EC_DSTRIPE and EC_CSTRIPE defaults that enable_ec()
scales to OSTCOUNT and drop the gates, so they run with 3 OSTs.
Scale sizes and offsets tied to a 4-stripe row to EC_DSTRIPE, and
put each PFL boundary in front of an EC component on a multiple of
the row width.  60d and 65c mix a 2+1 component with the default
layout and need 4 OSTs so the two differ.  With only one parity
stripe, 59a and 63c skip their two-OST part, and 58b, 61d and 65d,
which only fail two OSTs, are skipped.

Test-Parameters: trivial testlist=sanity-ec
Test-Parameters: testlist=sanity-ec ostcount=4
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I8a0d5693af6204d29defa5da9fbb505a123ebbe5

70272 master J –B ✓T ✓R 0/2

LU-20565 tests: use EC_* layouts in sanity-ec layout tests

reviewing Andreas Dilger · CI green — awaiting your review

testfix 2 unresolved M +73 −61

build #134936 SUCCESS · tests all Maloo sessions

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 21:19 Gerrit AI review for Lustre COMMIT_MSG:13 — (minor) Is "7 OST" accurate here? The old gates were 6, 7, 6, 7, 8, 6, 6 and 4 OSTs, so a 7-OST session only skipped 2f. On a 4-OST session,

Gerrit AI review for Lustre PS1 · 2026-10-31 21:19

(minor) Is "7 OST" accurate here? The old gates were 6, 7, 6, 7, 8, 6, 6 and 4 OSTs, so a 7-OST session only skipped 2f. On a 4-OST session, every subtest except 21c was skipped.

reply PS1 2026-10-31 21:19 Gerrit AI review for Lustre sanity-ec.sh:893 — (minor) This isn't a bug, but the row-width rule from the commit message is only applied to some subtests. 2a sets up the same `-N -E 128M -

Gerrit AI review for Lustre PS1 · 2026-10-31 21:19

(minor) This isn't a bug, but the row-width rule from the commit message is only applied to some subtests. 2a sets up the same `-N -E 128M -E -1 $EC_ARG` layout as 2b, and 1d mirrors 1k. Both keep 128M, which is not a multiple of 3M or 5M (OSTCOUNT 4, 5 or 7). lfs and LOD don't enforce the alignment, and the client EC code computes rows relative to the start of the component (lov_object.c and the resync code in liblustreapi_layout.c). If the alignment matters, should 1d, 1e, 2a, 3a and the others get it too, with the reason given in the commit message? If it doesn't, the extra `$end` variables could be dropped.

The component also has no `-c`, so its row width is the instantiated stripe count times the stripe size, not necessarily `${EC_DSTRIPE}M`. The same applies to the "2M" row in the 1f comment, which describes the 2+1 component that uses the default stripe count.

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: use EC_* layouts in sanity-ec layout tests

sanity-ec subtests 1f, 1h, 1k, 2b, 2f, 7b, 7c and 21c hard-coded an
EC 4+2 or 3+1 layout, most with a gate of 6 to 8 OSTs, although they
only check how parity components follow the data components, how a
parity mirror is split, or a migrate between EC layouts.  Most would
be skipped on the 4 and 7 OST test sessions.

Use the EC_ARG, EC_DSTRIPE and EC_CSTRIPE defaults that enable_ec()
scales to OSTCOUNT.  2b, 2f, 7b and 7c now run with 3 OSTs.  1f, 1h,
1k and 21c mix a fixed 2+1 layout with the default one, and need 4
OSTs so the two layouts differ.  Put each PFL boundary in front of
an EC component on a multiple of the row width.

Test-Parameters: trivial testlist=sanity-ec
Test-Parameters: testlist=sanity-ec ostcount=4
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: Ibb58f4778fdd722a8cc663b8f5a6f285dd3ebbe5

70277 master J –B ✓T ✓R 0/2

LU-20565 tests: use EC_* layouts in sanity-ec 67a-71d

reviewing Andreas Dilger · CI green — awaiting your review

testfix 1 unresolved M +35 −30

build #134941 SUCCESS · tests all Maloo sessions

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 20:58 Gerrit AI review for Lustre sanity-ec.sh:9277 — (minor) With one parity stripe, this check still runs before the 1-down part, so 69d can skip_env completely instead of only skipping its 2-

Gerrit AI review for Lustre PS1 · 2026-10-31 20:58

(minor) With one parity stripe, this check still runs before the 1-down part, so 69d can skip_env completely instead of only skipping its 2-down part as the commit message says.

ec_classify_osts() drops every OST that is parity in *any* component. With 3 OSTs (2+1), both PFL components use all three OSTs, and LOD only keeps a parity off its own component's data OSTs. If the two parities land on different OSTs, only one safe OST is left. The same can happen at 5 OSTs (3+2), where the parity union can reach 4.

If the patch is refreshed, consider the ordering already used by the PFL sparse test earlier in this file: require >= 1 safe OST here, run the 1-down check, `return 0` when EC_CSTRIPE < 2, and only then require 2 safe OSTs for the 2-down part.

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: use EC_* layouts in sanity-ec 67a-71d

sanity-ec degraded-read subtests 67a-71d had a gate of 8 OSTs, and
most hard-coded an EC 4+2 layout although the geometry did not
matter to what they tested.  They would be skipped on the 4 and 7
OST test sessions.

Use the EC_ARG, EC_DSTRIPE and EC_CSTRIPE defaults that enable_ec()
scales to OSTCOUNT and drop the gates, so they run with 3 OSTs.  Put
each PFL boundary in front of an EC component on a multiple of the
row width.  68d mixes a 2+1 component with the default layout and
needs 4 OSTs so the two differ.  With only one parity stripe, 69d
skips its two-OST part, and 69b and 71c, which only fail two OSTs,
are skipped.  68a (2+1), 68c (4+1) and 71b (with a 4+2 file) keep
their layouts, with the gate lowered to 3, 5 and 6 OSTs.

Test-Parameters: trivial testlist=sanity-ec
Test-Parameters: testlist=sanity-ec ostcount=4
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: Id80cd74077c3543b9b91ff95b32be302ef3ebbe5

70275 master J –B ✓T ✓R 0/2

LU-20565 tests: use EC_* layouts in sanity-ec 46d-52c

reviewing Andreas Dilger · CI green — awaiting your review

testfix 2 unresolved L +158 −111

build #134939 SUCCESS · tests all Maloo sessions

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 20:57 Gerrit AI review for Lustre COMMIT_MSG:9 — (minor) test_49a is inside the 46d-52c range and still hard-codes `-c 4 --ec 4+2` behind `(( OSTCOUNT >= 6 ))`, so it is still skipped on th

Gerrit AI review for Lustre PS1 · 2026-10-31 20:57

(minor) test_49a is inside the 46d-52c range and still hard-codes `-c 4 --ec 4+2` behind `(( OSTCOUNT >= 6 ))`, so it is still skipped on the 4 OST sessions. It has the same create/resync/disable-two-safe-OSTs/read structure as 49b, so the geometry doesn't seem to matter to it any more than it does to 49b-49d. It isn't converted in the later 58b-75e patches either. Should 49a get the same conversion, or is there a reason to keep its gate? If it stays, the message could say so, as it does for 52c. 49e and 51a keeping their gates is fine, since both depend on a specific raid-set/stripe-set geometry.

reply PS1 2026-10-31 20:57 Gerrit AI review for Lustre sanity-ec.sh:7067 — (suggestion) This isn't a bug, but this patch adds the same ten-line "fail one or two safe OSTs depending on EC_CSTRIPE" block to 49b, 49c,

Gerrit AI review for Lustre PS1 · 2026-10-31 20:57

(suggestion) This isn't a bug, but this patch adds the same ten-line "fail one or two safe OSTs depending on EC_CSTRIPE" block to 49b, 49c, 49d, 50a and 50b, and the 73a-75e patch later in the series adds more copies. If the patch is refreshed, consider a small helper next to ec_deactivate_ost(), e.g. one that picks `safe_osts[0]`, and `safe_osts[1]` when EC_CSTRIPE >= 2, deactivates them and leaves the list in an array that 49d can use for ec_reactivate_ost(). The `(( ${#safe_osts[@]} >= 2 ))` skip could then become `>= ${#down[@]}`, so it matches the number of OSTs actually failed.

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: use EC_* layouts in sanity-ec 46d-52c

sanity-ec degraded-read subtests 46d-52c hard-coded an EC 4+2
layout with a gate of 6 or 8 OSTs, although the geometry did not
matter to what they tested.  They would be skipped on the 4 OST test
sessions, and the 8 OST gates also on the 7 OST sessions.

Use the EC_ARG, EC_DSTRIPE and EC_CSTRIPE defaults that enable_ec()
scales to OSTCOUNT and drop the gates, so they run with 3 OSTs.
Scale sizes, offsets and stripe indexes tied to a 4-stripe row to
EC_DSTRIPE, and put each PFL boundary in front of an EC component
on a multiple of the row width.  With only one parity stripe the
concurrent-I/O subtests 49b-49d, 50a and 50b fail one OST instead of
two, and 52c, which needs a data and a parity OST down, is skipped.
52b now fails EC_CSTRIPE + 1 data OSTs to exceed the parity.

Test-Parameters: trivial testlist=sanity-ec
Test-Parameters: testlist=sanity-ec ostcount=4
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I957515862d0ba36067a7d40b3d3808f7ce3ebbe5

70278 master J –B ✓T ✓R 0/2

LU-20565 tests: use EC_* layouts in sanity-ec 73a-75e

reviewing Andreas Dilger · CI green — awaiting your review

testfix M +46 −46

build #134942 SUCCESS · tests all Maloo sessions

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 2d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: use EC_* layouts in sanity-ec 73a-75e

sanity-ec degraded-read subtests 73a-75e hard-coded an EC 4+2
layout with a gate of 6 or 8 OSTs, although the geometry did not
matter to what they tested.  They would be skipped on the 4 OST
test sessions, and 73x also on the 7 OST sessions.

Use the EC_ARG, EC_DSTRIPE and EC_CSTRIPE defaults that enable_ec()
scales to OSTCOUNT and drop the gates, so they run with 3 OSTs.
75b and 75e write one row of EC_DSTRIPE stripes per file, and 75e
checks for EC_CSTRIPE parity stripes.  With only one parity stripe,
73d and 73e skip their two-OST part.

Test-Parameters: trivial testlist=sanity-ec
Test-Parameters: testlist=sanity-ec ostcount=4
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: Icde35ecd386787a74b40519a22c2ceec263ebbe5

69517 master J –B ✓T ✓R +1/2

LU-20690 ec: implement STRIPE I/O scope

reviewing Keguang Xu · CI green — awaiting your review

218_review_prioritypt_ecroraid10-like 3 unresolved L +455 −99

build #134549 SUCCESS · tests all Maloo sessions

reviews: Ronnie Sahlberg +1 2026-10-28 05:23

no vote yet: Andreas Dilger, Zhenyu Xu

patchsets:

+4 earlier

PS1 10-08 PS2 10-10 PS3 10-13 PS4 10-14

PS5 10-17 PS6 10-22 Marc Vef PS7 10-23 PS8 10-27 PS9 10-27

threads: 2 need your reply · 1 waiting on others · 0 own notes · 0 bot

reply PS8 2026-10-27 12:04 Keguang Xu cl_object.h:2091 — Unlike mirror or other fields, `stripe=0` actually represents a valid value. While on the other side, `memset(0)` is widely used to clear th

Andreas Dilger PS8 · 2026-10-27 03:32

ci_designated_stripe_set is redundant: store idx + 1 so 0 means unset, the same convention as ci_designated_mirror.  That also removes lov_io_has_designated_stripe().  About 8 lines.

Keguang Xu PS8 · 2026-10-27 12:04

Unlike mirror or other fields, `stripe=0` actually represents a valid value. While on the other side, `memset(0)` is widely used to clear the structure where *0* by default means unset. That's why this extra `ci_designated_stripe_set` is introduced here, to make sure `stripe=0` stays as a recognizable/familiar value to reviewer(do not surprise them), and keep the `memset(0)` style intact.

reply PS9 2026-10-28 03:38 Gerrit AI review for Lustre lov_io.c:1319 — (style) Carried over from patchset 8, where it was Acknowledged: this isn't a bug, but this branch runs only for truncate. A truncate is nev

Gerrit AI review for Lustre PS9 · 2026-10-28 03:38

(style) Carried over from patchset 8, where it was Acknowledged: this isn't a bug, but this branch runs only for truncate. A truncate is never CIT_READ/CIT_WRITE, so lov_foreach_io_stripe() always visits every stripe here. If the patch is refreshed, the plain `for (stripe = 0; stripe < r0->lo_nr; stripe++)` loop would keep the churn, and conflicts with the EC read patches, smaller.

tags: 218_review_priority pt_ecro raid10-like

owner Keguang Xu · uploader Keguang Xu · PS9 uploaded 6d ago · NEW · open in Gerrit ↗

commit message

LU-20690 ec: implement STRIPE I/O scope

In the RAID10-like EC parity layout the p stripes of a raidset share
one file offset but hold different data, so read and write must name
one of them. Implement LL_IO_SCOPE_STRIPE for that.

ll_io_scope_validate() accepts STRIPE with ios_mirror_id in
[1, MIRROR_ID_MAX] and ios_stripe_idx below
LOV_EC_MAX_CODING_STRIPES. ios_stripe_idx must be 0 for FILE and
MIRROR. It is not checked against p here: the scope outlives the
layout and the I/O type is not known yet. ll_io_scope_init() handles
STRIPE like MIRROR and, for STRIPE, copies ios_stripe_idx into
cl_io::ci_designated_stripe and sets ci_designated_stripe_set.
OBJECT stays reserved.

ci_designated_stripe is an unsigned index. 0 is a valid stripe, so
ci_designated_stripe_set says whether it is in use. A zeroed cl_io
is unset. The index is meaningful only with ci_designated_mirror
set.

lov visits only the stripes an I/O touches through
lov_foreach_io_stripe(), and maps an offset to a stripe through
lov_io_stripe_number(). On a parity component, read and write use
only the designated stripe of each raidset and fail with -EINVAL if
it is unset or not less than p. Punch, truncate, fallocate and lseek
reach every stripe and ignore the index, so a leftover STRIPE scope
does not fail them.

These helpers target the bare LOV_PATTERN_PARITY encoding for stripe
picking. lov_page_init_composite() also refuses CPT_CACHEABLE pages
when ci_designated_mirror is set on any parity component
(lsm_entry_is_parity()), including RAID0|PARITY. That covers MIRROR
and STRIPE scopes and LL_IOC_FLR_SET_MIRROR: a designated parity
read that falls back to the page cache (fcntl clears O_DIRECT, or a
short O_DIRECT read finished through the cache) returns -EINVAL.
sanity-ec test_312 checks that case.

llapi_io_scope_set(3) and the uapi comment document STRIPE, and that
ios_layout_version 0 uses the fd version from the last lease release
or resync lease. LL_IOC_SET_IO_SCOPE reuses nr 177 with
LL_IOC_FLR_SET_MIRROR (different size) to save an ioctl number.
sanity-flr test_213a checks the ioctl limits for FILE, MIRROR and
STRIPE, that OBJECT stays reserved, that mirror resync still goes
through the SET_MIRROR alias, and that a STRIPE read of a non-parity
mirror ignores the index. sanity-ec test_311 checks that lseek,
truncate and fallocate succeed for any index, and skips below 6
OSTs. io_scope gains -b, -x, -r, -t, -f and -l for that.

Test-Parameters: testlist=sanity-flr
Test-Parameters: testlist=sanity-ec ostcount=8
Assisted-by: Cursor:Composer-2 llm_code_and_review_tools
Signed-off-by: Keguang Xu <kxu@ddn.com>
Change-Id: Ia8c9e729df4f8651ac4f9dad88d4aef8b056100d

70212 master J –B ✓T ✓R 0/2

LU-20565 tests: remove duplicate sanity-ec subtests

reviewing Andreas Dilger · CI green — awaiting your review

testfix L +7 −716

build #134845 SUCCESS · tests all Maloo sessions

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: remove duplicate sanity-ec subtests

Many sanity-ec degraded-read subtests repeated the same layout,
write, resync and fault as another subtest.  They would add run time
and setup cost without testing anything the other subtest did not.

Remove each subtest whose checks another subtest already makes:

- 11 (create 2+2): 12a creates the same layout and checks parity
- 30a (write stales parity, resync): 23a and 28a check the same
  ro/wp/stale transitions, 6b checks the data across the resync
- 41f (DIO read after buffered write): 41i does that after a
  buffered degraded read of the same file
- 43c, 44c (EIO when all reads fail): 44a, which also reads the
  file again after the fault is cleared
- 43d (fail 1, then 2 stripes): 63c fails 1, then 2, then 1 again
- 44b (repeat a 1-OST fault): 63a and 63c fail and clear faults
  repeatedly on the same file
- 45a (fail stripes 0 and 1): 43b is the same test
- 45c (fail only parity OSTs): 75a, which now fails every parity
  OST that holds no data rather than one, and also checks stat and
  getstripe
- 47a (2+1 then 4+2 PFL): 68d, which fails each safe OST in turn
- 47c (1M then 4M stripe PFL): 46d
- 47d (read across a PFL boundary): 42d
- 63b (every pair of safe OSTs): 58a is the same test
- 64b (stale parity read, then resync): 48c
- 64c (3 rewrite/resync rounds from mount2): 64a does 2 rounds
- 65a (3-component PFL): 65b uses 4 components of mixed geometry
  and stripe size
- 67b (rewrite and resync 3 times): 75b repeats overwrite, resync
  and degraded read 50 times, and 64a covers truncating rewrites

Test-Parameters: trivial testlist=sanity-ec
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I8d081a238b2ef8f7d0b84b96514092ec773ebbe5

70213 master J –B ✓T ✓R 0/2

LU-20565 tests: merge similar sanity-ec degraded-read subtests

reviewing Andreas Dilger · CI green — awaiting your review

testfix 1 unresolved XL +414 −1563

build #134846 SUCCESS · tests all Maloo sessions

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 10:55 Gerrit AI review for Lustre sanity-ec.sh:6039 — (suggestion) ec_start_read_fault() fails only the OST that holds data stripe 0. Several of these reads never touch that stripe, so they are

Gerrit AI review for Lustre PS1 · 2026-10-31 10:55

(suggestion) ec_start_read_fault() fails only the OST that holds data stripe 0. Several of these reads never touch that stripe, so they are served from healthy OSTs and pass whether or not reconstruction works. With 1M stripes and stripe 0 at [n*row, n*row + 1M):

    EC_DSTRIPE=4: size-1M+1M, 2M+2M, 1.25M+2M
    EC_DSTRIPE=5,6 (7+ OSTs): 3M+2M, size-1M+1M, 2M+2M, 1.25M+2M

Most of these were already vacuous in the old 41a/41c with 4+2. The 3M+2M read is new: with 4+2 it reached stripe 0 of the second row, but with EC_DSTRIPE >= 5 it covers only stripes 3-4, so on a 7-OST setup it no longer checks recovery.

If the patch is refreshed, one option is to fail the OST holding the stripe at each read's offset, e.g. ec_data_stripe_osts $tf $(( (off / m) % EC_DSTRIPE )), and pass that to ec_start_read_fault().

tags: testfix

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: merge similar sanity-ec degraded-read subtests

Many sanity-ec subtests ran the same create, write, resync and
degraded-read sequence and differed only in the file size, stripe
size, read offsets or the write before the resync.  Each repeated
the setup and the fault handling, and most hard-coded a 4+2 layout
with a gate of 6 or 8 OSTs, so they would be skipped on smaller
configurations.

Merge each group into one subtest that loops over the cases, and
keep the number of the subtest that remains:

- 61a: sub-stripe, partial-stripe, partial-row and multi-row file
  sizes with 64K, 256K, 1M and 4M stripes (42b 42c 46a 46b 46c
  61b)
- 58a: every pair of failed data OSTs, for 2+2 and for the default
  layout when it has 2 parity stripes (43b 45b 45d 52a 68b)
- 41a: aligned, unaligned and multi-row reads (41b 41c)
- 70a: overwrite then resync: across a row boundary, unaligned,
  after an extend, a partial row, either PFL component (67c 67d
  70b 70c 70d)
- 62a: staged, unaligned and sub-stripe appends (62b 62d)
- 60a: truncate mid-stripe, to a row boundary, to 0, and before an
  extend (60b 60c 73c)
- 32a: OST stop and start, then write and resync, twice (30c 33a)
- 34a: mirror resync, read and verify keep the size (34b)
- 20: parity component layout and stripe size for 4M, 1M and 64K
  stripes (1j)

Add ec_check_degraded() to read a file with the first, or each, data
OST that holds no parity failed, and optionally two of them.

The merged subtests use EC_ARG, EC_DSTRIPE and EC_CSTRIPE and size
files in rows of EC_DSTRIPE stripes, so they run with 3 OSTs (58a
with 4), and fail two OSTs only when EC_CSTRIPE >= 2.  Every
degraded read now compares the data with the healthy copy and
drops the client locks first: 41c read several ranges after one
drop_caches, 33a only checked that the read succeeded, and the
size of the unaligned appends in 62b was not checked.

Test-Parameters: trivial testlist=sanity-ec
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I430d4648463a5e78d21c6a2dd58b4ce7b03ebbe5

68455 master J –B ✓T ✓R 0/2

LU-6689 nodemap: add layout_ops rbac role

reviewing Sebastien Buisson · CI green — awaiting your review

4 unresolved XL +1354 −29

build #134778 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger

patchsets:

+23 earlier

PS1 08-28 PS2 08-31 PS3 09-03 PS4 09-03 PS5 09-03 PS6 09-04 PS7 09-07 PS8 09-08 PS9 09-10 PS10 09-15 PS11 09-17 PS12 09-18 PS13 09-21 PS14 09-23 PS15 09-24 PS16 09-25 PS17 09-28 PS18 09-28 PS19 10-09 PS20 10-14 PS21 10-20 PS22 10-22 PS23 10-23

PS24 10-23 PS25 10-27 PS26 10-28 PS27 10-28 PS28 10-29

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS28 2026-10-31 09:28 Gerrit AI review for Lustre mdt_handler.c:1809 — (suggestion) This runs for every client on the first write/truncate layout intent of every file, not only for nodemaps without layout_ops. E

Gerrit AI review for Lustre PS28 · 2026-10-31 09:28

(suggestion) This runs for every client on the first write/truncate layout intent of every file, not only for nodemaps without layout_ops. Even on a system where nobody ever removes the role (the default 'all'), each object pays two extra mo_xattr_get() calls under mot_som_mutex. Each SEL file also keeps a full copy of its layout in mot_sel_lmm for as long as it stays in cache.

The saved copy is already best effort (it is lost on restart or eviction, and a missing copy only means an EPERM on migrate). Could it be skipped at least when nodemap is not active? Then systems that don't use nodemap would not pay for it.

reply PS28 2026-10-31 09:28 Gerrit AI review for Lustre mdt_internal.h:1489 — (minor) Is this helper still needed? Its only caller is mdt_swap_layouts(), and that function has already called mdt_init_ucred() before the

Gerrit AI review for Lustre PS28 · 2026-10-31 09:28

(minor) Is this helper still needed? Its only caller is mdt_swap_layouts(), and that function has already called mdt_init_ucred() before the check. old_init_ucred() and new_init_ucred() both go through ucred_set_rbac_roles(), so mdt_ucred(info)->uc_rbac_layout_ops is valid there. The comment about the ucred not being initialized therefore doesn't match the caller.

If the patch is refreshed, using the ucred bit directly would drop the second copy of the nodemap lookup. It would also keep its error handling the same as every other rbac check.

reply PS28 2026-10-31 09:28 Gerrit AI review for Lustre mdt_open.c:1552 — (defect) MDS_OPEN_VOLATILE is trusted here before the lookup runs. It is a client-supplied flag, and nothing below checks that a volatile ob

Gerrit AI review for Lustre PS28 · 2026-10-31 09:28

(defect) MDS_OPEN_VOLATILE is trusted here before the lookup runs. It is a client-supplied flag, and nothing below checks that a volatile object actually gets created. If the name resolves to an existing regular file that has no layout yet (for example, one the user created with O_LOV_DELAY_CREATE), the lookup-positive branch still applies the client's EA:

    mdt_reint_open()          /* VOLATILE|HAS_EA|FMODE_WRITE, existing name */
      -> mdt_finish_open() -> mdt_mfd_open()   /* isreg, no MA_LOV */
      -> mdt_create_data() -> mdd_create_data()
         /* HAS_EA: dt_xattr_set(XATTR_NAME_LOV, sp_ea.eadata) */

So a client without layout_ops can set any layout on a named file. md_should_create() only needs FMODE_WRITE without DELAY_CREATE/LEASE. A stock client can't send this, but the same is true of the bare lustre.lov setxattr that the patch gates in mdt_reint_setxattr().

Could the exemption apply only when the object is actually created? For example, return -EPERM in the `mdt_object_exists(child)` branch when HAS_EA is set without the role (and this isn't a replay). mot_layout_norole is already set only on the `created` path.

reply PS28 2026-10-31 09:28 Gerrit AI review for Lustre sanity-sec.sh:9158 — (minor) The newest numeric tag in the tree appears to be 2.17.59 (LUSTRE-VERSION-GEN has DEFAULT_VERSION=2.17.60). Wouldn't a 2.17.60.1 gate

Gerrit AI review for Lustre PS28 · 2026-10-31 09:28

(minor) The newest numeric tag in the tree appears to be 2.17.59 (LUSTRE-VERSION-GEN has DEFAULT_VERSION=2.17.60). Wouldn't a 2.17.60.1 gate skip test_64m (and test_64n below) on builds of this patch until 2.17.60 is tagged? Or is the gate meant to be bumped at landing?

owner Sebastien Buisson · uploader Sebastien Buisson · PS28 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-6689 nodemap: add layout_ops rbac role

Add a new 'layout_ops' nodemap rbac role, so that clients can be
prevented from changing the layout of files and directories. When
the role is removed from a nodemap, files created by clients of
that nodemap can only inherit the layout from the default layout
of their parent directory.

The role is enforced on the MDT for following client-driven layout
modifications:
- specifying a layout when opening/creating a file, as done by
'lfs setstripe' on a file, and by the volatile file creation
behind 'lfs mirror extend';
- setting the default layout of a directory, as done by
'lfs setstripe' on a directory;
- modifying layout components via the lustre.lov.{add,del,set}
xattrs, as done by 'lfs setstripe --component-add/--component-del';
- swapping layouts, as done by 'lfs swap_layouts';
- swapping, merging or splitting layouts on close, as done by
'lfs mirror extend/split'.

Using 'lfs migrate' to migrate a file to a new copy that uses the
very same layout is always allowed, so that file data can be moved to
other OST objects without the role. SEL files migrate is always
allowed as well.

HSM import now requires the layout_ops role in addition to the hsm_ops
role, as well as HSM restore if it explicitly brings in a different
layout than what the file had when it was released.

Open replays are let through, because the layout they carry was
already accepted when the request was first processed.

Reading layouts is not affected, and neither is the instantiation
of layout components triggered by regular IO, nor PCC read-only
attach, which both go through a layout intent. The directory layout
in the DNE sense is still governed by the 'dne_ops' role.
PCC read-write attach is not affected either, be it explicit or on
create, as the only layout it sets is an empty released one.

Mirror resync ('lfs mirror resync') is not affected either: it does
not change the composite layout, only the stale flags, FLR state and
layout version of components that already exist, and it is the
counterpart of the mirror staling that regular writes - themselves
not gated - already perform.

Like all other roles, 'layout_ops' is part of the default 'all'
value, so the behavior is unchanged unless it is explicitly
removed from a nodemap.

Add sanity-sec test_64m and test_64n, and make test_64a and test_64i
request the new role where it calls 'lfs setstripe'. Also add
sanity-hsm test_412 to exercise the HSM part.

Test-Parameters: testlist=sanity-sec clientversion=2.16 env=SANITY_SEC_EXCEPT="31"
Test-Parameters: testlist=sanity-sec serverversion=2.16 env=SANITY_SEC_EXCEPT="5 45 47 48b 49 64a 64i"
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ia1cd0cc25cec4c83c41cfc842a244154f826347f

70175 master J –B ✓T ✓R +1/2

LU-20565 tests: run sanity-ec with fewer OSTs

reviewing Andreas Dilger · CI green — awaiting your review

4 unresolved M +31 −29

build #134789 SUCCESS · tests all Maloo sessions

reviews: Keguang Xu +1 2026-10-31 08:37

no vote yet: Ronnie Sahlberg

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 07:22 Andreas Dilger sanity-ec.sh:1501 — Strange. There was a full session with `testlist=sanity-ec ostcount=4` that passed, so either this subtest is not being run, or the stripe c

Gerrit AI review for Lustre PS1 · 2026-10-31 06:54

(minor) test_5a isn't in the list, but it seems to hit the same ENOSPC on 4 OSTs. It hard-codes `-c 4 --ec 4+1` with no OSTCOUNT gate, and a 4+1 raidset needs 5 distinct OSTs. For a single raidset, lod_stripe_count_min() gives the parity component its own cstripe count of unique OSTs, and the reuse/shrink fallbacks in lod_qos.c only run when ec_raidset_count(sc) > 1.

None of the later LU-20565 patches in the relation chain touch 5a either. Will the `ostcount=4` session still fail here? If so, 5a could use `-c $EC_DSTRIPE $EC_ARG` like the other subtests, or get a `(( OSTCOUNT >= 5 )) || skip` gate.

Andreas Dilger PS1 · 2026-10-31 07:22

Strange. There was a full session with `testlist=sanity-ec ostcount=4` that passed, so either this subtest is not being run, or the stripe count for this component is being shrunk to 3+1 or similar with 4 OSTs?

Either way I don't think this is a regression in this patch. It could be fixed if refreshed.

reply PS1 2026-10-31 07:22 Andreas Dilger sanity-ec.sh:1510 — Hmm, this is also checking stripe_count=4, so it isn't clear how this could pass with OSTCOUNT=4?

Andreas Dilger PS1 · 2026-10-31 07:22

Hmm, this is also checking stripe_count=4, so it isn't clear how this could pass with OSTCOUNT=4?

reply PS1 2026-10-31 07:22 Andreas Dilger sanity-ec.sh:1558 — I think that is an anti-feature of the tests that all of the components are aligned on raidset boundaries. That isn't always possible (eg.

Gerrit AI review for Lustre PS1 · 2026-10-31 06:54

(suggestion) This isn't a bug, but with the scaled geometry the 128M and 1G boundaries are no longer a multiple of the row width when EC_DSTRIPE is 3, 5 or 6 (4, 5, 7 and 8+ OSTs, with 1M stripes). For example, 128M % 3M = 2M. The later patches in this series put each PFL boundary in front of an EC component on a multiple of the row width. If the patch is refreshed, should 5b do the same, e.g. `$((32 * EC_DSTRIPE))M` and `$((256 * EC_DSTRIPE))M`, with the second write's seek moved to match?

Andreas Dilger PS1 · 2026-10-31 07:22

I think that is an anti-feature of the tests that all of the components are aligned on raidset boundaries.  That isn't always possible (eg. a 64KiB DoM first component with an EC second component).  The EC calculations should work properly in this case.

reply PS1 2026-10-31 07:22 Andreas Dilger sanity-ec.sh:4496 — Partial-raidset overwrites should also work properly.

Gerrit AI review for Lustre PS1 · 2026-10-31 06:54

(minor) The seek is scaled to the start of row 2, but count=4 isn't scaled. The old test overwrote exactly one full row (seek=8 count=4 with 4x1M rows). Now that only happens when EC_DSTRIPE is 4 or 2, and with 3, 5 or 6 data stripes it is a partial-row overwrite. If the full-row case was intended, count=$EC_DSTRIPE would keep it. Otherwise the commit message's "overwrite from EC_DSTRIPE" could say that only the offset is scaled.

Andreas Dilger PS1 · 2026-10-31 07:22

Partial-raidset overwrites should also work properly.

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20565 tests: run sanity-ec with fewer OSTs

On a filesystem with 4 OSTs, sanity-ec subtests 4e, 5b, 6d, 6e, 6f,
7a, 23a, 23b, 23c, 24a, 25a, 26a and 28a would fail in "lfs setstripe"
with "No space left on device (28)", because they hard-code an EC 4+2
geometry and the raidset allocator needs 6 distinct OSTs for it.

None of these subtests depends on that geometry, so use the EC_ARG,
EC_DSTRIPE and EC_CSTRIPE defaults that enable_ec() scales to OSTCOUNT
for the layout and the expected stripe counts.  Size the sanity-ec/26a
file and overwrite from EC_DSTRIPE so it still covers 5 full rows.

Test-Parameters: trivial testlist=sanity-ec
Test-Parameters: testlist=sanity-ec ostcount=4
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: Ie76079973f5ee62cb77888d448194913693ebbe5

69575 master J –B ✓T ✓R +2/2

LU-20690 utils: add EC parity-stripe I/O helpers

reviewing Keguang Xu · CI green — awaiting your review

pt_ecroraid10-like218_review_priority 1 unresolved L +244 −115

build #134756 SUCCESS · tests all Maloo sessions

reviews: Ronnie Sahlberg +1 2026-10-29 23:40 · Andreas Dilger +1 2026-10-31 07:00

patchsets:

+3 earlier

PS1 10-11 PS2 10-13 PS3 10-14

PS4 10-23 PS5 10-23 PS6 10-23 PS7 10-29 Ronnie Sahlberg PS8 10-29 Ronnie Sahlberg

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS8 2026-10-31 06:07 Gerrit AI review for Lustre lfs.c:17527 — (suggestion) 'lfs mirror write' now refuses a parity mirror, but 'lfs mirror copy' still writes into one. 'lfs mirror copy -i1 -o<parity_id>

Gerrit AI review for Lustre PS8 · 2026-10-31 06:07

(suggestion) 'lfs mirror write' now refuses a parity mirror, but 'lfs mirror copy' still writes into one. 'lfs mirror copy -i1 -o<parity_id>' does, and so does the documented '-o -1' form, because get_other_mirror_ids() collects every mirror including parity. The path is llapi_mirror_copy_many() -> llapi_mirror_write(), which overwrites the parity with raw data. LL_LEASE_RESYNC_DONE then marks that mirror in sync, so it is the same corruption the new write check prevents.

This predates the patch. verify_mirror_id_by_fd() now returns is_parity, though, so this call site could reject an explicit parity target, and the -1 case could skip parity mirrors. If the patch is refreshed, or in a follow-on, consider doing that.

tags: pt_ecro raid10-like 218_review_priority

owner Keguang Xu · uploader Ronnie Sahlberg · PS8 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-20690 utils: add EC parity-stripe I/O helpers

Prepare userspace for the RAID10-like EC parity layout, where the
p parity stripes of a raidset share one file offset and must be
addressed by (pos, parity index) through LL_IO_SCOPE_STRIPE.

Add internal llapi_mirror_read_parity() and
llapi_mirror_write_parity() (lustreapi_internal.h, not lustreapi.h),
which pin the I/O to one parity stripe with LL_IO_SCOPE_STRIPE.  They
share the pread/pwrite loop of llapi_mirror_read() and
llapi_mirror_write().

llapi_ec_write_parities() and llapi_ec_verify_parities() take the
parity component instead of stripe size and id, and do parity I/O
through the _parity helpers.  Offsets are unchanged: parity i is
still at ec_pos + i * stripe_size.  With the current RAID0|PARITY
encoding lov ignores the designated stripe, so resync and verify
behave as before.  The zero-fill fallback of parity punch, the hole
check of verify, and 'lfs mirror read' and 'lfs mirror copy' of a
parity mirror still use mirror I/O; they need the same change
before the RAID10 layout is enabled.

Add llapi_layout_is_parity_mirror() to the public API and use it in
lfs in place of its own is_parity_mirror() and mirror id lookup.
'lfs mirror write' now refuses a parity mirror and points to
'lfs mirror resync'.

Test-Parameters: testlist=sanity-ec ostcount=8
Signed-off-by: Keguang Xu <kxu@ddn.com>
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Assisted-by: Cursor:Grok-4.6/Opus5.5 llm_code_and_review_tools
Assisted-by: ClaudeCode:claude-opus-5-5
Change-Id: I58eae79087e99eb6a5751919a9da17970074c215

69392 master J –B ✓T ✓R 0/2

LU-12669 ec: wait for stripes that parity cannot rebuild

reviewing Andreas Dilger · CI green — awaiting your review

218_review_prioritypt_ecro L +289 −36

failed enforced test	platform	detail
review-dne-zfs-part-1	RHEL 9.8 / x86_64	ran 3 tests. 1 tests failed: sanity.	session

build #134706 SUCCESS · tests all Maloo sessions

no vote yet: Keguang Xu, Ronnie Sahlberg, Zhenyu Xu

patchsets: PS1 10-03 PS2 10-14 PS3 10-28

tags: 218_review_priority pt_ecro

owner Andreas Dilger · uploader Andreas Dilger · PS3 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-12669 ec: wait for stripes that parity cannot rebuild

A read of an EC file whose raid set has more data stripes on stopped
OSTs than parity can rebuild fails with -EAGAIN or -EIO, while the
same read of a file without parity waits for the OSTs to return,
e.g. after an OSS failover.  Adding parity to a file should not make
it less available than a plain striped file.

When CIT_EC_RD finds too many unavailable stripes or too little
parity, set the new cl_io::ci_ec_wait, which cl_io_loop() turns into
a restart, and record that raid set's range.  Restart the read up to
the start of that raid set as CIT_EC_RD, so earlier raid sets are
still rebuilt from parity, then read that raid set as a plain read
without the no-delay flag, which waits for the stripes' OSTs.  The
rest of the read goes back to the no-delay read, so later raid sets
are also rebuilt from parity.  A synchronous direct read whose aio
was freed for the CIT_EC_RD restart gets a new one.  A read of a
deactivated OST still fails at once.  Since waiting for the OSTs is
not an error, print a debug message instead of a console warning.

Change sanity-ec test_29a to check that such a buffered or direct
read waits for its OSTs and then returns the right data, and add
test_29b and test_29c for reads whose second or first raid set is
rebuilt from parity.

Test-Parameters: testlist=sanity-ec ostcount=8
Fixes: 783c13bb0b8f ("LU-12669 ec: recover data from parity")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I9956bab1b63744d0b52d03268b95c78ca73ebbe5

69712 master J –B ✓T ✓R 0/2

LU-20965 llite: change layout before zeroing truncated page

reviewing Sebastien Buisson · CI green — awaiting your review

pw_shepherd M +151 −4

build #134365 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger, Timothy Day

patchsets: PS1 10-15 PS2 10-19 PS3 10-20 PS4 10-23

tags: pw_shepherd

owner Sebastien Buisson · uploader Sebastien Buisson · PS4 uploaded 10d ago · NEW · open in Gerrit ↗

commit message

LU-20965 llite: change layout before zeroing truncated page

When truncating an encrypted file to a size that is not a multiple
of PAGE_SIZE, ll_io_zero_page() initializes a write IO on the page
at the new EOF. If the layout must change before writing to that
page, cl_io_rw_init() returns 1. This happens when the page is in a
PFL component not instantiated yet, or when the file is mirrored and
in sync. ll_io_zero_page() returns this 1 as is, so ll_setattr_raw()
does not truncate OST objects, and ftruncate() returns 1 with the
file size unchanged. It also skips cl_io_fini().

Call cl_io_fini() in that case, so that the layout intent is sent,
and restart the IO if needed, as cl_setattr_ost() does.

ll_io_zero_page() also ignored the designated mirror of the file.
'lfs mirror write' truncates the file with a designated mirror while
it holds a lease, and its truncate was silently skipped because of
the above. With the layout intent now sent, it fails with -ESTALE.
So pass the file to ll_io_zero_page() and set the designated mirror
on the IO, as cl_setattr_ost() does.
This also changes HSM behavior, now a truncate to an unaligned size of
a released encrypted file restores the file, the IO restarts, and
LLIF_DATA_MODIFIED is set so the file is marked dirty.

Finally, ll_io_zero_page() computed the offset of the page to zero
in an unsigned int, which wraps for a page at or beyond 4 GiB. So
compute the offset as a loff_t.

Add sanity-sec test_48e to truncate encrypted files to a size that
is not page aligned:
- extending into an uninstantiated PFL component;
- beyond 4 GiB, while a second mount holds a lockahead lock
- for a mirrored file in sync.
Extend sanity-hsm test_58 to check that truncating an encrypted file
to a size not page aligned restores it and mark it dirty.

Fixes: adf46db962f6 ("LU-12275 sec: support truncate for encrypted files")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Assisted-by: Claude:claude-opus-5-5
Change-Id: I2344a03db488445cf72abc2d1c21df8c31dbee31

69869 master J –B ✓T ✓R 0/2

LU-21005 nodemap: add servergroup property

reviewing Sebastien Buisson · CI green — awaiting your review

pw_shepherd 1 unresolved L +378 −3

build #134646 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger

patchsets: PS1 10-21 PS2 10-22 PS3 10-22 PS4 10-22 PS5 10-28 patrick bot

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS4 2026-10-27 17:01 Sebastien Buisson out_handler.c:956 — This sounds like a legitimate concern, even in case of rolling upgrade, although I cannot really confirm the likelihood of the scenario give

Gerrit AI review for Lustre PS4 · 2026-10-26 12:28

(defect) The commit message says older nodemaps come up with servergroup off and behaviour changes "until servergroup is set". The impact during that window looks worse than refused operations, though.

Targets load the cached nodemap config, including the active flag, at mount: nm_config_file_register_tgt()->nodemap_load_entries(). So on a nodemap-active DNE system that has just been upgraded, every MDT enforces this check right away. The admin can't set the property first: the MGS (usually with MDT0) has to be running the new code, and older targets drop the unknown flags2 bit when they rewrite their cache.

At MDT setup, lod_sub_recovery_thread() reads each peer's update llog through OSP, i.e. OUT_READ/OUT_ATTR_GET:

    lod_sub_prep_llog() -> osp_md_read() -> OUT_UPDATE -> -EPERM

-EPERM is not in the retry list (-ETIMEDOUT/-EAGAIN/-EIO), so the thread calls lod_sub_cancel_llog() and gives up on that peer's update log. Pending distributed transactions are then not replayed. OUT_UPDATE replays queued during target recovery would also fail with -EPERM.

Could enforcement be opt-in, for example only when at least one nodemap in the active config has servergroup set, or behind a global switch? Otherwise an ordinary upgrade seems to end with skipped DNE recovery and an LFSCK.

Sebastien Buisson PS4 · 2026-10-27 17:01

This sounds like a legitimate concern, even in case of rolling upgrade, although I cannot really confirm the likelihood of the scenario given here leading to skipped DNE recovery and LFSCK.
We would need a way to enforce the servergroup property but still allow initial server-to-server connections, at least temporarily. @adilger@thelustrecollective.com do you have an idea how we could do this? Or maybe it means that the servergroup property should not be off by default? (in which case we would probably need to reverse the semantic)

tags: pw_shepherd

owner Sebastien Buisson · uploader patrick bot · PS5 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-21005 nodemap: add servergroup property

Add new nodemap property 'servergroup', off by default. It marks a
nodemap whose members are Lustre servers. When nodemap is active,
targets reply with EPERM to server-only request types that are sent
from peers that belong to a nodemap with this property set to 0.
Requests received on the loopback NID 0@lo can only come from the
server node itself, so they are always accepted.

The servergroup property can be set on regular nodemaps and on the
default nodemap, but not on dynamic nodemaps, and it is never
inherited by new nodemaps. Nodemaps created with an older Lustre
version have it off after upgrade, which changes the behavior of
existing deployments until servergroup is set on the nodemap holding
the server NIDs.

Update lctl-nodemap-modify man page, and add nodemap.servergroup man
page.

Add sanity-sec test_107 to exercise this new property.
In test-framework, set servergroup on the default nodemap and on the
SSK server-to-server nodemap, so that existing tests activating
nodemap keep working with DNE.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Assisted-by: Claude:claude-opus-5-5
Change-Id: I9a0febb71464c74ba241ffd7f8f51afa16cd8068

67507 master J –B ✓T ✓R 0/2

LU-6142 obdclass: add/use OBD_ALLOC_PTR_ARRAY[_LARGE]()

reviewing Andreas Dilger · CI green — awaiting your review

pw_smallfix 1 unresolved L +173 −203

failed enforced test	platform	detail
review-dne-ipv6	RHEL 9.8 / x86_64	ran 4 tests. 1 tests failed: sanity.	session

build #134647 SUCCESS · tests all Maloo sessions

no vote yet: Alex Zhuravlev, Zhenyu Xu, Timothy Day

patchsets:

+5 earlier

PS1 07-24 PS2 07-25 PS3 07-25 PS4 09-28 patrick bot PS5 10-05 patrick bot

PS6 10-05 patrick bot PS7 10-06 patrick bot PS8 10-07 patrick bot PS9 10-27 patrick bot PS10 10-28 patrick bot

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS4 2026-09-28 07:28 patrick bot obd_support.h:868 — Patrick-Bot: Confirmed for RHEL8.10: the rocky8 4.18 kvmalloc_node() still returns kmalloc_node() for flags that are not GFP_KERNEL, so GFP_

Gerrit AI review for Lustre PS3 · 2026-07-27 20:07

Does this keep the vmalloc fallback on the older supported kernels? Before v5.15 (commit a421ef303008 "mm: allow !GFP_KERNEL allocations for kvmalloc") kvmalloc_node() starts with:

    if ((flags & GFP_KERNEL) != GFP_KERNEL)
        return kmalloc_node(size, flags, node);

GFP_NOFS lacks __GFP_FS, so on those kernels this degrades to a plain kmalloc_array() with no vmalloc fallback at all. lustre/ChangeLog still lists RHEL8.10 (4.18), RHEL9.x (5.14), SLES15 SP5 (5.14.21) and openEuler 22.03 (5.10) as supported.

That affects every OBD_ALLOC_LARGE() site converted here - pool_ptrs_alloc() (opp_max_ptr_pages * sizeof(void *) can be hundreds of KiB), desc->bd_enc_vec, desc->bd_vec, lut_reply_bitmap, llog_catalog_list()'s idarray, ldo_comp_entries - which will now return -ENOMEM under fragmentation where OBD_VMALLOC() used to succeed. The `(size) > KMALLOC_MAX_SIZE` guard that OBD_ALLOC_LARGE() carries for LU-8196 is gone too.

Would routing through OBD_ALLOC_LARGE() (with an overflow-checked size_mul()) keep both the overflow protection and the existing fallback?

patrick bot PS4 · 2026-09-28 07:28

Patrick-Bot: Confirmed for RHEL8.10: the rocky8 4.18 kvmalloc_node() still returns kmalloc_node() for flags that are not GFP_KERNEL, so GFP_NOFS kvmalloc_array() has no vmalloc fallback there. The rocky9 tree has the fix. Choosing between OBD_ALLOC_LARGE() with size_mul() and a compat wrapper is a design decision, so this stays open for the owner.

tags: pw_smallfix

owner Andreas Dilger · uploader patrick bot · PS10 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-6142 obdclass: add/use OBD_ALLOC_PTR_ARRAY[_LARGE]()

Reimplement OBD_ALLOC_PTR_ARRAY() and
OBD_ALLOC_PTR_ARRAY_LARGE() using the kernel
k[v]malloc_array() functions instead of open-coding
array sizes, and add OBD_ALLOC_ARRAY_GFP() and
OBD_ALLOC_ARRAY_LARGE_GFP() as the underlying array
allocators.  These simplify callers and avoid potential
bugs with overflow of calculations with 32-bit variables.

Convert OBD_ALLOC() and OBD_ALLOC_LARGE() and corresponding
OBD_FREE[_LARGE]() callers to the new macros where possible.
Callers converted to OBD_ALLOC_PTR_ARRAY_LARGE() now get the
vmalloc fallback of kvmalloc_array() rather than the
open-coded kmalloc-then-vmalloc of OBD_ALLOC_LARGE().

Remove OBD_ALLOC_WAIT(), OBD_ALLOC_PTR_WAIT() and
OBD_ALLOC_PTR_ARRAY_WAIT(); their few callers now use
OBD_ALLOC_GFP(..., GFP_KERNEL) directly.  Update
leak_finder.pl to parse the new allocation labels.

Change lds_def_comp_size_cnt to hold the maximum count of
entries in the lds_def_comp_entries array, rather than the
requested number of entries.  This avoids recalculating
the array size on each call so lod_free_def_comp_entries()
and lod_free_comp_buffer() can use OBD_FREE_PTR_ARRAY_LARGE().

Cleanup code style issues in related code.

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-sonnet-5
Change-Id: Ie0db250d017d228606d4d976132968d138500c1e

68764 master J –B ✓T ✓R 0/2

LU-20724 tests: add fsx option to drop client caches

reviewing Patrick Farrell · CI green — awaiting your review

pw_own M +171 −12

build #134561 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger, Oleg Drokin, Jian Yu, Timothy Day

patchsets:

+1 earlier

PS1 09-10

PS2 09-10 PS3 09-19 patrick bot PS4 09-20 patrick bot PS5 09-20 patrick bot PS6 10-27 patrick bot

tags: pw_own

owner Patrick Farrell · uploader patrick bot · PS6 uploaded 5d ago · NEW · open in Gerrit ↗

commit message

LU-20724 tests: add fsx option to drop client caches

-f only calls msync(MS_INVALIDATE), which leaves the DLM
locks in place, so the client still serves reads from
cache.

Add -Q P: a 1 in P chance per operation of syncfs() plus
writing "clear" to ldlm.namespaces.*.lru_size, cancelling
the unused locks and discarding the pages they covered.
Only client (osc and mdc) namespaces have an LRU to clear.
The sync is required: clearing the lru does not reliably
drop the cache with dirty pages still around.

A drop takes one mountpoint and a random subset of its
namespaces, leaving the rest of the cache to go stale
rather than making every mountpoint consistent at once.
The namespaces of a mountpoint are the ones named for its
client instance, so they are resolved once per path and
kept.

Clearing the lru needs root, so prove it works before
running any operations, and fail the run if it stops
working later, rather than finishing a run that never
dropped a cache.  The draws in test() only run when -Q is
given, so a seed replays the same without it, and they do
run while a -b replay is still simulating even though the
drop itself does not, so the replay follows the same
stream of random values.

Drops are logged to the oplog so logdump() shows them.
Take the operation number in that dump from the log entry
rather than counting entries.  That was already wrong
before this: a selected op that does no work returns from
test() without logging anything, so the count had drifted
from testcalls, which is what -b replays and what gendata()
stamps into the data.

Add -Q to sanityn test_16l.  Hoist the getopt option
string into a variable to keep it under 80 columns.

Assisted-by: ClaudeCode:claude-opus-5
Signed-off-by: Patrick Farrell <patrick@thelustrecollective.com>
Change-Id: I55fda8f3b4979e44982ba53ed5447b32d280a2b1

64927 master J –B ✓T ✓R 0/2

LU-20016 utils: add an EC computation benchmark

reviewing Ronnie Sahlberg · CI green — awaiting your review

ec2 pw_bigfixpt_ecro L +620 −2

build #134551 SUCCESS · tests all Maloo sessions

no vote yet: patrick bot, Hiroshi Nishida, Maximilian Dilger, Andreas Dilger, Timothy Day, Patrick Farrell

patchsets:

+11 earlier

PS1 03-26 PS2 03-27 PS3 03-27 PS4 05-13 Andreas Dilger PS5 05-13 Andreas Dilger PS6 05-13 Janitor Bot PS7 09-14 PS8 09-15 PS9 10-13 patrick bot PS10 10-13 patrick bot PS11 10-15 patrick bot

PS12 10-17 patrick bot PS13 10-24 patrick bot PS14 10-26 patrick bot PS15 10-26 patrick bot PS16 10-27 patrick bot

tags: pw_bigfix pt_ecro

owner Ronnie Sahlberg · uploader patrick bot · PS16 uploaded 6d ago · NEW · open in Gerrit ↗

commit message

LU-20016 utils: add an EC computation benchmark

Add ec_perf_bench to measure EC encode throughput and
compare systems and builds. It reports source data rates per
thread and in aggregate, along with the CPU model, speed,
online logical CPU count and EC implementation.

Note that in the ISA-L library, computing parities and
recovering lost data stripes is the same operation:
ec_encode_data(). The difference is not in the computations
but rather the content of the coefficient matrix.

The cost of these operations is primarily bound by the
number of stripes to compute which amusingly means that
recovering a single lost stripe should be FASTER than
generating two parities.

Build erasurecode before the rest of utils and add an
explicit rule for erasurecode/libec.a, so ec_perf_bench's
dependency on the static library is honored under parallel
make. Build libec.a with -fPIC: ec_perf_bench is the first
program linked against it with the PIE hardening flags.

Add sanity-ec test_0b, which runs ec_perf_bench for a few
geometries and logs the CPU, implementation and throughput.

Link the benchmark explicitly with libpthread, including when libcfs
is configured without pthread support.

Test-Parameters: trivial testlist=sanity-ec
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I61f409b631a8f4aefe0ceb2628e56f58f4c3a07c
Assisted-by: Codex:gpt-6.1-sol
Assisted-by: ClaudeCode:claude-sonnet-5
Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-sonnet-5-5
Signed-off-by: Patrickbot <patrick-bot@mulberrytree.us>

70065 master J –B ✓T ✓R +1/2

LU-21042 ldlm: revoke export locks with a blocking AST

reviewing Andreas Dilger · CI green — awaiting your review

L +360 −25

build #134522 SUCCESS · tests all Maloo sessions

reviews: Sebastien Buisson +1 2026-10-28 15:02

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 6d ago · NEW · open in Gerrit ↗

commit message

LU-21042 ldlm: revoke export locks with a blocking AST

ldlm_revoke_export_locks() is called on nodemap changes to make
clients drop cached permissions.  It cancelled the locks of the
export on the server only, as if the client had been evicted, so
the client kept its locks and its cached dentries and attributes.
Later changes by other clients or through another MDT no longer sent
a blocking AST to the client, and it would use stale data.  E.g.
after sanity-hsm test_411 changed a nodemap, test_500 would create a
directory on a remote MDT, and then stat would return -ENOENT from a
stale negative dentry.

Send a blocking AST for each granted IBITS and EXTENT lock of the
export instead, so the client cancels the locks and drops its cache.
A client that does not cancel is evicted by the lock callback
timeout.  Flock and group locks are not revoked.

The nodemap code calls ldlm_revoke_export_locks() with nodemap locks
held, so it must not wait for client replies.  Queue the export with
a reference to the ldlm_bl threads instead, which take the locks from
exp_lock_hash and send the ASTs.  An export that is already queued is
not queued again, and an export that is evicted or disconnected
before the revoke runs is skipped, since its locks are cancelled
without ASTs.  nodemap_modify and other nodemap changes return before
the revocation is complete.

The revoke ASTs are still sent synchronously from the ldlm_bl thread,
which waits for the replies, so a dead client holds that thread until
it is evicted.  Limit revoke work to about half of the maximum number
of ldlm_bl threads, so that dead clients cannot tie up the pool
needed for evictions and other blocking callbacks.  Sending the ASTs
asynchronously through ptlrpcd is left for later, since
ldlm_cb_interpret() would need an lu_env there.

Extend sanity-sec test_82 to check that a nodemap change makes the
client cancel its MDT0000 locks without an eviction.

Test-Parameters: trivial testlist=sanity-sec
Test-Parameters: mdscount=2 mdtcount=4 testlist=sanity-hsm
Fixes: 64a0a6c4559b ("LU-19157 nodemap: revoke export locks on nodemap change")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: I271021c58f6ae916c10ca3d0f37683a0503ebbe5

69916 master J –B ✓T ✓R +2/2

LU-21014 lod: keep EC raid sets on distinct OSTs in RR

reviewing Ronnie Sahlberg · CI green — awaiting your review

ec2 pt_ecro CI@PS2 master-next 5 unresolved S +8 −2

build #134322 SUCCESS · tests all Maloo sessions

reviews: Keguang Xu +1 2026-10-28 12:00 · Andreas Dilger +1 2026-10-27 02:27

patchsets: PS1 10-22 PS2 10-23 PS3 10-27 Andreas Dilger

threads: 5 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS2 2026-10-24 10:03 Gerrit AI review for Lustre COMMIT_MSG:19 — (minor) This fixes a placement bug from the commit that added the EC RR reuse path, lod_ec_add_data() and the lag_ost_avail gate in lod_chec

Gerrit AI review for Lustre PS2 · 2026-10-24 10:03

(minor) This fixes a placement bug from the commit that added the EC RR reuse path, lod_ec_add_data() and the lag_ost_avail gate in lod_check_and_reserve_ost(). That commit's message promises "-C: require unique OSTs within each raidset", which the RR loop reuse did not keep. Should this carry:

Fixes: c53abca9f9ee ("LU-19852 lod: raidset aware stripe allocator")

reply PS2 2026-10-27 02:27 Andreas Dilger lod_qos.c:779 — This says where EC reuse happens but not why it can't happen here. Please add that lag_ost_avail counts OSTs that RR skips (inactive, degra

Andreas Dilger PS2 · 2026-10-27 02:27

This says where EC reuse happens but not why it can't happen here.  Please add that lag_ost_avail counts OSTs that RR skips (inactive, degraded), so the loop could reuse an OST before every usable OST was placed.  Can go in the follow-on patch.

reply PS2 2026-10-27 02:27 Andreas Dilger lod_qos.c:1463 — Pre-existing and not critical for this release: lod_ec_add_data() only checks failure domains within the raid set, not OST identity. An OST

Andreas Dilger PS2 · 2026-10-27 02:27

Pre-existing and not critical for this release: lod_ec_add_data() only checks failure domains within the raid set, not OST identity.  An OST with failure domain 0 skips the check entirely.  When the list mixes domain 0 and non-zero domain OSTs, a "continue" here for a domain collision shifts the cyclic order, so a domain 0 OST can be placed twice in one raid set.  Separately, if lod_ec_add_data() returns short, the parity split is computed from the reduced stripe count, which may not keep the uniqueness the original split had.  Something for a later patch.

reply PS2 2026-10-27 02:27 Andreas Dilger lod_qos.c:1768 — With this change EC components never set overstriped in the RR loop, so LOV_PATTERN_OVERSTRIPING is always cleared, even when lod_ec_add_dat

Andreas Dilger PS2 · 2026-10-27 02:27

With this change EC components never set overstriped in the RR loop, so LOV_PATTERN_OVERSTRIPING is always cleared, even when lod_ec_add_data()/lod_ec_add_parities() do reuse OSTs afterward.  "lfs getstripe" then shows a -C EC layout with repeated OSTs but no overstriping.  This already happened with all OSTs active, and now also with an OST down.  Setting overstriped when lod_ec_add_data()/lod_ec_add_parities() place a reused OST would fix it.  Also, clearing the flag before an error return means the reserve=0 retry runs as a -c layout.  Not critical, could go in the follow-on patch.

reply PS2 2026-10-28 12:00 Keguang Xu lod_qos.c:2831 — +1

Andreas Dilger PS2 · 2026-10-27 02:27

lod_ost_alloc_qos() has the same issue: for an EC component with overstriping, QoS still reuses an OST at any point in its weighted random loop, and with failure domain 0 nothing stops two stripes of one raid set from landing on the same OST, even with all OSTs up.  The same "&& !lod_comp->llc_cstripe_count" check here would fix it, since QoS already calls lod_ec_add_data()/lod_ec_add_parities() afterward.  LU-9 weighted RR QoS allocation also fixes this, but it is unlikely to land for 2.18.0, so please fix it in a follow-on patch for 2.18.0.

Keguang Xu PS2 · 2026-10-28 12:00

+1

tags: pt_ecro

build/test results are from PS2 — PS3…PS3 changed only the commit message (votes copied, CI not re-run)

owner Ronnie Sahlberg · uploader Andreas Dilger · PS3 uploaded 6d ago · NEW · open in Gerrit ↗

commit message

LU-21014 lod: keep EC raid sets on distinct OSTs in RR

With overstriping, lod_check_and_reserve_ost() lets the RR loop reuse
an OST while lag_ost_avail says unique OSTs remain. That count includes
OSTs RR skips, such as inactive or degraded ones, so an EC data
component can reuse an OST before lod_ec_add_data() takes over, which
repeats the placed stripes assuming they are unique and so puts two
stripes of a raid set on one OST, e.g. -C 13 --ec 5+2 on 8 OSTs with
one down.

Leave all OST reuse for EC components to lod_ec_add_data() and
lod_ec_add_parities(), which keep each raid set on distinct OSTs.

Test-Parameters: trivial testlist=sanity-ec ostcount=8
Fixes: c53abca9f9ee ("LU-19852 lod: raidset aware stripe allocator")
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Assisted-by: ClaudeCode:claude-opus-5.5
Change-Id: I8fe6431de113351e92653780fe17db2e8a180697

69421 master J –B ✓T ✓R +2/2

LU-19823 lod: balance RR allocation across failure domains

reviewing Andreas Dilger · CI green — awaiting your review

218_review_prioritypt_ecro master-next 7 unresolved XL +982 −177

build #134288 SUCCESS · tests all Maloo sessions

reviews: Keguang Xu +1 2026-10-26 08:02 · Ronnie Sahlberg +1 2026-10-22 16:39

patchsets: PS1 10-05 PS2 10-10 PS3 10-10 PS4 10-13 PS5 10-22

threads: 7 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-10-11 05:27 Gerrit AI review for Lustre lod_qos.c:1665 — (defect) What happens when every OST that lu_rr_fdom_tgt() offers is rejected? This loop then calls lu_rr_fdom_tgt() exactly lrf_count times

Gerrit AI review for Lustre PS3 · 2026-10-11 05:27

(defect) What happens when every OST that lu_rr_fdom_tgt() offers is rejected? This loop then calls lu_rr_fdom_tgt() exactly lrf_count times. The domain counter advances by a full cycle and ends up where it started. The OST returned is the last one tried, which is lrfs_tgts[(next - 1) % lrf_count]: the OST most recently handed out from that counter. So the rotation doesn't move, and each such stripe lands on the same OST as the previous consumer of the counter.

This happens whenever all OSTs of a domain are on OSSs in lag_oss_avoid_array. lod_collect_avoidance() doesn't skip components whose llc_id is still LCME_ID_INVAL at create time. So for EC parity, the avoid list holds the OSSs of the file's own data stripes, not just those of other FLR mirrors. Some cases:

- one OSS with several domains, FLR or EC
- an HA pair whose OSTs share enclosures as domains, with EC data stripes on both OSSs

With the patch's own lod_rr_test harness, which has the same loop:

-f -o 4-16 -s 1 -d 2-4 -D -M 43 of 52 combinations fail
(last OST of each domain gets 5x the others)
-f -o 6-16 -s 1 -d 3-4 -D -e 1 47 of 55 fail
-o 12 -s 2 -d 3 -D -e 1 -c 2 OST7 450 objects, OST4 200, same domain

This looks different from LU-20925. That ticket covers the skew when only some of a domain's OSTs are avoided, while here the rotation stops completely, and it also hits EC parity, which has no mirrors. In the harness, taking one more lu_rr_fdom_tgt() after the loop when nothing qualified (so the counter advances by one net) brings the first two runs to 0 failures. It also leaves the existing -d, -d -M and -d -D -M results unchanged or better. Another option is to pass an offset to lu_rr_fdom_tgt(), so the probe doesn't touch the shared counter.

reply PS5 2026-10-26 08:02 Keguang Xu lod_qos.c:616 — (style) lod_oss_avoid_some -> lod_flr_has_unused_oss?

Keguang Xu PS5 · 2026-10-26 08:02

(style) lod_oss_avoid_some -> lod_flr_has_unused_oss?

reply PS5 2026-10-26 08:02 Keguang Xu lod_qos.c:632 — (style) lod_oss_avoided -> lod_flr_oss_used_by_peer ?

Keguang Xu PS5 · 2026-10-26 08:02

(style) lod_oss_avoided -> lod_flr_oss_used_by_peer ?

reply PS5 2026-10-26 08:02 Keguang Xu lod_qos.c:645 — (style) lod_fdom_tgt_skip -> lod_rr_fdom_ost_unusable?

Keguang Xu PS5 · 2026-10-26 08:02

(style) lod_fdom_tgt_skip -> lod_rr_fdom_ost_unusable?

reply PS5 2026-10-26 08:02 Keguang Xu lod_qos.c:1587 — (style) slot_ost_fallback?

Keguang Xu PS5 · 2026-10-26 08:02

(style) slot_ost_fallback?

reply PS4 2026-10-24 19:52 Andreas Dilger lod_rr_test.c:245 — This is only the test so efficiency is not the primary concern, but could be updated if patch is refreshed.

Gerrit AI review for Lustre PS4 · 2026-10-14 01:52

(minor) This loop calls oss_used(), which itself scans every OST, once for every OST, on every rr_alloc() call. It doesn't stop at the first unused OSS, and it also runs without failure domains (lqr_fdoms == NULL), where oss_avoid is never used. So every lod_rr_test invocation now does O(OSTs^2) work per layout, including the ones that existed before this patch.

Built locally with gcc -O2:

    -o 3-64 -s 1-4         5.8s -> 93s
    -o 3-48 -s 2-8 -M      2.9s -> 31s
    pre-existing 8 runs    22s  -> 245s
    full test_51g                  ~9 min

That will take much longer on CI VMs, and much longer still when the subtest-change sessions loop test_51g.

Gating the loop as lod_ost_alloc_rr() does, and stopping at the first match, brings each run back to about 5-10s with the same results (0 failed combinations for each test_51g case I reran):

    if (!rt->no_first && lqr->lqr_fdoms)
            for (i = 0; i < rt->ost_count && !oss_avoid; i++)

Andreas Dilger PS4 · 2026-10-24 19:52

This is only the test so efficiency is not the primary concern, but could be updated if patch is refreshed.

reply PS5 2026-10-24 19:52 Andreas Dilger sanity-ec.sh:5693 — Can be fixed if patch needs to be refreshed.

wc-checkpatch PS5 · 2026-10-22 16:54

(style)  'comma_list $(mdts_nodes' may be misspelled - perhaps 'mdts_nodes'?

Andreas Dilger PS5 · 2026-10-24 19:52

Can be fixed if patch needs to be refreshed.

tags: 218_review_priority pt_ecro

owner Andreas Dilger · uploader Andreas Dilger · PS5 uploaded 10d ago · NEW · open in Gerrit ↗

commit message

LU-19823 lod: balance RR allocation across failure domains

The RR array interleaves OSTs only by OSS, and EC and FLR layouts
that need their stripes in different failure domains skip the OSTs
of domains they already use.  Each skip advances the shared start
index, so the layouts lock into a pattern that uses some OSTs much
more than others and some not at all, e.g. 24 OSTs on 6 OSSs in 3
failure domains with 3-stripe EC layouts get 0-401 objects per OST.

Order the RR array in lu_rr_fill() by failure domain, then by OSS in
each domain, so that consecutive OSTs are in different failure
domains, unless OSSs have OSTs in several domains, when it is ordered
by OSS as before so consecutive OSTs are still on different OSSs.
Stagger OSSs and domains with the same OST count so that their OSTs do
not fall next to each other, and swap apart any OSTs still next to one
of their own OSS or domain, as with OSSs of different sizes.  Domains
of different sizes still need skips, so layouts that need distinct
domains now scan from a start reserved by lu_rr_reserve() without
advancing the shared index, use each array slot only to pick a failure
domain, and take the OSTs of each domain in turn in lu_rr_fdom_tgt(),
separately for the first stripes of each stripe count and component,
and likewise for the OSTs of each OSS without a failure domain,
skipping any the layout already uses.  Layouts that use several OSTs
of an OSS without a failure domain can still be up to about 1.3x
unbalanced within it, but a failure domain should be set for all OSTs.
A domain offers only one of its OSTs per slot, so if a layout is still
short of stripes, try the slot OSTs themselves before accepting fewer.
EC parity takes the OSTs of each domain like the later stripes, so it
does not shift the first stripes of the data.  FLR mirrors prefer the
OSTs of a domain on an OSS that the other mirrors do not use yet,
which can unbalance domains whose OSTs are on several OSSs (LU-20925).
Rebuild the RR arrays when an OST failure domain changes, counted in
lq_rr_gen, since it is only known once the OST has reported statfs.

Add failure domains to lod_rr_test and sanity test_51g, and add
sanity-ec test_35c to check RR balance with unequal domains.  Test
of lod_rr_test with 3-64 OSTs, 1-16 OSSs, 1-8 failure domains and
up to one stripe per domain, comparing OSTs in the same domain:

              combinations   24 OSTs, 6 OSSs, 3 domains, 3 stripes
              failed         objects per OST   #0 objects per OST
  before      16344/22165    0-401             0-399
  after           0/22165    300               100

sanity-ec test_35c, 8 OSTs in domains 1,1,1,1,2,3,2,3, 160 EC 2+1
files with qos_threshold_rr=100, objects per OST:

          OST0  OST1  OST2  OST3  OST4  OST5  OST6  OST7
  before    80    79     1     0    81    81    79    79
  after     41    41    39    39    80    80    80    80

Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: I7111ce7a05f6431ceeaba20213d3c3921e3ebbe5

69649 master J –B ✓T ✓R +1/2

LU-20860 doc: add more agent workflow guidance

reviewing Andreas Dilger · CI green — awaiting your review

18 unresolved L +268 −104

build #133712 SUCCESS · tests all Maloo sessions

reviews: Artem Blagodarenko +1 2026-10-15 13:40

no vote yet: Hiroshi Nishida, Patrick Farrell

patchsets: PS1 10-13 PS2 10-14

threads: 18 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-14 22:50 Andreas Dilger SKILL.md:65 — This could be more informative:

Andreas Dilger PS1 · 2026-10-14 22:50

This could be more informative:

    Code that should disappear in a later release, because it is intended
    as a short-term workaround for some bug in the code should be wrapped in
    `#if LUSTRE_VERSION_CODE < OBD_OCD_VERSION(X, Y, 53, 0)`, where Y is
    typically +4 or more from the current version (assume one release per year,
    and 2.20 may become 3.0).  It is compiled out silently once that version
    is reached, so can be found by grepping for these blocks when preparing
    for a release or for removal after that version has passed.  Compile test
    by increasing the build version beyond the release.

reply PS1 2026-10-14 22:50 Andreas Dilger SKILL.md:157 — add `... and autoconf/configure`

Andreas Dilger PS1 · 2026-10-14 22:50

add `... and autoconf/configure`

reply PS1 2026-10-14 22:50 Andreas Dilger SKILL.md:163 — Add:

Andreas Dilger PS1 · 2026-10-14 22:50

Add:

    - The new kernel API should be used directly in the code, and the compat
      macro should hide the difference, unless this is not possible due to
      missing (not extra) function arguments in the new API, otherwise use.
    - If the compat macro is small, define it directly in autoconf, otherwise
      add compat function(s) and map to it with #ifdef/#define in the header.

reply PS2 2026-10-28 21:47 Andreas Dilger SKILL.md:178 — That patch will be updated soon.

Gerrit AI review for Lustre PS2 · 2026-10-17 10:38

(defect) On current master this doesn't seem to be true. The build adds `-I$PWD/include` (config/lustre-toolchain.m4), not `include/lustre_compat`, so `#include <linux/fs.h>` gets only the kernel header. The tree has about 133 explicit `#include <lustre_compat/linux/...>` lines, including in lnet/. The bullet above still says `<lustre_compat/linux/fs.h>` wraps `<linux/fs.h>`. Shadowing kernel headers is what the open change https://review.whamcloud.com/67519 ("LU-18687 build: normalize lustre_compat headers") proposes, but it hasn't landed. An agent following this today would include the plain Linux header and miss the compat helper, which builds on new kernels and breaks on older ones. Could this describe the current practice until 67519 lands? The `<lustre_compat/...>` group could also go back into the include order at line 149; it was dropped there, so explicit compat includes now have no defined place.

Andreas Dilger PS2 · 2026-10-28 21:47

That patch will be updated soon.

reply PS2 2026-10-28 21:47 Andreas Dilger SKILL.md:183 — can be fixed if patch is refreshed

Gerrit AI review for Lustre PS2 · 2026-10-17 10:38

(typo) A word seems to be missing: "prefer new compatibility wrappers that transparently map to existing kernel functions".

Andreas Dilger PS2 · 2026-10-28 21:47

can be fixed if patch is refreshed

reply PS2 2026-10-28 21:47 Andreas Dilger SKILL.md:214 — not critical. if o2ib is built by default, that is OK, but some systems may need to specify the path explicitly

Gerrit AI review for Lustre PS2 · 2026-10-17 10:38

(minor) `--with-o2ib` defaults to "yes" in config/lustre-lnet.m4, so o2iblnd is already enabled by default. It is skipped (with only an AC_MSG_WARN) when the kernel has no in-kernel IB support and no OFED path is found. Would it be clearer to say "check that configure reports o2iblnd will be built; use `--with-o2ib=<OFED path>` if needed"?

Andreas Dilger PS2 · 2026-10-28 21:47

not critical. if o2ib is built by default, that is OK, but some systems may need to specify the path explicitly

reply PS1 2026-10-14 22:50 Andreas Dilger SKILL.md:74 — NIDs like 192.168.x.y, 10.x.y.z are OK and non-identifying.

Andreas Dilger PS1 · 2026-10-14 22:50

NIDs like 192.168.x.y, 10.x.y.z are OK and non-identifying.

reply PS1 2026-10-14 22:50 Andreas Dilger SKILL.md:74 — The non-LU ticket IDs and Jira should be split into a separate sentence, adding a caveat "... for patches submitted to a mainline/public rep

Andreas Dilger PS1 · 2026-10-14 22:50

The non-LU ticket IDs and Jira should be split into a separate sentence, adding a caveat "... for patches submitted to a mainline/public repo, other Jira identifiers may be allowed for private vendor repos/branches."

reply PS1 2026-10-14 22:50 Andreas Dilger SKILL.md:161 — add `comments, `

Andreas Dilger PS1 · 2026-10-14 22:50

add `comments, `

reply PS2 2026-10-19 16:43 Andreas Dilger SKILL.md:24 — Preferably `... the parent's LU ticket, then abandoned).` so it still fits in 80 columns

Gerrit AI review for Lustre PS2 · 2026-10-17 10:38

(typo) "which should use their parent LU ticket then abandoned" doesn't parse. Maybe "which should use their parent's LU ticket and then be abandoned"?

Andreas Dilger PS2 · 2026-10-19 16:43

Preferably `... the parent's LU ticket, then abandoned).` so it still fits in 80 columns

reply PS2 2026-10-19 16:43 Andreas Dilger SKILL.md:60 — The problem statement should be written in the past tense ("was previously", "held", "locked", "called", ...) for the behavior that is chang

Andreas Dilger PS2 · 2026-10-19 16:43

The problem statement should be written in the past tense ("was previously", "held", "locked", "called", ...) for the behavior that is changed by the patch, since the old (broken) behavior is no longer true once the patch is landed.

Otherwise it is confusing for the reader that may misinterpret the problem statement as a declaration of what the patch is itself implementing.

reply PS1 2026-10-14 22:50 Andreas Dilger SKILL.md:163 — Add:

Andreas Dilger PS1 · 2026-10-14 22:50

Add:

    - Additional CI testing can be requested on an existing build if needed
      to identify CI-only test failures, as long as the build remains in the
      cache and only modifies parameters accessible via `Test-Parameters:`,
      such as arch, distro, {mds,mdt,oss,ost}count, fstype, testlist, env
      variables (e.g. test-specific or global like debug masks and size),
      by submitting a `Test-Parameters:` line for each new session with the
      `Maloo Results` URL posted by Maloo for each build.

owner Andreas Dilger · uploader Andreas Dilger · PS2 uploaded 18d ago · NEW · open in Gerrit ↗

commit message

LU-20860 doc: add more agent workflow guidance

Add guidance that came up in recent patch work: when to start looking
for existing work, how to split and restack patches, which CI
follow-up an agent may do on its own patches, how to push debugging
patches, how to request extra CI sessions on a build, how to write
kernel compat code, fault injection and cheap, well-sized subtests,
which ZFS version to build against, and to leave master-next changes
alone.

Also address review comments on the first patch: link the ltvm repo
under github.com/lustre, note that auster leaves the filesystem
mounted after a failure, restrict skip_env to required packages, add
build, test and commit-msg traps found by other developers, and ask
for plain English.  Add Marc Vef as a reviewer.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5
Change-Id: I9bfaad9c4e0370bea7103ba2e69649fe883ebbe5

63330 master J –B ✓T ✓R -1/2−

LU-19403 llapi: Add device functions for nodemap

reviewing Nikos Papakonstantinou · CI green — awaiting your review (needs rebase first)

needs rebase 6 unresolved L +385 −50

CI green — awaiting your review (needs rebase first)

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked

build #133513 SUCCESS · tests all Maloo sessions

no vote yet: James Simmons, Qian Yingjin, Sebastien Buisson, Andreas Dilger

patchsets:

+18 earlier

PS1 01-12 PS2 01-28 PS3 01-28 PS4 02-09 PS5 02-10 PS6 02-12 PS7 02-12 Janitor Bot PS8 02-13 PS9 02-18 PS10 02-24 PS11 02-25 PS12 02-26 PS13 03-13 PS14 03-14 PS15 03-14 Janitor Bot PS16 03-14 PS17 03-31 PS18 07-28

PS19 08-10 PS20 08-21 PS21 09-16 PS22 09-17 PS23 10-13

threads: 6 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS22 2026-09-18 09:31 Gerrit AI review for Lustre COMMIT_MSG:19 — (minor) The body accounts for the three device helpers, the new test binary and the run_tests() change, but two other pieces of the diff are

Gerrit AI review for Lustre PS22 · 2026-09-18 09:31

(minor) The body accounts for the three device helpers, the new test binary and the run_tests() change, but two other pieces of the diff are not mentioned: the llapi_yaml_get_device_index() changes in liblustreapi_param.c, and the new llapi_strscpy() helper in lustreapi_internal.h.

llapi_strscpy() has no caller in this patch (it is picked up by later changes in the relation chain), so a reader landing on this commit has no way to tell why it appeared here. A sentence for each would help, or would they sit better in the parent patch that introduced llapi_obdname2devno()?

reply PS23 2026-10-14 08:31 Gerrit AI review for Lustre COMMIT_MSG:14 — (minor) "Fix an errno logged but not returned in their error path" reads like a change from an earlier patchset of this change rather than a

Gerrit AI review for Lustre PS23 · 2026-10-14 08:31

(minor) "Fix an errno logged but not returned in their error path" reads like a change from an earlier patchset of this change rather than against master. On master llapi_get_*_device() don't exist yet, and the obd.c helpers they replace never logged an errno. They printed "This command must be run on the MGS." and returned -ENODEV. If the patch is refreshed, could this sentence be dropped, or reworded to describe the actual behavior (an error is logged via llapi_error() and the same errno is returned)?

reply PS22 2026-09-18 09:31 Gerrit AI review for Lustre llapi_nodemap_test.c:29 — (nit) If the patch is refreshed: the comment promises more than the helper checks. llapi_obdname2devno() only returns early when the ioctl g

Gerrit AI review for Lustre PS22 · 2026-09-18 09:31

(nit) If the patch is refreshed: the comment promises more than the helper checks. llapi_obdname2devno() only returns early when the ioctl gives -EINVAL; for any other ioctl error it falls through to llapi_yaml_get_device_index(), so a true result here can come entirely from the Netlink path with the ioctl never having worked.

"Check that the MGS device number can be resolved" would match what it does.

reply PS22 2026-09-18 09:31 Gerrit AI review for Lustre llapi_nodemap_test.c:130 — (minor) This one has no ASSERTF(), so it always passes whatever it prints - it records the node's roles in the log but cannot detect a regre

Gerrit AI review for Lustre PS22 · 2026-09-18 09:31

(minor) This one has no ASSERTF(), so it always passes whatever it prints - it records the node's roles in the log but cannot detect a regression.

Since the six values it prints are related, could it assert the relationship instead? On a node where is_mgs() is true, llapi_get_mgs_device() must succeed, so

    ASSERTF(!mgs || mgs_device_accessible(), ...)

and the MDS/OSS equivalents would turn this into real coverage.

reply PS23 2026-10-14 08:31 Gerrit AI review for Lustre llapi_nodemap_test.c:77 — (suggestion) run_tests() forks each test, so every child starts with an empty static cache. As a result, nothing in the binary exercises the

Gerrit AI review for Lustre PS23 · 2026-10-14 08:31

(suggestion) run_tests() forks each test, so every child starts with an empty static cache. As a result, nothing in the binary exercises the "cache after first successful lookup" path that the commit message describes. If the patch is refreshed, consider calling llapi_get_mgs_device() a second time within this test and asserting that it returns the same device number.

reply PS23 2026-10-14 08:31 Gerrit AI review for Lustre liblustreapi_ioctl.c:285 — (minor) Only -EINVAL is remapped, so "device not present" can now come back as different errnos depending on which lookup path answered:

Gerrit AI review for Lustre PS23 · 2026-10-14 08:31

(minor) Only -EINVAL is remapped, so "device not present" can now come back as different errnos depending on which lookup path answered:

    ioctl(OBD_IOC_NAME2DEV) not found      -> -EINVAL -> -ENODEV
    llapi_yaml_get_device_index() not found -> -ENOENT
    /dev/obd unopenable, no "lustre" genl  -> -EOPNOTSUPP

Before this patch, get_mgs_device(), get_mds_device() and get_oss_device() in obd.c always returned -ENODEV. So, for example, `lctl barrier_freeze` on a node without the Lustre modules loaded now fails with EOPNOTSUPP instead of ENODEV.

Should the helpers return -ENODEV for every lookup failure, or should the kernel-doc spell out which errnos callers can expect? Same in the MDS and OSS variants below.

owner Nikos Papakonstantinou · uploader Nikos Papakonstantinou · PS23 uploaded 20d ago · NEW · open in Gerrit ↗

commit message

LU-19403 llapi: Add device functions for nodemap

Nodemap lctl commands look up the local MGS/MDS/OSS device
number, but that logic lived only in obdctl.

Add llapi_get_mgs_device()/get_mds_device()/get_oss_device()
wrappers around llapi_obdname2devno(), caching the device number
after the first successful lookup. Fix an errno logged but not
returned in their error path.

Add the llapi_nodemap_test binary and sanity-sec test_92;
run_tests() gains a NULL @lustre_dir case so this server-only
binary can share it instead of duplicating the fork/report loop.

Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com>
Change-Id: I095265bdccc8efd441c6c396c8ad267d1f6b7ca9

68840 master J !B ✓T ✓R 0/2

LU-18511 gss: carry out negotiation again on empty ccache

reviewing Sebastien Buisson · CI green — awaiting your review

L +242 −14

janitor run: 3 failing config(s), none unique to this patch — janitor results

build #132134 SUCCESS · tests all Maloo sessions

no vote yet: Bruno Faccini

patchsets:

+2 earlier

PS1 09-11 PS2 09-14

PS3 09-14 PS4 09-15 PS5 09-18 PS6 09-21 PS7 09-22

owner Sebastien Buisson · uploader Sebastien Buisson · PS7 uploaded 41d ago · NEW · open in Gerrit ↗

commit message

LU-18511 gss: carry out negotiation again on empty ccache

lgss_keyring prepares the root credential in
lkrb5_prepare_root_cred(), which checks the credentials cache and
obtains a new TGT from the keytab if needed, serialised with the
other lgss_keyring processes by a semaphore. But the credential
itself is not held, and it does not help to hold one: an INITIATE
gss credential obtained from the default cache only keeps a
krb5_ccache handle, and krb5_gss_init_sec_context() reads the
tickets from that cache when it is called, through
krb5_get_credentials().

So emptying the cache in between makes gss_init_sec_context()
return GSS_S_NO_CRED. This happens when something runs kdestroy or
kinit, and also when another lgss_keyring renews the root TGT,
because lkrb5_get_root_tgt_keytab() empties the cache with
krb5_cc_initialize() before filling it again with
krb5_cc_store_cred().

Such a failure is transient, and always recoverable for root, whose
credential can be obtained from the keytab again. But it is treated
as fatal: gss_kt_update() turns it into -EACCES and sets
PTLRPC_CTX_ERROR_BIT, then sptlrpc_req_refresh_ctx() only looks for
an already existing context for root, finds none just after a
flush, and completes the request with -EPERM. The application ends
up with an error on an operation that should merely have waited for
a new context.

So when the negotiation of a root context fails with GSS_S_NO_CRED,
get a new credential and carry the negotiation out again. This time
ask for a private copy of the credentials cache, in memory, made
while still holding the semaphore so that no other lgss_keyring can
be emptying the shared cache. Nothing outside of this process can
empty that copy, so the negotiation cannot fail again for the same
reason and a single retry is enough. Only the retry pays for it, so
the usual path keeps sharing its service tickets with the other
lgss_keyring processes.

While here, release the negotiation data before retrying, which the
existing retry path forgot to do, leaking the service name and the
partial context.

Also make 'lfs flushctx -k' run kdestroy before the ioctl that
flushes the contexts, instead of after it. Running it afterwards
empties the credentials cache just as the gss upcalls triggered by
that very flush are starting, which is the easiest way to hit this
race.

Add sanity-krb5 test_91, which recreates that ordering explicitly,
under dbench load.

Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: kerberos=true testlist=sanity-krb5 env=ONLY=91,ONLY_REPEAT=20
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Assisted-by: Claude:claude-opus-5
Change-Id: I9808862fa002a2b28a71f38687368e60b0ba0359

63283 master J –B ✓T ✓R 0/2

LU-19403 llapi: Move nodemap command function

reviewing Nikos Papakonstantinou · CI green — awaiting your review

3 unresolved L +156 −171

failed enforced test	platform	detail
review-dne-zfs-part-5 failed 2×	RHEL 9.7 / x86_64	ran 7 tests. 1 tests failed: lustre-rsync-test.	session

build #133514 SUCCESS · tests all Maloo sessions

no vote yet: Sebastien Buisson, Andreas Dilger

patchsets:

+32 earlier

PS1 01-08 PS2 01-08 Janitor Bot PS3 01-09 PS4 01-09 PS5 01-12 PS6 01-28 PS7 01-28 Janitor Bot PS8 01-28 PS9 01-30 PS10 01-30 PS11 02-09 PS12 02-10 PS13 02-10 PS14 02-10 PS15 02-12 PS16 02-12 Janitor Bot PS17 02-13 PS18 02-18 PS19 02-18 Janitor Bot PS20 02-18 PS21 02-18 Janitor Bot PS22 02-24 PS23 02-25 PS24 02-25 PS25 02-26 PS26 03-13 PS27 03-13 PS28 03-14 PS29 03-14 Janitor Bot PS30 03-14 PS31 03-31 PS32 07-28

PS33 08-10 PS34 08-21 PS35 09-16 PS36 09-17 PS37 10-13

threads: 3 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS32 2026-08-10 15:47 Nikos Papakonstantinou COMMIT_MSG:10 — I checked this and I saw that the fix is not that simple. For now I left it as it is and add a comment to the commit message that this is in

Gerrit AI review for Lustre PS32 · 2026-07-29 01:25

(minor) The body doesn't account for several things in the diff: that `nodemap_cmd()` is renamed to `llapi_nodemap_cmd()`, that error reporting switches from `fprintf(stderr)` to `llapi_error()`, and that `get_mds_device()`/`get_oss_device()` are deleted from obd.c together with their `do_disconnect()` and `cur_device` side effects.

"make all needed adjustments" leaves a reader to reconstruct those from the diff. The rename point was raised on patchset 30 and is still not in the message.

Nikos Papakonstantinou PS32 · 2026-08-10 15:47

I checked this and I saw that the fix is not that simple. For now I left it as it is and add a comment to the commit message that this is intentional, but if you think that we have to fix it, I would like to discuss a good way to do it.

reply PS34 2026-08-22 18:03 Gerrit AI review for Lustre COMMIT_MSG:12 — (minor) The body covers the rename, the llapi_error() switch and the dropped do_disconnect()/cur_device side effects, but not that the ioctl

Gerrit AI review for Lustre PS34 · 2026-08-22 18:03

(minor) The body covers the rename, the llapi_error() switch and the dropped do_disconnect()/cur_device side effects, but not that the ioctl call itself changed from l_ioctl() to llapi_ioctl_dev().

That one is worth a sentence: llapi_ioctl_dev() lazily does llapi_register_ioc_dev(OBD_DEV_ID, OBD_DEV_PATH), which is what lets the function work for a caller that never ran lctl's obd_initialize() — i.e. it is the part that actually delivers on "so it can be reused outside of lctl".

reply PS34 2026-08-22 18:03 Gerrit AI review for Lustre liblustreapi_nodemap.c:64 — (minor) Not a bug with today's callers (the longest passes 5 strings), but now that this is a shared entry point the varargs contract is wor

Gerrit AI review for Lustre PS34 · 2026-08-22 18:03

(minor) Not a bug with today's callers (the longest passes 5 strings), but now that this is a shared entry point the varargs contract is worth stating.

lustre_cfg_bufs_set() drops anything past LUSTRE_CFG_MAX_BUFCOUNT:

    if (index >= LUSTRE_CFG_MAX_BUFCOUNT)
            return;

so a caller passing more than 8 strings silently gets a truncated command rather than an error. Could the kernel-doc for `@...` mention the 8-argument limit, or the loop stop and return -EOVERFLOW when `i` reaches LUSTRE_CFG_MAX_BUFCOUNT?

owner Nikos Papakonstantinou · uploader Nikos Papakonstantinou · PS37 uploaded 20d ago · NEW · open in Gerrit ↗

commit message

LU-19403 llapi: Move nodemap command function

Move nodemap_cmd() from obdctl to liblustreapi as
llapi_nodemap_cmd(), so it can be reused outside of lctl. Error
reporting switches from fprintf(stderr) to llapi_error().

Unlike the old obdctl nodemap_cmd(), it does not call
do_disconnect() or update lctl's cur_device as a side effect of
selecting the MGS/MDS/OSS device: an interactive lctl session's
next bare device command may now target a different device than
the nodemap command just used.

Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com>
Change-Id: I19f510399b86b7e79302a8f862232ac290f32648

69479 master J –B ✓T ✓R +1/2

LU-20885 llite: use delay RPC for glimpse after all mirrors

reviewing Zhenyu Xu · CI green — awaiting your review

S +15 −2

18d

build #133021 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-10-14 18:05

no vote yet: Qian Yingjin

owner Zhenyu Xu · uploader Zhenyu Xu · PS1 uploaded 26d ago · NEW · open in Gerrit ↗

commit message

LU-20885 llite: use delay RPC for glimpse after all mirrors

__cl_glimpse_size() retries a glimpse while fast-switching FLR
mirrors, and forces io->ci_ndelay = 1 on every iteration so that the
glimpse lock is enqueued with a non-delay RPC (LDLM_FL_NDELAY).  When
the OST import is not connected, osc_lock_upcall() turns the enqueue
failure into -EAGAIN so the caller can try the next mirror.

Commit 4468f6c9d924 ("LU-16025 llite: adjust read count as file got
truncated") bounded that retry loop with ci_tried_all_mirrors to avoid
an endless restart on a concurrent truncate.  But once
ci_tried_all_mirrors is set the loop gives up and returns -EAGAIN to
userspace while io->ci_ndelay is still forced to 1, so the glimpse
never gets a chance to use a delay RPC that waits for the import to
reconnect.  Right after a client remount the OST imports are still
connecting, so a stat() on an FLR file fails with EAGAIN after only a
few mirror rounds instead of waiting for the connection, e.g. in
sanity-flr test_44d.

Stop forcing non-delay RPCs once all mirrors have been tried, and
allow one more restart so the glimpse is retried with a delay RPC
(ci_ndelay == 0) that waits for the imports to reconnect.  The
LU-16025 bound is kept: a delay RPC no longer returns the ndelay
-EAGAIN, so the loop still terminates on a concurrent truncate.

Test-Parameters: testlist=sanity-flr env=ONLY=44d,ONLY_REPEAT=10
Fixes: 4468f6c9d924 ("LU-16025 llite: adjust read count as file got truncated")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Assisted-by: Augment Agent:DeepSeek V4.1 Flash
Change-Id: I22bdf5384e3936aa6d6e606010e8e1e43c049707

69204 master J –B ✓T ✓R +0/2−

LU-20828 tests: run xfstests' fsx against Lustre

reviewing patrick bot · CI green — awaiting your review

bot-ci-generated 4 unresolved XL +4037 −0

23d

build #133308 SUCCESS · tests all Maloo sessions

reviews: Artem Blagodarenko +1 2026-10-09 21:03 · Patrick Farrell -1 2026-10-09 20:28

no vote yet: Jian Yu

patchsets: PS1 09-25 PS2 10-08 PS3 10-09 Patrick Farrell PS4 10-09

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS4 2026-10-09 21:03 Artem Blagodarenko sanity-fsx-xfs.sh:33 — About 40-48% of these ops are skipped on Lustre. fsx disables the unsupported operations at startup and still counts each one as an op: 1963

Artem Blagodarenko PS4 · 2026-10-09 21:03

About 40-48% of these ops are skipped on Lustre. fsx disables the unsupported operations at startup and still counts each one as an op: 1963 of 5000 are skipped in the buffered and AIO mixes, 2380 in the O_DIRECT mix. Skipped ops are collapse_range, insert_range, clone_range, dedupe_range, exchange_range, write_zeroes and write_atomic (plus copy_range in test 2 because of -E).

Because of -q none of this is visible in the test log. Could we either raise NUMOPS to compensate, or drop -q (or print the "disabling" lines once) so the log shows what was actually exercised?

reply PS4 2026-10-09 21:03 Artem Blagodarenko sanity-fsx-xfs.sh:40 — The file uses the default layout and fsx's default maximum length of 256 KiB, so it is always a single stripe on one OST (1 stripe, 4 MiB st

Artem Blagodarenko PS4 · 2026-10-09 21:03

The file uses the default layout and fsx's default maximum length of 256 KiB, so it is always a single stripe on one OST (1 stripe, 4 MiB stripe size in my run) and no I/O crosses a stripe or OST boundary.

Could be a follow-up, but a `$LFS setstripe -c -1 -S 64k` on the file, or a larger -l, would reach the multi-stripe paths, which is where I'd expect a Lustre-specific fsx to earn its keep.

reply PS4 2026-10-09 21:03 Artem Blagodarenko sanity-fsx-xfs.sh:45 — fsx defaults to seed 1, so every run executes the identical op sequence, and repeated CI runs add no new coverage. Tests 1 and 3 also run ex

Artem Blagodarenko PS4 · 2026-10-09 21:03

fsx defaults to seed 1, so every run executes the identical op sequence, and repeated CI runs add no new coverage. Tests 1 and 3 also run exactly the same sequence as each other (the --record-ops logs are byte-identical); only the submission path differs.

Would -S 0 be acceptable here? fsx logs the seed, so a failure stays reproducible, and sanityn test_16 already runs our own fsx with -S 0.

reply PS4 2026-10-10 14:08 Gerrit AI review for Lustre README:6 — (minor) Is the GPL-2.0 claim for fsx.c something xfstests actually states anywhere? Neighbouring xfstests sources such as ltp/fsstress.c and

Gerrit AI review for Lustre PS4 · 2026-10-10 14:08

(minor) Is the GPL-2.0 claim for fsx.c something xfstests actually states anywhere? Neighbouring xfstests sources such as ltp/fsstress.c and src/global.h carry an explicit SPDX tag, while fsx.c has only the 1991 NeXT "All Rights Reserved" notice, so its missing tag looks deliberate rather than an omission. The same fsx.c lineage already in this tree, lustre/tests/fsx.c, is licensed under the Apple Public Source License 1.1.

If the license is really only inferred from the rest of the project, it may be better to say so in the README (or cite where xfstests says it) than to state it as fact.

tags: bot-ci-generated

owner patrick bot · uploader patrick bot · PS4 uploaded 23d ago · NEW · open in Gerrit ↗

commit message

LU-20828 tests: run xfstests' fsx against Lustre

Lustre's fsx.c and xfstests' ltp/fsx.c share a distant ancestor
but diverged into different tools; the xfstests copy exercises
I/O paths ours never reaches. Vendor it and run it against a
single mount as a new sanity-fsx-xfs suite, with each test's
op mix kept in the script rather than in a patched fsx.c.

fsx.c and global.h are pinned to kdave/xfstests commit
a370dcbed43563f0462801e889e0eceb93c7cfad. fsx.c drops xfstests'
own statx.h for the system's struct statx, and skips the
atomic-write probe when the system headers are too old to
describe it.

xfstests normally builds fsx.c against its own generated
config.h; the needed HAVE_* macros are supplied on the command
line instead, without HAVE_XFS_XFS_H, so global.h does not pull
in xfsprogs headers this tree has no use for. fsx.c itself uses
getopt_long() and struct option without including <getopt.h>,
which upstream's own build reaches transitively through those
same xfsprogs headers; a small new compat header force-includes
it directly instead, and also silences a clang-only
warning-as-error on an llabs() that clang wrongly calls a no-op.

The O_DIRECT op mix is page-aligned and excludes copy_file_range
(-E), matching what was validated on LU-20828: page-aligned
O_DIRECT combined with copy_file_range corrupts file content on
Lustre, though not on every run. Reproduced here too (1 failure
in 3 x 20000 ops); a real bug, but a separate one from standing
up this suite. On a failure, fsx's replay files are kept in the
test logs.

Test-Parameters: trivial testlist=sanity-fsx-xfs
Assisted-by: ClaudeCode:claude-sonnet-5
Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Patrick Farrell <patrick@thelustrecollective.com>
Change-Id: I2af4cd2762e8fcec52859cb4cbd09746e561e7ed

54051 master J !B ✓T ✓R +0/2−

LU-15069 llite: remove skip_index

reviewing Patrick Farrell · CI green — awaiting your review (needs rebase first)

needs rebase 5 unresolved S +10 −26

56d

CI green — awaiting your review (needs rebase first)

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked

janitor run: 2 failing config(s), none unique to this patch — janitor results

5 optional test failure(s) — informational

build #119151 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-04-25 08:49 · Oleg Drokin -1 2026-08-22 06:52 · Timothy Day +1 2026-07-28 05:08

no vote yet: Qian Yingjin, Shaun Tancheff, Sebastien Buisson, Li Dongyang

patchsets:

+1 earlier

PS1 02-15

PS2 02-15 PS3 02-15 PS4 02-15 PS5 11-23 PS6 12-03

threads: 5 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS6 2026-08-09 22:24 Oleg Drokin COMMIT_MSG:25 — no Fixes: line for the patch that introduced this?

Oleg Drokin PS6 · 2026-08-09 22:24

no Fixes: line for the patch that introduced this?

reply PS6 2026-08-09 22:24 Oleg Drokin patchset comment — this patch was pushed 8 month ago and not refreshed since.

Oleg Drokin PS6 · 2026-08-09 22:24

this patch was pushed 8 month ago and not refreshed since.

that means no ai review (this is a note to myself to run it before pushing)

reply PS6 2026-08-12 05:13 Oleg Drokin rw.c:781 — this... sounds somewhat unintentional?

Gerrit AI review for Lustre PS6 · 2026-08-12 05:04

(defect) This drops the `if (skip_index)` condition but keeps its body, so the assignment two lines above is now dead - it is always overwritten. Removing skip_index should mean the code behaves as it did when skip_index was 0, i.e. the window end stays anchored at ras_window_start_idx:

    if (ras->ras_window_pages > 0)
        end_idx = ras->ras_window_start_idx + ras->ras_window_pages - 1;
    end_idx = *start_idx + ras->ras_window_pages - 1;   /* was skip_index only */

As written the RA window end is now anchored at ras_next_readahead_idx for every read, not just the mmap range case, which is a behaviour change the commit message describes as a simplification. Is that intended? If so it deserves a sentence in the message and the dead lines above should go.

The `ras->ras_window_pages > 0` guard also disappears with it. When the window is zero the new expression underflows to `*start_idx - 1`, and with `*start_idx == 0` that wraps to ULONG_MAX; the EOF clamp below then turns it into `[0, eof_index]`, i.e. a whole-file readahead window, where the old code fell through to the `end_idx == 0` RA_STAT_ZERO_WINDOW return. ras_detect_read_pattern() reaches that state - the stride-invalidation path sets `ras_window_pages = 0; ras_next_readahead_idx = index;` with index 0, and the mmap hit path in ras_update() sets `ras_window_pages = 0` without touching ras_next_readahead_idx.

Oleg Drokin PS6 · 2026-08-12 05:13

this... sounds somewhat unintentional?

I guess forcing whole file readahead would do wonders to tests that expect great readhead, but what about actual workloads? do we have any actual tests?

reply PS5 2025-11-25 05:30 Andreas Dilger sanity.sh:13083 — (style) prefer `((...))` for numeric comparisons

Andreas Dilger PS5 · 2025-11-25 05:30

(style) prefer `((...))` for numeric comparisons

reply PS6 2026-08-12 05:04 Gerrit AI review for Lustre sanity.sh:13083 — (style) this isn't a bug, but the numeric comparison was asked to move to `(( ... ))` on an earlier patchset and the line is still using `[

Gerrit AI review for Lustre PS6 · 2026-08-12 05:04

(style) this isn't a bug, but the numeric comparison was asked to move to `(( ... ))` on an earlier patchset and the line is still using `[ ... -eq ... ]`:

    (( miss == 2 )) || error "expected misses 2 but got $miss"

owner Patrick Farrell · uploader Patrick Farrell · PS6 uploaded 334d ago · NEW · open in Gerrit ↗

commit message

LU-15069 llite: remove skip_index

Skip_index seems to have been a confused attempt to change
mmap readahead behavior, but basically just turns off a
number of readahead optimizations in a weird way and
sometimes causes us not to read a page for - as far as I
can tell - no reason.  This causes misses in normal read
patterns.

It also skips the minimum page reservation logic, again for
no reason I can see - mmap reads should use the same logic
here as everything else.

Remove it entirely to simplify the code a little.

This also reduces the number of misses we take in a simple
readahead test, which is a nice indication of how this code
is affecting readahead.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ia953c8a8cb6dc175255ac316b4b3f02d78effae9

63164 master J !B ✓T ✓R +0/2−

LU-930 doc: man page improvements

reviewing Patrick Farrell · CI green — awaiting your review (needs rebase first)

needs rebase stalled 237d 3 unresolved L +249 −146

237d

CI green — awaiting your review (needs rebase first)

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #120257 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-01-10 01:09

patchsets: PS1 12-24 PS2 01-09 Marc Vef PS3 01-10 Andreas Dilger

threads: 2 need your reply · 1 waiting on others · 0 own notes · 0 bot

reply PS1 2026-01-10 01:05 Andreas Dilger COMMIT_MSG:7 — ```suggestion

Marc Vef PS1 · 2026-01-09 11:02

(defect) This should use a proper LU number

Andreas Dilger PS1 · 2026-01-10 01:05

```suggestion
LU-930 doc: man page improvements
```

Could just use the default "man page improvement" ticket...

reply PS1 2025-12-24 12:31 Andreas Dilger lctl.8:386 — Rather than adding sub-commands to lctl, it is better to add a separate manage for it. That allows proper formatting, subsections, examples

Andreas Dilger PS1 · 2025-12-24 12:31

Rather than adding sub-commands to lctl, it is better to add a separate manage for it.  That allows proper formatting, subsections, examples, etc.

owner Patrick Farrell · uploader Andreas Dilger · PS3 uploaded 296d ago · NEW · open in Gerrit ↗

commit message

LU-930 doc: man page improvements

Various man page improvements - adding missing
commands and options, fixing typos, fixing
format issues, etc.

Includes one trivial fix to lctl.c.

Focuses on man8.

This code was generated by Augment.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I222ea824b25b0a3104dc4a8bc8f56c10449045e4

52182 master J !B ✓T ✓R 0/2

LU-13814 osc: simplify osc_dio_page_submit

reviewing Patrick Farrell · CI green — awaiting your review

stalled 338d S +10 −14

338d

janitor run: 3 failing config(s), none unique to this patch — janitor results

2 optional test failure(s) — informational

build #119105 SUCCESS · tests all Maloo sessions

no vote yet: Qian Yingjin, Andreas Dilger, Zhenyu Xu

patchsets:

+32 earlier

PS1 08-30 PS2 08-30 PS3 08-30 PS4 08-31 PS5 09-05 PS6 09-09 PS7 09-09 PS8 09-10 PS9 09-10 PS10 09-10 PS11 09-10 PS12 09-10 PS13 09-10 PS14 09-10 PS15 09-12 PS16 09-14 PS17 09-15 PS18 09-24 PS19 09-25 PS20 09-26 PS21 09-26 PS22 10-05 PS23 10-05 PS24 10-20 PS25 10-22 PS26 10-22 PS27 02-27 PS28 10-28 PS29 11-03 PS30 04-28 PS31 05-01 PS32 07-04

PS33 07-14 PS34 11-06 PS35 11-09 PS36 11-16 PS37 11-16

owner Patrick Farrell · uploader Patrick Farrell · PS37 uploaded 350d ago · NEW · open in Gerrit ↗

commit message

LU-13814 osc: simplify osc_dio_page_submit

Remove more code from osc_dio_page_submit, with the goal of
eliminating it entirely.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id408aad01c2e81a126dd04807819aed9d29fca6b

52167 master J !B ✓T ✓R 0/2

LU-13814 osc: add osc_dio_completion

reviewing Patrick Farrell · CI green — awaiting your review

stalled 338d M +49 −32

338d

janitor run: 3 failing config(s), none unique to this patch — janitor results

2 optional test failure(s) — informational

build #119044 SUCCESS · tests all Maloo sessions

no vote yet: Qian Yingjin, Andreas Dilger, Zhenyu Xu

patchsets:

+31 earlier

PS32 07-14 PS33 11-06 PS34 11-09 PS35 11-16 PS36 11-16

owner Patrick Farrell · uploader Patrick Farrell · PS36 uploaded 350d ago · NEW · open in Gerrit ↗

commit message

LU-13814 osc: add osc_dio_completion

Specialize osc_completion to add the osc_dio_completion
version.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I4d202165ee6c4f50338b6a9381aa476b969d3593

▶

Awaiting review — CI not green yet

(13)

69703 master J –B ✓T ✗5R 0/2

LU-20690 ec: verify parity against data, allow p > k

reviewing Keguang Xu · blocked on the owner: enforced tests failed

raid10-likept_ecro218_review_priority 1 unresolved L +338 −193

blocked on the owner: enforced tests failed

awaiting review (CI not green yet)

failed enforced test	platform	detail
custom-1001	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-ec.	session
review-dne-subtest-change	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session

build #135173 SUCCESS · tests all Maloo sessions

no vote yet: Ronnie Sahlberg, Andreas Dilger

patchsets: PS1 10-15 PS2 10-15 PS3 10-26 PS4 11-02

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-10-30 08:58 Keguang Xu lod_lov.c:2242 — Thanks for catching this.

Gerrit AI review for Lustre PS3 · 2026-10-27 00:48

(defect) Can the MDS create a parity that this check then rejects?

With more than one raid set, lod_stripe_count_min() returns 0 for a parity, and lod_ost_alloc_rr() then reports success on a short allocation:

    lod_ost_alloc_rr()
      if (stripe_idx) rc = 0;
      lod_ec_add_parities()   /* may still be short, e.g. failure domains */
      /* Still short; remember how many we got */
      lod_comp->llc_stripe_count = stripe_idx;
      stripe_idx < stripe_count_min (0)  -> no -ENOSPC

lod_comp_prep_create() does not check the count afterwards, so the layout is stored with fewer than (n0 + n1) * p stripes. If the shortfall is a whole raid set, the count is still a multiple of p and passes lod_ec_parity_valid(). Before this patch such a layout went through open replay (LOV_MAGIC_DEFINED -> lod_qos_parse_config() -> lod_verify_striping()). Now the replay gets -EINVAL here, as do HSM release and xattr copies of the file.

Should RR fail a short EC parity the way lod_ost_alloc_qos() does ("we require the full stripe_count to avoid incomplete raid sets")? Otherwise this check is stricter than what the allocator guarantees.

Keguang Xu PS3 · 2026-10-30 08:58

Thanks for catching this.

Full parity geometry is nsets * p (nsets = ec_raidset_count from the data stripe count and k). RR can finish with stripe_idx < nsets * p when nsets > 1: lod_stripe_count_min() is 0, so a short lod_ec_add_parities() still returns success and sets llc_stripe_count = stripe_idx. QoS instead requires stripe_idx == nsets * p.

@rsahlberg@whamcloud.com @adilger@thelustrecollective.com: is stripe_idx < nsets * p for multi-raidset parity from RR intentional? A short layout (e.g. (nsets-1)*p) can break degraded read / EC recovery, leave some raid sets with no parity to rebuild from, and confuse resync/verify and mirror tools. QoS already refuses this. I think RR should match QoS (fail short EC parity) and the verifier should require count == nsets * p again — happy to follow up if you agree.

tags: raid10-like pt_ecro 218_review_priority

owner Keguang Xu · uploader Keguang Xu · PS4 uploaded 14h ago · NEW · open in Gerrit ↗

commit message

LU-20690 ec: verify parity against data, allow p > k

With the RAID10-like parity layout the p parities of a raidset share
one offset, so the parity of a component fits in its extent whatever
p is. Remove the p <= k checks in lfs and llapi; k+p <= 256 and
p <= 15 remain. Also drop the mirror-extend stripe_count < p guard
in ec_check_data_comp() so --ec 2+3 works on a two-stripe target.

An instantiated parity is checked against its data component in
lod_verify_ec_parity_comps() after lod_use_defined_striping() /
lod_parse_striping(), using lod_comp_find_linked() for the same
pairing rule as the allocator (covers layout-change replay). Require
a bidirectional link, matching k/p and stripe size, and parity stripe
count ec_raidset_count() * p. lod_verify_striping() checks k/p and
rejects an instantiated parity with stripe_count == 0 before
DEFINED object init.

Tests: sanity-ec 4a and 10 accept p > k, and 4b rejects k == 0 and
k > 255. 8m covers mirror extend --ec 2+3 on a two-stripe target.
300 covers PFL -E 2M --ec 2+3 create/resync/verify, and 305 covers
p > k multi-raidset create/resync/verify. llapi_layout_test test47
allows 3+4.

Test-Parameters: testlist=sanity-ec ostcount=8
Signed-off-by: Keguang Xu <kxu@ddn.com>
Assisted-by: Cursor:Composer llm_code_and_review_tools
Change-Id: I3878502e8ff739cb5ed420ff359e2008376303e7

68361 master J –B ✓T ↻R 0/2

LU-20690 ec: store EC parity like RAID10

reviewing Keguang Xu · awaiting review (CI not green yet)

ec2 pt_ecroraid10-like218_review_priority 11 unresolved XL +2237 −857

failed enforced test	platform	detail
review-dne-part-7 retesting	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-pcc.	session
review-dne-selinux-ssk-part-2 retesting failed 3×	RHEL 8.10 / x86_64	ran 7 tests. 1 tests failed: sanity-sec.	session

build #135085 SUCCESS · tests all Maloo sessions

no vote yet: Ronnie Sahlberg, Andreas Dilger

patchsets:

+8 earlier

PS1 08-26 PS7 09-03 PS8 09-07 PS9 09-10 PS10 09-11 PS11 09-23 PS12 10-08 PS13 10-08

PS17 10-23 PS18 10-29 PS19 10-29 Ronnie Sahlberg PS20 10-30 Ronnie Sahlberg PS21 11-01

threads: 11 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS21 2026-11-02 00:34 Gerrit AI review for Lustre COMMIT_MSG:110 — (minor) Two claims here don't match the code:

Gerrit AI review for Lustre PS21 · 2026-11-02 00:34

(minor) Two claims here don't match the code:

- lod_parse_striping() doesn't reject a flag/pattern mismatch. The new check is in lod_verify_striping(). lod_parse_striping() forces PARITY with lod_comp_set_ec_pattern() when LCME_FL_PARITY is set, and leaves a PARITY pattern without the flag as it is. lsm_unpackmd_comp_md_v1() rejects that second case, so the MDS would serve a layout that no client can unpack. Should lod_parse_striping() reject it as described, or should the text say lod_verify_striping()?
- lod_declare_layout_set() doesn't refuse to modify a parity component. It refuses a set.flags request that sets or clears LCME_FL_PARITY (the case sanity-ec 6g tests). Other flags, such as stale, can still be set on parity components.

reply PS21 2026-11-02 00:34 Gerrit AI review for Lustre COMMIT_MSG:174 — (minor) The patch also fixes the raid set mapping in degraded reads and resync, which counted rows and raid sets from e_start instead of fil

Gerrit AI review for Lustre PS21 · 2026-11-02 00:34

(minor) The patch also fixes the raid set mapping in degraded reads and resync, which counted rows and raid sets from e_start instead of file offset 0 (the wrong-parity rebuild described above). That counting came from landed commits, so a Fixes: line would help with tracking, even though EC has not shipped:

Fixes: 783c13bb0b ("LU-12669 ec: recover data from parity")

The resync side of the same problem dates from 03191be6f4 ("LU-12668 ec: add support to resync stale ec comps").

reply PS14 2026-10-14 09:17 Keguang Xu patchset comment — With the changing of layout from `PARITY|RAID0` TO RAID10-like `PARITY`, we not only store the layout header differently to disk, but also w

Andreas Dilger PS14 · 2026-10-14 03:41

Should the removal of the LOV_PATTERN_RAID0 from EC layouts be moved into an earlier patch?  That affects the on-disk layout of all the files and should be included into the 2.18.0 release, even if the 1+2 layout is not allowed initially.  If this patch doesn't make it then we could force 1+2 to be 2+2 at create time?  Not ideal, but probably easier to get landed.

Keguang Xu PS14 · 2026-10-14 09:17

With the changing of layout from `PARITY|RAID0` TO RAID10-like `PARITY`, we not only store the layout header differently to disk, but also we need to adjust our access path, say, in the parity component, stripes belong to the same raidset are arranged as RAID1 (logicall, they share the same file offset), stripes from different raidsets are stored as RAID0. To read/write data from/to parity, we need to designate stripe first.

We can move the layout changes into an earlier patch. However, for EC to function properly, all of these patches must be landed together as a single set.

reply PS21 2026-11-02 00:34 Gerrit AI review for Lustre lfs-mirror-read.1:30 — (minor) llapi_layout_parity_pos_to_io() starts each component's dump at the slot of the raid set that holds e_start, not at raid set 0. Take

Gerrit AI review for Lustre PS21 · 2026-11-02 00:34

(minor) llapi_layout_parity_pos_to_io() starts each component's dump at the slot of the raid set that holds e_start, not at raid set 0. Take -E 4M -c 1 -E eof -S 1M -c 8 --ec 4+1: the second component has O = 3M, e_start is in raid set 1, and its dump begins with the parity object of raid set 1, followed by raid set 0 of the next row. The order matches getstripe only when e_start is a multiple of the stripe width. lfs-getstripe.1 has the same issue ("dumps this RAID0 of all parity stripes"). If the patch is refreshed, could both pages say that the first row starts at the raid set of e_start?

reply PS9 2026-10-30 02:46 Ronnie Sahlberg cl_object.h:2022 — not part of this change any more. And don't want to touch 69517 where this now lives as that one has passed testing.

Andreas Dilger PS9 · 2026-09-10 19:32

(minor) "stripe" is not a good name for this, since it is potentially conflated with the data stripe number if we want to do single-stripe read/write for replacement.  Maybe "_parity_index" or similar?

Ronnie Sahlberg PS9 · 2026-10-30 02:46

not part of this change any more.  And don't want to touch 69517 where this now lives as that one has passed testing.

reply PS20 2026-10-30 02:49 Misc Code Checks Robot (Gatekeeper helper) lod_internal.h:751 — warn: lod_comp_shrink_to_extent():always true condition '(lod_comp->llc_stripe_count <= 65535) => (0-u16max <= u16max)'

Misc Code Checks Robot (Gatekeeper helper) PS20 · 2026-10-30 02:49

warn: lod_comp_shrink_to_extent():always true condition '(lod_comp->llc_stripe_count <= 65535) => (0-u16max <= u16max)'

reply PS21 2026-11-01 12:04 Misc Code Checks Robot (Gatekeeper helper) lod_internal.h:751 — warn: lod_comp_shrink_to_extent():always true condition '(lod_comp->llc_stripe_count <= 65535) => (0-u16max <= u16max)'

Misc Code Checks Robot (Gatekeeper helper) PS21 · 2026-11-01 12:04

warn: lod_comp_shrink_to_extent():always true condition '(lod_comp->llc_stripe_count <= 65535) => (0-u16max <= u16max)'

reply PS17 2026-10-27 03:38 Andreas Dilger lov_internal.h:160 — With RAID0|PARITY gone, lsme_parity_slots_per_raidset() is always 1 for parity and only multiplies poff at two places in lov_io.c. The six

Andreas Dilger PS17 · 2026-10-27 03:38

With RAID0|PARITY gone, lsme_parity_slots_per_raidset() is always 1 for parity and only multiplies poff at two places in lov_io.c.  The six geometry helpers (lsme_stripe_origin, lsme_stripes_per_slot, lsme_parity_single_slot, lsme_slot_count, lsme_stripe_slot, lsme_parity_slots_per_raidset) reduce to one rule: for parity, origin = e_start, width = (stripe_count / p) * stripe_size, slot = stripe / p; otherwise RAID0.  Two helpers would do, and the single-slot case can become an invalid entry.  About 25 lines.

reply PS17 2026-10-30 02:46 Ronnie Sahlberg lfs.c:4629 — Silent fallback is addressed. But not the quoting as raid0,overstriped should already be treated by yaml as a single string so quoting shoul

Andreas Dilger PS17 · 2026-10-27 03:17

This only matches the exact string "parity", but "lfs getstripe -y" prints "parity,overstriped" for a -C EC component, so "lfs setstripe --yaml" of that output silently falls back to the default pattern.  When the pattern has multiple flags, please quote it in the YAML output (pattern: "parity,overstriped") so it is a single scalar, and parse the comma-separated flags here instead of comparing whole strings, which would also cover "raid0,overstriped" and any later flag combinations.

Ronnie Sahlberg PS17 · 2026-10-30 02:46

Silent fallback is addressed. But not the quoting as raid0,overstriped should already be treated by yaml as a single string so quoting should not be needed here.

reply PS21 2026-11-02 00:34 Gerrit AI review for Lustre lfs.c:4553 — (suggestion) This isn't a bug, but the ignored[] list repeats the names in lov_pattern_names[] in liblustreapi_layout.c, and llapi_lov_strin

Gerrit AI review for Lustre PS21 · 2026-11-02 00:34

(suggestion) This isn't a bug, but the ignored[] list repeats the names in lov_pattern_names[] in liblustreapi_layout.c, and llapi_lov_string_pattern() already parses the comma-separated getstripe form. The copy will drift when a new pattern name is added. If the patch is refreshed, consider calling llapi_lov_string_pattern() and mapping LOV_PATTERN_MDT and LOV_PATTERN_OVERSTRIPING to the LLAPI_LAYOUT_* values. The "unknown_" prefix would still need handling here.

reply PS20 2026-10-31 15:29 Gerrit AI review for Lustre liblustreapi_layout.c:3229 — (minor) Carried over from the PS17 comment on liblustreapi_mirror.c, which asked for the parity-dump helpers to be static in lfs.c rather th

Gerrit AI review for Lustre PS20 · 2026-10-31 15:29

(minor) Carried over from the PS17 comment on liblustreapi_mirror.c, which asked for the parity-dump helpers to be static in lfs.c rather than new public API. mirror_dump_parity() moved, but llapi_layout_parity_pos_to_io() is still a global llapi_* function. liblustreapi.map exports `llapi_*`, so it is still in liblustreapi.so's ABI even though only lustreapi_internal.h declares it. If the patch is refreshed, could it move into lfs.c and use the public layout getters, or at least lose the llapi_ prefix so the map doesn't export it?

tags: pt_ecro raid10-like 218_review_priority

owner Keguang Xu · uploader Keguang Xu · PS21 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-20690 ec: store EC parity like RAID10

The k >= p restriction is not a Cauchy/ISA-L limit. It comes from
storing parity as RAID0|PARITY: the p parity objects of a raidset are
concatenated along the file offset, so raidset_count * p * stripe_size
does not fit in a finite PFL extent when p > k (for example
--ec 2+3 -E 2M). This patch changes the parity layout; the p <= k
checks in lfs and llapi stay until a following patch removes them.

Store EC parity as LOV_PATTERN_PARITY instead. A raidset is an EC
split of the stripe set, and the layout is like RAID10 in that it has
the same two axes:

  - across the raidsets of one stripe set: like RAID0, one
    stripe_size per raidset;
  - within one raidset: like RAID1, the p parity stripes share one
    file offset (the same object_off).

The RAID1 axis comes from recovery. A raidset of k data and p parity
stripes survives the loss of any p of its k + p stripes, just as a
RAID1 of p + 1 copies survives the loss of any p copies. RAID1 is the
k == 1 case, where each parity is a copy of the data (--ec 1+N). The
p parities protect the same data, so they share its offset: the
offset says which data is protected, the parity index says which
redundancy unit it is.

What differs from RAID1 is I/O, not the role. For k > 1 the parities
are not replicas: each holds a different parity, and a lost stripe is
rebuilt from any k survivors rather than copied from one. The parity
uses the extent of the data component.

RAID0 places data rows from file offset 0, whereas counting EC rows
and raidsets from e_start groups the wrong k stripes when e_start is
not a multiple of the data stripe width: the stripes EC groups as one
raidset are not the objects lod paired with that raidset's parity
OSTs, and a degraded read rebuilds a lost stripe from the wrong
parity. For example -E 4M -c 1 -E eof -S 1M -c 8 --ec 4+1 puts offset
4M on object 4 (raidset 1), which counting from e_start encodes into
the parity of raidset 0. When e_start is not a multiple of
stripe_size, as after a 64K DoM component, counting stripe_size
parity slots from e_start also splits stripes at
e_start + n * stripe_size and overflows e_end.

Count rows and raidsets from file offset 0, as RAID0 maps the data
objects:

  row = off / (stripe_count * stripe_size)
  ec_off = O
      + (row - row of e_start) * raidset_count * stripe_size
      + raidset * stripe_size + off % stripe_size

Data before e_start is taken as zero, and so is its parity, which is
not stored: the origin O moves back from e_start over the raidsets of
the first row that end before e_start, and over the part of the
raidset of e_start below e_start % stripe_size when e_start is in its
last data stripe. slot % raidset_count stays the raidset, so the
parity OSTs lod pairs are unchanged, and every stored parity byte has
data in the component. Common DoM and PFL layouts with an unaligned
e_start are supported and keep their parity within [e_start, e_end),
such as:
- -E 64K -L mdt -E 1M -c 1 -S 1M -E eof -c 4 -S 4M --ec 4+2;
- -E 64K -L mdt -E 16M -c 2 -S 1M -E eof -c 4 -S 4M --ec 2+2;
- -E 64K -L mdt -E eof -c 4 -S 1M --ec 2+2;
- -E 1M -c 1 -S 1M -E 64M -c 1 -S 4M -E eof -c 4 -S 4M --ec 4+2.
With --ec 1+N, ec_off is the data offset. O depends only on e_start,
the data stripe count, k and the stripe size, not on e_end. When
e_start is a multiple of the stripe width, O is e_start.

Add ec_raidset_count(), ec_raidset_first(), ec_stripe_to_raidset()
and ec_off_to_raidset_offset() next to ec_split_stripes() in
lustre_user.h, and struct ec_parity_geom with ec_parity_geom_init(),
ec_data_to_parity() and its inverse ec_parity_to_data(), with
ec_parity_width() for the parity bytes of one stripe set, and use
them in lov, lod and llapi instead of open-coding the k0 * n0 split.
lov caches O in lsme_parity_origin when the layout is unpacked and
returns it from lsme_stripe_origin(), and lov_parity_geom_init()
fails a degraded read whose pair does not match in stripe size,
raidsets or origin; the parity mirror EOF of such a pair is also
bounded by one parity row per stripe set from e_start. Raidset
objects before e_start read as zero in degraded reads and resync.
Drop lsme_parity_raid10() and lsme_parity_slots_per_raidset(), which
have no users left once every parity component is LOV_PATTERN_PARITY.

A component shorter than one stripe (e_end < e_start + stripe_size)
whose raidset continues past e_end (e.g. [64K, 1M) with -S 1M -c 2
--ec 2+2, or [1M, 4M) with -S 4M -c 2 --ec 2+2, where the second
stripe is never reached) still has no room for a full stripe_size
parity slot from e_start. ec_parity_fits() detects it:
llapi_layout_sanity() refuses it with LSE_EC_PARITY_FIT,
llapi_ec_check_comp_match() refuses to resync or verify it, and
lod_comp_prep_create() checks it again with the final data stripe
count, also for lfs mirror extend --ec.

The lov slot geometry, the STRIPE I/O scope and the per-stripe llapi
helpers were added by the previous patches and only take effect for
LOV_PATTERN_PARITY. This patch switches the encoding so they apply:
llapi_pattern_to_lov() and llapi_layout_to_lum() emit
LOV_PATTERN_PARITY, lov_pattern_supported() accepts it in place of
RAID0|PARITY (and lov_pattern_available() via that), and lod allocates
and verifies parity components with it. EC has not shipped, so the old
combination is not kept. Finite-extent stripe-count shrinking, now in
one lod_comp_shrink_to_extent() helper instead of three copies, is
skipped for parity components. lsm_unpackmd_comp_md_v1() and
lod_parse_striping() reject a component whose LCME_FL_PARITY flag and
LOV_PATTERN_PARITY disagree, since the parity geometry follows the
flag, and lod_declare_layout_set() refuses to modify a parity
component.

vvp_prep_size() no longer writes KMS into i_size for parity I/O.
ci_parity_eof is derived from i_size, and KMS is only a lower bound
of the data size, so a parity read within KMS shrank ci_parity_eof
for later parity I/O.

Userspace resync and verify move to the RAID10 offsets: one
stripe_size per raidset, each parity addressed by its stripe index,
leaving the part of the first parity slot below e_start alone. Hole
punch-or-clear and hole verify loop over the p parity stripes that
share an offset when they fall back to zero-fill or zero-content
reads.

lfs setstripe --yaml parses the pattern as the comma-separated flags
lfs getstripe -y prints, so "parity,overstriped" keeps overstriping.

lfs mirror read of a parity mirror still dumps every parity object in
RAID0 order, through mirror_dump_parity() in lfs, from e_start.  The
dump of a component stops at its e_end: when the extent is not a
multiple of the data stripe set, the raid sets of the last row past
e_end hold the data of the next component.  The glimpse lock of such
a read spans every component, so lov_lock_sub_init() no longer
rejects a glimpse because a component the I/O does not cover has
fewer parity stripes than the designated one; lov_io_iter_init()
checks the components the I/O covers.

Example: -E 128M -c 11 -S 1M --ec 4+2

  data/parity extent [0, 128M)
  raidsets [4,2], [4,2], [3,2] (raidset_count=3)
  parity has 6 OST objects
  one data stripe set = 11M
  one parity stripe set = 3M (not 1M flat, not 6M concat)

  stripeset raidset data_pos    ec_off  concat  objects
  --------- ------- ----------- ------- ------- -------
  0         0       [0, 4M)     0       0       0,1
  0         1       [4M, 8M)    1M      2M      2,3
  0         2       [8M, 11M)   2M      4M      4,5
  1         0       [11M, 15M)  3M      6M      0,1
  1         1       [15M, 19M)  4M      8M      2,3
  1         2       [19M, 22M)  5M      10M     4,5

The concat column is what RAID0|PARITY would have used, and is what
overflows a finite extent when p > k.

Tests: sanity-ec 6g rejects clearing LCME_FL_PARITY, 12c/12d/34c cover
holes and multi-raidset offsets, 30d and 31b-31e check the parity dump
size, 34d dumps a mixed-p mirror whose first extent is not a multiple
of the stripe set, 63d recovers reads from parity of raidset 1 in a
PFL component that does not start at 0, 301 covers an e_start that
stripe_size does not divide and checks that a component whose parity
would pass e_end is refused for that reason, 302 resyncs, verifies and
reads degraded common PFL layouts with an unaligned e_start, 303
recovers reads when e_start is not a multiple of the stripe width,
with equal and unequal raidsets, 304 covers zero-fill before e_start,
and 312 truncates at raidset and unaligned component boundaries.
llapi_layout_test asserts the component pattern is LLAPI_LAYOUT_PARITY
and 68 checks the parity placement over a range of geometries and
extents.

Test-Parameters: testlist=sanity-ec ostcount=8
Test-Parameters: testlist=sanity-ec ostcount=8 fstype=zfs
Signed-off-by: Keguang Xu <kxu@ddn.com>
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Assisted-by: Cursor:Composer/Opus5.5 llm_code_and_review_tools
Assisted-by: ClaudeCode:claude-opus-5-5 llm_code_and_review_tools
Assisted-by: ClaudeCode:claude-fable-5-1 llm_code_and_review_tools
Change-Id: I9574df44969ddd40776f028b3c326d150ea08eb5

66882 master J –B ✓T …R +1/2

LU-20211 mdt: trunc(0) layout replace on inactive stripes

reviewing Keguang Xu · awaiting review (CI not green yet)

pt_ecro218_review_priority 2 unresolved XL +1320 −64

11h

build #135174 SUCCESS · tests all Maloo sessions · test queue

reviews: Andreas Dilger +1 2026-11-02 04:54

no vote yet: Ronnie Sahlberg, Qian Yingjin, Zhenyu Xu

patchsets:

+29 earlier

PS1 06-26 PS13 07-27 PS14 07-28 PS15 07-31 PS16 08-05 PS17 08-06 PS18 08-07 PS19 08-12 PS20 08-20 PS21 08-25 PS22 08-25 PS23 09-02 PS24 09-08 PS25 09-08 PS26 09-09 PS27 09-11 PS28 09-17 PS29 10-08 Andreas Dilger PS30 10-08 Andreas Dilger PS31 10-08 Andreas Dilger PS32 10-12 PS33 10-12 PS34 10-13 Andreas Dilger PS35 10-15 Andreas Dilger PS36 10-15 Andreas Dilger PS37 10-16 Andreas Dilger PS38 10-19 Andreas Dilger PS39 10-22 Andreas Dilger PS40 10-26 Andreas Dilger

PS41 10-28 PS42 10-30 PS43 11-02 Andreas Dilger PS44 11-02 Andreas Dilger PS45 11-02 Andreas Dilger

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS41 2026-10-28 23:26 Ronnie Sahlberg cl_object.h:226 — Or change CL_FSYNC_LOCAL = 0 and a BUILD_BUG_ON(CL_FSYNC_LOCAL != 0) would make it safe if future code blindly clears the conf where t

Ronnie Sahlberg PS41 · 2026-10-28 23:14

The risk is for future code. A new layout-change caller that just zeroes conf would get CL_FSYNC_NONE without anyone noticing. Prune would start writeback but not wait for it, and the old layout would be torn down while dirty pages were still being written to its objects. The likely result is lost or misdirected writes, or crashes and assertion failures during cleanup. The code compiles fine and nothing warns, so this would be hard to trace back to the missing field.

Ronnie Sahlberg PS41 · 2026-10-28 23:26

Or change CL_FSYNC_LOCAL = 0     and a BUILD_BUG_ON(CL_FSYNC_LOCAL != 0)   would make it safe if future code blindly clears the conf where this enum is embedded.

reply PS41 2026-10-28 16:32 Gerrit AI review for Lustre mdd_object.c:4453 — (style) This isn't a bug, but the kernel-doc for this non-static function does not describe @env, @obj or @mlc, and has no Return: section.

Gerrit AI review for Lustre PS41 · 2026-10-28 16:32

(style) This isn't a bug, but the kernel-doc for this non-static function does not describe @env, @obj or @mlc, and has no Return: section. `kernel-doc -v` warns about all three parameters. If the patch is refreshed, consider adding them.

tags: pt_ecro 218_review_priority

owner Keguang Xu · uploader Andreas Dilger · PS45 uploaded 13h ago · NEW · open in Gerrit ↗

commit message

LU-20211 mdt: trunc(0) layout replace on inactive stripes

When truncate(0) hits a file whose layout references deactivated or
disconnected OSTs, and the client advertises
OBD_CONNECT2_LAYOUT_REPLACE, replace the layout: move the old layout
onto a volatile victim and instantiate a fresh layout on the file from
the existing pattern via QoS/RR.  A full or read-only OST is still
usable and does not trigger this.  If the new layout cannot be
allocated, keep the old layout and do a normal trunc(0).
Skip it for a lease holder (MDS_TRUNC_KEEP_LEASE): lfs mirror resync
and copy truncate only one mirror, and the in-sync mirror must stay.
An OST is offline if it was deactivated or its OSP import is not
connected, which is checked directly since LOD only learns of the
disconnection at its next statfs refresh.

After the layout swap, do not update the parent FID of the volatile
victim's objects, which are destroyed on close, nor of any object on
an offline OST.  That update reads the object's filter_fid first, and
the RPC would wait for an unreachable OST while LAYOUT EX is held.
An OST that reconnects between the declare and execute passes is also
skipped at execute, since no OSP sub-transaction was declared for it.

Skip replace when any component has LCME_FL_NOSYNC.  That flag freezes
the mirror's existing objects; swap would discard the snapshot.  The
NOSYNC test runs once, inside mdt_replace_unavailable_layout(), which
returns 0 so setattr falls through to a normal trunc(0).

mdt_reint_setattr() does not check NOSYNC.  It probes for an
unavailable layout before mdt_attr_set() and without a DLM lock, so a
healthy trunc(0) skips LAYOUT EX.  If the probe finds one,
mdt_attr_set() takes UPDATE|LAYOUT|XATTR EX in a single enqueue and
replaces the layout under it.  A second lock on the file while the
saved UPDATE lock is held could queue behind a setxattr waiting for
both, and deadlock (LU-3601).  The unavailable-stripe result is
re-checked under the lock, so a concurrent trunc(0) that already
replaced the layout does not allocate a second set of objects.

Extend the layout API with MD_LAYOUT_UNAVAILABLE (detect stripes on
offline OSTs) and MD_LAYOUT_INSTANTIATE (allocate from a source
pattern).  Add lod_generate_lovea_template() to emit uninstantiated
LOVEA templates that omit placement hints so new stripes use QoS/RR,
and trigger replacement from mdt_reint_setattr() via
mdt_replace_unavailable_layout().

Layout-type handling for templates / instantiate:
- DoM: never carries OST objects; force stripe_count to 0 and skip
  the OST object array so LOVEA size matches INIT DoM.
- PFL/DoM: keep the DoM component and replace only OST components.
- SEL: coerce LOV_MAGIC_SEL to COMP_V1 for lod_qos_parse_config();
  keep LCME_FL_EXTENSION and restamp SEL when LOVEA is written.
- EC: re-emit bound data/parity as unbound (IS_LINK_ID + shared
  link id) so lod_qos_prep_create() can pair before component IDs
  exist.

Layout replace creates a volatile victim via mdt_orphan_open().
Set mlc_no_inherit on MD_LAYOUT_INSTANTIATE so the new layout does
not inherit the local ROOT default pool from the orphan parent.
Zero mti_spec first defensively before the volatile create;
mdt_thread_info_reset() already clears it per request, and setattr
does not leave create state there.

After a successful swap, sync the object before replying so the new
layout is durable; replay skips replace.  On success, return
MS_LAYOUT_REPLACED (with OBD_MD_FLEASIZE) in the setattr reply so
the client applies the new LOVEA.  Before applying it, the client
discards its dirty pages past the returned size (all of them for
trunc(0)), so prune does not write them back to unavailable OSTs.
This lets a later server replace only the OST objects past a
non-zero truncate size.  The client reserves the default layout size
in the setattr reply buffer of any truncate.  A resent trunc(0)
setattr (lost or truncated reply) returns the current layout with
MS_LAYOUT_REPLACED, whether or not the original request replaced it;
the client ignores a layout whose generation is unchanged.

Add sanity/test_27xt (plain, PFL/DoM, SEL), sanity-ec/test_74b (EC),
and sanity-flr/test_50e (mirrors, resync), which stop the OST.  Add
sanity/test_27xv, which holds the replace with
OBD_FAIL_MDS_LAYOUT_REPLACE_PAUSE while a setxattr from a second
mount waits for the file, with sync_lock_cancel=never so the saved
UPDATE lock stays PW, to check for the deadlock.

Test-Parameters: testlist=sanity serverversion=2.17 env=SANITY_EXCEPT="24D 27D"
Test-Parameters: testlist=sanity-flr ostcount=5 serverversion=2.17 env=SANITY_FLR_EXCEPT="0a 34b"
Test-Parameters: testlist=sanity clientversion=2.17
Assisted-by: Cursor:Grok-4.6 llm_code_and_review_tools
Signed-off-by: Keguang Xu <kxu@ddn.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Assisted-by: Cursor:Composer
Change-Id: I07e0792a51e144d935cece4b028e17f659e4ccfd

69743 master J –B ✓T …R 0/2

LU-20455 tests: run subtests from separate .ltsh files

reviewing Andreas Dilger · awaiting review (CI not green yet)

XL +1744 −186

15h

build #135158 SUCCESS · tests all Maloo sessions · test queue

no vote yet: Alex Zhuravlev, Timothy Day, Patrick Farrell

patchsets:

+1 earlier

PS1 10-16

PS2 10-27 PS3 10-27 PS4 10-28 PS5 10-29 PS6 11-01

owner Andreas Dilger · uploader Andreas Dilger · PS6 uploaded 16h ago · NEW · open in Gerrit ↗

commit message

LU-20455 tests: run subtests from separate .ltsh files

Test scripts like sanity.sh (38000 lines) and conf-sanity.sh (13000
lines) are hard to read and navigate, and patches conflict often
because every new subtest is added to the same few files.

Allow subtests to be kept in lustre/tests/$TESTSUITE.d/*.ltsh files,
each holding one or more test_NNN() functions and the helpers they
share.  build_test_filter() finds the files and reads the YAML header
in the leading comment block of each one, run_test() first runs the
file subtests that sort before the subtest it is called for, and runs
the rest after the last subtest in $TESTSUITE.sh, before the suite's
own cleanup.  The suite scripts need no changes, and the subtests run
in numeric order interleaved with the subtests still in
$TESTSUITE.sh.  ONLY, EXCEPT, always_except(), EXCEPT_SLOW,
START_AT and STOP_AT apply to them as usual.  Each file is sourced
only when the first of its subtests runs.  The code is in the new
ltsh.sh, sourced by test-framework.sh, with the header parser in
ltsh.awk.

The header gives the summary passed to run_test(), the commits that
added or changed the subtest, whose first ticket is added to the
banner unless a Jira ticket is given, a description, a list of
keywords for the modules and features it exercises, and its typical
duration, which are not printed but help to find and understand the
subtest.  It replaces the usual skip checks at the start of a subtest:
MDT, OST and client counts, MDS and OSS node counts, server and client
versions, OSD type, separate MGS, required tools, free space and
inodes of the filesystem or of the least free MDT or OST, and SLOW,
with a "reason:" added to the skip message.  "start:" and "end:"
declare the filesystem state (mounted, unmounted with servers up,
stopped, or freshly formatted) that the subtest needs and leaves, by
default mounted.  The framework changes the state only as needed and
trusts "end:" rather than checking it, so that later the subtests can
be reordered to avoid needless remounts.  "formatted" has the
framework reformat before or after the subtest, skipping the reformat
at the start if the previous subtest reformatted at its end.  "fsck:"
checks the ldiskfs targets with run_e2fsck() after each subtest that
passes, or after the last subtest of the file if it passes.  This
setup runs in the subtest's subshell, so a failure fails only that
subtest.  "expect: must_fail" or "should_fail" runs the subtest with
run_test_must_fail() or run_test_should_fail(), where a header, setup
or fsck error is also the expected failure.  "modifies:" and
"observes:" list the shared state a subtest changes, such as fail_loc,
mounts or config, and the state it reads, such as stats or free space,
from a fixed list.  With PARALLEL=yes only subtests that give both
lists empty are run.  Lists in the header of the file add to those of
each subtest.  A file with one subtest can leave out "subtests:".  The
headers are valid YAML, so other tools can read them.  Errors in a
subtest's header keys fail that subtest, YAML errors fail all subtests
of that file, and a test_NNN() missing from the header fails as a
subtest of its own, rather than any of them stopping the whole suite.

conf-sanity-framework.sh uses its own reformat_and_config(), setup(),
start_servers() and stop_servers() for this, which keep a separate MGS
running, and keeps the client unmounted between subtests.

Move some subtests to exercise each part of this, with a description
and the list of commits of each from git history and the tickets.
Their version checks now use the "git describe" form of the commit
that added each feature, since the history of those lines is lost by
the move:
- sanity 27I: single subtest without "subtests:", OST count,
  modifies: and observes:
- sanity 230i, 230j: MDT count for both subtests with a different
  version check for each, modifies: and observes: only for 230j
- conf-sanity 151a, 151b: OSD type, start stopped, end formatted
- conf-sanity 153a, 153c: start formatted, run before and after 153b
  in conf-sanity.sh, shared helper, fsck after the last subtest.  Also
  mount $FSNAME there instead of a hard-coded "lustre".

Add sanity-ltsh.sh, in no test group, to test the framework itself
with the subtests in sanity-ltsh.d/: filesystem state changes, skips
from "requires:", header errors, fsck finding corruption and not
running after a failed subtest, and "expect:".  "fsck:" needs the
LU-20967 fix, or e2fsck finds the unnamed failure_domain file on the
OSTs.

Have the lustre-test-scripts agent skill put new subtests in .ltsh
files and describe how to move existing ones, and mention the files in
AGENTS.md.

Describe file format and header fields in lustre/tests/README-ltsh.md.
Install the files with the test scripts, and have prepare-commit-msg
run shellcheck on them as it does for *.sh files, with --shell=bash
since they have no #! line.  Also fix its pattern for newly added
files ("new file:"), which never matched, and default the line range
of a file that has no hunk in HEAD, which was an arithmetic error that
stopped all checks, so new *.sh files are checked as well.
checkpatch.pl applies its shell script checks to *.ltsh files, and
allows long lines that end in a commit reference like those of
"Fixes:", as the header "commit:" lists use.

Test-Parameters: testlist=sanity-ltsh
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: Ie5a6c96781d8ff41f40b3eee7c2aef18ef3ebbe5

70067 master J –B ✓T ✗3R +2/2

LU-20690 ec: copy parity in 1+N recovery reads

reviewing Zhenyu Xu · blocked on the owner: enforced tests failed

218_review_prioritypt_ecro 1 unresolved M +87 −14

blocked on the owner: enforced tests failed

awaiting review (CI not green yet)

failed enforced test	platform	detail
custom-1001	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session
review-dne-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-ec.	session

build #134529 SUCCESS · tests all Maloo sessions

reviews: Keguang Xu +1 2026-10-27 12:11 · Ronnie Sahlberg +1 2026-10-29 21:30

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-27 23:25 Gerrit AI review for Lustre COMMIT_MSG:29 — (minor) This fixes data corruption on degraded reads, so should it carry a Fixes: line? The wrong decode for a single-data-stripe raidset go

Gerrit AI review for Lustre PS1 · 2026-10-27 23:25

(minor) This fixes data corruption on degraded reads, so should it carry a Fixes: line? The wrong decode for a single-data-stripe raidset goes back to the original recovery code, not only to --ec 1+N:

Fixes: 783c13bb0b8f ("LU-12669 ec: recover data from parity")

lov_io_ec_rd_start() has always used the per-raidset dcount (sc.esc_k1 for the tail raidsets). llapi_ec_compute_parities() is also called with the per-raidset k, so it memcpy()s for those raidsets. ec_split_stripes(5, 2) gives n0=2 k0=2 n1=1 k1=1, so a `-c 5 --ec 2+2` file (and the same for -c 7, -c 9, ...) has a 1+2 tail raidset. Before this patch, losing that raidset's data stripe and parity 0 returned data * 2. That layout was valid before LU-20568 allowed --ec 1+N.

If the patch is refreshed, it may help to say in the body that this case is fixed as well, since it affects released layouts and matters when deciding on backports.

tags: 218_review_priority pt_ecro

owner Zhenyu Xu · uploader Zhenyu Xu · PS1 uploaded 6d ago · NEW · open in Gerrit ↗

commit message

LU-20690 ec: copy parity in 1+N recovery reads

A raidset with one data stripe stores a copy of that stripe in each of
its p parity objects: llapi_ec_compute_parities() memcpy()s the data
instead of calling ec_encode_data() when k == 1, so --ec 1+N writes p
identical copies.  The client recovery path did not know this and
always ran the Cauchy decode, where the coefficient of parity stripe i
is gf_inv(i + 1).  Only parity stripe 0 has coefficient 1, matching
what is on disk.  A read that lost the data stripe and parity stripe
0, and fell back to any other parity, thus returned data * 2, * 3,
... rather than the data.

Copy the first parity page that could be read over the error page
when the raidset has one data stripe, in lov_ec_recover_page().  The
RS setup is of no use in that case either, so skip the encode matrix
in lov_ec_bufs_alloc() and the scratch matrices, GF tables, matrix
inversion and ec_init_tables() in lov_ec_recover_init() when dcount
is 1.

Add sanity-ec test 40e, which fails the data OST and the OST of
parity stripe 0 in a 1+2 layout, so the read has to be rebuilt from
parity stripe 1.

Test-Parameters: testlist=sanity-ec ostcount=8
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Assisted-by: AugmentAgent:DeepSeek-V4.1-Flash
Change-Id: Idbe55e61f464d1e91bb5093bf1c41442dc588f72

63678 master J ✗1B ✓T ✗5R -2/2−

LU-19839 llite: add persistent nohybrid I/O flag

reviewing Patrick Farrell · blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; janitor: test failures unique to this patch; enforced tests failed

needs rebase 1 unique failure 20 unresolved M +124 −5

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; janitor: test failures unique to this patch; enforced tests failed

awaiting review (CI not green yet)

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity2@zfs:test_119l	NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews

failed enforced test	platform	detail
review-dne-selinux-ssk-part-2	RHEL 8.10 / x86_64	ran 5 tests. 1 tests failed: sanity-sec.	session
review-dne-zfs-part-1	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-zfs-part-1	RHEL 10.1 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-dne-zfs-subtest-change	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64	ran 6 tests. 1 tests failed: sanity-lnet.	session

build #127956 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger

patchsets:

+4 earlier

PS1 01-30 PS2 01-30 PS3 01-30 PS4 01-30

PS5 02-03 PS6 02-03 PS7 02-03 Janitor Bot PS8 07-16 Maximilian Dilger PS9 07-22 Maximilian Dilger

threads: 20 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS7 2026-02-04 06:05 Andreas Dilger COMMIT_MSG:15 — (minor) ... so it could also be set and checked via `chattr +t FILE` and `lsattr FILE` commands from e2fsprogs.

Andreas Dilger PS7 · 2026-02-04 06:05

(minor) ... so it could also be set and checked via `chattr +t FILE` and `lsattr FILE` commands from e2fsprogs.

reply PS9 2026-07-28 11:36 Gerrit AI review for Lustre COMMIT_MSG:21 — The "Changes:" list mentions adding LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE, but the same hunk also adds LUSTRE_COMPR_FL to that mask

Gerrit AI review for Lustre PS9 · 2026-07-28 11:36

The "Changes:" list mentions adding LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE, but the same hunk also adds LUSTRE_COMPR_FL to that mask and removes the duplicate LUSTRE_NOATIME_FL entry. Neither is explained. The COMPR change is an independent server-side behavior change - should it be split into its own patch?

The lustre/utils/lfs.c hunk (skipping the range validation for nohybrid) isn't described either.

reply PS9 2026-07-28 11:36 Gerrit AI review for Lustre COMMIT_MSG:25 — This changes UAPI flag definitions that the MDT interprets, so interop with an older server is worth an explicit test run. Consider adding s

Gerrit AI review for Lustre PS9 · 2026-07-28 11:36

This changes UAPI flag definitions that the MDT interprets, so interop with an older server is worth an explicit test run. Consider adding something like:

    Test-Parameters: testlist=sanity serverversion=2.16.0

reply PS9 2026-07-28 11:36 Gerrit AI review for Lustre lustre_idl.h:1891 — New wire flag, but wirecheck.c and the two wiretest.c copies don't look updated - the neighbouring LUSTRE_*_FL values have CHECK_VALUE_X ent

Gerrit AI review for Lustre PS9 · 2026-07-28 11:36

New wire flag, but wirecheck.c and the two wiretest.c copies don't look updated - the neighbouring LUSTRE_*_FL values have CHECK_VALUE_X entries. Same for LU_LADVISE_NOHYBRID, which needs a CHECK_VALUE next to the other LU_LADVISE_* ones (LU_LADVISE_AHEAD seems to have been missed earlier too).

reply PS9 2026-07-28 11:36 Gerrit AI review for Lustre lustre_idl.h:1919 — The compatibility claim points at the wrong side. Older clients aren't the problem; older servers are.

Gerrit AI review for Lustre PS9 · 2026-07-28 11:36

The compatibility claim points at the wrong side. Older clients aren't the problem; older servers are.

On a pre-patch MDS, LUSTRE_FL_USER_VISIBLE has no NOTAIL bit, so mdt_setattr_unpack() hits

    if (rec->sa_attr_flags & ~LUSTRE_FL_USER_VISIBLE)
            RETURN(-EOPNOTSUPP);

and lfs ladvise -a nohybrid fails with EOPNOTSUPP. Clearing the flag still "succeeds" as a no-op, so the two directions behave differently. Worth stating the required server version here.

reply PS9 2026-07-28 11:36 Gerrit AI review for Lustre lustre_idl.h:1929 — This was asked on patchset 7 and looks unaddressed: adding LUSTRE_COMPR_FL here is unrelated to nohybrid and changes server behavior.

Gerrit AI review for Lustre PS9 · 2026-07-28 11:36

This was asked on patchset 7 and looks unaddressed: adding LUSTRE_COMPR_FL here is unrelated to nohybrid and changes server behavior.

mdt_setattr_unpack() masks with LUSTRE_FL_USER_MODIFIABLE, and osd_attr_set() replaces the whole masked set, so this makes the compression flag both settable and clearable on the MDT inode by any client. Is that intended ahead of the compression work landing?

reply PS7 2026-02-04 06:05 Andreas Dilger lustre_idl.h:1909 — (defect) why is NOATIME being removed?

Andreas Dilger PS7 · 2026-02-04 06:05

(defect) why is NOATIME being removed?

reply PS7 2026-02-04 06:05 Andreas Dilger lustre_idl.h:1917 — Is the COMPR flag really user modifiable or just visible? And does it make sense to allow this to be set before CSDC is landed to master?

Andreas Dilger PS7 · 2026-02-04 06:05

Is the COMPR flag really user modifiable or just visible?  And does it make sense to allow this to be set before CSDC is landed to master?

reply PS7 2026-02-04 06:05 Andreas Dilger lustre_idl.h:1919 — Should this use NOHYBRID?

Andreas Dilger PS7 · 2026-02-04 06:05

Should this use NOHYBRID?

reply PS7 2026-02-04 06:05 Andreas Dilger file.c:5011 — Does this also set the flag directly on the inode, or is the inode here the root or parent directory?

Andreas Dilger PS7 · 2026-02-04 06:05

Does this also set the flag directly on the inode, or is the inode here the root or parent directory?

reply PS8 2026-07-16 22:16 Misc Code Checks Robot (Gatekeeper helper) file.c:5116 — parse ll_file_ioctl():error: Function too hairy. Giving up. 4 seconds

Misc Code Checks Robot (Gatekeeper helper) PS8 · 2026-07-16 22:16

parse ll_file_ioctl():error: Function too hairy.  Giving up. 4 seconds
warn: ll_file_ioctl():Function too hairy.  No more merges.

reply PS9 2026-07-28 11:36 Gerrit AI review for Lustre file.c:4994 — ll_inode2ext_flags() is not a full picture of the file's flags - it rebuilds them from inode->i_flags via ll_inode_to_ext_flags(), which onl

Gerrit AI review for Lustre PS9 · 2026-07-28 11:36

ll_inode2ext_flags() is not a full picture of the file's flags - it rebuilds them from inode->i_flags via ll_inode_to_ext_flags(), which only knows SYNC/NOATIME/APPEND/DIRSYNC/IMMUTABLE/ENCRYPT, plus the PROJINHERIT and (new) NOHYBRID lli_flags bits.

LUSTRE_NODUMP_FL and LUSTRE_NOCOMPR_FL are in LUSTRE_FL_USER_MODIFIABLE but have no i_flags or lli_flags representation, so they come back as 0 here. osd_attr_set() then does a wholesale replace:

    ei->i_flags = (ei->i_flags & ~LDISKFS_OSD_USER_MODIFIABLE) |
                  (attr->la_flags & LDISKFS_OSD_USER_MODIFIABLE);

so those bits get cleared on disk. chattr +d FILE followed by lfs ladvise -a nohybrid FILE should lose the 'd' flag.

The FS_IOC_SETFLAGS path avoids this by calling fileattr_get() first, which fetches body->mbo_flags from the MDT. Should this do the same before OR-ing in LUSTRE_NOHYBRID_FL?

Related: ll_set_project() builds op_attr_flags from ll_xflags_to_ext_flags() and also sets OP_XVALID_FLAGS, so lfs project on a file appears to clear the nohybrid flag for the same reason.

owner Patrick Farrell · uploader Maximilian Dilger · PS9 uploaded 102d ago · NEW · open in Gerrit ↗

commit message

LU-19839 llite: add persistent nohybrid I/O flag

Add a persistent flag to prevent hybrid I/O switching
for specific files. This uses the NOTAIL flag (0x00008000)
which is not used by Lustre/ext4 and unlikely to be used
in the future.

The flag is set via ladvise interface for discoverability
but uses FS_IOC_SETFLAGS internally for implementation.
When set, hybrid I/O will not switch the file from
buffered to direct I/O regardless of I/O size.

Changes:
- Define LUSTRE_NOTAIL_FL and LUSTRE_NOHYBRID_FL
- Add LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE
- Implement LU_LADVISE_NOHYBRID using FS_IOC_SETFLAGS
- Update ll_update_inode_flags to handle NOTAIL flag
- Add test_119l to verify nohybrid flag functionality

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I6a69293801114e2a3015ed87f2258828922ab767

66567 master J –B ✓T ✗10R 0/2

LU-20212 lov: skip invalid OST mirrors for aggregate I/O

reviewing Keguang Xu · blocked on the owner: enforced tests failed

4 unresolved L +216 −39

blocked on the owner: enforced tests failed

awaiting review (CI not green yet)

failed enforced test	platform	detail
review-dne-part-4	RHEL 9.7 / x86_64	ran 11 tests. 1 tests failed: sanity-flr.	session
review-dne-part-4	RHEL 10.1 / x86_64	ran 11 tests. 1 tests failed: sanity-flr.	session
review-dne-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-ec.	session
review-dne-subtest-change	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-flr.	session
review-dne-zfs-part-4	RHEL 9.8 / x86_64	ran 11 tests. 1 tests failed: sanity-flr.	session
review-dne-zfs-part-4	RHEL 10.1 / x86_64	ran 11 tests. 1 tests failed: sanity-flr.	session
review-dne-zfs-part-5	RHEL 9.7 / x86_64	ran 5 tests. 1 tests failed: sanityn.	session
review-dne-zfs-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-ec.	session
review-dne-zfs-subtest-change	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-flr.	session
review-zfs	RHEL 8.10 / x86_64	ran 10 tests. 1 tests failed: sanity-flr.	session

build #134448 SUCCESS · tests all Maloo sessions

no vote yet: Qian Yingjin, Andreas Dilger

patchsets:

+2 earlier

PS1 06-10 PS2 06-10

PS3 06-11 PS4 06-13 PS5 06-16 PS6 06-22 PS7 10-26

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-06-16 09:12 kg.xu patchset comment — I reused `lovsub_import_inactive()` from the EC series and incorporated logic to skip parity for `CIT_GLIMPSE`. Given these dependencies...S

Keguang Xu PS1 · 2026-06-10 03:33

Hi, this patch is part of 'FLR-ECRO: verify non-write RPCs do not block client': I was wondering if the `fail fast` logic on `stat()` should actually be a generic feature for all PFL components, instead of being scoped exclusively to EC files. If that makes sense, I'd prefer to rebase this against master and relocate the tests under sanity.sh. Thoughts?

kg.xu PS1 · 2026-06-16 09:12

I reused `lovsub_import_inactive()` from the EC series and incorporated logic to skip parity for `CIT_GLIMPSE`. Given these dependencies...So, let's keep it within this series.

reply PS4 2026-06-16 09:12 kg.xu patchset comment — I reused `lovsub_import_inactive()` from the EC series and incorporated logic to skip parity for `CIT_GLIMPSE`. Given these dependencies...S

Andreas Dilger PS4 · 2026-06-13 08:25

This patch should probably go directly on master, since I don't think it depends on any functionality that hasn't landed yet?

kg.xu PS4 · 2026-06-16 09:12

I reused `lovsub_import_inactive()` from the EC series and incorporated logic to skip parity for `CIT_GLIMPSE`. Given these dependencies...So, let's keep it within this series.

reply PS5 2026-06-16 10:34 kg.xu patchset comment — In lov_io_mirror_init(), if `ci_designated_mirror` is set, we can also check lov_mirror_has_inactive_stripe for `CIT_GLIMPSE/LSEEK/DATA_VERS

kg.xu PS5 · 2026-06-16 10:34

In lov_io_mirror_init(), if `ci_designated_mirror` is set, we can also check lov_mirror_has_inactive_stripe for `CIT_GLIMPSE/LSEEK/DATA_VERSION`. Since these io types all trigger OST aggregation, this check allows us to skip unnecessary RPCs.

In current impl, we only check `CIT_GLIMPSE` during mirror selection (where `ci_designated_mirror` is false).

reply PS4 2026-06-16 09:12 kg.xu sanity-ec.sh:3899 — An error returned:

wc-checkpatch PS4 · 2026-06-13 04:04

(style)  line length of 81 exceeds 80 columns

Andreas Dilger PS4 · 2026-06-13 08:25

How does this behave for a non-EC file with a deactivated OST stripe?  Does that return an error or just calculate the size without the stripe (which seems wrong)?

kg.xu PS4 · 2026-06-16 09:12

An error returned:
`stat: cannot statx '/mnt/lustre/f42c.sanity-ec': Cannot send after transport endpoint shutdown`

So the old code itself is correct already. New change only avoid sending unnecessary glimpse to mirrors with invalid OSTs.

owner Keguang Xu · uploader Keguang Xu · PS7 uploaded 7d ago · NEW · open in Gerrit ↗

commit message

LU-20212 lov: skip invalid OST mirrors for aggregate I/O

I/O that aggregates stripe-local results into one logical answer
(glimpse, lseek, data_version) cannot succeed on a mirror while
any relevant OST import is deactivated or invalid.

In lov_io_mirror_init(), skip mirrors with unreachable OSTs in the
I/O extent during FLR selection for aggregate I/O.  When every
mirror is skipped, glimpse (ndelay) returns -EAGAIN after backoff
so the caller retries, and -EIO once it has tried all mirrors
several rounds; lseek and data_version fail with -EIO at once.

For designated-mirror I/O, apply the same extent-scoped check after
resolving ci_designated_mirror; there is no alternate mirror to try,
so return -EIO immediately.

Detect unreachable stripes with lov_mirror_extent_unreachable(),
limited to stripes intersecting the current I/O extent, via
lov_sub_unreachable().

Skip parity mirrors for glimpse and data_version as well, so they
never return parity object size or version.  Size FLR ndelay rounds
by the data mirrors that can serve the I/O, not lo_mirror_count, for
I/O types that skip parity; this also excludes stale mirrors from
the round of a plain FLR read.  CIT_EC_RD skips that rotation.
Add sanity-flr 31b.

Test-Parameters: fortestonly
Assisted-by: Cursor:Grok-4.6/Opus5.5 llm_code_and_review_tools
Signed-off-by: Keguang Xu <kxu@ddn.com>
Change-Id: Iac1c6415127d46fd958016a998ceab68663e0d42

68618 master J !B ✓T ✗2R -1/2−

LU-20684 lod: timestamp component initialization

reviewing Praveen · blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; enforced tests failed

needs rebase 7 unresolved L +284 −19

11d

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; enforced tests failed

awaiting review (CI not green yet)

janitor run: 1 failing config(s), none unique to this patch — janitor results

failed enforced test	platform	detail
review-dne-zfs-part-7	RHEL 9.8 / x86_64	ran 6 tests. 1 tests failed: sanity-scrub.	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64	ran 6 tests. 1 tests failed: sanity-lnet.	session

build #131115 SUCCESS · tests all Maloo sessions

no vote yet: Etienne AUJAMES, Aurelien Degremont, Andreas Dilger

patchsets: PS1 09-03 PS2 09-04

threads: 7 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-09-04 22:09 Gerrit AI review for Lustre COMMIT_MSG:14 — (minor) "Add PFL, SEL replay, and HSM restore coverage" - the SEL case added to sanity-pfl test_22a exercises component repeat, not replay.

Gerrit AI review for Lustre PS1 · 2026-09-04 22:09

(minor) "Add PFL, SEL replay, and HSM restore coverage" - the SEL case added to sanity-pfl test_22a exercises component repeat, not replay. Is "replay" a typo for "repeat"?

Either way, nothing in the patch exercises the replay case that the llc_timestamp == 0 guard in lod_comp_set_init_timestamp() exists for. That branch only matters when lod_use_defined_striping() has already unpacked a timestamp from a client-supplied LOV_MAGIC_DEFINED layout, and none of the new tests reach it.

reply PS1 2026-09-04 22:09 Gerrit AI review for Lustre lfs-getstripe.1:128 — (minor) This describes lcme_timestamp inside the list of valid names for --component-flags, where the reader is looking for flag names to fi

Gerrit AI review for Lustre PS1 · 2026-09-04 22:09

(minor) This describes lcme_timestamp inside the list of valid names for --component-flags, where the reader is looking for flag names to filter on rather than output fields. Would it read better next to the other getstripe output fields?

Also, "may be retained" is hard to act on without knowing what it is retained from. Retained across a SEL repeat and across replay is what the code does - saying that explicitly would help.

The nosync entry a few lines below still says the timestamp is printed for replicas carrying that flag, which now reads as though nosync is the only case; every initialized component of a composite file prints one after this patch.

reply PS2 2026-09-06 19:09 Gerrit AI review for Lustre lfs-getstripe.1:120 — (typo) "when an uninitialized SEL component is repeated" has it the wrong way round - lod_layout_repeat_comp() asserts lod_comp_inited(lod_c

Gerrit AI review for Lustre PS2 · 2026-09-06 19:09

(typo) "when an uninitialized SEL component is repeated" has it the wrong way round - lod_layout_repeat_comp() asserts lod_comp_inited(lod_comp), so it is an *initialized* component that gets repeated, and it is the new (uninitialized) copy whose timestamp is cleared.

Also, line 116 says a mirror link ID "or" a timestamp is printed, but a component with both a nonzero link ID and a timestamp prints both lines, and a nosync component prints its timestamp even when it is zero.

reply PS2 2026-09-06 19:09 Gerrit AI review for Lustre lfs-getstripe.1:125 — (minor) .PP inside a .TP block resets the indent, so everything after it - the "If COMP_FLAG is specified ..." paragraph and the whole init/

Gerrit AI review for Lustre PS2 · 2026-09-06 19:09

(minor) .PP inside a .TP block resets the indent, so everything after it - the "If COMP_FLAG is specified ..." paragraph and the whole init/stale/prefer/nosync flag list - is now rendered at the option-tag margin instead of the description margin, as if it no longer belonged to --component-flags.

Compare `man -l Documentation/man1/lfs-getstripe.1` before and after; .IP restores the old layout.

reply PS1 2026-09-04 22:09 Gerrit AI review for Lustre sanity-pfl.sh:1182 — (style) This isn't a bug, but get_component_timestamp() is added verbatim here and in sanity-hsm.sh. If the patch is refreshed, would test-f

Gerrit AI review for Lustre PS1 · 2026-09-04 22:09

(style) This isn't a bug, but get_component_timestamp() is added verbatim here and in sanity-hsm.sh. If the patch is refreshed, would test-framework.sh be a better home so both suites (and any later ones) share one copy?

reply PS2 2026-09-06 19:09 Gerrit AI review for Lustre sanity-pfl.sh:2122 — (minor) Unlike the new 17a and sanity-hsm 1g, 22a creates its directory with `test_mkdir -p`, which under DNE uses dirstripe_index=$((base %

Gerrit AI review for Lustre PS2 · 2026-09-06 19:09

(minor) Unlike the new 17a and sanity-hsm 1g, 22a creates its directory with `test_mkdir -p`, which under DNE uses dirstripe_index=$((base % MDSCOUNT)) - so with MDSCOUNT=4 $tdir is created at MDT2 with -c 2 and the file's layout timestamp comes from mds3 or mds4, not mds1.

The [$before,$after] window is then read off a different node's clock, so this bound only holds while the MDS clocks agree. Would `mkdir_on_mdt0` here (as in 17a), or taking the date from the facet that actually owns the file, make it robust?

reply PS2 2026-09-06 19:09 Gerrit AI review for Lustre liblustreapi.c:2478 — (minor) Worth a thought: before this patch a timestamp line here was a rare corner case (nosync, or an EC link id), so `lfs getstripe -I1 --

Gerrit AI review for Lustre PS2 · 2026-09-06 19:09

(minor) Worth a thought: before this patch a timestamp line here was a rare corner case (nosync, or an EC link id), so `lfs getstripe -I1 --comp-flags $file` was effectively a single-value query. Now every initialized component of every composite file gets one, so that command returns two lines for the common case.

The patch handles it for the in-tree suites by adding get_component_flags(), which throws the second line away with `sed -n 1p`, but external scripts doing [ "$(lfs getstripe -I1 --comp-flags $f)" = init ] will break silently.

Would it be better to print the timestamp only in the labelled dump - i.e. under `verbose & ~VERBOSE_COMP_FLAGS`, the same condition already used for the "lcme_timestamp:" label - so --comp-flags on its own keeps returning just the flags?

owner Praveen · uploader Praveen · PS2 uploaded 58d ago · NEW · open in Gerrit ↗

commit message

LU-20684 lod: timestamp component initialization

Set lcme_timestamp when a composite layout component is first
initialized. Preserve timestamps used by replay, NOSYNC, and EC.
Avoid assigning timestamps to released placeholders.

Clear ordinary timestamps copied to new SEL components so they are
timestamped at their own initialization. Add PFL, SEL repeat and
replay, HSM restore, and layout intent replay coverage. Document the
timestamp semantics.

Parse component flags independently from the timestamp printed after
them. Resolve the generated SEL repeat component by ID instead of
assuming it has component ID 2.

Test-Parameters: testlist=replay-single,sanity-hsm,sanity-pcc
Test-Parameters: testlist=sanity-pfl,sanity-flr,sanity-dom
Test-Parameters: testlist=sanity-ec
Signed-off-by: Praveen Solanki <praveen.solanki@oracle.com>
Change-Id: I66d1e5ca5d2876f8185749d92ee905d38e1c7285

65672 master J ✗1B ✓T ✗2R 0/2

LU-20011 llapi: add flag-based layout sanity API

reviewing Nikos Papakonstantinou · blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

1 unique failure 4 unresolved M +141 −34

23d

blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

awaiting review (CI not green yet)

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity3@ldiskfs+DNE:test_271f	seen in 1 other review

failed enforced test	platform	detail
review-dne-selinux-ssk-part-1 crashed	RHEL 8.10 / x86_64	ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %%	session
review-ldiskfs-ubuntu failed 3×	RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64	ran 6 tests. 1 tests failed: sanity-lnet.	session

2 optional test failure(s) — informational

build #133020 SUCCESS · tests all Maloo sessions

no vote yet: Rajeev Mishra, Andreas Dilger, Nikitas Angelinas

patchsets:

+5 earlier

PS1 05-05 PS2 05-07 PS3 05-11 PS4 05-11 Janitor Bot PS5 05-15

PS6 05-18 PS7 06-12 PS8 06-26 PS9 09-01 PS10 09-16

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS10 2026-09-18 01:24 Gerrit AI review for Lustre COMMIT_MSG:20 — (minor) This tag points at the commit that added llapi_layout_v2_sanity(), which is a fair label for the API rework. But the failure in the

Gerrit AI review for Lustre PS10 · 2026-09-18 01:24

(minor) This tag points at the commit that added llapi_layout_v2_sanity(), which is a fair label for the API rework. But the failure in the subject line predates it: the LSE_INCOMPLETE_MIRROR / "Incomplete mirror - must go to EOF" rejection comes from

    4eca26ddab3186a68888862218fa8904f812e5a1 ("LU-10070 lod: SEL: Layout sanity checking")

and ee7dfc5ad1 only moved pool-name checking around (its lfs.c hunk removed a sanity call from mirror_create_sanity_check(), it did not touch mirror_split()). Since the Fixes: tag drives which maintenance branches get the cherry-pick, would a second line be more accurate?

    Fixes: 4eca26ddab31 ("LU-10070 lod: SEL: Layout sanity checking")

reply PS10 2026-09-18 01:24 Gerrit AI review for Lustre lustreapi.h:1612 — (minor) llapi_layout_v2_sanity() still has an in-tree consumer after this: rustreapi/rustreapi/src/layout.rs exposes it as Layout::sanity_v2

Gerrit AI review for Lustre PS10 · 2026-09-18 01:24

(minor) llapi_layout_v2_sanity() still has an in-tree consumer after this: rustreapi/rustreapi/src/layout.rs exposes it as Layout::sanity_v2() (behind the LUSTRE_2_16 feature), and there is no Rust wrapper for llapi_layout_sanity_check(). If the patch is refreshed, is it worth converting that binding too, so the deprecated entry point has no callers left?

reply PS10 2026-09-18 01:24 Gerrit AI review for Lustre llapi_layout_test.c:4662 — (style) Not a bug, but `path`, T68FILE and this unlink() are leftover boilerplate - test68 only builds an in-memory layout and never creates

Gerrit AI review for Lustre PS10 · 2026-09-18 01:24

(style) Not a bug, but `path`, T68FILE and this unlink() are leftover boilerplate - test68 only builds an in-memory layout and never creates or opens a file, so nothing here is used. Dropping them would also make it obvious the test needs no Lustre mount.

reply PS10 2026-09-18 01:24 Gerrit AI review for Lustre liblustreapi_layout.c:3282 — (style) This isn't a bug, but llapi_layout_file_comp_add() and llapi_layout_file_comp_set() were both moved to llapi_layout_sanity_check() i

Gerrit AI review for Lustre PS10 · 2026-09-18 01:24

(style) This isn't a bug, but llapi_layout_file_comp_add() and llapi_layout_file_comp_set() were both moved to llapi_layout_sanity_check() in this patch while llapi_layout_file_comp_del() was left on the bool form. Converting the third sibling would keep the three consistent.

owner Nikos Papakonstantinou · uploader Nikos Papakonstantinou · PS10 uploaded 47d ago · NEW · open in Gerrit ↗

commit message

LU-20011 llapi: add flag-based layout sanity API

`lfs mirror split` failed with "Incomplete mirror - must go to
EOF" on a file whose mirror was added to a partial PFL layout.

Introduce llapi_layout_sanity_check() with a bitmask-based
llapi_layout_checks enum to replace the old boolean-parameter
APIs. Deprecate llapi_layout_v2_sanity(), and update callers
(including lfs) to use the new interface; mirror_split() now
passes LLAPI_LAYOUT_CHECK_INCOMP_MIRROR to skip the EOF check
only, the mirror-count check still always runs.
Cover it with a new llapi_layout_test test68.

Fixes: ee7dfc5ad1 ("LU-17025 llapi: Verify stripe pool name")
Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com>
Change-Id: I952d346f0dd2db4eb724e3767724705969d7810e

64937 master J –B ✗T –R +0/2−

LU-20016 ec: ISA-L SIMD for userspace and kernel

reviewing Patrick Farrell · blocked on the owner: build failure

21 unresolved XL +36039 −19

30d

blocked on the owner: build failure

awaiting review (CI not green yet)

build #132756 FAILURE

reviews: Ronnie Sahlberg +1 2026-03-27 21:35 · Andreas Dilger -1 2026-04-06 21:18

no vote yet: Qian Yingjin, Hiroshi Nishida

patchsets: PS1 03-27 PS2 03-27 PS3 03-27

threads: 21 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-10-02 19:13 patrick bot COMMIT_MSG:14 — Patrick-Bot: Converting the NASM routines and maintaining a second assembly implementation needs a dedicated change with instruction-equival

Andreas Dilger PS1 · 2026-03-29 08:52

It would make sense to put an LLM to work on converting the NASM .asm files int GCC .S files.  It should be possible to have it iterate on getting the GCC-compiled code to generate the same x86 byte code, or at least compile and run correctly to pass whatever correctness tests exist for this code. 

In fact, a quick search shows that NASM already has a tool to do this:

    Automatically convert Intel NASM assembly to GNU Assembler (GAS/GASM)
    syntax using the intel2gas tool, which converts between NASM and AT&T
    syntax, or by utilizing NASM's built-in output capabilities to generate
    GAS-compatible objects.
    
    $ intel2gas -i input.asm -o output.s

These generated/converted .S files should be stored alongside the original .asm files in Git.  If NASM is available the originals can be compiled, but in the common case where NASM is not available the .S files would be compiled and linked into the kernel modules.

I think it is worthwhile to keep both, since it would be easier to update the .asm files from upstream ISA-L, but it might be some time before the .S files are update to match.

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Converting the NASM routines and maintaining a second assembly implementation needs a dedicated change with instruction-equivalence and runtime validation across the ISA variants. This exceeds small-fix scope.

reply PS3 2026-10-02 19:13 patrick bot COMMIT_MSG:7 — Patrick-Bot: Changed the commit subject to LU-19905, whose ticket covers importing ISA-L assembly. LU-20016 describes the benchmark tool. No

Andreas Dilger PS3 · 2026-04-06 21:18

This should probably use LU-19905, or change the description of LU-20016?

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Changed the commit subject to LU-19905, whose ticket covers importing ISA-L assembly. LU-20016 describes the benchmark tool. No JIRA fields were changed. These local edits remain unpublished: upload refuses the obsolete ancestry, and rebasing onto 64927 PS8 needs substantive EC integration choices.

reply PS3 2026-10-02 19:13 patrick bot COMMIT_MSG:71 — Patrick-Bot: Removed the duplicated performance table and its associated claims from this commit message. No aarch64 measurements are claime

Andreas Dilger PS3 · 2026-05-13 00:10

This table is the same between the x86 and aarch64 patches.  It would be useful to fix one or the other to have the correct data for that CPU architecture.

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Removed the duplicated performance table and its associated claims from this commit message. No aarch64 measurements are claimed, and change 64938 was not modified. These local edits remain unpublished: upload refuses the obsolete ancestry, and rebasing onto 64927 PS8 needs substantive EC integration choices.

reply PS3 2026-10-02 19:13 patrick bot ec_aarch64_neon.c:17 — Patrick-Bot: The comment now names the verified 5.14 (RHEL9) and 6.12 (RHEL10) NEON APIs and removes the unsupported claim about older kbuil

Andreas Dilger PS3 · 2026-04-06 21:18

(minor) It would be better to specify which kernel version is non-functional, so that this can become conditional upon a newer kernel (assuming there is a benefit to use SVE-optimized versions).

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: The comment now names the verified 5.14 (RHEL9) and 6.12 (RHEL10) NEON APIs and removes the unsupported claim about older kbuild rejecting SVE directives. Enabling kernel SVE remains separate work. These local edits remain unpublished: upload refuses the obsolete ancestry, and rebasing onto 64927 PS8 needs substantive EC integration choices.

reply PS3 2026-10-02 19:13 patrick bot ec_aarch64_neon.c:20 — Patrick-Bot: The comment now names Linux 5.14 (RHEL9) and 6.12 (RHEL10), whose headers provide the no-argument NEON API, and removes unverif

Andreas Dilger PS3 · 2026-04-06 21:18

(minor) Again, please specify kernel versions so that this can be fixed in the future for newer kernel versions.

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: The comment now names Linux 5.14 (RHEL9) and 6.12 (RHEL10), whose headers provide the no-argument NEON API, and removes unverified compatibility claims. These local edits remain unpublished: upload refuses the obsolete ancestry, and rebasing onto 64927 PS8 needs substantive EC integration choices.

reply PS3 2026-10-02 19:13 patrick bot ec_aarch64_neon.c:31 — Patrick-Bot: Changed the availability flag to bool and its assignments to true/false. These local edits remain unpublished: upload refuses t

Andreas Dilger PS3 · 2026-04-06 21:18

(style) `bool`?

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Changed the availability flag to bool and its assignments to true/false. These local edits remain unpublished: upload refuses the obsolete ancestry, and rebasing onto 64927 PS8 needs substantive EC integration choices.

reply PS3 2026-10-02 19:13 patrick bot ec_aarch64_neon.c:34 — Patrick-Bot: The NEON declarations remain local. A shared architecture header needs a signature audit and integration with the userspace and

wc-checkpatch PS3 · 2026-03-27 13:45

(style)  externs should be avoided in .c files

Andreas Dilger PS3 · 2026-04-06 21:18

These warnings are correct and the prototypes should be moved to a .h file that is included here and into the file where these functions are implemented.  Having the function prototype in a local .c file prevents the compiler from checking argument type/count against the implementation, and can result in hard-to-find bugs in the future.

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: The NEON declarations remain local. A shared architecture header needs a signature audit and integration with the userspace and assembly builds; that work remains for the owner.

reply PS3 2026-10-02 19:13 patrick bot ec_aarch64_neon.c:165 — Patrick-Bot: Changed the loop to rows >= 6 and removed case 6. The NEON wrapper matches scalar parity for rows 1..24 using scalar stand-ins

Andreas Dilger PS3 · 2026-04-06 21:18

(style) should this be changed to `rows >= 6` and remove the `case 6:` in the switch, as is done in `ec_encode_data_neon()` above?

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Changed the loop to rows >= 6 and removed case 6. The NEON wrapper matches scalar parity for rows 1..24 using scalar stand-ins under ASan/UBSan; actual ARM runtime validation remains outstanding. These local edits remain unpublished: upload refuses the obsolete ancestry, and rebasing onto 64927 PS8 needs substantive EC integration choices.

reply PS3 2026-10-02 19:13 patrick bot ec_base_aliases.c:1 — Patrick-Bot: Added BSD-3-Clause SPDX tags to 116 files with existing complete BSD-3-Clause notices. The two aarch64_label.h files have no li

Andreas Dilger PS1 · 2026-03-29 08:52

These files should all get SPDX headers with BSD 3-clause license markers

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Added BSD-3-Clause SPDX tags to 116 files with existing complete BSD-3-Clause notices. The two aarch64_label.h files have no license notice; their intended license needs confirmation. Existing GPL/BSD-2-Clause identifiers were preserved. The local additions are unpublished because upload is blocked by obsolete ancestry.

reply PS3 2026-10-02 19:13 patrick bot ec_benchmark.c:9 — Patrick-Bot: Adding deterministic known-answer encode/decode checks and fixed reference hashes changes the benchmark design and needs indepe

Andreas Dilger PS3 · 2026-04-06 21:18

Does it make sense to add a (post benchmark) step to confirm that the generated data is correct?  Going fast doesn't help if the parity is wrong...

Something like:
- use PRNG to generate repeated/consistent "random" data in the data stripes
- generate parity chunks under timer for benchmark
- calculate hash (adler32?) of "random" data chunks
- compare data chunk hashes to hard-coded values to confirm correct data
- calculate hash of generated parity chunks
- compare parity chunk hashes to hard-coded values from C code to verify parity
- reconstruct data chunks under timer for benchmark
- calculate hash of reconstructed data chunks
- compare data chunk hash to original value to confirm correct data reconstruction

Having static hash values compiled into the test module (vs. comparing against just-computed C hash values) helps confirm the assembly and C versions are operating correctly, and do not change over time for whatever reason.  This should also catch new ASM versions as they are added, or selected for new CPU architectures or variants.

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Adding deterministic known-answer encode/decode checks and fixed reference hashes changes the benchmark design and needs independently verified test vectors. This remains for the owner.

reply PS3 2026-10-02 19:13 patrick bot ec_benchmark.c:11 — Patrick-Bot: Moving the benchmark into a separate module requires build, symbol-interface and lifecycle changes. Its long non-preemptible ba

Andreas Dilger PS3 · 2026-04-06 21:18

Another option to avoid having this code resident in the kernel for the 0.0000001% of the time it is needed would be to put it into a dedicated `lustre/kunit/ec_benchmark.c` test module, so that `modprobe ec_benchmark` loads and runs the benchmark and prints the output to the console.  There is already an `ec_test.c`, but it might become more heavyweight in the future and having a dedicated `ec_benchmark.ko` module might be better.

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Moving the benchmark into a separate module requires build, symbol-interface and lifecycle changes. Its long non-preemptible batch also needs redesign. This remains for the owner.

reply PS3 2026-10-02 19:13 patrick bot ec_benchmark.c:122 — Patrick-Bot: Joined the final parameters onto the preceding declaration line. These local edits remain unpublished: upload refuses the obsol

Andreas Dilger PS3 · 2026-04-06 21:18

(style) can fit on previous line

patrick bot PS3 · 2026-10-02 19:13

Patrick-Bot: Joined the final parameters onto the preceding declaration line. These local edits remain unpublished: upload refuses the obsolete ancestry, and rebasing onto 64927 PS8 needs substantive EC integration choices.

owner Patrick Farrell · uploader Patrick Farrell · PS3 uploaded 220d ago · NEW · open in Gerrit ↗

commit message

LU-20016 ec: ISA-L SIMD for userspace and kernel

Integrate Intel ISA-L optimized assembly into Lustre's
erasure coding library for both userspace and kernel,
on x86_64 and aarch64. Add debugfs kernel benchmark
for measuring FPU save/restore overhead.

Userspace x86_64 (libec.a):
- 72 NASM assembly files (SSE/AVX/AVX2/AVX-512/GFNI)
- ec_multibinary.asm: runtime CPUID dispatch
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar when NASM unavailable

Userspace aarch64 (libec.a):
- NEON + SVE .S assembly + SVE C intrinsics
- ec_aarch64_dispatcher.c: getauxval() dispatch
- ec_aarch64_highlevel_func.c: N-vector dispatch

Kernel x86_64 (ec.ko):
- Pre-assemble ISA-L NASM .asm files into .o,
  link into ec.ko (71 assembly objects)
- ec_dispatch.c: boot_cpu_has() selects AVX2/AVX/SSE
  at module init, kernel_fpu_begin/end wrapping
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar in interrupt context

Kernel aarch64 (ec.ko):
- NEON .S assembly files (GAS format, direct kbuild)
- ec_aarch64_neon.c: kernel_neon_begin/end wrapping
  with may_use_simd() check, hwcap detection
- SVE excluded from kernel (toolchain portability)

Build system:
- lustre-erasurecode.m4: detect NASM, aarch64
- erasurecode/autoMakefile.am: three-way dispatch
- utils/Makefile.am: link against libec.a
- ec/Makefile.in: NASM pre-assembly for kernel

Bug fixes in ec_perf_bench.c:
- -p flag was setting k instead of p
- frag_ptrs allocated stripe_size pointers not m
- Inverted exit code and stale rc from getopt
- Use aligned_alloc(64) for data buffers

Kernel benchmark (debugfs):
- /sys/kernel/debug/lustre/ec/benchmark
- 4 phases: FPU overhead, SIMD+FPU per-call,
  SIMD+FPU amortized, C scalar baseline
- Input format (write to debugfs entry):
    echo 1 > .../ec/benchmark        (defaults)
    echo "k=5 p=2 s=128" > .../ec/benchmark
    cat .../ec/benchmark
  Parameters (key=value, space separated):
    k  data stripes     (default 5, max k+p=16)
    p  parity stripes   (default 2)
    s  stripe size in KB (default 128, max 65536)
  "echo 1" runs with all defaults. Unrecognized
  input returns -EINVAL. cat shows results or
  usage instructions if not yet run.

Fix gf_vect_mul_init naming to use _base suffix
consistently (aligns with ISA-L convention) in
both userspace and kernel ec_base.c.

Performance (5+2, 1 thread, QEMU VM, AVX2):

Stripe | Kernel  | Kernel  | Userspace | SIMD     | Kernel vs
Size   | SIMD    | Scalar  | SIMD      | Speedup  | Userspace
       |         |         |           | in Kernel|
-------|---------|---------|-----------|----------|----------
 64 KB | 17099   | 540     | 404345*   | 31.7x    | 0.04x*
128 KB | 16888   | 531     | 197511*   | 31.8x    | 0.09x*
256 KB | 17032   | 539     | 98401*    | 31.6x    | 0.17x*
  1 MB | 17142   | 538     | 24830     | 31.9x    | 0.69x
  4 MB | 10309   | 533     | 5385      | 19.3x    | 1.91x
 16 MB | 10954   | 534     | 1048      | 20.5x    | 10.5x

All throughput in MB/s. (*) Userspace numbers at
small stripes are inflated by cache residency.
At 4-16MB the kernel is faster -- its kvmalloc
pages are physically scattered, matching real
Lustre page cache behavior. Replicating this in
userspace is non-trivial and not worth the effort;
the kernel benchmark is authoritative.

FPU save/restore overhead was measured at 25-54 ns
on this platform (Ryzen 3700X) -- negligible vs
encode time (~39us at 128KB). The benchmark
measures this separately (Phase 1) so it can be
characterized on other hardware where it may be
more significant.

Generated with Claude Code + Tools

Test-Parameters: testlist=sanity-ec
Change-Id: If4c57c328c0e23bb6769dcbc45d52d43b03a7837
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>

66111 master J ✗45B ✓T ✗0R 0/2

DRAFT LU-20244 utils: add support to resync a region

reviewing Ronnie Sahlberg · blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

45 unique failures ec2 pt_ecro M +84 −36

75d

blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

awaiting review (CI not green yet)

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-ec@ldiskfs+DNE:test_1a	seen in 21 other reviews
sanity-ec@ldiskfs+DNE:test_1b	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1c	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1d	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1g	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_4e	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_5a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_5b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_6a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6b	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6d	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6e	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_7	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_23a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_23b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_23c	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_24a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_24b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_25a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_26a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_26b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_27a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_28a	seen in 22 other reviews
sanity-ec@zfs:test_1a	seen in 21 other reviews
sanity-ec@zfs:test_1b	seen in 23 other reviews
sanity-ec@zfs:test_1c	seen in 23 other reviews
sanity-ec@zfs:test_1d	seen in 23 other reviews
sanity-ec@zfs:test_1g	seen in 23 other reviews
sanity-ec@zfs:test_4e	seen in 23 other reviews
sanity-ec@zfs:test_5a	seen in 23 other reviews
sanity-ec@zfs:test_5b	seen in 22 other reviews
sanity-ec@zfs:test_6a	seen in 23 other reviews
sanity-ec@zfs:test_6b	seen in 23 other reviews
sanity-ec@zfs:test_6d	seen in 23 other reviews
sanity-ec@zfs:test_6e	seen in 23 other reviews
sanity-ec@zfs:test_7	seen in 23 other reviews
sanity-ec@zfs:test_23a	seen in 22 other reviews
sanity-ec@zfs:test_23b	seen in 22 other reviews
sanity-ec@zfs:test_24a	seen in 22 other reviews
sanity-ec@zfs:test_24b	seen in 22 other reviews
sanity-ec@zfs:test_25a	seen in 22 other reviews
sanity-ec@zfs:test_26a	seen in 22 other reviews
sanity-ec@zfs:test_26b	seen in 22 other reviews
sanity-ec@zfs:test_27a	seen in 22 other reviews
sanity-ec@zfs:test_28a	seen in 22 other reviews

no vote yet: Andreas Dilger, Zhenyu Xu

patchsets: PS1 05-25 PS2 05-25 PS3 05-25 PS4 08-18

tags: pt_ecro

owner Ronnie Sahlberg · uploader Ronnie Sahlberg · PS4 uploaded 75d ago · NEW · open in Gerrit ↗

commit message

DRAFT LU-20244 utils: add support to resync a region

Add support to resync EC for a region of a file instead of the entire
file.

Test-Parameters: testlist=sanity-ec
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I49a0ab53e7948c40086b532a0e6f9298d1bc8e1e

66112 master J ✗45B ✓T ✗0R 0/2

DRAFT LU-20244 utils: resync/verify stale mirrors in 1GB chunks.

reviewing Ronnie Sahlberg · blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

45 unique failures ec2 pt_ecro 3 unresolved S +30 −4

75d

blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

awaiting review (CI not green yet)

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-ec@ldiskfs+DNE:test_1a	seen in 21 other reviews
sanity-ec@ldiskfs+DNE:test_1b	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1c	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1d	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1g	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_4e	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_5a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_5b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_6a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6b	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6d	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6e	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_7	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_23a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_23b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_23c	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_24a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_24b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_25a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_26a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_26b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_27a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_28a	seen in 22 other reviews
sanity-ec@zfs:test_1a	seen in 21 other reviews
sanity-ec@zfs:test_1b	seen in 23 other reviews
sanity-ec@zfs:test_1c	seen in 23 other reviews
sanity-ec@zfs:test_1d	seen in 23 other reviews
sanity-ec@zfs:test_1g	seen in 23 other reviews
sanity-ec@zfs:test_4e	seen in 23 other reviews
sanity-ec@zfs:test_5a	seen in 23 other reviews
sanity-ec@zfs:test_5b	seen in 22 other reviews
sanity-ec@zfs:test_6a	seen in 23 other reviews
sanity-ec@zfs:test_6b	seen in 23 other reviews
sanity-ec@zfs:test_6d	seen in 23 other reviews
sanity-ec@zfs:test_6e	seen in 23 other reviews
sanity-ec@zfs:test_7	seen in 23 other reviews
sanity-ec@zfs:test_23a	seen in 22 other reviews
sanity-ec@zfs:test_23b	seen in 22 other reviews
sanity-ec@zfs:test_24a	seen in 22 other reviews
sanity-ec@zfs:test_24b	seen in 22 other reviews
sanity-ec@zfs:test_25a	seen in 22 other reviews
sanity-ec@zfs:test_26a	seen in 22 other reviews
sanity-ec@zfs:test_26b	seen in 22 other reviews
sanity-ec@zfs:test_27a	seen in 22 other reviews
sanity-ec@zfs:test_28a	seen in 22 other reviews

no vote yet: Andreas Dilger, Zhenyu Xu

patchsets: PS1 05-25 PS2 05-25 PS3 05-25 PS4 08-18

threads: 3 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-05-26 08:20 Andreas Dilger COMMIT_MSG:14 — I don't see this patch doing these things, mainly changing the buffer size?

Andreas Dilger PS3 · 2026-05-26 08:20

I don't see this patch doing these things, mainly changing the buffer size?

reply PS3 2026-05-26 08:20 Andreas Dilger lfs.c:14655 — How was this 1GiB buffer size selected?

Andreas Dilger PS3 · 2026-05-26 08:20

How was this 1GiB buffer size selected?

reply PS3 2026-05-26 08:20 Andreas Dilger lfs.c:14663 — The file was stat'dback on line 14591, so that could be used here instead of a new stat() call? If there is a reason the stat() info has go

Andreas Dilger PS3 · 2026-05-26 08:20

The file was stat'dback on line 14591, so that could be used here instead of a new stat() call?  If there is a reason the stat() info has gone stale since the start of this function, then that is probably a sign the file shouldn't be resync'd at this time.

tags: pt_ecro

owner Ronnie Sahlberg · uploader Ronnie Sahlberg · PS4 uploaded 75d ago · NEW · open in Gerrit ↗

commit message

DRAFT LU-20244 utils: resync/verify stale mirrors in 1GB chunks.

Instead of resyncing each stale mirror, one after the other, resync/verify
the mirrors one chunk (1GB region) of the file at a time.
This is so that if we are resyncing/verifying multiple stale mirrors at a time,
that we read data from the up-to-date mirror once, for generating/writing to the first
stale mirror and then re-read the original data from page-cache when generating/writing
to the other stale mirrors.

Otherwise if we resync/verify the whole mirrors one at a time, we risk for very large
files that the read data from the good mirror will fall out of cache and need te be
re-read for each stale submirror.

Test-Parameters: testlist=sanity-ec
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I279cdedf9b07c8f091747fcaa5b103852ee166c7

66110 master J ✗45B ✓T ✗0R 0/2

DRAFT LU-20244 tests: add tests for resync of FLR + EC

reviewing Ronnie Sahlberg · blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

45 unique failures ec2 pt_ecro M +148 −0

75d

blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

awaiting review (CI not green yet)

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-ec@ldiskfs+DNE:test_1a	seen in 21 other reviews
sanity-ec@ldiskfs+DNE:test_1b	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1c	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1d	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_1g	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_4e	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_5a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_5b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_6a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6b	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6d	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_6e	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_7	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_23a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_23b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_23c	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_24a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_24b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_25a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_26a	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_26b	seen in 22 other reviews
sanity-ec@ldiskfs+DNE:test_27a	seen in 23 other reviews
sanity-ec@ldiskfs+DNE:test_28a	seen in 22 other reviews
sanity-ec@zfs:test_1a	seen in 21 other reviews
sanity-ec@zfs:test_1b	seen in 23 other reviews
sanity-ec@zfs:test_1c	seen in 23 other reviews
sanity-ec@zfs:test_1d	seen in 23 other reviews
sanity-ec@zfs:test_1g	seen in 23 other reviews
sanity-ec@zfs:test_4e	seen in 23 other reviews
sanity-ec@zfs:test_5a	seen in 23 other reviews
sanity-ec@zfs:test_5b	seen in 22 other reviews
sanity-ec@zfs:test_6a	seen in 23 other reviews
sanity-ec@zfs:test_6b	seen in 23 other reviews
sanity-ec@zfs:test_6d	seen in 23 other reviews
sanity-ec@zfs:test_6e	seen in 23 other reviews
sanity-ec@zfs:test_7	seen in 23 other reviews
sanity-ec@zfs:test_23a	seen in 22 other reviews
sanity-ec@zfs:test_23b	seen in 22 other reviews
sanity-ec@zfs:test_24a	seen in 22 other reviews
sanity-ec@zfs:test_24b	seen in 22 other reviews
sanity-ec@zfs:test_25a	seen in 22 other reviews
sanity-ec@zfs:test_26a	seen in 22 other reviews
sanity-ec@zfs:test_26b	seen in 22 other reviews
sanity-ec@zfs:test_27a	seen in 22 other reviews
sanity-ec@zfs:test_28a	seen in 22 other reviews

no vote yet: Andreas Dilger, Zhenyu Xu

patchsets: PS1 05-25 PS2 05-25 PS3 05-25 PS4 08-18

tags: pt_ecro

owner Ronnie Sahlberg · uploader Ronnie Sahlberg · PS4 uploaded 75d ago · NEW · open in Gerrit ↗

commit message

DRAFT LU-20244 tests: add tests for resync of FLR + EC

We can have files that use FLR mirroring but at the same time also use
EC to protect the FLR mirrors.
It is probably not common to use as a persistent end user configuration
as Mirrorins and EC usually are meant for different purposes :
with Mirroring being prefered when performance outweights storage cost
and EC is preferred when storage costs outweigh performance.

But there will be situations where FLR+EC will be in use during a
transitionary period, for example when migrating an EC protected file
between different storage hierarchies.

As such we need to have a test for this situation and that we can
support and resync buth FLR and EC components at the same time.

Test-Parameters: testlist=sanity-ec
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: Idcac78327b501d484653dec6be0882b5dbb1fa6f

▶

Voted on current patchset

(12)

62158 master J –B ✓T ✓R +2/2

LU-12597 tests: remove comma_list for all_nodes

reviewing Andreas Dilger · you voted +1 on current PS

testfix 1 unresolved M +65 −70

build #135056 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-11-02 11:13 · Arshad Hussain +1 2026-11-02 08:28

no vote yet: patrick bot, Sebastien Buisson, Chakshu Kansal

patchsets:

+2 earlier

PS1 10-31 PS2 09-17 patrick bot

PS3 09-18 Patrick Farrell PS4 09-18 patrick bot PS5 09-23 patrick bot PS6 10-05 PS7 11-01 patrick bot

your previous vote: +1 on PS5 (2026-09-27 21:24) — now at PS7

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2026-09-18 15:31 patrick bot sanity-sec.sh:12755 — Patrick-Bot: Picked up in patchset 3 for the two files this patch already touches -- the remaining 16 comma_list $(all_*_nodes) sites in san

Gerrit AI review for Lustre PS2 · 2026-09-18 14:32

(minor) cleanup_103() sits right next to the cleanup_102() that was converted and still uses the same `comma_list $(all_server_nodes)` / `comma_list $(all_nodes)` pattern here and at lines 12763 and 12767.

The same is true elsewhere in this file (1294, 1792, 3224, 11641, 12814, 13111, 13151) and in test-framework.sh itself, where setup_load_ssk() and cleanup_unload_ssk() at lines 1728-1756 were missed. sanity.sh, sanity-compr.sh and replay-dual.sh have a few too. Should these be picked up in the same pass so the pattern doesn't get copied forward?

(The gather_logs() calls in recovery-scale-lib.sh and recovery-double-scale.sh do need comma_list, since they merge two lists.)

patrick bot PS3 · 2026-09-18 15:31

Patrick-Bot: Picked up in patchset 3 for the two files this patch already touches -- the remaining 16 comma_list $(all_*_nodes) sites in sanity-sec.sh (including cleanup_103) and test-framework.sh (generate_load_ssk, cleanup_unload_ssk). Each helper ends in comma_list, so the wrapper is idempotent; checked on a synthetic 4-MDS/8-OST/3-client config where comma_list $(helper) compares equal to $(helper) for all four helpers.

Left alone: the two gather_logs() sites, which do merge two lists, as you note. Also left alone are sanity.sh, sanity-compr.sh and replay-dual.sh -- those are mdts_nodes/osts_nodes rather than all_*, and pulling three new files into someone else's patch is the author's call rather than mine, so leaving this thread open for Andreas.

tags: testfix

owner Andreas Dilger · uploader patrick bot · PS7 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-12597 tests: remove comma_list for all_nodes

Remove comma_list for all_nodes, all_mdts_nodes, all_osts_nodes,
all_server_nodes, etc. usage in test-framework.sh, sanity-sec.sh
and obdfilter-survey.sh. These helpers already end in comma_list()
after the earlier patches in this series, so the outer call is a
no-op.

In load_modules() and unload_modules(), rename the local $list to
$nodes to match. Hoist the duplicated empty-list check in
unload_modules() so the lines this rename touches fit in 80
columns.

Assisted-by: ClaudeCode:claude-opus-5
Assisted-by: ClaudeCode:claude-sonnet-5
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Change-Id: I00109872350987564ca560ad93fdaba0443ebbe5
Signed-off-by: Patrickbot <patrick-bot@mulberrytree.us>

70191 master J –B ✓T ✓R +2/2

LU-20869 tests: speed up sanity-sec 27a and 27ab

reviewing patrick bot · you voted +1 on current PS

pw_own master-next M +34 −18

build #134883 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-10-30 16:11 · Andreas Dilger +1 2026-10-30 19:05

no vote yet: Patrick Farrell

patchsets: PS1 10-29 PS2 10-30 PS3 10-30

your previous vote: -1 on PS2 (2026-10-30 15:03) — now at PS3

tags: pw_own

owner patrick bot · uploader patrick bot · PS3 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-20869 tests: speed up sanity-sec 27a and 27ab

These tests restart every server to check that nodemap data
survives a restart: 27a twice and 27ab twice, each a full module
reload and remount of every target. That is about 160 seconds on
a 4 VM DNE CI setup, and the tests average 320 to 420 seconds
in CI.

What 27ab checks lives on the MGS, which re-reads the nodemap
index from disk when it restarts, and 27a mounts a client
against the fileset the MGS loaded. Restart just the MGS and
MDT0000, with the clients unmounted, and wait for the other
MDTs to reconnect to MDT0000. 27a keeps the full restart
with module reload when the MGS predates persistent filesets,
since an llog fileset is only replayed from the params llog once
the in-memory nodemap is freed; 27ac keeps it for the same
reason, and 27c keeps covering a full restart.

Fix 27ac's fileset sync checks, which used undefined variable
names and so checked nothing.

On a 2 MDT, 2 OST, 1 client setup 27a went from 289 s to 93
to 106 s and 27ab from 166 s to 54 to 63 s: about 300 s per run
of the two tests, roughly 700 hours per 30 days at 8300 runs of
each test.

Fixes: 40cd140b8597 ("LU-18357 ptlrpc: Support add and del alternate filesets")
Test-Parameters: trivial testlist=sanity-sec env=ONLY="27a 27ab 27ac"
Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Patrick Farrell <patrick@thelustrecollective.com>
Change-Id: I97e69b2d2211d854bf1b1776246cd665cde51d4e

69812 master J –B ✓T ✓R +2/2

LU-17898 test: sync quota setting

reviewing Hongchao Zhang · you voted +1 on current PS

master-next 4 unresolved XS +3 −1

build #134005 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-10-20 13:45 · Timothy Day +1 2026-10-28 20:10

no vote yet: Sergey Cheremencev

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-20 09:34 Gerrit AI review for Lustre COMMIT_MSG:7 — (style) This isn't a bug, but test-only changes normally use the "tests:" component tag, e.g. "LU-17898 tests: wait for quota sync in sanity

Gerrit AI review for Lustre PS1 · 2026-10-20 09:34

(style) This isn't a bug, but test-only changes normally use the "tests:" component tag, e.g. "LU-17898 tests: wait for quota sync in sanity-quota/19".

reply PS1 2026-10-20 09:34 Gerrit AI review for Lustre COMMIT_MSG:9 — (style) If the patch is refreshed, could the body be reworded so it's clearer? As written, "could not be synced ... during the following quo

Gerrit AI review for Lustre PS1 · 2026-10-20 09:34

(style) If the patch is refreshed, could the body be reworded so it's clearer? As written, "could not be synced ... during the following quota test, then it should wait" is hard to follow.

It would also help to name `wait_quota_synced()` and say why `-i 0` was added to `lfs setstripe`. The wait only checks OST0000, so the file has to be on that OST for the wait to mean anything.

reply PS1 2026-10-20 09:34 Gerrit AI review for Lustre sanity-quota.sh:3465 — (minor) If the patch is refreshed, consider deriving the value from `blimit` like the other callers do, e.g. `$((blimit * 1024))`, so it can

Gerrit AI review for Lustre PS1 · 2026-10-20 09:34

(minor) If the patch is refreshed, consider deriving the value from `blimit` like the other callers do, e.g. `$((blimit * 1024))`, so it can't drift if `blimit` changes.

reply PS1 2026-10-20 09:34 Gerrit AI review for Lustre sanity-quota.sh:3465 — (suggestion) This test checks that updating admin limits doesn't zero the operational limits. If the value on the QSD doesn't match, `wait_q

Gerrit AI review for Lustre PS1 · 2026-10-20 09:34

(suggestion) This test checks that updating admin limits doesn't zero the operational limits. If the value on the QSD doesn't match, `wait_quota_synced()` writes `quota_slave.force_reint=1`, and that re-fetches the global index from the QMT.

So if the "Update quota limits" step ever wrongly cleared the slave's copy, the forced reint would restore it and the test would pass. Would it be better to wait after the first `lfs setquota` instead? The second setquota uses the same limits, so `qmt_set_with_lqe()` doesn't mark the entry dirty or send a glimpse. A later write would then still catch a regression.

owner Hongchao Zhang · uploader Hongchao Zhang · PS1 uploaded 14d ago · NEW · open in Gerrit ↗

commit message

LU-17898 test: sync quota setting

In test_19 of sanity-quota, the quota setting could not be synced
between QMT and QSD during the following quota test, then it should
wait for the quota setting on OSTs to be synced.

Test-Parameters: trivial
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: I0a075f8190e636219a50939f7d14a292d8fb218e

69458 master J –B ✓T ✓R +2/2

LU-19157 nodemap: do not apply ban list on the MGS

reviewing Sebastien Buisson · you voted +1 on current PS

master-next M +60 −11

7 optional test failure(s) — informational

build #133015 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-10-09 14:42 · Patrick Farrell +1 2026-10-30 18:27

no vote yet: Andreas Dilger

patchsets: PS1 10-06 PS2 10-07

your previous vote: +1 on PS1 (2026-10-07 10:06) — now at PS2

owner Sebastien Buisson · uploader Sebastien Buisson · PS2 uploaded 26d ago · NEW · open in Gerrit ↗

commit message

LU-19157 nodemap: do not apply ban list on the MGS

The ban list is checked in tgt_request_handle(), which lets connect,
disconnect, ping and close through, but only for MDS and OST. On the
MGS, the MGS_CONNECT of a banned client gets -EPERM, after the export
has been created. The client treats this as a transient connect
failure, so the MGC import never reaches FULL and keeps reconnecting.

Fix this behavior by excepting the ban for requests needed by the MGC
to fetch the configuration. A banned client now gets its
configuration, connects to the MDT, and its first MDT request fails
with -EPERM, so the mount fails immediately with "Operation not
permitted".
Adding only MGS_CONNECT/MGS_DISCONNECT to the exempt list is not
enough. The MGS lock request would then fail with -EPERM, but the MGC
still treats a failed lock request as "MGS unreachable" and keeps on
retrying.

Improve sanity-sec test_81a to check that a mount from a banned
client fails within 30 seconds and with EPERM.

Fixes: f6f6009aa9 ("LU-20333 mgs: Associate nodemaps with exports on connect/reconnect")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Assisted-by: Claude:claude-opus-5-5
Change-Id: Icd5af8da27d57d96bcb081697dc0fa3403e4a20c

69357 master J !B ✓T ✓R +2/2

LU-20763 nodemap: split IAM batches into small transactions

reviewing Sebastien Buisson · you voted +1 on current PS

master-next M +144 −12

janitor run: 4 failing config(s), none unique to this patch — janitor results

build #132717 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-10-05 10:09 · Andreas Dilger +1 2026-10-29 01:14

patchsets: PS1 10-01 PS2 10-02

owner Sebastien Buisson · uploader Sebastien Buisson · PS2 uploaded 31d ago · NEW · open in Gerrit ↗

commit message

LU-20763 nodemap: split IAM batches into small transactions

nodemap_idx_insert_batch() and nodemap_idx_delete_batch() update any
number of nodemap IAM records in a single transaction. On ldiskfs each
IAM insert is declared with 16 credits and each delete with 4, so a
batch easily declares more credits than the maximum transaction size
of a small MGS journal:
- every fileset removal wipes the whole subid range reserved for the
  fileset, i.e. LUSTRE_NODEMAP_FILESET_SUBID_RANGE (256) deletes and
  1024 credits;
- a PATH_MAX long fileset is stored as 147 fragments, i.e. 2352
  credits to add it and 588 to delete it.

osd_trans_start() then clamps the transaction and osd_trans_exec_op()
complains:

  lustre-MDT0000: opcode 9: before 516 < left 1024, rollback = 8

To fix this, split batches into transactions of at most 16 inserts or
64 deletes, about 256 credits each. Rest of record processing stays
as is, so the undo logic of the callers is unchanged.

Add sanity-sec test_27g to check that fileset updates on the MGS do
not exceed the journal transaction size.

Fixes: da3536da22ab ("LU-18357 ptlrpc: Use nodemap IAM for persisting filesets")
Test-Parameters: trivial testlist=sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Assisted-by: Claude:claude-opus-5-5
Change-Id: Ib7303ab52116e8fae1e8e7e351ba492ca3c67268

70145 master J –B ✓T ✓R +1/2

LU-20896 llite: wait for OST for O_CIPHERTEXT EC reads

reviewing Andreas Dilger · you voted +1 on current PS

pt_ecro 2 unresolved M +89 −11

failed enforced test	platform	detail
review-ldiskfs	RHEL 8.10 / x86_64, RHEL 10.1 / x86_64	ran 5 tests. 1 tests failed: sanity-lnet.	session

build #134771 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-10-30 20:25

no vote yet: Sebastien Buisson

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 08:42 Gerrit AI review for Lustre COMMIT_MSG:14 — (minor) Is the i_size part accurate for the no-key case? Without the key, ll_merge_attr() already rounds i_size up to LUSTRE_ENCRYPTION_UNIT

Gerrit AI review for Lustre PS1 · 2026-10-31 08:42

(minor) Is the i_size part accurate for the no-key case? Without the key, ll_merge_attr() already rounds i_size up to LUSTRE_ENCRYPTION_UNIT_SIZE (the clear text size goes to lli_lazysize), so a buffered CIT_EC_RD read should already reach the end of the last encryption unit. Stopping at the clear text i_size looks like a with-key problem, and with the key that recovery path was already refused. If so, the sentence could just drop that clause.

reply PS1 2026-10-31 08:42 Gerrit AI review for Lustre file.c:2294 — (suggestion) This isn't a bug, but this rewind (iov_iter_revert(), iov_iter_reexpand(), ki_pos -= ci_bytes, ci_bytes = 0) now appears word f

Gerrit AI review for Lustre PS1 · 2026-10-31 08:42

(suggestion) This isn't a bug, but this rewind (iov_iter_revert(), iov_iter_reexpand(), ki_pos -= ci_bytes, ci_bytes = 0) now appears word for word in both branches. If the patch is refreshed, a small static helper such as ll_dio_rewind(args, io, bytes) would keep the two restart paths from drifting apart.

tags: pt_ecro

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-20896 llite: wait for OST for O_CIPHERTEXT EC reads

An O_CIPHERTEXT direct read of an EC file with an unavailable data
stripe would fail with -EIO or -EAGAIN with the key loaded, since the
buffered CIT_EC_RD recovery read decrypts the data.  Without the key,
that recovery read would leave the ciphertext uptodate in the page
cache, where a reader that later adds the key would see it as clear
text, and it would stop at i_size rather than at the end of the last
encryption unit.

Read the whole DIO request again without the no-delay flag instead,
which waits for the OST to return, as for a file without parity,
until the ciphertext can be rebuilt from parity directly.  A read
from a deactivated OST still fails with -EIO.  Also skip the EC DIO
read retry after a signal.

Add sanity-ec test_69e to check that such a read waits for a stopped
OST and then returns the ciphertext.

Fixes: bd9ebb7e51a6 ("LU-20896 llite: transfer ciphertext for O_CIPHERTEXT DIO")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: Icdd28a844cc6f41ce3c86be8f9b1ec2a0d3ebbe5

70078 master J –B ✓T ✓R +1/2

LU-21051 ssk: consolidate SSK key modes

reviewing Sebastien Buisson · you voted +1 on current PS

CI@PS2 S +18 −18

build #134620 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-10-29 12:07

no vote yet: Andreas Dilger

patchsets: PS1 10-27 PS2 10-28 PS3 10-29

your previous vote: +1 on PS2 (2026-10-28 17:55) — now at PS3

build/test results are from PS2 — PS3…PS3 changed only the commit message (votes copied, CI not re-run)

owner Sebastien Buisson · uploader Sebastien Buisson · PS3 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-21051 ssk: consolidate SSK key modes

Instead of duplicating the SSK key modes across multiple tests,
declare a common variable to hold the mode.

Test-Parameters: trivial testlist=sanity
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Assisted-by: Claude:claude-opus-5-5
Change-Id: I38e34b5a410cd20dbfd4668d94dbf6638e26a715

66527 master J –B ✓T ✓R +1/2

LU-19079 krb: use .lgss to specify principal

reviewing Sebastien Buisson · you voted +1 on current PS

L +210 −42

build #133488 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-10-13 15:00

no vote yet: Aurelien Degremont, Bruno Faccini, Andreas Dilger, Li Xi

patchsets:

+20 earlier

PS1 06-08 PS2 06-08 PS3 06-08 PS4 06-09 PS5 06-09 PS6 06-15 PS7 06-23 PS8 07-09 PS9 07-09 PS10 08-17 PS11 08-18 PS12 08-19 PS13 08-20 PS14 08-21 PS15 08-24 PS16 08-25 PS17 08-26 PS18 08-28 PS19 08-28 PS20 08-31

PS21 09-01 PS22 09-02 PS23 09-03 PS24 09-04 PS25 10-13

your previous vote: +1 on PS24 (2026-10-02 12:46) — now at PS25

owner Sebastien Buisson · uploader Sebastien Buisson · PS25 uploaded 20d ago · NEW · open in Gerrit ↗

commit message

LU-19079 krb: use .lgss to specify principal

Allow the .lgss file inside the client mount point to contain a
reference to the Kerberos principal to use, with the syntax:
krb5princ=<full Service Principal Name>

This is an alternative to the 'krb5princ' mount option, designed for
easier automated implementation.

Test-Parameters: trivial testlist=sanity
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1205f65faed3f60057e301b883ea646c88efcf88
Was-Change-Id: I776908e0ecad51ec3b3b79e5b392f95815d2e569
Was-Change-Id: I82cea28ab2fa2c86671271658e10f744a0f05841
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Marc Vef <mvef@whamcloud.com>

66564 b2_15 J ✓B ✓T ✓R +4/1

LU-19784 tests: mmp/test_8 wait time changes

reviewing Li Dongyang · you voted +1 on current PS

M +47 −48

68d

build #126492 SUCCESS · tests all Maloo sessions

reviews: Marc Vef +1 2026-08-26 08:10 · Andreas Dilger +1 2026-06-12 20:43 · Artem Blagodarenko +1 2026-06-12 21:04 · Alex Zhuravlev +1 2026-06-13 07:56

patchsets: PS1 06-10 PS2 06-12 Andreas Dilger PS3 06-12 Andreas Dilger PS4 06-12 Andreas Dilger

your previous vote: -1 on PS1 (2026-06-12 08:39) — now at PS4

owner Li Dongyang · uploader Andreas Dilger · PS4 uploaded 142d ago · NEW · backport · open in Gerrit ↗

commit message

LU-19784 tests: mmp/test_8 wait time changes

To prepare for the e2fsprogs/libext2fs changes
reducing the wait time when the MMP sequence
number is EXT4_MMP_SEQ_CLEAN,

LU-19784 ext2fs: mmp wait time ajustments
https://review.whamcloud.com/63478

We need to make sure the sequence number is not
EXT4_MMP_SEQ_CLEAN before running e2fsck.

We can use debugfs to change the MMP sequence number,
rename mark_mmp_block() to set_mmp_seq() to reflect
this.

Remove mmp_mark.sh, which is not needed any more,
and use set_mmp_seq() in test_9.

Make sure we tear down the dm flakey device in test_9
otherwise during mmp_fini()->disable_mmp() will corrupt
the filesystem.

Lustre-change: https://review.whamcloud.com/63488
Lustre-commit: ad051b359db12ef7e752abcf2aa8e34199ded041

Test-Parameters: trivial testlist=mmp
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I81b9d28061862a99b04fa92de0cc18a536c9d17b
Reviewed-by: Artem Blagodarenko <ablagodarenko@thelustrecollective.com>
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>

59801 master J !B ✗T –R +1/2−

LU-19109 llite: remove extra vvp_env_new_io call

reviewing Patrick Farrell · blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; build aborted

needs rebase stalled 147d 1 unresolved S +9 −5

147d

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; build aborted
you voted +1 on current PS
janitor run: 2 failing config(s), none unique to this patch — janitor results
build #114406 ABORTED
      
reviews:
        Marc Vef +1 2025-06-17 16:32 · Timothy Day +1 2025-06-17 16:37
      
no vote yet: Andreas Dilger
patchsets:
        
          PS1 06-17
        
          PS2 06-17
        
          PS3 06-17
        
          PS4 06-17
        
your previous vote: -1 on PS3 (2025-06-17 16:24) — now at PS4
threads: 0 need your reply · 1 waiting on others · 0 own notes · 0 bot
owner Patrick Farrell · uploader Patrick Farrell · PS4 uploaded 502d ago · NEW · open in Gerrit ↗

commit messageLU-19109 llite: remove extra vvp_env_new_io call

vvp_env_new_io is called twice in ll_file_io_generic, which
is confusing since the second call overwrites the first.

Fix this and a cleanup path mistake, where we assume the
IO was set up after the call to vvp_env_new_io, which is
wrong.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I6186e17db8b04f01fd37ea7ac5d4b69b30d0258c

63197 master J !B ✗T –R -3/2−

LU-19744 doc: bulk man page review

reviewing Patrick Farrell · blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; build failure

needs rebase stalled 240d L +367 −151

240d

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; build failure

you voted -1 on current PS

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #120071 FAILURE

reviews: Marc Vef -1 2026-03-04 14:34 · Andreas Dilger -1 2026-03-07 00:48

patchsets: PS1 12-29 PS2 12-29

owner Patrick Farrell · uploader Patrick Farrell · PS2 uploaded 307d ago · NEW · open in Gerrit ↗

commit message

LU-19744 doc: bulk man page review

Bulk review done by Augment and Claude Code.

Complete systematic review of Lustre man pages (sections 5, 7, 8)
with fixes for documentation errors, missing options, formatting
issues, and cross-references.

Key fixes include:
- Add missing options: --nomgs, --nidsfile, -q/--quiet, --catalog,
  -l, and others documented in usage but missing from man pages
- Fix formatting errors: .TH syntax, incomplete sentences, SEE ALSO
  sections
- Add missing cross-references to related man pages
- Fix typos and grammar errors
- Improve consistency across man pages
- Add missing NAME section to lnetctl.8
- Fix alphabetical ordering in SEE ALSO sections (18 nodemap files)

Files modified: 60+ man pages across sections 5, 7, and 8
Total changes: systematic improvements to documentation accuracy
and completeness

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I4896b64767e58c2b939b5a70db394a6f920949d5

58621 master J ✗1B ✓T ✗2R +3/2

LU-13814 llite: remove unnecessary smp_mb()

reviewing Patrick Farrell · blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

1 unique failure stalled 454d XS +0 −5

454d

blocked on the owner: janitor: test failures unique to this patch; enforced tests failed

you voted +1 on current PS

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
replay-single@ldiskfs+DNE:test_65a	seen in 66 other reviews

failed enforced test	platform	detail
review-dne-part-6 crashed	RHEL 9.4 / x86_64	ran 6 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %%	session
review-ldiskfs-ubuntu	RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64	ran 6 tests. 1 tests failed: sanity-sec.	session

2 optional test failure(s) — informational

build #112134 SUCCESS · tests all Maloo sessions

reviews: Qian Yingjin +1 2025-04-01 01:45 · Marc Vef +1 2025-04-01 09:10 · Andreas Dilger +1 2025-03-31 20:40

patchsets: PS1 03-31 PS2 03-31

owner Patrick Farrell · uploader Patrick Farrell · PS2 uploaded 580d ago · NEW · open in Gerrit ↗

commit message

LU-13814 llite: remove unnecessary smp_mb()

This smp_mb() was added as part of:
https://review.whamcloud.com/c/39542

To help protect the csi_sync_nr atomic.

But it was not justified there, and is not in fact needed.
IO submission has *several* memory barriers before another
thread can see the IO.  (Note there is a discussion on that
Gerrit suggesting it was unneeded.)

For example:
osc_queue_sync_pages uses a spinlock to put the extent on
the list(s) (after this point, it's available for ptlrpc to
make an RPC).  There are several other instances of
spinlocks, AND the use of csi_sync_nr is under memory
barriers (in atomic_dec_and_lock, or under a spinlock).

Let's remove this.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I8b81f8e02025cae801a980d2856993c6d4023716

CC'd — on your radar, review not requested

(16)

69422 master J –B ✓T ✓R 0/2

LU-8933 lod: leave inactive targets out of the RR array

cc Andreas Dilger ·

pt_ecro218_review_priority 2 unresolved L +675 −128

15h

failed enforced test	platform	detail
review-dne-part-1	RHEL 9.8 / x86_64	ran 3 tests. 1 tests failed: sanity.	session

build #135075 SUCCESS · tests all Maloo sessions

no vote yet: patrick bot, Keguang Xu, Ronnie Sahlberg

patchsets:

+4 earlier

PS1 10-05 PS2 10-10 PS3 10-13 PS4 10-31 patrick bot

PS5 10-31 PS6 10-31 patrick bot PS7 11-01 patrick bot PS8 11-01 patrick bot PS9 11-01

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS9 2026-11-01 20:19 Gerrit AI review for Lustre COMMIT_MSG:22 — (minor) Is "full" the right example here? A full MDT was already skipped before this patch: lod_mdt_alloc_rr() and lod_mdt_alloc_qos() call

Gerrit AI review for Lustre PS9 · 2026-11-01 20:19

(minor) Is "full" the right example here? A full MDT was already skipped before this patch: lod_mdt_alloc_rr() and lod_mdt_alloc_qos() call lod_statfs_check() on the same cached ltd_statfs that made lod_statfs_and_check() mark it inactive, and that rejects OS_STATFS_ENOSPC/ENOINO for MDTs. The case that is really new seems to be "deactivated by config" (or a statfs error other than -ENOTCONN). This can be fixed whenever the patch is next refreshed.

reply PS9 2026-11-01 20:19 Gerrit AI review for Lustre sanity.sh:7262 — (suggestion) Would this subtest pass without the patch too? "lctl --device %osp deactivate" goes ptlrpc_set_import_active(0) -> IMP_EVENT_IN

Gerrit AI review for Lustre PS9 · 2026-11-01 20:19

(suggestion) Would this subtest pass without the patch too? "lctl --device %osp deactivate" goes ptlrpc_set_import_active(0) -> IMP_EVENT_INACTIVE -> opd_imp_active = 0. After that, osp_statfs() returns -ENOTCONN, and lod_statfs_and_check() sets ltd_discon = 1 along with ltd_active = 0.

Before this patch, lod_mdt_alloc_rr() and lod_mdt_alloc_qos() already skipped ltd_discon MDTs. With MDSCOUNT=3, the old RR loop also gives MDSCOUNT - 1 stripes: the deactivated MDT is skipped, and the use_degraded pass finds the other remote MDT already used. So the stripe count and "not on MDT0001" checks seem to hold on the old code as well.

The new `!mdt->ltd_active` checks only matter for an MDT that is inactive but not ltd_discon, e.g. the lod_process_config() "osp.*.active=0" path before the next statfs, or a statfs error other than -ENOTCONN. If the patch is refreshed, could the test reach one of those, or check its result against an unpatched MDS, so that it fails without the fix?

tags: pt_ecro 218_review_priority

owner Andreas Dilger · uploader Andreas Dilger · PS9 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-8933 lod: leave inactive targets out of the RR array

Inactive OSTs stayed in the round-robin array, so every layout that
reached one skipped it and used up an index, and the OST after it in
the array got its share of first stripes as well as its own.

Leave inactive targets out of the array built by lod_qos_calc_rr(),
unless all of them are inactive, and rebuild the arrays when a target
becomes active or inactive, counted in lq_rr_gen along with failure
domain changes.  The RR allocators index the array by its own size
rather than the pool's.  The periodic statfs update still checks the
inactive targets, and notices when they become active again.  OSTs
that are full, read-only, or have max_create_count=0 are inactive for
this too, so they also leave the array until they can be used again.
This also stops directory stripes being allocated on an inactive MDT
that is not disconnected, e.g. one that is full or deactivated by
config, which lod_mdt_alloc_rr() and lod_mdt_alloc_qos() did not check
for.  Keep returning -EINPROGRESS when no target could be used and a
target of the pool is disconnected, or one left out of the array
became active since it was built, and don't let the smaller array
raise the rounds of an overstriped directory above
lod_max_stripes_per_mdt.  Overstriped OST layouts from the RR
allocator are likewise spread over the active OSTs only, e.g. "lfs
setstripe -C 8" with 1 of 8 OSTs inactive gets 8 stripes with 2 on one
OST, instead of 7 stripes.

Rebuilding the arrays after a target becomes active or inactive only
tries to take lq_rw_sem, and the allocators keep using the old array,
so creates don't wait behind allocators sleeping in precreate
(LU-15393).  Failure domain changes are now rebuilt the same way.
Only if the array stays stale for 10 * qos_maxage, or at once and then
every qos_maxage after an omitted target becomes active, does one
thread wait at most 2s for it, as QoS allocation does.  Move that
timed wait to lu_down_write_timeout() for both, and the backoff to
lu_rr_should_wait() and lu_rr_wait_failed(), which lod_rr_test checks.
Count the targets that become active in lq_active_gen for this.
Also avoid a static checker warning about lrfs_count - 1 underflowing
in lu_rr_fdoms_init().

Add sanity test_51i to check the first stripe balance and overstriping
with OST0001 deactivated, and test_51k to check that RR and QoS
directory stripes skip a deactivated MDT.  test_51i waits until every
OST has enough precreated objects, using up the short FID sequence
tail of any OST that has too few.  With 8 OSTs and 800 2-stripe files
test_51i gives these first stripes per active OST:

  before  50-200
  after   113-115

Check up to 32 OSTs in sanity test_51g, or 64 with SLOW=yes, and
check the OSSs used by other mirrors once per layout in lod_rr_test
rather than once per OST, so that test_51g takes about 20s.

Use the renamed timer cleanup API where available, with a fallback for
older kernels.

Fixes: ced540165ef5 ("LU-16623 lod: handle object allocation consistently")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Patrickbot <patrick-bot@mulberrytree.us>
Change-Id: I8361eea7d707559ba65a9377fa642ca7833ebbe5

69694 master J –B ✓T ✓R 0/2

LU-20958 llite: only update blocks after a flush

cc Andreas Dilger ·

218_review_prioritypt_ecro 1 unresolved M +116 −10

18h

build #135070 SUCCESS · tests all Maloo sessions

no vote yet: Keguang Xu, Zhenyu Xu

patchsets: PS1 10-14 PS2 10-21 patrick bot PS3 10-29 PS4 10-29 PS5 11-01

threads: 1 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS5 2026-11-01 20:12 Gerrit AI review for Lustre vvp_io.c:1412 — (minor) If the patch is refreshed: this says i_size is not shrunk, but ll_merge_attr_try() -> ll_merge_attr_nolock(..., true) still shrinks

Gerrit AI review for Lustre PS5 · 2026-11-01 20:12

(minor) If the patch is refreshed: this says i_size is not shrunk, but ll_merge_attr_try() -> ll_merge_attr_nolock(..., true) still shrinks i_size when cat_kms_partial is clear, i.e. once every stripe has had a lock granted (the truncate-on-another-client case described in the commit message). Maybe "so i_size is not shrunk while some stripe size is unknown", to match the ll_merge_attr_nolock() kernel-doc.

tags: 218_review_priority pt_ecro

owner Andreas Dilger · uploader Andreas Dilger · PS5 uploaded 1d ago · NEW · open in Gerrit ↗

commit message

LU-20958 llite: only update blocks after a flush

vvp_io_fsync_end() merges the stripe attributes into the inode after
every flush and OST sync, to update the blocks for LSOM.  Neither
takes a lock on the stripes, so the merged size and blocks are only
what this client last cached for each stripe.  That replaces a valid
i_size, e.g. a strict SOM size from the MDS, with a smaller one, down
to 0, for example in the local flush before lseek(SEEK_DATA).  An
lseek of an EC file with a stripe on an unavailable OST then finds no
data before this EOF.

Only update i_blocks there, which is what LSOM needed, and not while
i_size and i_blocks came from the MDS.

A write only locks the stripes it covers, and a stripe that this
client never held a lock on only has the size last cached for it, 0
if never seen, so the merge after a write, an O_SYNC write or a tiny
write could also shrink i_size, e.g. to 1MiB after a write at offset
0 of a 3MiB file with three 1MiB stripes whose size came from the
MDS.  Have lov_merge_lvb_kms() report such a stripe in the new
cl_attr::cat_kms_partial, and since a write never makes a file
shorter, let these merges only grow i_size then.  When the size of
every stripe is known, they may still shrink i_size, e.g. after a
truncate on another client.  If that truncate changed the layout, a
write merge can keep a stale larger i_size until a glimpse, which
only affects the lazy SOM size sent at close.

Add sanity-flr test_44j, which checks that an fsync does not
shrink the strict SOM size of a mirrored file.

Test-Parameters: testlist=sanity-flr
Fixes: dfb08bbf77a1 ("LU-16465 llite: fix LSOM blocks for ftruncate and close")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5-5 ltvm llm_code_and_review_tools
Change-Id: Id5561dd9af3aec5f966ae4db4bc055bb213ebbe5

69941 master J –B ✓T ✗1R -1/2−

LU-12669 ec: EC feature test build (DO NOT LAND)

cc Maximilian Dilger · blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; enforced tests failed

needs rebase pt_ecro 75 unresolved XL +14455 −1606

failed enforced test	platform	detail
custom-1002	RHEL 9.7 / x86_64	ran 3 tests. 1 tests failed: sanity-ec.	session

build #134777 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger

patchsets:

+2 earlier

PS1 10-22 PS2 10-22

PS3 10-22 PS4 10-26 PS5 10-27 PS6 10-28 PS7 10-29

threads: 75 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-22 17:30 Misc Code Checks Robot (Gatekeeper helper) file.c:5403 — parse ll_file_ioctl():error: Function too hairy. Giving up. 5 seconds

Misc Code Checks Robot (Gatekeeper helper) PS1 · 2026-10-22 17:30

parse ll_file_ioctl():error: Function too hairy.  Giving up. 5 seconds
warn: ll_file_ioctl():Function too hairy.  No more merges.

reply PS2 2026-10-22 18:10 Misc Code Checks Robot (Gatekeeper helper) file.c:5410 — parse ll_file_ioctl():error: Function too hairy. Giving up. 5 seconds

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-10-22 18:10

parse ll_file_ioctl():error: Function too hairy.  Giving up. 5 seconds
warn: ll_file_ioctl():Function too hairy.  No more merges.

reply PS3 2026-10-22 18:19 Misc Code Checks Robot (Gatekeeper helper) file.c:5410 — parse ll_file_ioctl():error: Function too hairy. Giving up. 5 seconds

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-10-22 18:19

parse ll_file_ioctl():error: Function too hairy.  Giving up. 5 seconds
warn: ll_file_ioctl():Function too hairy.  No more merges.

reply PS4 2026-10-26 19:52 Misc Code Checks Robot (Gatekeeper helper) file.c:5398 — parse ll_file_ioctl():error: Function too hairy. Giving up. 5 seconds

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-10-26 19:52

parse ll_file_ioctl():error: Function too hairy.  Giving up. 5 seconds
warn: ll_file_ioctl():Function too hairy.  No more merges.

reply PS5 2026-10-27 20:12 Misc Code Checks Robot (Gatekeeper helper) file.c:5428 — parse ll_file_ioctl():error: Function too hairy. Giving up. 4 seconds

Misc Code Checks Robot (Gatekeeper helper) PS5 · 2026-10-27 20:12

parse ll_file_ioctl():error: Function too hairy.  Giving up. 4 seconds
warn: ll_file_ioctl():Function too hairy.  No more merges.

reply PS6 2026-10-28 15:44 Misc Code Checks Robot (Gatekeeper helper) file.c:5428 — parse ll_file_ioctl():error: Function too hairy. Giving up. 4 seconds

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-10-28 15:44

parse ll_file_ioctl():error: Function too hairy.  Giving up. 4 seconds
warn: ll_file_ioctl():Function too hairy.  No more merges.

reply PS2 2026-10-22 18:10 Misc Code Checks Robot (Gatekeeper helper) lov_io.c:4036 — warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS2 · 2026-10-22 18:10

warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

reply PS3 2026-10-22 18:19 Misc Code Checks Robot (Gatekeeper helper) lov_io.c:4036 — warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS3 · 2026-10-22 18:19

warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

reply PS4 2026-10-26 19:52 Misc Code Checks Robot (Gatekeeper helper) lov_io.c:4065 — warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS4 · 2026-10-26 19:52

warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

reply PS5 2026-10-27 20:12 Misc Code Checks Robot (Gatekeeper helper) lov_io.c:4051 — warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS5 · 2026-10-27 20:12

warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

reply PS6 2026-10-28 15:44 Misc Code Checks Robot (Gatekeeper helper) lov_io.c:4097 — warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS6 · 2026-10-28 15:44

warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

reply PS7 2026-10-29 17:25 Misc Code Checks Robot (Gatekeeper helper) lov_io.c:4100 — warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

Misc Code Checks Robot (Gatekeeper helper) PS7 · 2026-10-29 17:25

warn: lov_io_ec_rd_start():comparing different units: 'j < nfolios' 'unit_array_size < unit_page'

owner Maximilian Dilger · uploader Maximilian Dilger · PS7 uploaded 3d ago · NEW · open in Gerrit ↗

commit message

LU-12669 ec: EC feature test build (DO NOT LAND)

A test vehicle, not for landing. It squashes the open changes below,
at the patchsets listed, onto master 40b80bc048 (v2_17_60) so that
they can be built and tested together: every open change with both the
218_review_priority and pt_ecro hashtags as of 2026-10-28, plus the
changes marked * or +. It is refreshed as they get new patchsets.

68361 (LU-20690) stores EC parity as LOV_PATTERN_PARITY and no longer
accepts the RAID0|PARITY encoding of master, so EC files must be
created with this build.

64441 (LU-12187 layout pattern negotiation) is applied after 62868,
which drops the module parameters 64441 negotiates with.  The local
fix after 64441 keeps the negotiation with FLR EC on by default
(enable_flr_ec=1, mdt_enable_flr_ec=1).

64919 ps52 and 69392 both make an EC read that parity cannot rebuild
wait for its stripes' OSTs instead of failing: 64919 changes sanity-ec
test_29a to expect it, and 69392 implements it (cl_io::ci_ec_wait),
with the same test_29a and a new test_29b.

Included, in apply order (change/patchset):
   69349/8   LU-12669 ec: serialize concurrent EC recovery readers
   69350/7   LU-12669 ec: create lock placeholder for reconstructed
             pages
   69702/1   LU-20708 llite: rebuild faulting EC pages from parity
   69694/3   LU-20958 llite: only update blocks after a flush
   64086/48  LU-20958 lov: handle unavailable EC stripes in lseek
   69828/2   LU-20958 llite: drop strict SOM size with the UPDATE lock
   62868/4   LU-12187 llite: enable FLR EC support unconditionally
   69762/7   LU-20287 lov: don't block setattr on unavailable OST
   68747/14  LU-20718 ec: ignore stale EC fields from old servers
   68932/4   LU-20718 lod: scrub stale EC fields from old layouts
   69699/5   LU-20963 lod: use a reconnecting OST only when needed
   66882/41  LU-20211 mdt: trunc(0) layout replace on inactive stripes
 * 69305/6   LU-8933 lod: balance round-robin first stripe start
   69421/5   LU-19823 lod: balance RR allocation across failure
             domains
   69422/3   LU-8933 lod: leave inactive targets out of the RR array
   69426/5   LU-19823 lod: spread failover groups over the RR array
   69697/7   LU-20967 osd-ldiskfs: name the failure_domain local file
   69111/14  LU-20797 llite: add SET I/O scope ioctl
   69516/7   LU-20690 lov: slot-based stripe geometry for EC parity
   69517/9   LU-20690 ec: implement STRIPE I/O scope
   69575/7   LU-20690 utils: add EC parity-stripe I/O helpers
   68361/17  LU-20690 ec: like-RAID10 parity layout
   69703/3   LU-20690 ec: verify parity against data, allow p > k
   69908/1   LU-12669 ec: fix degraded read page leak and short read
 + 69913/1   LU-20708 osc: let read faults switch mirrors on a bad OSC
 + 69914/1   LU-12328 llite: move a failed DIO read to the next mirror
 + 69493/5   LU-19823 lod: unique failure domains per EC raid set
 + 69824/3   LU-20212 obdclass: don't block no-delay statfs on others
 + 69352/3   LU-20200 utils: report unreachable targets in lfs df -l
 + 64919/52  LU-12669 lov: slow and dead OST detection for degraded
             reads
   local     EC feature merge fix: 64919 with 64086 and 68361
 + 69709/2   LU-12669 lov: peer-relative deadline for EC raid set
             reads
   69710/2   LU-12669 lov: split EC reads at a raid set parity cannot
             cover
 + 64441/32  LU-12187 lov: MDS layout pattern negotiation
   local     local: 64441 on 62868: FLR EC on by default, fail_loc
             0x143c
   69392/3   LU-12669 ec: wait for stripes that parity cannot rebuild
 * 69768/4   LU-11962 flr: fix the handling of foreign mirrors
 + 69769/2   LU-11962 flr: fix FLR and SOM state after a layout swap
   70067/1   LU-20690 ec: copy parity in 1+N recovery reads
 + 69916/3   LU-21014 lod: keep EC raid sets on distinct OSTs in RR

* untagged parent of a tagged change
+ untagged, added on request

local = fix for conflicts with master or between the changes, not in
Gerrit.

Conflicts resolved by hand: 69828, 62868, 66882, 69422, 69516, 69517,
68361, 69703, 69908, 69914, 64919, 69709, 69710, 64441, 69392, 69768,
69769.

Test-Parameters: fortestonly ostcount=8
Test-Parameters: fortestonly testlist=sanity-ec ostcount=8
Assisted-by: ClaudeCode:claude-opus-5.5 llm_code_and_review_tools
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Change-Id: I647c485c7b6b92acdd4484f94663243d191a0b37

62952 master J –B ✓T ✓R 0/2

LU-19687 lov: skip foreign layouts for regular file IO

cc Alex Zhuravlev ·

19 unresolved S +43 −2

build #134759 SUCCESS · tests all Maloo sessions

no vote yet: Andreas Dilger, Zhenyu Xu, Oleg Drokin, Nikitas Angelinas, Patrick Farrell

patchsets:

+16 earlier

PS1 12-08 PS2 12-09 PS3 12-09 PS4 01-09 PS5 01-28 PS6 02-11 PS7 02-18 PS8 03-10 PS9 03-12 PS10 03-30 Andreas Dilger PS11 04-29 PS12 04-29 PS13 05-02 PS14 05-29 PS15 05-29 PS16 06-08

PS17 06-12 PS18 06-23 PS19 09-16 PS20 10-23 PS21 10-29

threads: 19 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS19 2026-09-18 04:16 Gerrit AI review for Lustre COMMIT_MSG:20 — (minor) This removes a client LBUG, so a Fixes: tag would help decide which maintenance branches need it. The LASSERT came in with 61a002cd8

Gerrit AI review for Lustre PS19 · 2026-09-18 04:16

(minor) This removes a client LBUG, so a Fixes: tag would help decide which maintenance branches need it. The LASSERT came in with 61a002cd8631 ("LU-13602 flr: skip unknown FLR component types"), but it only became reachable for a plain PFL file once a broken component started being converted to an in-memory foreign entry:

    Fixes: 902fe290e51d ("LU-17261 lov: ignore broken components")

reply PS20 2026-10-26 15:24 Gerrit AI review for Lustre COMMIT_MSG:16 — (typo) "such a write" -> "such as a write".

Gerrit AI review for Lustre PS20 · 2026-10-26 15:24

(typo) "such a write" -> "such as a write".

reply PS20 2026-10-26 15:24 Gerrit AI review for Lustre COMMIT_MSG:19 — (minor) If the patch is refreshed, could this also name the new OBD_FAIL_LOV_INVALID_OSTIDX2 fail_loc in lod_gen_component_ea()? The test_21

Gerrit AI review for Lustre PS20 · 2026-10-26 15:24

(minor) If the patch is refreshed, could this also name the new OBD_FAIL_LOV_INVALID_OSTIDX2 fail_loc in lod_gen_component_ea()? The test_210c/d sentence is the only hint that the patch also changes MDS code.

reply PS19 2026-10-23 00:26 Oleg Drokin patchset comment — sanity-flr test_210c/test_210d fail in all 7 interop sessions with a b2_15 or b2_16 MDS

Oleg Drokin PS19 · 2026-10-23 00:26

sanity-flr test_210c/test_210d fail in all 7 interop sessions with a b2_15 or b2_16 MDS
in lustre-master-next build 1018 ("can dd" / "can truncate"), and pass in the other 50:
https://testing.whamcloud.com/test_sets/5655b63e-5787-4566-9872-2ea78d70e78d
https://testing.whamcloud.com/test_sets/7a2a037d-1b7c-4dd2-bae4-33f33a05cb9a
The tests depend on fail_loc 0x1440 (OBD_FAIL_LOV_INVALID_OSTIDX2) on mds1, which only
exists with this patch's lod_lov.c change, so an older MDS hands out a valid layout.
Please add an MDS version guard next to the client one, e.g.
  (( $MDS1_VERSION >= $(version_code 2.17.<landing version>) )) ||
          skip "need MDS with OBD_FAIL_LOV_INVALID_OSTIDX2"

reply PS20 2026-10-29 10:58 Alex Zhuravlev obd_support.h:659 — oh, thanks!

Marc Vef PS20 · 2026-10-29 10:35

(conflict) this line has a neighbouring line conflict with https://review.whamcloud.com/c/fs/lustre-release/+/69573

69573 is ready. I just rebased 69573 on the current master. Since this patch needs to be refreshed again, I think it makes sense to rebase this patch onto 69573 so they can land together

Alex Zhuravlev PS20 · 2026-10-29 10:58

oh, thanks!

reply PS21 2026-10-31 06:55 Gerrit AI review for Lustre obd_support.h:660 — (style) This isn't a bug, but 0x1440 sits between 0x1439 and 0x143a, so the list is out of order and 0x143b-0x143f are skipped. If the patch

Gerrit AI review for Lustre PS21 · 2026-10-31 06:55

(style) This isn't a bug, but 0x1440 sits between 0x1439 and 0x143a, so the list is out of order and 0x143b-0x143f are skipped. If the patch is refreshed, could it use the next free value (0x143b) after OBD_FAIL_LLITE_DIO_FALLBACK? A name that says what it does, e.g. OBD_FAIL_LOV_INVALID_OSTIDX_COMP0 (every stripe of component 0 gets a bad index), would also read better than a "2" suffix.

reply PS10 2026-04-27 22:25 Nikitas Angelinas lod_lov.c:830 — cfs_fail_val isn't used in the added tests, fwiw

Nikitas Angelinas PS10 · 2026-04-27 22:25

cfs_fail_val isn't used in the added tests, fwiw

reply PS10 2026-04-27 22:25 Nikitas Angelinas lod_lov.c:830 — the braces aren't necessary and could be removed; same for the CFS_FAIL_CHECK() below

Nikitas Angelinas PS10 · 2026-04-27 22:25

the braces aren't necessary and could be removed; same for the CFS_FAIL_CHECK() below

reply PS10 2026-04-29 07:34 Alex Zhuravlev lod_lov.c:832 — well, no, I wanted to make it looking "mostly fine"

Nikitas Angelinas PS10 · 2026-04-27 22:25

Could this be cpu_to_le32(0xffffffff), same as below?

Alex Zhuravlev PS10 · 2026-04-29 07:34

well, no, I wanted to make it looking "mostly fine"

reply PS19 2026-09-18 04:16 Gerrit AI review for Lustre lod_lov.c:846 — (style) Not a bug, but if the patch is refreshed: ltd_tgts_size+1 wants spaces around the +, and the two arms of the same if spell the same

Gerrit AI review for Lustre PS19 · 2026-09-18 04:16

(style) Not a bug, but if the patch is refreshed: ltd_tgts_size+1 wants spaces around the +, and the two arms of the same if spell the same test differently - !comp_idx here versus comp_idx == 0 just below.

reply PS20 2026-10-26 15:24 Gerrit AI review for Lustre lod_lov.c:848 — (style) This isn't a bug, but now that the opening line is "} else if (", the continuation line should be re-aligned to start after the open

Gerrit AI review for Lustre PS20 · 2026-10-26 15:24

(style) This isn't a bug, but now that the opening line is "} else if (", the continuation line should be re-aligned to start after the open parenthesis. It still has the old "if (" indentation.

reply PS19 2026-09-18 04:16 Gerrit AI review for Lustre lov_io.c:2356 — (defect) Skipping the entry here means lov_io_layout_at() reports "no component at this offset", and lov_foreach_io_layout() uses exactly th

Gerrit AI review for Lustre PS19 · 2026-09-18 04:16

(defect) Skipping the entry here means lov_io_layout_at() reports "no component at this offset", and lov_foreach_io_layout() uses exactly that as its loop terminator:

    ind = lov_io_layout_at(lio, lov_io_extent(lio, ind)->e_end)

Take a mixed layout - comp0 [0,4M) regular, comp1 [4M,8M) foreign, comp2 [8M,EOF) regular. The loop in lov_io_iter_init() starts at comp0, then the advance step looks up offset 4M: comp1 is now skipped and comp2 does not cover 4M, so it returns -1 and the loop ends. comp2 never reaches lis_active, so a CIT_FSYNC or CIT_DATA_VERSION over the whole file quietly does nothing for it. The foreign component is not skipped, it truncates the iteration.

Everywhere else the tree handles a foreign component by skipping it in the loop body, so iteration continues - lov_io_iter_init() at line 1094 and lov_attr_get_composite() in lov_object.c. Those checks become unreachable with this change, as do lov_io.c:1251 and the lsm_entry_is_foreign() term at lov_io.c:1461, since lov_io_layout_at() can no longer return a foreign index.

Would it be better to have lov_io_layout_at() keep returning the foreign entry and let those per-caller checks decide? That steps over the foreign extent instead of stopping at it, and matches the PCC-RO / HSM-in-PFL case raised earlier in the review, where the regular components around a foreign one should stay usable.

owner Alex Zhuravlev · uploader Alex Zhuravlev · PS21 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-19687 lov: skip foreign layouts for regular file IO

If the client sees a bad layout in lsm_unpackmd_comp_md_v1(),
for example if the OST index is invalid after the retry loop in
lsme_unpack_comp()->lsme_unpack() waits for newly-added OSTs to
appear in the client configuration (LU-17334) and fails, then
lsm_unpackmd_comp_md_v1() changes the component type to use
LOV_MAGIC_FOREIGN (in memory only).

If a regular IO (such a write) meets a foreign component layout
in lov_io_layout_at(), then just skip it instead of LASSERT().

Add sanity-flr.sh test_210c and test_210d to verify this case.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I51c6dbc196d70d8d03a165b9877f7617b85dfac1

70150 master J –B ✓T ✓R 0/2

LU-19823 lod: fit EC raid sets to the OSTs without domains

cc Andreas Dilger ·

pt_ecro 2 unresolved M +80 −24

build #134751 SUCCESS · tests all Maloo sessions

threads: 2 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS1 2026-10-31 04:32 Gerrit AI review for Lustre lfs-setstripe.1:419 — (minor) Following the lod_ec_fit_fdoms() question: when STRIPE_COUNT is larger than the available OSTs, the raid sets can end up below 3/4 o

Gerrit AI review for Lustre PS1 · 2026-10-31 04:32

(minor) Following the lod_ec_fit_fdoms() question: when STRIPE_COUNT is larger than the available OSTs, the raid sets can end up below 3/4 of DATA_STRIPES (e.g. `-c 12 --ec 6+3` on 7 OSTs gives 4+3 and 3+3). If that is intended, could this sentence say the floor only applies when the stripe count fits the OSTs?

reply PS1 2026-10-31 04:32 Gerrit AI review for Lustre lod_qos.c:3882 — (minor) Once count is clamped to nost, can the raid sets drop below the K - K/4 floor that the commit message and man page promise? The earl

Gerrit AI review for Lustre PS1 · 2026-10-31 04:32

(minor) Once count is clamped to nost, can the raid sets drop below the K - K/4 floor that the commit message and man page promise? The early return here skips the kmin check, and RR/QoS then re-split the stripes they place.

Example, without domains (or with 8 OSTs in 8 domains), `-c 12 --ec 6+3`:

    count = min(12, 8) = 8
    ec_split_stripes(8, 6) -> 2 x 4, k = 4
    nfdom 8 >= k + p 7 -> return 0
    lod_ost_alloc_rr() places 8, lod_ec_shrink_data_geometry() -> 2 x 4

The file gets two 4+3 raid sets, but kmin for 6 is 5. On 7 OSTs it gets 4+3 and 3+3. With domains configured the parent returned ENOSPC for both cases (k2 = 4 < kmin), so for those setups this is a behaviour change.

At the same time `-c 6 --ec 6+3` on 7 OSTs still fails, because k2 = 4 < kmin = 5. So the same 6+3 request is refused at K=4 when it is narrow and accepted at K=3 or 4 when it is wider than the OSTs. Should the clamped path also compare k against kmin, or is the floor meant to apply only to the parity fit?

tags: pt_ecro

owner Andreas Dilger · uploader Andreas Dilger · PS1 uploaded 4d ago · NEW · open in Gerrit ↗

commit message

LU-19823 lod: fit EC raid sets to the OSTs without domains

With "lfs setstripe -c 6 --ec 6+3" on 8 OSTs, QoS shrank the single
raid set to 5+3 so that its parity fit, while RR placed all 6 data
stripes and then failed with ENOSPC when it found only 2 OSTs left
for the parity.  Which one ran depended on the free space balance of
the OSTs, so sanity-ec test_27h passed or failed depending on the
earlier tests.

lod_ec_fit_fdoms() already fits the raid sets to the failure domains
before either allocator places the data.  Without failure domains,
count each active OST as a domain of its own, so RR and QoS fit the
same K.  A -c component wider than the active OSTs is fitted by the
stripes it can place, and if its raid sets must shrink, still to at
least 3/4 of the requested K.  As with domains, a layout may be
stored with the smaller K of its raid sets.  Describe this in
lfs-setstripe.1.

Have sanity-ec test_27h check that a layout which can't fit fails, and
one that can gets fewer data stripes, with both the RR and QoS
allocators.

Fixes: c53abca9f9ee ("LU-19852 lod: raidset aware stripe allocator")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: Ief45b5df095e03fd96065b85eb6b1008bd3ebbe5

64919 master J –B ✓T ✓R +1/2

LU-12669 lov: slow and dead OST detection for degraded reads

cc Maximilian Dilger ·

pt_ecro 4 unresolved XL +2386 −78

build #134398 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger +1 2026-10-24 07:13

no vote yet: Ronnie Sahlberg, Zhenyu Xu

patchsets:

+47 earlier

PS1 03-26 PS2 03-26 PS3 03-29 Andreas Dilger PS4 03-29 Patrick Farrell PS5 03-29 Patrick Farrell PS6 03-29 Patrick Farrell PS7 03-29 Ronnie Sahlberg PS8 03-30 Ronnie Sahlberg PS9 03-31 Ronnie Sahlberg PS10 03-31 Patrick Farrell PS11 03-31 Ronnie Sahlberg PS12 04-01 Ronnie Sahlberg PS13 04-01 Ronnie Sahlberg PS14 04-07 Ronnie Sahlberg PS15 04-08 Patrick Farrell PS16 04-09 Patrick Farrell PS17 04-09 Patrick Farrell PS18 04-09 Patrick Farrell PS19 04-09 Patrick Farrell PS20 04-09 Patrick Farrell PS21 04-09 Patrick Farrell PS22 04-09 Patrick Farrell PS23 04-09 Patrick Farrell PS24 04-10 Patrick Farrell PS25 04-10 Patrick Farrell PS26 04-10 Patrick Farrell PS27 04-10 Patrick Farrell PS28 04-10 Patrick Farrell PS29 04-14 Patrick Farrell PS30 04-14 Patrick Farrell PS31 04-14 Patrick Farrell PS32 04-15 Patrick Farrell PS33 04-15 Patrick Farrell PS34 04-15 Patrick Farrell PS35 04-21 PS36 04-25 PS37 05-04 PS38 05-20 Marc Vef PS39 06-04 Ronnie Sahlberg PS40 07-13 Marc Vef PS41 07-13 Marc Vef PS42 07-13 Marc Vef PS43 07-15 Marc Vef PS44 07-26 Marc Vef PS45 07-26 Marc Vef PS46 08-04 PS47 08-19

PS48 09-26 PS49 10-05 Andreas Dilger PS50 10-05 Andreas Dilger PS51 10-15 Andreas Dilger PS52 10-24 Andreas Dilger

threads: 4 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS50 2026-10-05 18:53 Gerrit AI review for Lustre COMMIT_MSG:62 — (minor) Earlier review agreed to keep the lov_io_set_range(), readahead and via_hybrid_switched fixes in this patch. The io_latency_stats ar

Gerrit AI review for Lustre PS50 · 2026-10-05 18:53

(minor) Earlier review agreed to keep the lov_io_set_range(), readahead and via_hybrid_switched fixes in this patch. The io_latency_stats array fix is newer, though, and is a different case. The arrays are allocated with PTLRPC_MAX_BRW_BITS - PAGE_SHIFT entries, but brw_interpret() indexes them with fls(page_count) - 1. A maximum-size RPC therefore writes one histogram past the end of the allocation. d67cf23fb18a is already in 2.17.0, so this heap overflow needs to reach b2_17, and the EC degraded-read feature will not be backported there. Could the OSC_LAT_BUCKETS change go in a separate patch with its own Fixes: line, so it can be cherry-picked on its own?

reply PS52 2026-10-26 15:10 Gerrit AI review for Lustre lov_io.c:3631 — (defect) lov_sub_unavailable() is evaluated again for every parity stripe on each pass, and the answer can change between the passes. Pass 0

Gerrit AI review for Lustre PS52 · 2026-10-26 15:10

(defect) lov_sub_unavailable() is evaluated again for every parity stripe on each pass, and the answer can change between the passes. Pass 0 does a synchronous lov_ec_read_parity_stripe() (cl_sync_io_wait(), no soft deadline) for each available stripe, and a slow mark is time-bound, so cl_read_slow_until can expire while pass 0 is reading.

Example, err_nr == 2, pcount == 2, parity p0 slow-marked: pass 0 skips p0 and reads p1. p0's mark expires during that read. Pass 1 now sees p0 as available, so `lov_sub_unavailable(los) != pass` skips it again, par_stripes_used stays 1, and the read fails with "not enough parity stripes" -EIO. Before this patch the single loop would have read p0 and recovered.

The opposite flip (a stripe already read in pass 0 gets marked slow by another reader's soft timeout) would read it again and put the same index twice into ecr_parity_used[], which makes the decode matrix singular (-EIO from gf_invert_matrix()).

Could the availability of each parity stripe be sampled once before the passes (e.g. a small bitmap of pcount bits) so each stripe is tried exactly once?

reply PS52 2026-10-26 15:10 Gerrit AI review for Lustre lov_lock.c:238 — (minor) Is it intended that this also matches stripes that have no sub-IO only because they are past EOF? lov_ecio_add_data_sub() caps data_

Gerrit AI review for Lustre PS52 · 2026-10-26 15:10

(minor) Is it intended that this also matches stripes that have no sub-IO only because they are past EOF? lov_ecio_add_data_sub() caps data_ext at ec_inode_size, so in a small file the other data stripes of the expanded raid set get no sub-IO, while io_ext here is the uncapped [lis_pos, lis_endpos). Those sub-locks on healthy OSTs are then turned into placeholders instead of being enqueued.

That may well be harmless (no pages are rebuilt there), but it is not what the comment says ("as it routes around the OST, is rebuilt"), and it widens the placeholder use from unavailable OSTs to every stripe past EOF of a degraded read. If the patch is refreshed, either intersect with the EOF-capped extent here, or document the past-EOF case in the comment and commit message.

reply PS52 2026-10-26 15:10 Gerrit AI review for Lustre osc_request.c:4231 — (minor) osc_setup_common() is also called by mdc_setup(), and DoM reads go through the same osc_queue_sync_pages()/osc_build_rpc() path, so

Gerrit AI review for Lustre PS52 · 2026-10-26 15:10

(minor) osc_setup_common() is also called by mdc_setup(), and DoM reads go through the same osc_queue_sync_pages()/osc_build_rpc() path, so a mirrored file with a DoM component gets soft deadlines on its MDT reads with slow_read_pct=300. But slow_read_pct, slow_read_min_ms and slow_read_stats are only registered for osc, so an admin cannot see or turn this off for the MDC (`osc.*.slow_read_pct=0` does not cover it). Should MDC either get the same tunables, or start with cl_slow_read_pct = 0?

tags: pt_ecro

owner Maximilian Dilger · uploader Andreas Dilger · PS52 uploaded 9d ago · NEW · open in Gerrit ↗

commit message

LU-12669 lov: slow and dead OST detection for degraded reads

A read of an EC file whose data stripe is on a dead OST waits for
the RPC's adaptive timeout, 7 s in production, before it is
reconstructed from parity; a slow OSS extends that with early
replies, and even a dead import costs the FLR retry rounds first.

Give such reads a soft deadline and route around the OST.  A
request past it fails with -ETIMEDOUT without failing the import or
touching the adaptive timeout.  Each OSC keeps a smoothed OST_READ
latency and deviation per RPC size, as for the TCP retransmission
timer, and a top-level no-delay read of an EC or mirrored file gets
a deadline, counted from its send, of the largest of
osc.*.slow_read_pct (default 300) percent of the smoothed latency,
that latency plus four deviations, and osc.*.slow_read_min_ms
(default 250 ms for a non-rotational OST, 2 s otherwise).  An
abandoned read marks the import slow for four times its deadline;
each further soft timeout before a successful read doubles the
deadline and the mark, up to the adaptive timeout.

LOV treats a deactivated, invalid, disconnected or slow-marked import
as unavailable, but not one reconnecting from idle.  A read of a dead
stripe is reconstructed at once when its raid set's own parity covers
its dead stripes, and of a slow-marked one only when it covers all of
its unavailable stripes; otherwise the read waits for them, e.g. until
an OSS failover completes.  A read spanning several raid sets waits if
any of them cannot be rebuilt.  With no in-sync parity the file reads
like a plain mirror.  A read of part of a raid set, whose
reconstruction reads many times its size, gets no soft deadline and is
not reconstructed around a slow stripe when most of the set's other
stripes are on OSTs running at over twice their median latency since
mount and over an eighth of slow_read_min_ms.  A plain mirror's
buffered reads get the same deadline and rotate mirrors on expiry,
unless no other mirror can serve the read.  An EC read skips the lock
and read of a stripe it reconstructs by having no sub-IO for it,
rather than through the lso_status shared with other readers.
osc.*.slow_read_stats shows the estimates and counters.

A 1MiB read of stripe 0 of a 4+2 EC file on an 8-OST VM, with one
30s stall of its OST (OBD_FAIL_OST_BRW_PAUSE_BULK), then 200 1MiB
direct and 200 4KiB buffered random reads while 4 writers load all
OSTs:

  slow_read_pct            0 (off)    300 (default)
  direct read, stall       30053ms    2069ms
  buffered read, stall     30055ms    2048ms
  reads under load: 0 soft timeouts, no extra parity or survivor reads

Also fix lov_io_set_range() rounding the reconstruction range end
past its component, which LBUGged "uncovered page!"; the
ra_cur_pages leak in ll_readahead_handle_work() when
cl_io_rw_init() fails, which stopped readahead for the mount; and
do_file_read_iter() never clearing via_hybrid_switched, which ran
later O_DIRECT AIO reads on the same lu_env synchronously.  Also fix
ll_readahead_handle_work() passing a page index and count to
cl_io_rw_init() as bytes, which routed async readahead of FLR and EC
files by the wrong extent, and the client io_latency_stats arrays
being one size short for a maximum-size RPC.

Add sanity-ec 54a-54k, and change test_29a to check that a read with
more stripes lost than parity waits for their OSTs.

Test-Parameters: testlist=sanity-ec ostcount=8
Test-Parameters: testlist=sanity-ec ostcount=8 fstype=zfs
Test-Parameters: testlist=sanity,sanity-flr ostcount=8
Fixes: c2791674260b ("LU-12043 llite: improve single-thread read performance")
Fixes: 3fc4243da7ef ("LU-13802 llite: tag switched hybrid IOs")
Fixes: 783c13bb0b8f ("LU-12669 ec: recover data from parity")
Fixes: d67cf23fb18a ("LU-18993 osc: Add IO RPC latency statistics by size")
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Change-Id: I0913e1ee977c9850193c92835edb185b0aedc6d4

63144 master J –B ✓T ✓R 0/2

LU-18158 sec: retry mechanism for most syscalls

cc Sebastien Buisson ·

XL +1096 −116

build #134563 SUCCESS · tests all Maloo sessions

no vote yet: Martin Schwenke, Andreas Dilger, Lai Siyao, Li Xi

patchsets:

+27 earlier

PS1 12-22 PS2 12-22 Janitor Bot PS3 12-23 PS4 01-07 PS5 02-26 PS6 03-02 PS7 03-05 PS8 03-05 PS9 03-10 PS10 03-23 PS11 06-12 PS12 06-12 PS13 07-06 PS14 07-07 PS15 07-09 PS16 08-13 PS17 08-14 PS18 08-17 PS19 08-18 PS20 09-04 PS21 09-07 PS22 09-08 PS23 09-10 PS24 09-20 PS25 09-30 PS26 10-02 PS27 10-05

PS28 10-06 PS29 10-07 PS30 10-09 PS31 10-23 PS32 10-27

owner Sebastien Buisson · uploader Sebastien Buisson · PS32 uploaded 5d ago · NEW · open in Gerrit ↗

commit message

LU-18158 sec: retry mechanism for most syscalls

In addition to file/dir create, a number of other syscalls can
benefit from a retry mechanism in case of -EACCES. This can happen if
the client was not able to send the correct supplementary groups.
After the first failed attempt, the failure reply from the
server includes a hint with a possible supplementary group and ACLs.
The client uses this hint from the reply and retries with alternative
supplementary groups.

The syscalls handled by this patch are:
- open
- get/setattr
- mkdir
- unlink
- rename
- link

rename is special because unlike other syscalls handled by this patch,
access control is carried out based on permissions of source file,
source directory and target directory. As we do not have enough room
in the MDS_REINT_RENAME request, we proceed in 2 steps. If the rename
operation initially fails with -EACCES, the server hints the client
with the ACLs of the source directory, and the client tries again the
rename with this additional information. If it fails again with
-EACCES, the clients sets the MDS_RENAME_AGAIN op bias, so that the
server knows it needs to hint with the target dir ACLs this time in
the reply. The client then tries one last time the rename with this
additional information.

For the reply to have room for the ACL, RMF_ACL is added to the
MDS_REINT_UNLINK, MDS_REINT_LINK, MDS_REINT_RENAME and
MDS_REINT_CREATE_ACL reply formats, and to MDS_REINT_CREATE_SYM so
that it keeps matching MDS_REINT_CREATE_ACL, which is the format the
client uses to send a symlink create. The client and the server both
set the ACL reply buffer size to 0 as long as the hint is not needed.

sanity-sec test_80 is added to exercise this.

Test-Parameters: serverversion=2.15 testlist=sanity env=EXCEPT="24I 24v 27cg 51b 160a 230k 230w 271f 272 277 421d 807"
Fixes: e603ddadd5 ("LU-18158 sec: hint client in case of failed reint open")
Fixes: e826e8bb77 ("LU-17961 sec: support supplementary groups from client")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6497b21356dfbfc648c58dec0f9c1868655f6364
Was-Change-Id: I515467cb7fb5c26a5496f8cac9fb4749723bae5a
Was-Change-Id: I424affdafb1fd9ce375cf30f6e5c8af1716a3899
Was-Change-Id: If7ebbe2b8e3a897f433737693b2ab63b3dae59a4
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Martin Schwenke <mschwenke@ddn.com>

68073 master J –B ✓T ✓R 0/2

LU-20354 quota: refresh QMT usage cache via MDT/OSP proxy

cc Keguang Xu ·

3 unresolved L +759 −66

15d

build #133812 SUCCESS · tests all Maloo sessions

no vote yet: Sergey Cheremencev, Andreas Dilger, Hongchao Zhang

patchsets:

+1 earlier

PS1 08-18

PS2 09-02 PS3 09-04 PS5 09-10 PS6 10-14 PS7 10-16

threads: 3 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS7 2026-10-18 02:10 Gerrit AI review for Lustre mdt_handler.c:6122 — (typo) Missing space before `*/`.

Gerrit AI review for Lustre PS7 · 2026-10-18 02:10

(typo) Missing space before `*/`.

reply PS7 2026-10-18 02:10 Gerrit AI review for Lustre qmt_entry.c:1344 — (minor) The extra members don't really wait for a later refresh. After qmt_refresh_lqe_usg() returns 0, qmt_get_usage() re-sums with check_t

Gerrit AI review for Lustre PS7 · 2026-10-18 02:10

(minor) The extra members don't really wait for a later refresh. After qmt_refresh_lqe_usg() returns 0, qmt_get_usage() re-sums with check_ttl=false over the live sarr, not over the copied idxs. So a member added during the walk is summed at whatever age its slot has.

Example: `lctl pool_add` adds OST N to a named pool while the serial Q_GETOQUOTA walk is running. OST N already has a slot on the global lqe from an old DQACQ (slots are only freed with the lqe), but that slot expired long ago. Refresh never touches N, and the following sum serves N's old value as part of a fresh aggregate. A member with no slot gets -ESTALE, which is fine.

This only matters once qmt_usg_serve is enabled, and the result is one stale reply, not corruption. But either the comment or the post-refresh sum should change, e.g. sum only the idxs that refresh handled, or redo the refresh if op_count changed.

reply PS5 2026-09-11 05:49 Gerrit AI review for Lustre qmt_handler.c:776 — (style) This isn't a bug, but the comment sits directly above an assignment that runs for every command in this case group, while the text o

Gerrit AI review for Lustre PS5 · 2026-09-11 05:49

(style) This isn't a bug, but the comment sits directly above an assignment that runs for every command in this case group, while the text only describes the two commands handled by the condition below it. The old comment explained the assignment itself. Moving this line down onto the `if` would read better.

owner Keguang Xu · uploader Keguang Xu · PS7 uploaded 17d ago · NEW · open in Gerrit ↗

commit message

LU-20354 quota: refresh QMT usage cache via MDT/OSP proxy

Wire qmt_refresh_lqe_usg() to pull stale or missing per-slave
usage through MDT: local OSD for the DOM MDT, OSP Q_GETOQUOTA
for OSTs. The cache stays on the global DT lqe. The caller
passes that lqe and the pool; refresh walks the pool sarr
(alloc outside the lock, copy under it). Fresh slots,
including DQACQ-warmed zeros, are left alone. Remote MDT
DOM fetch is still TODO.

On fetch failure, keep a cached value when the slot exists,
otherwise fall back to granted from that slave's index.
Degraded values are back-dated to expire after about 60s
so the next refresh retries soon.

qmt_sum_lqe_usg() takes check_ttl: true for a cache lookup,
false after refresh so freshly stored slots are served even
when ttl is 0.

qc_id is already a filesystem ID. MDS/OSS NIDs belong in a
trusted nodemap, so OFD's CLIENT_TO_FS map is identity and
no skip-idmap handshake is needed.

qmt_get_usage() sums and refreshes only when qmt_usg_serve is
on. The flag stays off, so GETQUOTA still leaves usage unset.
qmt_usg_fallback counts a granted fallback or a failed serve.

Expose qmt.*.usage_cache_stats (hit/miss/fetch_rpc/fallback).
Add sanity-quota/310, skipped until qmt_usg_serve is on.
Extend qmt_usg_test for ttl 0 with check_ttl true/false.

Test-Parameters: trivial testlist=sanity-quota
Assisted-by: Cursor:Grok-4.6 llm_code_and_review_tools
Signed-off-by: Keguang Xu <kxu@ddn.com>
Change-Id: Ied2f4f1e3c7ee6611a69a57e8064d46a59ba845f

69026 master J ✗1B ✓T ✗1R -2/2−

LU-17217 nodemap: add client version policy

cc Timothy Day · Andreas Dilger pinged you: “@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here? If this pa”

needs rebase 1 unique failure 12 unresolved L +805 −3

20d

Andreas Dilger pinged you: “@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here? If this pa”

blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; janitor: test failures unique to this patch; enforced tests failed

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
sanity-hsm@ldiskfs+DNE:test_254b	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

failed enforced test	platform	detail
review-ldiskfs-dne-arm	RHEL 8.10 / x86_64, Rocky 9.8 / aarch64	ran 5 tests. 1 tests failed: sanity.	session

build #131922 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger -1 2026-09-18 04:21

no vote yet: Patrick Farrell

1 ping waiting on you:

Gerrit AI review for Lustre PS1 · 2026-09-18 09:49

(minor) Adding the bit changes NODEMAP_RAISE_PRIV_NONE from 0xfffffe00 to 0xfffffc00, and that value is stored on disk rather than being a pure in-memory sentinel.

nodemap_idx_cluster_roles_modify() writes a NODEMAP_CLUSTER_ROLES record whenever rbac != NODEMAP_RBAC_ALL, and the record carries ncrr_privs = nmf_raise_privs, i.e. the old 0xfffffe00 for a nodemap whose raise_privs is 'none'. nodemap_cluster_roles_helper() then assigns it back verbatim with no masking.

So after an upgrade, a nodemap that merely had a non-default rbac comes back with bit 0x200 set, and

    if ((nodemap->nm_parent_nm->nmf_raise_privs & priv) && priv != NODEMAP_RAISE_PRIV_RBAC)
            return true;

short-circuits the new NODEMAP_RAISE_PRIV_VERSION check, so a dynamic child can override a parent policy that was meant to be locked. raise_privs would also read back as 'version_policy' for a nodemap the admin set to 'none'.

This encoding predates the patch (CAPS and DENY_MNT had the same effect when they were added) so it probably belongs in its own change rather than blocking this one - but is it worth masking the loaded value to the known bits, or is the intent that unknown bits stay raised?

Andreas Dilger PS1 · 2026-09-19 08:08

@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here?  If this patch is introducing the mismatch between the flags then it makes sense to fix it here as well instead of separate patch.

Sebastien Buisson PS1 · 2026-09-25 15:55

`NODEMAP_RAISE_PRIV_NONE` was made on the same model as `NODEMAP_RBAC_NONE`, but the sematic is quite the opposite actually. Rbac roles are all present by default, and we remove some of them to lower client capabilities. 'Raise privs' are all absent by default, and we add some of them to grant privileges. So I think we should set `ncrr_privs` to 0 when `nmf_raise_privs` is `NODEMAP_RAISE_PRIV_NONE`. That would avoid the problem mentioned here when we upgrade a system after new privs have been added, and that would be a simple fix. I could push a quick patch, so that it is fixed right now and does not wait for this bigger patch.

threads: 12 need your reply · 0 waiting on others · 0 own notes · 0 bot

Gerrit AI review for Lustre PS1 · 2026-09-18 09:49

(minor) "older servers never see it" only holds while no policy is set. Once one is set the NODEMAP_CLUSTER_VERSION record is written to the MGS index and distributed to every server, including older ones - which is exactly the case that matters.

The subid-unknown fallback in nodemap_process_keyrec() that ignores it with a warning only landed in 2.15.55, so a 2.15.x LTS server does not have it. Could the sentence say instead that the policy is simply not enforced on servers that predate it?

reply PS1 2026-09-18 04:21 Andreas Dilger lctl-nodemap-set-version-policy.8:18 — Is there a reason not to include the 4th version digit? That is needed for versions like 2.18.0-tlc7 or whatever.

Andreas Dilger PS1 · 2026-09-18 04:21

Is there a reason not to include the 4th version digit?  That is needed for versions like 2.18.0-tlc7 or whatever.

Timothy Day PS1 · 2026-09-17 20:20

We probably want more glob examples.

Andreas Dilger PS1 · 2026-09-18 04:21

I'm not sure that "glob" is actually the right paradigm for this, if it ends up being converted to a number anyway. I think that storing a min and max numerical range for allow or deny would match the functionality better:

    --policy allow --min 2.16.0.22 --max 2.18.1.12

Andreas Dilger PS1 · 2026-09-18 04:21

This could be reduced to `__u32 nvpr_min` and `__u32 nvpr_max`.  If this is the only such record per nodemap, then it could contain a min/max allow and min/max block version (16 bytes) and still have space left for some other parameters.  It would be possible to have separate allow and block records do describe multiple non-overlapping regions (e.g. blacklist specific versions), but I don't know if that is needed yet?

I'm not sure if there are other parameters we could store within a single record?

reply PS1 2026-09-18 04:21 Andreas Dilger lustre_disk.h:392 — (style) align fields in struct declarations

Andreas Dilger PS1 · 2026-09-18 04:21

(style) align fields in struct declarations

Gerrit AI review for Lustre PS1 · 2026-09-18 09:49

(minor) Adding the bit changes NODEMAP_RAISE_PRIV_NONE from 0xfffffe00 to 0xfffffc00, and that value is stored on disk rather than being a pure in-memory sentinel.

nodemap_idx_cluster_roles_modify() writes a NODEMAP_CLUSTER_ROLES record whenever rbac != NODEMAP_RBAC_ALL, and the record carries ncrr_privs = nmf_raise_privs, i.e. the old 0xfffffe00 for a nodemap whose raise_privs is 'none'. nodemap_cluster_roles_helper() then assigns it back verbatim with no masking.

So after an upgrade, a nodemap that merely had a non-default rbac comes back with bit 0x200 set, and

    if ((nodemap->nm_parent_nm->nmf_raise_privs & priv) && priv != NODEMAP_RAISE_PRIV_RBAC)
            return true;

short-circuits the new NODEMAP_RAISE_PRIV_VERSION check, so a dynamic child can override a parent policy that was meant to be locked. raise_privs would also read back as 'version_policy' for a nodemap the admin set to 'none'.

This encoding predates the patch (CAPS and DENY_MNT had the same effect when they were added) so it probably belongs in its own change rather than blocking this one - but is it worth masking the loaded value to the known bits, or is the intent that unknown bits stay raised?

Andreas Dilger PS1 · 2026-09-19 08:08

@sbuisson@ddn.com or @mvef@whamcloud.com could you comment on the intent here?  If this patch is introducing the mismatch between the flags then it makes sense to fix it here as well instead of separate patch.

Sebastien Buisson PS1 · 2026-09-25 15:55

`NODEMAP_RAISE_PRIV_NONE` was made on the same model as `NODEMAP_RBAC_NONE`, but the sematic is quite the opposite actually. Rbac roles are all present by default, and we remove some of them to lower client capabilities. 'Raise privs' are all absent by default, and we add some of them to grant privileges. So I think we should set `ncrr_privs` to 0 when `nmf_raise_privs` is `NODEMAP_RAISE_PRIV_NONE`. That would avoid the problem mentioned here when we upgrade a system after new privs have been added, and that would be a simple fix. I could push a quick patch, so that it is fixed right now and does not wait for this bigger patch.

reply PS1 2026-09-17 20:43 patrick bot mdt_handler.c:7593 — **[AI review - opus]**

patrick bot PS1 · 2026-09-17 20:43

**[AI review - opus]**

(suggestion) Refusing on the reconnect path means a client that is in the recovery set and is now blocked can never come back: ptlrpc_connect_interpret() sets IMPF_DEACTIVE on -EACCES, so the target waits on it for the whole recovery window before evicting it. nmf_deny_mount avoids this by checking in mdt_get_root() rather than at connect.

Is it worth releasing the recovery slot (or only enforcing on an initial connect) so a version policy change does not stretch out the next recovery?

Andreas Dilger PS1 · 2026-09-18 04:21

With numeric ranges this would be possible - the child nodemaps could be allowed to increase the deny range or decrease the accept range.

reply PS1 2026-09-17 20:43 patrick bot nodemap_handler.c:3935 — **[AI review - opus]**

patrick bot PS1 · 2026-09-17 20:43

**[AI review - opus]**

(defect) Deleting the record when the policy goes back to allow:* assumes the nodemap's inherited value is also the default, but nodemap_inherit_properties() copies nmf_version_policy/nm_version_glob from the 'default' nodemap for every non-dynamic nodemap.

So with a policy set on 'default':

    lctl nodemap_set_version_policy --name default --policy block --glob '2.15.*'
    lctl nodemap_add c0                 # c0 inherits block:2.15.* in memory, no record written
    lctl nodemap_set_version_policy --name c0 --policy allow

the last command takes the is_default() branch, deletes a key that was never inserted, and leaves no NODEMAP_CLUSTER_VERSION record for c0. nodemap_idx_delete_batch() swallows -ENOENT and still calls nodemap_inc_version(), so the other servers re-read the config and rebuild c0 through nodemap_cluster_rec_helper() -> nodemap_create() -> nodemap_inherit_properties(), which hands it back block:2.15.*. The MGS reports allow:* for c0 while the MDTs and OSTs keep refusing those clients, and the MGS agrees again after a restart.

nm_sepol is deliberately not inherited from 'default' (see the src->nm_id == LUSTRE_NODEMAP_DEFAULT_ID case); would the same treatment work here, or should the record be written whenever the policy differs from what would be inherited?

reply PS1 2026-09-18 04:21 Andreas Dilger nodemap_handler.c:4002 — we don't really need such a verbose comment

Andreas Dilger PS1 · 2026-09-18 04:21

we don't really need such a verbose comment

reply PS1 2026-09-17 20:20 Timothy Day nodemap_handler.c:4007 — This might be a bit spammy.

Timothy Day PS1 · 2026-09-17 20:20

This might be a bit spammy.

Timothy Day PS1 · 2026-09-17 20:21

There's probably a smarter, more concise way to test this using some kind of fake client.

Andreas Dilger PS1 · 2026-09-18 04:21

One option would be to have a fail_loc plus fail_val that sets the client version sent to the servers at connect time.  The 32-bit fail_val can store the encoded version so it would be possible to report any arbitrary version code.  This doesn't allow emulating ocd_connect_flags, but I don't think that matters in this case.

owner Timothy Day · uploader Timothy Day · PS1 uploaded 45d ago · NEW · open in Gerrit ↗

commit message

LU-17217 nodemap: add client version policy

Let admins control which Lustre client versions may connect,
using nodemap rather than a separate policy framework. Each
nodemap gets a version_policy property, shown as POLICY:GLOB,
where GLOB is matched against the client "major.minor.patch"
version when it connects to an MDT or OST:

  allow:GLOB       only clients matching GLOB may connect
  block:GLOB       clients matching GLOB are refused

The default is allow:*, which accepts every client. A refused
client gets -EACCES at connect time, so the mount fails and the
import is deactivated rather than retried forever. Server to
server connections (MDS, MDS_MDS and lightweight) are never
checked, so a policy cannot partition the servers.

The policy is stored as a new NODEMAP_CLUSTER_VERSION record in
the nodemap config, so it is distributed to all servers like the
other nodemap properties. The record is only written when a
policy is set, so older servers never see it. Dynamic nodemaps
inherit the policy from their parent and may only change it if
the parent has no policy, or raises the new version_policy
privilege via child_raise_privileges.

Add "lctl nodemap_set_version_policy --name NM --policy
{allow|block} [--glob GLOB]", a man page, and sanity-sec
tests 86a (mount enforcement) and 86b (dynamic nodemap
privilege).

Test-Parameters: trivial testlist=sanity-sec env=ONLY="86a 86b"
Signed-off-by: Timothy Day <timday@thelustrecollective.com>
Change-Id: I380802f736a87dd29ad746efff9d3c386a6a6964

63627 b2_15 J !B ✓T ✓R +2/1

LU-19539 osd-zfs: use osd_dmu_write() wrapper for xattrs

cc Olaf Faaland ·

LTS15 XS +1 −1

23d

janitor run: 1 failing config(s), none unique to this patch — janitor results

build #126459 SUCCESS · tests all Maloo sessions

reviews: Tony Hutter +1 2026-06-12 19:52 · Alex Zhuravlev +1 2026-06-12 08:45

no vote yet: Oleg Drokin

patchsets: PS1 01-28 PS2 05-11 PS3 06-12

owner Olaf Faaland · uploader Olaf Faaland · PS3 uploaded 143d ago · NEW · backport · open in Gerrit ↗

commit message

LU-19539 osd-zfs: use osd_dmu_write() wrapper for xattrs

When setting xattrs use the osd_dmu_write() wrapper which
internally calls the _by_dnode() version of dmu_write().  This
both avoids an extra lookup and hold on the dnode.  Furthermore,
it removes the osd-zfs dependency on the dmu_write() symbol
which will add an flags argument in OpenZFS 2.5.

Lustre-change: https://review.whamcloud.com/62101
Lustre-commit: 84f5994257b16910849f4ab3b27b3c49a45b1adc

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Change-Id: I30ba7abbcf0eefb3dee01b4061baaace2d971842
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>

58978 master J !B ✗T –R ✗

LU-13371 api: add llapi_getdirstripe function

cc Patrick Farrell · blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked; build failure; Patrick Farrell voted -1

needs rebaseCR -1 M +230 −7

59d

janitor run: build failed — no tests ran — janitor results

build #112925 FAILURE

reviews: Patrick Farrell -1 2025-04-29 20:01

no vote yet: Andreas Dilger

patchsets: PS1 04-25 PS2 04-25 PS3 04-27

owner Patrick Farrell · uploader Patrick Farrell · PS3 uploaded 554d ago · NEW · open in Gerrit ↗

commit message

LU-13371 api: add llapi_getdirstripe function

Add a new llapi_getdirstripe() function to the Lustre API to
retrieve directory striping information. This function is
similar to llapi_file_get_stripe() but specifically for
directory striping information.

This function will make it easier for applications to
retrieve directory striping information without having to
directly use the LL_IOC_LMV_GETSTRIPE ioctl.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I41da7e83c6d67bc182d008524d08a9088405cc05

61956 master J !B ✓T ✓R -2/2−

LU-19264 ldlm: server notify lock clients to reclaim LRU locks

cc Qian Yingjin · blocked on the owner: needs rebase — checkpatch: cannot be cherry-picked

needs rebase stalled 132d 19 unresolved M +240 −8

132d

janitor run: 1 failing config(s), none unique to this patch — janitor results

5 optional test failure(s) — informational

build #117706 SUCCESS · tests all Maloo sessions

reviews: Andreas Dilger -1 2025-10-24 07:56

no vote yet: kg.xu

patchsets: PS1 10-22 PS2 10-22 PS3 10-22

threads: 19 need your reply · 0 waiting on others · 0 own notes · 0 bot

reply PS3 2025-10-24 07:54 Andreas Dilger COMMIT_MSG:16 — This should be reworded to make it clear that these issues only exist in the old version of the code where the lock server is making the dec

Andreas Dilger PS3 · 2025-10-24 07:54

This should be reworded to make it clear that these issues only exist in the old version of the code where the lock server is making the decision for which locks to cancel. Something like:
```
If the lock server selects specific locks to cancel,
this can lead ...
```

reply PS3 2025-10-24 07:54 Andreas Dilger COMMIT_MSG:17 — `... in active use on the client ...`

Andreas Dilger PS3 · 2025-10-24 07:54

`... in active use on the client ...`

reply PS3 2025-10-24 07:54 Andreas Dilger COMMIT_MSG:18 — `Server-selected lock revocation ...`

Andreas Dilger PS3 · 2025-10-24 07:54

`Server-selected lock revocation ...`

reply PS3 2025-10-24 07:54 Andreas Dilger COMMIT_MSG:24 — Please describe the mechanism implemented.

Andreas Dilger PS3 · 2025-10-24 07:54

Please describe the mechanism implemented.

reply PS3 2025-11-11 10:39 Andreas Dilger patchset comment — > IMHO I'd rather keep the CLV/SLV mechnism since it DO give the clients

Andreas Dilger PS3 · 2025-11-10 09:41

If we are moving forward with a new server-side notification mechanism to cancel client locks (instead of the CLV/SLV exchange), then we should consider to disable this mechanism completely once all clients support the new mechanism.

The old mechanism generates a non-trivial amount of overhead on the client to update the CLV for each namespace (it was originally done on 1-second intervals for *all* namespaces on the client, but had to be scaled down to one namespace per 1-second interval to avoid impacting client performance).

If the server starts sending the "magic cancel" then the client could stop calculating the CLV and the server can stop sending it as well.

kg.xu PS3 · 2025-11-10 14:02

IMHO I'd rather keep the *CLV/SLV* mechnism since it `DO` give the clients the choice to select the locks that they prefer to return. With "magic cancel", clients still need to figure out which locks they should cancel, it's just another form of *CLV*?
The issue to the SLV is it's not aligned with CLV in some degree right now, I'm on it and will upload a patch hoping to fix it(with CLV intact).
Still, we need reclaim notification here as the safety net that should be relied upon once *CLV/SLV* broken.

Andreas Dilger PS3 · 2025-11-11 10:39

> IMHO I'd rather keep the CLV/SLV mechnism since it DO give the clients
 > the choice to select the locks that they prefer to return.
 > With "magic cancel", clients still need to figure out which locks
 > they should cancel, it's just another form of CLV?

Sorry, I don't understand your comment.  In both cases, CLV/SLV and "magic cancel" (or set_info), the client needs to decide *which* locks to cancel, while the server decides *how many* to cancel (though I guess the client could still use this as a "hint" and cancel more or fewer locks if needed).  We *want* the client to decide this, since it has more information about the current lock usage than the server.

One benefit of "magic cancel" is that it is a simpler mechanism to understand, a clear notification to cancel "some" locks.  The CLV/SLV mechanism has (AFAIK) been at least partly broken for many years and nobody can ever get it working.  The main benefit of CLV/SLV (AFAIK) is that the client is essentially notified of a "rate" to cancel locks, so the server does not need to continually send RPCs to each client.  I guess the same could be done with "magic cancel", at which point they are nearly the same thing.

It would definitely be awesome of you could fix this code to be more robust.

reply PS3 2025-10-30 16:09 Andreas Dilger ldlm_lockd.c:2447 — (defect) I would prefer that this check all lock resource fields are {0,0,0,0} so that we can have other "special-case" requests in the futu

Andreas Dilger PS3 · 2025-10-30 16:09

(defect) I would prefer that this check all lock resource fields are {0,0,0,0} so that we can have other "special-case" requests in the future.

reply PS3 2025-10-30 16:09 Andreas Dilger ldlm_lockd.c:2454 — Presumably this CERROR() is for debugging only? There may be reason the client cannot cancel these locks (eg. it just cancelled locks befor

Andreas Dilger PS3 · 2025-10-24 07:54

(style) spaces around that '='

(typo) "to" is not needed here

Andreas Dilger PS3 · 2025-10-30 16:09

Presumably this CERROR() is for debugging only?  There may be reason the client cannot cancel these locks (eg. it just cancelled locks before this RPC was received).

reply PS3 2025-10-24 07:54 Andreas Dilger ldlm_lockd.c:2456 — Should this send the reply *before* shrinking the LRU (to avoid RPC timeout and resend), or are the cancelled locks added into this reply RP

Andreas Dilger PS3 · 2025-10-24 07:54

Should this send the reply *before* shrinking the LRU (to avoid RPC timeout and resend), or are the cancelled locks added into this reply RPC?

reply PS3 2025-10-28 09:17 kg.xu ldlm_reclaim.c:240 — (typo) across

kg.xu PS3 · 2025-10-28 09:17

(typo) across
(style) comment part: @params ...

reply PS3 2025-10-30 16:09 Andreas Dilger ldlm_reclaim.c:241 — (minor) it isn't clear that the number of clients holding locks is needed? Clients holding no (or few) locks do not need to be sent any cal

Andreas Dilger PS3 · 2025-10-30 16:09

(minor) it isn't clear that the number of clients holding locks is needed?  Clients holding no (or few) locks do not need to be sent any callback. The important thing is the ratio of each export's locks out of the total number of granted locks.

reply PS3 2025-11-10 14:02 kg.xu ldlm_reclaim.c:254 — I was taking some extreme case here, say, there might be 10 clients having 90k locks each, in the case of 5% cancel rate, per client needs t

kg.xu PS3 · 2025-11-10 06:04

Selecting users to return locks is a bit tricky, a threshold derived from `total_to_cancel / total_count` should be picked with caution,
- If small, then it takes a heavy user (20,000 locks) much more iterations to return all its locks compared with a moderate user(1,000 locks?), is that fair?
- If large, then in a environment where locks are distributed relatively evenly, do we still be able to collect back enough locks?

Qian Yingjin PS3 · 2025-11-10 07:01

Yes, in the second patch:https://review.whamcloud.com/c/fs/lustre-release/+/62146/2
It will send more information about server lock load such as total granted lock, total granted lock of this NS, memory pressure (in Percent: ldlm_lock_threshold * 100 / ldlm_lock_limit), and minimal lock cancel count and whether it is a force...

After receive these information, the client can decide lock revoking accordingly.

Andreas Dilger PS3 · 2025-11-10 09:41

Maybe I'm misunderstanding here, but this seems straight forward to me.

For example, if the server has 1M locks and it wants to cancel 50k cancels/1M locks = 5 cancels/100 locks = 5% to reduce memory pressure.  There are two cases (but they work out the same): 

If the lock allocation is relatively uniform (e.g. there are 1000 clients and they each have 1000 locks), then the same number of locks will be requested to be cancelled from each client (i.e. 1000 locks/client * 5% = 50 locks/client).  

If the lock allocation is non-uniform (e.g. 1000 clients, but 100 clients have 9100 locks each = 910k locks, and 900 clients have 100 locks each = 90k locks) then the 100 clients with a larger fraction of the total locks will also be asked to cancel more locks (i.e. 910k * 5% = 45 or 46) and the 900 clients with fewer locks will be asked to cancel a smaller number (i.e. 100 * 5% = 5 locks).

If it gets to the extreme where the server needs to cancel locks, but quantization makes the number of locks to cancel below 1 for all of the clients (e.g. cancel 500 locks from 1000 clients that all have the same number of locks), then either the server picks the clients with the most locks to cancel 1 each, or randomly picks 500 clients to cancel one, and then next time the other 500 clients will have 1 more lock and should be selected to cancel one lock.

kg.xu PS3 · 2025-11-10 14:02

I was taking some extreme case here, say, there might be 10 clients having 90k locks each, in the case of 5% cancel rate, per client needs to return 4.5k locks. 

Here we capped the *count* to *remains*(512 at most) per round, which means it needs 4.5k/512 ~= 9 iterations to have heavy clients return all their locks back, on the contrary, it takes clients with fewer locks 1~2 rounds to return their locks. That's why I mention `unfair` here. Put it another way, the more locks a client holds, the less locks he/she may need to return in the end.

Besides, RPC here is `no_delay`, it might cause traffic spike?

reply PS3 2025-10-24 07:54 Andreas Dilger ldlm_reclaim.c:257 — I think yes, there should be some small number of locks left on the client? Otherwise, the client locks may be thrashing on a few locks tha

Andreas Dilger PS3 · 2025-10-24 07:54

I think yes, there should be some small number of locks left on the client?  Otherwise, the client locks may be thrashing on a few locks that the client needs (eg. user home directory), even though some other client may have thousands of locks.

owner Qian Yingjin · uploader Qian Yingjin · PS3 uploaded 376d ago · NEW · open in Gerrit ↗

commit message

LU-19264 ldlm: server notify lock clients to reclaim LRU locks

In large-scale Lustre cluster with a number of clients, the server
side lock namespace must manage all locks granted to clients. And
these locks are cached in server side memory. Due to the limited
server memory capacity, the server must reclaim locks when it is
under memory pressure. However, when lock reclaim is triggered,
the LRU or frequency of locks cached on the server lock namespace
may not reflect recent used accurately.
This can lead to premature reclaim of locks that are actually
still in active use or may be reused in the near future by the
client. The lock revocation will drop the cache data or metadata
protected by the lock on the client. This may result in
unnecessary network traffic, re-reading the data (such as
memory resident library files) dropped from cache wrongly and
degraded system performance.

To solve this problem, this patch implements a mechanism to let
the lock server to notify clients, which have much accurate and
realtime information about lock usage, to make better decision for
lock reclaim on client side.

Signed-off-by: Yingjin Qian <qian@ddn.com>
Change-Id: I55a7d5876e5dfbdfbfc84d073e86dc0156f886cd

66539 b2_16 B ✗T –R +2/1

LU-19743 tests: Allow multiple FID formats on debugfs tests

cc Andreas Dilger · blocked on the owner: build failure

stalled 137d M +120 −108

137d

build #126706 FAILURE

reviews: Alex Deiter +1 2026-06-10 07:43 · Li Dongyang +1 2026-06-10 00:12

patchsets: PS1 06-09 PS2 06-09

owner Andreas Dilger · uploader Andreas Dilger · PS2 uploaded 146d ago · NEW · backport · open in Gerrit ↗

commit message

LU-19743 tests: Allow multiple FID formats on debugfs tests

Since new debugfs FID output will not output brace in the future,
related tests must support both DFID and DFID_NOBRACE format FIDs
for test interop compatibility.

Lustre-change: https://review.whamcloud.com/63195
Lustre-commit: e1f272796c51f5335b03f5d60a626ef90ec4d1d7

Test-Parameters: trivial
Signed-off-by: Aurélien Cedeyn <acedeyn@ddn.com>
Change-Id: I9c78b051091307eba4760d3220c0caf3d0056efa
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
Reviewed-by: Alex Deiter <adeiter@ddn.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>

63626 b2_15 J ✗6B ✓T ✓R +2/1

LU-19193 osd-zfs: fix blocksize prediction

cc Olaf Faaland ·

stalled 138d M +73 −11

138d

janitor flagged test failures unique to this patch — superseded (tests have since passed) — full janitor results

unique failing test	history
replay-single1@zfs+DNE:test_29	seen in 1 other review
replay-single1@zfs+DNE:test_30	seen in 1 other review
replay-single1@zfs+DNE:test_31	seen in 1 other review
replay-single1@zfs+DNE:test_32	seen in 1 other review
replay-single1@zfs+DNE:test_33a	seen in 1 other review
replay-single2@zfs:test_132a	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews

build #126457 SUCCESS · tests all Maloo sessions

reviews: Gian-Carlo DeFazio +1 2026-06-16 21:14 · Alexey Lyashkov +1 2026-06-12 08:41

no vote yet: Andreas Dilger, Alex Zhuravlev, Brian Behlendorf, Oleg Drokin

patchsets: PS1 01-28 PS2 06-12

owner Olaf Faaland · uploader Olaf Faaland · PS2 uploaded 143d ago · NEW · backport · open in Gerrit ↗

commit message

LU-19193 osd-zfs: fix blocksize prediction

1) non-zero offsets shouldn't this logic to grow blocksize
2) appending writes should increase blocksize larger than
   the chunk being written

and enable sanity/312 back

Lustre-change: https://review.whamcloud.com/61103
Lustre-commit: d3dde0b63b65fc2a9fbd68534421f7bbb8ba8936

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iada44aba64ea07862421f7ee452672794732ee32
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

65822 b2_15 J ✓B ✓T ✓R +2/1

LU-19435 osd-zfs: support for 2.4

cc Olaf Faaland ·

stalled 143d M +78 −28

143d

build #126458 SUCCESS · tests all Maloo sessions

reviews: Shaun Tancheff +1 2026-06-12 08:43 · Jian Yu +1 2026-06-12 08:43

no vote yet: Akash B

patchsets: PS1 05-11 PS2 06-12

owner Olaf Faaland · uploader Olaf Faaland · PS2 uploaded 143d ago · NEW · backport · open in Gerrit ↗

commit message

LU-19435 osd-zfs: support for 2.4

zfs commit zfs-2.3.99-340-g734eba251
  Wire O_DIRECT also to Uncached I/O (#17218)
dmu_write_by_dnode() now has flags argument
dmu_assign_arcbuf_by_dbuf() now has flags argument

zfs commit zfs-2.3.99-373-gac2e57952
  dmu_tx: make DMU_TX_* flags an enum
DMU_TX_WAIT may be an enum

Lustre-change: https://review.whamcloud.com/61589
Lustre-commit: bc3fe9e4bb1524ac64eff5529b4ba15675cf2e3b

Test-Parameters: trivial
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I2b1694b39f4d4a83a178d70937a2e48019ed42a0
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Akash B <akash-b@hpe.com>

52235 master J ✗46B ✗T –R 0/2

LU-13814 osc: rename osc_prep_transfer_page

cc Patrick Farrell · blocked on the owner: janitor: test failures unique to this patch; build failure

46 unique failures ★ stalled 550d S +19 −20

550d

janitor flagged test failures unique to this patch — full janitor results

unique failing test	history
recovery-small@zfs:test_155	NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews
sanity1@zfs:test_56x	seen in 38 other reviews
sanity1@zfs:test_56xB	seen in 6 other reviews
sanity1@zfs:test_56xa	seen in 38 other reviews
sanity1@zfs:test_56xab	seen in 3 other reviews
sanity1@zfs:test_56xc	seen in 34 other reviews
sanity1@zfs:test_56ej	seen in 6 other reviews
sanity2@zfs:test_119e	seen in 15 other reviews
sanity2@zfs:test_119f	seen in 14 other reviews
sanity2@zfs:test_119g	seen in 14 other reviews
sanity2@zfs:test_119h	seen in 14 other reviews
sanity2@zfs:test_119p	seen in 3 other reviews
sanity2@zfs:test_119q	seen in 3 other reviews
sanity2@zfs:test_398o	seen in 19 other reviews
sanity2@zfs:test_398s	seen in 3 other reviews
sanity-flr@ldiskfs+DNE:test_0g	seen in 31 other reviews
sanity-flr@ldiskfs+DNE:test_0h	seen in 33 other reviews
sanity-flr@ldiskfs+DNE:test_0j	seen in 32 other reviews
sanity-flr@ldiskfs+DNE:test_36d	seen in 31 other reviews
sanity-flr@ldiskfs+DNE:test_37	seen in 31 other reviews
sanity-flr@ldiskfs+DNE:test_38	seen in 25 other reviews
sanity-flr@ldiskfs+DNE:test_44b	seen in 4 other reviews
sanity-flr@ldiskfs+DNE:test_50a	seen in 35 other reviews
sanity-flr@ldiskfs+DNE:test_50b	seen in 38 other reviews
sanity-flr@ldiskfs+DNE:test_50d	seen in 34 other reviews
sanity-flr@ldiskfs+DNE:test_61a	seen in 38 other reviews
sanity-flr@ldiskfs+DNE:test_61c	seen in 29 other reviews
sanity-flr@ldiskfs+DNE:test_70a	seen in 70 other reviews
sanity-flr@ldiskfs+DNE:test_200a	seen in 30 other reviews
sanity-flr@ldiskfs+DNE:test_200b	seen in 29 other reviews
sanity-flr@zfs:test_0g	seen in 32 other reviews
sanity-flr@zfs:test_0h	seen in 34 other reviews
sanity-flr@zfs:test_0j	seen in 33 other reviews
sanity-flr@zfs:test_36d	seen in 31 other reviews
sanity-flr@zfs:test_37	seen in 35 other reviews
sanity-flr@zfs:test_38	seen in 31 other reviews
sanity-flr@zfs:test_44b	seen in 3 other reviews
sanity-flr@zfs:test_61a	seen in 39 other reviews
sanity-flr@zfs:test_61c	seen in 32 other reviews
sanity-flr@zfs:test_70a	seen in 59 other reviews
sanity-flr@zfs:test_200a	seen in 31 other reviews
sanity-flr@zfs:test_200b	seen in 32 other reviews
sanity-hsm@ldiskfs+DNE:test_607b	seen in 4 other reviews
sanity-hsm@zfs:test_607b	seen in 4 other reviews
sanityn@ldiskfs+DNE:test_16j	seen in 27 other reviews
sanityn@zfs:test_16j	seen in 30 other reviews

build #112977 FAILURE

no vote yet: Andreas Dilger

patchsets:

+37 earlier

PS1 09-02 PS2 09-03 PS3 09-03 PS4 09-03 PS5 09-03 PS6 09-03 PS7 09-03 PS8 09-05 PS9 09-07 PS10 09-09 PS11 09-09 PS12 09-10 PS13 09-10 PS14 09-10 PS15 09-10 PS16 09-10 PS17 09-10 PS18 09-10 PS19 09-10 PS20 09-10 PS21 09-10 PS22 09-10 PS23 09-12 PS24 09-14 PS25 09-14 PS26 09-15 PS27 09-15 PS28 09-16 PS29 09-24 PS30 09-25 PS31 09-26 PS32 09-26 PS33 10-05 PS34 10-05 PS35 10-20 PS36 10-22 PS37 10-22

PS38 12-26 PS39 02-27 PS40 10-28 PS41 11-03 PS42 05-01

owner Patrick Farrell · uploader Patrick Farrell · PS42 uploaded 550d ago · NEW · open in Gerrit ↗

commit message

LU-13814 osc: rename osc_prep_transfer_page

osc_transfer_page_init is a better match for osc_page_init.

Test-Parameters: trivial
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id12eec932eb4aab00139f08532b218c12e6ffba4

Merged — last 30 days

(5)

69824 master J –B ✓T ✓R +4/2

LU-20212 obdclass: don't block no-delay statfs on others

mine

pt_ecro M +46 −5

reviews: Andreas Dilger +1 2026-11-01 02:14 · Maximilian Dilger +1 2026-11-01 02:14 · Oleg Drokin +2 2026-11-01 02:14

patchsets: PS1 10-19 PS2 10-22 PS3 10-22

tags: pt_ecro

owner Marc Vef · uploader Oleg Drokin · PS4 uploaded 1d ago · MERGED · open in Gerrit ↗

commit message

LU-20212 obdclass: don't block no-delay statfs on others

"lfs df -l" and reading osc.*.kbytesfree hung for as long as an OST
was down whenever a plain "lfs df" was already waiting for that OST.
Lazystatfs df hung the same way behind a plain "lfs df" waiting for a
down MDT.

obd_statfs() serializes statfs RPCs per device with obd_dev_mutex so that many threads do not send the same RPC at once. A blocking
caller holds the mutex while its RPC waits for recovery, so a no-delay
caller waited on the mutex instead of failing fast.

Serialize no-delay callers on their own obd_statfs_nodelay_mutex.
Their RPCs do not wait for recovery, so they only wait for each other
briefly, and each device has at most one blocking and one no-delay
obd_statfs() RPC in flight.

Add recovery-small test_116 to check that "lfs df -l" and kbytesfree
return while "lfs df" waits for a stopped OST.

Fixes: 1c41a6ac390b ("LU-12368 obdclass: don't send multiple statfs RPCs")
Test-Parameters: testlist=recovery-small env=ONLY=116,ONLY_REPEAT=10
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I950ca70a4e1740e5bdf06151916a7ae86931da3d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/69824
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
Reviewed-by: Maximilian Dilger <mdilger@whamcloud.com>

68932 master J –B ✓T ✓R +4/2

LU-20718 lod: scrub stale EC fields from old layouts

mine

pt_ecro218_review_priority218_review_blocker 10 unresolved M +141 −0

reviews: Keguang Xu +1 2026-11-01 02:09 · Andreas Dilger +1 2026-11-01 02:09 · Oleg Drokin +2 2026-11-01 02:09

patchsets: PS1 09-15 PS2 09-15 PS3 10-09 PS4 10-16 Andreas Dilger

tags: pt_ecro 218_review_priority 218_review_blocker

owner Marc Vef · uploader Oleg Drokin · PS5 uploaded 1d ago · MERGED · open in Gerrit ↗

commit message

LU-20718 lod: scrub stale EC fields from old layouts

Before EC was introduced, servers wrote lcme_timestamp only for
NOSYNC components, memset only the layout header and reused the EA
buffer, so every other component was persisted with stale bytes in
what EC now reads as lcme_time_and_id and the EC stripe counts.  An
MDS upgraded from such a server hands those bytes to every client,
and trusts a stale stripe count on an uninstantiated component: it
instantiates it as an EC component or fails the write with -EINVAL.

Zero the EA before packing it.  Wherever a layout is parsed, clear a
timestamp outside a plausible range, and the link id and stripe
counts of a data component that no parity component, which such a
server never wrote, links to; the layout is clean the next time it
is written.  Apply the same to the EA returned by lod_xattr_get().

Fixes: 472e0f27d411 ("LU-19545 ec: tight binding parity mirror to data mirror")
Test-Parameters: testlist=sanity-pfl clientversion=2.17
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools
Change-Id: Id823aeb4c95c3f0ca2e318f982ff7cd0147eff01
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/68932
Reviewed-by: Keguang Xu <kxu@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

68747 master J –B ✓T ✓R +7/2

LU-20718 ec: ignore stale EC fields from old servers

mine

218_review_priority218_review_blockerpt_ecropw_smallfix 1 unresolved M +135 −8

reviews: Praveen +1 2026-11-01 02:09 · Ronnie Sahlberg +1 2026-11-01 02:09 · Andreas Dilger +1 2026-11-01 02:09 · Maximilian Dilger +1 2026-11-01 02:09 · Oleg Drokin +2 2026-11-01 02:09 · Timothy Day +1 2026-11-01 02:09

patchsets:

+9 earlier

PS1 09-09 PS2 09-09 PS3 09-15 PS4 09-15 PS5 09-16 PS6 09-16 PS7 10-09 patrick bot PS8 10-09 patrick bot PS9 10-13 patrick bot

PS10 10-16 Andreas Dilger PS11 10-16 Andreas Dilger PS12 10-18 patrick bot PS13 10-18 patrick bot PS14 10-19

tags: 218_review_priority 218_review_blocker pt_ecro pw_smallfix

owner Marc Vef · uploader Oleg Drokin · PS15 uploaded 1d ago · MERGED · open in Gerrit ↗

commit message

LU-20718 ec: ignore stale EC fields from old servers

Before EC was introduced, servers wrote lcme_timestamp only for
NOSYNC components, memset only the layout header and reused the EA
buffer, so every other component was persisted with stale bytes in
what EC now reads as lcme_time_and_id and the EC stripe counts.  A
stale link id makes llapi_layout_merge() flag an on-disk component
LCME_FL_IS_LINK_ID and the layout fails sanity ("lfs setstripe
--component-add": "Wrong flags", sanity-pfl test_18); a stale
timestamp prints as a date in the year 56546 (sanity-pfl test_19e).

Read the timestamp only for NOSYNC components or EC layouts, and a
data component's link id and stripe counts only while a parity
component, which such a server never wrote, links to it.  Print the
link id and timestamp only for layouts with a parity component, and
the timestamp also for NOSYNC components.

The kernel client has the same problem: with a stale
lcme_dstripe_count a plain mirror looks like EC data,
lov_io_mirror_init() sets ci_cross_ec, and when reading the mirror
fails the client switches to a CIT_EC_RD read that fails with "no
updated parity component to restore data: rc = -5" (sanity-flr
test_34b against 2.17 servers).  Apply the same rule in
lsm_unpackmd_comp_md_v1().

The client EC stripe counts share storage with a foreign
component's HSM flags, so neither unpack them for a foreign
component nor pack its flags back as stripe counts.

Fixes: 472e0f27d411 ("LU-19545 ec: tight binding parity mirror to data mirror")
Test-Parameters: testlist=sanity-pfl serverversion=2.17
Test-Parameters: testlist=sanity-flr serverversion=2.17
Test-Parameters: testlist=sanity-pcc
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Assisted-by: ClaudeCode:claude-opus-5.5 ltvm,llm_code_and_review_tools
Assisted-by: Codex:gpt-6.1-sol
Change-Id: If01e51b47d267e9da0512d978cae998fabc2301f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/68747
Reviewed-by: Praveen <praveen.solanki@oracle.com>
Reviewed-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@thelustrecollective.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Maximilian Dilger <mdilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

69815 master J –B ✓T ✓R +4/2

LU-20992 tests: skip sanity-pfl/28 for old MDS

mine

218_review_blocker218_review_prioritytestfix S +6 −4

reviews: Andreas Dilger +1 2026-10-28 16:02 · Sebastien Buisson +1 2026-10-28 16:02 · Oleg Drokin +2 2026-10-28 16:02

patchsets: PS1 10-19 PS2 10-19

tags: 218_review_blocker 218_review_priority testfix

owner Marc Vef · uploader Oleg Drokin · PS3 uploaded 5d ago · MERGED · open in Gerrit ↗

commit message

LU-20992 tests: skip sanity-pfl/28 for old MDS

sanity-pfl test_28 fails in every interop session against 2.15,
2.16 and 2.17.0 servers, including lustre-master's own full runs:
"lfs migrate: cannot get group lock: Invalid argument (22)".  The
test needs the lod_ost_alloc_specific() fallback that landed with
it in v2_17_53-25-gd2aaf9d4c4; an older MDS still rejects the stale
stripe offset of the uninitialized component.

Skip the test when the MDS predates that change.

Fixes: d2aaf9d4c4b8 ("LU-19519 lod: allow migration of PFL with invalid OST")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-pfl env=ONLY=28
Test-Parameters: testlist=sanity-pfl serverversion=2.17 env=ONLY=28
Assisted-by: ClaudeCode:Opus-5.5 llm_code_and_review_tools
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I53a6c6460d34db3967182f9f83df7ef763132c28
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/69815
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>

65580 master J –B ✓T ✓R +4/2

LU-20203 utils: add lctl nodemap_info --list-properties

mine

patrick-bot M +76 −24

11d

reviews: Sebastien Buisson +1 2026-10-21 17:16 · Oleg Drokin +2 2026-10-21 17:16 · Timothy Day +1 2026-10-21 17:16

patchsets:

+2 earlier

PS1 04-29 PS2 04-29

PS3 08-25 PS4 09-25 patrick bot PS5 09-25 patrick bot PS6 09-25 patrick bot PS7 10-01 patrick bot

your previous vote: -1 on PS2 (2026-04-29 21:14) — now at PS8

tags: patrick-bot

owner Marc Vef · uploader Oleg Drokin · PS8 uploaded 11d ago · MERGED · open in Gerrit ↗

commit message

LU-20203 utils: add lctl nodemap_info --list-properties

"lctl nodemap_info --list" printed the table of available nodemap
properties along with the global nodemap state and the list of defined
nodemaps.  The property descriptions are static reference material and
do not belong in the output of every list query, where a long list of
defined nodemaps pushes them off screen.

Move the property table to its own "--list-properties" option, which
prints the descriptions and nothing else and queries no nodemap state.
"--list" now prints only the global nodemap state and the defined
nodemaps.  "--list-properties" is mutually exclusive with "--list",
"--name" and "--property".

Also add the properties missing from the description table: banlist,
child_raise_privileges, enable_cap_mask, gssonly_identification,
identity and parent.

Mark properties unavailable for the default nodemap and
keep a reciprocal table reference at their definitions.

Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools
Assisted-by: ClaudeCode:claude-sonnet-5
Assisted-by: Codex:gpt-6.1-sol
Fixes: b8e3b427f478 ("LU-18715 utils: Extend lctl nodemap_info with property values")
Test-Parameters: trivial testlist=sanity-sec
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8747b49d1cdb31701e595497342180bf85707bec
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/65580
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@thelustrecollective.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>