LU-0000 tests: sanity-ec sanity check Check whether sanity-ec can complete properly. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: If5bfe7bf33b28f8afb58473b7590a11c453c0659
LU-20554 quota: preserve partial OST usage
lov_quotactl() returns -ENETDOWN after summing quota usage from
reachable OSTs when an expected target is inactive. quotactl_ioctl()
does not copy this subtotal for -ENETDOWN, so lfs quota reports only
the MDT usage.
Copy dqb_curspace for -ENETDOWN, but leave QIF_SPACE clear. This
allows lfs quota to report usage from the reachable OSTs and MDTs
while retaining brackets to mark the result as incomplete.
Add sanity-quota test 16c to verify that usage from reachable OSTs is
included while another expected OST is unavailable.
Fixes: 001b8dbfacb7 ("LU-3963 cleanup: C89 and build cleanups")
Test-Parameters: testlist=sanity-quota env=ONLY=16c
Signed-off-by: Sergey Noskov <sergey.noskov.sn@gmail.com>
Change-Id: I17252ef34ff32dbdd0561c1570ce66f31cacfb87
| unique failing test | history |
|---|---|
| sanity-hsm@zfs:test_254b | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 10.1 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
The code doesn't seem to match this. gssiam_get_nodemap() calls strlen(gssiam_info->lii_desc.lid_principal) unconditionally, there is no -EINVAL return when the principal is missing. Was that part dropped when the patch was refreshed?
mdt_get_info() only adds
} else if (KEY_IS(KEY_GSSIAM_AUTHENTICATE)) {
rc = mdt_get_info_gssiam_authenticate(tsi);
and mdt_get_info_gssiam_authenticate() checks mdt_seq_site(mdt)->ss_node_id but never looks at tsi->tsi_exp. Is the peer check described here actually in this revision?
This adds a new on-wire struct (gssiam_auth_rep) exchanged between MDTs, so a mixed-version server pair can hit it. Would a Test-Parameters: line requesting interop coverage (serverversion=...) be worth adding here?
(minor) The tree is at 2.17.57 (LUSTRE-VERSION-GEN), and l_gssiam_upcall.8 added earlier in this series already says 2.17.57, so this and the `Added in commit v2_17_56` comment below look a version behind.
(minor) auth_permission ends up hashed twice - once through the new `options` parameter of lustre_gssiam_hash_key() and again via lustre_hash_fnv_1a_64_continue() just below. The new parameter is documented as the client's mount/auth options (lid_options is what tgt_gssiam_get_entry() passes), so feeding a permission mask into the same slot makes the two callers mean different things by it. Should the nodemap name pass the options here and keep the permission to the explicit continue()?
This kernel-doc block describes mdt_get_local_mdt0() with an @mdt argument and a pointer return, but it sits on mdt_gssiam_authenticate_remote(), which takes (lut, cache, entry) and returns int. Looks like a leftover from an earlier revision.
Same here - the block documents mdt_authenticate_gssiam() with @lut/@cache/@entry, but the function below is mdt_gssiam_projid_cb(mdt0_obd, path, projid, resolved_path). None of the documented parameters exist.
(style) These two blank lines, and the pair added inside mdt_prepare() around line 7261, don't look related to GSSIAM and aren't mentioned in the commit message - leftovers from an earlier revision? There are also three consecutive blank lines after mdt_gssiam_authenticate_remote() around line 3024.
(minor) lid is only partially filled in below - lid_principal and lid_projid keep whatever was on the stack. Nothing reads them on this path today, but tgt_gssiam_get_entry() passes the whole descriptor down to the cache ops, so `struct lustre_gssiam_desc lid = { 0 };` would be safer if the patch is refreshed.
gar_expire stays 0 on this path, and that seems to defeat the expired-token signalling on non-MDT0 targets.
tgt_gssiam_rpc_auth_callback() builds "expired:0:", tgt_gssiam_parse_downcall() sets ue_expire = ktime_get_seconds() + 0, and upcall_cache_downcall() keeps that value because it is non-zero. Back in upcall_cache_get_entry(), check_unlink_entry() sees `now < ue_expire` is false and sets UC_CACHE_EXPIRED, so tgt_gssiam_verify() fails the UC_CACHE_IS_VALID() test and returns -EACCES before it can reach
if (entry->u.gssiam.gd_auth_permission & GSSIAM_AUTH_EXPIRED)
GOTO(out_put, rc = -EAGAIN);
The client then gets GSS_S_DEFECTIVE_CREDENTIAL instead of GSS_S_CREDENTIALS_EXPIRED. Should this reply carry a small non-zero grace value so the entry is still valid when the caller inspects it?
(style) Three blank lines between the function and the next comment block; one is enough.
Can lid_principal be NULL here?
tgt_gssiam_parse_downcall() only fills gd_principal when the downcall carries a non-empty third field:
if (principal_str && *principal_str)
OBD_STRDUP(gssiam->gd_principal, principal_str);
but it sets gd_auth_permission from the first field regardless, so a downcall of "rw:600" or "rw:600:" leaves a valid RW entry with no principal. tgt_gssiam_info_alloc() then skips the strdup, tgt_gssiam_verify() returns 0 because the permission is neither DENY nor EXPIRED, and gssiam_handle_init() reaches this strlen() on a NULL pointer.
The local upcall path is still inert in this patch (no sptlrpc/gssiam_server downcall file yet), but once it is wired up a site-supplied auth helper that omits the principal would panic the MDS here.
LU-19921 target: verify GSSIAM tokens on the target When the target gets the init context RPC, it will call tgt_gssiam_verify() to verify the token through the external GSSIAM server by the l_gssiam_auth upcall. Once authenticated, the token is used to generate the security context, which is attached to the export. This patch includes: - tgt_gssiam_verify() and the cache interactions to validate tokens. - l_gssiam_auth utility to mock the external authentication. - tgt_gssiam_local_upcall() to execute the upcall. - tgt_gssiam_rpc_authenticate() and the callbacks to handle authentication via GET_INFO RPC. - Parsing of the downcall payload returned by the authentication server. - Attaching the upcall entry to the export, and destroying the gssiam upcall entry when all of its exports are disconnected. - ofd_dev.c: implement ofd_authenticate_gssiam() and initialize target GSSIAM callbacks in lut_tsi. - lustre_idl.h: define UAPI wire struct gssiam_auth_rep. - Documentation/man8: add l_gssiam_auth.8 and Makefile entry. Additionally, address several robustness and cleanup items: - Order l_gssiam_auth.8 alphabetically in man8 Makefile.am. - In gssiam_get_nodemap(), switch nodemap name derivation from raw token to the authenticated principal (lid_principal), and return -EINVAL if the principal is missing. - In lustre_gssiam.h, add auth options argument to lustre_gssiam_hash_key() so cache and nodemap keys account for mount and auth options. - Verify sender export is a target peer in mdt_get_info() before handling GSSIAM authentication. - Reply GSSIAM_AUTH_DENY when principal is NULL or empty, use cpu_to_le32/cpu_to_le64 for gar_permission/gar_projid/ gar_expire in struct gssiam_auth_rep, and propagate transient server error codes in mdt_get_info_gssiam_authenticate(). - Verify export obd and peer NID match in sec_gssiam_srv.c before accepting handle from class_conn2export(). - In tgt_gssiam.c: assign token_len after allocation, pass demoted RO permission to gssiam_nodemap_name(), initialize str_len = 0, and use tgt_gssiam_cache directly in attach/ detach cache entry helpers. Signed-off-by: Di Wang <ddiwang@google.com> Change-Id: I9fbd92e7e5dcdbc0039f90f547ce3841b005615e
(style) The summary reads as past tense; house style is imperative. "update mirror split for EC support" would match the rest of the tree.
(minor) The body calls out `test_44f` and the `test_7b` rewrite by name, but sanity-ec `test_7e`, `test_7f`, `test_7g` and the new `identify_ec_mirrors()` helper (~240 new lines) aren't mentioned anywhere. Worth a sentence so a reader knows the new coverage is intentional and what it exercises.
@mvef@whamcloud.com, @rsahlberg@whamcloud.com, I see that `--force-no-ec` is listed in LU-19548, but I don't recall what the realistic use case is for that option? Leaving an unanchored EC mirror behind on a file doesn't seem very useful, and there doesn't seem to be any way to reconnect it to a new data mirror even if it was created (nor assurance that the OSTs used in the data mirror are suitable for that EC). (Apologies in advance to Max, but ...) I'm wondering if `--force-no-ec` option should be removed, and EC mirrors should always be moved to the victim file or deleted? That would remove a lot of complexity in the code that I doesn't seem to add any real value.
(style) The .TH date is older than the lfs-mirror-delete.1 date set in the same patch, and predates the current content. Worth bumping to the refresh date.
(minor) The victim now gets two mirrors, and lod_declare_layout_merge() rejects any merge source that has more than one:
if (le16_to_cpu(merge_lcm->lcm_mirror_count))
RETURN(-EBUSY);
So `lfs mirror extend -N -f NEW_FILE` (documented in lfs-mirror-extend.1) fails with EBUSY on a file produced this way, unlike every other split victim. Should this paragraph say so, until extend learns to merge a data+parity pair?
(style) function comment should follow the style as `mdd_split_ea`.
(question) For a data+parity victim, if either side was *STALE* on the source, should the new file always set data *in-sync* and parity *STALE*? The victim has no other data mirror, so data has to be the non-stale primary or the layout is unusable (all-stale is -EPERM; in-sync parity as primary is -EUCLEAN). Only when both were already in-sync should both stay in-sync. Can current mask accomplish this or not?
(defect) llapi_mirror_find_stale() skips LCME_FL_PARITY, so a remaining
parity mirror always yields comp_size == 0 and this returns false.
lfs mirror split/delete then skips the last-good-copy resync even when
every remaining data mirror is stale.
Should remaining parity mirrors be ignored here the same way
data_mirror_remains() ignores them? Co-split also takes the paired
parity, so that parity is not a copy that will survive the split.
Example: file has D1 (EC data, in-sync after a write), P1 (parity),
and D2 (data, stale).
lfs mirror split --mirror-id D1
last_non_stale_mirror(D1) sees remaining mirrors {P1, D2}. P1 is
skipped by find_stale, so comp_size == 0 and the function returns
false. Resync is skipped, co-split removes D1+P1, and the source is
left with only stale D2.
(minor) The comment is narrower than the condition. `mflags & MF_DESTROY` with a NULL victim_file is also true for `lfs mirror split -d` (which cannot take -f at all), for `lfs pcc detach` (`MF_DESTROY | MF_FOREIGN`), and for the internal `mirror_split(name, 1, NULL, MF_DESTROY, NULL)` call. Naming only `lfs mirror delete` makes the flag look narrower than it is.
(minor) mirror_split() also backs `lfs mirror delete` (SO_MIRROR_DELETE sets MF_DESTROY) and `lfs pcc detach`, so a user who typed `lfs mirror delete` is told the mirror is being "split". sanity-ec test_7f greps for this text from a `lfs mirror delete` run, so that wording is on a tested path. Also, every other message in this function is prefixed with `progname`; this one isn't, so in a pipeline it isn't obvious which tool emitted it.
Skipping the orphan parity comp leaves `comp_array[i].lrc_synced` at the value `_mirror_find_stale()` set, which is unconditionally `true`:
comp[idx].lrc_synced = true; /* liblustreapi_layout.c */
Back in lfs_mirror_resync_file() every entry with `lrc_synced` is fed to LL_LEASE_RESYNC_DONE, and lod_declare_update_sync_pending() then does `llc_flags &= ~LCME_FL_STALE` for each id.
So `lfs mirror resync` returns success and drops LCME_FL_STALE from a parity component it never recomputed - `lfs getstripe` afterwards shows it as in sync. Would `comp_array[i].lrc_synced = false;` before the `continue` be more honest here?
LU-19548 lfs: updated mirror split for EC support
An EC data mirror and its parity mirror form a pair joined by a
bidirectional link (lcme_mirror_link_id). Splitting one mirror
without the other strands a parity mirror that protects no data,
or leaves a data mirror with no EC protection.
When splitting a data mirror, the MDS now splits (or destroys) its
paired parity mirror in the same atomic operation. The pair is
found via the mirror link id rather than by assuming the data and
parity components are adjacent in the layout, and the link is only
followed while it still describes an intact pair: the linked mirror
must exist, carry LCME_FL_PARITY and link back. A link failing
those tests is stale, and following it would co-split an unrelated
mirror. The link on any mirror left behind is cleared so no
dangling reference remains.
A parity mirror can still end up with no data mirror to pair with,
since a stale link is not followed and is cleared instead. lfs
mirror verify and lfs mirror resync resolve a parity component to
its data component through that link, in
__llapi_layout_find_data_comp_by_parity(), so both used to fail the
whole file with -ENOENT; once anything was written to the file the
orphan was marked stale and could never be resynced again. Skip a
parity component that has no data component to pair with instead.
mirror_split() used its "purge" flag for two things: to mean "the
mirror is deleted by handing fd in as the victim", and as the guard
for the retry that copes with an old MDS rejecting fd == fdv. The
flag started out true for every split, so a split to -f NEW_FILE or
to the default victim file also retried after an -EINVAL or -EBUSY
from the MDS, re-opened with O_CREAT|O_EXCL the victim it had
already created, and reported "create victim file failed: File
exists" in place of the real cause. The same flag guards the close
of the victim descriptor, so those splits leaked one descriptor per
file and splitting a batch of files in a single command ran out of
descriptors partway through. Start "purge" as the condition it
describes.
Such a failure was also invisible to a script, since mirror_split()
returned the leftover 0 of an earlier helper on several error paths:
a split that could not create its victim file, or that named a
mirror, component or pool the file does not have, printed an error
and exited 0. Set an error on those paths. Add sanity-flr test_44f.
A split is refused if it would remove every mirror, and also if it
would leave the file without a data mirror. Parity mirrors alone
cannot serve reads: with no data mirror the layout is no longer
FLR, so lov_io_mirror_init() returns at its !lov_is_flr() check
ahead of the parity checks, the parity mirror is picked for IO, and
reads hand back parity stripes as file data. A lone data+parity
pair therefore cannot be taken apart by splitting the data mirror;
split the parity mirror instead to turn the file back into a plain
data file.
The warning printed when a parity mirror is split names the data
mirror that loses EC protection, and is skipped for a parity mirror
whose link has been cleared, which protects no data to begin with.
Splitting a parity mirror directly no longer requires -d. That
restriction, added by commit dfb93e4978b0 ("LU-19548 lfs: mirror
split -d for parity mirrors"), existed to keep a parity-only
layout off a real file. Refusing to strand the data side now covers
the source file, and a victim file holding only the parity mirror
reports size 0, because lov_attr_get_composite() does not take size
from parity components, so it cannot pass parity off as file data.
A warning is printed instead, since removing the parity eliminates
EC protection of the data mirror. test_7b is updated for the new
behaviour.
A victim that receives a data+parity pair keeps its LCME_FL_STALE
markers, so a parity mirror that was stale at split time no longer
lands in a victim that claims an in-sync pair.
lod_declare_layout_purge() refused a purge buffer holding more than
one mirror. lfs mirror delete of a data mirror now hands it the
data+parity pair, so the check is relaxed from lcm_mirror_count
!= 0 to > 1.
Also fix two endian bugs that predate this work in the rewritten
function, harmless on little-endian but wrong on big-endian:
lcm_flags was converted twice, and the __u16 lcm_entry_count was
written with cpu_to_le32().
collect_mirror_id() fills an array owned by its caller but never
knew how large that array was, and every caller passed 128 entries
while lod.*.mirror_count_max accepts up to LUSTRE_MIRROR_COUNT_MAX
mirrors. Splitting a mirror out of a file with more mirrors than
that wrote past the end of a stack array. Carry the array size in
struct collect_ids_data, check it in the callback, and size the
arrays LUSTRE_MIRROR_COUNT_MAX so the check cannot be reached.
Fixes: 23b2d4781899 ("LU-10420 flr: split a mirror from mirrored file")
Fixes: c6e7c0788d7c ("LU-10258 lfs: lfs mirror copy command")
Fixes: b2d73351e646 ("LU-14521 flr: delete mirror without volatile file")
Fixes: fb790204ce3e ("LU-17908 layout: preserve non-FLR state layout flags")
Test-Parameters: testlist=sanity-ec ostcount=8
Test-Parameters: testlist=sanity-flr
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Assisted-by: ClaudeCode:Opus-4.8 llm_code_and_review_tools
Change-Id: I69ca706127e8e0b0e26c88525ac5509e22fa912a
LU-12669 ec: recover data from parity
Restore read data when some OSTs are unavailable from updated parity
codes.
When normal read fails, the read would switch to CIT_EC_RD, and the
ec read inner IO would expand to cover the whole raid set stripes, the
extent lock of the ec_rd would cover the available data OSTs, then
it reads pages from available data objects and parity objects,
calculates and fill in the missing data pages so that the original
outer IO can pick them up and copy to user buffer.
The ec read outer IO is to keep track of the original read position
and count, and the ec read inner IO is to expand the IO to cover the
whole raid set stripes.
lov_ec_read_stripe_pages() would read all pages for one data stripe
across all page positions in a recovery group in a single
cl_io_submit_sync call. And lov_ec_read_parity_stripe() would read
all pages for one parity stripe in a single cl_io_submit_sync call.
Also catch DIO read failure and switch to CIT_EC_RD to recover data
from parity (buffered IO).
For EC recovery reads, the page beyong the end_index has no DLM lock
(EC recovery only locks the actual read range), so return -EIO to stop
the kernel from retrying. And for EC recovery read, verify DLM lock
coverage for every page, pages on good stripes need to be read from
OSTs and have locks, pages on deactivated stripes should already be in
the page cache from EC reconstruction. If a page without lock coverage
reaches ll_readpage(), it maps to a failed stripe -- return -EIO to
prevent LBUG in osc_req_attr_set().
Introduce a new cl_page_alloc_sub() to allocate a cl_page for a
sub-object. The sub-object is the lovsub object for a parity stripe,
the difference it's from cl_page_alloc() is that the page index of the
sub-object is relative to the parity object, not to the file.
This patch also fixes a resync crash on overstriped parity mirrors. If
a file's parity mirror was set overstriped ("-C, --overstripe-count"),
the lov_pattern_supported/available() check failed causing that mirror
not being created and the write to fail.
Fixes: e90c6a428e27 ("LU-12188 uapi: add LOV_PATTERN_PARITY")
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I74977a9148256e2604d948979b5f0b944786e4e0
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.7 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
(minor) "stop handing out EC layouts" doesn't quite match the code. mdt_pattern_types is only read in mdt_connect_internal(); nothing in mdt/ or lod/ consults it when a layout is created, so an MDT with parity removed will still create EC layouts on request. What it stops is clients *using* parity components, and only for clients that mount afterwards. Worth noting too that the negotiated mask does not gate creation on the client either: with lustre.enable_flr_ec=0 (the default) a client can still create an EC file via llite.*.enable_erasure_coding and then get -EIO writing it. Should layout creation be gated on the negotiated mask as well?
This baseline omits LOV_PATTERN_COMPRESS, but lov_pattern_supported() right below still lists `LOV_PATTERN_RAID0 | LOV_PATTERN_COMPRESS` as a pattern this client's IO stack handles.
So for a compressed component lov_lsme_usable() -> lov_pattern_available_mds() computes
pattern_base = 0x801
0x801 & ~LOV_MDS_PATTERN_SUPPORT_217 = 0x800 /* != 0 */
and returns false, on both branches: the compat branch uses this macro, and the negotiated branch uses the MDS mask, which is capped by MDT_PATTERN_TYPES_SUPPORTED = baseline | PARITY. There is no value of mdt.*.pattern_types that can put COMPRESS back (conf-sanity test_163a asserts `+compress` is silently dropped).
The result is that lov_init_composite() leaves every compressed component !lle_valid and skips lco_init(), so lov_io_rw_iter_init() returns -EAGAIN/-EIO for a plain compressed file. This is reachable today: lsme_unpack() accepts COMPRESS components when llite.*.enable_compression is set (lov_ea.c), and lod_generate_lovea() packs them.
Should the mask be derived from what the client actually supports (i.e. include COMPRESS in both the 2.17 baseline and MDT_PATTERN_TYPES_SUPPORTED), rather than a hard-coded RAID0/MDT/OVERSTRIPING set?
(style) F_HOLE and F_RELEASED are layout flags rather than pattern types, and every consumer strips them first (`pattern & ~LOV_PATTERN_F_MASK` in lov_pattern_available_mds(), `& ~(F_RELEASED | F_MASK)` in lov_pattern_supported()). lov_pattern_bit2str() also has no names for bits 30/31, so they never show up in mdt.*.pattern_types or lov.*.mds_pattern_support. They only put two bits nobody reads on the wire in ocd_pattern_support. Could the macro just be RAID0 | MDT | OVERSTRIPING?
(minor) `data` here comes from obd_get_info(KEY_CONN_DATA) on sbi->ll_md_exp, which is the LMV export, and lmv_get_info() forwards that key to `lmv_tgt(lmv, 0)` only. So on DNE the stored mask is whatever MDT index 0 negotiated; mdt.<fs>-MDT0001.pattern_types has no effect on any client, even though the parameter is per-MDT and the man page shows a single-MDT example. Should the client intersect the masks from all MDTs, or should the caveat be documented?
(minor) This version gate was flagged on an earlier patchset ("this version should be updated when the patch is refreshed") and answered "Done", but it is still 2.17.52.224 here and in test_163a/test_163b.
The things these tests check (`pattern_support` in the mdc/osc import, mdt.*.pattern_types) only exist from this patch, so any server in 2.17.53..2.17.57 passes the gate without the feature. The tests do fall through to a skip on the missing parameter, but the gate should be the version this lands in.
LU-12187 lov: MDS layout pattern negotiation
Add 'obd_connect_data::ocd_pattern_support' to allow the client
and MDS to negotiate the file layout patterns that they support.
This prevents clients from trying to use layout types not
recognized by the MDS, and allows the MDS to (potentially)
convert/filter existing file layouts to a format that the
client understands.
This will be used by FLR-EC to negotiate whether the client
and MDS support LOV_PATTERN_PARITY layouts, and others in
the future.
Add LOV_MDS_PATTERN_SUPPORT_217 for compat with pre-2.18
MDS that do not send ocd_pattern_support. Generalize
lov_pattern_available_mds() to check all patterns against
the MDS-negotiated mask. The client sends its supported patterns
to the MDS, which responds with the intersection of both sets.
The client stores the negotiated mask only when the server
grants OBD_CONNECT2_FLR_EC. target_handle_connect() echoes
un-negotiated ocd fields back from the request, so an ungated
store would let the client's own request value defeat the
pre-2.18 fallback.
Gate pattern usability at layout init rather than at parse time:
lov_init_composite() consults lov_lsme_usable() (pattern support
intersected with the MDS-negotiated mask), and leaves unusable
components un-initialized and !lle_valid, the same as components
with an unknown pattern. Parsing itself stays pattern-agnostic
to preserve lsme alloc/free symmetry.
Adds mdt.*.pattern_types which gives an administrator the
ability to enable or disable layout types at runtime, and a
read-only lov.*.mds_pattern_support which exposes the
effective negotiated mask on the client. The mdt_enable_flr_ec
module parameter now only sets the initial pattern_types value
when an MDT starts up, so mdt.*.pattern_types is what changes
the advertised types at runtime. Add man pages for both
parameters and for the renamed lustre.enable_flr_ec parameter.
Rename the 'llite_enable_flr_ec' module parameter to
'enable_flr_ec', so it is accessed as 'lustre.enable_flr_ec'
per LU-14144 convention. Add the 'lustre' module to the libcfs
parameter path list so the client module parameters are reachable
via 'lctl {get,set}_param --module' instead of a hard-coded
/sys/module path.
Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=conf-sanity env=ONLY="163 163a 163b"
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Change-Id: Iaef716e28014be5b91dd50b117dd0881f5c37f37
(style) line length of 83 exceeds 80 columns
(style) "contains" -> "has"
LU-10026 csdc: prefer uncompressed mirror for read When accessing a mirrored file with both compressed and uncompressed components, choose uncompressed components for read. csdc: prefer uncompressed mirror for write When writing to mirrored files with both compressed and uncompressed mirrors, prefer the uncompressed components to write, and that is better for performance, more compatible with older clients, and better fits the model of compressing files after initial write. DDN-bug-id: EX-6510 Was-Change-Id: I62a117d5cc3d34e2c0c96d1a9ade8eef0a2d1291 Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@gmail.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I043b27bd891c039901075a08c76630f8f0f9f182
(minor) the date should be updated
LU-10026 utils: add 'lfs find' support for compressed file * Add "--comp-flags=[^]compress" to locate file with/without compressed components. * Add "--comp-flags=[^]nocompr" to locate file with/without setting component compress preference. * Add "[!] --layout=compress" to locate file with/without compressed components. * Add "[!] --compress-type=<compress-type>" to locate compressed file with/without specified compress algorithm. * Add "[!] --compress-level=[+-]<compress-level>" to locate compressed file with/without specified compress level. * Add "[!] --compress-chunk=[+-]<compress-chunk>" to locate compressed file with/without specified compress chunk in KiB. utils: add -Z option for 'lfs getstripe/find' Add support for "lfs getstripe -Z" to get the last instantiated component compression information. Add support for "lfs find -Z <type>[:[+-]<level>]" to keep consistent options with "lfs setstripe -Z". DDN-bug-id: EX-6856 Was-Change-Id: Ia5e2a2ea6937dc2b46e224cec808504a196b974c utils: logical AND for 'lfs find' compression exprs All search expressions provided to 'lfs find' must be combined as a logical AND. Fix newly added options for compression support, so that they comply with this logical AND. DDN-bug-id: EX-7714 Was-Change-Id: I3b28cd87c1d304df6d04753b413d46f5abcfe16e csdc: don't set compression layout when disabled When llite_enable_compression is disabled (lfs set_param llite.*.enable_compression=0), we should check it before sending it to MDS lest we get a file with compressed component which we cannot handle. DDN-bug-id: EX-7593 Was-Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f utils: support 'lfs find --printf %LZ' Add support for "lfs find --printf %LZ" to print the compression type:level of the last instantiated component of a file. DDN-bug-id: EX-6856 Was-Change-Id: Iaf1b6c031b06c70e7b5be51354697aa6bdcc9850 csdc: right error code in case compression is disabled Currently, if compression is disabled on a client the error message is confusing "Cannot set layout EA: Unknown error 524" Let's replace the error code from ENOTSUPP to EOPNOTSUPP, so error messages become more informative ctl get_param -n llite.*.enable_compression 0 lfs setstripe -i 0 -c 1 -E -1 -Z lz4 /mnt/lustre/foo.txt Cannot set layout EA: Operation not supported lfs setstripe: cannot create composite file '/mnt/lustre/foo.txt': Inappropriate ioctl for device This message could be even more informative, but this version is already much better. DDN-bug-id: EX-10577 Was-Change-Id: I7fd67d5d3d20bc7de169f35f6fa63b53f2bfe718 Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ic36946738c3463fd862aeca4ee2e2c2ed85eff84
this patch also conflicts with https://review.whamcloud.com/c/fs/lustre-release/+/62906 that is in master-next already
From Claude: (minor) fp_compr_lvl is pre-seeded with (unsigned int)-1 as the \"no level given\" sentinel, but llapi_parse_compress_type() can legitimately parse a level of -1 (range is -COMPR_LEVEL_MAX..COMPR_LEVEL_MAX), which also lands as (unsigned int)-1. So `lfs find -Z gzip:-1` takes this false branch and the level filter is silently dropped - it behaves like `-Z gzip`. A separate \"level present\" bool would avoid the collision.
Oleg, this comment seems misplaced? I don't see this related to anything here.
LU-10026 csdc: set compress component for file
* 'lfs setstripe' to support compress component
--compress|-Z <type>[:<level>]
Set component compression algorithm <type> and compress <level>;
--compress-chunk=<size>
Set compress data chunk size in KiB used by the compression
algorithm, the value will be adjusted to power-of-two multiples of
the base 64KiB.
Example:
$ lfs setstripe -Eeof -Z lz4:5 --compress-chunk=512 <file>
* 'lfs getstripe' to show compress component parameters
Display component's compression parameters if possible.
--compress-type|--compr-type
Print only the compress type if possible.
--compress-level|--compr-level
Print only the compress level if possible.
--compress-chunk|--compr-chunk
Print only the compress chunk size in KiB if possible.
Example:
$ lfs getstripe <file>
lcme_compr_type: lz4
lcme_compr_lvl: 5
lcme_compr_chunk_kb: 512
lmm_pattern: raid0,compress
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ife0382469cbc5099e0c6dc96534bb169ddeff61e
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| custom-1002 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-subtest-change failed 2× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-zfs-subtest-change failed 7× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-ldiskfs-dne-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-12668 ec: Add tests for computing the parity coverage When we write or verify the parities we no longer use the whole stripe, instead we compute the coverage of what ranges of parity is important and what can be ignored. This is based on SEEK_DATA/SEEK_HOLE and EOF. Add tests that we compute these ranges correctly. Test-Parameters: trivial Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: Iaee9d0ccb0875f6d515a8adc322fa3cd37bfe0f6
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1002 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-sec. | session |
| custom-2001 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-sec. | session |
LU-20213 dbg: force timeout This patch attempts to reproduce LU-20213. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 serverjob=lustre-b2_15 serverbuildno=117 Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 serverjob=lustre-b2_16 serverbuildno=6 Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I8674a4a7bf061ed0a475826fbfb2a7b89d6d1487
LU-20213 dbg: revert patch Revert "LU-19763 socklnd: Check sendpage_ok for ZC" This reverts commit bf67df3212759c40b434c8182663f88ee23ad618. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 serverjob=lustre-b2_15 serverbuildno=117 Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Id54398db09df332e861b4c7cb9538f630bf98aca
This paragraph describes a change that does not seem to be in the diff. lustre/lov/lov_page.c isn't touched, and lov_page_init_composite() has no append exception either before or after this commit:
stripe_cached = lio->lis_cached_entry != LIS_CACHE_ENTRY_NONE &&
page->cp_type == CPT_TRANSIENT;
Was this paragraph left over from an earlier version of the patch?
This fixes wrong-stripe routing and a wrong file size, so a Fixes: tag would help decide which maintenance branches need it. The "DIO is already split by stripe" assumption that this patch repairs was introduced by:
Fixes: d31647c017a3 ("LU-13799 lov: Improve DIO submit")
That commit made lov_io_submit() splice every DIO page onto the stripe of the first page, and 14db1faa0fbe ("LU-13799 lov: Cache stripe offset calculation") then extended the same assumption to the per-page stripe cache. The assumption never held for the append path.
The clamp to the component extent lives inside this branch, so nothing bounds the batch when the component at `start` has a single stripe. For a layout like `-E 1M -c 1 -E -1 -c 4`, a 4MB DIO append from offset 0 gets no split at all and the batch crosses the component boundary; lov_page_init_composite() then hits
if (!lov_io_layout_at_confirm(lio, entry, offset))
return -ENODATA;
for the pages past 1M and the write fails.
The non-append path below clamps unconditionally - it starts at MAX_LFS_FILESIZE and applies min(next, lse->lsme_extent.e_end) outside the stripe_count test. Should this do the same, so the commit message's "matching what the non-append path already does" actually holds?
lio->lis_io_endpos is an absolute file offset set once in lov_io_init():
lio->lis_io_endpos = crw_pos + crw_bytes;
For append that crw_pos is only the estimate taken in ll_file_io_generic(); the offset the write actually lands at is chosen later by vvp_io_write_start() from i_size_read(). After the first iteration `start` tracks that real position, so if the file grew in the meantime `start` can run past lis_io_endpos.
Two consequences: the loop stops once next reaches lis_io_endpos even though bytes remain in the iov_iter (short write from an O_APPEND write(), where before the patch the whole count went out in one iteration), and on the next line `next - start` goes negative into the size_t crw_bytes.
Should the append path clamp against the number of bytes still outstanding rather than an absolute endpos, and guard next >= start?
Setting ci_continue here turns a single append iteration into several, and cl_io_loop() drops the lock between them:
cl_io_lock() -> cl_io_start() -> cl_io_end() -> cl_io_unlock()
vvp_io_write_lock() takes [0, wr_append_lockpos] PW for append precisely so the whole write lands atomically at EOF, and vvp_io_write_start() re-reads i_size_read() under that lock on every iteration. With the split, another client can revoke the lock and append between our iterations, so a single 4MB O_APPEND write() can end up with a foreign client's data interleaved in the middle of it.
Is there a way to keep the append as one locked iteration and instead bound the cl_dio_pages batch itself (in ll_direct_IO()/cl_dio_pages_init()) at the stripe boundary?
This contradicts the assertion 14 lines above, which says the opposite about the same queue:
/* it could only be mirror read to get here therefore
* the pages will be transient. */
LASSERT(page->cp_type == CPT_TRANSIENT);
If transient pages genuinely never reach lov_io_submit(), that earlier LASSERT can never be satisfied and would LBUG the client on any buffered read that lands on an uninstantiated component. Worth resolving one way or the other while this function is being touched.
There is no OST count precondition, so on a single-OST setup `-c 2` silently yields one stripe and the test passes without exercising anything. Consider:
(( OSTCOUNT >= 2 )) || skip_env "needs at least 2 OSTs"
Also, the file is empty here, so the append starts at offset 0 and is already stripe aligned. Appending to a file that is not a multiple of the stripe size (say seed it with 512KB first) is the case where the split has to get the first partial stripe right.
Both md5sums read the same OST data - the first one populates the page cache from the OSTs after the DIO write - so this compares the file against itself rather than against what was written. Writing from a temp file (or `dd`ing a known pattern) and comparing that checksum would actually catch mis-routed data; right now only the size check can fail. On a related note, the earlier suggestion on this hunk to use `cancel_lru_locks osc` instead of drop_caches doesn't appear to have been picked up in the current revision.
LU-19900 lov: fix DIO with O_APPEND stripe routing When a file is opened with O_APPEND and written via direct IO, all data is incorrectly routed to stripe 0 instead of being distributed across stripes. The root cause is that lov_io_rw_iter_init() does not split append writes at stripe boundaries. Each DIO iteration processes a cl_dio_pages batch, which lov_dio_pages_init_composite() assigns to a single stripe. Without splitting, the entire write goes to one stripe, causing incorrect file sizes (e.g. 7MB instead of 4MB for a 4MB write to a 2-stripe file) and data corruption. Fix by adding stripe boundary splitting to the append path of lov_io_rw_iter_init(), matching what the non-append path already does. Each iteration now covers at most one stripe_size worth of data, so lov_dio_pages_init_composite() correctly assigns each batch to a single stripe. Also add an LASSERT in lov_io_submit() confirming that DIO pages (CPT_TRANSIENT) never reach that path - they use lov_dio_submit() instead. Also revert the append exception for the DIO stripe cache in lov_page_init_composite() since it is no longer needed now that iterations are single-stripe. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I66304a6692eaab9c68a03159e51cf3d27c465a83
LU-10026 osc: add compression to can_merge_pages Some BRW flags are OK to have on only some pages in a BRW, others are not. can_merge_pages has a whitelist of the flags which are safe to have on only some pages in a BRW, and prints a warning if other flags are seen. Add compression to the white list, because while all pages in an niobuf must be compressed, it is normal to have only some pages in a BRW compressed. Prior to this patch, this warning was printing during normal usage of compression. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ia262d4fc878e5328bd956865047e997aa77946f0
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56ab | seen in 21 other reviews |
(defect) The message describes a "sec: page_pools shrinker fix" (element_size / ppp_idle_idx / IDLE_IDX_MAX changes) and adds sanity-compr test_2000 gated on client 2.17.53 "for compression OOM fix", but there is no page_pools.c change anywhere in the diff, so test_2000 can't pass on this commit. A few hunks also aren't accounted for by the message: the ll_lov_getstripe_ea_info() switch to md_getattr() when filename is NULL in llite/file.c, and the LINVRNT->LASSERT change in cl_io_rw_init(). Are these meant to be part of this patch? Also, since this touches the OST_WRITE/OST_READ wire path (OBD_BRW_COMPRESSED, o_size/OBD_MD_FLSIZE), a Test-Parameters: line requesting client/server interop testing would be worthwhile.
(defect) this needs to add: ``` Test-Parameters: testlist=sanity-compr ``` otherwise that test session is not being run on master.
(defect) 0x418 is already used by OBD_FAIL_OSC_FIEMAP two lines above, so this fault injection point collides with it (triggering one fires the other). 0x41a or 0x420 appear free.
(defect) On the decompress path *type is llch_compr_type read from storage/the wire, so a corrupt header carrying LL_COMPR_TYPE_BEST or LL_COMPR_TYPE_FAST would LBUG the client here. The sibling change that replaced an assert with EIO ("we shouldn't assert on values read from storage") applies - return -EIO instead of asserting?
(minor) If fill_cpga() fails here, `dst` (obtained just above from obd_pool_get_objects) has not yet been stored in bp_cmp_chunk, and the out: path only releases `src`, so the chunk buffer looks leaked on this error return.
(defect) compressed_pages comes from llch.llch_compr_size, which is read straight out of the server reply by is_chunk_start() (that only checks llch_magic). It is bounded above only by page_count - i, not by pages_per_chunk. merge_chunk() then copies compressed_pages * PAGE_SIZE bytes into `src`, a single chunk_size buffer from obd_pool_get_objects(&src, buf_bits). A corrupt or hostile compr_size larger than one chunk overflows `src`. The LASSERT(src_size <= chunk_size) on the next line runs only after the copy (and asserting on wire data is itself unsafe). Can compressed_pages be bounded against pages_per_chunk before merging, returning -EUCLEAN otherwise?
(defect) On a resend, osc_brw_redo_request() re-enters this function with aa->aa_ppga. For a compressed write that array is the cpga built by compress_request()/fill_cpga() - bare brw_pages allocated with OBD_ALLOC_PTR and freed with a plain OBD_FREE, not embedded in an osc_async_page. brw_page2oap(pga[0]) then points before the allocation, so oap2cl_page() and the following clpage->cp_inode / cp_type / cp_compr_type reads touch invalid memory. -EINPROGRESS/recoverable resends are common, so this looks like a client crash on any compressed-write resend. Should the resend derive inode/compressed from aa_ncppga instead?
(style) This loop is indented with spaces instead of tabs.
LU-10026 osc: osc brw request compression This patch adds client-side compression/decompression. The client-side data compression project (CSDC) reduces storage and network utilization by leveraging the more plentiful memory and CPU resources on the local client. Data is sent compressed over the network, saved directly to storage on the server side, and decompressed back on the client side. Uncompressed data is kept in client page cache, all while being functionally transparent to the end user and application. As an example, a test file is compressed and decompressed. The resulting file is compared with the original one. The test case shows 2.5x compression ratio: 356K /mnt/lustre/d460.sanity/sanity.sh 884K /tmp/cmp-46ofie/decompressed_sanity.sh Compression should read whole chunk even if offset and size differ. Let's modify readahead to force reading data from the offset and size multiple to the chunk size. utils: fix cp_comp_type size cp_comp_type should be 8 bits, as llch_compr_type and all associated variables are declared as u8. So remove useless cp_comp_enabled and fix code to test for compressed component with cp_comp_type against LL_COMPR_TYPE_NONE. And update LL_COMPR_TYPE_MAX value to 255 to avoid conflicts with future compression types. DDN-bug-id: EX-7775 Was-Change-Id: Ia15868ac0ac003b62942540a57f782226ae8c141 tests: test compression without bzip2/HDF5 Run as much of sanity test_460a compression tests as possible, even if bunzip2 or HDF5 file are unavailable. Print a clear message in test_84 if bunzip2 unavailable. DDN-bug-id: EX-6127 Was-Change-Id: I36251834f636600eb9b0194ccd14c8b203da32e5 lov: refactor lov_io_lsme_at lov_io_lsme_at needs some minor changes to be called from lov_io_slice_init(). DDN-bug-id: EX-7601 Was-Change-Id: I0611d66052e22d349932eb26257369e07b9b8167 osc: don't check for start inside the chunk Chunk size is the same for the whole request and every chunk offset is multiple to a chunk size. No need to search for compression header in every page. It is enough to check every with offset multiple to a chunk size. DDN-bug-id: EX-7818 Was-Change-Id: Ie2ef645130656279e152ea1f7e6db01cb33836ca osc: minor compression cleanups This cleans up some style and argument issues I found made the code a little harder to follow. DDN-bug-id: EX-8270 Was-Change-Id: Ia3492ae79acf6c83d724cc91b0201c7872325853 osc: move common CSDC code to the library CSDC repacks a chunk on the server side in case of the partial rewrite. There are routines that can be shared between client and server. This patch moves common compression code to the libcfs. DDN-bug-id: EX-7601 Was-Change-Id: I824211a3435b0479f7a3b8f08598a5b567b67d3c osc: use correct count Using the number of bytes in the compressed page creates gaps in the RDMA, which IB memory registration cannot accept. Fix this by always setting count to PAGE_SIZE for compressed pages and otherwise using the count from the original source page. Setting PAGE_SIZE for compressed pages is valid because client only does compression for aligned IO, except for the trailing chunk. For the trailing chunk, the file size is set on the server, so any trailing bytes are ignored. DDN-bug-id: EX-8245 Was-Change-Id: Ied89d3ac328fb6020079392f5a8812ad5637b4a4 csdc: remove holes from struct ll_compr_hdr This patch reorganizes struct ll_compr_hdr to remove alignment holes. DDN-bug-id: EX-8353 Was-Change-Id: I59800b00e3a17972d621bae21ba06509a39b1036 osc: apply compressed flag to dst page The existing code to apply brw flags to compressed pages has two issues: 1. The dst_page is NOT an osc async page, it is a bare BRW page. This means the brw_page2oap macro isn't right, because there is no oap page. Because oap_brw_flags is actually oap_brw_page.flag, we don't ever access the memory pointed at by OAP, just use it to find an offset back in to the brw page. This means the flags are set correctly, but we still shouldn't use this macro. 2. However, the function then overwrites these flags by copying from a page in the source, so OBD_BRW_COMPRESSED is lost. Add OBD_BRW_COMPRESSED when we set flags. This ensures the flag is actually sent to the server on compressed IO. This was not causing any problems because the server does not actually use the OBD_BRW_COMPRESSED flag yet. (EX-7601 uses this flag) DDN-bug-id: EX-7601 Was-Change-Id: Ia94cdc803868ce16a0b66fd58578ec8b2d00cbae osc: remove unused 'wrkmem' compress_chunk() takes a wrkmem buffer, which it does not use. Remove this and its allocation in compress_request. DDN-bug-id: EX-7601 Was-Change-Id: I6f236f018f5b79c57cc8725ca0f95125810a4064 osc: walk chunk unaligned RPC correctly For decompression, the client must start looking for compressed chunks at a chunk aligned offset. Implement this in decompress_request. DDN-bug-id: EX-7601 Was-Change-Id: I3273135990ddf51e8b3c651734e19350e91f659c ofd: add obd level compression lib Some compression functions will be used by several areas of of Lustre, so they need to be in obdclass. This moves merge_chunk and unmerge_chunk there and adds the ability for them to merge lnbs. This is used in a future patch. DDN-bug-id: EX-7601 Was-Change-Id: If4a318119bb7685e41adb9f3b31a66074031e6ac osc: remove cpga fill bits cpga fill bits are not needed now that we don't support compression and encryption. DDN-bug-id: EX-7601 Was-Change-Id: I13c2278e085e9b288bd896585947e28e2ea505ca osc: only set compressed flag on compressed pages The code accidentally sets the compressed flag on all pages processed through fill_cpga, even if they're not compressed. Oops. Also stop setting pg->index on the pages in the compressed pga, this is only used by encryption and that's no longer supported with compression. DDN-bug-id: EX-7601 Was-Change-Id: I313fd943a18b71cd52493852a6884f30d187e52f osc: use pages_left in unmerge_chunk Since we have compressed chunks < chunk_size (if they're after EOF), we must use pages_left in unmgerge_chunk or it will go off the end of the page array. This also lets us remove the workaround where unmerge_chunk would skip pages that were not present. unmerge_chunk always works with a known and complete set of pages, so this check is unneeded. We should also check that our count of bytes is correct when we finish. DDN-bug-id: EX-7600 Was-Change-Id: I88896307990ff839514e54e9a7e18390a457e5d8 osc: rename 'done' Rename the ambiguous 'done' and remove it where not used. DDN-bug-id: EX-7601 Was-Change-Id: I8fb88b7a91fcc7dbd5ce2d29a61c18330fc0cda3 osc: cleanup compression variables Make usage of the compression variables more readable. DDN-bug-id: EX-7601 Was-Change-Id: I6daff56b56877c8f36e02303cc0579ba7faa731b ofd: make compress_chunk take chunk_bits Chunk bits is used everywhere, have compress_chunk convert to log bits rather than have the callers do it. DDN-bug-id: EX-7601 Was-Change-Id: Ic01bb749425cb95d9c5717965d692a18138ceeb7 obd: move module load to function This is a trivial code change to make alloc_compr a bit shorter. DDN-bug-id: EX-7601 Was-Change-Id: I0a790afe7afebde1d223420d9a578529da6ff7e5 osc: variable cleanup in decompress_req Use type and lvl variables in decompress_request. Remove an unused variable and an assert which can never fire. DDN-bug-id: EX-7601 Was-Change-Id: Ieff57411a2a41215fd368d731614801bd0f43e38 osc: replace assert with error We shouldn't assert on values read from storage, instead if they are incorrect, we should give EIO. DDN-bug-id: EX-7601 Was-Change-Id: Icda213e3c5a90a848c9b008788e92ee49e2efcb1 osc: rearrange compress_request A trivial rearrangement of compress_request to make it more readable before redoing the core logic. DDN-bug-id: EX-7601 Was-Change-Id: I1d34cd2a2a6d84bc30cc7dae8eb07586c4837f7d osc: give compress_request explicit success Compress_request has explicit failure handling, but the success handling just follows the failure handling. This is confusing - on failure, we do: page_count = *pcount then immediately do: *pcount = page_count It also sets *orig_pga = pga on success OR failure, which is wrong because compress_request may have modified pga and then failed. DDN-bug-id: EX-7601 Was-Change-Id: I121ec71cfe35babc4a572951e93f7581887ade80 osc: remove &pga usage in compress_request The usage of 'pga' and '&pga' in compress_request is confusing, but also, compress_request modifies &pga by allocating a new compressed page array. Except if we fail in compress_request, we free that new page array. This means failing in compress_request replaces 'pga' with a pointer to freed memory. Instead, create an explicit cpga pointer in the caller and use that. This allows compress_request to fail safely. DDN-bug-id: EX-7601 Was-Change-Id: Idaf592103c57b0e9ce76ab520a69b819d4f37be9 obd: move type switching to alloc_compr callers The code is much cleaner if we can eliminated applied type and handle that issue once per compression or decompression rather than for every chunk. This requires moving the type switching inside alloc_compr. (Also improve some error messages - alloc_compr can fail with ENOMEM as well.) The compression code currently allocates a transform for every chunk on the client. This is relatively cheap, but it also complicates the code by repeatedly checking if a particular compression type is supported (this is the "applied type" code). Moving alloc_compr to compress/decompress request makes the code much simpler. DDN-bug-id: EX-7601 Was-Change-Id: I162e81577db721a9715d57b3f262fcabbcbf308a osc: rename pages_in_chunk Chunks can have variable numbers of pages in them. DDN-bug-id: EX-7601 Was-Change-Id: If199d777367569e62c21305f6e4b9f3e4cce6d06 osc: allow multiple chunks in read It's rare, but reads can sometimes have multiple discontiguous chunks. Update decompress_request to handle this case. DDN-bug-id: EX-7601 Was-Change-Id: I880af95db285dce76db3610e8140a0f54baa401b ofd: do not overwrite rc in unmerge_chunk unmerge_chunk should not be responsible for setting the lnb rc, because this overwrites the result of any previous activity on the lnb. Plus, unmerge_chunk can't fail. DDN-bug-id: EX-7601 Was-Change-Id: Id1ce590c7f1da3ab7faddbd685d264a33c08d639 osc: calculate compressed size reduction accurately Compression reduces space used if it results in allocating at least one fewer block on disk. Modify the checks in compress_chunk to reflect this, rather than using the simpler "reduce size by at least 4K" calculation. Also do not attempt to compress chunks if they are less than 4K in size, since they can't possibly get a space benefit. This improved my measured ratio on a version of the Linux kernel source data set from 1.24 to 1.56, so this is significant for datasets with many small files. (This version of the source had large incompressible files removed, to focus on smaller files. The unmodified data set would not improve as much.) Note this is still short of our estimates, so either the estimate or Lustre still needs adjustment. TBD. DDN-bug-id: EX-7601 Was-Change-Id: I815706914b88de4f532a674d773769aa3a64d218 osc: rewrite compress_request The existing version of compress_request can't handle discontiguous RPCs. Rewrite the logic to handle this case properly. This also implements kms handling. If a write chunks ends at the known minimum size, we know this write is after all other data in the file and so there is no compressed data under it. This means we can compress this chunk. DDN-bug-id: EX-7601 Was-Change-Id: I8a912d9e279d04c8ff07de39e63a1ec9b490d921 osc: handle partial chunks in decompress_request Now that we have compression for incomplete chunks at the end of files, decompress_request needs to handle these chunks. This patch modifies it to understand compressed chunks which are less than chunk_size pages. DDN-bug-id: EX-7601 Was-Change-Id: I877550fa0d418def406e0308392a5336ec9f3ab6 osc: add check to decompress_request decompress_request should check to see if there's room in the RPC for the decompressed data, since this can occur if there's a bug or data corruption, and otherwise we will go past the end of the RPC during decompression. DDN-bug-id: EX-7601 Was-Change-Id: Ib1bf19bf39701b72f0f5a61b2aaff2f2fdad1897 osc: debug fix in decompress_request Debug message had an incorrect subtraction. DDN-bug-id: EX-7601 Was-Change-Id: I5daf360766ca77b98dc5af3d72c42ac38f5782bc obd: add 'lvl' for best and fast 'best' and 'fast' compression types must also set a level, because not all levels are supported by all algorithms. Rather than trying to be clever, just use simple universally supported values, except for lz4fast, where we special case this, because otherwise '0' is the slowest setting (and lz4fast is likely to remain our default fastest). DDN-bug-id: EX-6269 Was-Change-Id: I7c29659d4f027af2e44285ae38e4c9d91e35509a osc: decompress with algorithm from server Data may not be compressed with the compression type and level from the layout, so we must use the compression type and level from storage for decompression. DDN-bug-id: EX-6269 Was-Change-Id: Ib4cdccf294ef631a25147413d7f5c1a847c9504e osc: handle different compression types Allow the client to handle different compression types in a single component. This shouldn't happen normally, but it may happen in the future if there is dynamic compression algorithm selection for "fast" or "best" types (e.g. compress based on available CPU and network bandwidth or RPC backlog). DDN-bug-id: EX-6269 Was-Change-Id: Ide2731c60a68584e7cbb474bee88a17e9a7b8fec osc: add COMPR_GAP check to compress_request Currently, compress_request will build the compression buffer (calling merge_chunk()) for requests which are less than the minimum compression gap. This is noticed in the compression code when it checks if there's enough data to attempt compression, but we can do a trivial check in compress_request() to save that work. Also fix a few minor style things. This is not an important fix, but I discovered it while investigating another issue and it's trivial to resolve. DDN-bug-id: EX-7601 Was-Change-Id: Ieb32e6297e10d229f23c58e2ef4d933ce3dda4f2 lustre: add uncompressed size to compression header It's useful to have the uncompressed size of the data in the compression header. Also, we have three checksum fields - compressed, uncompressed, and header, but in practice, checksumming the compressed data including the header is enough to cover all of these. This patch cleans up all of this at the same time. DDN-bug-id: EX-8851 Was-Change-Id: Ie82e0dbe9c862ddc88999b109cea1f27577dbbff csdc: rename "cp_comp_*" to "cp_compr_*" This patch renames "cp_comp_type", "cp_comp_level", and "cp_chunk_log_bits" to use "compr" in the name to be consistent with other variable names. DDN-bug-id: EX-8353 Was-Change-Id: I428ff3a789b33da02832dee02f316b02d97137e2 osc: Do not iterate over chunk pages If osc_decompress() knows chunk size, no need to iterate other all pages in chunk. Thay can be skipped. DDN-bug-id: EX-7729 Was-Change-Id: Ib84b060075c55c97eba9f74ef017c0a956e85b12 csdc: Fix the upper mergeable chunk pointer If the full chunk is followed by un mergeable page, the upper mergeable chunk pointer is occasionally set to this unmanageable page. The chunk size is calculated wrongly then and the next condition suggest not to compress this chunk, because its size is not equal to the expected size. The pointer should be moved to the first instruction after the can_merge_pages(). DDN-bug-id: EX-9192 Was-Change-Id: I09fedc770c8bbcac4864b32372a941da5e0c7ac3 obdclass: reset bits after decompression as uncompressed data can be less than chunk/page, but still be visiable to userspace as a part of a sparse file. DDN-bug-id: EX-9873 Was-Change-Id: I4114b0704fb685013f4e03cf2d80ccde2cc8c87f osc: preserve compressed pages for OST_WRITE replay it's incorrect to release compressed pages right after reply as we may resend them during OST_WRITE replay. DDN-bug-id: EX-9895 Was-Change-Id: I3edc16d6556ddd60735d2f14fe879fc0f45231d7 csdc: Fix the next available algorithm selection Currently, if a chosen algorithm is not available, then next preferable is chosen, but an error code is not cleared so __alloc_compr() returns this wrong code. Data if written uncompressed while it can be compressed with next appropriate algorithm. This patch adds this error code clearing. Test is provided. DDN-bug-id: EX-9889 Was-Change-Id: I59f65058a0fe9b108de3d4ba7cf5950f18e32204 ptlrpc: reset refcount reusing pages for new req when a set of compressed pages are reused for a redo RPC, we have to reset refcounter. DDN-bug-id: EX-10061 Was-Change-Id: I16c46d857d60cf4dd0f6317190e7c99b2667e135 ptlrpc: drop extra reference to compressed pages if the request is not replayable, so brw_commit() won't be called. DDN-bug-id: EX-10184 Was-Change-Id: Ib29ab19b69fd15ef339cc18c5dfef17217a32cb1 osc: remove noisy warning if we can't compress data on the client side for a reason (e.g. too random), we just send data uncompressed, makes sense to hide the error message confusing customers: kernel: LustreError: 936336:0:(lustre_compr.c:345:compress_chunk()) exafs-OST0007-osc-ff49193c293f3800: Compression failed, type 5, lvl 5, -22 DDN-bug-id: EX-10593 Was-Change-Id: Icbc8be38b0372f8f67690e7d94384a16ab02cbe4 osc: fix osc_compress ENOMEM handling Send already compressed chunks in compress_request even if can't allocate memory for the next chunks. This patch also fixes statistic in case of ENOMEM. Without that write_chunks_incompressible hasn't been incremented in case of ENOMEM. Add sanity-compr_2002 to check write_chunks_compressible and write_chunks_incompressible in case of ENOMEM DDN-bug-id: EX-11035 Was-Change-Id: I3fc58ece6cf22582166ea77e4a217bc7a9efd77d csdc: remove buffer double sizes Pools now (POOL_ADD_PAGE is defined) allocate additional page for memory misuse debug, so no need to allocate double-sized buffers to prevent lz4 decompress problem. DDN-bug-id: EX-8276 Was-Change-Id: Iea04943ae0f95c8983c5ba7ec9a2c63003afe351 utils: fix LL_COMPR_TYPE wire checks Fix LL_COMPR_TYPE* wire checks to avoid duplication. The value of LL_COMPR_TYPE_UNCHANGED is also declared as 255 so that is does not conflict with other potential real compression types in the future. DDN-bug-id: EX-7775 Was-Change-Id: Iab9830f09f0778e1e1f3b1ea4c9878ce1017de8d sec: page_pools shrinker fix page_pool shrinker should consider ppp_free_pages as a number of memory regions with the size of 2^pool_order pages. Otherwhise it startis shrinking only when becomes too large, for example 2GB instead of 64MB. It often might be the reason of OOM especially when compressed files with different chunk-sizes have been written. In a such case several pools created for different chunk sizes might hold gygabytes of memory without any chance to release it. Fix element_size to return correct element size instead of number of pages. Because of that elements in page_pools had below sizes: order 0 = 4096 order 1 = 2 order 2 = 4 .... order 17 = 131072 This made possible to have elements with sizes 2,4,8,... bytes that have been allocated with OBD_VMALLOC. Now all sizes are miltiple of PAGE_SIZE: order 0 = 4096 order 1 = 8192 ... order 5 = 131072 Change the logic around ppp_idle_idx. 1. Recalculate ppp_idle_idx in __sptlrpc_pool_put_pages the same way as it done in __sptlrpc_pool_get_pages. It is possible that ppp_idle_idx might become 0 or very small (less than 10). If there is no new allocations it would be impossible to free anything from the pool during 40 seconds(CACHE_QUIESCENT_PERIOD) despite the large amount of free elements ready to shrinking. 2. In pool_shrink_count when ppp_idle_idx == IDLE_IDX_MAX, it means no one accessed pool for CACHE_QUIESCENT_PERIOD (40 seconds), i.e. we may release as much as possible pages. Earlier in a such case pool_shrink_count always returned 0. Another words it was impossible to shrink anything from the pool if there was no allocations for 40s. Add sanity-compr 2000 that reproduces the OOM problem on a client without this patch. DDN-bug-id: EX-10094 Was-Change-Id: I6b55ba67b0d21cdffdb57034e8e66063745f796e osc: Add BRW_COMPRESSED flag to reads We need to add the BRW_COMPRESSED flag to reads so servers can know if the client is able to decompress data. This lets servers decide if a client can be sent compressed data and the result won't be nonsense/corruption. This is important for future support of GPU direct, where the server will need to do the decompression. DDN-bug-id: EX-6269 Was-Change-Id: I36b5b73f983ce8f2e5297c3e9dc778a5eca54e6a osc: don't discard decompress_request error The error handling for decompress_request is unusual - non-zero returns are just discarded. And rc2 is just discarded. The read() doesn't fail or get a short read. Fix this so if decompression fails with an error. DDN-bug-id: EX-7807 Was-Change-Id: Idd01947c7375c9586a64f064dd6ee0ac2308ea86 osc: save compressed object size CSDC uses a sparse file feature. A client writes compressed data chunks to the original offsets so the same data is expected to be read from the same offsets. There are no writes after the last compressed chunk, so no "hole" after the last compressed chunk. Compressed file size (based on OST objects size) is smaller than the original on the "original last chunk size - compressed last chunk size" delta. Object size should be set to uncompressed size. This size is used to calculate file size and to remove the workaround of not compressing the last chunk in the file. DDN-bug-id: EX-7600 Was-Change-Id: I387c282e1cf788c3b8f6230ef555d73ffffe49c1 osd: save compressed object size on zfs "osc: save compressed object size" added means to transfer object size to the osd and added ldiskfs support. This patch adds saving objec size to the ZFS backend. Currently this fix submitted as separete patch, for testing purpouse, but can be marged to the main patch latter. DDN-bug-id: EX-7600 Was-Change-Id: I99e29e3f756a070b5f3cece12c4ca58f668a2ecf ldiskfs: fix detection of compressed extent The code in ldiskfs_map_inode_pages which detects a compressed extent checks the first lnb for that extent, but it's possible for some lnbs and not others to be compressed in a given extent, so we must check all of them. This occurs when multiple writes have been combined in to one RPC. If we don't detect compression correctly, we won't set the file size correctly and we'll get data corruption. DDN-bug-id: EX-7601 Was-Change-Id: I11d50bdc45c40d93bb1b829fcd930165b7626432 llite: DIO fallback on compressed files Fully supporting direct I/O on compressed files is tricky because we cannot pull the full chunk in to the page cache (because there is no page cache for DIO). So instead we fall back to buffered I/O for DIO on compressed files. This patch adds the check and a test for this. llite: allow aligned DIO with compression DDN-bug-id: EX-6127 DDN-bug-id: EX-7601 Was-Change-Id: I8224ef9b8ad1d912d8a11eccad37d3dff8dd8498 If a DIO is fully aligned to compression chunk boundaries, it is safe to do on a compressed file, so allow it. Was-Change-Id: If8fa3397c9424254538738f4d77f9f50d1c21129 lov: set IO property check in lov_io_init_composite Move compression IO check down to lov_io_init_composite(). DDN-bug-id: EX-9853 Was-Change-Id: I6d4ce3388d8eef788d31dabf4b3a5dd5eff5091f llite: DIO fallback on compressed files Direct I/O on compressed files is still not perfect for now, so we'd fall back to buffered I/O for DIO on compressed files. DDN-bug-id: EX-6266 Was-Change-Id: I2255638e4fbde766abf061989d07b37ad9f369a1 llite: Compute compression chunk ranges Determine the edges of any leading and trailing compression chunks touched by this IO and store them in the cl_io struct. The functionality in this patch also allows us to adjust the lock and read rounding to do them more intelligently, this will be done in a future patch. DDN-bug-id: EX-7601 Was-Change-Id: I526563ea347fb0246f97f3532b823c4345c3fa27 llite: round LDLM lock requests to chunk When we do IO with compression, we may need to 'fill' the compression chunk, reading up pages which have already been written to storage, so we can compress the whole chunk. Doing this safely requires that any dlmlock we're using always covers the full chunk. The easiest way to do this is to round the entire locking process to include leading or trailing compression chunks. DDN-bug-id: EX-7601 Was-Change-Id: I3c365844561d0da909e6290f4b58ef2211c2d255 csdc: is_chunk_start should return header copy In is_chunk_start() *ret_header = header; ... kunmap_atomic(header); ret_header is used after is_chunk_start(). The header copy should be returned from is_chunk_start() for safe work. DDN-bug-id: EX-9878 Was-Change-Id: Ib5e828d6b61e90dcd70c28589931a4490cf19c22 Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I9b41ab815db3df9ad7bdea5fca4c093cbda8814b
(minor) The comment above still says the acceleration factor is mapped "keeping 1-9 as-is", but from/to now apply a +1/-1 offset, so the encode and decode boundaries no longer line up at 9. from_lz4fast_level(9) returns 10 (the `level < 10` path, level+1), but to_lz4fast_level(10) takes the `>= 10` path and returns 11. So `lfs setstripe -Z lz4fast:9` reads back as level 11 via `lfs getstripe --compr-level` (8 round-trips, 9 does not). Was the boundary meant to be `level <= 10` on decode (or `level < 9` on encode)?
(style) These crypto headers are #include'd in the middle of the file, interleaved between the new static inline definitions, rather than grouped with the other includes near the top. If the patch is refreshed, consider moving them up. (There's also a stray double blank line at the end of the block.)
(typo) "Cyrpto" -> "Crypto" here and in the ll_crypto_acomp_set_level() comment below.
(defect) ll_crypto_comp_set_level() and ll_crypto_acomp_set_level() (and COMPR_LEVEL_MASK / the *_MOD_NAME macros) don't appear to be called or referenced anywhere in the tree. If nothing uses them yet, this looks like it belongs in the kernel patch that consumes them rather than in a utils level-mapping change.
(defect) This branch is entered when `level` is NULL (`!level`), but then dereferences it. The pre-patch code returned early for exactly the `!level` case, so a caller of this exported liblustreapi function that passes NULL for `level` (only interested in the type) now crashes. Both in-tree callers happen to pass non-NULL, but the guard still contradicts the body. Something like:
if (!level)
return 0;
if (!ptr) {
*level = -1;
return 0;
}
LU-10026 utils: update handling of compr level for lz4 The way lz4 compression level or acceleration factor is handled needs to be adapted in order to match what is provided by the lz4 userspace tool: - any level between 0 and 2 is interpreted as the default lz4 acceleration factor of 1; - any level from 3 and up to 16 is interpreted as a compression level for internal lz4hc. Increasing the compression level trades CPU time for improved compression ratio; - acceleration factor can be specified for the lz4fast compression type, from 1 to 26. This acceleration factor trades compression ratio for faster speed. Test-Parameters: forjanitoronly Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I4711217c1a6601f29f78d262567da5998f657fc9
(suggestion) This new list of currently supported types omits zstd (and fast/best), but the untouched line just above still advertises "1-21 for zstd" as a valid level range. If zstd is not currently supported, consider reconciling the two so the level-range hint doesn't contradict this list.
(defect) Is the lz4 default really 1? When no level is given, llapi_parse_compress_type() passes level = -1, and from_lz4_level(-1) returns LZ4HC_DEFAULT_CLEVEL - 1 = 8, which lfs getstripe maps back via to_lz4_level() to 9. So the effective default looks like 9 (highest ratio / slowest), not 1. The LZ4_ACCELERATION_DEFAULT = 1 case only applies when an explicit level < 3 is given, not for the unspecified default. Compare gzip just above, where the documented default of 6 does match from_gzip_level(-1).
LU-10026 man: document supported compression types Update lfs-setstripe to mention the currently supported compression types: gzip, lz4, lz4fast and lzo. Test-Parameters: forjanitoronly Test-Parameters: trivial Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I9b587516d27b882bec5855da4948f489e5a0041f
(minor) `off` is computed at the call site as vma->vm_pgoff<<PAGE_SHIFT and passed in, but never used in the body. Drop the parameter, or use it.
(suggestion) The block is keyed on the file open mode, so an fd opened O_RDWR that only creates a PROT_READ mapping is rejected with -EOPNOTSUPP even though it never writes the file (a common open-O_RDWR-then-mmap-read pattern). Since read mmap of compressed files is supported, would keying on the mapping's write intent (vma->vm_flags VM_WRITE/VM_SHARED) rather than the open mode let those read-only mappings through while still avoiding the write SIGBUS?
(minor) The message names cl_io_rw_init(), but this path calls cl_io_init().
(defect) These treat O_RDONLY/O_WRONLY/O_RDWR as bitmask flags, but the access mode is a 2-bit field. O_RDONLY is 0, so `fd_flags & O_RDONLY` is always 0, and for an O_RDONLY fd `fd_flags & O_RDWR` is also 0, so mmap_mode stays 0 (PROT_NONE) instead of PROT_READ.
Mask and compare instead:
switch (fd_flags & O_ACCMODE) {
case O_RDONLY: mmap_mode = PROT_READ; break;
case O_WRONLY: mmap_mode = PROT_WRITE; break;
case O_RDWR: mmap_mode = PROT_READ | PROT_WRITE; break;
}
Because of this, the `oSMc` case in test_460e maps PROT_NONE, so "readonly mmap should work" passes without ever exercising a readable mapping.
LU-10026 llite: allow mmap reads of compressed files mmap reads of compressed files work, so we should only block writes. We cannot block the actual fault operations because that will cause the application to get a SIGBUS, so we check the file open mode when we go to create the memory mapping. llite: disable mmap on compressed files disable mmap(2) on compressed files until well tested. DDN-bug-id: EX-6265 Was-Change-Id: I3464a03b16708edcd0692bc9db337eb8473ea047 llite: use CIT_MISC in check_compression ll_mmap_check_compression uses a CIT_FAULT io type and initializes it with cl_io_rw_init. This doesn't initialize the IO correctly and this sometimes results in the following crash during io_fini, because the ft_page pointer is initialized to something else by cl_io_rw_init: BUG: unable to handle kernel NULL pointer dereference at 000000000000004a RIP: 0010:cl_pagevec_put+0x9f/0x3a0 [obdclass] vvp_io_fault_fini+0x21/0x40 [lustre] cl_io_fini+0x7a/0x230 [obdclass] ll_mmap_check_compression+0x403/0x540 [lustre] DDN-bug-id: EX-7812 Was-Change-Id: Ib1f270f071370d6c045bb9d799ab5b7b41a6c6be Test-Parameters: forjanitoronly Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I636f398fd247ddcd153f94bf8116440540e8469c
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56ab | seen in 22 other reviews |
(minor) These CDEBUG() pairs print start_idx/end_idx before and after the rounding with no descriptive text, on the D_SEC (security) mask which is unrelated to readahead. Same at lines 813 and 816. They look like leftover debugging scaffolding; if kept, D_READA would be the right mask.
(defect) vui_ra_pages is initialized once per read syscall in vvp_io_read_start() and only reset at IO teardown, but ll_readahead() runs once per page (ll_readpage()->ll_io_read_page()->ll_readahead()). Won't this += accumulate across every not-uptodate page of a compressed read, progressively inflating the RA window enlarge at lines 806-808 and ria_end_idx_min at line 863? Also, the delta is taken from end_idx before the EOF clamp just below (lines 819-828), so vui_ra_pages can grow past the file size. ria_end_idx_min then feeds ll_read_ahead_pages() at lines 478-479, which raises ria_end_idx without an ria_eof guard - so readahead can extend past eof_index, the opposite of the "restrict readahead to eof" goal in the commit message.
LU-10026 llite: getting stripe info optimization ll_lov_getstripe_ea_info() is expensive call and should be avoided if possible. Let's use cached chunk size rather than get it from stripe info every time. llite: restrict readahead to eof Compressed file readahead rounding needs to come before readahead is limited to EOF. DDN-bug-id: EX-7601 Was-Change-Id: I4e9e7fe63301c08efcb05f170726735593a9431d llite: fix chunk_bits usage in readahead For the minimum compression chunk size, chunk bits is zero, so we cannot use if (chunk_bits) to determine if we're doing compression. This also fixes two other things: 1. A rounding error when rounding to chunk 2. Move rounding of end_idx to before first usage of end_idx, so calculation of number of pages is correct Without this, when the user reads 1 page or less, readahead will calculate the readahead page count as 0 and will exit without reading the chunk. DDN-bug-id: EX-6127 Was-Change-Id: I273506fd4f6ed5f0b8b5020357fd7caf0531e61c Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Id08487ec782f797e242e3f673c4a4dd8d526c9cc
The message (and the sanity-pfl/sanity test updates) cover only lzo, but the new error path also fires for zstd, since it likewise has no level mapping. Worth naming zstd here and adding coverage for it, or clarifying why zstd is excluded from the documented 1-21 level range.
(defect) This whitelists only fast/best to carry a level; every other type with a NULL ctn_from_compr_level falls through to the error. That is lzo (intended) but also zstd. So `lfs setstripe -Z zstd:5` now fails, whereas pre-patch it stored the level. lfs-setstripe.1:232 (added in this same LU-10026 series) says zstd takes a level 1-21. Is the zstd rejection intended? If zstd should not take a level, the man page needs updating too; if it should, zstd wants a from_compr_level mapping rather than the error path.
LU-10026 utils: error for invalid compression level Not all compression types accept a compression level (e.g. lzo in its kernel implementation). For those, return an error if a compression level is provided on 'lfs setstripe' command-line. Test-Parameters: forjanitoronly Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I6c367c4bfd76cc81c890a89ba9f994f4fd9f4f80
this is not helpful and needs to be updated
(minor) The body says decompress_request() has "a significant performance cost for checking if data is compressed", but it already returns immediately when the first page's cp_compr_type is LL_COMPR_TYPE_NONE. For uncompressed files that check is cheap; the scan cost only applies to compressed data, which is the case that still needs decompressing. Could the justification be tightened, and the read/write asymmetry this creates (writes still compress off the layout while reads stop decompressing) be noted?
(defect) This gates decompression on the global llite_enable_compression instead of the data's own cp_compr_type.
decompress_request() already returns early when pga[0]'s cp_compr_type is LL_COMPR_TYPE_NONE:
type = clpage->cp_compr_type;
/* no compression */
if (type == LL_COMPR_TYPE_NONE)
RETURN(0);
so uncompressed reads were never paying for a full scan.
The concern is a genuinely compressed read reaching here with the tunable off. lsm_lmm_verify_v1v3() only rejects a fresh compressed-layout unpack when compression is disabled; a cached lsm survives a runtime toggle. On read, set_page_compression()->__set_page_compression() still fills cp_compr_type from that cached lsme, and the write path still compresses off cp_compr_type regardless of the global.
After echo 0 > ...enable_compression on a client that already holds the compressed file's layout, does a cache-cold read here hand back the still-compressed bytes undecompressed?
LU-10026 osc: don't do decomp. with compression disabled decompress_request likely has a significant performance cost for checking if data is compressed, so we should not call it when compression is disabled. This is a stop-gap solution for the preview until we can have the server tell the client if data is compressed, as described in EX-7818. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Iba8feba3ab0fe620d0594f59c2c6ddea25faeb4f
The commit message only describes disabling fallocate, but the diff also adds a `rsync --sparse` decompression round-trip (the `decomp2` / "fiomap case" block) to compress_type() in sanity.sh. That change is about the sparse-copy/FIEMAP read path, not fallocate, and isn't mentioned here. Is it meant to be part of this patch, or should it be split into its own change with its own Change-Id?
(suggestion) This only checks lsm_entries[0]. lsme_compr_type is set per component (lov_ea.c copies lcme_compr_type for each LOV_PATTERN_COMPRESS entry), so a composite/PFL file could have an uncompressed component 0 and a compressed later component. Would fallocate over such a file still be allowed even though it targets a compressed component? (The ll_fiemap() guard has the same entry[0]-only limitation.)
(minor) `lio` and `loo` here just re-derive values already in scope: the outer `lio` (declared at the top of the function) is the same `cl2lov_io(env, ios)`, and `loo` is `lio->lis_object`. The inner `struct lov_io *lio` also shadows the outer one. This could just use the existing `lio->lis_object` without the extra declarations.
(minor) fallocate(2) is directly user-triggerable, so a CWARN here lets an unprivileged caller flood the console by repeatedly calling fallocate on a compressed file. The equivalent compressed-file guard in ll_fiemap() (lov_object.c around line 2021) returns -EOPNOTSUPP silently. Consider CDEBUG(D_VFSTRACE, ...) or a rate-limited message instead.
LU-10026 lov: fallocate is not allowed for compressed files Client Side Data Compression allocates blocks after a compression. It is impossible to preallocate blocks for the whole file, so fallocate should be disabled in case of compression. Test-Parameters: forjanitoronly Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ie834ace183fdcec0d7d6f747237e0964c3c4120b
LU-10026 llite: restrict overwrites during preview EX-7601 is an issue where when modifying a compressed file we do not correctly read-up and re-write existing compressed data. To avoid this, we can only allow writes which are not aligned to compression chunk size when they are not overwriting existing data, ie, when they are extending the file. This returns EINVAL for all writes to compressed files which are not either chunk aligned or extending the file. This should prevent users from hitting the data corruption issue but still allows some basic usage. This is intended just for the preview period. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I0eea01e2249866a074afd0d0642fe6dce9a49664
(style) The body explains why compression must be disabled for encrypted files, but that logic (compressed = 0; goto skip_compression) already lives in the parent compression patch. What this change actually does — throttling the existing CWARN to once per 24h and adding `always_except LU-10026 66` in sanity-sec.sh — isn't mentioned. Could the message describe the change it makes?
(minor) There is no test_66 in sanity-sec.sh (tests jump 65 -> 68), so this exclusion is a no-op in this tree. Which patch adds sanity-sec test_66, and should the always_except land together with it? Worth also confirming sanity-sec is the intended suite rather than sanity-compr.
LU-10026 sec: disable compression for encrypted files In case a read-modify-write io pattern is carried out on a compressed file, it has to be handled on server side. But because encryption cannot be done on server side for security reasons, we are not able to handle that kind of io pattern if the file is encrypted + compressed. So just disable compression for all encrypted files. Test-Parameters: forjanitoronly Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I84881fb1235f015d022751d4cce2d43a7231c2b4
(defect) `req` was just allocated above (ptlrpc_request_alloc_pool for writes, ptlrpc_request_alloc for reads) and every other error path in this function frees it with ptlrpc_request_free(req) before returning. These two early returns leave without freeing it, and *reqp is only assigned on the success path, so the caller can't release it either:
if (compressed && !page_access_allowed)
RETURN(-EINVAL);
if (inode && IS_ENCRYPTED(inode) && !page_access_allowed)
RETURN(-EINVAL);
For OST_WRITE the request comes from osc_rq_pool, so a GPU-direct write to a compressed or encrypted file leaks a pooled request each time and can eventually drain the pool. Should both routes free req first?
LU-10026 osc: disable CPU-access features for RDMA only pages Pages which cannot be accessed by the CPU are referred to as RDMA only pages. If pages cannot be accessed by the CPU, it is impossible for us to do compression, encryption, checksums, or short-io (data-in-RPC) on them. This patch disables compression and encryption for these pages and cleans up the code so checksums and short-io are disabled by the same code. The only user of RDMA only pages today is Nvidia's GPU direct, so this patch disables compression and encryption with GPU direct. NB: We eventually intend to handle compression for GPU direct with server side compress/decompress. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Iad9311617cddf27d3ff75a17429499c573067ea0
(minor) The body explains the DoM+compress block, but two hunks pull the other way and aren't mentioned: lov_pattern_available() gains LOV_PATTERN_MDT | LOV_PATTERN_COMPRESS, and llapi_pattern_to_lov() gains MDT|COMPRESS and RAID0|COMPRESS cases. Both make the combination more acceptable, not less. Could the message cover why they belong in a patch whose stated purpose is to disallow it?
(defect) This makes the server accept an MDT|COMPRESS component (lod_lov.c / lov_ea.c call lov_pattern_available()), yet lov_pattern_supported() still excludes it, so client IO marks such an entry invalid (lov_object.c sets lle_valid = 0). The only guard against creating it lives in lfs.c; a liblustreapi caller doing llapi_layout_pattern_set(MDT) + llapi_layout_compress_set() bypasses that and produces a layout the server stores but IO cannot use. If DoM+compress is unsupported, should the block live in the library/server rather than only the CLI?
(defect) errno is set to a negative value here; every other error path in comp_args_to_layout() uses errno = EINVAL. A negative errno breaks strerror(errno)/perror on this -1 return. Should be errno = EINVAL.
(minor) These two cases look unreachable. The COMPRESS bit never reaches llc_pattern: llapi_layout_pattern_set() rejects any pattern carrying it, and llapi_layout_from_lum() stores compression in llc_compr_type, not the pattern. The bit is OR'd into lmm_pattern separately in llapi_layout_to_lum() (blob->lmm_pattern |= LOV_PATTERN_COMPRESS). Was a real caller intended, or can these be dropped?
LU-10026 csdc: handle data compression on MDT Do not support setting data compression component on DoM until data compression on MDT implemented. DDN-bug-id: EX-7806 Was-Change-Id: I3794460140f08a073377c418dd56e7dda907d96d Test-Parameters: forjanitoronly Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I8b5c4d5fd18033e796b1c7555c37251e4f7f9a49
(minor) The diff also touches Documentation/man1/lfs-mirror-extend.1 (documenting compression options for SETSTRIPE_OPTIONS), which the message doesn't mention and which is unrelated to the migrate fix. Is that hunk meant to be part of this patch, or should it be split out / called out in the body?
(minor) This prints `fd`, which is always -1 on this path, so the message reads "Cannot open file X: -1" with no useful detail. The error is in errno from open() – strerror(errno) would be the informative value. Also note the caller (lfs_migrate/lfs.c) already emits its own llapi_error() when this returns < 0, so this adds a duplicate line. Using llapi_error(LLAPI_MSG_ERROR, errno, ...) here (as the ENOSPC path below does) would decode errno and respect the message level instead of writing to stderr unconditionally.
LU-10026 lfs: migrate compressed file without stripe info lfs migrate file without specifying stripe info will get layout info from the file as the target layout template, and llapi_layout_get_by_xattr() tries to convert LOV_PATTERN_* values to user scope LLAPI_LAYOUT_* values, while LOV_PATTERN_COMPRESS is missed in this conversion. This patch add a function llapi_pattern_from_lov() to handle this conversion specifically. This patch also add more error messages for llapi_layout_file_open(). Test-Parameters: forjanitoronly Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I49a43cc7761cd2baed7a5da7d4e7cff2152ff9bb
| unique failing test | history |
|---|---|
| racer@ldiskfs+DNE:test_1 | seen in 44 other reviews |
| sanity2@ldiskfs+DNE:test_160h | seen in 2 other reviews |
| sanity-pcc@ldiskfs+DNE:test_20 | seen in 23 other reviews |
(minor) This is an on-wire change: the client now fills bytes in the obdo (formerly o_padding_4) that are sent to the OST/MDS. Test-Parameters: forjanitoronly doesn't exercise interoperability. Consider requesting old-server/new-client and new-server/old-client runs with serverjob/serverbuildno and clientjob/clientbuildno so the padding-reuse stays wire-compatible with unpatched peers.
(minor) This comment reword looks unrelated to sending compression info to the server; is it meant to be in this patch? Also the code computes the chunk size with COMPR_CHUNK_MIN_BITS (COMPR_GET_CHUNK_SIZE()), not LOV_MIN_STRIPE_BITS; they happen to both be 16, but referencing the constant the code actually uses would be clearer.
This isn't a bug, but after this patch lov_lsm2layout() has no remaining callers (both lov_io.c and lovsub_object.c now call lov_lsm2layout2()). Folding the olc parameter into lov_lsm2layout() and updating those two call sites would avoid the version-suffixed name and the now-dead wrapper.
LU-10026 csdc: sending compression info to server Client fills in layout compression info into obdo and passes it to server. Test-Parameters: forjanitoronly Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ieb5d7b3609da41f35f8622ed6116f19ce7567ddb
LU-10026 csdc: prohibit set compression upon encrypted file Setting compression layout component upon encrypted file is not allowed for now. This patch add this check on MDS in creating file with layout, adding/merging new mirror to existing file. Test-Parameters: forjanitoronly Test-Parameters: testlist=sanity-sec env=ONLY=67,PTLDEBUG=-1 Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I60d9f4bfce3a498f1eb3994c6276afb9d89c99a7
| unique failing test | history |
|---|---|
| sanity-lfsck@zfs:test_18c | seen in 25 other reviews |
| sanity-scrub@ldiskfs+DNE:test_9 | seen in 5 other reviews |
This is false.
These modules do not compile on newer kernels (6.18, at least). I think it should be possible to build without these.
just as the smatch implies this should really be IS_ERR, I don't think filp_open can return NULL?
LU-10026 lustre: add lz4 and lz4hc kernel modules lz4 and lz4hc kernel modules implement compression according to the lz4 and lz4hc algorithms respectively, through the kernel Crypto API. lz4 module provides 2 cipher drivers under the generic name 'lz4': * lz4-lustre-generic of type compression * lz4-lustre-scomp of type scomp lz4hc module provides 2 cipher drivers under the generic name 'lz4hc': * lz4hc-lustre-generic of type compression * lz4hc-lustre-scomp of type scomp lz4 and lz4hc kernel module sources are copied from linux v6.1-rc5, and renamed to llz4.c and llz4hc.c respectively to avoid name collisions. Use of vmalloc has been changed to kvmalloc since it is faster in most cases. They implement the Crypto API interface, and rely on the lz4/lz4hc kernel library for compression implementation. They have been modified to grok a compression acceleration/level, as read from the top 4 bits of the crypto_tfm flags, and pass it to the underlying library. The lz4/lz4hc library sources are also copied from linux v6.1-rc5 and built statically, so lz4_compress, lz4_decompress and lz4hc_compress sources have been "de-modulified", and EXPORT_SYMBOLs removed. Headers have also been copied from linux v6.1-rc5 for consistency, and source files modified to include the copied headers instead of the system headers. All aforementioned sources are located in the lustre_compat/crypto/lz4 directory. The lz4/lz4hc modules are built only if the kernel does not provide them. This is verified by checking if the CONFIG_CRYPTO_LZ4 and CONFIG_CRYPTO_LZ4HC kernel config options are defined. This patch provides unit testing of several compression modules, such as lz4, lz4hc, lzo and gzip. This is done via a new test kernel module kcompr.ko, and new sanity test_84. File lustre/tests/kernel/kcompr.c contains examples of how to call the compression/decompression routines. EX-7998 lustre: Fix build with crypto_tfm_ctx on kernel 6.2 In recent kernels, commit e634ac4 "crypto: api - Add crypto_tfm_ctx_dma" moved crypto_tfm_ctx into algapi.h. So this file must be included from lustre_crypto.h. Was-Change-Id: If10f721355da3e58ca541c17615e978334e8d718 EX-7683 utils: always try to use our own lz4/lz4hc lz4/lz4hc provided by the kernel do not grok a compression level. The built-in lz4/lz4hc do, so always build them as dedicated kernel modules llz4.ko and llz4hc.ko, with the same .cra_name but with a slightly higher .cra_priority = 110, so that they are preferred over the in-kernel modules if any. And try to manually load the llz4/llz4hc kernel modules when a file requires compression with the corresponding alg. This is a "one-shot" try that allows us to prefer our modules that has level support, but continues to at least compress/decompress files even if our own modules are not available. Was-Change-Id: I0bdf267f998e21df81e460250a653aed34e3215d EX-bug-id: EX-6275 Test-Parameters: testlist=sanity env=ONLY=84 Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I74ce95ff18194c6052d291588d7e8c79596a7f23
We aggregate attributes from multiple objects for other reasons (e.g. maxbytes, timestamps, size, etc.). The current implementation looks "mostly OK" and will work for basic linear IO patterns, where the file transitions from non-rotational to rotational. However, any other kind of IO pattern (e.g. HDF5 where some writes are at the beginning of the file and others at the end) might have a serious issue? IMHO, it would be better to aggregate this information once from the object (maybe from the OSC's the object is allocated on) and then be done with it? Storing a threshold when the file transitions from non-rotational to rotational would be best, as this could be computed once based on the layout. It is fairly unlikely that a file would have HDD stripes in the middle and flash at the beginning and end. Alternately (probably better) is to store the nonrot state in each layout component (which is a perfect 1:1 mapping), and then this can be checked at IO submission time to see what type of storage it is covering.
Hm, OK. I don't think we can do this as easily as we'd like - the issue is we have to have this information very early. I can consider this, though, your points are good ones. The good thing is this isn't essential to merging the core feature, particularly since we're leaving it off by default for 2.16.
Actually, an OST is (generally) only going to be rotational or non-rotational, and this is already returned to the client via statfs, so the clients should have full knowledge whether any IO is on flash or disk.
LU-13802 llite: add file nonrotational check This patch adds the ability to note whether or not the last IO to a file hit a rotational or a non-rotational device. This will be used in selecting the cutover thresholds for hybrid IO, since DIO is synchronous and much slower on spinning disk. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I75a9970f91b1776ed6f04ac0d000a9ba576df75a
(defect) This version check should be `2.15.8`. Test 9 was never run for this patch as it was skipped.
More strictly, it should be "> 2.15.8" because the 2.15.8.0 release didn't have this fix either. Any patch built on b2_15 will be at least 2.15.8.1, so that test will pass. More importantly, this except file is only going to be used for interop testing with *older clients* running with this on the server, and we need this `mmp.ex` file **on master as well**, so that unpatched/older clients will also skip the test when running against master servers (which is where most of the testing is being run).
LU-19784 tests: mmp/test_8 wait time changes To prepare for the e2fsprogs/libext2fs changes reducing the wait time when the MMP sequence number is EXT4_MMP_SEQ_CLEAN, LU-19784 ext2fs: mmp wait time ajustments https://review.whamcloud.com/63478 We need to make sure the sequence number is not EXT4_MMP_SEQ_CLEAN before running e2fsck. We can use debugfs to change the MMP sequence number, rename mark_mmp_block() to set_mmp_seq() to reflect this. Remove mmp_mark.sh, which is not needed any more, and use set_mmp_seq() in test_9. Make sure we tear down the dm flakey device in test_9 otherwise during mmp_fini()->disable_mmp() will corrupt the filesystem. Lustre-change: https://review.whamcloud.com/63488 Lustre-commit: ad051b359db12ef7e752abcf2aa8e34199ded041 Test-Parameters: trivial testlist=mmp Signed-off-by: Li Dongyang <dongyangli@ddn.com> Change-Id: I81b9d28061862a99b04fa92de0cc18a536c9d17b Reviewed-by: Artem Blagodarenko <ablagodarenko@thelustrecollective.com> Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
LU-10026 osc: clear oi_write_osclock in lock fini func Move osc_io::oi_write_osclock clearance in osc_lock_fini() as it's set in osc_lock_init(). Compression IO could possibly expand lock region and osc_lock_set_writer() could access a osc_io that is not accessed in osc_io_iter_init(), so that osc_io_rw_iter_fini() miss clearing osc_io's oi_writer_osclock. This patch moves the oi_write_osclock clearance in lock fini function to match its creation in osc_lock_init(). Test-Parameters: forjanitoronly Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ied42f5befc1abd76aa10a7666eadb9a58e1f1783
LU-10026 llite: skip hole/data lseek() on compressed file Doesn't execute real lseek() on compressed file with SEEK_HOLE/SEEK_DATA origin but consider file always non-sparsed and do generic lseek() only test: get size before fallback to generic If ll_file_seek() falls back to generic_file_llseek_size() then make sure inode size is reliable DDN-bug-id: EX-7786 Was-Change-Id: Ieb8c90e66fb19675ba41f0147c0d9cdaf29ea20a Test-Parameters: forjanitoronly Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I2cd587cc0205e85758e06bbaafafe0e2959e0ade
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
LU-10026 lustre: add lgzip kernel module lgzip kernel module implements compression according to the deflate/zlib algorithm, through the kernel Crypto API. It provides 2 cipher drivers under the generic name 'deflate': * deflate-lustre-generic of type compression * deflate-lustre-scomp of type scomp Note the 'deflate' name is identical to the in-kernel module, but lgzip registers it with a slightly higher .cra_priority = 110, so that it is preferred over the in-kernel module. Our 'deflate' is also different in that it accepts a compression level as explained below. lgzip kernel module sources are copied from linux v6.2-rc5 and renamed to gzip.c to avoid name collisions. It implements the Crypto API interface, and rely on the deflate/zlib kernel library for compression implementation. It has been modified to grok a compression level, as read from the top 4 bits of the crypto_tfm flags, and pass it to the underlying library. The deflate/zlib library sources are also copied from linux v6.2-rc5 and built statically. Headers have also been copied from linux v6.2-rc5 for consistency, and source files modified to include the copied headers instead of the system headers. All aforementioned sources are located in the lustre_compat/crypto/gzip directory. The lgzip module is always built with Lustre. This patch enhances the test kernel module kcompr.ko to exercise the compression level of the provided 'deflate' module. It also tries to manually load the lgzip kernel module when a file requires compression with the 'delfate' alg. This is a "one-shot" try that allows us to prefer our module that has level support, but continues to at least compress/decompress files even if our own module is not available. EX-bug-id: EX-6206 Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I10b0ec03cf7d0e4fe3dd2898e77f357667209755
courtesy of gerrit not accepting test results at times you did not get your early warning. This introduces a sleeping under spinlock right at server init time and so every test has it https://testing.whamcloud.com/gerrit-janitor/63997/results.html [ 817.527107] alg: No test for zstdfast (zstdfast-lustre-generic) [ 817.534750] BUG: sleeping function called from invalid context at include/linux/sched/mm.h:226 [ 817.541176] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 26101, name: cryptomgr_test [ 817.550885] CPU: 3 PID: 26101 Comm: cryptomgr_test Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #2 [ 817.559137] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 817.565102] Call Trace: [ 817.566955] ? dump_stack+0xbb/0x10e [ 817.575504] ? ___might_sleep.cold.92+0xd9/0x107 [ 817.580904] ? __might_sleep+0x59/0xc0 [ 817.583799] ? slab_pre_alloc_hook.constprop.64+0x11f/0x1d0 [ 817.590298] ? kmem_cache_alloc_node_trace+0x5e/0x470 [ 817.595682] ? __get_vm_area_node+0xd6/0x220 [ 817.599009] ? __vmalloc_node_range+0x79/0x3c0 [ 817.603765] ? __zstd_compress+0x100/0x210 [lzstd] [ 817.606239] ? __vmalloc_node+0x4e/0x90 [ 817.608884] ? __zstd_compress+0x100/0x210 [lzstd] [ 817.615296] ? vzalloc+0x4e/0x60 [ 817.616509] ? __zstd_compress+0x100/0x210 [lzstd] [ 817.620607] ? zstd_scompress+0x32/0x60 [lzstd] [ 817.625250] ? scomp_acomp_comp_decomp+0x129/0x200 [ 817.627994] ? scomp_acomp_compress+0x17/0x20 [ 817.633796] ? test_acomp+0x2cc/0x5a0 [ 817.635204] ? cryptomgr_probe+0x150/0x150 [ 817.638300] ? alg_test_comp+0xc8/0xf0 [ 817.640295] ? alg_test+0x1f0/0x390 [ 817.642645] ? do_raw_spin_unlock+0x75/0x190 [ 817.646249] ? _raw_spin_unlock_irq+0x12/0x50 [ 817.648952] ? finish_task_switch+0x125/0x560 [ 817.651578] ? __schedule+0x369/0xcb0 [ 817.656565] ? do_raw_spin_unlock+0x75/0x190 [ 817.660800] ? cryptomgr_probe+0x150/0x150 [ 817.663411] ? cryptomgr_test+0x37/0x70 [ 817.665939] ? kthread+0x1d1/0x200 [ 817.669914] ? set_kthread_struct+0x70/0x70 [ 817.673595] ? ret_from_fork+0x1f/0x30 [ 817.689553] alg: No test for zstdfast (zstdfast-lustre-scomp)
LU-10026 lustre: add lzstd kernel module
lzstd kernel module implements compression according to the zstd
algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'zstd':
* zstd-lustre-generic of type compression
* zstd-lustre-scomp of type scomp
And 2 cipher drivers under the generic name 'zstdfast':
* zstdfast-lustre-generic of type compression
* zstdfast-lustre-scomp of type scomp
Note the 'zstd' name is identical to the in-kernel module, but lzstd
registers it with a slightly higher .cra_priority = 110, so that it is
preferred over the in-kernel module.
Our 'zstd' is also different in that it accepts a compression level
as explained below. 'zstdfast' is handling the 'negative' compression
levels, interpreted as an acceleration factor.
lzstd kernel module sources are copied from linux v6.9. This module
implements the Crypto API interface, and rely on the zstd kernel
library for compression implementation. It has been modified to grok
a compression level, as read from the top 4 bits of the crypto_tfm
flags, and pass it to the underlying library.
The zstd library sources are also copied from linux v6.9 and built
statically. Headers have also been copied from linux v6.9 for
consistency, and source files modified to include the copied headers
instead of the system headers.
In order to maximize compatibility with various distributions, we copy
xxhash sources from linux v6.9, and link these statically.
All aforementioned sources are located in the
lustre_compat/crypto/zstd directory.
The lzstd module is always built with Lustre.
This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'zstd' module.
It also tries to manually load the lzstd kernel module when a file
requires compression with the 'zstd' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.
EX-9678 csdc: defer memory allocation in zstd/zstdfast
Zstandard normally proceeds to memory allocation for its context
(workspace) while in the init phase, which corresponds to a call to
crypto_alloc_comp() when used via the Crypto API.
At this stage we cannot be aware of the compression level or the chunk
size. So we have to make zstd allocate a workspace that can fit any
level/chunk. And the higher the level is, the larger the workspace
needs to be. Problem is zstd is very memory eager, and by default for
the higher compression level it requires a workspace of 650 MiB. This
is insane compared to our usual compression chunk sizes.
So we defer workspace initialization to the first time zstd actually
compresses data. This corresponds to a call to crypto_comp_compress()
when used via the Crypto API. And by that time we know the compression
level and chunk size, which enables us to make zstd allocate a finer
grained workspace, that just fits our actual needs. This saves a lot
on memory allocation, as can be seen from these values (workspace size
expressed in bytes):
no src hint 64kB src
level 1 cwksp size 582392 303864
level 2 cwksp size 779000 402168
level 3 cwksp size 1303288 664312
level 4 cwksp size 2614008 1057528
level 5 cwksp size 3662584 1057528
level 6 cwksp size 3662584 1057528
level 7 cwksp size 6808312 1057528
level 8 cwksp size 6808312 1057528
level 9 cwksp size 13099768 1057528
level 11 cwksp size 25682680 1319672
level 13 cwksp size 34071288 1468856
level 15 cwksp size 67625720 1791095
level 17 cwksp size 50997688 1791095
level 19 cwksp size 85196598 1791095
level 21 cwksp size 336854838 1791095
level 22 cwksp size 680943414 1791095
And of course because we reduce contention on memory a lot, we improve
performance when zstd is used via the Crypto API, which is done in
osc_brw_prep_request() for ever RPC that involves a file compressed
with zstd.
Was-Change-Id: I470331ea0f2e550761ce7223e7be7e1373701bd2
Was-Change-Id: Idf7382093feec21bed5b6e62001a085a386736d2
EX-bug-id: EX-9678
Test-Parameters: testlist=sanity env=ONLY=84
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I113345cd10f41197683bfe127d373b8628364f88
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_127f | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-5 crashed | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
(typo) ".patch" Writes for fscrypt files *should* always be full-block writes? Why/how would writes to fscrypt files be partial blocks? The client should be sending a full encrypted 4KiB block, but then set the size on the inode without truncating it.
Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext4 for fscypt with write through (direct I/O) mode. Although writes from clients are always full-block writes, but the size calls via lnb offset + len may be not full-block aligned, we use this size for the actual size of the file?
LU-12916 osd: use writeback for small writes in ldiskfs This patch implements the writeback caching for small writes on Lustre OSD (ldiskfs). The writes are not synchronous any more on OST. It marks the page as dirty and create buffer heads mapping pages to the physical blocks. The kernel writeback mechanism will handle the actual I/O. This patch should improve the performance for I/O cases such as small writes from many files (i.e. mdtest-hard-write) or unaligned I/O on a shared file (i.e. ior-hard-write). Add a ldiskfs patch "ext4-writeback.patch" to not zero out non-full page during writeback for a encrypted file. Without this ldiskfs patch, it will fail sanity-sec/test_37. According to the benchmark results, this patch can improve the IO500 mdtest-hard performance and the performance of the real application Nek5000 turbulent pipe with more than 10%. Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I9b5aa568a49d5b39d04656fdacb94c3a6ec5a7ff
One thing that needs caution here is to "repair" a parent FID on an object that is actually (incorrectly) referenced by two MDT layouts. If the parent FID is missing/unset/corrupt, that is an easy case. If the parent FID is potentially valid, then the OST should contact the MDT owning the current parent FID to confirm or deny whether it has a layout referencing that OST object. I'm pretty sure that LFSCK is already fixing this case, so I'm wondering if it makes sense to push this OST object off to LFSCK in this case?
In this pfid repair case that is triggered through RPCs, I wanted to be conservative and only repair either unset or corrupt parent FIDs (that do not pass `fid_is_sane()`). This keeps the repair operation local to the OST. I can improve the commit message in this regard. It looks like the case you mentioned is handled by `lfsck_layout_check_parent()` and `LLIT_MULTIPLE_REFERENCED`. There is also `ofd_verify_ff()` which triggers the "inconsistency thread" and contacts the MDT. I initially considered it for checking if a parent FID needs repair but it is relying on the "lfsck_instance". I think it also contacts the MDT. In any case, this counters the idea of keeping this repair lightweight and local.
Having a more comprehensive commit message will help explain motivation and caveats in case there is an issue with this in the future.
warning: sparse:symbol 'ofd_can_repair_pfid' was not declared. Should it be static?
(defect) this should be fixed, or will fail compilation on some distros. It needs to be declared static (or declared in a header if it is used in other files, but that is not the case here)
OK, good. I didn't see this before my initial comment. I only saw the commit message and wanted to highlight this issue, but it seems you are already handling the common case. Should this be annotated with `likely(...)`? Is this called only for "repair" or on new objects also?
(style) you can use `[[ ... ~= "ids" ]]` to do a substring match
(style) instead of adding this to a single increasingly complex test case, also with separate version requirements, it seems better to add a separate test case for this?
LU-18756 ofd: Add PFID repair to resource ID repair The resource ID repair checks whether a UID/GID/PROJID is not set on an OST object yet. If all or some IDs are not set and the requirements are met, the IDs are updated asynchronously, e.g., as part of a read operation. This patch adds support for repairing the parent FID stored in the "trusted.fid" xattr field for OST objects which references the corresponding MDT inode. Repairing the PFID is done in the same transaction as the ID repair. It can therefore repair IDs, the PFID or both, depending on the current state of the object. As a fast path, the PFID is stored in the "ofo_ff.ff_parent" field in the "ofd_object", which is also used to check whether repair is required before loading the xattr, if the field is not yet set. Sanity-sec 75b is modified to exercise PFID repair in the three configurations: IDs-only missing, PFID-only missing, or both missing. Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Iafa97b1204a96ef3349009a340fd812528ae43ef
Shouldn't this RPC be using the "trusted" nodemap, so that it already skips ID mapping? I'm not totally against making this code more robust even without the "trusted" nodemap, but I wonder if this will find enough other issues that prevent it from being useful (e.g. RBAC rules to prevent access entirely).
Unfortunately, we have the same issue for trusted nodemaps because we get the MDS nodemap instead of the client nodemap from the export. So, the type of nodemap is not relevant here. The issue was the below ID mapping assumption that the incoming IDs represent the "CLIENT_ID" and must be mapped to the FS_ID. This is a fair assumption whenever the client sends RPCs directly to the OST, but it is incorrect when the MDS sends RPCs to the OST because, in this case, the MDS already mapped the "CLIENT_ID" to the "FS_ID" and sends the latter to the OST. Mapping the "FS_ID" again here with the "NODEMAP_CLIENT_TO_FS" type (assuming the same nodemap applies to both client and MDS) would squash it both for idmapping and offset cases. Back to the "trusted" nodemap case: This is the same issue. The MDS skips id mapping, and the OST actually receives the trusted "CLIENT_ID" from the MDS. However, the OST does not attempt mapping for the trusted client nodemap, but for the MDS nodemap. If the MDS is trusted, this would be fine. Otherwise, the IDs are also squashed. I verified this is an issue for trusted and untrusted nodemaps, and this patch fixes both.
I think what Andreas means is that servers are _supposed to_, if correctly configured, be in a "TrustedSystems" nodemap with admin=1 and trusted=1. So in this case, when the OSS processes the SETATTR request from the MDS, it would naturally keep untouched the ids the MDS put in the request. I guess this patch is still nice to have in order to protect against incorrect nodemap configurations that do not have that "TrustedSystems" nodemap. But maybe we could try to cover more cases by changing the way nodemap_add_member() fetches the nodemap whose reference is stored in the export. Today it uses the provided nid. But it could check for the connect flags on the export, as you are doing below for OBD_CONNECT_MDS. Assuming they are already set when nodemap_add_member() gets called, we could point to an in-memory only privileged nodemap (always created when modules load, like "server_nm") if we detect we are dealing with a server-to-server export. This way, even if the "TrustedSystems" nodemap is missing in the configuration, we would treat the server-to-server exchanges as privileged from a nodemap standpoint. Here in tgt_ost_body_unpack(), that would mean the call to nodemap_map_id() would keep ids as they are set in the request coming from the MDS.
I see, thanks for clarifying. Yes, in this case, this issue would not occur since mapping is skipped here. So, we could go more in the direction of protecting against such misconfigurations. I would agree that it would be better to enforce this globally (for server<->server communication) instead of on an individual RPC level. I will look into your idea, Sébastien, thank you.
I think the right approach for now would be to report an error at mount if there are nodemaps configured for the MDS and OSS but they have ID mapping enabled, and/or are not marked trusted.
LU-18835 target: Skip id mapping for OST_SETATTR from MDS When nodemaps are in use, the uid/gid tags on OST objects become squashed when "chown"-like operations are issued. Valid OST_SETATTR RPCs to modify the UID and GID on the OST objects are asynchrounously sent by the MDS (via OSP through "osp_attr_set()") after successfully applying the UID/GID to the MDT object. Here, the ID mapping is already done on the MDS and the (mapped) FS IDs are sent to the OSTs. This causes two issues: (1), The RPC target layer for OST RPCs however assumes that _all_ RPCs come from the client and maps the ids for "NODEMAP_CLIENT_TO_FS". And (2), when OST_SETATTR RPCs are sent from the MDS to the OSTs, the export is the calling MDS, which may resolve to a different nodemap than the client (e.g., "default"). Either issue causes an incorrect UID and GID which is then applied to the OST object. This patch fixes this issue by skipping the "NODEMAP_CLIENT_TO_FS" mapping on the RPC target layer for OST_SETATTR RPCs _if_ they originate from the MDS because, in this case, the UID/GID are already correctly mapped. Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Iffa7b1054c3b0858d4986351e2b86b03dd93d057
LU-18756 tests: sanity-sec 75a sync resource IDs
After sanity-sec 75a setups the test namespace, it creates and writes
to files before changing the user to the correct tenant user which
carries out the commands. Later, when the ID check is enabled, it
could happen that the corresponding OST object still has the initial
user ID set and that the "chown" changes have not yet been propagated
from the MDT to the OSTs. In such a case, the test fails the ID check
because the OST object has the wrong IDs set.
This patch adds an additional step after namespace creation to
synchronize ID changes to and between the servers before continuing
with the test.
Fixes: 719f6763fe ("LU-18756 sec: add resource id check to oss and mds")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=75
Test-Parameters: testlist=sanity-sec env=ONLY=75a,ONLY_REPEAT=100
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I07d602375fd34334dc5047cfecea60f09f03bd42
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lnet. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-13814 osc: add osc_dio_page_submit This patch adds the osc_dio_page_submit and specializes osc_page_submit to take only BIO pages. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I048bbcd25ffee28fbbd8fee26017796b5e3586bd
LU-0000 tests: sanity-ec sanity check Check whether sanity-ec can complete properly. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: If5bfe7bf33b28f8afb58473b7590a11c453c0659
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1002 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-sec. | session |
| custom-2001 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-sec. | session |
LU-20213 dbg: force timeout This patch attempts to reproduce LU-20213. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 serverjob=lustre-b2_15 serverbuildno=117 Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 serverjob=lustre-b2_16 serverbuildno=6 Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I8674a4a7bf061ed0a475826fbfb2a7b89d6d1487
LU-20213 dbg: revert patch Revert "LU-19763 socklnd: Check sendpage_ok for ZC" This reverts commit bf67df3212759c40b434c8182663f88ee23ad618. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 Test-Parameters: testlist=sanity-sec env=ONLY=47,ONLY_REPEAT=50 serverjob=lustre-b2_15 serverbuildno=117 Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Id54398db09df332e861b4c7cb9538f630bf98aca
LU-10026 osc: add compression to can_merge_pages Some BRW flags are OK to have on only some pages in a BRW, others are not. can_merge_pages has a whitelist of the flags which are safe to have on only some pages in a BRW, and prints a warning if other flags are seen. Add compression to the white list, because while all pages in an niobuf must be compressed, it is normal to have only some pages in a BRW compressed. Prior to this patch, this warning was printing during normal usage of compression. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ia262d4fc878e5328bd956865047e997aa77946f0
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56ab | seen in 21 other reviews |
(defect) The message describes a "sec: page_pools shrinker fix" (element_size / ppp_idle_idx / IDLE_IDX_MAX changes) and adds sanity-compr test_2000 gated on client 2.17.53 "for compression OOM fix", but there is no page_pools.c change anywhere in the diff, so test_2000 can't pass on this commit. A few hunks also aren't accounted for by the message: the ll_lov_getstripe_ea_info() switch to md_getattr() when filename is NULL in llite/file.c, and the LINVRNT->LASSERT change in cl_io_rw_init(). Are these meant to be part of this patch? Also, since this touches the OST_WRITE/OST_READ wire path (OBD_BRW_COMPRESSED, o_size/OBD_MD_FLSIZE), a Test-Parameters: line requesting client/server interop testing would be worthwhile.
(defect) this needs to add: ``` Test-Parameters: testlist=sanity-compr ``` otherwise that test session is not being run on master.
(defect) 0x418 is already used by OBD_FAIL_OSC_FIEMAP two lines above, so this fault injection point collides with it (triggering one fires the other). 0x41a or 0x420 appear free.
(defect) On the decompress path *type is llch_compr_type read from storage/the wire, so a corrupt header carrying LL_COMPR_TYPE_BEST or LL_COMPR_TYPE_FAST would LBUG the client here. The sibling change that replaced an assert with EIO ("we shouldn't assert on values read from storage") applies - return -EIO instead of asserting?
(minor) If fill_cpga() fails here, `dst` (obtained just above from obd_pool_get_objects) has not yet been stored in bp_cmp_chunk, and the out: path only releases `src`, so the chunk buffer looks leaked on this error return.
(defect) compressed_pages comes from llch.llch_compr_size, which is read straight out of the server reply by is_chunk_start() (that only checks llch_magic). It is bounded above only by page_count - i, not by pages_per_chunk. merge_chunk() then copies compressed_pages * PAGE_SIZE bytes into `src`, a single chunk_size buffer from obd_pool_get_objects(&src, buf_bits). A corrupt or hostile compr_size larger than one chunk overflows `src`. The LASSERT(src_size <= chunk_size) on the next line runs only after the copy (and asserting on wire data is itself unsafe). Can compressed_pages be bounded against pages_per_chunk before merging, returning -EUCLEAN otherwise?
(defect) On a resend, osc_brw_redo_request() re-enters this function with aa->aa_ppga. For a compressed write that array is the cpga built by compress_request()/fill_cpga() - bare brw_pages allocated with OBD_ALLOC_PTR and freed with a plain OBD_FREE, not embedded in an osc_async_page. brw_page2oap(pga[0]) then points before the allocation, so oap2cl_page() and the following clpage->cp_inode / cp_type / cp_compr_type reads touch invalid memory. -EINPROGRESS/recoverable resends are common, so this looks like a client crash on any compressed-write resend. Should the resend derive inode/compressed from aa_ncppga instead?
(style) This loop is indented with spaces instead of tabs.
LU-10026 osc: osc brw request compression This patch adds client-side compression/decompression. The client-side data compression project (CSDC) reduces storage and network utilization by leveraging the more plentiful memory and CPU resources on the local client. Data is sent compressed over the network, saved directly to storage on the server side, and decompressed back on the client side. Uncompressed data is kept in client page cache, all while being functionally transparent to the end user and application. As an example, a test file is compressed and decompressed. The resulting file is compared with the original one. The test case shows 2.5x compression ratio: 356K /mnt/lustre/d460.sanity/sanity.sh 884K /tmp/cmp-46ofie/decompressed_sanity.sh Compression should read whole chunk even if offset and size differ. Let's modify readahead to force reading data from the offset and size multiple to the chunk size. utils: fix cp_comp_type size cp_comp_type should be 8 bits, as llch_compr_type and all associated variables are declared as u8. So remove useless cp_comp_enabled and fix code to test for compressed component with cp_comp_type against LL_COMPR_TYPE_NONE. And update LL_COMPR_TYPE_MAX value to 255 to avoid conflicts with future compression types. DDN-bug-id: EX-7775 Was-Change-Id: Ia15868ac0ac003b62942540a57f782226ae8c141 tests: test compression without bzip2/HDF5 Run as much of sanity test_460a compression tests as possible, even if bunzip2 or HDF5 file are unavailable. Print a clear message in test_84 if bunzip2 unavailable. DDN-bug-id: EX-6127 Was-Change-Id: I36251834f636600eb9b0194ccd14c8b203da32e5 lov: refactor lov_io_lsme_at lov_io_lsme_at needs some minor changes to be called from lov_io_slice_init(). DDN-bug-id: EX-7601 Was-Change-Id: I0611d66052e22d349932eb26257369e07b9b8167 osc: don't check for start inside the chunk Chunk size is the same for the whole request and every chunk offset is multiple to a chunk size. No need to search for compression header in every page. It is enough to check every with offset multiple to a chunk size. DDN-bug-id: EX-7818 Was-Change-Id: Ie2ef645130656279e152ea1f7e6db01cb33836ca osc: minor compression cleanups This cleans up some style and argument issues I found made the code a little harder to follow. DDN-bug-id: EX-8270 Was-Change-Id: Ia3492ae79acf6c83d724cc91b0201c7872325853 osc: move common CSDC code to the library CSDC repacks a chunk on the server side in case of the partial rewrite. There are routines that can be shared between client and server. This patch moves common compression code to the libcfs. DDN-bug-id: EX-7601 Was-Change-Id: I824211a3435b0479f7a3b8f08598a5b567b67d3c osc: use correct count Using the number of bytes in the compressed page creates gaps in the RDMA, which IB memory registration cannot accept. Fix this by always setting count to PAGE_SIZE for compressed pages and otherwise using the count from the original source page. Setting PAGE_SIZE for compressed pages is valid because client only does compression for aligned IO, except for the trailing chunk. For the trailing chunk, the file size is set on the server, so any trailing bytes are ignored. DDN-bug-id: EX-8245 Was-Change-Id: Ied89d3ac328fb6020079392f5a8812ad5637b4a4 csdc: remove holes from struct ll_compr_hdr This patch reorganizes struct ll_compr_hdr to remove alignment holes. DDN-bug-id: EX-8353 Was-Change-Id: I59800b00e3a17972d621bae21ba06509a39b1036 osc: apply compressed flag to dst page The existing code to apply brw flags to compressed pages has two issues: 1. The dst_page is NOT an osc async page, it is a bare BRW page. This means the brw_page2oap macro isn't right, because there is no oap page. Because oap_brw_flags is actually oap_brw_page.flag, we don't ever access the memory pointed at by OAP, just use it to find an offset back in to the brw page. This means the flags are set correctly, but we still shouldn't use this macro. 2. However, the function then overwrites these flags by copying from a page in the source, so OBD_BRW_COMPRESSED is lost. Add OBD_BRW_COMPRESSED when we set flags. This ensures the flag is actually sent to the server on compressed IO. This was not causing any problems because the server does not actually use the OBD_BRW_COMPRESSED flag yet. (EX-7601 uses this flag) DDN-bug-id: EX-7601 Was-Change-Id: Ia94cdc803868ce16a0b66fd58578ec8b2d00cbae osc: remove unused 'wrkmem' compress_chunk() takes a wrkmem buffer, which it does not use. Remove this and its allocation in compress_request. DDN-bug-id: EX-7601 Was-Change-Id: I6f236f018f5b79c57cc8725ca0f95125810a4064 osc: walk chunk unaligned RPC correctly For decompression, the client must start looking for compressed chunks at a chunk aligned offset. Implement this in decompress_request. DDN-bug-id: EX-7601 Was-Change-Id: I3273135990ddf51e8b3c651734e19350e91f659c ofd: add obd level compression lib Some compression functions will be used by several areas of of Lustre, so they need to be in obdclass. This moves merge_chunk and unmerge_chunk there and adds the ability for them to merge lnbs. This is used in a future patch. DDN-bug-id: EX-7601 Was-Change-Id: If4a318119bb7685e41adb9f3b31a66074031e6ac osc: remove cpga fill bits cpga fill bits are not needed now that we don't support compression and encryption. DDN-bug-id: EX-7601 Was-Change-Id: I13c2278e085e9b288bd896585947e28e2ea505ca osc: only set compressed flag on compressed pages The code accidentally sets the compressed flag on all pages processed through fill_cpga, even if they're not compressed. Oops. Also stop setting pg->index on the pages in the compressed pga, this is only used by encryption and that's no longer supported with compression. DDN-bug-id: EX-7601 Was-Change-Id: I313fd943a18b71cd52493852a6884f30d187e52f osc: use pages_left in unmerge_chunk Since we have compressed chunks < chunk_size (if they're after EOF), we must use pages_left in unmgerge_chunk or it will go off the end of the page array. This also lets us remove the workaround where unmerge_chunk would skip pages that were not present. unmerge_chunk always works with a known and complete set of pages, so this check is unneeded. We should also check that our count of bytes is correct when we finish. DDN-bug-id: EX-7600 Was-Change-Id: I88896307990ff839514e54e9a7e18390a457e5d8 osc: rename 'done' Rename the ambiguous 'done' and remove it where not used. DDN-bug-id: EX-7601 Was-Change-Id: I8fb88b7a91fcc7dbd5ce2d29a61c18330fc0cda3 osc: cleanup compression variables Make usage of the compression variables more readable. DDN-bug-id: EX-7601 Was-Change-Id: I6daff56b56877c8f36e02303cc0579ba7faa731b ofd: make compress_chunk take chunk_bits Chunk bits is used everywhere, have compress_chunk convert to log bits rather than have the callers do it. DDN-bug-id: EX-7601 Was-Change-Id: Ic01bb749425cb95d9c5717965d692a18138ceeb7 obd: move module load to function This is a trivial code change to make alloc_compr a bit shorter. DDN-bug-id: EX-7601 Was-Change-Id: I0a790afe7afebde1d223420d9a578529da6ff7e5 osc: variable cleanup in decompress_req Use type and lvl variables in decompress_request. Remove an unused variable and an assert which can never fire. DDN-bug-id: EX-7601 Was-Change-Id: Ieff57411a2a41215fd368d731614801bd0f43e38 osc: replace assert with error We shouldn't assert on values read from storage, instead if they are incorrect, we should give EIO. DDN-bug-id: EX-7601 Was-Change-Id: Icda213e3c5a90a848c9b008788e92ee49e2efcb1 osc: rearrange compress_request A trivial rearrangement of compress_request to make it more readable before redoing the core logic. DDN-bug-id: EX-7601 Was-Change-Id: I1d34cd2a2a6d84bc30cc7dae8eb07586c4837f7d osc: give compress_request explicit success Compress_request has explicit failure handling, but the success handling just follows the failure handling. This is confusing - on failure, we do: page_count = *pcount then immediately do: *pcount = page_count It also sets *orig_pga = pga on success OR failure, which is wrong because compress_request may have modified pga and then failed. DDN-bug-id: EX-7601 Was-Change-Id: I121ec71cfe35babc4a572951e93f7581887ade80 osc: remove &pga usage in compress_request The usage of 'pga' and '&pga' in compress_request is confusing, but also, compress_request modifies &pga by allocating a new compressed page array. Except if we fail in compress_request, we free that new page array. This means failing in compress_request replaces 'pga' with a pointer to freed memory. Instead, create an explicit cpga pointer in the caller and use that. This allows compress_request to fail safely. DDN-bug-id: EX-7601 Was-Change-Id: Idaf592103c57b0e9ce76ab520a69b819d4f37be9 obd: move type switching to alloc_compr callers The code is much cleaner if we can eliminated applied type and handle that issue once per compression or decompression rather than for every chunk. This requires moving the type switching inside alloc_compr. (Also improve some error messages - alloc_compr can fail with ENOMEM as well.) The compression code currently allocates a transform for every chunk on the client. This is relatively cheap, but it also complicates the code by repeatedly checking if a particular compression type is supported (this is the "applied type" code). Moving alloc_compr to compress/decompress request makes the code much simpler. DDN-bug-id: EX-7601 Was-Change-Id: I162e81577db721a9715d57b3f262fcabbcbf308a osc: rename pages_in_chunk Chunks can have variable numbers of pages in them. DDN-bug-id: EX-7601 Was-Change-Id: If199d777367569e62c21305f6e4b9f3e4cce6d06 osc: allow multiple chunks in read It's rare, but reads can sometimes have multiple discontiguous chunks. Update decompress_request to handle this case. DDN-bug-id: EX-7601 Was-Change-Id: I880af95db285dce76db3610e8140a0f54baa401b ofd: do not overwrite rc in unmerge_chunk unmerge_chunk should not be responsible for setting the lnb rc, because this overwrites the result of any previous activity on the lnb. Plus, unmerge_chunk can't fail. DDN-bug-id: EX-7601 Was-Change-Id: Id1ce590c7f1da3ab7faddbd685d264a33c08d639 osc: calculate compressed size reduction accurately Compression reduces space used if it results in allocating at least one fewer block on disk. Modify the checks in compress_chunk to reflect this, rather than using the simpler "reduce size by at least 4K" calculation. Also do not attempt to compress chunks if they are less than 4K in size, since they can't possibly get a space benefit. This improved my measured ratio on a version of the Linux kernel source data set from 1.24 to 1.56, so this is significant for datasets with many small files. (This version of the source had large incompressible files removed, to focus on smaller files. The unmodified data set would not improve as much.) Note this is still short of our estimates, so either the estimate or Lustre still needs adjustment. TBD. DDN-bug-id: EX-7601 Was-Change-Id: I815706914b88de4f532a674d773769aa3a64d218 osc: rewrite compress_request The existing version of compress_request can't handle discontiguous RPCs. Rewrite the logic to handle this case properly. This also implements kms handling. If a write chunks ends at the known minimum size, we know this write is after all other data in the file and so there is no compressed data under it. This means we can compress this chunk. DDN-bug-id: EX-7601 Was-Change-Id: I8a912d9e279d04c8ff07de39e63a1ec9b490d921 osc: handle partial chunks in decompress_request Now that we have compression for incomplete chunks at the end of files, decompress_request needs to handle these chunks. This patch modifies it to understand compressed chunks which are less than chunk_size pages. DDN-bug-id: EX-7601 Was-Change-Id: I877550fa0d418def406e0308392a5336ec9f3ab6 osc: add check to decompress_request decompress_request should check to see if there's room in the RPC for the decompressed data, since this can occur if there's a bug or data corruption, and otherwise we will go past the end of the RPC during decompression. DDN-bug-id: EX-7601 Was-Change-Id: Ib1bf19bf39701b72f0f5a61b2aaff2f2fdad1897 osc: debug fix in decompress_request Debug message had an incorrect subtraction. DDN-bug-id: EX-7601 Was-Change-Id: I5daf360766ca77b98dc5af3d72c42ac38f5782bc obd: add 'lvl' for best and fast 'best' and 'fast' compression types must also set a level, because not all levels are supported by all algorithms. Rather than trying to be clever, just use simple universally supported values, except for lz4fast, where we special case this, because otherwise '0' is the slowest setting (and lz4fast is likely to remain our default fastest). DDN-bug-id: EX-6269 Was-Change-Id: I7c29659d4f027af2e44285ae38e4c9d91e35509a osc: decompress with algorithm from server Data may not be compressed with the compression type and level from the layout, so we must use the compression type and level from storage for decompression. DDN-bug-id: EX-6269 Was-Change-Id: Ib4cdccf294ef631a25147413d7f5c1a847c9504e osc: handle different compression types Allow the client to handle different compression types in a single component. This shouldn't happen normally, but it may happen in the future if there is dynamic compression algorithm selection for "fast" or "best" types (e.g. compress based on available CPU and network bandwidth or RPC backlog). DDN-bug-id: EX-6269 Was-Change-Id: Ide2731c60a68584e7cbb474bee88a17e9a7b8fec osc: add COMPR_GAP check to compress_request Currently, compress_request will build the compression buffer (calling merge_chunk()) for requests which are less than the minimum compression gap. This is noticed in the compression code when it checks if there's enough data to attempt compression, but we can do a trivial check in compress_request() to save that work. Also fix a few minor style things. This is not an important fix, but I discovered it while investigating another issue and it's trivial to resolve. DDN-bug-id: EX-7601 Was-Change-Id: Ieb32e6297e10d229f23c58e2ef4d933ce3dda4f2 lustre: add uncompressed size to compression header It's useful to have the uncompressed size of the data in the compression header. Also, we have three checksum fields - compressed, uncompressed, and header, but in practice, checksumming the compressed data including the header is enough to cover all of these. This patch cleans up all of this at the same time. DDN-bug-id: EX-8851 Was-Change-Id: Ie82e0dbe9c862ddc88999b109cea1f27577dbbff csdc: rename "cp_comp_*" to "cp_compr_*" This patch renames "cp_comp_type", "cp_comp_level", and "cp_chunk_log_bits" to use "compr" in the name to be consistent with other variable names. DDN-bug-id: EX-8353 Was-Change-Id: I428ff3a789b33da02832dee02f316b02d97137e2 osc: Do not iterate over chunk pages If osc_decompress() knows chunk size, no need to iterate other all pages in chunk. Thay can be skipped. DDN-bug-id: EX-7729 Was-Change-Id: Ib84b060075c55c97eba9f74ef017c0a956e85b12 csdc: Fix the upper mergeable chunk pointer If the full chunk is followed by un mergeable page, the upper mergeable chunk pointer is occasionally set to this unmanageable page. The chunk size is calculated wrongly then and the next condition suggest not to compress this chunk, because its size is not equal to the expected size. The pointer should be moved to the first instruction after the can_merge_pages(). DDN-bug-id: EX-9192 Was-Change-Id: I09fedc770c8bbcac4864b32372a941da5e0c7ac3 obdclass: reset bits after decompression as uncompressed data can be less than chunk/page, but still be visiable to userspace as a part of a sparse file. DDN-bug-id: EX-9873 Was-Change-Id: I4114b0704fb685013f4e03cf2d80ccde2cc8c87f osc: preserve compressed pages for OST_WRITE replay it's incorrect to release compressed pages right after reply as we may resend them during OST_WRITE replay. DDN-bug-id: EX-9895 Was-Change-Id: I3edc16d6556ddd60735d2f14fe879fc0f45231d7 csdc: Fix the next available algorithm selection Currently, if a chosen algorithm is not available, then next preferable is chosen, but an error code is not cleared so __alloc_compr() returns this wrong code. Data if written uncompressed while it can be compressed with next appropriate algorithm. This patch adds this error code clearing. Test is provided. DDN-bug-id: EX-9889 Was-Change-Id: I59f65058a0fe9b108de3d4ba7cf5950f18e32204 ptlrpc: reset refcount reusing pages for new req when a set of compressed pages are reused for a redo RPC, we have to reset refcounter. DDN-bug-id: EX-10061 Was-Change-Id: I16c46d857d60cf4dd0f6317190e7c99b2667e135 ptlrpc: drop extra reference to compressed pages if the request is not replayable, so brw_commit() won't be called. DDN-bug-id: EX-10184 Was-Change-Id: Ib29ab19b69fd15ef339cc18c5dfef17217a32cb1 osc: remove noisy warning if we can't compress data on the client side for a reason (e.g. too random), we just send data uncompressed, makes sense to hide the error message confusing customers: kernel: LustreError: 936336:0:(lustre_compr.c:345:compress_chunk()) exafs-OST0007-osc-ff49193c293f3800: Compression failed, type 5, lvl 5, -22 DDN-bug-id: EX-10593 Was-Change-Id: Icbc8be38b0372f8f67690e7d94384a16ab02cbe4 osc: fix osc_compress ENOMEM handling Send already compressed chunks in compress_request even if can't allocate memory for the next chunks. This patch also fixes statistic in case of ENOMEM. Without that write_chunks_incompressible hasn't been incremented in case of ENOMEM. Add sanity-compr_2002 to check write_chunks_compressible and write_chunks_incompressible in case of ENOMEM DDN-bug-id: EX-11035 Was-Change-Id: I3fc58ece6cf22582166ea77e4a217bc7a9efd77d csdc: remove buffer double sizes Pools now (POOL_ADD_PAGE is defined) allocate additional page for memory misuse debug, so no need to allocate double-sized buffers to prevent lz4 decompress problem. DDN-bug-id: EX-8276 Was-Change-Id: Iea04943ae0f95c8983c5ba7ec9a2c63003afe351 utils: fix LL_COMPR_TYPE wire checks Fix LL_COMPR_TYPE* wire checks to avoid duplication. The value of LL_COMPR_TYPE_UNCHANGED is also declared as 255 so that is does not conflict with other potential real compression types in the future. DDN-bug-id: EX-7775 Was-Change-Id: Iab9830f09f0778e1e1f3b1ea4c9878ce1017de8d sec: page_pools shrinker fix page_pool shrinker should consider ppp_free_pages as a number of memory regions with the size of 2^pool_order pages. Otherwhise it startis shrinking only when becomes too large, for example 2GB instead of 64MB. It often might be the reason of OOM especially when compressed files with different chunk-sizes have been written. In a such case several pools created for different chunk sizes might hold gygabytes of memory without any chance to release it. Fix element_size to return correct element size instead of number of pages. Because of that elements in page_pools had below sizes: order 0 = 4096 order 1 = 2 order 2 = 4 .... order 17 = 131072 This made possible to have elements with sizes 2,4,8,... bytes that have been allocated with OBD_VMALLOC. Now all sizes are miltiple of PAGE_SIZE: order 0 = 4096 order 1 = 8192 ... order 5 = 131072 Change the logic around ppp_idle_idx. 1. Recalculate ppp_idle_idx in __sptlrpc_pool_put_pages the same way as it done in __sptlrpc_pool_get_pages. It is possible that ppp_idle_idx might become 0 or very small (less than 10). If there is no new allocations it would be impossible to free anything from the pool during 40 seconds(CACHE_QUIESCENT_PERIOD) despite the large amount of free elements ready to shrinking. 2. In pool_shrink_count when ppp_idle_idx == IDLE_IDX_MAX, it means no one accessed pool for CACHE_QUIESCENT_PERIOD (40 seconds), i.e. we may release as much as possible pages. Earlier in a such case pool_shrink_count always returned 0. Another words it was impossible to shrink anything from the pool if there was no allocations for 40s. Add sanity-compr 2000 that reproduces the OOM problem on a client without this patch. DDN-bug-id: EX-10094 Was-Change-Id: I6b55ba67b0d21cdffdb57034e8e66063745f796e osc: Add BRW_COMPRESSED flag to reads We need to add the BRW_COMPRESSED flag to reads so servers can know if the client is able to decompress data. This lets servers decide if a client can be sent compressed data and the result won't be nonsense/corruption. This is important for future support of GPU direct, where the server will need to do the decompression. DDN-bug-id: EX-6269 Was-Change-Id: I36b5b73f983ce8f2e5297c3e9dc778a5eca54e6a osc: don't discard decompress_request error The error handling for decompress_request is unusual - non-zero returns are just discarded. And rc2 is just discarded. The read() doesn't fail or get a short read. Fix this so if decompression fails with an error. DDN-bug-id: EX-7807 Was-Change-Id: Idd01947c7375c9586a64f064dd6ee0ac2308ea86 osc: save compressed object size CSDC uses a sparse file feature. A client writes compressed data chunks to the original offsets so the same data is expected to be read from the same offsets. There are no writes after the last compressed chunk, so no "hole" after the last compressed chunk. Compressed file size (based on OST objects size) is smaller than the original on the "original last chunk size - compressed last chunk size" delta. Object size should be set to uncompressed size. This size is used to calculate file size and to remove the workaround of not compressing the last chunk in the file. DDN-bug-id: EX-7600 Was-Change-Id: I387c282e1cf788c3b8f6230ef555d73ffffe49c1 osd: save compressed object size on zfs "osc: save compressed object size" added means to transfer object size to the osd and added ldiskfs support. This patch adds saving objec size to the ZFS backend. Currently this fix submitted as separete patch, for testing purpouse, but can be marged to the main patch latter. DDN-bug-id: EX-7600 Was-Change-Id: I99e29e3f756a070b5f3cece12c4ca58f668a2ecf ldiskfs: fix detection of compressed extent The code in ldiskfs_map_inode_pages which detects a compressed extent checks the first lnb for that extent, but it's possible for some lnbs and not others to be compressed in a given extent, so we must check all of them. This occurs when multiple writes have been combined in to one RPC. If we don't detect compression correctly, we won't set the file size correctly and we'll get data corruption. DDN-bug-id: EX-7601 Was-Change-Id: I11d50bdc45c40d93bb1b829fcd930165b7626432 llite: DIO fallback on compressed files Fully supporting direct I/O on compressed files is tricky because we cannot pull the full chunk in to the page cache (because there is no page cache for DIO). So instead we fall back to buffered I/O for DIO on compressed files. This patch adds the check and a test for this. llite: allow aligned DIO with compression DDN-bug-id: EX-6127 DDN-bug-id: EX-7601 Was-Change-Id: I8224ef9b8ad1d912d8a11eccad37d3dff8dd8498 If a DIO is fully aligned to compression chunk boundaries, it is safe to do on a compressed file, so allow it. Was-Change-Id: If8fa3397c9424254538738f4d77f9f50d1c21129 lov: set IO property check in lov_io_init_composite Move compression IO check down to lov_io_init_composite(). DDN-bug-id: EX-9853 Was-Change-Id: I6d4ce3388d8eef788d31dabf4b3a5dd5eff5091f llite: DIO fallback on compressed files Direct I/O on compressed files is still not perfect for now, so we'd fall back to buffered I/O for DIO on compressed files. DDN-bug-id: EX-6266 Was-Change-Id: I2255638e4fbde766abf061989d07b37ad9f369a1 llite: Compute compression chunk ranges Determine the edges of any leading and trailing compression chunks touched by this IO and store them in the cl_io struct. The functionality in this patch also allows us to adjust the lock and read rounding to do them more intelligently, this will be done in a future patch. DDN-bug-id: EX-7601 Was-Change-Id: I526563ea347fb0246f97f3532b823c4345c3fa27 llite: round LDLM lock requests to chunk When we do IO with compression, we may need to 'fill' the compression chunk, reading up pages which have already been written to storage, so we can compress the whole chunk. Doing this safely requires that any dlmlock we're using always covers the full chunk. The easiest way to do this is to round the entire locking process to include leading or trailing compression chunks. DDN-bug-id: EX-7601 Was-Change-Id: I3c365844561d0da909e6290f4b58ef2211c2d255 csdc: is_chunk_start should return header copy In is_chunk_start() *ret_header = header; ... kunmap_atomic(header); ret_header is used after is_chunk_start(). The header copy should be returned from is_chunk_start() for safe work. DDN-bug-id: EX-9878 Was-Change-Id: Ib5e828d6b61e90dcd70c28589931a4490cf19c22 Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I9b41ab815db3df9ad7bdea5fca4c093cbda8814b
(minor) The comment above still says the acceleration factor is mapped "keeping 1-9 as-is", but from/to now apply a +1/-1 offset, so the encode and decode boundaries no longer line up at 9. from_lz4fast_level(9) returns 10 (the `level < 10` path, level+1), but to_lz4fast_level(10) takes the `>= 10` path and returns 11. So `lfs setstripe -Z lz4fast:9` reads back as level 11 via `lfs getstripe --compr-level` (8 round-trips, 9 does not). Was the boundary meant to be `level <= 10` on decode (or `level < 9` on encode)?
(style) These crypto headers are #include'd in the middle of the file, interleaved between the new static inline definitions, rather than grouped with the other includes near the top. If the patch is refreshed, consider moving them up. (There's also a stray double blank line at the end of the block.)
(typo) "Cyrpto" -> "Crypto" here and in the ll_crypto_acomp_set_level() comment below.
(defect) ll_crypto_comp_set_level() and ll_crypto_acomp_set_level() (and COMPR_LEVEL_MASK / the *_MOD_NAME macros) don't appear to be called or referenced anywhere in the tree. If nothing uses them yet, this looks like it belongs in the kernel patch that consumes them rather than in a utils level-mapping change.
(defect) This branch is entered when `level` is NULL (`!level`), but then dereferences it. The pre-patch code returned early for exactly the `!level` case, so a caller of this exported liblustreapi function that passes NULL for `level` (only interested in the type) now crashes. Both in-tree callers happen to pass non-NULL, but the guard still contradicts the body. Something like:
if (!level)
return 0;
if (!ptr) {
*level = -1;
return 0;
}
LU-10026 utils: update handling of compr level for lz4 The way lz4 compression level or acceleration factor is handled needs to be adapted in order to match what is provided by the lz4 userspace tool: - any level between 0 and 2 is interpreted as the default lz4 acceleration factor of 1; - any level from 3 and up to 16 is interpreted as a compression level for internal lz4hc. Increasing the compression level trades CPU time for improved compression ratio; - acceleration factor can be specified for the lz4fast compression type, from 1 to 26. This acceleration factor trades compression ratio for faster speed. Test-Parameters: forjanitoronly Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I4711217c1a6601f29f78d262567da5998f657fc9
(suggestion) This new list of currently supported types omits zstd (and fast/best), but the untouched line just above still advertises "1-21 for zstd" as a valid level range. If zstd is not currently supported, consider reconciling the two so the level-range hint doesn't contradict this list.
(defect) Is the lz4 default really 1? When no level is given, llapi_parse_compress_type() passes level = -1, and from_lz4_level(-1) returns LZ4HC_DEFAULT_CLEVEL - 1 = 8, which lfs getstripe maps back via to_lz4_level() to 9. So the effective default looks like 9 (highest ratio / slowest), not 1. The LZ4_ACCELERATION_DEFAULT = 1 case only applies when an explicit level < 3 is given, not for the unspecified default. Compare gzip just above, where the documented default of 6 does match from_gzip_level(-1).
LU-10026 man: document supported compression types Update lfs-setstripe to mention the currently supported compression types: gzip, lz4, lz4fast and lzo. Test-Parameters: forjanitoronly Test-Parameters: trivial Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I9b587516d27b882bec5855da4948f489e5a0041f
(minor) `off` is computed at the call site as vma->vm_pgoff<<PAGE_SHIFT and passed in, but never used in the body. Drop the parameter, or use it.
(suggestion) The block is keyed on the file open mode, so an fd opened O_RDWR that only creates a PROT_READ mapping is rejected with -EOPNOTSUPP even though it never writes the file (a common open-O_RDWR-then-mmap-read pattern). Since read mmap of compressed files is supported, would keying on the mapping's write intent (vma->vm_flags VM_WRITE/VM_SHARED) rather than the open mode let those read-only mappings through while still avoiding the write SIGBUS?
(minor) The message names cl_io_rw_init(), but this path calls cl_io_init().
(defect) These treat O_RDONLY/O_WRONLY/O_RDWR as bitmask flags, but the access mode is a 2-bit field. O_RDONLY is 0, so `fd_flags & O_RDONLY` is always 0, and for an O_RDONLY fd `fd_flags & O_RDWR` is also 0, so mmap_mode stays 0 (PROT_NONE) instead of PROT_READ.
Mask and compare instead:
switch (fd_flags & O_ACCMODE) {
case O_RDONLY: mmap_mode = PROT_READ; break;
case O_WRONLY: mmap_mode = PROT_WRITE; break;
case O_RDWR: mmap_mode = PROT_READ | PROT_WRITE; break;
}
Because of this, the `oSMc` case in test_460e maps PROT_NONE, so "readonly mmap should work" passes without ever exercising a readable mapping.
LU-10026 llite: allow mmap reads of compressed files mmap reads of compressed files work, so we should only block writes. We cannot block the actual fault operations because that will cause the application to get a SIGBUS, so we check the file open mode when we go to create the memory mapping. llite: disable mmap on compressed files disable mmap(2) on compressed files until well tested. DDN-bug-id: EX-6265 Was-Change-Id: I3464a03b16708edcd0692bc9db337eb8473ea047 llite: use CIT_MISC in check_compression ll_mmap_check_compression uses a CIT_FAULT io type and initializes it with cl_io_rw_init. This doesn't initialize the IO correctly and this sometimes results in the following crash during io_fini, because the ft_page pointer is initialized to something else by cl_io_rw_init: BUG: unable to handle kernel NULL pointer dereference at 000000000000004a RIP: 0010:cl_pagevec_put+0x9f/0x3a0 [obdclass] vvp_io_fault_fini+0x21/0x40 [lustre] cl_io_fini+0x7a/0x230 [obdclass] ll_mmap_check_compression+0x403/0x540 [lustre] DDN-bug-id: EX-7812 Was-Change-Id: Ib1f270f071370d6c045bb9d799ab5b7b41a6c6be Test-Parameters: forjanitoronly Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I636f398fd247ddcd153f94bf8116440540e8469c
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56ab | seen in 22 other reviews |
(minor) These CDEBUG() pairs print start_idx/end_idx before and after the rounding with no descriptive text, on the D_SEC (security) mask which is unrelated to readahead. Same at lines 813 and 816. They look like leftover debugging scaffolding; if kept, D_READA would be the right mask.
(defect) vui_ra_pages is initialized once per read syscall in vvp_io_read_start() and only reset at IO teardown, but ll_readahead() runs once per page (ll_readpage()->ll_io_read_page()->ll_readahead()). Won't this += accumulate across every not-uptodate page of a compressed read, progressively inflating the RA window enlarge at lines 806-808 and ria_end_idx_min at line 863? Also, the delta is taken from end_idx before the EOF clamp just below (lines 819-828), so vui_ra_pages can grow past the file size. ria_end_idx_min then feeds ll_read_ahead_pages() at lines 478-479, which raises ria_end_idx without an ria_eof guard - so readahead can extend past eof_index, the opposite of the "restrict readahead to eof" goal in the commit message.
LU-10026 llite: getting stripe info optimization ll_lov_getstripe_ea_info() is expensive call and should be avoided if possible. Let's use cached chunk size rather than get it from stripe info every time. llite: restrict readahead to eof Compressed file readahead rounding needs to come before readahead is limited to EOF. DDN-bug-id: EX-7601 Was-Change-Id: I4e9e7fe63301c08efcb05f170726735593a9431d llite: fix chunk_bits usage in readahead For the minimum compression chunk size, chunk bits is zero, so we cannot use if (chunk_bits) to determine if we're doing compression. This also fixes two other things: 1. A rounding error when rounding to chunk 2. Move rounding of end_idx to before first usage of end_idx, so calculation of number of pages is correct Without this, when the user reads 1 page or less, readahead will calculate the readahead page count as 0 and will exit without reading the chunk. DDN-bug-id: EX-6127 Was-Change-Id: I273506fd4f6ed5f0b8b5020357fd7caf0531e61c Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Id08487ec782f797e242e3f673c4a4dd8d526c9cc
The message (and the sanity-pfl/sanity test updates) cover only lzo, but the new error path also fires for zstd, since it likewise has no level mapping. Worth naming zstd here and adding coverage for it, or clarifying why zstd is excluded from the documented 1-21 level range.
(defect) This whitelists only fast/best to carry a level; every other type with a NULL ctn_from_compr_level falls through to the error. That is lzo (intended) but also zstd. So `lfs setstripe -Z zstd:5` now fails, whereas pre-patch it stored the level. lfs-setstripe.1:232 (added in this same LU-10026 series) says zstd takes a level 1-21. Is the zstd rejection intended? If zstd should not take a level, the man page needs updating too; if it should, zstd wants a from_compr_level mapping rather than the error path.
LU-10026 utils: error for invalid compression level Not all compression types accept a compression level (e.g. lzo in its kernel implementation). For those, return an error if a compression level is provided on 'lfs setstripe' command-line. Test-Parameters: forjanitoronly Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I6c367c4bfd76cc81c890a89ba9f994f4fd9f4f80
this is not helpful and needs to be updated
(minor) The body says decompress_request() has "a significant performance cost for checking if data is compressed", but it already returns immediately when the first page's cp_compr_type is LL_COMPR_TYPE_NONE. For uncompressed files that check is cheap; the scan cost only applies to compressed data, which is the case that still needs decompressing. Could the justification be tightened, and the read/write asymmetry this creates (writes still compress off the layout while reads stop decompressing) be noted?
(defect) This gates decompression on the global llite_enable_compression instead of the data's own cp_compr_type.
decompress_request() already returns early when pga[0]'s cp_compr_type is LL_COMPR_TYPE_NONE:
type = clpage->cp_compr_type;
/* no compression */
if (type == LL_COMPR_TYPE_NONE)
RETURN(0);
so uncompressed reads were never paying for a full scan.
The concern is a genuinely compressed read reaching here with the tunable off. lsm_lmm_verify_v1v3() only rejects a fresh compressed-layout unpack when compression is disabled; a cached lsm survives a runtime toggle. On read, set_page_compression()->__set_page_compression() still fills cp_compr_type from that cached lsme, and the write path still compresses off cp_compr_type regardless of the global.
After echo 0 > ...enable_compression on a client that already holds the compressed file's layout, does a cache-cold read here hand back the still-compressed bytes undecompressed?
LU-10026 osc: don't do decomp. with compression disabled decompress_request likely has a significant performance cost for checking if data is compressed, so we should not call it when compression is disabled. This is a stop-gap solution for the preview until we can have the server tell the client if data is compressed, as described in EX-7818. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Iba8feba3ab0fe620d0594f59c2c6ddea25faeb4f
The commit message only describes disabling fallocate, but the diff also adds a `rsync --sparse` decompression round-trip (the `decomp2` / "fiomap case" block) to compress_type() in sanity.sh. That change is about the sparse-copy/FIEMAP read path, not fallocate, and isn't mentioned here. Is it meant to be part of this patch, or should it be split into its own change with its own Change-Id?
(suggestion) This only checks lsm_entries[0]. lsme_compr_type is set per component (lov_ea.c copies lcme_compr_type for each LOV_PATTERN_COMPRESS entry), so a composite/PFL file could have an uncompressed component 0 and a compressed later component. Would fallocate over such a file still be allowed even though it targets a compressed component? (The ll_fiemap() guard has the same entry[0]-only limitation.)
(minor) `lio` and `loo` here just re-derive values already in scope: the outer `lio` (declared at the top of the function) is the same `cl2lov_io(env, ios)`, and `loo` is `lio->lis_object`. The inner `struct lov_io *lio` also shadows the outer one. This could just use the existing `lio->lis_object` without the extra declarations.
(minor) fallocate(2) is directly user-triggerable, so a CWARN here lets an unprivileged caller flood the console by repeatedly calling fallocate on a compressed file. The equivalent compressed-file guard in ll_fiemap() (lov_object.c around line 2021) returns -EOPNOTSUPP silently. Consider CDEBUG(D_VFSTRACE, ...) or a rate-limited message instead.
LU-10026 lov: fallocate is not allowed for compressed files Client Side Data Compression allocates blocks after a compression. It is impossible to preallocate blocks for the whole file, so fallocate should be disabled in case of compression. Test-Parameters: forjanitoronly Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ie834ace183fdcec0d7d6f747237e0964c3c4120b
LU-10026 llite: restrict overwrites during preview EX-7601 is an issue where when modifying a compressed file we do not correctly read-up and re-write existing compressed data. To avoid this, we can only allow writes which are not aligned to compression chunk size when they are not overwriting existing data, ie, when they are extending the file. This returns EINVAL for all writes to compressed files which are not either chunk aligned or extending the file. This should prevent users from hitting the data corruption issue but still allows some basic usage. This is intended just for the preview period. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I0eea01e2249866a074afd0d0642fe6dce9a49664
(style) The body explains why compression must be disabled for encrypted files, but that logic (compressed = 0; goto skip_compression) already lives in the parent compression patch. What this change actually does — throttling the existing CWARN to once per 24h and adding `always_except LU-10026 66` in sanity-sec.sh — isn't mentioned. Could the message describe the change it makes?
(minor) There is no test_66 in sanity-sec.sh (tests jump 65 -> 68), so this exclusion is a no-op in this tree. Which patch adds sanity-sec test_66, and should the always_except land together with it? Worth also confirming sanity-sec is the intended suite rather than sanity-compr.
LU-10026 sec: disable compression for encrypted files In case a read-modify-write io pattern is carried out on a compressed file, it has to be handled on server side. But because encryption cannot be done on server side for security reasons, we are not able to handle that kind of io pattern if the file is encrypted + compressed. So just disable compression for all encrypted files. Test-Parameters: forjanitoronly Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I84881fb1235f015d022751d4cce2d43a7231c2b4
(defect) `req` was just allocated above (ptlrpc_request_alloc_pool for writes, ptlrpc_request_alloc for reads) and every other error path in this function frees it with ptlrpc_request_free(req) before returning. These two early returns leave without freeing it, and *reqp is only assigned on the success path, so the caller can't release it either:
if (compressed && !page_access_allowed)
RETURN(-EINVAL);
if (inode && IS_ENCRYPTED(inode) && !page_access_allowed)
RETURN(-EINVAL);
For OST_WRITE the request comes from osc_rq_pool, so a GPU-direct write to a compressed or encrypted file leaks a pooled request each time and can eventually drain the pool. Should both routes free req first?
LU-10026 osc: disable CPU-access features for RDMA only pages Pages which cannot be accessed by the CPU are referred to as RDMA only pages. If pages cannot be accessed by the CPU, it is impossible for us to do compression, encryption, checksums, or short-io (data-in-RPC) on them. This patch disables compression and encryption for these pages and cleans up the code so checksums and short-io are disabled by the same code. The only user of RDMA only pages today is Nvidia's GPU direct, so this patch disables compression and encryption with GPU direct. NB: We eventually intend to handle compression for GPU direct with server side compress/decompress. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Iad9311617cddf27d3ff75a17429499c573067ea0
(minor) The body explains the DoM+compress block, but two hunks pull the other way and aren't mentioned: lov_pattern_available() gains LOV_PATTERN_MDT | LOV_PATTERN_COMPRESS, and llapi_pattern_to_lov() gains MDT|COMPRESS and RAID0|COMPRESS cases. Both make the combination more acceptable, not less. Could the message cover why they belong in a patch whose stated purpose is to disallow it?
(defect) This makes the server accept an MDT|COMPRESS component (lod_lov.c / lov_ea.c call lov_pattern_available()), yet lov_pattern_supported() still excludes it, so client IO marks such an entry invalid (lov_object.c sets lle_valid = 0). The only guard against creating it lives in lfs.c; a liblustreapi caller doing llapi_layout_pattern_set(MDT) + llapi_layout_compress_set() bypasses that and produces a layout the server stores but IO cannot use. If DoM+compress is unsupported, should the block live in the library/server rather than only the CLI?
(defect) errno is set to a negative value here; every other error path in comp_args_to_layout() uses errno = EINVAL. A negative errno breaks strerror(errno)/perror on this -1 return. Should be errno = EINVAL.
(minor) These two cases look unreachable. The COMPRESS bit never reaches llc_pattern: llapi_layout_pattern_set() rejects any pattern carrying it, and llapi_layout_from_lum() stores compression in llc_compr_type, not the pattern. The bit is OR'd into lmm_pattern separately in llapi_layout_to_lum() (blob->lmm_pattern |= LOV_PATTERN_COMPRESS). Was a real caller intended, or can these be dropped?
LU-10026 csdc: handle data compression on MDT Do not support setting data compression component on DoM until data compression on MDT implemented. DDN-bug-id: EX-7806 Was-Change-Id: I3794460140f08a073377c418dd56e7dda907d96d Test-Parameters: forjanitoronly Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I8b5c4d5fd18033e796b1c7555c37251e4f7f9a49
(minor) The diff also touches Documentation/man1/lfs-mirror-extend.1 (documenting compression options for SETSTRIPE_OPTIONS), which the message doesn't mention and which is unrelated to the migrate fix. Is that hunk meant to be part of this patch, or should it be split out / called out in the body?
(minor) This prints `fd`, which is always -1 on this path, so the message reads "Cannot open file X: -1" with no useful detail. The error is in errno from open() – strerror(errno) would be the informative value. Also note the caller (lfs_migrate/lfs.c) already emits its own llapi_error() when this returns < 0, so this adds a duplicate line. Using llapi_error(LLAPI_MSG_ERROR, errno, ...) here (as the ENOSPC path below does) would decode errno and respect the message level instead of writing to stderr unconditionally.
LU-10026 lfs: migrate compressed file without stripe info lfs migrate file without specifying stripe info will get layout info from the file as the target layout template, and llapi_layout_get_by_xattr() tries to convert LOV_PATTERN_* values to user scope LLAPI_LAYOUT_* values, while LOV_PATTERN_COMPRESS is missed in this conversion. This patch add a function llapi_pattern_from_lov() to handle this conversion specifically. This patch also add more error messages for llapi_layout_file_open(). Test-Parameters: forjanitoronly Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I49a43cc7761cd2baed7a5da7d4e7cff2152ff9bb
| unique failing test | history |
|---|---|
| racer@ldiskfs+DNE:test_1 | seen in 44 other reviews |
| sanity2@ldiskfs+DNE:test_160h | seen in 2 other reviews |
| sanity-pcc@ldiskfs+DNE:test_20 | seen in 23 other reviews |
(minor) This is an on-wire change: the client now fills bytes in the obdo (formerly o_padding_4) that are sent to the OST/MDS. Test-Parameters: forjanitoronly doesn't exercise interoperability. Consider requesting old-server/new-client and new-server/old-client runs with serverjob/serverbuildno and clientjob/clientbuildno so the padding-reuse stays wire-compatible with unpatched peers.
(minor) This comment reword looks unrelated to sending compression info to the server; is it meant to be in this patch? Also the code computes the chunk size with COMPR_CHUNK_MIN_BITS (COMPR_GET_CHUNK_SIZE()), not LOV_MIN_STRIPE_BITS; they happen to both be 16, but referencing the constant the code actually uses would be clearer.
This isn't a bug, but after this patch lov_lsm2layout() has no remaining callers (both lov_io.c and lovsub_object.c now call lov_lsm2layout2()). Folding the olc parameter into lov_lsm2layout() and updating those two call sites would avoid the version-suffixed name and the now-dead wrapper.
LU-10026 csdc: sending compression info to server Client fills in layout compression info into obdo and passes it to server. Test-Parameters: forjanitoronly Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ieb5d7b3609da41f35f8622ed6116f19ce7567ddb
LU-10026 csdc: prohibit set compression upon encrypted file Setting compression layout component upon encrypted file is not allowed for now. This patch add this check on MDS in creating file with layout, adding/merging new mirror to existing file. Test-Parameters: forjanitoronly Test-Parameters: testlist=sanity-sec env=ONLY=67,PTLDEBUG=-1 Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I60d9f4bfce3a498f1eb3994c6276afb9d89c99a7
LU-10026 osc: clear oi_write_osclock in lock fini func Move osc_io::oi_write_osclock clearance in osc_lock_fini() as it's set in osc_lock_init(). Compression IO could possibly expand lock region and osc_lock_set_writer() could access a osc_io that is not accessed in osc_io_iter_init(), so that osc_io_rw_iter_fini() miss clearing osc_io's oi_writer_osclock. This patch moves the oi_write_osclock clearance in lock fini function to match its creation in osc_lock_init(). Test-Parameters: forjanitoronly Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ied42f5befc1abd76aa10a7666eadb9a58e1f1783
LU-10026 llite: skip hole/data lseek() on compressed file Doesn't execute real lseek() on compressed file with SEEK_HOLE/SEEK_DATA origin but consider file always non-sparsed and do generic lseek() only test: get size before fallback to generic If ll_file_seek() falls back to generic_file_llseek_size() then make sure inode size is reliable DDN-bug-id: EX-7786 Was-Change-Id: Ieb8c90e66fb19675ba41f0147c0d9cdaf29ea20a Test-Parameters: forjanitoronly Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I2cd587cc0205e85758e06bbaafafe0e2959e0ade
courtesy of gerrit not accepting test results at times you did not get your early warning. This introduces a sleeping under spinlock right at server init time and so every test has it https://testing.whamcloud.com/gerrit-janitor/63997/results.html [ 817.527107] alg: No test for zstdfast (zstdfast-lustre-generic) [ 817.534750] BUG: sleeping function called from invalid context at include/linux/sched/mm.h:226 [ 817.541176] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 26101, name: cryptomgr_test [ 817.550885] CPU: 3 PID: 26101 Comm: cryptomgr_test Kdump: loaded Tainted: G W O -------- - - 4.18.0rh8.10-debug #2 [ 817.559137] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 817.565102] Call Trace: [ 817.566955] ? dump_stack+0xbb/0x10e [ 817.575504] ? ___might_sleep.cold.92+0xd9/0x107 [ 817.580904] ? __might_sleep+0x59/0xc0 [ 817.583799] ? slab_pre_alloc_hook.constprop.64+0x11f/0x1d0 [ 817.590298] ? kmem_cache_alloc_node_trace+0x5e/0x470 [ 817.595682] ? __get_vm_area_node+0xd6/0x220 [ 817.599009] ? __vmalloc_node_range+0x79/0x3c0 [ 817.603765] ? __zstd_compress+0x100/0x210 [lzstd] [ 817.606239] ? __vmalloc_node+0x4e/0x90 [ 817.608884] ? __zstd_compress+0x100/0x210 [lzstd] [ 817.615296] ? vzalloc+0x4e/0x60 [ 817.616509] ? __zstd_compress+0x100/0x210 [lzstd] [ 817.620607] ? zstd_scompress+0x32/0x60 [lzstd] [ 817.625250] ? scomp_acomp_comp_decomp+0x129/0x200 [ 817.627994] ? scomp_acomp_compress+0x17/0x20 [ 817.633796] ? test_acomp+0x2cc/0x5a0 [ 817.635204] ? cryptomgr_probe+0x150/0x150 [ 817.638300] ? alg_test_comp+0xc8/0xf0 [ 817.640295] ? alg_test+0x1f0/0x390 [ 817.642645] ? do_raw_spin_unlock+0x75/0x190 [ 817.646249] ? _raw_spin_unlock_irq+0x12/0x50 [ 817.648952] ? finish_task_switch+0x125/0x560 [ 817.651578] ? __schedule+0x369/0xcb0 [ 817.656565] ? do_raw_spin_unlock+0x75/0x190 [ 817.660800] ? cryptomgr_probe+0x150/0x150 [ 817.663411] ? cryptomgr_test+0x37/0x70 [ 817.665939] ? kthread+0x1d1/0x200 [ 817.669914] ? set_kthread_struct+0x70/0x70 [ 817.673595] ? ret_from_fork+0x1f/0x30 [ 817.689553] alg: No test for zstdfast (zstdfast-lustre-scomp)
LU-10026 lustre: add lzstd kernel module
lzstd kernel module implements compression according to the zstd
algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'zstd':
* zstd-lustre-generic of type compression
* zstd-lustre-scomp of type scomp
And 2 cipher drivers under the generic name 'zstdfast':
* zstdfast-lustre-generic of type compression
* zstdfast-lustre-scomp of type scomp
Note the 'zstd' name is identical to the in-kernel module, but lzstd
registers it with a slightly higher .cra_priority = 110, so that it is
preferred over the in-kernel module.
Our 'zstd' is also different in that it accepts a compression level
as explained below. 'zstdfast' is handling the 'negative' compression
levels, interpreted as an acceleration factor.
lzstd kernel module sources are copied from linux v6.9. This module
implements the Crypto API interface, and rely on the zstd kernel
library for compression implementation. It has been modified to grok
a compression level, as read from the top 4 bits of the crypto_tfm
flags, and pass it to the underlying library.
The zstd library sources are also copied from linux v6.9 and built
statically. Headers have also been copied from linux v6.9 for
consistency, and source files modified to include the copied headers
instead of the system headers.
In order to maximize compatibility with various distributions, we copy
xxhash sources from linux v6.9, and link these statically.
All aforementioned sources are located in the
lustre_compat/crypto/zstd directory.
The lzstd module is always built with Lustre.
This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'zstd' module.
It also tries to manually load the lzstd kernel module when a file
requires compression with the 'zstd' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.
EX-9678 csdc: defer memory allocation in zstd/zstdfast
Zstandard normally proceeds to memory allocation for its context
(workspace) while in the init phase, which corresponds to a call to
crypto_alloc_comp() when used via the Crypto API.
At this stage we cannot be aware of the compression level or the chunk
size. So we have to make zstd allocate a workspace that can fit any
level/chunk. And the higher the level is, the larger the workspace
needs to be. Problem is zstd is very memory eager, and by default for
the higher compression level it requires a workspace of 650 MiB. This
is insane compared to our usual compression chunk sizes.
So we defer workspace initialization to the first time zstd actually
compresses data. This corresponds to a call to crypto_comp_compress()
when used via the Crypto API. And by that time we know the compression
level and chunk size, which enables us to make zstd allocate a finer
grained workspace, that just fits our actual needs. This saves a lot
on memory allocation, as can be seen from these values (workspace size
expressed in bytes):
no src hint 64kB src
level 1 cwksp size 582392 303864
level 2 cwksp size 779000 402168
level 3 cwksp size 1303288 664312
level 4 cwksp size 2614008 1057528
level 5 cwksp size 3662584 1057528
level 6 cwksp size 3662584 1057528
level 7 cwksp size 6808312 1057528
level 8 cwksp size 6808312 1057528
level 9 cwksp size 13099768 1057528
level 11 cwksp size 25682680 1319672
level 13 cwksp size 34071288 1468856
level 15 cwksp size 67625720 1791095
level 17 cwksp size 50997688 1791095
level 19 cwksp size 85196598 1791095
level 21 cwksp size 336854838 1791095
level 22 cwksp size 680943414 1791095
And of course because we reduce contention on memory a lot, we improve
performance when zstd is used via the Crypto API, which is done in
osc_brw_prep_request() for ever RPC that involves a file compressed
with zstd.
Was-Change-Id: I470331ea0f2e550761ce7223e7be7e1373701bd2
Was-Change-Id: Idf7382093feec21bed5b6e62001a085a386736d2
EX-bug-id: EX-9678
Test-Parameters: testlist=sanity env=ONLY=84
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I113345cd10f41197683bfe127d373b8628364f88
One thing that needs caution here is to "repair" a parent FID on an object that is actually (incorrectly) referenced by two MDT layouts. If the parent FID is missing/unset/corrupt, that is an easy case. If the parent FID is potentially valid, then the OST should contact the MDT owning the current parent FID to confirm or deny whether it has a layout referencing that OST object. I'm pretty sure that LFSCK is already fixing this case, so I'm wondering if it makes sense to push this OST object off to LFSCK in this case?
In this pfid repair case that is triggered through RPCs, I wanted to be conservative and only repair either unset or corrupt parent FIDs (that do not pass `fid_is_sane()`). This keeps the repair operation local to the OST. I can improve the commit message in this regard. It looks like the case you mentioned is handled by `lfsck_layout_check_parent()` and `LLIT_MULTIPLE_REFERENCED`. There is also `ofd_verify_ff()` which triggers the "inconsistency thread" and contacts the MDT. I initially considered it for checking if a parent FID needs repair but it is relying on the "lfsck_instance". I think it also contacts the MDT. In any case, this counters the idea of keeping this repair lightweight and local.
Having a more comprehensive commit message will help explain motivation and caveats in case there is an issue with this in the future.
warning: sparse:symbol 'ofd_can_repair_pfid' was not declared. Should it be static?
(defect) this should be fixed, or will fail compilation on some distros. It needs to be declared static (or declared in a header if it is used in other files, but that is not the case here)
OK, good. I didn't see this before my initial comment. I only saw the commit message and wanted to highlight this issue, but it seems you are already handling the common case. Should this be annotated with `likely(...)`? Is this called only for "repair" or on new objects also?
(style) you can use `[[ ... ~= "ids" ]]` to do a substring match
(style) instead of adding this to a single increasingly complex test case, also with separate version requirements, it seems better to add a separate test case for this?
LU-18756 ofd: Add PFID repair to resource ID repair The resource ID repair checks whether a UID/GID/PROJID is not set on an OST object yet. If all or some IDs are not set and the requirements are met, the IDs are updated asynchronously, e.g., as part of a read operation. This patch adds support for repairing the parent FID stored in the "trusted.fid" xattr field for OST objects which references the corresponding MDT inode. Repairing the PFID is done in the same transaction as the ID repair. It can therefore repair IDs, the PFID or both, depending on the current state of the object. As a fast path, the PFID is stored in the "ofo_ff.ff_parent" field in the "ofd_object", which is also used to check whether repair is required before loading the xattr, if the field is not yet set. Sanity-sec 75b is modified to exercise PFID repair in the three configurations: IDs-only missing, PFID-only missing, or both missing. Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Iafa97b1204a96ef3349009a340fd812528ae43ef
(style) line length of 83 exceeds 80 columns
(style) "contains" -> "has"
LU-10026 csdc: prefer uncompressed mirror for read When accessing a mirrored file with both compressed and uncompressed components, choose uncompressed components for read. csdc: prefer uncompressed mirror for write When writing to mirrored files with both compressed and uncompressed mirrors, prefer the uncompressed components to write, and that is better for performance, more compatible with older clients, and better fits the model of compressing files after initial write. DDN-bug-id: EX-6510 Was-Change-Id: I62a117d5cc3d34e2c0c96d1a9ade8eef0a2d1291 Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@gmail.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I043b27bd891c039901075a08c76630f8f0f9f182
(defect) How is this header meant to be byte-swapped? A bitfield can't be run through cpu_to_le64() or __swab64s(), so a big-endian client storing LLCH_MAGIC lays down c0 39 8e 55 da 7a where a little-endian one lays down 7a da 55 8e 39 c0, and neither can then recognise the other's chunks. wirecheck can't pin bitfield placement either, which is what the generated "is a bitfield and cannot be checked" comments mean. The tree already solves exactly this for lcme_timestamp:48 by overlaying it with a __u64 lcme_time_and_id that lustre_swab_lov_comp_md_v1() swabs (lustre/ptlrpc/pack_generic.c:2591). A similar union overlay here, or a plain __u32 + __u16 magic, would make the byte order expressible. The llch_compr_level / llch_chunk_log_bits nibbles have the same problem, since their order inside byte 11 also flips with endianness. Nothing consumes the struct yet, so this is cheapest to settle now while the format is still free to change.
This makes sense. We can't actually change the on-disk or wire format, but I don't think there are any big-endian users today so maintaining only the little-endian format is required.
The nameless union needs more complexity to handle, while changing this to separate fields seems simpler (it is only accessed in a few places):
```
#define LLCH_MAGIC_LO 0x8E55DA7A
#define LLCH_MAGIC_HI 0xC039 /* "compressdata" */
struct ll_compr_hdr {
__u32 llch_magic_lo; /* LLCH_MAGIC_LO */
__u16 llch_magic_hi; /* LLCH_MAGIC_HI */
__u8 llch_header_size; /* for future extensions */
__u8 llch_extra_flags;
```
It's too bad it wasn't split as "0xC0398E55" and "DA7A", but oh well.
Done
Valid concern. The bitfield nibbles llch_compr_level:4 / llch_chunk_lum_bits:4 are not byte-swappable and their layout is implementation-defined on big-endian. Since nothing consumes this struct yet, will replace the bitfield byte with a plain __u8 and use masks/shifts (analogous to how the timestat nibbles are handled), so the byte order is explicit. Noted for the first patch that reads or writes llch_compr_level.
(suggestion) Using 0 for "no checksum" is ambiguous, since crc32 over real data can legitimately come out 0 and such a chunk would then silently skip verification. Both llch_flags and llch_extra_flags are unused so far, so a "checksum present" bit would be unambiguous. Same applies to llch_hdr_csum.
I don't think we are using the checksums yet, so this seems reasonable when we get there.
Done
Good point. A genuine crc32 of 0 would be indistinguishable from 'no checksum'. Will add a LLCH_FL_COMPR_CSUM / LLCH_FL_HDR_CSUM flag pair to llch_flags to signal checksum presence explicitly, so 0 in the csum field is unambiguous. llch_flags and llch_extra_flags are unused so far, so the slot is available.
Does this now match where these fields appear when the file is auto-generated?
Is there a reason this was moved? It just seems like needless churn.
LU-10026 csdc: add ll_compr_hdr structure It is an important structure for CSDC feature. Will be used in the next patches. Also adds LL_COMPR_TYPE_UNCHANGED enum value (used to signal that the compression type should not be changed on update) and the LL_LZ4FAST_DEF_LEVEL define (default level for the LZ4FAST algorithm). Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I26f2fc7c0212f1a4cad36805c75aa766d6aed548
(minor) the date should be updated
LU-10026 utils: add 'lfs find' support for compressed file * Add "--comp-flags=[^]compress" to locate file with/without compressed components. * Add "--comp-flags=[^]nocompr" to locate file with/without setting component compress preference. * Add "[!] --layout=compress" to locate file with/without compressed components. * Add "[!] --compress-type=<compress-type>" to locate compressed file with/without specified compress algorithm. * Add "[!] --compress-level=[+-]<compress-level>" to locate compressed file with/without specified compress level. * Add "[!] --compress-chunk=[+-]<compress-chunk>" to locate compressed file with/without specified compress chunk in KiB. utils: add -Z option for 'lfs getstripe/find' Add support for "lfs getstripe -Z" to get the last instantiated component compression information. Add support for "lfs find -Z <type>[:[+-]<level>]" to keep consistent options with "lfs setstripe -Z". DDN-bug-id: EX-6856 Was-Change-Id: Ia5e2a2ea6937dc2b46e224cec808504a196b974c utils: logical AND for 'lfs find' compression exprs All search expressions provided to 'lfs find' must be combined as a logical AND. Fix newly added options for compression support, so that they comply with this logical AND. DDN-bug-id: EX-7714 Was-Change-Id: I3b28cd87c1d304df6d04753b413d46f5abcfe16e csdc: don't set compression layout when disabled When llite_enable_compression is disabled (lfs set_param llite.*.enable_compression=0), we should check it before sending it to MDS lest we get a file with compressed component which we cannot handle. DDN-bug-id: EX-7593 Was-Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f utils: support 'lfs find --printf %LZ' Add support for "lfs find --printf %LZ" to print the compression type:level of the last instantiated component of a file. DDN-bug-id: EX-6856 Was-Change-Id: Iaf1b6c031b06c70e7b5be51354697aa6bdcc9850 csdc: right error code in case compression is disabled Currently, if compression is disabled on a client the error message is confusing "Cannot set layout EA: Unknown error 524" Let's replace the error code from ENOTSUPP to EOPNOTSUPP, so error messages become more informative ctl get_param -n llite.*.enable_compression 0 lfs setstripe -i 0 -c 1 -E -1 -Z lz4 /mnt/lustre/foo.txt Cannot set layout EA: Operation not supported lfs setstripe: cannot create composite file '/mnt/lustre/foo.txt': Inappropriate ioctl for device This message could be even more informative, but this version is already much better. DDN-bug-id: EX-10577 Was-Change-Id: I7fd67d5d3d20bc7de169f35f6fa63b53f2bfe718 Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ic36946738c3463fd862aeca4ee2e2c2ed85eff84
this patch also conflicts with https://review.whamcloud.com/c/fs/lustre-release/+/62906 that is in master-next already
From Claude: (minor) fp_compr_lvl is pre-seeded with (unsigned int)-1 as the \"no level given\" sentinel, but llapi_parse_compress_type() can legitimately parse a level of -1 (range is -COMPR_LEVEL_MAX..COMPR_LEVEL_MAX), which also lands as (unsigned int)-1. So `lfs find -Z gzip:-1` takes this false branch and the level filter is silently dropped - it behaves like `-Z gzip`. A separate \"level present\" bool would avoid the collision.
Oleg, this comment seems misplaced? I don't see this related to anything here.
LU-10026 csdc: set compress component for file
* 'lfs setstripe' to support compress component
--compress|-Z <type>[:<level>]
Set component compression algorithm <type> and compress <level>;
--compress-chunk=<size>
Set compress data chunk size in KiB used by the compression
algorithm, the value will be adjusted to power-of-two multiples of
the base 64KiB.
Example:
$ lfs setstripe -Eeof -Z lz4:5 --compress-chunk=512 <file>
* 'lfs getstripe' to show compress component parameters
Display component's compression parameters if possible.
--compress-type|--compr-type
Print only the compress type if possible.
--compress-level|--compr-level
Print only the compress level if possible.
--compress-chunk|--compr-chunk
Print only the compress chunk size in KiB if possible.
Example:
$ lfs getstripe <file>
lcme_compr_type: lz4
lcme_compr_lvl: 5
lcme_compr_chunk_kb: 512
lmm_pattern: raid0,compress
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ife0382469cbc5099e0c6dc96534bb169ddeff61e
Shouldn't this RPC be using the "trusted" nodemap, so that it already skips ID mapping? I'm not totally against making this code more robust even without the "trusted" nodemap, but I wonder if this will find enough other issues that prevent it from being useful (e.g. RBAC rules to prevent access entirely).
Unfortunately, we have the same issue for trusted nodemaps because we get the MDS nodemap instead of the client nodemap from the export. So, the type of nodemap is not relevant here. The issue was the below ID mapping assumption that the incoming IDs represent the "CLIENT_ID" and must be mapped to the FS_ID. This is a fair assumption whenever the client sends RPCs directly to the OST, but it is incorrect when the MDS sends RPCs to the OST because, in this case, the MDS already mapped the "CLIENT_ID" to the "FS_ID" and sends the latter to the OST. Mapping the "FS_ID" again here with the "NODEMAP_CLIENT_TO_FS" type (assuming the same nodemap applies to both client and MDS) would squash it both for idmapping and offset cases. Back to the "trusted" nodemap case: This is the same issue. The MDS skips id mapping, and the OST actually receives the trusted "CLIENT_ID" from the MDS. However, the OST does not attempt mapping for the trusted client nodemap, but for the MDS nodemap. If the MDS is trusted, this would be fine. Otherwise, the IDs are also squashed. I verified this is an issue for trusted and untrusted nodemaps, and this patch fixes both.
I think what Andreas means is that servers are _supposed to_, if correctly configured, be in a "TrustedSystems" nodemap with admin=1 and trusted=1. So in this case, when the OSS processes the SETATTR request from the MDS, it would naturally keep untouched the ids the MDS put in the request. I guess this patch is still nice to have in order to protect against incorrect nodemap configurations that do not have that "TrustedSystems" nodemap. But maybe we could try to cover more cases by changing the way nodemap_add_member() fetches the nodemap whose reference is stored in the export. Today it uses the provided nid. But it could check for the connect flags on the export, as you are doing below for OBD_CONNECT_MDS. Assuming they are already set when nodemap_add_member() gets called, we could point to an in-memory only privileged nodemap (always created when modules load, like "server_nm") if we detect we are dealing with a server-to-server export. This way, even if the "TrustedSystems" nodemap is missing in the configuration, we would treat the server-to-server exchanges as privileged from a nodemap standpoint. Here in tgt_ost_body_unpack(), that would mean the call to nodemap_map_id() would keep ids as they are set in the request coming from the MDS.
I see, thanks for clarifying. Yes, in this case, this issue would not occur since mapping is skipped here. So, we could go more in the direction of protecting against such misconfigurations. I would agree that it would be better to enforce this globally (for server<->server communication) instead of on an individual RPC level. I will look into your idea, Sébastien, thank you.
I think the right approach for now would be to report an error at mount if there are nodemaps configured for the MDS and OSS but they have ID mapping enabled, and/or are not marked trusted.
LU-18835 target: Skip id mapping for OST_SETATTR from MDS When nodemaps are in use, the uid/gid tags on OST objects become squashed when "chown"-like operations are issued. Valid OST_SETATTR RPCs to modify the UID and GID on the OST objects are asynchrounously sent by the MDS (via OSP through "osp_attr_set()") after successfully applying the UID/GID to the MDT object. Here, the ID mapping is already done on the MDS and the (mapped) FS IDs are sent to the OSTs. This causes two issues: (1), The RPC target layer for OST RPCs however assumes that _all_ RPCs come from the client and maps the ids for "NODEMAP_CLIENT_TO_FS". And (2), when OST_SETATTR RPCs are sent from the MDS to the OSTs, the export is the calling MDS, which may resolve to a different nodemap than the client (e.g., "default"). Either issue causes an incorrect UID and GID which is then applied to the OST object. This patch fixes this issue by skipping the "NODEMAP_CLIENT_TO_FS" mapping on the RPC target layer for OST_SETATTR RPCs _if_ they originate from the MDS because, in this case, the UID/GID are already correctly mapped. Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Iffa7b1054c3b0858d4986351e2b86b03dd93d057
LU-18756 tests: sanity-sec 75a sync resource IDs
After sanity-sec 75a setups the test namespace, it creates and writes
to files before changing the user to the correct tenant user which
carries out the commands. Later, when the ID check is enabled, it
could happen that the corresponding OST object still has the initial
user ID set and that the "chown" changes have not yet been propagated
from the MDT to the OSTs. In such a case, the test fails the ID check
because the OST object has the wrong IDs set.
This patch adds an additional step after namespace creation to
synchronize ID changes to and between the servers before continuing
with the test.
Fixes: 719f6763fe ("LU-18756 sec: add resource id check to oss and mds")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=75
Test-Parameters: testlist=sanity-sec env=ONLY=75a,ONLY_REPEAT=100
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I07d602375fd34334dc5047cfecea60f09f03bd42
| unique failing test | history |
|---|---|
| sanity-pcc@zfs:test_22 | seen in 4 other reviews |
the reason is now known and should be included here. Note for me to update later
(defect) The resend is rebuilt from the uncompressed arrays, so besides aa_requested_nob the niobuf/page counts also change (compression yields fewer niobufs), yet only aa_requested_nob is propagated back into `aa` before `new_req->rq_async_args = request->rq_async_args` overwrites new_aa. After the resend new_aa->aa_nio_count still holds the compressed count, while the reply's RMF_RCS vector is sized from the uncompressed `niocount` in osc_brw_prep_request().
check_write_rcs(req, aa->aa_requested_nob, aa->aa_nio_count, ...)
check_write_rcs() then only inspects the first aa_nio_count per-niobuf rcs, so a server-side write error on a niobuf past the compressed count would be skipped. aa_page_count is similarly stale. Should aa_nio_count (and aa_page_count) be updated here the same way as aa_requested_nob?
LU-10026 osc: do not compress resends There's some issue with doing compression on resent requests, so this patch works around it with two things: 1. Use the uncompressed page array for resend (this was always necessary unless we modified resend to know it already had compressed pages as input) 2. Disable compression on resend (not clear why 1. wasn't enough) csdc: Update async_args after resend It is decided to send an uncompressed request on redo. osc_brw_prep_request() processes uncompressed data and prepares a request, so some parts of the old request are outdated. Let's update the old request with information from the new one. DDN-bug-id: EX-8814 Was-Change-Id: Idb1c6ee9db64cb1f2ea1c1562b1c5aae443263e3 Test-Parameters: forjanitoronly Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I5fbbdc2771f8c2c7b5c28f0b70d89b8b6015147f
(minor) The Fixes: commit only touched lustre/obdclass/page_pools.c and never added this test. subtest_2000() and the dd count=5000 that this patch tunes down were both introduced by 452599da95 ("LU-10026 osc: osc brw request compression", Gerrit 65770). If the patch stays standalone, should Fixes: point at 452599da95 instead? (This also lines up with the existing suggestion to fold it into 65770, which would drop the tag entirely.)
(minor) Both echoes print the same "count $count" label, one before and one after the clamp, so a log reader can't tell the raw 70%-of-RAM value from the clamped one. Consider labelling them differently or dropping the first.
LU-10026 tests: change sanity-compr 2000
Make dd write 70% of RAM but less than 5GB.
The test case is too agressive and sometimes
causes OOM as client have to preserve
compressed pages for OST_WRITE replay(see
EX-9895 for details).
Fixes: 14fc034e983e ("LU-18086 obdclass: page_pools shrinker fix")
Test-Parameters: forjanitoronly
Test-Parameters: trivial testlist=sanity-compr env=ONLY=2000,ONLY_REPEAT=10
Test-Parameters: trivial env=ONLY=2000,SLOW=yes testlist=sanity-compr
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I594b963ca358c5adbf077bcd59378dd13506ccad
| unique failing test | history |
|---|---|
| recovery-small@ldiskfs:test_155 | seen in 4 other reviews |
LU-10026 osd: osd_bufs_put does not always handle all pages osd_bufs_put asserts that the dio pages used after are always zero, but there's no reason for this to be true and compression specifically violates this by using 1 page at a time. Without this patch, we hit this assert and crash when nonrotational = 1. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: If6bdb11f254c260e2da4cabe11a82693a468e6fb
LU-10026 tests: skip sanity-compr for old servers Skip for old servers. Test-Parameters: forjanitoronly Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: Ia42f1f32fe166bae84a7c38c02d6a218303cc51e
| unique failing test | history |
|---|---|
| sanity-pcc@ldiskfs+DNE:test_22 | seen in 1 other review |
| sanity-quota@zfs:test_1b | seen in 48 other reviews |
LU-20203 utils: reorder lctl nodemap_info --list output
The "lctl nodemap_info --list" command prints three sections in this
order: global nodemap state, defined nodemaps, and available
parameters. Put the available parameters first.
Generated with Claude Code + Tools
Fixes: b8e3b427f478 ("LU-18715 utils: Extend lctl nodemap_info with property values")
Test-Parameters: trivial
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I8747b49d1cdb31701e595497342180bf85707bec
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| custom-1002 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-subtest-change failed 2× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-zfs-subtest-change failed 7× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-ldiskfs-dne-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-12668 ec: Add tests for computing the parity coverage When we write or verify the parities we no longer use the whole stripe, instead we compute the coverage of what ranges of parity is important and what can be ignored. This is based on SEEK_DATA/SEEK_HOLE and EOF. Add tests that we compute these ranges correctly. Test-Parameters: trivial Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: Iaee9d0ccb0875f6d515a8adc322fa3cd37bfe0f6
| unique failing test | history |
|---|---|
| sanity-lfsck@zfs:test_18c | seen in 25 other reviews |
| sanity-scrub@ldiskfs+DNE:test_9 | seen in 5 other reviews |
This is false.
These modules do not compile on newer kernels (6.18, at least). I think it should be possible to build without these.
just as the smatch implies this should really be IS_ERR, I don't think filp_open can return NULL?
LU-10026 lustre: add lz4 and lz4hc kernel modules lz4 and lz4hc kernel modules implement compression according to the lz4 and lz4hc algorithms respectively, through the kernel Crypto API. lz4 module provides 2 cipher drivers under the generic name 'lz4': * lz4-lustre-generic of type compression * lz4-lustre-scomp of type scomp lz4hc module provides 2 cipher drivers under the generic name 'lz4hc': * lz4hc-lustre-generic of type compression * lz4hc-lustre-scomp of type scomp lz4 and lz4hc kernel module sources are copied from linux v6.1-rc5, and renamed to llz4.c and llz4hc.c respectively to avoid name collisions. Use of vmalloc has been changed to kvmalloc since it is faster in most cases. They implement the Crypto API interface, and rely on the lz4/lz4hc kernel library for compression implementation. They have been modified to grok a compression acceleration/level, as read from the top 4 bits of the crypto_tfm flags, and pass it to the underlying library. The lz4/lz4hc library sources are also copied from linux v6.1-rc5 and built statically, so lz4_compress, lz4_decompress and lz4hc_compress sources have been "de-modulified", and EXPORT_SYMBOLs removed. Headers have also been copied from linux v6.1-rc5 for consistency, and source files modified to include the copied headers instead of the system headers. All aforementioned sources are located in the lustre_compat/crypto/lz4 directory. The lz4/lz4hc modules are built only if the kernel does not provide them. This is verified by checking if the CONFIG_CRYPTO_LZ4 and CONFIG_CRYPTO_LZ4HC kernel config options are defined. This patch provides unit testing of several compression modules, such as lz4, lz4hc, lzo and gzip. This is done via a new test kernel module kcompr.ko, and new sanity test_84. File lustre/tests/kernel/kcompr.c contains examples of how to call the compression/decompression routines. EX-7998 lustre: Fix build with crypto_tfm_ctx on kernel 6.2 In recent kernels, commit e634ac4 "crypto: api - Add crypto_tfm_ctx_dma" moved crypto_tfm_ctx into algapi.h. So this file must be included from lustre_crypto.h. Was-Change-Id: If10f721355da3e58ca541c17615e978334e8d718 EX-7683 utils: always try to use our own lz4/lz4hc lz4/lz4hc provided by the kernel do not grok a compression level. The built-in lz4/lz4hc do, so always build them as dedicated kernel modules llz4.ko and llz4hc.ko, with the same .cra_name but with a slightly higher .cra_priority = 110, so that they are preferred over the in-kernel modules if any. And try to manually load the llz4/llz4hc kernel modules when a file requires compression with the corresponding alg. This is a "one-shot" try that allows us to prefer our modules that has level support, but continues to at least compress/decompress files even if our own modules are not available. Was-Change-Id: I0bdf267f998e21df81e460250a653aed34e3215d EX-bug-id: EX-6275 Test-Parameters: testlist=sanity env=ONLY=84 Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I74ce95ff18194c6052d291588d7e8c79596a7f23
We aggregate attributes from multiple objects for other reasons (e.g. maxbytes, timestamps, size, etc.). The current implementation looks "mostly OK" and will work for basic linear IO patterns, where the file transitions from non-rotational to rotational. However, any other kind of IO pattern (e.g. HDF5 where some writes are at the beginning of the file and others at the end) might have a serious issue? IMHO, it would be better to aggregate this information once from the object (maybe from the OSC's the object is allocated on) and then be done with it? Storing a threshold when the file transitions from non-rotational to rotational would be best, as this could be computed once based on the layout. It is fairly unlikely that a file would have HDD stripes in the middle and flash at the beginning and end. Alternately (probably better) is to store the nonrot state in each layout component (which is a perfect 1:1 mapping), and then this can be checked at IO submission time to see what type of storage it is covering.
Hm, OK. I don't think we can do this as easily as we'd like - the issue is we have to have this information very early. I can consider this, though, your points are good ones. The good thing is this isn't essential to merging the core feature, particularly since we're leaving it off by default for 2.16.
Actually, an OST is (generally) only going to be rotational or non-rotational, and this is already returned to the client via statfs, so the clients should have full knowledge whether any IO is on flash or disk.
LU-13802 llite: add file nonrotational check This patch adds the ability to note whether or not the last IO to a file hit a rotational or a non-rotational device. This will be used in selecting the cutover thresholds for hybrid IO, since DIO is synchronous and much slower on spinning disk. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I75a9970f91b1776ed6f04ac0d000a9ba576df75a
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_127f | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-5 crashed | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
(typo) ".patch" Writes for fscrypt files *should* always be full-block writes? Why/how would writes to fscrypt files be partial blocks? The client should be sending a full encrypted 4KiB block, but then set the size on the inode without truncating it.
Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext4 for fscypt with write through (direct I/O) mode. Although writes from clients are always full-block writes, but the size calls via lnb offset + len may be not full-block aligned, we use this size for the actual size of the file?
LU-12916 osd: use writeback for small writes in ldiskfs This patch implements the writeback caching for small writes on Lustre OSD (ldiskfs). The writes are not synchronous any more on OST. It marks the page as dirty and create buffer heads mapping pages to the physical blocks. The kernel writeback mechanism will handle the actual I/O. This patch should improve the performance for I/O cases such as small writes from many files (i.e. mdtest-hard-write) or unaligned I/O on a shared file (i.e. ior-hard-write). Add a ldiskfs patch "ext4-writeback.patch" to not zero out non-full page during writeback for a encrypted file. Without this ldiskfs patch, it will fail sanity-sec/test_37. According to the benchmark results, this patch can improve the IO500 mdtest-hard performance and the performance of the real application Nek5000 turbulent pipe with more than 10%. Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I9b5aa568a49d5b39d04656fdacb94c3a6ec5a7ff
This paragraph describes a change that does not seem to be in the diff. lustre/lov/lov_page.c isn't touched, and lov_page_init_composite() has no append exception either before or after this commit:
stripe_cached = lio->lis_cached_entry != LIS_CACHE_ENTRY_NONE &&
page->cp_type == CPT_TRANSIENT;
Was this paragraph left over from an earlier version of the patch?
This fixes wrong-stripe routing and a wrong file size, so a Fixes: tag would help decide which maintenance branches need it. The "DIO is already split by stripe" assumption that this patch repairs was introduced by:
Fixes: d31647c017a3 ("LU-13799 lov: Improve DIO submit")
That commit made lov_io_submit() splice every DIO page onto the stripe of the first page, and 14db1faa0fbe ("LU-13799 lov: Cache stripe offset calculation") then extended the same assumption to the per-page stripe cache. The assumption never held for the append path.
The clamp to the component extent lives inside this branch, so nothing bounds the batch when the component at `start` has a single stripe. For a layout like `-E 1M -c 1 -E -1 -c 4`, a 4MB DIO append from offset 0 gets no split at all and the batch crosses the component boundary; lov_page_init_composite() then hits
if (!lov_io_layout_at_confirm(lio, entry, offset))
return -ENODATA;
for the pages past 1M and the write fails.
The non-append path below clamps unconditionally - it starts at MAX_LFS_FILESIZE and applies min(next, lse->lsme_extent.e_end) outside the stripe_count test. Should this do the same, so the commit message's "matching what the non-append path already does" actually holds?
lio->lis_io_endpos is an absolute file offset set once in lov_io_init():
lio->lis_io_endpos = crw_pos + crw_bytes;
For append that crw_pos is only the estimate taken in ll_file_io_generic(); the offset the write actually lands at is chosen later by vvp_io_write_start() from i_size_read(). After the first iteration `start` tracks that real position, so if the file grew in the meantime `start` can run past lis_io_endpos.
Two consequences: the loop stops once next reaches lis_io_endpos even though bytes remain in the iov_iter (short write from an O_APPEND write(), where before the patch the whole count went out in one iteration), and on the next line `next - start` goes negative into the size_t crw_bytes.
Should the append path clamp against the number of bytes still outstanding rather than an absolute endpos, and guard next >= start?
Setting ci_continue here turns a single append iteration into several, and cl_io_loop() drops the lock between them:
cl_io_lock() -> cl_io_start() -> cl_io_end() -> cl_io_unlock()
vvp_io_write_lock() takes [0, wr_append_lockpos] PW for append precisely so the whole write lands atomically at EOF, and vvp_io_write_start() re-reads i_size_read() under that lock on every iteration. With the split, another client can revoke the lock and append between our iterations, so a single 4MB O_APPEND write() can end up with a foreign client's data interleaved in the middle of it.
Is there a way to keep the append as one locked iteration and instead bound the cl_dio_pages batch itself (in ll_direct_IO()/cl_dio_pages_init()) at the stripe boundary?
This contradicts the assertion 14 lines above, which says the opposite about the same queue:
/* it could only be mirror read to get here therefore
* the pages will be transient. */
LASSERT(page->cp_type == CPT_TRANSIENT);
If transient pages genuinely never reach lov_io_submit(), that earlier LASSERT can never be satisfied and would LBUG the client on any buffered read that lands on an uninstantiated component. Worth resolving one way or the other while this function is being touched.
There is no OST count precondition, so on a single-OST setup `-c 2` silently yields one stripe and the test passes without exercising anything. Consider:
(( OSTCOUNT >= 2 )) || skip_env "needs at least 2 OSTs"
Also, the file is empty here, so the append starts at offset 0 and is already stripe aligned. Appending to a file that is not a multiple of the stripe size (say seed it with 512KB first) is the case where the split has to get the first partial stripe right.
Both md5sums read the same OST data - the first one populates the page cache from the OSTs after the DIO write - so this compares the file against itself rather than against what was written. Writing from a temp file (or `dd`ing a known pattern) and comparing that checksum would actually catch mis-routed data; right now only the size check can fail. On a related note, the earlier suggestion on this hunk to use `cancel_lru_locks osc` instead of drop_caches doesn't appear to have been picked up in the current revision.
LU-19900 lov: fix DIO with O_APPEND stripe routing When a file is opened with O_APPEND and written via direct IO, all data is incorrectly routed to stripe 0 instead of being distributed across stripes. The root cause is that lov_io_rw_iter_init() does not split append writes at stripe boundaries. Each DIO iteration processes a cl_dio_pages batch, which lov_dio_pages_init_composite() assigns to a single stripe. Without splitting, the entire write goes to one stripe, causing incorrect file sizes (e.g. 7MB instead of 4MB for a 4MB write to a 2-stripe file) and data corruption. Fix by adding stripe boundary splitting to the append path of lov_io_rw_iter_init(), matching what the non-append path already does. Each iteration now covers at most one stripe_size worth of data, so lov_dio_pages_init_composite() correctly assigns each batch to a single stripe. Also add an LASSERT in lov_io_submit() confirming that DIO pages (CPT_TRANSIENT) never reach that path - they use lov_dio_submit() instead. Also revert the append exception for the DIO stripe cache in lov_page_init_composite() since it is no longer needed now that iterations are single-stripe. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I66304a6692eaab9c68a03159e51cf3d27c465a83
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
LU-10026 lustre: add lgzip kernel module lgzip kernel module implements compression according to the deflate/zlib algorithm, through the kernel Crypto API. It provides 2 cipher drivers under the generic name 'deflate': * deflate-lustre-generic of type compression * deflate-lustre-scomp of type scomp Note the 'deflate' name is identical to the in-kernel module, but lgzip registers it with a slightly higher .cra_priority = 110, so that it is preferred over the in-kernel module. Our 'deflate' is also different in that it accepts a compression level as explained below. lgzip kernel module sources are copied from linux v6.2-rc5 and renamed to gzip.c to avoid name collisions. It implements the Crypto API interface, and rely on the deflate/zlib kernel library for compression implementation. It has been modified to grok a compression level, as read from the top 4 bits of the crypto_tfm flags, and pass it to the underlying library. The deflate/zlib library sources are also copied from linux v6.2-rc5 and built statically. Headers have also been copied from linux v6.2-rc5 for consistency, and source files modified to include the copied headers instead of the system headers. All aforementioned sources are located in the lustre_compat/crypto/gzip directory. The lgzip module is always built with Lustre. This patch enhances the test kernel module kcompr.ko to exercise the compression level of the provided 'deflate' module. It also tries to manually load the lgzip kernel module when a file requires compression with the 'delfate' alg. This is a "one-shot" try that allows us to prefer our module that has level support, but continues to at least compress/decompress files even if our own module is not available. EX-bug-id: EX-6206 Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I10b0ec03cf7d0e4fe3dd2898e77f357667209755
| unique failing test | history |
|---|---|
| sanity-hsm@ldiskfs+DNE:test_254b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-12668 ec: Allow resyncing parities for nosync data mirrors
The data and ec mirrors should be treated independently as there are
valid use cases for both data being nosync but not ec as well as the
opposite case. Remove the restriction that we do not re-sync ec when
data has been flagged as nosync.
Fixes: 03191be6f4 ("LU-12668 ec: add support to resync stale ec comps")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: Ia02c49ab64afd645dc201f142141a841f524e506
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-subtest-change crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-subtest-change crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-dne-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
LU-19566 tests: test EC with lfsck EC parity mirror components do not contain regular file data, so it's essential they be recognized as parity components after an lfsck repair. This test verifies this functionality. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0384d7676043cc23f52da7df41385dc61809114a
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
(typo) field is actually named `ff_comp_flags`
(style) rather than duplicating this code each time, it should just incrementally decode the new parts:
```
} else /* if (size >= sizeof(struct filter_fid_217) */ {
struct filter_fid_217 *ff_old = (struct filter_fid_217 *)dst;
ost_layout_cpu_to_le(&ff_old->ff_layout, &src->ff_layout);
ff_old->ff_layout_version = cpu_to_le32(src->ff_layout_version);
ff_old->ff_range = cpu_to_le32(src->ff_range);
}
if (size >= offsetof(dst->ff_comp_flags) + sizeof(dst->ff_comp_flags)) {
dst->ff_comp_flags = cpu_to_le32(src->ff_comp_flags);
}
```
(style) same as above - process each chunk incrementally instead of duplicating code
(defect) this comment should not be removed
Oh, hmm. There was another patch looking to use one of the uid_h fields:
https://review.whamcloud.com/64468 ("LU-18847 mdt: version layout checking")
but it is using the `cr_fsuid_h` field in `struct mdt_rec_create`, so this should be fine.
(style) better to use `enum lov_comp_md_entry_flags` to make it more clear which "flags" these are, and they can be found more easily. The wirecheck.c code will ensure that the enum size remains consistent.
(minor) I think this is a misnomer to write "for filter_fid", since it is primarily used for components. Maybe `layout flags for components and filter_fid`?
(style) this shouldn't be in the comment here, as it will invariably become outdated in the future
(style) should there be a helper function for this, like:
```
/* return minimum size of in-use filter_fid to maximize interop with old tools */
size_t filter_fid_sizeof(struct filter_fid *ff)
{
BUILD_BUG_ON(sizeof(*ff) >
sizeof(struct filter_fid_217) + sizeof(ff->ff_comp_flags));
/* duplicate next chunk when new fields are added to filter_fid */
/* if (!ff->ff_new_field && ff->ff_previous_last_field)
return sizeof(struct filter_fid_next); */
if (!ff->ff_comp_flags /* && ff->ff_layout_version */)
return sizeof(struct filter_fid_217);
return sizeof(*ff);
}
```
(minor) `ff_size = filter_fid_sizeof(ff);`
(minor) `ff_size = filter_fid_sizeof(ff);`
(minor) It isn't clear if there is any value to printing this field, especially *always* printing it (maybe vs. only printing it if it is non-zero for some reason)
(style) 'version_code 2.17' may be misspelled - perhaps 'version 2.16.x should be used'?
(minor) update to 2.17.52.52 at least
LU-19566 lustre: add layout flags to lfsck EC parity mirror components do not contain regular file data, so it's essential they be recognized as parity components after an lfsck repair. They are distinguished by a component flag, so we must add component flag support to lfsck. This support can be landed without the rest of EC support, so this is based on master. There is a test for lfsck + EC in: https://review.whamcloud.com/c/62489 Details: Extended struct filter_fid from 52 to 56 bytes by adding __u32 ff_flags field to store component flags (like LCME_FL_INIT, LCME_FL_PARITY) in OST objects. Created versioned structure (filter_fid_217) for backward compatibility with old 52-byte filter_fid. Modified lu_orphan_rec_v3 to add lor_comp_flags field (replacing lor_padding_1) to store component flags from filter_fid. This allows LFSCK to preserve component flags during reconstruction. Extended wire protocol by adding o_comp_flags field to struct obdo (replacing o_padding_4) and added OBD_MD_FLCOMPFLAGS flag to indicate when component flags are valid in the obdo. Updated client to set o_comp_flags in the obdo. Modified OFD to read o_comp_flags from obdo and store it in ff_flags during writes, setattr, punch, and fallocate operations. Updated LFSCK to read ff_flags from filter_fid and restore component flags when reconstructing layouts from orphan OST objects. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I716dbf24db48cc50385a2fa9bcc446d291380814
LU-20554 quota: preserve partial OST usage
lov_quotactl() returns -ENETDOWN after summing quota usage from
reachable OSTs when an expected target is inactive. quotactl_ioctl()
does not copy this subtotal for -ENETDOWN, so lfs quota reports only
the MDT usage.
Copy dqb_curspace for -ENETDOWN, but leave QIF_SPACE clear. This
allows lfs quota to report usage from the reachable OSTs and MDTs
while retaining brackets to mark the result as incomplete.
Add sanity-quota test 16c to verify that usage from reachable OSTs is
included while another expected OST is unavailable.
Fixes: 001b8dbfacb7 ("LU-3963 cleanup: C89 and build cleanups")
Test-Parameters: testlist=sanity-quota env=ONLY=16c
Signed-off-by: Sergey Noskov <sergey.noskov.sn@gmail.com>
Change-Id: I17252ef34ff32dbdd0561c1570ce66f31cacfb87
| unique failing test | history |
|---|---|
| sanity-hsm@zfs:test_254b | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 10.1 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
The code doesn't seem to match this. gssiam_get_nodemap() calls strlen(gssiam_info->lii_desc.lid_principal) unconditionally, there is no -EINVAL return when the principal is missing. Was that part dropped when the patch was refreshed?
mdt_get_info() only adds
} else if (KEY_IS(KEY_GSSIAM_AUTHENTICATE)) {
rc = mdt_get_info_gssiam_authenticate(tsi);
and mdt_get_info_gssiam_authenticate() checks mdt_seq_site(mdt)->ss_node_id but never looks at tsi->tsi_exp. Is the peer check described here actually in this revision?
This adds a new on-wire struct (gssiam_auth_rep) exchanged between MDTs, so a mixed-version server pair can hit it. Would a Test-Parameters: line requesting interop coverage (serverversion=...) be worth adding here?
(minor) The tree is at 2.17.57 (LUSTRE-VERSION-GEN), and l_gssiam_upcall.8 added earlier in this series already says 2.17.57, so this and the `Added in commit v2_17_56` comment below look a version behind.
(minor) auth_permission ends up hashed twice - once through the new `options` parameter of lustre_gssiam_hash_key() and again via lustre_hash_fnv_1a_64_continue() just below. The new parameter is documented as the client's mount/auth options (lid_options is what tgt_gssiam_get_entry() passes), so feeding a permission mask into the same slot makes the two callers mean different things by it. Should the nodemap name pass the options here and keep the permission to the explicit continue()?
This kernel-doc block describes mdt_get_local_mdt0() with an @mdt argument and a pointer return, but it sits on mdt_gssiam_authenticate_remote(), which takes (lut, cache, entry) and returns int. Looks like a leftover from an earlier revision.
Same here - the block documents mdt_authenticate_gssiam() with @lut/@cache/@entry, but the function below is mdt_gssiam_projid_cb(mdt0_obd, path, projid, resolved_path). None of the documented parameters exist.
(style) These two blank lines, and the pair added inside mdt_prepare() around line 7261, don't look related to GSSIAM and aren't mentioned in the commit message - leftovers from an earlier revision? There are also three consecutive blank lines after mdt_gssiam_authenticate_remote() around line 3024.
(minor) lid is only partially filled in below - lid_principal and lid_projid keep whatever was on the stack. Nothing reads them on this path today, but tgt_gssiam_get_entry() passes the whole descriptor down to the cache ops, so `struct lustre_gssiam_desc lid = { 0 };` would be safer if the patch is refreshed.
gar_expire stays 0 on this path, and that seems to defeat the expired-token signalling on non-MDT0 targets.
tgt_gssiam_rpc_auth_callback() builds "expired:0:", tgt_gssiam_parse_downcall() sets ue_expire = ktime_get_seconds() + 0, and upcall_cache_downcall() keeps that value because it is non-zero. Back in upcall_cache_get_entry(), check_unlink_entry() sees `now < ue_expire` is false and sets UC_CACHE_EXPIRED, so tgt_gssiam_verify() fails the UC_CACHE_IS_VALID() test and returns -EACCES before it can reach
if (entry->u.gssiam.gd_auth_permission & GSSIAM_AUTH_EXPIRED)
GOTO(out_put, rc = -EAGAIN);
The client then gets GSS_S_DEFECTIVE_CREDENTIAL instead of GSS_S_CREDENTIALS_EXPIRED. Should this reply carry a small non-zero grace value so the entry is still valid when the caller inspects it?
(style) Three blank lines between the function and the next comment block; one is enough.
Can lid_principal be NULL here?
tgt_gssiam_parse_downcall() only fills gd_principal when the downcall carries a non-empty third field:
if (principal_str && *principal_str)
OBD_STRDUP(gssiam->gd_principal, principal_str);
but it sets gd_auth_permission from the first field regardless, so a downcall of "rw:600" or "rw:600:" leaves a valid RW entry with no principal. tgt_gssiam_info_alloc() then skips the strdup, tgt_gssiam_verify() returns 0 because the permission is neither DENY nor EXPIRED, and gssiam_handle_init() reaches this strlen() on a NULL pointer.
The local upcall path is still inert in this patch (no sptlrpc/gssiam_server downcall file yet), but once it is wired up a site-supplied auth helper that omits the principal would panic the MDS here.
LU-19921 target: verify GSSIAM tokens on the target When the target gets the init context RPC, it will call tgt_gssiam_verify() to verify the token through the external GSSIAM server by the l_gssiam_auth upcall. Once authenticated, the token is used to generate the security context, which is attached to the export. This patch includes: - tgt_gssiam_verify() and the cache interactions to validate tokens. - l_gssiam_auth utility to mock the external authentication. - tgt_gssiam_local_upcall() to execute the upcall. - tgt_gssiam_rpc_authenticate() and the callbacks to handle authentication via GET_INFO RPC. - Parsing of the downcall payload returned by the authentication server. - Attaching the upcall entry to the export, and destroying the gssiam upcall entry when all of its exports are disconnected. - ofd_dev.c: implement ofd_authenticate_gssiam() and initialize target GSSIAM callbacks in lut_tsi. - lustre_idl.h: define UAPI wire struct gssiam_auth_rep. - Documentation/man8: add l_gssiam_auth.8 and Makefile entry. Additionally, address several robustness and cleanup items: - Order l_gssiam_auth.8 alphabetically in man8 Makefile.am. - In gssiam_get_nodemap(), switch nodemap name derivation from raw token to the authenticated principal (lid_principal), and return -EINVAL if the principal is missing. - In lustre_gssiam.h, add auth options argument to lustre_gssiam_hash_key() so cache and nodemap keys account for mount and auth options. - Verify sender export is a target peer in mdt_get_info() before handling GSSIAM authentication. - Reply GSSIAM_AUTH_DENY when principal is NULL or empty, use cpu_to_le32/cpu_to_le64 for gar_permission/gar_projid/ gar_expire in struct gssiam_auth_rep, and propagate transient server error codes in mdt_get_info_gssiam_authenticate(). - Verify export obd and peer NID match in sec_gssiam_srv.c before accepting handle from class_conn2export(). - In tgt_gssiam.c: assign token_len after allocation, pass demoted RO permission to gssiam_nodemap_name(), initialize str_len = 0, and use tgt_gssiam_cache directly in attach/ detach cache entry helpers. Signed-off-by: Di Wang <ddiwang@google.com> Change-Id: I9fbd92e7e5dcdbc0039f90f547ce3841b005615e
LU-19079 krb: use .lgss to specify principal Allow the .lgss file inside the client mount point to contain a reference to the Kerberos principal to use, with the syntax: krb5princ=<full Service Principal Name> This is an alternative to the 'krb5princ' mount option, designed for easier automated implementation. Test-Parameters: trivial testlist=sanity Test-Parameters: testgroup=review-dne-selinux-ssk-part-1 Test-Parameters: testgroup=review-dne-selinux-ssk-part-2 Test-Parameters: kerberos=true testlist=sanity-krb5 Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I1205f65faed3f60057e301b883ea646c88efcf88 Was-Change-Id: I776908e0ecad51ec3b3b79e5b392f95815d2e569 Was-Change-Id: I82cea28ab2fa2c86671271658e10f744a0f05841 Reviewed-by: Li Xi <lixi@ddn.com> Reviewed-by: Marc Vef <mvef@whamcloud.com>
LU-19079 krb: mount-point specific principal By default, the principal used by the Lustre client to authenticate against the KDC is the first valid one found in the system keytab (usually /etc/krb5.keytab). This means we cannot choose which principal to use for Lustre. However, there is a need to support multiple Kerberos principals at the same time on the client, for instance in the context of GSS based identification to allow different mount points on the same client to be considered part of different nodemaps. To address this need, this patch introduces a new client mount option 'krb5princ', to allow specifying the full Service Principal Name (SPN) to use for Kerberos authentication of this specific client mount. To pass this information to the kernel, we resort to a kernel keyring key whose descriptor is in the form "lustre:<fsname>:<client uuid>", and whose payload is the principal name to use. Having the client UUID in the key desc allows request_key() to find this client-specific key by putting the UUID into the callout info for context negotiation that happens in userspace. There, the SPN contained in the key payload allows to filter out the principals fetched by the krb5 primitives. Note that we have to make use of a temporary kernel keyring key, as the UUID is computed at mount in kernel space, whereas the krb5princ option is handled in userspace by the mount command before filling the superblock. Lastly, Kerberos credentials cache (krb5cc) on the client must be created for each principal being used, to support multiple principals on the same node. sanity-krb5 test_12 is added to exercise the new krb5princ client mount option. Test-Parameters: trivial Test-Parameters: testgroup=review-dne-selinux-ssk-part-1 Test-Parameters: testgroup=review-dne-selinux-ssk-part-2 Test-Parameters: kerberos=true testlist=sanity-krb5 Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: Ic0e059bfadbd37c11fcdfec3884eacf4c028337f Reviewed-by: Marc Vef <mvef@whamcloud.com> Reviewed-by: Li Xi <lixi@ddn.com>
(style) The summary reads as past tense; house style is imperative. "update mirror split for EC support" would match the rest of the tree.
(minor) The body calls out `test_44f` and the `test_7b` rewrite by name, but sanity-ec `test_7e`, `test_7f`, `test_7g` and the new `identify_ec_mirrors()` helper (~240 new lines) aren't mentioned anywhere. Worth a sentence so a reader knows the new coverage is intentional and what it exercises.
@mvef@whamcloud.com, @rsahlberg@whamcloud.com, I see that `--force-no-ec` is listed in LU-19548, but I don't recall what the realistic use case is for that option? Leaving an unanchored EC mirror behind on a file doesn't seem very useful, and there doesn't seem to be any way to reconnect it to a new data mirror even if it was created (nor assurance that the OSTs used in the data mirror are suitable for that EC). (Apologies in advance to Max, but ...) I'm wondering if `--force-no-ec` option should be removed, and EC mirrors should always be moved to the victim file or deleted? That would remove a lot of complexity in the code that I doesn't seem to add any real value.
(style) The .TH date is older than the lfs-mirror-delete.1 date set in the same patch, and predates the current content. Worth bumping to the refresh date.
(minor) The victim now gets two mirrors, and lod_declare_layout_merge() rejects any merge source that has more than one:
if (le16_to_cpu(merge_lcm->lcm_mirror_count))
RETURN(-EBUSY);
So `lfs mirror extend -N -f NEW_FILE` (documented in lfs-mirror-extend.1) fails with EBUSY on a file produced this way, unlike every other split victim. Should this paragraph say so, until extend learns to merge a data+parity pair?
(style) function comment should follow the style as `mdd_split_ea`.
(question) For a data+parity victim, if either side was *STALE* on the source, should the new file always set data *in-sync* and parity *STALE*? The victim has no other data mirror, so data has to be the non-stale primary or the layout is unusable (all-stale is -EPERM; in-sync parity as primary is -EUCLEAN). Only when both were already in-sync should both stay in-sync. Can current mask accomplish this or not?
(defect) llapi_mirror_find_stale() skips LCME_FL_PARITY, so a remaining
parity mirror always yields comp_size == 0 and this returns false.
lfs mirror split/delete then skips the last-good-copy resync even when
every remaining data mirror is stale.
Should remaining parity mirrors be ignored here the same way
data_mirror_remains() ignores them? Co-split also takes the paired
parity, so that parity is not a copy that will survive the split.
Example: file has D1 (EC data, in-sync after a write), P1 (parity),
and D2 (data, stale).
lfs mirror split --mirror-id D1
last_non_stale_mirror(D1) sees remaining mirrors {P1, D2}. P1 is
skipped by find_stale, so comp_size == 0 and the function returns
false. Resync is skipped, co-split removes D1+P1, and the source is
left with only stale D2.
(minor) The comment is narrower than the condition. `mflags & MF_DESTROY` with a NULL victim_file is also true for `lfs mirror split -d` (which cannot take -f at all), for `lfs pcc detach` (`MF_DESTROY | MF_FOREIGN`), and for the internal `mirror_split(name, 1, NULL, MF_DESTROY, NULL)` call. Naming only `lfs mirror delete` makes the flag look narrower than it is.
(minor) mirror_split() also backs `lfs mirror delete` (SO_MIRROR_DELETE sets MF_DESTROY) and `lfs pcc detach`, so a user who typed `lfs mirror delete` is told the mirror is being "split". sanity-ec test_7f greps for this text from a `lfs mirror delete` run, so that wording is on a tested path. Also, every other message in this function is prefixed with `progname`; this one isn't, so in a pipeline it isn't obvious which tool emitted it.
Skipping the orphan parity comp leaves `comp_array[i].lrc_synced` at the value `_mirror_find_stale()` set, which is unconditionally `true`:
comp[idx].lrc_synced = true; /* liblustreapi_layout.c */
Back in lfs_mirror_resync_file() every entry with `lrc_synced` is fed to LL_LEASE_RESYNC_DONE, and lod_declare_update_sync_pending() then does `llc_flags &= ~LCME_FL_STALE` for each id.
So `lfs mirror resync` returns success and drops LCME_FL_STALE from a parity component it never recomputed - `lfs getstripe` afterwards shows it as in sync. Would `comp_array[i].lrc_synced = false;` before the `continue` be more honest here?
LU-19548 lfs: updated mirror split for EC support
An EC data mirror and its parity mirror form a pair joined by a
bidirectional link (lcme_mirror_link_id). Splitting one mirror
without the other strands a parity mirror that protects no data,
or leaves a data mirror with no EC protection.
When splitting a data mirror, the MDS now splits (or destroys) its
paired parity mirror in the same atomic operation. The pair is
found via the mirror link id rather than by assuming the data and
parity components are adjacent in the layout, and the link is only
followed while it still describes an intact pair: the linked mirror
must exist, carry LCME_FL_PARITY and link back. A link failing
those tests is stale, and following it would co-split an unrelated
mirror. The link on any mirror left behind is cleared so no
dangling reference remains.
A parity mirror can still end up with no data mirror to pair with,
since a stale link is not followed and is cleared instead. lfs
mirror verify and lfs mirror resync resolve a parity component to
its data component through that link, in
__llapi_layout_find_data_comp_by_parity(), so both used to fail the
whole file with -ENOENT; once anything was written to the file the
orphan was marked stale and could never be resynced again. Skip a
parity component that has no data component to pair with instead.
mirror_split() used its "purge" flag for two things: to mean "the
mirror is deleted by handing fd in as the victim", and as the guard
for the retry that copes with an old MDS rejecting fd == fdv. The
flag started out true for every split, so a split to -f NEW_FILE or
to the default victim file also retried after an -EINVAL or -EBUSY
from the MDS, re-opened with O_CREAT|O_EXCL the victim it had
already created, and reported "create victim file failed: File
exists" in place of the real cause. The same flag guards the close
of the victim descriptor, so those splits leaked one descriptor per
file and splitting a batch of files in a single command ran out of
descriptors partway through. Start "purge" as the condition it
describes.
Such a failure was also invisible to a script, since mirror_split()
returned the leftover 0 of an earlier helper on several error paths:
a split that could not create its victim file, or that named a
mirror, component or pool the file does not have, printed an error
and exited 0. Set an error on those paths. Add sanity-flr test_44f.
A split is refused if it would remove every mirror, and also if it
would leave the file without a data mirror. Parity mirrors alone
cannot serve reads: with no data mirror the layout is no longer
FLR, so lov_io_mirror_init() returns at its !lov_is_flr() check
ahead of the parity checks, the parity mirror is picked for IO, and
reads hand back parity stripes as file data. A lone data+parity
pair therefore cannot be taken apart by splitting the data mirror;
split the parity mirror instead to turn the file back into a plain
data file.
The warning printed when a parity mirror is split names the data
mirror that loses EC protection, and is skipped for a parity mirror
whose link has been cleared, which protects no data to begin with.
Splitting a parity mirror directly no longer requires -d. That
restriction, added by commit dfb93e4978b0 ("LU-19548 lfs: mirror
split -d for parity mirrors"), existed to keep a parity-only
layout off a real file. Refusing to strand the data side now covers
the source file, and a victim file holding only the parity mirror
reports size 0, because lov_attr_get_composite() does not take size
from parity components, so it cannot pass parity off as file data.
A warning is printed instead, since removing the parity eliminates
EC protection of the data mirror. test_7b is updated for the new
behaviour.
A victim that receives a data+parity pair keeps its LCME_FL_STALE
markers, so a parity mirror that was stale at split time no longer
lands in a victim that claims an in-sync pair.
lod_declare_layout_purge() refused a purge buffer holding more than
one mirror. lfs mirror delete of a data mirror now hands it the
data+parity pair, so the check is relaxed from lcm_mirror_count
!= 0 to > 1.
Also fix two endian bugs that predate this work in the rewritten
function, harmless on little-endian but wrong on big-endian:
lcm_flags was converted twice, and the __u16 lcm_entry_count was
written with cpu_to_le32().
collect_mirror_id() fills an array owned by its caller but never
knew how large that array was, and every caller passed 128 entries
while lod.*.mirror_count_max accepts up to LUSTRE_MIRROR_COUNT_MAX
mirrors. Splitting a mirror out of a file with more mirrors than
that wrote past the end of a stack array. Carry the array size in
struct collect_ids_data, check it in the callback, and size the
arrays LUSTRE_MIRROR_COUNT_MAX so the check cannot be reached.
Fixes: 23b2d4781899 ("LU-10420 flr: split a mirror from mirrored file")
Fixes: c6e7c0788d7c ("LU-10258 lfs: lfs mirror copy command")
Fixes: b2d73351e646 ("LU-14521 flr: delete mirror without volatile file")
Fixes: fb790204ce3e ("LU-17908 layout: preserve non-FLR state layout flags")
Test-Parameters: testlist=sanity-ec ostcount=8
Test-Parameters: testlist=sanity-flr
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Assisted-by: ClaudeCode:Opus-4.8 llm_code_and_review_tools
Change-Id: I69ca706127e8e0b0e26c88525ac5509e22fa912a
LU-12669 ec: recover data from parity
Restore read data when some OSTs are unavailable from updated parity
codes.
When normal read fails, the read would switch to CIT_EC_RD, and the
ec read inner IO would expand to cover the whole raid set stripes, the
extent lock of the ec_rd would cover the available data OSTs, then
it reads pages from available data objects and parity objects,
calculates and fill in the missing data pages so that the original
outer IO can pick them up and copy to user buffer.
The ec read outer IO is to keep track of the original read position
and count, and the ec read inner IO is to expand the IO to cover the
whole raid set stripes.
lov_ec_read_stripe_pages() would read all pages for one data stripe
across all page positions in a recovery group in a single
cl_io_submit_sync call. And lov_ec_read_parity_stripe() would read
all pages for one parity stripe in a single cl_io_submit_sync call.
Also catch DIO read failure and switch to CIT_EC_RD to recover data
from parity (buffered IO).
For EC recovery reads, the page beyong the end_index has no DLM lock
(EC recovery only locks the actual read range), so return -EIO to stop
the kernel from retrying. And for EC recovery read, verify DLM lock
coverage for every page, pages on good stripes need to be read from
OSTs and have locks, pages on deactivated stripes should already be in
the page cache from EC reconstruction. If a page without lock coverage
reaches ll_readpage(), it maps to a failed stripe -- return -EIO to
prevent LBUG in osc_req_attr_set().
Introduce a new cl_page_alloc_sub() to allocate a cl_page for a
sub-object. The sub-object is the lovsub object for a parity stripe,
the difference it's from cl_page_alloc() is that the page index of the
sub-object is relative to the parity object, not to the file.
This patch also fixes a resync crash on overstriped parity mirrors. If
a file's parity mirror was set overstriped ("-C, --overstripe-count"),
the lov_pattern_supported/available() check failed causing that mirror
not being created and the write to fail.
Fixes: e90c6a428e27 ("LU-12188 uapi: add LOV_PATTERN_PARITY")
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I74977a9148256e2604d948979b5f0b944786e4e0
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.7 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
(minor) "stop handing out EC layouts" doesn't quite match the code. mdt_pattern_types is only read in mdt_connect_internal(); nothing in mdt/ or lod/ consults it when a layout is created, so an MDT with parity removed will still create EC layouts on request. What it stops is clients *using* parity components, and only for clients that mount afterwards. Worth noting too that the negotiated mask does not gate creation on the client either: with lustre.enable_flr_ec=0 (the default) a client can still create an EC file via llite.*.enable_erasure_coding and then get -EIO writing it. Should layout creation be gated on the negotiated mask as well?
This baseline omits LOV_PATTERN_COMPRESS, but lov_pattern_supported() right below still lists `LOV_PATTERN_RAID0 | LOV_PATTERN_COMPRESS` as a pattern this client's IO stack handles.
So for a compressed component lov_lsme_usable() -> lov_pattern_available_mds() computes
pattern_base = 0x801
0x801 & ~LOV_MDS_PATTERN_SUPPORT_217 = 0x800 /* != 0 */
and returns false, on both branches: the compat branch uses this macro, and the negotiated branch uses the MDS mask, which is capped by MDT_PATTERN_TYPES_SUPPORTED = baseline | PARITY. There is no value of mdt.*.pattern_types that can put COMPRESS back (conf-sanity test_163a asserts `+compress` is silently dropped).
The result is that lov_init_composite() leaves every compressed component !lle_valid and skips lco_init(), so lov_io_rw_iter_init() returns -EAGAIN/-EIO for a plain compressed file. This is reachable today: lsme_unpack() accepts COMPRESS components when llite.*.enable_compression is set (lov_ea.c), and lod_generate_lovea() packs them.
Should the mask be derived from what the client actually supports (i.e. include COMPRESS in both the 2.17 baseline and MDT_PATTERN_TYPES_SUPPORTED), rather than a hard-coded RAID0/MDT/OVERSTRIPING set?
(style) F_HOLE and F_RELEASED are layout flags rather than pattern types, and every consumer strips them first (`pattern & ~LOV_PATTERN_F_MASK` in lov_pattern_available_mds(), `& ~(F_RELEASED | F_MASK)` in lov_pattern_supported()). lov_pattern_bit2str() also has no names for bits 30/31, so they never show up in mdt.*.pattern_types or lov.*.mds_pattern_support. They only put two bits nobody reads on the wire in ocd_pattern_support. Could the macro just be RAID0 | MDT | OVERSTRIPING?
(minor) `data` here comes from obd_get_info(KEY_CONN_DATA) on sbi->ll_md_exp, which is the LMV export, and lmv_get_info() forwards that key to `lmv_tgt(lmv, 0)` only. So on DNE the stored mask is whatever MDT index 0 negotiated; mdt.<fs>-MDT0001.pattern_types has no effect on any client, even though the parameter is per-MDT and the man page shows a single-MDT example. Should the client intersect the masks from all MDTs, or should the caveat be documented?
(minor) This version gate was flagged on an earlier patchset ("this version should be updated when the patch is refreshed") and answered "Done", but it is still 2.17.52.224 here and in test_163a/test_163b.
The things these tests check (`pattern_support` in the mdc/osc import, mdt.*.pattern_types) only exist from this patch, so any server in 2.17.53..2.17.57 passes the gate without the feature. The tests do fall through to a skip on the missing parameter, but the gate should be the version this lands in.
LU-12187 lov: MDS layout pattern negotiation
Add 'obd_connect_data::ocd_pattern_support' to allow the client
and MDS to negotiate the file layout patterns that they support.
This prevents clients from trying to use layout types not
recognized by the MDS, and allows the MDS to (potentially)
convert/filter existing file layouts to a format that the
client understands.
This will be used by FLR-EC to negotiate whether the client
and MDS support LOV_PATTERN_PARITY layouts, and others in
the future.
Add LOV_MDS_PATTERN_SUPPORT_217 for compat with pre-2.18
MDS that do not send ocd_pattern_support. Generalize
lov_pattern_available_mds() to check all patterns against
the MDS-negotiated mask. The client sends its supported patterns
to the MDS, which responds with the intersection of both sets.
The client stores the negotiated mask only when the server
grants OBD_CONNECT2_FLR_EC. target_handle_connect() echoes
un-negotiated ocd fields back from the request, so an ungated
store would let the client's own request value defeat the
pre-2.18 fallback.
Gate pattern usability at layout init rather than at parse time:
lov_init_composite() consults lov_lsme_usable() (pattern support
intersected with the MDS-negotiated mask), and leaves unusable
components un-initialized and !lle_valid, the same as components
with an unknown pattern. Parsing itself stays pattern-agnostic
to preserve lsme alloc/free symmetry.
Adds mdt.*.pattern_types which gives an administrator the
ability to enable or disable layout types at runtime, and a
read-only lov.*.mds_pattern_support which exposes the
effective negotiated mask on the client. The mdt_enable_flr_ec
module parameter now only sets the initial pattern_types value
when an MDT starts up, so mdt.*.pattern_types is what changes
the advertised types at runtime. Add man pages for both
parameters and for the renamed lustre.enable_flr_ec parameter.
Rename the 'llite_enable_flr_ec' module parameter to
'enable_flr_ec', so it is accessed as 'lustre.enable_flr_ec'
per LU-14144 convention. Add the 'lustre' module to the libcfs
parameter path list so the client module parameters are reachable
via 'lctl {get,set}_param --module' instead of a hard-coded
/sys/module path.
Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=conf-sanity env=ONLY="163 163a 163b"
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Change-Id: Iaef716e28014be5b91dd50b117dd0881f5c37f37
(defect) This version check should be `2.15.8`. Test 9 was never run for this patch as it was skipped.
More strictly, it should be "> 2.15.8" because the 2.15.8.0 release didn't have this fix either. Any patch built on b2_15 will be at least 2.15.8.1, so that test will pass. More importantly, this except file is only going to be used for interop testing with *older clients* running with this on the server, and we need this `mmp.ex` file **on master as well**, so that unpatched/older clients will also skip the test when running against master servers (which is where most of the testing is being run).
LU-19784 tests: mmp/test_8 wait time changes To prepare for the e2fsprogs/libext2fs changes reducing the wait time when the MMP sequence number is EXT4_MMP_SEQ_CLEAN, LU-19784 ext2fs: mmp wait time ajustments https://review.whamcloud.com/63478 We need to make sure the sequence number is not EXT4_MMP_SEQ_CLEAN before running e2fsck. We can use debugfs to change the MMP sequence number, rename mark_mmp_block() to set_mmp_seq() to reflect this. Remove mmp_mark.sh, which is not needed any more, and use set_mmp_seq() in test_9. Make sure we tear down the dm flakey device in test_9 otherwise during mmp_fini()->disable_mmp() will corrupt the filesystem. Lustre-change: https://review.whamcloud.com/63488 Lustre-commit: ad051b359db12ef7e752abcf2aa8e34199ded041 Test-Parameters: trivial testlist=mmp Signed-off-by: Li Dongyang <dongyangli@ddn.com> Change-Id: I81b9d28061862a99b04fa92de0cc18a536c9d17b Reviewed-by: Artem Blagodarenko <ablagodarenko@thelustrecollective.com> Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lnet. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-13814 osc: add osc_dio_page_submit This patch adds the osc_dio_page_submit and specializes osc_page_submit to take only BIO pages. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I048bbcd25ffee28fbbd8fee26017796b5e3586bd
| unique failing test | history |
|---|---|
| sanity-lfsck@ldiskfs+DNE:test_18g | seen in 13 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18h | seen in 13 other reviews |
(minor) LU-19536 in JIRA is titled "fault_in_iov_iter_readable() with a spinlock held", and the other changes landed under it (DIO pool ENOMEM drain/retry, DIO buffer double-free, wait queue for DIO copy sync) are all in the unaligned-DIO copy machinery. Grant/dirty accounting for DIO writes reads like a separate topic. Would a dedicated ticket be a better home, or is LU-19536 intended as an umbrella for all of the DIO work?
(minor) The comment doesn't quite match what happens. osc_wake_cache_waiters() is only `wake_up(&cli->cl_cache_waiters)`, so nothing re-enters this function - there is no recursion. The actual hazard is that osc_enter_cache() evaluates this as the condition of wait_event_idle_exclusive_timeout_cmd() on that same queue, so the wake would land on the evaluating task itself (and osc_enter_cache() already wakes the queue once it succeeds). Worth rewording so the rationale for the no-wake variant is accurate.
(minor) This bakes in an assumption that any `cl_dio_pages` reaching the OSC is embedded in a `cl_sub_dio`, which isn't part of the `cio_dio_submit`/osc_queue_dio_pages() contract.
A few lines down the same sdio is already derived from the page anchor:
ext->oe_csd = anchor->csi_dio_aio;
Since `is_aio` is only needed inside the `!ext->oe_rw` branch, could it just read `ext->oe_csd->csd_ll_aio->cda_is_aio` there and drop the container_of()?
Please do this improvement.
LU-19536 osc: enforce dirty limits for regular DIO writes
osc_queue_dio_pages() checked only server grant before queuing a
write. For regular non-AIO DIO, parallel submission could therefore
bypass the per-OSC max_dirty_mb and global obd_max_dirty_pages limits.
Add osc_reserve_dio_grant() to validate the local dirty limit,
atomically reserve global dirty pages, and reserve server grant under
the client lock. Check dirty pressure first so a simultaneous grant
shortage cannot hide the dirty limit.
When a regular DIO extent cannot enter dirty accounting, submit it
synchronously and wait for that extent before forming more unaccounted
RPCs. Preserve existing no-grant and true AIO paths. Keep buffered
osc_enter_cache_try() accounting unchanged.
Remove the obsolete DIO grant branch from osc_queue_sync_pages().
Since LU-13814, DIO requests use osc_queue_dio_pages(). Rename
__osc_unreserve_grant() to osc_unreserve_grant_no_wake() to document
the no-wakeup rollback needed by waitqueue callers. Add __must_hold
annotations to document the locking requirements of grant and cache
helpers.
Add sanity test_398v using one exact OSC, warming its grant, and
pausing OST bulk completion. Verify that four DIO RPCs do not all
overlap at the dirty limit. Check final dirty counters and compare the
client and server grant changes around the tested DIO.
Fixes: 8efbad8ff4ed ("LU-13814 osc: add osc_queue_dio_pages")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I115232216ae77740f0a779994e8a020eedeebfef
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_63c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
(suggestion) This is the last DIO assumption left in lov_io_submit(), and the patch's own premise says it can no longer hold: every page that reaches here now comes from a cl_page_list, and the only CPT_TRANSIENT pages in the tree are built by ll_direct_rw_pages() into a cl_dio_pages, which goes to lov_dio_submit() instead. So an empty page arriving here would be CPT_CACHEABLE and this would LBUG.
In practice the branch looks unreachable too - lov_page_init_empty() does SetPageUptodate(), and all four submit paths (ll_readpage(), ll_read_ahead_page(), ll_prepare_partial_page(), ll_io_zero_page()) skip uptodate pages before queueing. If so the whole block is dead and could go with the rest of the DIO leftovers.
The comment is stale either way: cl_page_prep() here was removed by 169f076ae0 ("LU-13814 clio: remove cl_page_prep for transients").
(minor) Not a bug, but with `dio` dropped the only remaining value in this message is `ext->oe_srvlock`, and the enclosing condition is `dlmlock == NULL && !ext->oe_srvlock` - so it always prints `srvlock: 0`. If the patch is refreshed, either drop that field or print something that varies, e.g. `oe_rw`.
LU-19536 osc: remove dead DIO handling from sync path Since LU-13814 routed DIO through cio_dio_submit() and cl_dio_pages, transient DIO pages no longer reach lov_io_submit(), osc_io_submit(), or osc_queue_sync_pages(). The DIO-specific branches and setup left in the ordinary page-list path are therefore dead. Remove the stale DIO handling from the LOV and OSC submit paths, including OBD_BRW_NOCACHE setup, the oe_dio assignment, and the sync_io anchor setup. The ordinary LOV path now always groups pages by stripe, while osc_io_submit() always prepares its pages. OBD_BRW_NOCACHE is set only by osc_dio_submit() before it calls osc_queue_dio_pages(). Since oe_dio is always false in the sync path, simplify extent priority handling and omit oe_dio from the NOLCK diagnostic. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com> Change-Id: I63711a7f3bc3699680e1c09d3349291ccebb75ec
This strips local_admin regardless of where the bit came from, so an explicit request is now silently discarded rather than refused.
Starting from `rbac=all` then `rbac=-file_perms` (stored value 0xfffffdfe):
lctl nodemap_modify --name default --property rbac=+local_admin
rbac_mask becomes 0xfffffffe, the bit is cleared here, the value matches old_rbac and nodemap_set_rbac() returns 0. Before this change the same command returned -EINVAL. The mixed form `rbac=local_admin,+dne_ops` behaves the same way, while the pure absolute `rbac=local_admin,dne_ops` is still rejected.
The commit message says absolute sets are left alone so `rbac=local_admin` is still rejected — should the incremental form keep rejecting it too when the user names the role, e.g. only clear the bit when `param` does not contain "local_admin"? Otherwise the admin gets rc = 0 for a request that was not carried out.
Seems legit?
This probe cannot fire here: test_93 already requires MGS_VERSION >= 2.17.52 at the top, which is far newer than the version that added nodemap.default.rbac. Elsewhere in the suite the same probe runs before any setup; here it sits after ~90 lines of assertions that have already passed, and skip() calls exit 0, so a skip at this point would report the whole subtest as skipped.
If the intent is to gate the new section on the server carrying this fix, a version check would say that directly, and it needs to be an `if` block rather than `skip` so the earlier results are not thrown away:
if (( MGS_VERSION >= $(version_code 2.17.57) )); then
...
fi
The bit this patch actually manipulates is never checked. Adding `check_not_contains "$val" local_admin` here (and after the +all,-dne_ops step) would catch a future change that stops clearing it. A case asserting that `--property rbac=local_admin` on default still fails would also cover the second half of the commit message, which is currently untested.
LU-19901 nodemap: fix incremental RBAC updates on default nodemap
When applying incremental RBAC updates (+/-) on the default nodemap,
local_admin must be dropped after parse. NODEMAP_RBAC_ALL includes
that bit, which is not valid on default, so leftover local_admin from
stored "all" or from +all inside the string turns a valid update into
-EINVAL. Leave absolute sets alone so rbac=local_admin is still
rejected.
Add test coverage for -file_perms from all and +all,-dne_ops on
default.
Fixes: fa2726559112 ("LU-19901 nodemap: support incremental RBAC role updates")
Signed-off-by: Chakshu Kansal <ckansal@ddn.com>
Change-Id: I0ae8c2b763ac0733ea64be115a1eb01df45ae013
LU-20390 enc: fix subdir mount of an encrypted directory
When an encrypted directory is used as a subdirectory mount, its
".fscrypt" dir is served from the real fs root via the lu_root_fid
redirect. This exposes three problems addressed by this patch:
- Once 'fscrypt unlock' loads the key, llcrypt encrypts the
".fscrypt" name so it can no longer be looked up. Resolve it by
its plaintext name in ll_prepare_lookup()/ll_setup_filename()
regardless of key state, and drop the now-redundant -ENOENT
handling that only covered the keyless case.
- This consequently prevents the ".fscrypt" dentry from being
invalidated when the key loads, thus avoiding the subtree to detach
from s_root and escape shrink_dcache_for_umount(), which was leaving
inodes pinned whose cl_objects trip lu_device_fini() at umount.
- 'fscrypt lock' wrongly reports busy files because the mount root
inode is pinned for the mount lifetime; skip it in
check_for_busy_inodes().
Investigated with the help of Claude Code + Tools.
Fixes: c12378fba7f0 ("LU-15176 sec: present .fscrypt in subdir mount")
Test-Parameters: mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec clientdistro=ubuntu2604 serverdistro=el8.10 env=ONLY="54"
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ib0576f1c31416bc10dc742e5ef330eff4b5ec541
(typo) "Highlighs" -> "Highlights".
(minor) There is no Test-Parameters: line. This is ZFS-only functionality, so a default review run will not exercise any of the new code. Something like:
Test-Parameters: fstype=zfs testlist=sanity env=ONLY=150f
Also, an earlier revision asked for this to be marked "fortestonly" while it is still under development; that is still not in the message, and the latest patchset comment says the test needs more work.
The default is documented as 0, but osd_handler.c initialises `osd_fallocate_zero_blocks = 2` and osd_device_init0() copies that into od_fallocate_zero_blocks, so a freshly mounted ZFS target comes up with 2. Combined with the DESCRIPTION below (0 and 1 "will always return -EOPNOTSUPP"), the page currently reads as though fallocate is off by default when in fact punch is on. Should this say 2?
(style) this isn't a bug, but the second `.EX` block is never closed with `.EE`, so groff stays in no-fill constant-width mode and AVAILABILITY and SEE ALSO render as example text. The first example above, and every block in osd-zfs.fzap_blockshift.4, close with `.EE`.
(minor) This debug line and the `svname` local are unrelated to fallocate - `svname` exists only to feed this one CDEBUG. Could it be dropped, or split into its own patch? od_svname is already set up by osd_device_init() and printed by the surrounding code.
The start is rounded up to the next encryption unit, but the end is left as-is, so the rounding is only half applied. With LUSTRE_ENCRYPT_FL and an unaligned end (e.g. sanity 150f punches [4000, 7000)), dmu_free_range() zeroes ciphertext in the middle of the unit that spans `end`. The AES block containing `end` then no longer decrypts, so a few bytes past the requested end read back as garbage on the client. Should end be rounded down to LUSTRE_ENCRYPTION_MASK for symmetry? (The ldiskfs fallocate-punch path does no rounding at all, so dropping the rounding entirely would also be consistent - but doing one end and not the other looks unintended.)
+1
(typo) "osd_declare_falloate()" -> "osd_declare_fallocate()".
The first test is subsumed by the second (`-1 <= 1`), so only `<= 1` is doing any work here. The message also says "ZB == -1 or ZB <= -1", which does not describe what is checked - `ZB <= 1` was presumably meant.
+1; and in CDEBUG(ZB <= 1) not ZB <= -1
(minor) `check_set_fallocate()` applies the new value with `do_nodes $(osts_nodes) ...`, so on a config where the MDS is a separate node the MDT copy of `osd-zfs.*.fallocate_zero_blocks` keeps its default. With `DOM=yes` the punch at offset 4096 below lands on the DoM component, so `osd_declare_fallocate()` on the ZFS MDT returns -EOPNOTSUPP and the whole subtest ends in `skip_eopnotsupp` rather than testing anything. The guard also only looks at `$ost1_FSTYPE`, so a ZFS MDT with ldiskfs OSTs never even reaches `check_set_fallocate`. Since the patch enables the punch path for MDT devices too, is it worth setting the parameter on the MDS as well?
(minor) `fa_mode`/`mode` moved above the memo check, so every call now does a remote `lctl get_param` even when `$supported` is already cached, and `${!fstype}` is read before the `eval export $fstype=...` fallback a few lines down (giving `osd-.<svc>.fallocate_zero_blocks`).
Both go away if these two `local`s stay where the parameter is actually needed, i.e. below the cache check and the fstype fixup.
Probing with `punch` here also changes what `check_set_fallocate_or_skip()` does on ZFS, and that helper is used by ~20 subtests that need fallocate *preallocation*.
On a ZFS OST the new `osd-zfs.*.fallocate_zero_blocks` parameter now exists, so `check_fallocate_supported ost1 punch` returns its value instead of failing:
old_mode="0"
[[ -z "$new_mode" && "$old_mode" != "-1" ]] -> return 0
Previously the bare `check_fallocate_supported` hit the "fallocate on zfs doesn't consume space" branch, `old_mode` was empty, and the callers skipped.
So sanity-quota/1h, ost-pools/24, conf-sanity/133, sanity-ec and the sanity 150* preallocate cases no longer skip on ZFS and run into the `error`/`quota_error` paths instead (e.g. ost-pools.sh `fallocate -l$((towrite * 1024)) $tmpfile || error "can't fallocate"`).
Should `check_set_fallocate()` still probe `alloc` when the caller is not asking for punch, so those tests keep skipping?
(minor) `fa_mode` is built from ${!fstype} here, but ${facet}_FSTYPE is only defaulted from facet_fstype() further down, so the fallback no longer has any effect on the parameter path. Same for `version`: get_lustre_env() only exports MDS1_VERSION/OST1_VERSION, so for any facet other than mds1/ost1 the arithmetic below expands to `(( < NNN ))` and bash reports a syntax error.
No caller passes such a facet today, but moving the fstype fallback above these two lines would keep the guard meaningful. The `do_facet` on the next line also runs before the cached-result early return just below, so every call now costs a remote command.
LU-14157 osd-zfs: Add fallocate PUNCH for ZFS
This patch add fallocate -p (PUNCH) feature for ZFS
After a successful call, subsequent reads from this
range will return zeroes. Also, when using this option,
--keep-size is implied
Highlighs of this patch
- adds fallocate PUNCH for ZFS
- adds extra debug info under osd_device_init0()
- changes sanity/150f to exercise zfs punch operations.
- Add check for "alloc" under sanity test so that fallocate
exercising generic fallocate would not call into ZFS.
test which "check_fallocate_or_skip ost1 alloc" were added is
- 101m, 150bb, 150c, 150d, 150e, 150g, 150h, 253, 600[a-d]
- Add check for "alloc" under sanity-quota test so that fallocate
exercising generic fallocate would not call into ZFS.
test which "check_fallocate_or_skip ost1 alloc" were added is
- 1h, 78a, 78A
- Add check for "alloc" under ost-pools test so that fallocate
exercising generic fallocate would not call into ZFS.
test which "check_fallocate_or_skip ost1 alloc" were added is
- 29, 31
All ZFS punch test are moved into sanity/150f
- changes check_fallocate_supported() function under
test-framework so it would allow only "punch" operation
for ZFS
- changes check_fallocate_or_skip() to take extra args
which represent the fallocate operation
- changes check_set_fallocate() so it can handle ZFS
- Adds fallocate_zero_blocks tunables to control fallocate
operation on OSD ZFS
- Adds man page osd-zfs.fallocate_zero_blocks.4 under
Documentation/man4
Was-Change-Id: Ib9a55924a2f4a1dfa28ca9516e19099103c81105
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ib9a55924a2f4a1dfa28ca9516e19099103c81102
(suggestion) With `VERBOSE_EC_MAP` in `VERBOSE_DEFAULT`, plain `lfs getstripe` on a parity component now prints `lcme_dstripe_count`, then `lcme_cstripe_count`, then `lcme_ec: k+p` - the same two numbers a second time. Would restricting `lcme_ec` to the `--ec-map` dump, where the two counts are not printed, avoid the duplication in the normal layout output?
(style) An earlier patchset asked that new fields could be added at the end of the raidset mapping without breaking this test, and it was marked done, but these patterns still match the complete flow mapping including the closing `}`. Adding a key turns `parity_stripes: "0-1" }` into `parity_stripes: "0-1", ... }` and every one of these greps fails. Matching only the leading fields would leave the format open to extension.
(style) Not a bug, but the `%4s` with a `" "` argument was raised on an earlier patchset and marked done; the indentation here still goes through `%4s` rather than literal spaces in the format string. The raidset item lines went the other way - they used to be plain `" - %d: ..."` and are now `"%6s- %d: ...", " "`.
(style) This isn't a bug, but every other key in this dump lands its value in the same column - `lcme_id:`, `lcme_ec:` and `lcme_dstripe_count:` all pad to 25 characters. `lcme_ec_raidset_count: ` sits two columns further right and `lcme_ec_raidsets: []` two columns further left, so the block reads ragged. A shorter key that fits the existing padding, e.g. `lcme_ec_nr_raidsets`, would line up with the rest.
(style) `sc.esc_k0`/`sc.esc_k1` and the `data_start`/`parity_start` cursors are all `int`, but they are printed with `%u`. `%d` matches the declared types. Same in the second loop.
(minor) When `-I` names a data component, the `lcme_id` shown is the data component, but `parity_stripes` are indices into the linked parity component whose id never appears anywhere in the output. Would it be worth also printing the parity `lcme_id` (or `lcme_mirror_link_id`) so those ranges can be resolved to a component?
LU-20036 lfs: add --ec-map getstripe output for EC raidsets
Add lfs getstripe --ec-map and VERBOSE_EC_MAP to print
lcme_ec_raidset_count and lcme_ec_raidsets for EC layouts.
$ lfs getstripe -I$comp_id --ec-map file
components:
- lcme_id: 131074
lcme_ec: 3+2
lcme_ec_raidset_count: 3
lcme_ec_raidsets:
- 0: { ec_data_count: 3, data_stripes: "0-2",
parity_stripes: "0-1" }
- 1: { ec_data_count: 2, data_stripes: "3-4",
parity_stripes: "2-3" }
- 2: { ec_data_count: 2, data_stripes: "5-6",
parity_stripes: "4-5" }
Default/-v include the mapping on parity components in the
normal layout dump. --ec-map (fp_ec_map_only) prints only
lcme_id plus the raidset map and skips sub_layout; without
-I every parity component is shown, with -I either a data
or parity component is accepted and the peer is found via
lcme_mirror_link_id.
Defer the "components:" header until the first matching
entry so a file with no parity component is not a YAML
null key; the warning goes to stderr and stdout is empty.
An uninstantiated later PFL component still prints lcme_ec
as k+p with lcme_ec_raidset_count: 0 and lcme_ec_raidsets: [].
Document in lfs-getstripe(1) and ec.txt; add sanity-ec
test_31f and test_31g.
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Keguang Xu <kxu@ddn.com>
Change-Id: I31aed1b518ecd4d528d7af8b8d6ba9a7ec4bf07c
no Fixes: line for the patch that introduced this?
this patch was pushed 8 month ago and not refreshed since. that means no ai review (this is a note to myself to run it before pushing)
(defect) This drops the `if (skip_index)` condition but keeps its body, so the assignment two lines above is now dead - it is always overwritten. Removing skip_index should mean the code behaves as it did when skip_index was 0, i.e. the window end stays anchored at ras_window_start_idx:
if (ras->ras_window_pages > 0)
end_idx = ras->ras_window_start_idx + ras->ras_window_pages - 1;
end_idx = *start_idx + ras->ras_window_pages - 1; /* was skip_index only */
As written the RA window end is now anchored at ras_next_readahead_idx for every read, not just the mmap range case, which is a behaviour change the commit message describes as a simplification. Is that intended? If so it deserves a sentence in the message and the dead lines above should go.
The `ras->ras_window_pages > 0` guard also disappears with it. When the window is zero the new expression underflows to `*start_idx - 1`, and with `*start_idx == 0` that wraps to ULONG_MAX; the EOF clamp below then turns it into `[0, eof_index]`, i.e. a whole-file readahead window, where the old code fell through to the `end_idx == 0` RA_STAT_ZERO_WINDOW return. ras_detect_read_pattern() reaches that state - the stride-invalidation path sets `ras_window_pages = 0; ras_next_readahead_idx = index;` with index 0, and the mmap hit path in ras_update() sets `ras_window_pages = 0` without touching ras_next_readahead_idx.
this... sounds somewhat unintentional? I guess forcing whole file readahead would do wonders to tests that expect great readhead, but what about actual workloads? do we have any actual tests?
(style) prefer `((...))` for numeric comparisons
(style) this isn't a bug, but the numeric comparison was asked to move to `(( ... ))` on an earlier patchset and the line is still using `[ ... -eq ... ]`:
(( miss == 2 )) || error "expected misses 2 but got $miss"
LU-15069 llite: remove skip_index Skip_index seems to have been a confused attempt to change mmap readahead behavior, but basically just turns off a number of readahead optimizations in a weird way and sometimes causes us not to read a page for - as far as I can tell - no reason. This causes misses in normal read patterns. It also skips the minimum page reservation logic, again for no reason I can see - mmap reads should use the same logic here as everything else. Remove it entirely to simplify the code a little. This also reduces the number of misses we take in a simple readahead test, which is a nice indication of how this code is affecting readahead. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia953c8a8cb6dc175255ac316b4b3f02d78effae9
LU-20078 osc: call osc_extent_tree_dump() without object lock held
The panic of next_extent() in osc_extent_tree_dump() is caused by:
- osc_extent_tree_dump() walks the extent rbtree without holding
object lock: osc_object_lock().
- LU-19014 added a new debug call to osc_extent_tree_dump() in
osc_cache_writeback_range() for the IO_PRIO_DIRTY_EXCEEDED path.
- next_extent() assumes the passed exctent is still in the tree
and asserts on RB_EMPTY_NODE(); This is reasonable for normal
tree walkers under the object lock, but unsafe for an unlocked
debug dump.
This patch fixes it by removing the debug code added by LU-19014:
osc_extent_tree_dump().
This patch also fixes the similar call of osc_extent_tree_dump() in
osc_enter_cache() by adding object lock.
Fixes: c413d2ede5 ("LU-19014 memcg: fix client hang in
balance_dirty_page()")
Change-Id: I7802d951f1c01d92a9240e7b4d395104b52c8969
Signed-off-by: Yingjin Qian <qian@ddn.com>
(style) This isn't a bug, but the new members break the numbering the struct already uses - sl_dv1/sl_dv2 pair up, while the FIDs are sl_fid/sl_fid2. sl_fid1/sl_fid2 (and msl_fid1/msl_fid2 in mdc_swap_layouts) would read consistently, and these are uapi names that are awkward to change once they land.
I wouldn't be against using `fid1` here.
The whole point of this struct is that its size is frozen at 32 bytes, since that is what encodes the compat command number. Nothing pins it though - check_swap_layout() in wirecheck.c gains CHECK_MEMBER for sl_fid/sl_fid2 but no CHECK_STRUCT(lustre_swap_layouts_old), so a later edit to the struct would move LL_IOC_LOV_SWAP_LAYOUTS_OLD with no test failure. Worth adding it next to the existing lustre_swap_layouts checks? This isn't a bug, but the placement reads oddly too: a struct definition in the middle of the LL_IOC number block, with the _OLD macro ahead of the primary one. The LL_IOC_SET_LEASE / LL_IOC_SET_LEASE_OLD pair a few lines below is ordered the other way.
(style) Block comments use a trailing */ on a separate line
This should also/instead be masked at the server so that it cannot be sent by a misbehaving/malicious client.
(style) this comment is not matching the referenced code, since the masking was done earlier (where a similar comment already exists).
(defect?) If we have an OBD_CONNECT flag for this then we can change the CLOSE path/RPC to also send the FIDs, instead of working around it. The server already needs to handle this operation, so no point in working around it on the client.
The new _OLD command is handled here, but ll_dir_ioctl() still matches only the new number:
case LL_IOC_LOV_SWAP_LAYOUTS:
RETURN(-EPERM);
After this change that case no longer catches an old-header binary, so a swap ioctl on a directory fd falls through to the default arm and ends up in ll_iocontrol() -> obd_iocontrol(cmd, ll_dt_exp, 0, NULL, uarg), returning -ENOTTY instead of the deliberate -EPERM. Should dir.c pick up LL_IOC_LOV_SWAP_LAYOUTS_OLD alongside it?
parse ll_file_ioctl():error: Function too hairy. Giving up. 10 seconds warn: ll_file_ioctl():Function too hairy. No more merges.
(minor) Can snd_lmm be NULL at this point? mdd_get_lmm_gen(snd_lmm, &snd_gen) one line above already dereferences it - mdd_lmm_gen() reads lmm->lmm_magic with no NULL guard - and the earlier `fst_buf == NULL && snd_buf == NULL` return plus the reorder above guarantee snd_buf->lb_buf is set. So only the fst_lmm half of this test does anything.
This scan is a verbatim copy of the one inside the composite branch - dummy-slot skip, ostid_le_to_cpu(), ostid_to_fid(), lu_fid_eq(). Pulling it into a small helper taking (objs, stripe_count, fid) would keep the two paths from drifting if the match logic ever changes.
This isn't a bug, but the size test is the same expression already evaluated in the SINGLE_STRIPE block above. By the time control reaches here, a short message with SINGLE_STRIPE set has already returned -EPROTO, and without SINGLE_STRIPE mdd_swap_layouts() never looks at fid1/fid2 - so both arms behave identically. Hoisting it into a bool next to the earlier check (or dropping the branch and always passing the pointers) would make the invariant obvious.
This is missing a bunch of flags, and this array is positional so it matters:
```
"perfstats", /* 0x100000000000 */
+ "unknown", /* 0x200000000000 */
+ "unknown", /* 0x400000000000 */
+ "unknown", /* 0x800000000000 */
+ "unknown", /* 0x1000000000000 */
"single_stripe_swap", /* 0x2000000000000 */
```
Using `"unknown"` rather than the actual flag name is deliberate so that each patch still adds its own name. Otherwise, if all the names are added in one patch and not in the actual feature patch then this will be missed if the patch is backported.
(minor) Are these two guards reachable in the build they land in? For LUSTRE_VERSION_CODE < 3.4.53 (which is where 2.17.x sits) the block just above returns early whenever SWAP_LAYOUTS_WITH_DV12 is clear:
if (!(msl->msl_flags & SWAP_LAYOUTS_WITH_DV12) || size < ...)
return 0;
so the FIDs are only ever swabbed when the client also asked for DV12. ll_swap_layouts() always ORs WITH_DV12 in, so no in-tree client trips this, but mdt_swap_layouts() validates SINGLE_STRIPE independently of WITH_DV12 - a client that sets SINGLE_STRIPE alone reaches mdd_lmm_find_ost_obj() with un-swabbed FIDs on a mixed-endian pair and gets a confusing -ENOENT. Should the early return be keyed on the size check alone, or SINGLE_STRIPE added to the condition?
(style) preferabl, reverse the condition so that it works properly in both cases:
```
if ((msl->msl_flags & SWAP_LAYOUTS_WITH_DV12) &&
size >= offsetof(struct mdc_swap_layouts, msl_dv2) + sizeof(msl->msl_dv2))
#endif
{
__swab64s(&msl->msl_dv1);
__swab64s(&msl->msl_dv2);
}
```
LU-20534 mdd: add single-stripe layout swap Allow swap_layouts to exchange one OST object between two files, selected by OST object FID. Negotiate with OBD_CONNECT2_ONE_OBJ_SWAP and expose llapi_swap_stripe() for tools such as lfs migrate_stripe. Skip mdd_dom_fixup() and HSM xattr rewrite for SINGLE_STRIPE so MDT DoM data and archive state stay with the parent file. Reject CLOSE|SINGLE_STRIPE; find objects in COMP_V1 and SEL layouts. Test-Parameters: trivial Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I603ad3810c5f3efc1860f477b283acb0c35df1a7
| unique failing test | history |
|---|---|
| sanity-hsm@ldiskfs+DNE:test_252 | seen in 6 other reviews |
(minor) The body doesn't account for several things in the diff: that `nodemap_cmd()` is renamed to `llapi_nodemap_cmd()`, that error reporting switches from `fprintf(stderr)` to `llapi_error()`, and that `get_mds_device()`/`get_oss_device()` are deleted from obd.c together with their `do_disconnect()` and `cur_device` side effects. "make all needed adjustments" leaves a reader to reconstruct those from the diff. The rename point was raised on patchset 30 and is still not in the message.
I checked this and I saw that the fix is not that simple. For now I left it as it is and add a comment to the commit message that this is intentional, but if you think that we have to fix it, I would like to discuss a good way to do it.
It seems like this patch is still too large to review effectively. I think it makes sense to split into smaller parts, like moving `llapi_nodemap_cmd()` in one patch (already quite large), and then adding the individual commands like `llapi_nodemap_fileset_add()`, `llapi_nodemap_fileset_del()` in smaller increments. It doesn't have to be one patch per function, as long as they are related and around 100-200 lines if possible. That is enough to easily review without getting bogged down. To make those patches more independent, rather than adding them "A", "B", ... in order in the file (which forces a strong dependency because of context "A B C D E F"), you could add "C", then "B" and "E" are not conflicting, then "A", "D", "F" are not conflicting. That avoids gratuitous patch refreshes, which require retesting and re-review.
I am concerned that all the error messages are now gone, from this function and all others in this file. We spent quite some time improving the information returned by 'lctl nodemap_' commands in case incorrect arguments are provided, and removing the error messages is basically ruining this effort. We need to find a way to still print those when invoked from lctl.
Acknowledged
I put back the error handling for `lctl modemap_` commands in obd.c. Some warnings are still in liblustreapi_nodemap.c.
(minor) these should be fixed in the original patch where they are added. Also, this would be better to print a more useful message, like `"error: %s: packing ioctl buffer"` or similar, since the "invalid" part will be printed from the error code (whatever it is, maybe "out of memory"). (minor) newline `\n` is not needed for `llapi_error()` messages
(style) similarly, "error: %s: unpacking ioctl buffer")
(minor) newline not needed
(minor) newline not needed
(minor) newline not needed
(minor) newline not needed (minor) "invalid" should be printed by the error string. (defect) `llapi_error()` already prints `strerror()` and the numeric error code internally, it should not be part of the message string, just `"error: packing ioctl buffer for request %08x"` is enough. Same for all the other `llapi_error()` calls in this file
This should already be handled by `llapi_get_mds_device()`?
Yes but in case of dynamic flag true, we don't call `llapi_get_mds_device()`. That's why it is there I think.
(minor) For a library entry point this description is incomplete — with dynamic=false on a node that is not the MGS, the code sets dynamic=true and retries, so the ioctl goes to the local MDS/OSS rather than the MGS:
if (!is_mgs()) {
dynamic = true;
goto getdev;
}
Worth documenting that fallback here, since a caller passing false is unlikely to expect it.
(minor) The word "invalid" here describes the wrong thing: llapi_error() appends the real error, so an -ENOMEM from llapi_ioctl_pack() prints as "error: invalid ioctl request: 000001cd: Cannot allocate memory (12)". Something like "error: packing ioctl buffer for request %08x" describes what actually failed. Same for the two messages below, where the operation is the ioctl itself rather than an "invalid" one. This was raised on patchset 24 alongside the duplicated strerror()/newline points; those two were fixed, this one still applies.
LU-19403 llapi: Move nodemap command function Move nodemap_cmd() from obdctl to liblustreapi as llapi_nodemap_cmd(), so it can be reused outside of lctl. Error reporting switches from fprintf(stderr) to llapi_error(), consistent with the rest of liblustreapi. llapi_nodemap_cmd() is a library function and, unlike the old obdctl nodemap_cmd(), does not call do_disconnect() or update lctl's cur_device as a side effect of selecting the MGS/MDS/OSS device for the ioctl. This is an intentional behavior change: a following bare device command in an interactive lctl session after a nodemap_* command may now target a different device than the nodemap command just used. Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com> Change-Id: I19f510399b86b7e79302a8f862232ac290f32648
(minor) This corrects a user-visible failure (SEEK_DATA/SEEK_HOLE on a designated parity mirror returning ENXIO), so it would help to carry a Fixes: tag.
e21b93b5f7b4 introduced ci_parity_io/ci_parity_eof and converted vvp_prep_size() and ll_direct_IO_impl() to the parity EOF, but left vvp_io_lseek_start()/vvp_io_lseek_end() clamping on i_size, which is exactly what this patch changes.
Fixes: e21b93b5f7b4 ("LU-19631 llite: fix EOF handling for EC parity mirrors")
(minor) The lseek_test.c hunk also carries two cleanups the message doesn't mention: the open() failure message switching from `error %d`/errno to `strerror(errno)`, and the re-indent of the `return -1;` in the getopt `default:` case. Both are harmless, but a line in the body would keep the diff free of surprises.
(style) This isn't a bug, but the em dash here is a non-ASCII character (U+2014); a plain "-" or a reworded sentence keeps the file 7-bit like the rest of llite.
(suggestion) This glimpses on every designated-mirror SEEK_DATA/SEEK_HOLE, not just parity ones, and "harmless for data mirrors" is doing a lot of work - it is a whole-file `[0, CL_PAGE_EOF]` CEF_GLIMPSE|CEF_MUST enqueue per lseek, which osc_enqueue_base() can only short-circuit if a cached lock already covers the whole object. That lands on a hot loop: llapi_mirror_data_seek() sets the mirror and then does SEEK_DATA + SEEK_HOLE, and it is called per data segment / per raidset from llapi_mirror_resync_many_params(), llapi_ec_resync_or_verify_raidset() and llapi_ec_resync_or_verify_comp(). Resync of a sparse file now pays two extra glimpses per segment against a mirror whose size it never uses. Can this be limited to the case that needs it (parity mirror), or hoisted so a resync pass glimpses once rather than per segment?
(minor) The eof selection keys off `ci_parity_io`, but the glimpse that makes `ci_parity_eof` meaningful is keyed off `fd_designated_mirror > 0` in ll_file_seek(). Those two gates are not the same set. lov_io_mirror_init() also sets `ci_parity_io` on the non-designated mirror-selection path (the `io->ci_parity_io = lov_mirror_entry(obj, index)->lre_parity;` near the end), and the parity skip just above it only covers CIT_READ/CIT_FAULT, not CIT_LSEEK. So a plain SEEK_DATA/SEEK_HOLE that lands on a parity mirror (preferred data mirror not `lre_valid`) takes this branch with a `ci_parity_eof` derived from whatever i_size happened to be at cl_io_init() time, i.e. before the ll_merge_attr() on the line above ever ran. Previously that case used the refreshed i_size. Would gating on `io->ci_designated_mirror` here, or skipping parity mirrors for CIT_LSEEK the same way as CIT_READ, make the two ends agree? The comment above should probably be narrowed too - "after glimpse of data size" only holds for the designated-mirror path.
LU-12668 ec: allow SEEK_DATA/HOLE on parity mirror lseek Parity mirror lseek was rejected at VVP with -ENXIO because vvp_io_lseek_start() and vvp_io_lseek_end() clamped against inode i_size. Parity components do not contribute to cat_size, so i_size only reflects the data mirror and cannot serve as parity EOF. In ll_file_seek(), call ll_glimpse_size() before designated- mirror SEEK_DATA/HOLE so lov_io_mirror_init() can compute ci_parity_eof from a current data size. In vvp_io_lseek_start() and vvp_io_lseek_end(), use ci_parity_eof for parity I/O and i_size for data I/O. Keep start >= eof -> -ENXIO (same as data and iomap) and end result > eof -> -ENXIO so SEEK_HOLE may still return eof for the implicit hole at end of file. Extend lseek_test with -m for designated mirror I/O, and switch sanity-ec 12c-12f from check_parity_read to SEEK_DATA/HOLE checks that verify hole range boundaries instead of sampling 4k at fixed offsets. Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Keguang Xu <kxu@ddn.com> Change-Id: I26eb4c19d26f01e38ababf303de1df5492056193
| unique failing test | history |
|---|---|
| sanity-lfsck@ldiskfs+DNE:test_9a | seen in 70 other reviews |
LU-20011 llapi: add flag-based layout sanity API
Introduce llapi_layout_sanity_check() with a bitmask-based
llapi_layout_checks enum to replace the old boolean-parameter
APIs. Rework the internal llapi_layout_sanity_args to store
the bitmask instead of multiple boolean fields, and add an
ONDISK flag for future use.
Deprecate llapi_layout_v2_sanity(), and update callers
(including lfs) to use the new llapi_layout_sanity_check()
interface.
Fixes: ee7dfc5ad1 ("LU-17025 llapi: Verify stripe pool name")
Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com>
Change-Id: I952d346f0dd2db4eb724e3767724705969d7810e
(minor) it is confusing to me that this is using `p` = parity and `m` = total stripe count. I thought the standard was `k` = data, `m` = parity, as with patch https://review.whamcloud.com/61965 ("LU-12187 lfs: add EC setstripe support with k+m notation").
It would be useful if this printed some information about the local CPU architecture (e.g. `model name` and `cpu MHz` from `/proc/cpuinfo` or equivalent from `lscpu`) and the EC implementation that was selected, so that we have some way to identify the results that are generated. Otherwise, we get a bunch of numbers and no way to identify how they relate.
LU-20016: Simple benchmark tool for EC computations Note that in the ISL-L library, computing parities and recovering lost data stripes is the same operation: ec_encode_data(). The difference is not in the computations but rather the content of the coefficient matrix. The cost of these operations are primarily bound by the number of stripes to compute which amusingly means that recovering a single lost stripe should be FASTER than generating two parities. Test-Parameters: trivial testlist=sanity-ec Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I61f409b631a8f4aefe0ceb2628e56f58f4c3a07c
(defect) This should use a proper LU number
```suggestion LU-930 doc: man page improvements ``` Could just use the default "man page improvement" ticket...
Rather than adding sub-commands to lctl, it is better to add a separate manage for it. That allows proper formatting, subsections, examples, etc.
LU-930 doc: man page improvements Various man page improvements - adding missing commands and options, fixing typos, fixing format issues, etc. Includes one trivial fix to lctl.c. Focuses on man8. This code was generated by Augment. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I222ea824b25b0a3104dc4a8bc8f56c10449045e4
LU-13814 osc: simplify osc_dio_page_submit Remove more code from osc_dio_page_submit, with the goal of eliminating it entirely. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id408aad01c2e81a126dd04807819aed9d29fca6b
LU-13814 osc: add osc_dio_completion Specialize osc_completion to add the osc_dio_completion version. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4d202165ee6c4f50338b6a9381aa476b969d3593
LU-19921 ptlrpc: register gssiam policy This is the final patch in the GSSIAM (Identity and Access Management) security flavor series. It calls sptlrpc_register_policy(&sptlrpc_gssiam_policy) in sptlrpc_gssiam_init() and sptlrpc_unregister_policy() in sptlrpc_gssiam_exit() to enable the GSSIAM policy for client/server authentication and authorization. It also improves error-unwind handling in sptlrpc_gssiam_init() by introducing out-of-line tunables_cleanup and cache_cleanup labels after policy registration. Signed-off-by: Di Wang <ddiwang@google.com> Change-Id: Ic5e1bc4f7d0db18abcc7f52525d4a8e0efdae7c3
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_160g | seen in 3 other reviews |
| sanityn@zfs:test_18 | seen in 8 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 10.1 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
(minor) The migration bullet only names nodemap_config_set_active(), but nodemap_clear_dynamic_nodemaps() also changes: GSSIAM nodemaps are now skipped there, so they survive a config reload instead of being deleted. That is a user-visible behavior change on the MGS path (mgs_handler.c calls it) and is worth a line of its own in the body.
When nmf_gssiam_managed is set and fs_projid is MDT_INVALID_PROJID, neither half of this condition can be true, so the function returns 0 and the resource is allowed. The UID/GID half is gated on !nmf_gssiam_managed, so nothing is checked at all in that case.
fs_projid is MDT_INVALID_PROJID whenever the object's attributes come back without LA_PROJID. That is not hypothetical on ZFS:
/* osd-zfs/osd_object.c, osd_object_sa_fini()/attr path */
} else {
la->la_projid = ZFS_DEFAULT_PROJID;
la->la_valid &= ~LA_PROJID;
}
and the whole #else branch when ZFS_PROJINHERIT is not defined. Both mdt_check_resource_ids() and __ofd_check_resource_ids() then pass MDT_INVALID_PROJID.
So on an OSD that doesn't report a project ID, a GSSIAM tenant gets a weaker check than a plain nodemap rather than a stronger one. Was failing open intended here, or should the missing-projid case fall back to the UID/GID test (or deny)?
This hands the nodemap over to the new config by moving nm_hash and nm_sha_hash, but nm_ranges is left pointing into old_config.
nodemap_config_dealloc() only walks the nodemaps still in old_config->nmc_nodemap_hash, so a migrated nodemap's ranges are never range_delete()d before OBD_FREE_PTR(config). Each lu_nid_range is leaked, and range->rn_tree still points at storage owned by the old config (or, for a dynamic nodemap, at the parent range's rn_subtree, whose lu_nid_range is freed by range_destroy() during the same dealloc).
A later nodemap_del() on the migrated nodemap reaches:
range_delete_generic()
nm_range_remove(range, &range->rn_tree->nmrt_range_interval_root)
which is then a use-after-free. Before that the ranges are simply invisible to nodemap_classify_nid(), since that walks active_config->nmc_range_tree.
nodemap_add_range_helper() has explicit dynamic-nodemap handling, so a dynamic nodemap can carry ranges. Should the ranges (and nmc_netmask_setup entries) be re-inserted into new_config here, or should nodemap_migrate_one() refuse to migrate a nodemap whose nm_ranges is non-empty?
(defect) nmf_trust_client_ids is one of the properties check_privs_for_op() governs (see its kerneldoc list), and nodemap_set_trust_client_ids() gates on it:
if (!check_privs_for_op(nodemap, NODEMAP_RAISE_PRIV_TRUSTED,
trust_client_ids))
GOTO(out_putref, rc = -EPERM);
Only NODEMAP_RAISE_PRIV_ADMIN and NODEMAP_RAISE_PRIV_RO are checked here, yet the block below raises nmf_trust_client_ids from the value inherited from the parent to 1. For a parent with nmf_trust_client_ids = 0 that does not grant NODEMAP_RAISE_PRIV_TRUSTED, that is a privilege raise the admin's policy forbids. Should NODEMAP_RAISE_PRIV_TRUSTED be checked as well?
(defect) This gate tests three fields but the block it guards sets six, plus the idmap and the offsets:
nodemap->nmf_gss_identify = 1;
nodemap->nmf_trust_client_ids = 1;
nodemap->nmf_map_mode = NODEMAP_MAP_PROJID;
nodemap_del_offset_helper(nodemap);
A freshly created dynamic nodemap inherits nm_squash_projid, nmf_readonly_mount and nmf_allow_root_access from its parent (nodemap_inherit_properties()). If the parent has nm_squash_projid already set to the tenant's projid - e.g. an admin ran `lctl nodemap_modify --name gssiam --property squash_projid` - and readonly/allow_root match the inherited values, the very first call falls through the whole block and returns 0 without configuring anything.
Would keying the skip on the flags being already set (or dropping the fast path on first configuration) close that?
(defect) Setting nmf_trust_client_ids here looks like it disables the projid check that the rest of the patch adds.
__nodemap_map_id() short-circuits on that flag before it can ever reach the squash label:
if (id_type == NODEMAP_PROJID &&
!(nodemap->nmf_map_mode & NODEMAP_MAP_PROJID))
GOTO(offset, found_id);
if (nodemap->nmf_trust_client_ids)
GOTO(offset, found_id);
map:
...
squash:
if (id_is_squashed)
*id_is_squashed = true;
nodemap_del_offset_helper() on the next line zeroes nm_offset_start_projid/nm_offset_limit_projid, so the FS_TO_CLIENT un-offset block earlier in __nodemap_map_id() is skipped too and there is no other route to squash.
That leaves nodemap_id_is_squashed(nm, id, NODEMAP_PROJID, NODEMAP_FS_TO_CLIENT) always false for a GSSIAM-managed nodemap, so the new branch in nodemap_check_resource_ids() can never return -ECHRNG. Since the UID/GID half of that condition is gated on !nmf_gssiam_managed, the net effect is that nodemap_check_resource_ids() always allows for these nodemaps - weaker than before, not stricter.
The same flag also means the {projid, projid} idmap inserted just above and nm_squash_projid are never consulted, so client-supplied projids pass through unmapped. Was nmf_trust_client_ids meant to be 0 here?
(minor) Can root actually be NULL here? The if/else-if chain above covers all six combinations of the two enums, so every valid call assigns it. This hunk also isn't mentioned in the commit message - is it a leftover from debugging a different problem, or is there a caller passing an out-of-range id_type/tree_type that should be fixed instead?
LU-19921 ptlrpc: update nodemap for gssiam Export and add a few functions and flags for gssiam. - Export nodemap_lookup_unlocked() and nodemap_getref(), and make nodemap_lookup_and_lock() and nodemap_unlock_and_put() non-static in nodemap_internal.h. - Add nodemap_gssiam_attrs_update() to update the gssiam attrs on the nodemap. - Add nmf_gssiam_managed flag to struct lu_nodemap to track and identify GSSIAM-managed dynamic nodemaps. - Add identity to nodemap and man page Documentation/man4/nodemap.identity.4. - Add project ID check in nodemap_check_resource_ids() for GSSIAM-managed nodemaps (keyed off nmf_gssiam_managed), which replaces the UID/GID squash checks for GSSIAM nodemaps. - Migrate dynamic nodemap from old config to new config during nodemap_config_set_active(). Reassign unique nm_id during migration to avoid ID collisions. - Use mo_attr_get() instead of mdt_obj2dt->dt_attr_get() in mdt_check_resource_ids(), since mdt_obj2dt will refer to OSD object, which can be NULL for remote object. Additionally, address several robustness and cleanup items: - Add a nodemap_unlink() helper to safely remove a partially registered nodemap from global hashes, and use it to clean up on lprocfs_nodemap_register() failure in nodemap_add(). - Add an nm_pde NULL guard in lprocfs_nodemap_remove() to prevent null pointer dereferences, and replace CDEBUG_LIMIT with CERROR on sub-nodemap deletion failure in nodemap_del(). - Skip saving dynamic nodemaps in nodemap_save_config_cache() as they should not persist across config updates. - Introduce OBD_STRDUP macro in obd_support.h to use it in nodemap_set_identity(), and update OBD_FREE_STR() to set the pointer to NULL after freeing. - Include <linux/atomic.h> in lustre_nodemap.h for atomic operations on config counts. - Hold active_config_lock around nm_identity reads in procfs to prevent race with concurrent nodemap_set_identity() updates. Signed-off-by: Di Wang <ddiwang@google.com> Change-Id: I4266d36804c645789d6df497cc2b4920c3e03f39
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_160g | seen in 1 other review |
EEXIST from the O_EXCL open does not quite mean "the file already exists", so keep_file can end up set for a file this call did create.
lfs setstripe -c 1 <dangling symlink>
open(2) fails O_CREAT|O_EXCL with EEXIST for any symlink "regardless of where the symbolic link points", so this sets keep_file and the retry at the else branch then follows the link and creates the target. On a restricted setstripe the ioctl returns -EACCES, the message says "existing file left unchanged" although the file was just created, remove() is skipped, and an empty file is left behind instead of falling back to the default layout. The same window opens if another process unlinks the name between the two opens, and a caller passing O_TRUNC has the file truncated by the retry before the same message is printed.
Would it work to do the retry without O_CREAT (keep_file already means "it exists") and only go back to the creating path on ENOENT?
This predates the patch, but since the block is being rewritten anyway: fd was closed above and never reassigned, so this returns the stale positive descriptor. lfs.c then does `if (result >= 0) { close(result); result = 0; }` and reports success after printing the error. `rc = remove(name)` also stores -1 rather than -errno, so rc isn't usable either. Setting `fd = -errno;` before the goto would cover both. Same thing at the llapi_file_create_foreign() copy below.
(suggestion) The four-way open ladder, the EEXIST/ENOENT retry pair, the /proc/self/fd readlink block and the reworked EACCES handling are now near-identical copies in llapi_file_open_param() and here. The two already differ in small ways (the O_EXCL/O_CREAT guards on the retries), so a shared helper that returns the fd plus the resolved path would keep a future fix from landing in only one of them.
(style) This isn't a bug, but `len` here shadows the outer `size_t len` holding the foreign LOV EA length declared at the top of the function; that outer value is still needed below for `lfm->lfm_length` and the memcpy(). Naming this one `rlen` (or reusing `rc`) would keep the two apart.
LU-20451 llapi: keep existing file on restricted setstripe
When setstripe is restricted (llite.*.enable_setstripe_gid), the
ioctl returns -EACCES to 'lfs' before the -EEXIST "layout already
set" check. llapi_file_open_param() and llapi_file_create_foreign()
treated that as "the placeholder I just created cannot be striped"
and called remove(name), so 'lfs setstripe' on an existing file
deleted the user's data.
Open with O_EXCL so remove() only drops a file we created; if the
name already exists, reopen without O_CREAT and leave the file
untouched. O_EXCL fails with EEXIST on a dangling symlink as well,
so when the O_CREAT-less reopen then returns ENOENT, retry the
create once more without O_EXCL: it only instantiates the link
target, like any create through a symlink. If that setstripe is
refused, remove the created target (resolved via /proc/self/fd)
rather than the symlink and fall back to the default layout.
Report -EEXIST for a file that is kept, not the -EACCES from the
ioctl: an unrestricted setstripe on an existing file already fails
with -EEXIST, so the restriction does not change the error 'lfs'
exits with and no new errno reaches llapi callers.
Also return -errno instead of the already-closed fd when remove()
fails, so 'lfs' does not report success after the error.
Add sanity test_27W coverage.
Assisted-by: ClaudeCode:Fable-5
Fixes: 5b99b881c412 ("LU-17628 lfs: add lfs_setstripe admin restrict")
Signed-off-by: Sohei Koyama <skoyama@ddn.com>
Change-Id: I3f9b20d27b4cae1b714136e38a525e7d81c4df24
| unique failing test | history |
|---|---|
| sanity-ec@ldiskfs+DNE:test_1a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_1b | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1c | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1d | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1g | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_4e | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_5a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_5b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_6a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6b | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6d | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6e | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_7 | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_23a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_23b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_23c | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_24a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_24b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_25a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_26a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_26b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_27a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_28a | seen in 22 other reviews |
| sanity-ec@zfs:test_1a | seen in 21 other reviews |
| sanity-ec@zfs:test_1b | seen in 23 other reviews |
| sanity-ec@zfs:test_1c | seen in 23 other reviews |
| sanity-ec@zfs:test_1d | seen in 23 other reviews |
| sanity-ec@zfs:test_1g | seen in 23 other reviews |
| sanity-ec@zfs:test_4e | seen in 23 other reviews |
| sanity-ec@zfs:test_5a | seen in 23 other reviews |
| sanity-ec@zfs:test_5b | seen in 22 other reviews |
| sanity-ec@zfs:test_6a | seen in 23 other reviews |
| sanity-ec@zfs:test_6b | seen in 23 other reviews |
| sanity-ec@zfs:test_6d | seen in 23 other reviews |
| sanity-ec@zfs:test_6e | seen in 23 other reviews |
| sanity-ec@zfs:test_7 | seen in 23 other reviews |
| sanity-ec@zfs:test_23a | seen in 22 other reviews |
| sanity-ec@zfs:test_23b | seen in 22 other reviews |
| sanity-ec@zfs:test_24a | seen in 22 other reviews |
| sanity-ec@zfs:test_24b | seen in 22 other reviews |
| sanity-ec@zfs:test_25a | seen in 22 other reviews |
| sanity-ec@zfs:test_26a | seen in 22 other reviews |
| sanity-ec@zfs:test_26b | seen in 22 other reviews |
| sanity-ec@zfs:test_27a | seen in 22 other reviews |
| sanity-ec@zfs:test_28a | seen in 22 other reviews |
DRAFT LU-20244 utils: add support to resync a region Add support to resync EC for a region of a file instead of the entire file. Test-Parameters: testlist=sanity-ec Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I49a0ab53e7948c40086b532a0e6f9298d1bc8e1e
| unique failing test | history |
|---|---|
| sanity-ec@ldiskfs+DNE:test_1a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_1b | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1c | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1d | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1g | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_4e | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_5a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_5b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_6a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6b | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6d | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6e | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_7 | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_23a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_23b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_23c | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_24a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_24b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_25a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_26a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_26b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_27a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_28a | seen in 22 other reviews |
| sanity-ec@zfs:test_1a | seen in 21 other reviews |
| sanity-ec@zfs:test_1b | seen in 23 other reviews |
| sanity-ec@zfs:test_1c | seen in 23 other reviews |
| sanity-ec@zfs:test_1d | seen in 23 other reviews |
| sanity-ec@zfs:test_1g | seen in 23 other reviews |
| sanity-ec@zfs:test_4e | seen in 23 other reviews |
| sanity-ec@zfs:test_5a | seen in 23 other reviews |
| sanity-ec@zfs:test_5b | seen in 22 other reviews |
| sanity-ec@zfs:test_6a | seen in 23 other reviews |
| sanity-ec@zfs:test_6b | seen in 23 other reviews |
| sanity-ec@zfs:test_6d | seen in 23 other reviews |
| sanity-ec@zfs:test_6e | seen in 23 other reviews |
| sanity-ec@zfs:test_7 | seen in 23 other reviews |
| sanity-ec@zfs:test_23a | seen in 22 other reviews |
| sanity-ec@zfs:test_23b | seen in 22 other reviews |
| sanity-ec@zfs:test_24a | seen in 22 other reviews |
| sanity-ec@zfs:test_24b | seen in 22 other reviews |
| sanity-ec@zfs:test_25a | seen in 22 other reviews |
| sanity-ec@zfs:test_26a | seen in 22 other reviews |
| sanity-ec@zfs:test_26b | seen in 22 other reviews |
| sanity-ec@zfs:test_27a | seen in 22 other reviews |
| sanity-ec@zfs:test_28a | seen in 22 other reviews |
I don't see this patch doing these things, mainly changing the buffer size?
How was this 1GiB buffer size selected?
The file was stat'dback on line 14591, so that could be used here instead of a new stat() call? If there is a reason the stat() info has gone stale since the start of this function, then that is probably a sign the file shouldn't be resync'd at this time.
DRAFT LU-20244 utils: resync/verify stale mirrors in 1GB chunks. Instead of resyncing each stale mirror, one after the other, resync/verify the mirrors one chunk (1GB region) of the file at a time. This is so that if we are resyncing/verifying multiple stale mirrors at a time, that we read data from the up-to-date mirror once, for generating/writing to the first stale mirror and then re-read the original data from page-cache when generating/writing to the other stale mirrors. Otherwise if we resync/verify the whole mirrors one at a time, we risk for very large files that the read data from the good mirror will fall out of cache and need te be re-read for each stale submirror. Test-Parameters: testlist=sanity-ec Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I279cdedf9b07c8f091747fcaa5b103852ee166c7
| unique failing test | history |
|---|---|
| sanity-ec@ldiskfs+DNE:test_1a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_1b | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1c | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1d | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_1g | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_4e | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_5a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_5b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_6a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6b | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6d | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_6e | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_7 | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_23a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_23b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_23c | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_24a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_24b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_25a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_26a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_26b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_27a | seen in 23 other reviews |
| sanity-ec@ldiskfs+DNE:test_28a | seen in 22 other reviews |
| sanity-ec@zfs:test_1a | seen in 21 other reviews |
| sanity-ec@zfs:test_1b | seen in 23 other reviews |
| sanity-ec@zfs:test_1c | seen in 23 other reviews |
| sanity-ec@zfs:test_1d | seen in 23 other reviews |
| sanity-ec@zfs:test_1g | seen in 23 other reviews |
| sanity-ec@zfs:test_4e | seen in 23 other reviews |
| sanity-ec@zfs:test_5a | seen in 23 other reviews |
| sanity-ec@zfs:test_5b | seen in 22 other reviews |
| sanity-ec@zfs:test_6a | seen in 23 other reviews |
| sanity-ec@zfs:test_6b | seen in 23 other reviews |
| sanity-ec@zfs:test_6d | seen in 23 other reviews |
| sanity-ec@zfs:test_6e | seen in 23 other reviews |
| sanity-ec@zfs:test_7 | seen in 23 other reviews |
| sanity-ec@zfs:test_23a | seen in 22 other reviews |
| sanity-ec@zfs:test_23b | seen in 22 other reviews |
| sanity-ec@zfs:test_24a | seen in 22 other reviews |
| sanity-ec@zfs:test_24b | seen in 22 other reviews |
| sanity-ec@zfs:test_25a | seen in 22 other reviews |
| sanity-ec@zfs:test_26a | seen in 22 other reviews |
| sanity-ec@zfs:test_26b | seen in 22 other reviews |
| sanity-ec@zfs:test_27a | seen in 22 other reviews |
| sanity-ec@zfs:test_28a | seen in 22 other reviews |
DRAFT LU-20244 tests: add tests for resync of FLR + EC We can have files that use FLR mirroring but at the same time also use EC to protect the FLR mirrors. It is probably not common to use as a persistent end user configuration as Mirrorins and EC usually are meant for different purposes : with Mirroring being prefered when performance outweights storage cost and EC is preferred when storage costs outweigh performance. But there will be situations where FLR+EC will be in use during a transitionary period, for example when migrating an EC protected file between different storage hierarchies. As such we need to have a test for this situation and that we can support and resync buth FLR and EC components at the same time. Test-Parameters: testlist=sanity-ec Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: Idcac78327b501d484653dec6be0882b5dbb1fa6f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-subtest-change | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-ldiskfs-dne | RHEL 8.10 / x86_64 | ran 2 tests. 1 tests failed: lustre-initialization. | session |
(typo) The patch adds 35b through 35h, but the message only lists 35b-35g. 35h (past-EOF component) isn't accounted for.
This testing is unrelated to EC and could go into sanity-flr.sh to avoid contention in this file.
(minor) this could be tested with a minimum of 2 OSTs without loss of coverage.
(style) prefer lowercase for local variables, unless you are an emphatic fan of 80s New Wave music.
(style) no need for linefeed escape at end of line after `|`
(style) continued line should be indented
(suggestion) For a sparse component this re-seeks once per stripe cycle: when llapi_data_seek() reports data past region_end the inner loop breaks, and the next iteration calls it again from file_pos + swidth even though the previous answer already said where the next data is. On Lustre each SEEK_DATA/SEEK_HOLE is an OST RPC, so a large mostly-hole component costs 2 RPCs per stripe width with nothing to copy. Could the returned data_off be carried across iterations to skip straight to the region that contains it?
(minor) migrate_open_files() opens both the source and the volatile with O_CIPHERTEXT so `lfs migrate` can move objects of an encrypted file when the client has no key. Without it, ll_file_open_encrypt() (llite/crypto.c) returns -ENOKEY here, and ll_lookup_it() also refuses to create a volatile in an encrypted directory without O_CIPHERTEXT|O_DIRECT. So migrate_stripe can't be used on an encrypted file unless the key is loaded, which is awkward for the admin OST-drain case this is aimed at. Worth adding O_CIPHERTEXT here and in migrate_stripe_create_volatile()?
This st_size is sampled well before the data version is captured, so it can already be stale by the time the copy starts.
Order today:
fstat(fd_src) /* comp_end = min(comp_end, st_size) */
migrate_stripe_create_volatile() /* MDT create + OST object alloc */
migrate_stripe_get_ost_fid() x2 /* two fgetxattr() */
llapi_group_lock() / llapi_get_data_version(&dv_src)
A writer that appends between the fstat() and the dv_src read extends the file past the stale comp_end. The copy loop stops at comp_end so those bytes are never copied, and SWAP_LAYOUTS_CHECK_DV1 compares against dv_src which was read *after* the write, so the swap succeeds instead of returning -EAGAIN. Other stripes still carry the larger size, so after the swap that region reads back as zeros.
Would re-stating the size after dv_src (or under the group lock, for --lock-copy) close this? As written, --lock-copy is not fully atomic either, since the group lock is taken after the fstat.
(minor) This should preferentially use an API function like patch https://review.whamcloud.com/58514 ("LU-18842 lustreapi: create volatile file with a layout")
LU-20541 lfs: Implement migrate_stripe command and add test Add lfs migrate_stripe to move one OST object of a file onto another OST using a single-stripe volatile partner and llapi_swap_stripe(). Copy follows absolute LOV stripe numbering and object offsets, keeps holes sparse, and ftruncates the volatile so trailing holes still carry file size without SOM. Match source uid/gid on the volatile for mdd_swap_layouts(). Default path is optimistic (DV check at swap); --lock-copy holds a group lock across copy and swap. Document in lfs-migrate_stripe.1. sanity-ec 35b-35g cover content, holes, PFL offsets, size, absolute striping, and root on user files. Test-Parameters: trivial Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I48be7e597544c6856fdcf8edbd181b9f8d3db263
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 crashed | RHEL 10.1 / x86_64 | ran 11 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-1 | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
Several hunks aren't covered by the description, which makes it hard to tell what is intentional here:
- the new `llapi_strscpy()` in lustreapi_internal.h (nothing calls it)
- the `llapi_obdname2devno()` changes: moving `strlen()` below the NULL check, and copying into `namebuf`
- `yaml_get_device_index()`: the two new cleanup paths, the `-EOPNOTSUPP` -> `-EINVAL` change at the `error:` label, and the log-prefix change
- the lazy `OBD_DEV_ID` registration in `llapi_ioctl_dev()`
- the new `llapi_nodemap_test` binary and `sanity-sec test_92`
The last two are the interesting ones for a reader, since they change behaviour for every liblustreapi consumer, not just nodemap.
Also, the summary and body describe the change only in the abstract ("device functions", "getter functions"). Naming `llapi_get_mgs_device()` / `llapi_get_mds_device()` / `llapi_get_oss_device()` in the body would make this findable with `git log --grep` later.
(style) `Change-Id:` is above `Signed-off-by:` here; the usual order is `Signed-off-by:` first. That ordering normally comes out of the Gerrit commit hook, so it may just need reinstalling.
The lib/libcfs/l_ioctl.c hunk isn't mentioned anywhere in the description. Making llapi_register_ioc_dev() idempotent changes a shared libcfs entry point that LNet also uses (lustre/utils/portals.c:285, lnet/utils/lnetconfig/liblnetconfig.c:591), not just the OBD path described here. Worth a sentence so a reader can tell it is deliberate rather than a stray hunk.
The two llapi_obdname2devno() changes are crash fixes, not cleanups: strlen(name) ran before the NULL check, and ioc_inlbuf1 = (char *)name had llapi_ioctl_unpack() memcpy the reply back through a const char * (liblustreapi_ioctl.c:212). Both arrived with the immediately preceding patch in this series, so a Fixes: tag belongs here:
Fixes: 50f3cfb37abc ("LU-19276 llapi: add llapi_changelog_(de)register")
That sha still changes on every rebase while 61340 is unlanded, though. Would it be cleaner to fold both fixes into 61340 instead, so the tree never has a window where llapi_obdname2devno(NULL) segfaults?
(typo) The tree is at 2.17.54, so the next release is 2.18.0. Other man3 pages added recently say "since release 2.18.0"; 2.21.0 is three releases out.
(minor) b8a39fde46 is this change's own patchset 18, so it will not exist in master once this lands. The other man3 pages use a git describe id filled in at merge time, e.g.
.\" Added in commit v2_17_52-151-g86b0c90666
(minor) These land in the middle of the `llapi_search_*` / fsname block, but they're implemented in liblustreapi_ioctl.c and are thin wrappers around `llapi_obdname2devno()` - grouping them next to it (around line 1590) would keep related declarations together. Also, three new public entry points are added here with no man page. The parent LU-19276 patch added Documentation/man3/llapi_changelog_register.3 and friends in the same change; should this one add man3 pages (and the Documentation/man3/Makefile.am entries) for the new functions too?
(style) Stray blank line - the declarations that used to sit here moved down next to llapi_obdname2devno(), so this can go.
(minor) When the slot is already taken this returns success without looking at @dev_name, so registering a different path under the same id silently keeps the old one and the caller can't tell. Every in-tree caller passes the same path per id today, so nothing breaks. But for a public entry point, would returning -EEXIST when @dev_name differs from the registered name be safer than reporting success for a registration that didn't happen?
(style) please keep alphabetic ordering
(style) Raised earlier in the review and still open - alphabetically this belongs next to /llapi_hsm_test rather than appended at the end.
(minor) This test can't fail. When `llapi_get_mgs_device()` returns an error it prints "Skipping" and returns, the child exits 0, and `run_server_tests()` records a pass. Same shape in tests 1, 3, 4 and 5.
So the only assertions in the binary are tests 6 and 7, and both exercise `llapi_obdname2devno()` rather than the new getters - a broken `llapi_get_mds_device()` would still let `sanity-sec test_92` pass on every node.
Since each test already gates on `is_mgs()`/`is_mds()`/`is_oss()`, could the failure become an assertion once that gate passes?
ASSERTF(rc >= 0, "llapi_get_mgs_device() failed on MGS: rc=%d", rc);
LU-19403 llapi: Add device functions for nodemap Add llapi_get_mgs_device(), llapi_get_mds_device() and llapi_get_oss_device(), getter functions for the local MGS, MDS and OSS device numbers, used by nodemap-related lctl commands. Fix llapi_obdname2devno() to copy the device name into a local writable buffer before the ioctl, since llapi_ioctl_unpack() writes the kernel reply back into it and the caller's string may be read-only; also move the strlen() call after the NULL check. Add lazy OBD_DEV_ID registration to llapi_ioctl_dev(), so external applications that call llapi_obdname2devno() or llapi_changelog_register() without going through lctl's obd_initialize() still get a working device ioctl path. Fix two socket leaks in yaml_get_device_index() on its early -EOPNOTSUPP return paths, and correct the errno returned when the Netlink request itself fails to emit from -EOPNOTSUPP to -EINVAL, since Netlink is known to be available at that point. Add the llapi_nodemap_test binary and sanity-sec test_92 to exercise these functions on MGS, MDS and OSS nodes. Signed-off-by: Nikos Papakonstantinou <npapakonstantinou@ddn.com> Change-Id: I095265bdccc8efd441c6c396c8ad267d1f6b7ca9
| unique failing test | history |
|---|---|
| sanity2@zfs:test_119l | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 10.1 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-subtest-change | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
(minor) ... so it could also be set and checked via `chattr +t FILE` and `lsattr FILE` commands from e2fsprogs.
The "Changes:" list mentions adding LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE, but the same hunk also adds LUSTRE_COMPR_FL to that mask and removes the duplicate LUSTRE_NOATIME_FL entry. Neither is explained. The COMPR change is an independent server-side behavior change - should it be split into its own patch? The lustre/utils/lfs.c hunk (skipping the range validation for nohybrid) isn't described either.
This changes UAPI flag definitions that the MDT interprets, so interop with an older server is worth an explicit test run. Consider adding something like:
Test-Parameters: testlist=sanity serverversion=2.16.0
New wire flag, but wirecheck.c and the two wiretest.c copies don't look updated - the neighbouring LUSTRE_*_FL values have CHECK_VALUE_X entries. Same for LU_LADVISE_NOHYBRID, which needs a CHECK_VALUE next to the other LU_LADVISE_* ones (LU_LADVISE_AHEAD seems to have been missed earlier too).
The compatibility claim points at the wrong side. Older clients aren't the problem; older servers are.
On a pre-patch MDS, LUSTRE_FL_USER_VISIBLE has no NOTAIL bit, so mdt_setattr_unpack() hits
if (rec->sa_attr_flags & ~LUSTRE_FL_USER_VISIBLE)
RETURN(-EOPNOTSUPP);
and lfs ladvise -a nohybrid fails with EOPNOTSUPP. Clearing the flag still "succeeds" as a no-op, so the two directions behave differently. Worth stating the required server version here.
This was asked on patchset 7 and looks unaddressed: adding LUSTRE_COMPR_FL here is unrelated to nohybrid and changes server behavior. mdt_setattr_unpack() masks with LUSTRE_FL_USER_MODIFIABLE, and osd_attr_set() replaces the whole masked set, so this makes the compression flag both settable and clearable on the MDT inode by any client. Is that intended ahead of the compression work landing?
(defect) why is NOATIME being removed?
Is the COMPR flag really user modifiable or just visible? And does it make sense to allow this to be set before CSDC is landed to master?
Should this use NOHYBRID?
Does this also set the flag directly on the inode, or is the inode here the root or parent directory?
parse ll_file_ioctl():error: Function too hairy. Giving up. 4 seconds warn: ll_file_ioctl():Function too hairy. No more merges.
ll_inode2ext_flags() is not a full picture of the file's flags - it rebuilds them from inode->i_flags via ll_inode_to_ext_flags(), which only knows SYNC/NOATIME/APPEND/DIRSYNC/IMMUTABLE/ENCRYPT, plus the PROJINHERIT and (new) NOHYBRID lli_flags bits.
LUSTRE_NODUMP_FL and LUSTRE_NOCOMPR_FL are in LUSTRE_FL_USER_MODIFIABLE but have no i_flags or lli_flags representation, so they come back as 0 here. osd_attr_set() then does a wholesale replace:
ei->i_flags = (ei->i_flags & ~LDISKFS_OSD_USER_MODIFIABLE) |
(attr->la_flags & LDISKFS_OSD_USER_MODIFIABLE);
so those bits get cleared on disk. chattr +d FILE followed by lfs ladvise -a nohybrid FILE should lose the 'd' flag.
The FS_IOC_SETFLAGS path avoids this by calling fileattr_get() first, which fetches body->mbo_flags from the MDT. Should this do the same before OR-ing in LUSTRE_NOHYBRID_FL?
Related: ll_set_project() builds op_attr_flags from ll_xflags_to_ext_flags() and also sets OP_XVALID_FLAGS, so lfs project on a file appears to clear the nohybrid flag for the same reason.
LU-19839 llite: add persistent nohybrid I/O flag Add a persistent flag to prevent hybrid I/O switching for specific files. This uses the NOTAIL flag (0x00008000) which is not used by Lustre/ext4 and unlikely to be used in the future. The flag is set via ladvise interface for discoverability but uses FS_IOC_SETFLAGS internally for implementation. When set, hybrid I/O will not switch the file from buffered to direct I/O regardless of I/O size. Changes: - Define LUSTRE_NOTAIL_FL and LUSTRE_NOHYBRID_FL - Add LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE - Implement LU_LADVISE_NOHYBRID using FS_IOC_SETFLAGS - Update ll_update_inode_flags to handle NOTAIL flag - Add test_119l to verify nohybrid flag functionality Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6a69293801114e2a3015ed87f2258828922ab767
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne-arm failed 3× | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity. | session |
I see why the header file doesn't work because of the typo in this #ifdef, PPP->PP.
lustre_disk.h is really a server only header. The below nodemap code should be in #ifdef HAVE_SERVER_SUPPORT. Is that not the case?
Thanks for your reminder.
LU-18398 lctl: validate nodemap name in user space Move nodemap_name_is_valid() to lustre_disk.h, so that both kernel and userspace can do nodemap name validation by it conveniently. Also, sanity-sec.sh test_7 is modified to verify this patch. Test-Parameters: trivial testlist=sanity-sec Signed-off-by: Emoly Liu <emoly@whamcloud.com> Change-Id: Id3ed2cebcedb1a34d2f2981849d64a60a879a665
| unique failing test | history |
|---|---|
| sanity-ec@ldiskfs+DNE:test_1a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_1b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_1c | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_1d | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_1g | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_4e | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_5a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_5b | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_6a | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_6b | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_6d | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_6e | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_7 | seen in 22 other reviews |
| sanity-ec@ldiskfs+DNE:test_23a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_23b | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_23c | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_24a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_24b | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_25a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_26a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_26b | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_27a | seen in 21 other reviews |
| sanity-ec@ldiskfs+DNE:test_28a | seen in 21 other reviews |
| sanity-ec@zfs:test_1a | seen in 21 other reviews |
| sanity-ec@zfs:test_1b | seen in 22 other reviews |
| sanity-ec@zfs:test_1c | seen in 22 other reviews |
| sanity-ec@zfs:test_1d | seen in 22 other reviews |
| sanity-ec@zfs:test_1g | seen in 22 other reviews |
| sanity-ec@zfs:test_4e | seen in 22 other reviews |
| sanity-ec@zfs:test_5a | seen in 22 other reviews |
| sanity-ec@zfs:test_5b | seen in 21 other reviews |
| sanity-ec@zfs:test_6a | seen in 22 other reviews |
| sanity-ec@zfs:test_6b | seen in 22 other reviews |
| sanity-ec@zfs:test_6d | seen in 22 other reviews |
| sanity-ec@zfs:test_6e | seen in 22 other reviews |
| sanity-ec@zfs:test_7 | seen in 22 other reviews |
| sanity-ec@zfs:test_23a | seen in 21 other reviews |
| sanity-ec@zfs:test_23b | seen in 21 other reviews |
| sanity-ec@zfs:test_24a | seen in 21 other reviews |
| sanity-ec@zfs:test_24b | seen in 21 other reviews |
| sanity-ec@zfs:test_25a | seen in 21 other reviews |
| sanity-ec@zfs:test_26a | seen in 21 other reviews |
| sanity-ec@zfs:test_26b | seen in 21 other reviews |
| sanity-ec@zfs:test_27a | seen in 21 other reviews |
| sanity-ec@zfs:test_28a | seen in 21 other reviews |
Hi, this patch is part of 'FLR-ECRO: verify non-write RPCs do not block client': I was wondering if the `fail fast` logic on `stat()` should actually be a generic feature for all PFL components, instead of being scoped exclusively to EC files. If that makes sense, I'd prefer to rebase this against master and relocate the tests under sanity.sh. Thoughts?
I reused `lovsub_import_inactive()` from the EC series and incorporated logic to skip parity for `CIT_GLIMPSE`. Given these dependencies...So, let's keep it within this series.
This patch should probably go directly on master, since I don't think it depends on any functionality that hasn't landed yet?
I reused `lovsub_import_inactive()` from the EC series and incorporated logic to skip parity for `CIT_GLIMPSE`. Given these dependencies...So, let's keep it within this series.
In lov_io_mirror_init(), if `ci_designated_mirror` is set, we can also check lov_mirror_has_inactive_stripe for `CIT_GLIMPSE/LSEEK/DATA_VERSION`. Since these io types all trigger OST aggregation, this check allows us to skip unnecessary RPCs. In current impl, we only check `CIT_GLIMPSE` during mirror selection (where `ci_designated_mirror` is false).
(style) line length of 81 exceeds 80 columns
How does this behave for a non-EC file with a deactivated OST stripe? Does that return an error or just calculate the size without the stripe (which seems wrong)?
An error returned: `stat: cannot statx '/mnt/lustre/f42c.sanity-ec': Cannot send after transport endpoint shutdown` So the old code itself is correct already. New change only avoid sending unnecessary glimpse to mirrors with invalid OSTs.
LU-20212 lov: skip invalid OST mirrors for aggregate I/O I/O that aggregates stripe-local results into one logical answer (glimpse, lseek, data_version) cannot succeed on a mirror while any relevant OST import is deactivated or invalid. In lov_io_mirror_init(), skip mirrors with inactive OSTs in the I/O extent during FLR selection for aggregate I/O (glimpse, lseek, data_version). Only glimpse returns -EAGAIN after mirror-round backoff so the caller can retry (glimpse is ndely=1); lseek and data_version fail with -EIO. For designated-mirror I/O, apply the same extent-scoped inactive stripe check after resolving ci_designated_mirror; there is no alternate mirror to try, so return -EIO immediately. Detect dead stripes with lov_mirror_extent_has_inactive_stripe(), limited to stripes intersecting the current I/O extent. Skip parity mirrors for glimpse as well. Add test sanity-ec 42a,42b. Test-Parameters: testlist=sanity-ec Signed-off-by: kxu <kxu@ddn.com> Change-Id: Iac1c6415127d46fd958016a998ceab68663e0d42
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_27D | seen in 92 other reviews |
| sanity1@zfs:test_27D | seen in 96 other reviews |
| sanity-hsm@zfs:test_12q | seen in 60 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-19823 lod: Make the stripe allocators failure domain aware. Add failure domain awareness to the RR and QOS allocators when/if configured. A failure domain value of 0 means not configured/not used. For now we only implement this for EC. Later, we might enable this for FLR Mirroring too. Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I7bf09636939915b866eb97d9952bab2968d7718e
It would make sense to put an LLM to work on converting the NASM .asm files int GCC .S files. It should be possible to have it iterate on getting the GCC-compiled code to generate the same x86 byte code, or at least compile and run correctly to pass whatever correctness tests exist for this code.
In fact, a quick search shows that NASM already has a tool to do this:
Automatically convert Intel NASM assembly to GNU Assembler (GAS/GASM)
syntax using the intel2gas tool, which converts between NASM and AT&T
syntax, or by utilizing NASM's built-in output capabilities to generate
GAS-compatible objects.
$ intel2gas -i input.asm -o output.s
These generated/converted .S files should be stored alongside the original .asm files in Git. If NASM is available the originals can be compiled, but in the common case where NASM is not available the .S files would be compiled and linked into the kernel modules.
I think it is worthwhile to keep both, since it would be easier to update the .asm files from upstream ISA-L, but it might be some time before the .S files are update to match.
Are these bugs in the upstream ISA-L code that should be pushed back to them?
Similarly, is this code from upstream ISA-L, or a test wrapper that we developed?
This should probably use LU-19905, or change the description of LU-20016?
This table is the same between the x86 and aarch64 patches. It would be useful to fix one or the other to have the correct data for that CPU architecture.
(minor) this should have a warning that the C version may only be 1/20-1/30th as fast, at least until the .S versions are available.
(minor) It would be better to specify which kernel version is non-functional, so that this can become conditional upon a newer kernel (assuming there is a benefit to use SVE-optimized versions).
(minor) Again, please specify kernel versions so that this can be fixed in the future for newer kernel versions.
(style) `bool`?
(style) externs should be avoided in .c files
These warnings are correct and the prototypes should be moved to a .h file that is included here and into the file where these functions are implemented. Having the function prototype in a local .c file prevents the compiler from checking argument type/count against the implementation, and can result in hard-to-find bugs in the future.
(defect?) According to the comments for `kernel_neon_begin()`: ``` * Unless called from non-preemptible task context, @state must point to a * caller provided buffer that will be used to preserve the task's kernel mode * FPSIMD context when it is scheduled out, or if it is interrupted by kernel * mode FPSIMD occurring in softirq context. May be %NULL otherwise. */ void kernel_neon_begin(struct user_fpsimd_state *state) ``` I'm thinking that this would be declared on the stack in e.g. `gf_vect_dot_prod()` and passed to `ec_neon_usable()->kernel_neon_begin()` so that it can also be passed (as required) to `kernel_neon_end()`: ``` * The value of @state must match the value passed to the preceding call to * kernel_neon_begin(). */ void kernel_neon_end(struct user_fpsimd_state *state) ``` The `struct user_fpsimd_state` is 528 bytes, so a _bit_ heavy to put on the stack, but at the same time this is (very likely) to be on a PAGE_SIZE=64KiB system so stack space should be available for this, and it will not have a deep call chain below this point. I think doing a `kmalloc()` and `kfree()` for each call would add noticeable overhead and should be avoided it possible. Alternatives would include having a dedicated slab for this, so that there is a per-CPU cache (528 does not fit into standard slabs well) and it likely has local allocations objects cached in the slab.
(style) should this be changed to `rows >= 6` and remove the `case 6:` in the switch, as is done in `ec_encode_data_neon()` above?
LU-20016 ec: ISA-L SIMD for userspace and kernel
Integrate Intel ISA-L optimized assembly into Lustre's
erasure coding library for both userspace and kernel,
on x86_64 and aarch64. Add debugfs kernel benchmark
for measuring FPU save/restore overhead.
Userspace x86_64 (libec.a):
- 72 NASM assembly files (SSE/AVX/AVX2/AVX-512/GFNI)
- ec_multibinary.asm: runtime CPUID dispatch
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar when NASM unavailable
Userspace aarch64 (libec.a):
- NEON + SVE .S assembly + SVE C intrinsics
- ec_aarch64_dispatcher.c: getauxval() dispatch
- ec_aarch64_highlevel_func.c: N-vector dispatch
Kernel x86_64 (ec.ko):
- Pre-assemble ISA-L NASM .asm files into .o,
link into ec.ko (71 assembly objects)
- ec_dispatch.c: boot_cpu_has() selects AVX2/AVX/SSE
at module init, kernel_fpu_begin/end wrapping
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar in interrupt context
Kernel aarch64 (ec.ko):
- NEON .S assembly files (GAS format, direct kbuild)
- ec_aarch64_neon.c: kernel_neon_begin/end wrapping
with may_use_simd() check, hwcap detection
- SVE excluded from kernel (toolchain portability)
Build system:
- lustre-erasurecode.m4: detect NASM, aarch64
- erasurecode/autoMakefile.am: three-way dispatch
- utils/Makefile.am: link against libec.a
- ec/Makefile.in: NASM pre-assembly for kernel
Bug fixes in ec_perf_bench.c:
- -p flag was setting k instead of p
- frag_ptrs allocated stripe_size pointers not m
- Inverted exit code and stale rc from getopt
- Use aligned_alloc(64) for data buffers
Kernel benchmark (debugfs):
- /sys/kernel/debug/lustre/ec/benchmark
- 4 phases: FPU overhead, SIMD+FPU per-call,
SIMD+FPU amortized, C scalar baseline
- Input format (write to debugfs entry):
echo 1 > .../ec/benchmark (defaults)
echo "k=5 p=2 s=128" > .../ec/benchmark
cat .../ec/benchmark
Parameters (key=value, space separated):
k data stripes (default 5, max k+p=16)
p parity stripes (default 2)
s stripe size in KB (default 128, max 65536)
"echo 1" runs with all defaults. Unrecognized
input returns -EINVAL. cat shows results or
usage instructions if not yet run.
Fix gf_vect_mul_init naming to use _base suffix
consistently (aligns with ISA-L convention) in
both userspace and kernel ec_base.c.
Performance (5+2, 1 thread, QEMU VM, AVX2):
Stripe | Kernel | Kernel | Userspace | SIMD | Kernel vs
Size | SIMD | Scalar | SIMD | Speedup | Userspace
| | | | in Kernel|
-------|---------|---------|-----------|----------|----------
64 KB | 17099 | 540 | 404345* | 31.7x | 0.04x*
128 KB | 16888 | 531 | 197511* | 31.8x | 0.09x*
256 KB | 17032 | 539 | 98401* | 31.6x | 0.17x*
1 MB | 17142 | 538 | 24830 | 31.9x | 0.69x
4 MB | 10309 | 533 | 5385 | 19.3x | 1.91x
16 MB | 10954 | 534 | 1048 | 20.5x | 10.5x
All throughput in MB/s. (*) Userspace numbers at
small stripes are inflated by cache residency.
At 4-16MB the kernel is faster -- its kvmalloc
pages are physically scattered, matching real
Lustre page cache behavior. Replicating this in
userspace is non-trivial and not worth the effort;
the kernel benchmark is authoritative.
FPU save/restore overhead was measured at 25-54 ns
on this platform (Ryzen 3700X) -- negligible vs
encode time (~39us at 128KB). The benchmark
measures this separately (Phase 1) so it can be
characterized on other hardware where it may be
more significant.
Generated with Claude Code + Tools
Test-Parameters: testlist=sanity-ec
Change-Id: If4c57c328c0e23bb6769dcbc45d52d43b03a7837
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-12187 llite: enable FLR EC support unconditionally Remove module parameter guards and enable FLR EC support unconditionally. This patch is intended to be applied after the guarded development phase is complete. Changes: - Remove mdt_enable_flr_ec module parameter - Remove llite_enable_flr_ec module parameter - Add OBD_CONNECT2_FLR_EC unconditionally to client connect flags - Change ll_enable_erasure_coding default from 0 to 1 - Remove MODOPTS_MDT and MODOPTS_LLITE settings from test framework Test-Parameters: ignore Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I431d309f85c8964e13a11eaaa729d944e9228c29
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_27D | seen in 92 other reviews |
| sanity1@zfs:test_27D | seen in 96 other reviews |
| sanity-hsm@zfs:test_12q | seen in 60 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-20139 lod: Add failure-domains to FLR allocations Add failure-domain awareness to FLR mirrored files. If overstriping is requested we allow the same failure domain to be re-used within a component but not across components in different mirrors. Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: Id8f4beb23ef20f805cbf47ba743b2291c8d4a9ae
| unique failing test | history |
|---|---|
| racer@ldiskfs+DNE:test_2 | seen in 62 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
(comment inconsistency) below only chunks 0 and 2 are written
(defect?) is this the correct conclusion to draw here? It looks like `bytes_read == 0` could be ambiguous: The caller `llapi_ec_verify_hole_by_seek_or_zero_content()` via `llapi_mirror_data_seek()` only found data in the range of `[data_off, data_off + data_len)` but `data_len` was later replaced before calling this function: ``` /* SEEK_DATA hit data inside the range; confirm it is all zero */ data_len = end - data_pos; ``` The loop here runs until `end`. So if then `pos` (in this function here) is already at the end of the parity mirror's objects, `llapi_mirror_read()` will return 0 since there is nothing more to read. There is a similar guard in `llapi_ec_compute_parities()` which seems to be correct since it clamps `to_read` to `end_pos`. This isn't happening here, so should this case just `return 0;`? After all, IIUC, we only want to prove that the rest of the range is a hole, which would be true for zero-filled tails. Also, it looks like `llapi_ec_verify_parities()` handles a similar case correctly for short read, zero-filled tail cases. Granted, this seems a quite the narrow case... For this to happen, I guess SEEK_DATA needs to report data at the start of the range while the object ends inside it. But we are here in the verify case, so it seems we should catch that.
Actually, I changed my mind. Let's fix this in another patch if needed. Since this patch here is now ready to land, I don't think it is worth to update it for this narrow case
(style) This isn't correct kernel-doc I think (should be one arg per line). There are also some args missing that should be included. In general, I think it would be nice to have kernel-doc for all the changed functions here. It is _mostly_ clear by the naming but some functions have a lot of arguments so documentation would be preferred even if they are `static` functions
LU-12668 ec: ensure hole ranges for raidset parity Complete the block-granularity EC parity resync/verify path so partial writes and verification stay consistent with sparse parity mirrors. When a raidset has no data coverage, punch-or-zero or seek-or-zero-check that the parity mirror is holed for the entire raidset through llapi_ec_hole_range_verify_or_clear(), shared with the stripeset-level hole path and with clearer mismatch diagnostics. On osd-zfs, punch falls back to zero-fill via llapi_mirror_punch_or_clear(); verify uses llapi_ec_verify_hole_by_seek_or_zero_content() when SEEK_DATA may report zero-filled extents instead of a true hole. For raidsets that contain data, in the resync path, clear the full parity raidset once (punch-or-zero) and then write the @cov ranges. Previously llapi_ec_write_parities() only wrote @cov and did no clearing, so stale parity could remain outside those intervals. Clearing the raidset once up front removes that stale parity and keeps the resync path simpler. In the verify path, @cov is irrelevant: the whole stripe must be compared, covering both data-backed ranges and zero-filled ranges, otherwise inconsistency would be missed. Right after @cov is created, expand and merge coverage intervals to page boundaries via ec_cov_page_align_and_merge(), since files in resync operation is opened with O_DIRECT. Add sanity-ec/12d,12e,12f. Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Keguang Xu <kxu@ddn.com> Change-Id: I3f68bab823277679a3e1804bbd2262ce7b6904a8
This points to an abandoned patch and can be removed
LU-19066 ofd: add os_failure_domain to struct obd_statfs Add a new field to lfs df to show the failure domain. Update "lfs df --output=" so that when used to just print a single field we do not pad it to a fixed width with leading spaces. This makes it easier to parse the outputs in tests. Example: lfs df --output=domain Also add a ltq_failure_domain field to lu_tgt_qos and set it when grab the statfs data for an object. We do not yet use this field for anything but will need this information later once we add failure domain awareness to the allocator. Test-Parameters: trivial Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: Ic5bcc66b7570ad74886a550b77d91a235e72756d
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-2 retesting | RHEL 10.1 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-zfs-part-5 failed 2× crashed | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
Going forward, all of these lines should be replaced with a label: Assisted-by: ClaudeCode:MODEL_VERSION [TOOLNAME ...] https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution
Why is this for `fortestonly`? Should this be consolidated with another patch, or that annotation be removed?
Probably it was generated initially via AI, and nobody removed that designation as the patch was being updated?
(typo) The third path is `lov_io_lseek_end()`, which is the `.cio_end` entry in `lov_io_ops[CIT_LSEEK]` and runs from `cl_io_end()`, not from unlock. `lov_io_unlock()` is a separate op. Should this read "during sub-IO end"?
[Marc Bot] (style) This attribution line was flagged on patchset 34 and is still unresolved: it should use the `Assisted-by:` label format described at https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution instead of the free-form line.
"three independent paths" does not seem to hold for the third one.
The LSEEK sub-lock enqueue happens in cl_lockset_lock(), which cl_io_lock() runs *after* every cio_lock(), so the stripe is already marked by the time lov_io_call(cl_io_start) runs and the sub-IO is skipped there. A sub-IO that never started still has:
sub_io->ci_result = 0 /* lov_io_sub_init() */
sub_io->u.ci_lseek.ls_result = -ENXIO /* inherited from the parent in lov_io_sub_inherit(); ll_lseek() seeds it */
lov_io_lseek_end() already ignores both (`ci_result == 0` is a no-op, `sub_off == -ENXIO` hits the existing continue), so there is nothing for the third hunk to catch.
Also, ci_result propagation happens in .cio_end (lov_io_lseek_end), not during unlock.
(style) This was raised on an earlier patchset and the line is unchanged: tool attribution should use the `Assisted-by:` trailer format documented at https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution rather than a free-form sentence.
This guard sits in `lov_io_call()`, which is the shared dispatcher for four different ops:
lov_io_lock() -> lov_io_call(cl_io_lock)
lov_io_start() -> lov_io_call(cl_io_start)
lov_io_iter_fini() -> lov_io_call(lov_io_iter_fini_wrapper)
lov_io_unlock() -> lov_io_call(lov_io_unlock_wrapper)
All four are registered for CIT_LSEEK, so a stripe marked LSS_READ_ERR also skips `cl_io_unlock()` and `cl_io_iter_fini()` on its sub-IO, not just `cl_io_start()`. Its `ci_state` then goes CIS_LOCKED -> CIS_IO_FINISHED (set by `lov_io_end_wrapper()` in `lov_io_lseek_end()`) -> CIS_FINI, never passing through CIS_UNLOCKED/CIS_IT_ENDED.
Nothing leaks today because `osc_io_ops[CIT_LSEEK]` registers only cio_start/cio_end/cio_fini and the LSEEK DLM lock is taken on the top IO by `vvp_io_lseek_lock()`. But the intent is only to skip the data-fetch pass -- would putting the check in `lov_io_start()` (or keying it on `iofunc == cl_io_start`) keep the cleanup passes balanced?
(style) This isn't a bug, but `str` reads like a string; the rest of this file spells it `stripe` (see `lov_io_lseek_end()` a few hundred lines down, which uses `index`/`stripe` for the same two values). Worth matching if the patch is refreshed.
[Marc Bot] (defect) Skipping the stripe treats its extents as holes, but on an EC file that data is still readable via parity reconstruction. If the only data between ls_start and the next healthy-stripe data lives on the degraded stripe, SEEK_DATA returns the later offset, or -ENXIO if none, so sparse-aware tools (cp, tar) silently drop data that read() would return. Also, if every sub-IO in lis_active is skipped (e.g. single-stripe data component), offset stays -ENXIO and SEEK_HOLE fails with -ENXIO even though ls_start < file size, which breaks the SEEK_HOLE contract of a virtual hole at EOF. Would it be safer to treat a degraded stripe's covered range as data instead of skipping it, or to return an error rather than a misleading data map? The commit message claim that the remaining healthy stripes provide valid seek results does not hold in these cases.
(defect) Skipping cl_io_start() for the degraded stripe means its allocation map never contributes to the seek result, and lseek does not reconstruct anything from parity the way CIT_EC_RD does. So the answer is not "the remaining healthy stripes are sufficient" -- it is an answer computed from an incomplete extent map.
Concrete case, 4+2 EC, one data OST deactivated, data written only in the range that maps to that stripe:
lseek(fd, 0, SEEK_DATA)
-> degraded sub skipped, others report -ENXIO
-> offset stays -ENXIO, lseek fails
and with data further out on a healthy stripe it returns that later offset instead. Sparse-aware copies (cp --sparse, tar, rsync) would silently drop the bytes that a plain read() still returns via parity.
SEEK_HOLE has the mirror problem: if every sub-IO covering ls_start is skipped, `offset` stays -ENXIO and ll_lseek() returns -ENXIO for an offset below i_size, which breaks the "there is always a virtual hole at EOF" contract.
Is returning an error preferable to returning a wrong offset here? Alternatively, could the degraded stripe's range be reported as data (conservative) rather than dropped?
+1; error should be preferable compared with a wrong offset
(typo) This comment uses a non-ASCII em dash; the rest of the tree is plain ASCII. Plain "-" or "--" instead.
This hunk looks like it has no effect. Any stripe marked LSS_READ_ERR was already skipped by the new check in lov_io_call(), so its sub-IO never reached cl_io_start(): ci_result is still 0 from lov_io_sub_init(), and ls_result is still the -ENXIO that lov_io_sub_inherit() copied from the parent. The `if (io->ci_result == 0)` assignment and the `sub_off == -ENXIO` continue below both already handle that. Is there a path where a sub-IO is marked LSS_READ_ERR but still ran? If not, dropping this hunk would keep the two skip conditions from having to stay in sync.
LU-12668 lov: handle ESHUTDOWN for LSEEK on EC files When an OST hosting a data stripe of an EC file is deactivated, SEEK_DATA/SEEK_HOLE fails with ESHUTDOWN because the error propagates through the LOV layer during sub-lock enqueue. Fix this by marking the degraded stripe LSS_READ_ERR in lov_lock_enqueue (same mechanism used by CIT_EC_RD for parity recovery), then skipping those stripes in lov_io_call and lov_io_lseek_end. This avoids ESHUTDOWN leaking through three independent paths: sub-lock enqueue, sub-IO function dispatch, and sub-IO ci_result propagation during unlock. Add lov_lsm_has_parity() helper to check if any layout entry has parity, needed because CIT_LSEEK locks the data component while parity lives in a separate entry. This fixes lfs mirror verify failing on EC files when any data OST is deactivated. Generated with Claude Code + Tools Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5cce4e0ea51c68b0c6fda1d83b694af19cad57bd
(defect) looks like the ubuntu build failing is related to folios, so this should be ```suggestion pgoff_t index = folio_index_page(vmpage); ```
(defect) same here ```suggestion pgoff_t index = folio_index_page(vmpage); ```
LU-12669 ec: fix EC recovery read for encrypted files During mirror resync with O_CIPHERTEXT flag for encrypted files: - Parity data is generated from encrypted file data - The parity is stored in encrypted form on the OST During EC recovery read: - Data pages read from OSC are decrypted by the OSC layer - Parity pages read from OST are still encrypted - EC recovery (`lov_ec_recover_page()`) tries to recover using mismatched data: - Decrypted data pages + Encrypted parity pages = WRONG recovery In this patch, we fix EC recovery read for encrypted files as follows: 1. Before reading data pages from OSC, we allocate bounce pages for encrypted files. 2. OSC will save the encrypted data to these bounce pages before decryption. 3. LOV will use these bounce pages for EC recovery instead of decrypted pages. - Pass encrypted data + encrypted parity to `lov_ec_recover_page()` - This produces encrypted recovered data 4. After EC recovery, we decrypt the recovered pages in-place - The recovered data is in encrypted form - Decrypt it in-place so it can be used by the read operation And if bounce page allocation fails, we will encrypt the decrypted data pages in-place before EC recovery. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: Ida4722d05d79bcbee4e680e75e94b005579adcac Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-ubuntu retesting | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
LU-12669 llite: make AIO reads on EC file synchronous For sync DIO, EC recovery on read failure runs inline in ll_file_io_generic after cl_sync_io_wait_recycle. AIO has the same recovery requirement but cannot run recovery from the sub-DIO completion path -- ll_file_io_generic returns -EIOCBQUEUED to the VFS before the BRW completion fires, so the syscall context recovery needs (the lu_env, the user's iov_iter, the range lock) is gone by the time the read error is known. Rather than building a workqueue-based async recovery path, make AIO reads on EC files behave as sync DIO: after cl_io_loop, if an AIO read on an EC layout (io->ci_cross_ec, set by lov_io_mirror_init during cl_io_rw_init) got -EIOCBQUEUED, set cda_no_aio_complete and cda_creator_free to disarm the async ki_complete path, then drop into the same wait + restart code sync DIO uses. The VFS calls ki_complete itself when we return. The flag flip is race-free because the submission ref on cda_sync is still held -- end_io cannot fire until we drop it in cl_sync_io_wait_recycle. This disables async semantics for every AIO read on an EC file, not only those that need recovery -- we cannot tell at submission time whether reconstruction will be needed, so the sync conversion fires unconditionally on -EIOCBQUEUED for an EC layout. As a side effect, the range_lock taken by ll_file_io_generic is now held until all sub-DIOs have drained, instead of being dropped while they are still in flight. Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Assisted-by: Opus:4.8 llm_code_and_review_tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I394a3610e33b29ead8f5adb52dfa21db6b721944
LU-17891 lfsck: fix lcm in __lfsck_layout_repair_dangling
When LPF_DELAY_CREATE_OSTOBJ is enabled,
__lfsck_layout_repair_dangling() loads LOV EA into lovea.
For PFL files, it erroneously assigned:
struct lov_comp_md_v1 *lcm = buf->lb_buf;
Since buf points to filter_fid instead of LOV EA, lcm->lcm_entry_count
read 0, skipping the component search loop and returning without creating the OST object for PFL files, but still counting them.
Fix lcm to point to lovea->lb_buf so PFL components are searched and
repaired during phase 2 dangling reference repair.
Test-Parameters: testlist=sanity-lfsck env=ONLY="14b 18d 18e",ONLY_MINUTES=30
Fixes: e2cdf469b022 ("LU-8998 pfl: layout LFSCK handles PFL file")
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Di Wang <ddiwang@google.com>
Change-Id: I007455f52d3cbe07be4e4228005afaba98f3bcec
LU-19109 llite: remove extra vvp_env_new_io call vvp_env_new_io is called twice in ll_file_io_generic, which is confusing since the second call overwrites the first. Fix this and a cleanup path mistake, where we assume the IO was set up after the call to vvp_env_new_io, which is wrong. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6186e17db8b04f01fd37ea7ac5d4b69b30d0258c
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56ab | seen in 5 other reviews |
this 88 of 500 sounds strange to me.. is it still failing?
I guess the intent is that the patch passed 5x as many iterations as without the patch, so it is improving the situation to some extent.
if this was the real problem introducing commit, it should be Fixes: header below.
The pieces of this patch don't add up. While I understand it improves the situation, now that that was demonstrated, it's time for a real human to look into why and make a patch that actually makes sense.
is this the actual weight bearing "fix" of this patch?
this comment seems wrong. We can never get here via unevict-clear (osc_unevict_cache_shrink call I guess?) because it sets reason to SK_REASON_UNEVICT_LRU, but that case is already handled above. That leaves a call for normal reclaim from osc_lru_reclaim?
I am not sure how this statement makes any sense?
LU-19487 osc: fix shrinker loop in osc_lru_list_shrink
Commit 109e32dc23 ("LU-19223 osc: stop after scanning")
changed the scan-limit tracking in osc_lru_list_shrink()
from a countdown to a count-up, and added a
--pages_scanned adjustment in the cl_object-switch path
to avoid counting object-switch iterations as scans.
When LRU pages belong to many different cl_objects, the
decrement causes pages_scanned to stagnate near zero,
making the loop run far longer than max_pages_to_scan
allows. The kernel's do_shrink_slab then re-calls the
shrinker because nr_scanned stays low, creating a
CPU-bound infinite loop that hangs the system.
Remove the --pages_scanned adjustment. The cl_object
switch does real work (dropping the spinlock, calling
cl_io_init) and should count toward the scan limit.
For forced scans (unevict-clear, cache-limit reduction,
etc.), increase the scan budget to target<<1 without
capping at lru_in_list so that object-switch overhead
does not prevent the shrinker from reaching its target.
Reproduced on a single-node test setup: sanityn test_16g
hung at iteration 17 of 100 without the fix (system
required hard reboot), passed 88 of 500 iterations
cleanly with the fix applied.
Generated with Claude Code + Tools
Test-Parameters: testlist=sanityn env=ONLY=16g,ONLY_REPEAT=10
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I7c007a1c021067e0b166b00045223ad8a927b5aa
LU-19744 doc: bulk man page review Bulk review done by Augment and Claude Code. Complete systematic review of Lustre man pages (sections 5, 7, 8) with fixes for documentation errors, missing options, formatting issues, and cross-references. Key fixes include: - Add missing options: --nomgs, --nidsfile, -q/--quiet, --catalog, -l, and others documented in usage but missing from man pages - Fix formatting errors: .TH syntax, incomplete sentences, SEE ALSO sections - Add missing cross-references to related man pages - Fix typos and grammar errors - Improve consistency across man pages - Add missing NAME section to lnetctl.8 - Fix alphabetical ordering in SEE ALSO sections (18 nodemap files) Files modified: 60+ man pages across sections 5, 7, and 8 Total changes: systematic improvements to documentation accuracy and completeness Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4896b64767e58c2b939b5a70db394a6f920949d5
| unique failing test | history |
|---|---|
| replay-single@ldiskfs+DNE:test_65a | seen in 66 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-6 crashed | RHEL 9.4 / x86_64 | ran 6 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
LU-13814 llite: remove unnecessary smp_mb() This smp_mb() was added as part of: https://review.whamcloud.com/c/39542 To help protect the csi_sync_nr atomic. But it was not justified there, and is not in fact needed. IO submission has *several* memory barriers before another thread can see the IO. (Note there is a discussion on that Gerrit suggesting it was unneeded.) For example: osc_queue_sync_pages uses a spinlock to put the extent on the list(s) (after this point, it's available for ptlrpc to make an RPC). There are several other instances of spinlocks, AND the use of csi_sync_nr is under memory barriers (in atomic_dec_and_lock, or under a spinlock). Let's remove this. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8b81f8e02025cae801a980d2856993c6d4023716
LU-18158 sec: retry mechanism for most syscalls
In addition to file/dir create, a number of other syscalls can
benefit from a retry mechanism in case of -EACCES. This can happen if
the client was not able to send the correct supplementary groups.
After the first failed attempt, the failure reply from the
server includes a hint with a possible supplementary group and ACLs.
The client uses this hint from the reply and retries with alternative
supplementary groups.
The syscalls handled by this patch are:
- open
- get/setattr
- mkdir
- unlink
- rename
- link
rename is special because unlike other syscalls handled by this patch,
access control is carried out based on permissions of source file,
source directory and target directory. As we do not have enough room
in the MDS_REINT_RENAME request, we proceed in 2 steps. If the rename
operation initially fails with -EACCES, the server hints the client
with the ACLs of the source directory, and the client tries again the
rename with this additional information. If it fails again with
-EACCES, the clients sets the MDS_RENAME_AGAIN op bias, so that the
server knows it needs to hint with the target dir ACLs this time in
the reply. The client then tries one last time the rename with this
additional information.
For the reply to have room for the ACL, RMF_ACL is added to the
MDS_REINT_UNLINK, MDS_REINT_LINK, MDS_REINT_RENAME and
MDS_REINT_CREATE_ACL reply formats, and to MDS_REINT_CREATE_SYM so
that it keeps matching MDS_REINT_CREATE_ACL, which is the format the
client uses to send a symlink create. The client and the server both
set the ACL reply buffer size to 0 as long as the hint is not needed.
sanity-sec test_80 is added to exercise this.
Test-Parameters: serverversion=2.15 testlist=sanity env=EXCEPT="24I 230k 230w 272 807"
Fixes: e603ddadd5 ("LU-18158 sec: hint client in case of failed reint open")
Fixes: e826e8bb77 ("LU-17961 sec: support supplementary groups from client")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6497b21356dfbfc648c58dec0f9c1868655f6364
Was-Change-Id: I515467cb7fb5c26a5496f8cac9fb4749723bae5a
Was-Change-Id: I424affdafb1fd9ce375cf30f6e5c8af1716a3899
Was-Change-Id: If7ebbe2b8e3a897f433737693b2ab63b3dae59a4
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Martin Schwenke <mschwenke@ddn.com>
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 retesting | RHEL 9.7 / x86_64 | ran 11 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-2 retesting | RHEL 10.1 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-ldiskfs-ubuntu retesting | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
(minor) The ll_readahead_handle_work() piece fixes a pre-existing bug that has nothing to do with EC: ll_ra_count_get() reserves the pages and nothing puts them back when cl_io_rw_init() fails, and ll_ra_count_put() is the only decrement of ra_cur_pages. A Fixes: line would let the maintenance branches pick it up:
Fixes: c2791674260b ("LU-12043 llite: improve single-thread read performance")
(defect) The lov_io_set_range() hunk fixes an LBUG that 84c1a4a074 introduced, and that commit is three patches back in this same unlanded series. That leaves 84c1a4a074, 13af17f231 and b718cadc7b each panicking the client on a two-component EC layout, so the series is not bisectable and those revisions cannot be tested on their own.
Can the clamp be folded into 84c1a4a074 instead? If it has to stay a separate patch, it needs its own tag alongside the existing one:
Fixes: 84c1a4a07423 ("LU-12669 ec: recover data from parity")
Good catch. Definitely best to check for the inactive OSC device early. This would be set some seconds after the OST goes offline, so IO shouldn't wait to detect that every time.
Max, I rebased this on Bobi's latest patch - you were based on an older version
(minor) This label bypasses cl_io_fini(). cl_io_init()'s contract is that the caller calls cl_io_fini() no matter what it returned, and the commit message points out that this exit stops being rare once a dead import can fail an EC read at init. Should the new path run cl_io_fini(env, io) before dropping the reservation?
(defect) On a component with lsme_dstripe_count == 0 this can push eoff well past lio->lis_endpos.
Neither place that sets the cycle end rounds it to a recovery group on that branch: lov_io_set_range() skips its end-rounding block when dstripe_count == 0, and lov_io_ec_rd_iter_init() only rounds when dstripe_count > 1. So lis_endpos is just the request end, while eoff becomes soff + RGs, and RGs there is ss * 4 (or ss * lo_nr).
lov_ec_read_stripe_pages() classifies pages against eoff alone, so the pages in [lis_endpos, soff + RGs) are grabbed and submitted. lov_io_lock() enqueued only [ec_inner.crw_pos, +crw_bytes) for this cycle, and lov_ecio_add_data_sub() skipped those stripes (lov_stripe_intersects() false against the cycle extent), so lov_sub_get() allocates a fresh sub-IO with no DLM lock -- the same "uncovered page!" LBUG in osc_req_attr_set() that the lov_io_set_range() hunk is fixing.
Worked example, layout "-E 128M --ec 4+2 -E 512M -c 4" (EC component followed by a plain one, as in the mirror layouts in this suite), read [0, 130M) with a dead OST:
cycle in the plain component: lis_pos 128M, lis_endpos 130M
dcount 4, ss 1M => RGs 4M, soff 128M
eoff was min(132M, 130M) = 130M, now min(132M, 512M) = 132M
stripes 2 and 3 cover [130M, 132M) -- outside the enqueued lock
Before the change those pages were EC_DPG_ZERO and never touched. Should the clamp stay bounded by lio->lis_endpos on the dcount == 0 branch, where there is no recovery group to complete?
(minor) The sibling switch below clears the retry count before returning:
io->ci_switch_ec_io = 1;
io->ci_need_restart = 1;
io->ci_ndelay_tried = 0;
RETURN(-ENODATA);
This one leaves ci_ndelay_tried alone, and ll_file_io_generic() carries it across the restart (retried = io->ci_ndelay_tried). So if the read had already restarted once before the import went invalid, the CIT_EC_RD pass lands on ndelay_tried: with a non-zero count and can still take the 10 ms schedule_timeout_interruptible() and set ci_tried_all_mirrors -- the backoff the commit message says this path avoids. Should it reset the count too?
error: lov_lock_enqueue():'osc' dereferencing possible ERR_PTR()
error: lov_lock_enqueue():'osc' dereferencing possible ERR_PTR()
[AI review - fable] (defect) Including CIT_READ here looks dangerous. After the continue, the top lock is granted without covering this stripe, and nothing in the normal read path checks lso_status (only the EC_RD and LSEEK paths do), so pages on the dead stripe are still generated and submitted, and osc_build_rpc() -> cl_req_attr_set() -> osc_req_attr_set() hits the "uncovered page!" LBUG in osc_object.c when osc_dlmlock_at_pgoff() finds nothing - the very crash the commit message wants to avoid. The lov_io_mirror_init() check does not guard this path when ci_cross_ec is unset (a read of a non-EC component of a file whose layout has parity entries elsewhere - lov_lsm_has_parity() is file-wide, and lov_io_ec_rd_start() explicitly supports dcount == 0 components), or when the import goes inactive after cl_io_init(). Before this change the enqueue failed fast with -ESHUTDOWN and the IO returned an error or restarted into EC_RD instead of crashing. Should the skip be limited to CIT_EC_RD, letting CIT_READ fail the enqueue so the existing restart logic switches to EC_RD? As a side effect the CIT_READ marking also leaves a stale LSS_READ_ERR on the lovsub (the normal read path never resets it), which lov_io_lseek_end() then treats as degraded even after the OST is reactivated.
error: lov_lock_enqueue():'osc' dereferencing possible ERR_PTR()
error: lov_lock_enqueue():'osc' dereferencing possible ERR_PTR()
(style) This isn't a bug, but the include groups are inverted here - `<lustre_osc.h>` is a lustre header and belongs before the local `"lov_cl_internal.h"`. The same include added to lov_io.c in this patch is ordered that way.
LU-12668 lov: proactive dead-OST detection for degraded reads
Instead of letting a read proceed to an inactive OST and fail deep in
the BRW/lock path, detect dead imports during mirror selection and
route to the EC recovery path.
- lov_io_mirror_init() rejects a candidate mirror that has a data
stripe on a deactivated or invalid import, so FLR rotation can still
pick an intact mirror. Only when no mirror is intact does it set
ci_switch_ec_io, which also skips the FLR backoff sleep -- we
already know the OSTs are dead. The scan covers the whole mirror
rather than just the stripes the I/O touches: see the comment on
lov_ec_has_inactive_stripe() for why bounding it to the I/O extent
is not safe until the size path handles unreachable stripes.
- lov_ecio_add_data_sub() marks such stripes errored up front, so the
recovery loop reconstructs them instead of building a sub-IO that
cl_io_iter_init() or the lock enqueue would reject.
lov_io_set_range() rounded the CIT_EC_RD range end up to a raid-set
boundary using the geometry of the component holding the read end, but
that rounded value can land past the component. The next component
anchors its raid sets at its own e_start, so the end fell mid-raid-set
there: lov_io_ec_rd_iter_init() cut the cycle short at it while
lov_io_ec_rd_start() still read a whole recovery group, leaving pages
outside the lock lov_io_lock() had enqueued. osc_req_attr_set() then
LBUGged ("uncovered page!") from brw_queue_work and panicked the
client. Clamp the rounded end at the component boundary; the read
cannot need data past it, since the request itself ended inside that
component and each component is erasure coded independently.
Reproduced with a -E 4M -c 3 --ec 2+1, -E -1 -c 5 --ec 4+1 layout.
ll_readahead_handle_work() leaked its ra_cur_pages reservation when
cl_io_rw_init() failed. That exit is rare today but becomes routine
once a dead import can fail an EC read at init, and ll_ra_count_put()
is the counter's only decrement, so readahead would stop for the whole
mount and never recover. Release the reservation from a single exit
label that every path past ll_ra_count_get() reaches, and call
cl_io_fini() on the failed init, which cl_io_init() requires no matter
what it returned.
Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools
Test-Parameters: testlist=sanity-ec ostcount=8
Test-Parameters: testlist=sanity-ec ostcount=8 fstype=zfs
Test-Parameters: testlist=sanity ostcount=8
Fixes: c2791674260b ("LU-12043 llite: improve single-thread read performance")
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I0913e1ee977c9850193c92835edb185b0aedc6d4
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 crashed | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. %% THIS TEST SESSION CRASHED %% | session |
| custom-1002 crashed | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-subtest-change failed 30× | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-zfs-subtest-change failed 29× | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
(minor) The range here starts at 49b, but 49a "test concurrent reads during EC recovery" is added by this patch too. Should it read 49a-50b? Every other test added is accounted for by one of the ranges.
(minor) Two hunks are not accounted for by any of the ranges above: the new `[[ "$SLOW" == "no" ]] && EXCEPT_SLOW="74a 75b"` line, and the `head -n1` fix in `enable_ec()` for the multi-mount `get_param` output. The `EXCEPT_SLOW` one also changes an existing test: 74a is not added by this patch and is not named in any of the range lists, so a reader can't tell from the message that it is being moved onto the SLOW list. Worth a sentence for each.
(defect) With 3 of 4 data stripes unreadable and pcount=2, -EIO is the only correct outcome; returning reconstructed-but-wrong bytes is silent corruption. Treating it as an acceptable pass means a real reconstruction bug in this path would go unnoticed.
48a, 48c and 64b in this same patch take the opposite position ("handing back silently wrong data would be a corruption bug, so assert against it"). Should 52b assert rc != 0 instead?
(minor) These eight files (2M..16M) are never removed. The same applies to the extra files in 71b/71c/71d, 73b's .pure, 73c's .zero and the directories in 75e/75f; 73e is the only new test that registers a cleanup. Added up, the new tests write on the order of 1.5G and leave most of it in place for the rest of the run, which on a small test filesystem can push later subtests into ENOSPC. A `stack_trap "rm -f ..."` next to each creation would keep the footprint bounded.
(suggestion) Both of these are parked against the feature ticket itself. 53a in particular is described as intermittently reconstructing wrong data, which is a silent-corruption symptom rather than a test problem, and 41d is an OSC retry loop that never terminates. Would separate LU tickets referenced here keep them from being lost when LU-12668 is closed? The convention elsewhere in this file (12a -> LU-19631, 5b/12b -> LU-20435) is one ticket per known failure.
(minor) A fresh `stack_trap` is pushed on every call, including repeat calls for the same index, so loop-driven tests accumulate identical cleanup entries: 75b registers 50, 74a 20, and 58a/63b/68b one per pair. Each entry re-runs `ec_apply_fault`, which is a `do_nodes` to every OSS plus a `cancel_lru_locks osc`, so teardown does that work dozens of times over. Registering the trap only when the bit was not already set would make it one entry per OST.
(minor) When no stripe list is given this walks every data stripe and returns the first parity-free one, which can be a stripe that holds no data. For a sub-raid-set file that is the vacuous-pass mode this helper is documented as preventing: 42a writes 512K into a 1M-stripe 4+2 layout, so only stripe 0 has data. If stripe 0's OST happens to double as parity, the fault is armed on stripe 1's (empty) object and the checksum comparison succeeds without reconstructing anything. `ec_data_stripe_osts()` handles the analogous case by calling `skip_env`. Would returning non-zero (so `ec_start_read_fault()` skips) be safer than falling through to a later stripe?
(minor) 44c and 44a are the same test - same layout, same `ec_start_all_reads_fail`, same EIO check, same clear-and-reread - differing only in the error strings. 43c is that body minus the reread. Could these collapse into one? While here, 43c's description says "too many OST failures (3+ OSTs)" but `ec_start_all_reads_fail` uses `fail_val=0`, which fails every OST, not three.
(minor) The comment says "an OST that is a safe data OST for all files", but only `$f2` is classified. `$f1` (2+1, 3 objects) and `$f3` (2+2, 4 objects) get whatever the allocator gave them, so on an 8-OST config the victim frequently holds none of their objects and the `$s1`/`$s3` comparisons pass without any recovery running. 71a, 71c, 71d and 75e/75f have the same shape (classify one file, assert on all of them). That is defensible for a batch test, but here the comment claims something stronger than the code does.
(defect) `safe_osts` here still comes from the `ec_classify_osts $tf` above, but the SEEK_DATA/SEEK_HOLE checks below run on `$tfs`, which was created separately and gets its own object placement from the allocator.
So the OST taken out need not hold any of `$tfs`'s stripes, and on a run where it doesn't, the two `lseek_test` assertions execute against a fully healthy file.
73c already documents and avoids exactly this ("safe_osts still describes $tf; $tf2 has its own object placement") by calling `ec_data_stripe_osts $tf2 0` first. Should 75d do the same for `$tfs`?
Related: the comment says the seek "crosses the degraded stripe", but the data lives at 5M, i.e. stripe 1 with `-c 4 -S 1M`, while `safe_osts[0]` is just the lowest-numbered data OST.
(minor) The negative-index handling, and the "Index -1 is the last stripe" note in the header comment, appear to be unreachable: all eleven callers pass 0..4. Worth dropping the branch and the doc line unless a caller is coming.
(style) The suite convention is a `#define` comment naming the fault right above the line that arms it, so a reader does not have to look up the bare hex. `ec_start_all_reads_fail()` and 41d both do this; this call site and the one in `ec_apply_fault()` do not.
#define OBD_FAIL_OST_BRW_READ_BULK 0x20f
ec_ost_fail_loc 0x20f $(( 0x10000 | mask ))
The value itself is right (obd_support.h has 0x20f), it is only the annotation that is missing.
LU-12668 tests: add EC recovery tests Add sanity-ec coverage for erasure-coding recovery. Each test writes an EC file, resyncs parity, fails one or more OSTs, and verifies the client reconstructs the data from parity (CIT_EC_RD) against the pre-failure checksum. Failure is injected with OBD_FAIL_OST_BRW_READ_BULK so the OSC import stays active and only bulk reads fail, which drives genuine parity reconstruction. The fault is set on the OSS nodes, where tgt_brw_read() evaluates it, and osc.*.resend_count is dropped to 1 for the duration so the injected -EIO reaches the LOV layer instead of being absorbed by an OSC resend. Victims are chosen by stripe rather than by OST index. ec_pick_data_ost() walks a file's data stripes in order and takes the first whose OST does not also carry parity: a file smaller than one raid set holds data on stripe 0 alone, so picking the lowest OST index instead would arm the fault on an object the read never reaches and the test would pass without exercising recovery. ec_check_fault_index() skips when a target OST index is >= 16, which cfs_fail_index() cannot express in its 16-bit fail_val bitmask. ec_mirror_victims() fails one parity-free OST in every data mirror, since a file with more than one data copy would otherwise answer the read from an intact mirror rather than reconstructing anything. Reads that check a sub-range cancel their locks first: a range re-read after a whole-file read is otherwise served from the page cache, issues no BRW RPC, and so never reaches the injected fault. Geometry and I/O patterns (40b-44c): - 2+1, 2+2 and 4+1 EC; partial, offset, mmap, direct and async reads; single, maximum and progressive OST failure; graceful failure when too many OSTs are gone Failure placement and layout (45a-48c): - consecutive, non-consecutive, boundary and parity-only OST failures; 64K, 256K and 4M stripe sizes; multiple EC and mixed EC/non-EC PFL components; stale and partially stale parity Concurrency and multi-mount (49a-50b): - concurrent readers over a file with two failed data OSTs; background writes, mirror resync and OST reactivation during recovery; both mounts reading the same EC file Core recovery (51a-53a): - EOF boundary recovery at RAID set / stripe set boundaries; degradation limits and mixed parity+data failure; recovery at non-zero read offsets Layout patterns (55a): - file-size boundaries (1 byte .. multi-stripe) Edge cases (58a-62d): - parity_used combinations and stripe rotation; sparse files with holes; recovery after truncate; sub-stripe files; append writes Multi-target (63a-66d): - OST failure cycling and multi-mount coordination; 3-4 component PFL with per-component EC geometry; multiple EC mirror pairs Write patterns (67a-71d): - writes to healthy stripes during degraded mode and overwrite cycles; varied geometries; O_DIRECT writes; partial and mid-file overwrites; batch recovery of many files Stress and admin (73a-73e, 74a, 75a-75f): - random reads, fallocate, truncate-extend, O_APPEND, and large (128M) file recovery; reads racing with OST deactivation; stat/getattr, stress loops, lfs mirror verify, directory-inherited layout, and stripe rotation Tests 41d and 53a are added but listed in always_except. A degraded mmap read never completes: the OSC alternates between "too many resent retries" and osc_brw_redo_request() forever, so -EIO never reaches the LOV layer and CIT_FAULT never switches to CIT_EC_RD. A recovery read at a non-zero offset intermittently reconstructs wrong data. Both stay off until those are fixed. Test 65c and the sparse half of 75d are skipped on ZFS. Both build a sparse file and resync it, which needs lseek to report the holes so resync knows which stripe sets to skip, and ZFS does not report them reliably for dirty data. Test 12b describes the same problem. Assisted-by: ClaudeCode:opus llm_code_and_review_tools Test-Parameters: trivial testlist=sanity-ec ostcount=8 Test-Parameters: trivial testlist=sanity-ec ostcount=8 fstype=zfs Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Max Dilger <mdilger@whamcloud.com> Change-Id: I5a06cd166487e0bff7bfdb6a39414af3f12c4326
(defect) `GOTO()` only evaluates its second argument, it never assigns to `rc`:
# define GOTO(label, rc) do { ((void)(rc)); goto label; } while (0)
The other hunks work because `rc = 1` is itself an assignment, but `rc ?: 1` has no side effect, so `rc` is still 0 on the -ENODATA path. It reaches `stop:` with 0 and lfsck_layout_trans_stop() flips it back to 1, which is exactly the case this patch is removing elsewhere.
Would `GOTO(unlock2, rc = rc ?: 1)`, or a plain `if (!rc) rc = 1;` before the goto, be clearer here? This is also the only `GOTO(..., x ?: y)` in the tree, so it is easy to misread as an assignment.
LU-17891 lfsck: fix dangling object repair handling
Change the `GOTO(unlock2, rc = 0)` returns to use `rc = 1` instead.
This return value is reversed in lfsck_layout_trans_stop(), so using
`1` means nothing was repaired and it will return `0` to the caller
and avoid logging "Create the lost OST-object as required" and
counting this object being repaired when nothing was done.
Found during code review of another patch.
Test-Parameters: trivial testlist=sanity-lfsck
Fixes: 17cc912fd5b4 ("LU-8288 lfsck: handle dangling LOV EA reference")
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Change-Id: I733932b315ae186ed1f99b0f7dd65b49d4985999
(suggestion) The body says an interop run skips the new subtests rather than failing them, but nothing here exercises that. Would adding a second line such as `Test-Parameters: testlist=sanity-ec ostcount=8 serverversion=2.17.55` prove the version gate actually skips against an older MDS?
(minor) The extra storage for EC(D+P) is PARITY/DATA of the file size, not (DATA+PARITY)/DATA - (DATA+PARITY)/DATA is the total. The 8+2 example further down gets this right ("1.25x the file size, vs 2x for a plain mirror"), so the two read as contradicting each other. Reword as total storage, or use PARITY/DATA here?
(defect) Should this refuse a data component that is already linked, instead of overwriting its link id?
`data_lcme->lcme_time_and_id` may already carry the mirror id of an existing parity mirror. The old id is dropped here and replaced with the newly merged one, so the earlier parity mirror ends up half-linked from the data side - the exact state the `!link_id` check a few lines up refuses.
The only guard against that today is client-side, in `build_parity_layout_for_mirror()`, and it is not race-free: `llapi_layout_get_by_path()` reads the layout before `mirror_extend_layout()` takes any lease, so two `lfs mirror extend --mirror-id N --ec D+P` runs on different clients both see mirror N unprotected and both merge. The second merge re-stamps the data component; `verify_new_mirror_id()` then reports -EBUSY, but the layout has already been committed. `llapi_layout_comp_add_parity_only()` is also a new public, man-paged entry point, so any application can build the same merge buffer directly.
The result is a file with two parity mirrors on one data mirror, which the commit message says is rejected for now. `lod_verify_striping()` already re-checks the k+m bound with "Cannot trust userspace; enforce here too" - the same reasoning seems to apply to the link.
if (lcme_timestamp_id_unpack(time_and_id))
GOTO(out, rc = -EEXIST);
This needs to be fixed before the EC feature goes into production, but is still under discussion and not the target of this patch.
(minor) `llapi_layout_pool_name_get()` never sets ENODATA - it snprintf()s `comp->llc_pool_name` (empty when there is no pool) and returns 0, failing only on a NULL current component, a NULL `dest`, or a foreign pattern (EINVAL). So the `errno != ENODATA` carve-out never triggers and the comment describes behaviour the API doesn't have. If it stays, the non-ENODATA failure leaves `rc` at -1 until the next assignment overwrites it, which is easy to trip over later.
LU-19548 lfs: mirror extend support for EC
Add three EC modes to 'lfs mirror extend':
--mirror-id N --ec D+P attach a parity mirror to existing data
mirror N (N >= 1; id 0 is reserved for
non-FLR layouts and is rejected)
-N [K] --ec D+P add K new data+parity mirror pairs
(K defaults to 1)
--ec D+P attach a parity mirror to the file's lone
data mirror, for the non-FLR case where the
user just wants EC protection on a plain
file; a file with 2+ mirrors errors out
asking for --mirror-id or -N
As a regular extend never leaves a new mirror stale, the new mirrors
are in sync when the command returns: data mirrors are populated by a
copy before the layout merge, parity mirrors are computed in place
with the machinery 'lfs mirror resync' uses. A partial extend resyncs
the parity mirrors it did attach before reporting the error, and one
that was merged but could not be marked stale is reported with the
command that removes it, since resync finds mirrors by LCME_FL_STALE
and so cannot repair that one.
A parity mirror is attached by merging a layout that holds only the
new parity components, while the data mirror they protect stays where
it is. The new llapi_layout_comp_add_parity_only() builds that layout;
llapi_layout_comp_add_ec() cannot, as it derives the parity extent,
stripe size and pool by searching for the data component in the same
buffer, and pairs the two with a transient link id that is resolved
during creation. Client-side layout sanity likewise had to stop
requiring a data component for every parity component.
Splitting the pair across two files needs two changes in lod.
lod_qos_prep_create() sizes a parity component from the data component
it protects and rejects one that has none; the data component of an
extend is not in the volatile file being created, so the client sends
its stripe count as the parity component's own and lod derives the
raidset count from that. lod_declare_layout_merge() copies the
incoming entries verbatim, so only the parity -> data half of the link
would be stored; it now also stamps the matching data component with
the mirror id assigned to the parity component, without which 'lfs
mirror split' cannot see the pair from the data side and leaves the
parity mirror behind. A link naming a mirror that is not in the
merged layout means the data mirror was split away after the client
read it, and is refused rather than stored as a half-linked pair that
resync would then fail on for the life of the file.
'lfs mirror extend --ec' also refuses a data mirror that is stale or
'nosync'. Parity is computed from the content of the named mirror, and
resyncing that mirror afterwards does not re-stale its parity, so the
parity would silently protect superseded data; a nosync mirror is
skipped by resync, leaving the parity uncomputed.
The rider in lod_qos_prep_create() rejecting an EC data stripe count
of zero is unrelated to extend: lod_verify_striping() bounds only
k + m, so a k == 0 layout reaches ec_split_stripes() and divides by
zero in the kernel.
Attaching a second parity mirror of a different geometry to one data
mirror, which would let a file's EC protection change without
rewriting its data, is not implemented. It is rejected for now, and no
test asserts that rejection.
An MDS that predates this change refuses a parity-only create buffer
outright, so the new sanity-ec subtests are gated on the server
version and an interop run skips them instead of failing.
Components EC cannot be sized for are named rather than left to a bare
errno: a Data-on-MDT component has no OST stripes to spread a raidset
over, and a self-extending component's extent does not stay matched to
its parity component, both as 'lfs setstripe --ec' refuses them. --ec
is refused with -f as the man page says it is, and a second --ec on
one command line no longer reports success after printing a usage
error. A '-N=TOTAL' that finds the file already at TOTAL mirrors adds
none, as it warns, instead of falling through to the auto mode that a
bare --ec selects.
struct collect_ids_data moves ahead of the new EC-extend helpers,
which reuse it and collect_mirror_id() to gather a file's distinct
mirror ids.
Fixes: 8c5f5d3ec9b1 ("LU-12668 lod: bind ec mirror to data mirror during creation")
Test-Parameters: testlist=sanity-ec ostcount=8
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Assisted-by: ClaudeCode:Fable-5 llm_code_and_review_tools
Change-Id: I8d670f0558521063af425e2c75a56b00cdd294d7
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
LU-xxxxx lfs: allow --ex=1+p p>=1 in lfs setstripe Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I03abe044301d95fbe168fba32f109bbb09d7bd53
(defect) This adds a new OBD_CONNECT2 flag and a new mbo_t_state bit, i.e. it changes what old and new peers exchange. Per the interop rules a wire change should request interop coverage, e.g.
Test-Parameters: testlist=sanity,sanity-flr serverversion=2.15.6
Test-Parameters: testlist=sanity clientversion=2.15.6
Worth adding so a mismatched client/server pair is actually exercised.
(minor) A handful of hunks are not accounted for by the message and look like they came along for the ride: - sanity.sh: blank lines removed in test_17o, test_27Ke, test_64d, run_acl_subtest(), test_119f and test_271ba. - lod_object.c: the comment above the striping-data declare in lod_declare_striped_create() reflowed onto one line. Could those be dropped, or called out in the body?
Hi Keguang, thanks for working on this. I think it will be quite useful for maintaining filesystem accessibility with an offline OST. Is the layout replacement done for all of the file components, or only components that are using an offline OST? That would be more efficient, if it is practical to implement. Bobijam, can you please review the interaction with the layout handling.
Thanks Andreas. The patch is based on `mo_swap_layouts` which operates on file granularity. I've searched the codebase, there's no component granularity swap yet, neither in HSM part. While this patch does not introduce that functionality, implementing component-level swap and HSM appears to be a reasonable and valuable extension for future work?
Hi, patch is a bit of large, following is a quick sketch to streamline your code review process.
Server side:
```
mdt_reint_setattr()
│ size==0 && client has OBD_CONNECT2_LAYOUT_REPLACED
├─ mo_layout_check(UNAVAILABLE) # inactive OSTs?
└─ mdt_replace_unavailable_layout()
├─ orphan victim
├─ instantiate(from original layout template) # new OSTs via QoS/RR
└─ swap_layouts # old data→victim, new layout→file
→ reply: new LOVEA + MS_LAYOUT_REPLACED + size=0
```
Client:
```
ll_setattr → md_setattr → MDS
↓ reply
ll_update_inode → cl_file_inode_init
│ MS_LAYOUT_REPLACED + size=0
│ → coc_prune_mode = DISCARD
└─ lov_layout_change → prune(DISCARD) # drop stale cache, no writeback to bad OSTs
→ install new layout
```
(minor) The bit value matches what was reserved on patchset 16, but the name doesn't: the request was
#define OBD_CONNECT2_LAYOUT_REPLACE 0x8000000000000ULL /* trunc(0) new layout */
The name leaks out to users through obd_connect_names[], so it is easier to settle now than after landing. Renaming would also touch exp_connect_layout_replaced(), the lprocfs entry, wirecheck.c and both wiretest.c copies.
(minor) Carried over from an earlier patchset and still open: the bit value matches what was reserved, but the reserved name was OBD_CONNECT2_LAYOUT_REPLACE (no trailing D), with the comment /* trunc(0) new layout */. The name is user-visible through obd_connect_names[], so it is cheaper to settle before landing. Renaming also touches exp_connect_layout_replaced(), the lprocfs_status.c entry, wirecheck.c and both wiretest.c copies.
(minor) exp_connect_parity() has no callers anywhere in the tree, and the commit message doesn't mention it. Is it left over from an earlier revision, or meant for a follow-on patch? If it stays, the name doesn't match what it tests: the flag is OBD_CONNECT2_FLR_EC (erasure coding support), not parity specifically. exp_connect_flr_ec() would read closer to the neighbouring exp_connect_flr().
This feature by itself should not discard any cached data. The `truncate(0)` request should already return an `LDLM_FL_DISCARD_DATA` flag in the DLM lock callback, and layout lock cancellation should be sufficient to update the layout on the clients.
`LDLM_FL_DISCARD_DATA` and layout-lock cancel on trunc are mainly for *other clients* (extent holders discard on conflict cancel; layout holders invalidate and refresh).
The failure we hit is on the *truncating client*, which applies the new LOVEA from the MDS setattr reply — not via those other-client paths:
```
ll_setattr_raw
└─ ll_md_setattr ← MDS: size=0, maybe layout replace
└─ ll_update_inode
└─ cl_file_inode_init ← apply new LOVEA from reply
└─ lov_layout_change
└─ prune(LOCAL) ← writeback to old/unavailable OSTs → -EINVAL
└─ cl_setattr_ost ← later; CEF_DISCARD_DATA asks other
clients to discard on conflict cancel
```
So neither extent `DISCARD_DATA` nor layout cancel closes this window for the truncating client. Other heuristics (e.g. client-local `ATTR_SIZE==0`) could justify discard, but plumbing them here is awkward. `MS_LAYOUT_REPLACED` + `size==0` is explicit and local: trunc made the old cache obsolete; replace marks that this reply applies a new layout and must not LOCAL-flush. Migrate / swap_layouts stay on `CL_FSYNC_LOCAL`. Complexity looks acceptable.
This seems like an independent fix that could be landed early?
Standalone this is a tiny cleanup. With `as_template`, shrink becomes `as_template || (!inited …) → 1`, which would also hit INIT DoM and disagree with lod_comp_md_size() (still sized as 0). The DoM special-case is only to keep `as_template` from forcing that bogus slot.
(style) This isn't a bug, but the assignment is redundant now that the whole struct is cleared by the memset just above.
(minor) The availability check runs before mdt_replace_unavailable_layout() takes MDS_INODELOCK_LAYOUT|XATTR, and nothing re-checks once the lock is held. Two concurrent trunc(0) calls on the same file both see "unavailable" and both replace: the second builds a template from the layout the first just installed, allocates a second set of objects, and swaps the first set onto its own victim to be destroyed, bumping the layout generation twice. Would moving mo_layout_check() under the layout lock avoid that?
LU-20211 mdt: trunc(0) layout replace on inactive stripes When truncate(0) hits a file whose layout references inactive OSTs, and the client advertises OBD_CONNECT2_LAYOUT_REPLACED, replace the layout: move the old layout onto a volatile victim and instantiate a fresh layout on the file from the existing pattern via QoS/RR. Extend the layout API with MD_LAYOUT_UNAVAILABLE (detect stripes on inactive OSTs) and MD_LAYOUT_INSTANTIATE (allocate from a source pattern). Teach lod_generate_lovea() to emit uninstantiated LOVEA templates (as_template), omit placement hints so new stripes use QoS/RR, and trigger replacement from mdt_reint_setattr() via mdt_replace_unavailable_layout(). Layout-type handling for as_template / instantiate: - DoM: never carries OST objects; force stripe_count to 0 and skip the OST object array so LOVEA size matches INIT DoM. - PFL/DoM: keep the DoM component and replace only OST components. - SEL: coerce LOV_MAGIC_SEL to COMP_V1 for lod_qos_parse_config(); keep LCME_FL_EXTENSION and restamp SEL when LOVEA is written. - EC: re-emit bound data/parity as unbound (IS_LINK_ID + shared link id) so lod_qos_prep_create() can pair before component IDs exist. mdt_orphan_open() reused a stale mti_spec from a prior create, which could leave sp_cr_job_xattr set and trigger an undeclared job xattr set LBUG. Clear mti_spec for internal orphan creates, reset sp_cr_job_xattr in mdt_thread_info_reset(), and declare job xattr credits in mdd_declare_create_object(). On success, return MS_LAYOUT_REPLACED (with OBD_MD_FLEASIZE) in the setattr reply. The client then applies the new LOVEA via layout change / prune. By default prune uses CL_FSYNC_LOCAL and would writeback dirty pages still cached against the old layout to the unavailable OSTs (fail or hang). When MS_LAYOUT_REPLACED is seen with size 0, set coc_prune_mode to CL_FSYNC_DISCARD so prune discards that stale OSC dirty cache locally instead of writeback; VFS page cache is still dropped by ll_truncate_inode_pages_final() as usual. Add sanity/test_27xt (plain), 27xu (PFL/DoM), 27xv (SEL), 27xw (EC), and sanity-flr/test_50e/50f. Signed-off-by: Keguang Xu <kxu@ddn.com> Change-Id: I07e0792a51e144d935cece4b028e17f659e4ccfd
This should be reworded to make it clear that these issues only exist in the old version of the code where the lock server is making the decision for which locks to cancel. Something like: ``` If the lock server selects specific locks to cancel, this can lead ... ```
`... in active use on the client ...`
`Server-selected lock revocation ...`
Please describe the mechanism implemented.
If we are moving forward with a new server-side notification mechanism to cancel client locks (instead of the CLV/SLV exchange), then we should consider to disable this mechanism completely once all clients support the new mechanism. The old mechanism generates a non-trivial amount of overhead on the client to update the CLV for each namespace (it was originally done on 1-second intervals for *all* namespaces on the client, but had to be scaled down to one namespace per 1-second interval to avoid impacting client performance). If the server starts sending the "magic cancel" then the client could stop calculating the CLV and the server can stop sending it as well.
IMHO I'd rather keep the *CLV/SLV* mechnism since it `DO` give the clients the choice to select the locks that they prefer to return. With "magic cancel", clients still need to figure out which locks they should cancel, it's just another form of *CLV*? The issue to the SLV is it's not aligned with CLV in some degree right now, I'm on it and will upload a patch hoping to fix it(with CLV intact). Still, we need reclaim notification here as the safety net that should be relied upon once *CLV/SLV* broken.
> IMHO I'd rather keep the CLV/SLV mechnism since it DO give the clients > the choice to select the locks that they prefer to return. > With "magic cancel", clients still need to figure out which locks > they should cancel, it's just another form of CLV? Sorry, I don't understand your comment. In both cases, CLV/SLV and "magic cancel" (or set_info), the client needs to decide *which* locks to cancel, while the server decides *how many* to cancel (though I guess the client could still use this as a "hint" and cancel more or fewer locks if needed). We *want* the client to decide this, since it has more information about the current lock usage than the server. One benefit of "magic cancel" is that it is a simpler mechanism to understand, a clear notification to cancel "some" locks. The CLV/SLV mechanism has (AFAIK) been at least partly broken for many years and nobody can ever get it working. The main benefit of CLV/SLV (AFAIK) is that the client is essentially notified of a "rate" to cancel locks, so the server does not need to continually send RPCs to each client. I guess the same could be done with "magic cancel", at which point they are nearly the same thing. It would definitely be awesome of you could fix this code to be more robust.
(defect) I would prefer that this check all lock resource fields are {0,0,0,0} so that we can have other "special-case" requests in the future.
(style) spaces around that '=' (typo) "to" is not needed here
Presumably this CERROR() is for debugging only? There may be reason the client cannot cancel these locks (eg. it just cancelled locks before this RPC was received).
Should this send the reply *before* shrinking the LRU (to avoid RPC timeout and resend), or are the cancelled locks added into this reply RPC?
(typo) across (style) comment part: @params ...
(minor) it isn't clear that the number of clients holding locks is needed? Clients holding no (or few) locks do not need to be sent any callback. The important thing is the ratio of each export's locks out of the total number of granted locks.
Selecting users to return locks is a bit tricky, a threshold derived from `total_to_cancel / total_count` should be picked with caution, - If small, then it takes a heavy user (20,000 locks) much more iterations to return all its locks compared with a moderate user(1,000 locks?), is that fair? - If large, then in a environment where locks are distributed relatively evenly, do we still be able to collect back enough locks?
Yes, in the second patch:https://review.whamcloud.com/c/fs/lustre-release/+/62146/2 It will send more information about server lock load such as total granted lock, total granted lock of this NS, memory pressure (in Percent: ldlm_lock_threshold * 100 / ldlm_lock_limit), and minimal lock cancel count and whether it is a force... After receive these information, the client can decide lock revoking accordingly.
Maybe I'm misunderstanding here, but this seems straight forward to me. For example, if the server has 1M locks and it wants to cancel 50k cancels/1M locks = 5 cancels/100 locks = 5% to reduce memory pressure. There are two cases (but they work out the same): If the lock allocation is relatively uniform (e.g. there are 1000 clients and they each have 1000 locks), then the same number of locks will be requested to be cancelled from each client (i.e. 1000 locks/client * 5% = 50 locks/client). If the lock allocation is non-uniform (e.g. 1000 clients, but 100 clients have 9100 locks each = 910k locks, and 900 clients have 100 locks each = 90k locks) then the 100 clients with a larger fraction of the total locks will also be asked to cancel more locks (i.e. 910k * 5% = 45 or 46) and the 900 clients with fewer locks will be asked to cancel a smaller number (i.e. 100 * 5% = 5 locks). If it gets to the extreme where the server needs to cancel locks, but quantization makes the number of locks to cancel below 1 for all of the clients (e.g. cancel 500 locks from 1000 clients that all have the same number of locks), then either the server picks the clients with the most locks to cancel 1 each, or randomly picks 500 clients to cancel one, and then next time the other 500 clients will have 1 more lock and should be selected to cancel one lock.
I was taking some extreme case here, say, there might be 10 clients having 90k locks each, in the case of 5% cancel rate, per client needs to return 4.5k locks. Here we capped the *count* to *remains*(512 at most) per round, which means it needs 4.5k/512 ~= 9 iterations to have heavy clients return all their locks back, on the contrary, it takes clients with fewer locks 1~2 rounds to return their locks. That's why I mention `unfair` here. Put it another way, the more locks a client holds, the less locks he/she may need to return in the end. Besides, RPC here is `no_delay`, it might cause traffic spike?
I think yes, there should be some small number of locks left on the client? Otherwise, the client locks may be thrashing on a few locks that the client needs (eg. user home directory), even though some other client may have thousands of locks.
LU-19264 ldlm: server notify lock clients to reclaim LRU locks In large-scale Lustre cluster with a number of clients, the server side lock namespace must manage all locks granted to clients. And these locks are cached in server side memory. Due to the limited server memory capacity, the server must reclaim locks when it is under memory pressure. However, when lock reclaim is triggered, the LRU or frequency of locks cached on the server lock namespace may not reflect recent used accurately. This can lead to premature reclaim of locks that are actually still in active use or may be reused in the near future by the client. The lock revocation will drop the cache data or metadata protected by the lock on the client. This may result in unnecessary network traffic, re-reading the data (such as memory resident library files) dropped from cache wrongly and degraded system performance. To solve this problem, this patch implements a mechanism to let the lock server to notify clients, which have much accurate and realtime information about lock usage, to make better decision for lock reclaim on client side. Signed-off-by: Yingjin Qian <qian@ddn.com> Change-Id: I55a7d5876e5dfbdfbfc84d073e86dc0156f886cd
LU-19743 tests: Allow multiple FID formats on debugfs tests Since new debugfs FID output will not output brace in the future, related tests must support both DFID and DFID_NOBRACE format FIDs for test interop compatibility. Lustre-change: https://review.whamcloud.com/63195 Lustre-commit: e1f272796c51f5335b03f5d60a626ef90ec4d1d7 Test-Parameters: trivial Signed-off-by: Aurélien Cedeyn <acedeyn@ddn.com> Change-Id: I9c78b051091307eba4760d3220c0caf3d0056efa Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com> Reviewed-by: Alex Deiter <adeiter@ddn.com> Reviewed-by: Li Dongyang <dongyangli@ddn.com>
| unique failing test | history |
|---|---|
| replay-single1@zfs+DNE:test_29 | seen in 1 other review |
| replay-single1@zfs+DNE:test_30 | seen in 1 other review |
| replay-single1@zfs+DNE:test_31 | seen in 1 other review |
| replay-single1@zfs+DNE:test_32 | seen in 1 other review |
| replay-single1@zfs+DNE:test_33a | seen in 1 other review |
| replay-single2@zfs:test_132a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-19193 osd-zfs: fix blocksize prediction 1) non-zero offsets shouldn't this logic to grow blocksize 2) appending writes should increase blocksize larger than the chunk being written and enable sanity/312 back Lustre-change: https://review.whamcloud.com/61103 Lustre-commit: d3dde0b63b65fc2a9fbd68534421f7bbb8ba8936 Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Change-Id: Iada44aba64ea07862421f7ee452672794732ee32 Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-19539 osd-zfs: use osd_dmu_write() wrapper for xattrs When setting xattrs use the osd_dmu_write() wrapper which internally calls the _by_dnode() version of dmu_write(). This both avoids an extra lookup and hold on the dnode. Furthermore, it removes the osd-zfs dependency on the dmu_write() symbol which will add an flags argument in OpenZFS 2.5. Lustre-change: https://review.whamcloud.com/62101 Lustre-commit: 84f5994257b16910849f4ab3b27b3c49a45b1adc Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Change-Id: I30ba7abbcf0eefb3dee01b4061baaace2d971842 Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-by: Tony Hutter <hutter2@llnl.gov>
LU-19435 osd-zfs: support for 2.4 zfs commit zfs-2.3.99-340-g734eba251 Wire O_DIRECT also to Uncached I/O (#17218) dmu_write_by_dnode() now has flags argument dmu_assign_arcbuf_by_dbuf() now has flags argument zfs commit zfs-2.3.99-373-gac2e57952 dmu_tx: make DMU_TX_* flags an enum DMU_TX_WAIT may be an enum Lustre-change: https://review.whamcloud.com/61589 Lustre-commit: bc3fe9e4bb1524ac64eff5529b4ba15675cf2e3b Test-Parameters: trivial Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I2b1694b39f4d4a83a178d70937a2e48019ed42a0 Reviewed-by: Jian Yu <yujian@whamcloud.com> Reviewed-by: Akash B <akash-b@hpe.com>
LU-13371 api: add llapi_getdirstripe function Add a new llapi_getdirstripe() function to the Lustre API to retrieve directory striping information. This function is similar to llapi_file_get_stripe() but specifically for directory striping information. This function will make it easier for applications to retrieve directory striping information without having to directly use the LL_IOC_LMV_GETSTRIPE ioctl. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I41da7e83c6d67bc182d008524d08a9088405cc05
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 6 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 3 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 6 other reviews |
| sanity2@zfs:test_119e | seen in 15 other reviews |
| sanity2@zfs:test_119f | seen in 14 other reviews |
| sanity2@zfs:test_119g | seen in 14 other reviews |
| sanity2@zfs:test_119h | seen in 14 other reviews |
| sanity2@zfs:test_119p | seen in 3 other reviews |
| sanity2@zfs:test_119q | seen in 3 other reviews |
| sanity2@zfs:test_398o | seen in 19 other reviews |
| sanity2@zfs:test_398s | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 3 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 4 other reviews |
| sanity-hsm@zfs:test_607b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 osc: rename osc_prep_transfer_page osc_transfer_page_init is a better match for osc_page_init. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id12eec932eb4aab00139f08532b218c12e6ffba4
LU-10911 utils: fix coverity defects in EC resync/verify
Address three Coverity defects introduced by the EC parity
resync/verify support:
- is_parity_mirror() discarded the llapi_layout_comp_iterate()
return value. Check it and treat an iteration error as "not a
parity mirror", matching every other comp_iterate caller in
lfs.c (CHECKED_RETURN).
- llapi_ec_resync_or_verify_comp() passed sysconf(_SC_PAGESIZE),
which may return -1, directly as the posix_memalign() alignment
argument. Store it in a checked variable first, mirroring the
existing idiom in this file (NEGATIVE_RETURNS).
- llapi_ec_check_comp_match() evaluated (esc_n0 + esc_n1) *
llc_cstripe_count as a signed int that was then sign-extended to
the uint64_t stripe count. Cast to uint64_t to match the existing
idiom (SIGN_EXTENSION).
Each flagged condition is unreachable with valid layouts, so there is
no functional change; the fixes silence the defects and keep the EC
code consistent with the surrounding idioms.
Assisted-by: ClaudeCode:Opus-4.8 llm_code_and_review_tools
CoverityID: 504254 ("Unchecked return value")
CoverityID: 504243 ("Argument cannot be negative")
CoverityID: 504237 ("Suspicious implicit sign extension")
Fixes: dfb93e4978b0 ("LU-19548 lfs: mirror split -d for parity mirrors")
Fixes: 03191be6f459 ("LU-12668 ec: add support to resync stale ec comps")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia8ad92d5932be270bf782fe9a4aae8313eb83c03
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/66898
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
LU-16489 obdclass: handle LLOG_SKIP_PLAIN on client builds
Commit 2ed9e1bcf762 ("LU-14291 obdclass: unwind server code out of
llog cat code") moved the LLOG_SKIP_PLAIN handling in
llog_cat_process_cb() into the CONFIG_LUSTRE_FS_SERVER only
section. On client builds the conversion to zero is now compiled
out, so the LLOG_SKIP_PLAIN value returned by a plain llog
processing callback, e.g. chlg_read_cat_process_cb() skipping a
fully consumed plain llog via llog_is_plain_skipable(), is passed
up to the catalog level llog_process_thread(), which stops
processing the whole catalog instead of continuing with the next
plain llog. llog_cat_process() then returns the positive
LLOG_SKIP_PLAIN value, which callers like chlg_load() do not treat
as an error.
As a result a client side changelog reader that starts at a record
beyond the end of the first plain llog of the changelog catalog
silently receives no records at all. lustre_rsync then reports
"Changelog records consumed: 0" and misses all pending changes,
causing the frequent lustre-rsync-test test_2c "Failure in
replication; differences found" failures seen since the patch
landed.
Move the LLOG_SKIP_PLAIN check out of the server-only section
again; skipping a plain llog is client functionality. The
LLOG_DEL_PLAIN / LLOG_DEL_RECORD handling stays server-only since
llog_cat_cleanup() modifies the catalog.
Assisted-by: ClaudeCode:Fable-5 llm_code_and_review_tools
Test-Parameters: testlist=lustre-rsync-test env=ONLY=2c,ONLY_REPEAT=100
Test-Parameters: testlist=lustre-rsync-test env=ONLY=2c,ONLY_REPEAT=100 fstype=zfs
Fixes: 2ed9e1bcf762 ("LU-14291 obdclass: unwind server code out of llog cat code")
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I124d7ecb1fcd8e642b0d960fd708c9728e0563dd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/67221
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Timothy Day <timday@thelustrecollective.com>
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>