| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-2 retesting | RHEL 10.1 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-zfs-part-5 failed 2× crashed | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
Going forward, all of these lines should be replaced with a label: Assisted-by: ClaudeCode:MODEL_VERSION [TOOLNAME ...] https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution
Why is this for `fortestonly`? Should this be consolidated with another patch, or that annotation be removed?
Probably it was generated initially via AI, and nobody removed that designation as the patch was being updated?
(typo) The third path is `lov_io_lseek_end()`, which is the `.cio_end` entry in `lov_io_ops[CIT_LSEEK]` and runs from `cl_io_end()`, not from unlock. `lov_io_unlock()` is a separate op. Should this read "during sub-IO end"?
[Marc Bot] (style) This attribution line was flagged on patchset 34 and is still unresolved: it should use the `Assisted-by:` label format described at https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution instead of the free-form line.
"three independent paths" does not seem to hold for the third one.
The LSEEK sub-lock enqueue happens in cl_lockset_lock(), which cl_io_lock() runs *after* every cio_lock(), so the stripe is already marked by the time lov_io_call(cl_io_start) runs and the sub-IO is skipped there. A sub-IO that never started still has:
sub_io->ci_result = 0 /* lov_io_sub_init() */
sub_io->u.ci_lseek.ls_result = -ENXIO /* inherited from the parent in lov_io_sub_inherit(); ll_lseek() seeds it */
lov_io_lseek_end() already ignores both (`ci_result == 0` is a no-op, `sub_off == -ENXIO` hits the existing continue), so there is nothing for the third hunk to catch.
Also, ci_result propagation happens in .cio_end (lov_io_lseek_end), not during unlock.
(style) This was raised on an earlier patchset and the line is unchanged: tool attribution should use the `Assisted-by:` trailer format documented at https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution rather than a free-form sentence.
This guard sits in `lov_io_call()`, which is the shared dispatcher for four different ops:
lov_io_lock() -> lov_io_call(cl_io_lock)
lov_io_start() -> lov_io_call(cl_io_start)
lov_io_iter_fini() -> lov_io_call(lov_io_iter_fini_wrapper)
lov_io_unlock() -> lov_io_call(lov_io_unlock_wrapper)
All four are registered for CIT_LSEEK, so a stripe marked LSS_READ_ERR also skips `cl_io_unlock()` and `cl_io_iter_fini()` on its sub-IO, not just `cl_io_start()`. Its `ci_state` then goes CIS_LOCKED -> CIS_IO_FINISHED (set by `lov_io_end_wrapper()` in `lov_io_lseek_end()`) -> CIS_FINI, never passing through CIS_UNLOCKED/CIS_IT_ENDED.
Nothing leaks today because `osc_io_ops[CIT_LSEEK]` registers only cio_start/cio_end/cio_fini and the LSEEK DLM lock is taken on the top IO by `vvp_io_lseek_lock()`. But the intent is only to skip the data-fetch pass -- would putting the check in `lov_io_start()` (or keying it on `iofunc == cl_io_start`) keep the cleanup passes balanced?
(style) This isn't a bug, but `str` reads like a string; the rest of this file spells it `stripe` (see `lov_io_lseek_end()` a few hundred lines down, which uses `index`/`stripe` for the same two values). Worth matching if the patch is refreshed.
[Marc Bot] (defect) Skipping the stripe treats its extents as holes, but on an EC file that data is still readable via parity reconstruction. If the only data between ls_start and the next healthy-stripe data lives on the degraded stripe, SEEK_DATA returns the later offset, or -ENXIO if none, so sparse-aware tools (cp, tar) silently drop data that read() would return. Also, if every sub-IO in lis_active is skipped (e.g. single-stripe data component), offset stays -ENXIO and SEEK_HOLE fails with -ENXIO even though ls_start < file size, which breaks the SEEK_HOLE contract of a virtual hole at EOF. Would it be safer to treat a degraded stripe's covered range as data instead of skipping it, or to return an error rather than a misleading data map? The commit message claim that the remaining healthy stripes provide valid seek results does not hold in these cases.
(defect) Skipping cl_io_start() for the degraded stripe means its allocation map never contributes to the seek result, and lseek does not reconstruct anything from parity the way CIT_EC_RD does. So the answer is not "the remaining healthy stripes are sufficient" -- it is an answer computed from an incomplete extent map.
Concrete case, 4+2 EC, one data OST deactivated, data written only in the range that maps to that stripe:
lseek(fd, 0, SEEK_DATA)
-> degraded sub skipped, others report -ENXIO
-> offset stays -ENXIO, lseek fails
and with data further out on a healthy stripe it returns that later offset instead. Sparse-aware copies (cp --sparse, tar, rsync) would silently drop the bytes that a plain read() still returns via parity.
SEEK_HOLE has the mirror problem: if every sub-IO covering ls_start is skipped, `offset` stays -ENXIO and ll_lseek() returns -ENXIO for an offset below i_size, which breaks the "there is always a virtual hole at EOF" contract.
Is returning an error preferable to returning a wrong offset here? Alternatively, could the degraded stripe's range be reported as data (conservative) rather than dropped?
+1; error should be preferable compared with a wrong offset
(typo) This comment uses a non-ASCII em dash; the rest of the tree is plain ASCII. Plain "-" or "--" instead.
This hunk looks like it has no effect. Any stripe marked LSS_READ_ERR was already skipped by the new check in lov_io_call(), so its sub-IO never reached cl_io_start(): ci_result is still 0 from lov_io_sub_init(), and ls_result is still the -ENXIO that lov_io_sub_inherit() copied from the parent. The `if (io->ci_result == 0)` assignment and the `sub_off == -ENXIO` continue below both already handle that. Is there a path where a sub-IO is marked LSS_READ_ERR but still ran? If not, dropping this hunk would keep the two skip conditions from having to stay in sync.
LU-12668 lov: handle ESHUTDOWN for LSEEK on EC files When an OST hosting a data stripe of an EC file is deactivated, SEEK_DATA/SEEK_HOLE fails with ESHUTDOWN because the error propagates through the LOV layer during sub-lock enqueue. Fix this by marking the degraded stripe LSS_READ_ERR in lov_lock_enqueue (same mechanism used by CIT_EC_RD for parity recovery), then skipping those stripes in lov_io_call and lov_io_lseek_end. This avoids ESHUTDOWN leaking through three independent paths: sub-lock enqueue, sub-IO function dispatch, and sub-IO ci_result propagation during unlock. Add lov_lsm_has_parity() helper to check if any layout entry has parity, needed because CIT_LSEEK locks the data component while parity lives in a separate entry. This fixes lfs mirror verify failing on EC files when any data OST is deactivated. Generated with Claude Code + Tools Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5cce4e0ea51c68b0c6fda1d83b694af19cad57bd
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-ubuntu retesting | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
That seems like a pretty big hammer, basically breaking AIO completely for EC files, rather than just the recovery path? Or is the comment wrong and this is triggered only for EC recovery when `-EIOCBQUEUED` is returned? Would it be better to push **all** EC recovery to a workqueue instead of keeping it directly in the IO path, then it can fire the AIO completion when the reconstruction is complete, and synchronous readers would wait on the completion?
Andreas, yes, correct. This is essentially making AIO synchronous for all reads on EC files because we can't tell at submission time if we need reconstruction. So, this patch is just a stopgap to allow AIO to work for degraded reads, albeit with collateral damage. I'd agree that a workqueue is the better long-term approach, however, I'm not familiar enough with the code path at the minute here to gauge the effort required. So, I'd keep this patch as a short-term fix for AIO for degraded reads, and open a ticket so we address this properly with a workqueue. Do you agree? If yes, I open the ticket
(minor) aio_ec_sync deliberately survives the goto restart at the end of the function, and it has to. cda_no_aio_complete and cda_creator_free stay set on the cl_dio_aio across restarts, so a later pass that ends with anything other than -EIOCBQUEUED (cl_io_rw_init() failing, or cl_io_loop() returning a hard error) still needs both !is_aio || aio_ec_sync tests to be true. If it were cleared at restart, __cl_sync_io_note() would skip the free because creator_free is 1, and the creator branch would skip it too, leaking the cl_dio_aio and its cda_obj reference. This isn't a bug as written, but the coupling between a stack bool and two flags on a heap struct isn't obvious. Could the comment above say the flag is intentionally sticky, so it doesn't get reset in a later cleanup?
on refresh
LU-12669 llite: make AIO reads on EC file synchronous For sync DIO, EC recovery on read failure runs inline in ll_file_io_generic after cl_sync_io_wait_recycle. AIO has the same recovery requirement but cannot run recovery from the sub-DIO completion path -- ll_file_io_generic returns -EIOCBQUEUED to the VFS before the BRW completion fires, so the syscall context recovery needs (the lu_env, the user's iov_iter, the range lock) is gone by the time the read error is known. Rather than building a workqueue-based async recovery path, make AIO reads on EC files behave as sync DIO: after cl_io_loop, if an AIO read on an EC layout (io->ci_cross_ec, set by lov_io_mirror_init during cl_io_rw_init) got -EIOCBQUEUED, set cda_no_aio_complete and cda_creator_free to disarm the async ki_complete path, then drop into the same wait + restart code sync DIO uses. The VFS calls ki_complete itself when we return. The flag flip is race-free because the submission ref on cda_sync is still held -- end_io cannot fire until we drop it in cl_sync_io_wait_recycle. This disables async semantics for every AIO read on an EC file, not only those that need recovery -- we cannot tell at submission time whether reconstruction will be needed, so the sync conversion fires unconditionally on -EIOCBQUEUED for an EC layout. As a side effect, the range_lock taken by ll_file_io_generic is now held until all sub-DIOs have drained, instead of being dropped while they are still in flight. Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Assisted-by: Opus:4.8 llm_code_and_review_tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I394a3610e33b29ead8f5adb52dfa21db6b721944
| unique failing test | history |
|---|---|
| sanity-lfsck@ldiskfs+DNE:test_18g | seen in 13 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18h | seen in 13 other reviews |
(minor) LU-19536 in JIRA is titled "fault_in_iov_iter_readable() with a spinlock held", and the other changes landed under it (DIO pool ENOMEM drain/retry, DIO buffer double-free, wait queue for DIO copy sync) are all in the unaligned-DIO copy machinery. Grant/dirty accounting for DIO writes reads like a separate topic. Would a dedicated ticket be a better home, or is LU-19536 intended as an umbrella for all of the DIO work?
(minor) The comment doesn't quite match what happens. osc_wake_cache_waiters() is only `wake_up(&cli->cl_cache_waiters)`, so nothing re-enters this function - there is no recursion. The actual hazard is that osc_enter_cache() evaluates this as the condition of wait_event_idle_exclusive_timeout_cmd() on that same queue, so the wake would land on the evaluating task itself (and osc_enter_cache() already wakes the queue once it succeeds). Worth rewording so the rationale for the no-wake variant is accurate.
(minor) This bakes in an assumption that any `cl_dio_pages` reaching the OSC is embedded in a `cl_sub_dio`, which isn't part of the `cio_dio_submit`/osc_queue_dio_pages() contract.
A few lines down the same sdio is already derived from the page anchor:
ext->oe_csd = anchor->csi_dio_aio;
Since `is_aio` is only needed inside the `!ext->oe_rw` branch, could it just read `ext->oe_csd->csd_ll_aio->cda_is_aio` there and drop the container_of()?
Please do this improvement.
LU-19536 osc: enforce dirty limits for regular DIO writes
osc_queue_dio_pages() checked only server grant before queuing a
write. For regular non-AIO DIO, parallel submission could therefore
bypass the per-OSC max_dirty_mb and global obd_max_dirty_pages limits.
Add osc_reserve_dio_grant() to validate the local dirty limit,
atomically reserve global dirty pages, and reserve server grant under
the client lock. Check dirty pressure first so a simultaneous grant
shortage cannot hide the dirty limit.
When a regular DIO extent cannot enter dirty accounting, submit it
synchronously and wait for that extent before forming more unaccounted
RPCs. Preserve existing no-grant and true AIO paths. Keep buffered
osc_enter_cache_try() accounting unchanged.
Remove the obsolete DIO grant branch from osc_queue_sync_pages().
Since LU-13814, DIO requests use osc_queue_dio_pages(). Rename
__osc_unreserve_grant() to osc_unreserve_grant_no_wake() to document
the no-wakeup rollback needed by waitqueue callers. Add __must_hold
annotations to document the locking requirements of grant and cache
helpers.
Add sanity test_398v using one exact OSC, warming its grant, and
pausing OST bulk completion. Verify that four DIO RPCs do not all
overlap at the dirty limit. Check final dirty counters and compare the
client and server grant changes around the tested DIO.
Fixes: 8efbad8ff4ed ("LU-13814 osc: add osc_queue_dio_pages")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I115232216ae77740f0a779994e8a020eedeebfef
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 crashed | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. %% THIS TEST SESSION CRASHED %% | session |
| custom-1002 crashed | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-subtest-change failed 30× | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-zfs-subtest-change failed 29× | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
(minor) The range here starts at 49b, but 49a "test concurrent reads during EC recovery" is added by this patch too. Should it read 49a-50b? Every other test added is accounted for by one of the ranges.
(minor) Two hunks are not accounted for by any of the ranges above: the new `[[ "$SLOW" == "no" ]] && EXCEPT_SLOW="74a 75b"` line, and the `head -n1` fix in `enable_ec()` for the multi-mount `get_param` output. The `EXCEPT_SLOW` one also changes an existing test: 74a is not added by this patch and is not named in any of the range lists, so a reader can't tell from the message that it is being moved onto the SLOW list. Worth a sentence for each.
(defect) With 3 of 4 data stripes unreadable and pcount=2, -EIO is the only correct outcome; returning reconstructed-but-wrong bytes is silent corruption. Treating it as an acceptable pass means a real reconstruction bug in this path would go unnoticed.
48a, 48c and 64b in this same patch take the opposite position ("handing back silently wrong data would be a corruption bug, so assert against it"). Should 52b assert rc != 0 instead?
(minor) These eight files (2M..16M) are never removed. The same applies to the extra files in 71b/71c/71d, 73b's .pure, 73c's .zero and the directories in 75e/75f; 73e is the only new test that registers a cleanup. Added up, the new tests write on the order of 1.5G and leave most of it in place for the rest of the run, which on a small test filesystem can push later subtests into ENOSPC. A `stack_trap "rm -f ..."` next to each creation would keep the footprint bounded.
(suggestion) Both of these are parked against the feature ticket itself. 53a in particular is described as intermittently reconstructing wrong data, which is a silent-corruption symptom rather than a test problem, and 41d is an OSC retry loop that never terminates. Would separate LU tickets referenced here keep them from being lost when LU-12668 is closed? The convention elsewhere in this file (12a -> LU-19631, 5b/12b -> LU-20435) is one ticket per known failure.
(minor) A fresh `stack_trap` is pushed on every call, including repeat calls for the same index, so loop-driven tests accumulate identical cleanup entries: 75b registers 50, 74a 20, and 58a/63b/68b one per pair. Each entry re-runs `ec_apply_fault`, which is a `do_nodes` to every OSS plus a `cancel_lru_locks osc`, so teardown does that work dozens of times over. Registering the trap only when the bit was not already set would make it one entry per OST.
(minor) When no stripe list is given this walks every data stripe and returns the first parity-free one, which can be a stripe that holds no data. For a sub-raid-set file that is the vacuous-pass mode this helper is documented as preventing: 42a writes 512K into a 1M-stripe 4+2 layout, so only stripe 0 has data. If stripe 0's OST happens to double as parity, the fault is armed on stripe 1's (empty) object and the checksum comparison succeeds without reconstructing anything. `ec_data_stripe_osts()` handles the analogous case by calling `skip_env`. Would returning non-zero (so `ec_start_read_fault()` skips) be safer than falling through to a later stripe?
(minor) 44c and 44a are the same test - same layout, same `ec_start_all_reads_fail`, same EIO check, same clear-and-reread - differing only in the error strings. 43c is that body minus the reread. Could these collapse into one? While here, 43c's description says "too many OST failures (3+ OSTs)" but `ec_start_all_reads_fail` uses `fail_val=0`, which fails every OST, not three.
(minor) The comment says "an OST that is a safe data OST for all files", but only `$f2` is classified. `$f1` (2+1, 3 objects) and `$f3` (2+2, 4 objects) get whatever the allocator gave them, so on an 8-OST config the victim frequently holds none of their objects and the `$s1`/`$s3` comparisons pass without any recovery running. 71a, 71c, 71d and 75e/75f have the same shape (classify one file, assert on all of them). That is defensible for a batch test, but here the comment claims something stronger than the code does.
(defect) `safe_osts` here still comes from the `ec_classify_osts $tf` above, but the SEEK_DATA/SEEK_HOLE checks below run on `$tfs`, which was created separately and gets its own object placement from the allocator.
So the OST taken out need not hold any of `$tfs`'s stripes, and on a run where it doesn't, the two `lseek_test` assertions execute against a fully healthy file.
73c already documents and avoids exactly this ("safe_osts still describes $tf; $tf2 has its own object placement") by calling `ec_data_stripe_osts $tf2 0` first. Should 75d do the same for `$tfs`?
Related: the comment says the seek "crosses the degraded stripe", but the data lives at 5M, i.e. stripe 1 with `-c 4 -S 1M`, while `safe_osts[0]` is just the lowest-numbered data OST.
(minor) The negative-index handling, and the "Index -1 is the last stripe" note in the header comment, appear to be unreachable: all eleven callers pass 0..4. Worth dropping the branch and the doc line unless a caller is coming.
(style) The suite convention is a `#define` comment naming the fault right above the line that arms it, so a reader does not have to look up the bare hex. `ec_start_all_reads_fail()` and 41d both do this; this call site and the one in `ec_apply_fault()` do not.
#define OBD_FAIL_OST_BRW_READ_BULK 0x20f
ec_ost_fail_loc 0x20f $(( 0x10000 | mask ))
The value itself is right (obd_support.h has 0x20f), it is only the annotation that is missing.
LU-12668 tests: add EC recovery tests Add sanity-ec coverage for erasure-coding recovery. Each test writes an EC file, resyncs parity, fails one or more OSTs, and verifies the client reconstructs the data from parity (CIT_EC_RD) against the pre-failure checksum. Failure is injected with OBD_FAIL_OST_BRW_READ_BULK so the OSC import stays active and only bulk reads fail, which drives genuine parity reconstruction. The fault is set on the OSS nodes, where tgt_brw_read() evaluates it, and osc.*.resend_count is dropped to 1 for the duration so the injected -EIO reaches the LOV layer instead of being absorbed by an OSC resend. Victims are chosen by stripe rather than by OST index. ec_pick_data_ost() walks a file's data stripes in order and takes the first whose OST does not also carry parity: a file smaller than one raid set holds data on stripe 0 alone, so picking the lowest OST index instead would arm the fault on an object the read never reaches and the test would pass without exercising recovery. ec_check_fault_index() skips when a target OST index is >= 16, which cfs_fail_index() cannot express in its 16-bit fail_val bitmask. ec_mirror_victims() fails one parity-free OST in every data mirror, since a file with more than one data copy would otherwise answer the read from an intact mirror rather than reconstructing anything. Reads that check a sub-range cancel their locks first: a range re-read after a whole-file read is otherwise served from the page cache, issues no BRW RPC, and so never reaches the injected fault. Geometry and I/O patterns (40b-44c): - 2+1, 2+2 and 4+1 EC; partial, offset, mmap, direct and async reads; single, maximum and progressive OST failure; graceful failure when too many OSTs are gone Failure placement and layout (45a-48c): - consecutive, non-consecutive, boundary and parity-only OST failures; 64K, 256K and 4M stripe sizes; multiple EC and mixed EC/non-EC PFL components; stale and partially stale parity Concurrency and multi-mount (49a-50b): - concurrent readers over a file with two failed data OSTs; background writes, mirror resync and OST reactivation during recovery; both mounts reading the same EC file Core recovery (51a-53a): - EOF boundary recovery at RAID set / stripe set boundaries; degradation limits and mixed parity+data failure; recovery at non-zero read offsets Layout patterns (55a): - file-size boundaries (1 byte .. multi-stripe) Edge cases (58a-62d): - parity_used combinations and stripe rotation; sparse files with holes; recovery after truncate; sub-stripe files; append writes Multi-target (63a-66d): - OST failure cycling and multi-mount coordination; 3-4 component PFL with per-component EC geometry; multiple EC mirror pairs Write patterns (67a-71d): - writes to healthy stripes during degraded mode and overwrite cycles; varied geometries; O_DIRECT writes; partial and mid-file overwrites; batch recovery of many files Stress and admin (73a-73e, 74a, 75a-75f): - random reads, fallocate, truncate-extend, O_APPEND, and large (128M) file recovery; reads racing with OST deactivation; stat/getattr, stress loops, lfs mirror verify, directory-inherited layout, and stripe rotation Tests 41d and 53a are added but listed in always_except. A degraded mmap read never completes: the OSC alternates between "too many resent retries" and osc_brw_redo_request() forever, so -EIO never reaches the LOV layer and CIT_FAULT never switches to CIT_EC_RD. A recovery read at a non-zero offset intermittently reconstructs wrong data. Both stay off until those are fixed. Test 65c and the sparse half of 75d are skipped on ZFS. Both build a sparse file and resync it, which needs lseek to report the holes so resync knows which stripe sets to skip, and ZFS does not report them reliably for dirty data. Test 12b describes the same problem. Assisted-by: ClaudeCode:opus llm_code_and_review_tools Test-Parameters: trivial testlist=sanity-ec ostcount=8 Test-Parameters: trivial testlist=sanity-ec ostcount=8 fstype=zfs Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Max Dilger <mdilger@whamcloud.com> Change-Id: I5a06cd166487e0bff7bfdb6a39414af3f12c4326
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: recovery-small, lustre-rsync-test. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 7 tests. 2 tests failed: recovery-small, lustre-rsync-test. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: recovery-small, lustre-rsync-test. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
How is this better than just incrementing rpcs_in_flight under cl_loi_list_lock and decrementing it on failure? Does this mean we are now double counting some RPCs until the new counter is decremented?
In the upstream kernel this is expressed as:
Reported-by:
Or:
Suggested-by:
why remove this comment?
(style)
for ((i = 0; i < 16; i++)); do
(style) can this just grep for `5:` in the output?
$LCTL get_param -n osc.$osc.rpc_stats | grep "^5:" &&
error "found more than 4 RPCs in flight" || true
I guess this needs to only follow the `rpcs in flight:` section, so possibly:
```
$LCTL get_param -n osc.$osc.rpc_stats | grep -A 8 "rpcs in flight:" |
grep "^[5-9]:" && error ...`
```
This also detects the case where 5 RPCs-in-fight are somehow all skipped...
LU-19755 osc: fix race in max_rpcs_in_flight check When multiple ptlrpcd threads process RPCs concurrently, they can all pass the osc_max_rpc_in_flight() check before any of them has incremented the in-flight counter. This happens because the check is done under cl_loi_list_lock but the counter is incremented later in osc_send_*_rpc() after the lock is released. Fix this by adding a cl_pending_in_flight counter that reserves a slot while still holding the lock. This counter is included in rpcs_in_flight() so concurrent threads see the reservation and wait appropriately. Add OBD_FAIL_OSC_DELAY_RPC to allow testing this fix by injecting a delay in the race window, and sanity test 55d to verify the limit is respected under concurrency. Thanks to Jinshan Xiong of Google for reporting this issue and suggesting the fix. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8ceacf3d9040d94fc89ab54a39bd98a4fb35ae1d
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 crashed | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
LU-19956 osc: fix race from direct ops_transfer_pinned clear The ops_transfer_pinned flag in osc_page is paired with a cl_page reference -- the flag and the ref must always be managed together through the osc_page_transfer_get/put() accessors. osc_completion() violated this by clearing the flag directly and dropping the ref with a separate cl_page_put(). This decoupled the flag from the ref, opening a race on weakly-ordered architectures. The initial fix (complete first, then transfer_put) introduced a re-submission crash: once cl_page_complete() transitions the page to CPS_CACHED, a concurrent write can re-submit it via osc_page_cache_add -> osc_page_transfer_get, which asserts the pin is clear. If the old completion has not yet called transfer_put, the assert fires. Fix osc_completion to release the transfer pin BEFORE cl_page_complete(). While still in CPS_PAGEOUT, the state machine prevents any other transition, so the transfer_put is race-free. Take a temporary cl_page ref first: transfer_put drops the pin's ref (cp_ref 2->1), and cl_page_complete's end_page_writeback can make the page reclaimable on another CPU. A TLA+ formal model (formal_models/clio/TransferPin.tla) with the NoPinWhileCached invariant verifies this fix catches the re-submission crash in all three buggy variants (original, LASSERT-in-delete, complete-first). Generated with Claude Code + Tools Test-Parameters: testlist=sanity env=ONLY=80a Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ib8416c753e13fe6eb8e11790b63687fbc2c8a26d
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_63c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
(suggestion) This is the last DIO assumption left in lov_io_submit(), and the patch's own premise says it can no longer hold: every page that reaches here now comes from a cl_page_list, and the only CPT_TRANSIENT pages in the tree are built by ll_direct_rw_pages() into a cl_dio_pages, which goes to lov_dio_submit() instead. So an empty page arriving here would be CPT_CACHEABLE and this would LBUG.
In practice the branch looks unreachable too - lov_page_init_empty() does SetPageUptodate(), and all four submit paths (ll_readpage(), ll_read_ahead_page(), ll_prepare_partial_page(), ll_io_zero_page()) skip uptodate pages before queueing. If so the whole block is dead and could go with the rest of the DIO leftovers.
The comment is stale either way: cl_page_prep() here was removed by 169f076ae0 ("LU-13814 clio: remove cl_page_prep for transients").
(minor) Not a bug, but with `dio` dropped the only remaining value in this message is `ext->oe_srvlock`, and the enclosing condition is `dlmlock == NULL && !ext->oe_srvlock` - so it always prints `srvlock: 0`. If the patch is refreshed, either drop that field or print something that varies, e.g. `oe_rw`.
LU-19536 osc: remove dead DIO handling from sync path Since LU-13814 routed DIO through cio_dio_submit() and cl_dio_pages, transient DIO pages no longer reach lov_io_submit(), osc_io_submit(), or osc_queue_sync_pages(). The DIO-specific branches and setup left in the ordinary page-list path are therefore dead. Remove the stale DIO handling from the LOV and OSC submit paths, including OBD_BRW_NOCACHE setup, the oe_dio assignment, and the sync_io anchor setup. The ordinary LOV path now always groups pages by stripe, while osc_io_submit() always prepares its pages. OBD_BRW_NOCACHE is set only by osc_dio_submit() before it calls osc_queue_dio_pages(). Since oe_dio is always false in the sync path, simplify extent priority handling and omit oe_dio from the NOLCK diagnostic. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com> Change-Id: I63711a7f3bc3699680e1c09d3349291ccebb75ec
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
A few hunks aren't covered by the description, so it's hard to tell which are intentional:
- obd.h reorders op_code/op_xvalid/op_bias/op_cli_flags (a packing cleanup, unrelated to stale merges);
- obd.h converts `bool op_new_layout` into a bitfield, which touches a field this feature never uses;
- llite/file.c changes the LL_IOC_LEASE unlock error paths from `rc` to `rc2` and folds it in at the end.
That last one is a real user-visible fix, not a cleanup: today `GOTO(out_lease_close, rc = -EINVAL)` (and -EFAULT/-EBADF/-EPERM/-ENOMEM) is immediately overwritten by `rc = ll_lease_close_intent()`, so LL_LEASE_RESYNC_DONE / LAYOUT_MERGE / LAYOUT_SPLIT return the lease type instead of the error. Would it be better as its own patch with
Fixes: f172b116885 ("LU-10092 llite: Add persistent cache on client")
so it can be reviewed and backported on its own?
This adds a new on-wire field (cd_merge_flags) and a new client-set flag value, so a Test-Parameters: line requesting interop coverage would be useful here, e.g. an older serverversion and an older clientversion run.
LL_LEASE_ALLOW_STALE is a modifier bit but the switch still matches lil_flags exactly, so every future combination has to be enumerated as its own case. Anything unenumerated (LL_LEASE_LAYOUT_SPLIT | LL_LEASE_ALLOW_STALE, or the bit on its own) falls into `default:` and quietly releases the lease with bias = 0, returning the lease type as if it had worked. Would `switch (ioc->lil_flags & ~LL_LEASE_ALLOW_STALE)` plus a check that the modifier is only accepted for MERGE read better?
All of these bool -> 1 are unnecessary and in files not otherwise touched - let's leave them out?
I'm not sure whether coverity would complain about the transform bitwise from/to boolean value kinda of warning, so I made this change.
This isn't a bug that I can trigger today, but now that merge can mark every component of the new mirror stale, should this pass LVF_ALL_STALE like lod_declare_layout_split() does at the equivalent point? That's the flag that turns on the "can not set all stale mirrors" check in lod_parse_striping(), and merge is currently the one stale-producing path that skips it.
There doesn't seem to be any negotiation for this. An MDT that predates the patch never looks at cd_merge_flags - mdt_close_handle_layouts() just calls mo_xattr_set(..., LU_XATTR_MERGE) - so a new client asking for a stale merge against an older server gets the mirror merged with no LCME_FL_STALE set, and rc = 0 back. The result is a mirror that was never written being advertised as up to date, which reads can then be served from. Should this be gated on a new OBD_CONNECT2_* flag, with the client either failing with -EOPNOTSUPP or falling back to the sync path when the server doesn't advertise it?
The declare above passes LU_XATTR_MERGE_STALE, but the execute phase here still passes plain LU_XATTR_MERGE. It works only because lod_declare_layout_merge() builds the merged layout at declare time, and because LU_XATTR_MERGE_STALE would actually break the execute path - lod_xattr_set() has
if (!(fl & LU_XATTR_MERGE))
LASSERT(equi(...));
which would fire for an existing file if MERGE_STALE were passed instead.
Would it be safer to define LU_XATTR_MERGE_STALE as an extra bit used together with LU_XATTR_MERGE rather than in place of it? Then every existing `fl & LU_XATTR_MERGE` test (lod_object.c) and `fl == LU_XATTR_MERGE` test (mdd_xattr_set()) stays correct, and declare/execute can use the same value.
This is the only caller of mirror_extend_layout() and it hardcodes sync=true, so the `if (sync)` branch is always taken and `data->lil_flags |= LL_LEASE_ALLOW_STALE` is never executed. Grepping the tree, LL_LEASE_ALLOW_STALE is only referenced here and in ll_file_unlock_lease(). So as it stands `lfs mirror extend` can't create a stale mirror and the entire mdc/mdt/mdd/lod path added by this patch is unreachable from any shipped tool. Is the intent to add an `lfs mirror extend` option (plus the matching Documentation/man8/lfs-mirror-extend.8 text) in this patch, or is that coming in a follow-on?
check_close_data() is called inside the CHECK_COND_START(CONFIG_LUSTRE_FS_SERVER) block, so the generated assertions in both wiretest.c copies end up under `#ifdef CONFIG_LUSTRE_FS_SERVER`. struct close_data is packed by the client in mdc_close_intent_pack(), so a client-only build never verifies the layout of a struct it puts on the wire. The other client-visible structs (check_swap_layout(), check_hsm_*()) are called outside that block - should this one be too?
LU-18746 flr: allow merge stale mirror This patch makes it possible to merge a stale mirror to an existing file, so that we can append several mirrors w/o writing data to those mirrors first, and after we've finished the merge we'd do the mirror resync later. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: I0d730566ea9b238aeac5d2e27e77c719a1c73308
| unique failing test | history |
|---|---|
| conf-sanity1@ldiskfs+DNE:test_25 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| conf-sanity2@ldiskfs+DNE:test_50a | seen in 1 other review |
| conf-sanity3@ldiskfs+DNE:test_101a | seen in 6 other reviews |
| conf-sanity4@ldiskfs+DNE:test_151a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| conf-sanity1@zfs:test_30a | seen in 3 other reviews |
| conf-sanity2@zfs:test_51 | seen in 2 other reviews |
| recovery-small@ldiskfs+DNE:test_52 | seen in 8 other reviews |
| recovery-small@zfs:test_52 | seen in 6 other reviews |
| replay-dual@ldiskfs+DNE:test_28 | seen in 1 other review |
| sanity2@ldiskfs+DNE:test_123aa | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity3@ldiskfs+DNE:test_398m | seen in 2 other reviews |
| sanity-slow@ldiskfs+DNE:test_255a | seen in 17 other reviews |
| sanity1@zfs:test_56xb | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_123aa | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-quota@ldiskfs+DNE:test_33 | seen in 1 other review |
| sanity-quota@zfs:test_13 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@ldiskfs+DNE:test_18 | seen in 4 other reviews |
| sanity-sec@zfs:test_18 | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_43k | seen in 2 other reviews |
| sanityn@zfs:test_45j | seen in 4 other reviews |
(minor) duplicate code block? I can't see any difference
Doh, "readable" vs. "writeable". I was confused that they were both checking HAVE_FAULT_IN_IOV_ITER_READABLE...
(style) this would be more clear if the same #ifdef was not checked twice:
```
#ifdef HAVE_FAULT_IN_IOV_ITER_READABLE
#define ll_iov_iter_fault_in_readable(iov, bytes) \
fault_in_iov_iter_readable(iov, bytes)
#define ll_iov_iter_fault_in_writeable(iov, bytes) \
fault_in_iov_iter_writeable(iov, bytes)
#else
#define ll_iov_iter_fault_in_readable(iov, bytes) \
iov_iter_fault_in_readable(iov, bytes)
#define ll_iov_iter_fault_in_writeable(iov, bytes) \
iov_iter_fault_in_writeable(iov, bytes)
#endif
```
(defect) it looks like this will leak cdp->cdp_pages if an error is returned.
(defect) same
(minor) this should be moved to the end and cleaned up in one place:
```
if (unlikely(result != page_count)) {
CDEBUG(D_PAGE, "ll_release_user_pages() result=%ld, page_count=%ld\n", result, page_count);
if (result >= 0)
- return -EFAULT;
+ result = -EFAULT;
+ GOTO(out_free, size = result);
}
pvec->ldp_count = page_count;
+out_free:
+ if (size < 0) {
+ ll_release_user_pages(pvec->ldp_pages, page_count);
+ pvec->ldp_pages = NULL;
+ }
return size;
```
LU-0000 llite: fault in pages before get_user_pages We must fault in the user pages before get_user_pages, otherwise we can livelock with the mmap sem. Not sure about the requirement for the other case, on newer kernels - but it's worth a try. Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8a9a8094101e37a12d59482efb6a788231233837
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 failed 2× | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
I'm sort of confused about the logic here - why do we have to replace it? What does that do? And how can this page already be in the radix tree? I guess it's not "this" page, it's just another page at the same index. So this seems like it might be a race with removal in cl_page_delete/osc_page_delete? If it is, then how do we avoid the other thread removing *this* page from the tree, since it's done by index? It looks to me like vvp_page_delete() makes the page inaccessible in the page cache(?) by resetting the private pointer, but I'm not 100% sure. Then osc_page_delete() is called after. I guess my point is I'm not sure this is solving rather than hiding the problem. I guess perhaps if the page is in the process of being removed (which is why we're getting a new page), we can just do that removal ourselves first? Is that the logic here? I guess that seems sound, but what about the deleting thread possibly removing this page?
I think we must investigate why the page is already in the tree?
yes, I agree that it's strange to find an osc_page didn't successfully deleted from the radix tree, I'm just working on a workaround here as the customer find out certain version kernel does not have this issue while some does, so I'd guess that certain version kernel has some bug in the radix tree implementation.
LU-19037 osc: initialize osc_object::oo_tree Initialize the radix tree of osc_object::oo_tree. Lustre-change: https://review.whamcloud.com/59340 Lustre-commit: 1f4d2013929bd61f775e6c98a4122fe143b4242b Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Change-Id: I77a19cb08ccc52f4eb3457cd3367884b84624054
| unique failing test | history |
|---|---|
| sanity2@zfs:test_119l | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 10.1 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-subtest-change | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
(minor) ... so it could also be set and checked via `chattr +t FILE` and `lsattr FILE` commands from e2fsprogs.
The "Changes:" list mentions adding LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE, but the same hunk also adds LUSTRE_COMPR_FL to that mask and removes the duplicate LUSTRE_NOATIME_FL entry. Neither is explained. The COMPR change is an independent server-side behavior change - should it be split into its own patch? The lustre/utils/lfs.c hunk (skipping the range validation for nohybrid) isn't described either.
This changes UAPI flag definitions that the MDT interprets, so interop with an older server is worth an explicit test run. Consider adding something like:
Test-Parameters: testlist=sanity serverversion=2.16.0
New wire flag, but wirecheck.c and the two wiretest.c copies don't look updated - the neighbouring LUSTRE_*_FL values have CHECK_VALUE_X entries. Same for LU_LADVISE_NOHYBRID, which needs a CHECK_VALUE next to the other LU_LADVISE_* ones (LU_LADVISE_AHEAD seems to have been missed earlier too).
The compatibility claim points at the wrong side. Older clients aren't the problem; older servers are.
On a pre-patch MDS, LUSTRE_FL_USER_VISIBLE has no NOTAIL bit, so mdt_setattr_unpack() hits
if (rec->sa_attr_flags & ~LUSTRE_FL_USER_VISIBLE)
RETURN(-EOPNOTSUPP);
and lfs ladvise -a nohybrid fails with EOPNOTSUPP. Clearing the flag still "succeeds" as a no-op, so the two directions behave differently. Worth stating the required server version here.
This was asked on patchset 7 and looks unaddressed: adding LUSTRE_COMPR_FL here is unrelated to nohybrid and changes server behavior. mdt_setattr_unpack() masks with LUSTRE_FL_USER_MODIFIABLE, and osd_attr_set() replaces the whole masked set, so this makes the compression flag both settable and clearable on the MDT inode by any client. Is that intended ahead of the compression work landing?
(defect) why is NOATIME being removed?
Is the COMPR flag really user modifiable or just visible? And does it make sense to allow this to be set before CSDC is landed to master?
Should this use NOHYBRID?
Does this also set the flag directly on the inode, or is the inode here the root or parent directory?
parse ll_file_ioctl():error: Function too hairy. Giving up. 4 seconds warn: ll_file_ioctl():Function too hairy. No more merges.
ll_inode2ext_flags() is not a full picture of the file's flags - it rebuilds them from inode->i_flags via ll_inode_to_ext_flags(), which only knows SYNC/NOATIME/APPEND/DIRSYNC/IMMUTABLE/ENCRYPT, plus the PROJINHERIT and (new) NOHYBRID lli_flags bits.
LUSTRE_NODUMP_FL and LUSTRE_NOCOMPR_FL are in LUSTRE_FL_USER_MODIFIABLE but have no i_flags or lli_flags representation, so they come back as 0 here. osd_attr_set() then does a wholesale replace:
ei->i_flags = (ei->i_flags & ~LDISKFS_OSD_USER_MODIFIABLE) |
(attr->la_flags & LDISKFS_OSD_USER_MODIFIABLE);
so those bits get cleared on disk. chattr +d FILE followed by lfs ladvise -a nohybrid FILE should lose the 'd' flag.
The FS_IOC_SETFLAGS path avoids this by calling fileattr_get() first, which fetches body->mbo_flags from the MDT. Should this do the same before OR-ing in LUSTRE_NOHYBRID_FL?
Related: ll_set_project() builds op_attr_flags from ll_xflags_to_ext_flags() and also sets OP_XVALID_FLAGS, so lfs project on a file appears to clear the nohybrid flag for the same reason.
LU-19839 llite: add persistent nohybrid I/O flag Add a persistent flag to prevent hybrid I/O switching for specific files. This uses the NOTAIL flag (0x00008000) which is not used by Lustre/ext4 and unlikely to be used in the future. The flag is set via ladvise interface for discoverability but uses FS_IOC_SETFLAGS internally for implementation. When set, hybrid I/O will not switch the file from buffered to direct I/O regardless of I/O size. Changes: - Define LUSTRE_NOTAIL_FL and LUSTRE_NOHYBRID_FL - Add LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE - Implement LU_LADVISE_NOHYBRID using FS_IOC_SETFLAGS - Update ll_update_inode_flags to handle NOTAIL flag - Add test_119l to verify nohybrid flag functionality Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6a69293801114e2a3015ed87f2258828922ab767
This paragraph describes a change that does not seem to be in the diff. lustre/lov/lov_page.c isn't touched, and lov_page_init_composite() has no append exception either before or after this commit:
stripe_cached = lio->lis_cached_entry != LIS_CACHE_ENTRY_NONE &&
page->cp_type == CPT_TRANSIENT;
Was this paragraph left over from an earlier version of the patch?
This fixes wrong-stripe routing and a wrong file size, so a Fixes: tag would help decide which maintenance branches need it. The "DIO is already split by stripe" assumption that this patch repairs was introduced by:
Fixes: d31647c017a3 ("LU-13799 lov: Improve DIO submit")
That commit made lov_io_submit() splice every DIO page onto the stripe of the first page, and 14db1faa0fbe ("LU-13799 lov: Cache stripe offset calculation") then extended the same assumption to the per-page stripe cache. The assumption never held for the append path.
The clamp to the component extent lives inside this branch, so nothing bounds the batch when the component at `start` has a single stripe. For a layout like `-E 1M -c 1 -E -1 -c 4`, a 4MB DIO append from offset 0 gets no split at all and the batch crosses the component boundary; lov_page_init_composite() then hits
if (!lov_io_layout_at_confirm(lio, entry, offset))
return -ENODATA;
for the pages past 1M and the write fails.
The non-append path below clamps unconditionally - it starts at MAX_LFS_FILESIZE and applies min(next, lse->lsme_extent.e_end) outside the stripe_count test. Should this do the same, so the commit message's "matching what the non-append path already does" actually holds?
lio->lis_io_endpos is an absolute file offset set once in lov_io_init():
lio->lis_io_endpos = crw_pos + crw_bytes;
For append that crw_pos is only the estimate taken in ll_file_io_generic(); the offset the write actually lands at is chosen later by vvp_io_write_start() from i_size_read(). After the first iteration `start` tracks that real position, so if the file grew in the meantime `start` can run past lis_io_endpos.
Two consequences: the loop stops once next reaches lis_io_endpos even though bytes remain in the iov_iter (short write from an O_APPEND write(), where before the patch the whole count went out in one iteration), and on the next line `next - start` goes negative into the size_t crw_bytes.
Should the append path clamp against the number of bytes still outstanding rather than an absolute endpos, and guard next >= start?
Setting ci_continue here turns a single append iteration into several, and cl_io_loop() drops the lock between them:
cl_io_lock() -> cl_io_start() -> cl_io_end() -> cl_io_unlock()
vvp_io_write_lock() takes [0, wr_append_lockpos] PW for append precisely so the whole write lands atomically at EOF, and vvp_io_write_start() re-reads i_size_read() under that lock on every iteration. With the split, another client can revoke the lock and append between our iterations, so a single 4MB O_APPEND write() can end up with a foreign client's data interleaved in the middle of it.
Is there a way to keep the append as one locked iteration and instead bound the cl_dio_pages batch itself (in ll_direct_IO()/cl_dio_pages_init()) at the stripe boundary?
This contradicts the assertion 14 lines above, which says the opposite about the same queue:
/* it could only be mirror read to get here therefore
* the pages will be transient. */
LASSERT(page->cp_type == CPT_TRANSIENT);
If transient pages genuinely never reach lov_io_submit(), that earlier LASSERT can never be satisfied and would LBUG the client on any buffered read that lands on an uninstantiated component. Worth resolving one way or the other while this function is being touched.
I read that `drop_caches` is not considered reliable. It might be better to use `cancel_lru_locks osc`.
Andreas, out of interest, do you have some more info on this as to why?
There is no OST count precondition, so on a single-OST setup `-c 2` silently yields one stripe and the test passes without exercising anything. Consider:
(( OSTCOUNT >= 2 )) || skip_env "needs at least 2 OSTs"
Also, the file is empty here, so the append starts at offset 0 and is already stripe aligned. Appending to a file that is not a multiple of the stripe size (say seed it with 512KB first) is the case where the split has to get the first partial stripe right.
Both md5sums read the same OST data - the first one populates the page cache from the OSTs after the DIO write - so this compares the file against itself rather than against what was written. Writing from a temp file (or `dd`ing a known pattern) and comparing that checksum would actually catch mis-routed data; right now only the size check can fail. On a related note, the earlier suggestion on this hunk to use `cancel_lru_locks osc` instead of drop_caches doesn't appear to have been picked up in the current revision.
LU-19900 lov: fix DIO with O_APPEND stripe routing When a file is opened with O_APPEND and written via direct IO, all data is incorrectly routed to stripe 0 instead of being distributed across stripes. The root cause is that lov_io_rw_iter_init() does not split append writes at stripe boundaries. Each DIO iteration processes a cl_dio_pages batch, which lov_dio_pages_init_composite() assigns to a single stripe. Without splitting, the entire write goes to one stripe, causing incorrect file sizes (e.g. 7MB instead of 4MB for a 4MB write to a 2-stripe file) and data corruption. Fix by adding stripe boundary splitting to the append path of lov_io_rw_iter_init(), matching what the non-append path already does. Each iteration now covers at most one stripe_size worth of data, so lov_dio_pages_init_composite() correctly assigns each batch to a single stripe. Also add an LASSERT in lov_io_submit() confirming that DIO pages (CPT_TRANSIENT) never reach that path - they use lov_dio_submit() instead. Also revert the append exception for the DIO stripe cache in lov_page_init_composite() since it is no longer needed now that iterations are single-stripe. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I66304a6692eaab9c68a03159e51cf3d27c465a83
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 34 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 7 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 34 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 6 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 9 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 7 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 7 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 39 other reviews |
| sanity1@zfs:test_56xB | seen in 11 other reviews |
| sanity1@zfs:test_56xa | seen in 39 other reviews |
| sanity1@zfs:test_56xab | seen in 8 other reviews |
| sanity1@zfs:test_56xc | seen in 35 other reviews |
| sanity1@zfs:test_56ej | seen in 11 other reviews |
| sanity2@zfs:test_119e | seen in 18 other reviews |
| sanity2@zfs:test_119f | seen in 17 other reviews |
| sanity2@zfs:test_119g | seen in 17 other reviews |
| sanity2@zfs:test_119h | seen in 17 other reviews |
| sanity2@zfs:test_119p | seen in 7 other reviews |
| sanity2@zfs:test_119q | seen in 7 other reviews |
| sanity2@zfs:test_398o | seen in 22 other reviews |
| sanity2@zfs:test_398s | seen in 7 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 10 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 33 other reviews |
| sanity-flr@zfs:test_0h | seen in 35 other reviews |
| sanity-flr@zfs:test_0j | seen in 34 other reviews |
| sanity-flr@zfs:test_36d | seen in 32 other reviews |
| sanity-flr@zfs:test_37 | seen in 36 other reviews |
| sanity-flr@zfs:test_38 | seen in 33 other reviews |
| sanity-flr@zfs:test_44b | seen in 9 other reviews |
| sanity-flr@zfs:test_61a | seen in 40 other reviews |
| sanity-flr@zfs:test_61c | seen in 33 other reviews |
| sanity-flr@zfs:test_70a | seen in 60 other reviews |
| sanity-flr@zfs:test_200a | seen in 33 other reviews |
| sanity-flr@zfs:test_200b | seen in 33 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 11 other reviews |
| sanity-hsm@zfs:test_607b | seen in 11 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 29 other reviews |
| sanityn@zfs:test_16j | seen in 32 other reviews |
LU-13814 osc: assert transfer pages identical At this point, we're about to stop using cl_page for DIO, and the transfer pages created with the cl_page and those created separately for DIO should be identical. Let's assert that for every value. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6c71f6c2a3e48a65f5abb18d7beb0699f7577d92
We aggregate attributes from multiple objects for other reasons (e.g. maxbytes, timestamps, size, etc.). The current implementation looks "mostly OK" and will work for basic linear IO patterns, where the file transitions from non-rotational to rotational. However, any other kind of IO pattern (e.g. HDF5 where some writes are at the beginning of the file and others at the end) might have a serious issue? IMHO, it would be better to aggregate this information once from the object (maybe from the OSC's the object is allocated on) and then be done with it? Storing a threshold when the file transitions from non-rotational to rotational would be best, as this could be computed once based on the layout. It is fairly unlikely that a file would have HDD stripes in the middle and flash at the beginning and end. Alternately (probably better) is to store the nonrot state in each layout component (which is a perfect 1:1 mapping), and then this can be checked at IO submission time to see what type of storage it is covering.
Hm, OK. I don't think we can do this as easily as we'd like - the issue is we have to have this information very early. I can consider this, though, your points are good ones. The good thing is this isn't essential to merging the core feature, particularly since we're leaving it off by default for 2.16.
Actually, an OST is (generally) only going to be rotational or non-rotational, and this is already returned to the client via statfs, so the clients should have full knowledge whether any IO is on flash or disk.
LU-13802 llite: add file nonrotational check This patch adds the ability to note whether or not the last IO to a file hit a rotational or a non-rotational device. This will be used in selecting the cutover thresholds for hybrid IO, since DIO is synchronous and much slower on spinning disk. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I75a9970f91b1776ed6f04ac0d000a9ba576df75a
error: 'io' may be used uninitialized in this function [-Werror=maybe-uninitialized]
I was wondering if the Janitor flags this. Technically, I don't think we can get here since `rc` would be -ENOMEM if we jumped out in line :2006 where `io` would not be set yet. Looks like that still needs to be fixed (compile failed). I guess it'd be easiest to keep the first call to `vvp_env_new_io()` and drop the second one?
LU-19109 llite: remove extra vvp_env_new_io call vvp_env_new_io is called twice in ll_file_io_generic, which is confusing since the second call overwrites the first. Fix this and a cleanup path mistake, where we assume the IO was set up after the call to vvp_env_new_io, which is wrong. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6186e17db8b04f01fd37ea7ac5d4b69b30d0258c
It would make sense to put an LLM to work on converting the NASM .asm files int GCC .S files. It should be possible to have it iterate on getting the GCC-compiled code to generate the same x86 byte code, or at least compile and run correctly to pass whatever correctness tests exist for this code.
In fact, a quick search shows that NASM already has a tool to do this:
Automatically convert Intel NASM assembly to GNU Assembler (GAS/GASM)
syntax using the intel2gas tool, which converts between NASM and AT&T
syntax, or by utilizing NASM's built-in output capabilities to generate
GAS-compatible objects.
$ intel2gas -i input.asm -o output.s
These generated/converted .S files should be stored alongside the original .asm files in Git. If NASM is available the originals can be compiled, but in the common case where NASM is not available the .S files would be compiled and linked into the kernel modules.
I think it is worthwhile to keep both, since it would be easier to update the .asm files from upstream ISA-L, but it might be some time before the .S files are update to match.
Are these bugs in the upstream ISA-L code that should be pushed back to them?
Similarly, is this code from upstream ISA-L, or a test wrapper that we developed?
This should probably use LU-19905, or change the description of LU-20016?
This table is the same between the x86 and aarch64 patches. It would be useful to fix one or the other to have the correct data for that CPU architecture.
(minor) this should have a warning that the C version may only be 1/20-1/30th as fast, at least until the .S versions are available.
(minor) It would be better to specify which kernel version is non-functional, so that this can become conditional upon a newer kernel (assuming there is a benefit to use SVE-optimized versions).
(minor) Again, please specify kernel versions so that this can be fixed in the future for newer kernel versions.
(style) `bool`?
(style) externs should be avoided in .c files
These warnings are correct and the prototypes should be moved to a .h file that is included here and into the file where these functions are implemented. Having the function prototype in a local .c file prevents the compiler from checking argument type/count against the implementation, and can result in hard-to-find bugs in the future.
(defect?) According to the comments for `kernel_neon_begin()`: ``` * Unless called from non-preemptible task context, @state must point to a * caller provided buffer that will be used to preserve the task's kernel mode * FPSIMD context when it is scheduled out, or if it is interrupted by kernel * mode FPSIMD occurring in softirq context. May be %NULL otherwise. */ void kernel_neon_begin(struct user_fpsimd_state *state) ``` I'm thinking that this would be declared on the stack in e.g. `gf_vect_dot_prod()` and passed to `ec_neon_usable()->kernel_neon_begin()` so that it can also be passed (as required) to `kernel_neon_end()`: ``` * The value of @state must match the value passed to the preceding call to * kernel_neon_begin(). */ void kernel_neon_end(struct user_fpsimd_state *state) ``` The `struct user_fpsimd_state` is 528 bytes, so a _bit_ heavy to put on the stack, but at the same time this is (very likely) to be on a PAGE_SIZE=64KiB system so stack space should be available for this, and it will not have a deep call chain below this point. I think doing a `kmalloc()` and `kfree()` for each call would add noticeable overhead and should be avoided it possible. Alternatives would include having a dedicated slab for this, so that there is a per-CPU cache (528 does not fit into standard slabs well) and it likely has local allocations objects cached in the slab.
(style) should this be changed to `rows >= 6` and remove the `case 6:` in the switch, as is done in `ec_encode_data_neon()` above?
LU-20016 ec: ISA-L SIMD for userspace and kernel
Integrate Intel ISA-L optimized assembly into Lustre's
erasure coding library for both userspace and kernel,
on x86_64 and aarch64. Add debugfs kernel benchmark
for measuring FPU save/restore overhead.
Userspace x86_64 (libec.a):
- 72 NASM assembly files (SSE/AVX/AVX2/AVX-512/GFNI)
- ec_multibinary.asm: runtime CPUID dispatch
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar when NASM unavailable
Userspace aarch64 (libec.a):
- NEON + SVE .S assembly + SVE C intrinsics
- ec_aarch64_dispatcher.c: getauxval() dispatch
- ec_aarch64_highlevel_func.c: N-vector dispatch
Kernel x86_64 (ec.ko):
- Pre-assemble ISA-L NASM .asm files into .o,
link into ec.ko (71 assembly objects)
- ec_dispatch.c: boot_cpu_has() selects AVX2/AVX/SSE
at module init, kernel_fpu_begin/end wrapping
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar in interrupt context
Kernel aarch64 (ec.ko):
- NEON .S assembly files (GAS format, direct kbuild)
- ec_aarch64_neon.c: kernel_neon_begin/end wrapping
with may_use_simd() check, hwcap detection
- SVE excluded from kernel (toolchain portability)
Build system:
- lustre-erasurecode.m4: detect NASM, aarch64
- erasurecode/autoMakefile.am: three-way dispatch
- utils/Makefile.am: link against libec.a
- ec/Makefile.in: NASM pre-assembly for kernel
Bug fixes in ec_perf_bench.c:
- -p flag was setting k instead of p
- frag_ptrs allocated stripe_size pointers not m
- Inverted exit code and stale rc from getopt
- Use aligned_alloc(64) for data buffers
Kernel benchmark (debugfs):
- /sys/kernel/debug/lustre/ec/benchmark
- 4 phases: FPU overhead, SIMD+FPU per-call,
SIMD+FPU amortized, C scalar baseline
- Input format (write to debugfs entry):
echo 1 > .../ec/benchmark (defaults)
echo "k=5 p=2 s=128" > .../ec/benchmark
cat .../ec/benchmark
Parameters (key=value, space separated):
k data stripes (default 5, max k+p=16)
p parity stripes (default 2)
s stripe size in KB (default 128, max 65536)
"echo 1" runs with all defaults. Unrecognized
input returns -EINVAL. cat shows results or
usage instructions if not yet run.
Fix gf_vect_mul_init naming to use _base suffix
consistently (aligns with ISA-L convention) in
both userspace and kernel ec_base.c.
Performance (5+2, 1 thread, QEMU VM, AVX2):
Stripe | Kernel | Kernel | Userspace | SIMD | Kernel vs
Size | SIMD | Scalar | SIMD | Speedup | Userspace
| | | | in Kernel|
-------|---------|---------|-----------|----------|----------
64 KB | 17099 | 540 | 404345* | 31.7x | 0.04x*
128 KB | 16888 | 531 | 197511* | 31.8x | 0.09x*
256 KB | 17032 | 539 | 98401* | 31.6x | 0.17x*
1 MB | 17142 | 538 | 24830 | 31.9x | 0.69x
4 MB | 10309 | 533 | 5385 | 19.3x | 1.91x
16 MB | 10954 | 534 | 1048 | 20.5x | 10.5x
All throughput in MB/s. (*) Userspace numbers at
small stripes are inflated by cache residency.
At 4-16MB the kernel is faster -- its kvmalloc
pages are physically scattered, matching real
Lustre page cache behavior. Replicating this in
userspace is non-trivial and not worth the effort;
the kernel benchmark is authoritative.
FPU save/restore overhead was measured at 25-54 ns
on this platform (Ryzen 3700X) -- negligible vs
encode time (~39us at 128KB). The benchmark
measures this separately (Phase 1) so it can be
characterized on other hardware where it may be
more significant.
Generated with Claude Code + Tools
Test-Parameters: testlist=sanity-ec
Change-Id: If4c57c328c0e23bb6769dcbc45d52d43b03a7837
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
This should use LU-19905.
LU-20016 ec: kernel SIMD, aarch64 support, benchmark
Add kernel-space SIMD erasure coding via ISA-L's
proven NASM assembly (pre-assembled at build time)
and aarch64 NEON assembly. Add debugfs benchmark
for measuring FPU save/restore overhead.
Kernel x86_64 (ec.ko):
- Pre-assemble ISA-L NASM .asm files into .o,
link into ec.ko (71 assembly objects)
- ec_dispatch.c: boot_cpu_has() selects AVX2/AVX/SSE
at module init, kernel_fpu_begin/end wrapping
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar in interrupt context
Kernel aarch64 (ec.ko):
- NEON .S assembly files (GAS format, direct kbuild)
- ec_aarch64_neon.c: kernel_neon_begin/end wrapping
with may_use_simd() check, hwcap detection
- SVE excluded from kernel (toolchain portability)
Userspace aarch64 (libec.a):
- NEON + SVE .S assembly + SVE C intrinsics
- ec_aarch64_dispatcher.c: getauxval() dispatch
- ec_aarch64_highlevel_func.c: N-vector dispatch
Kernel benchmark (debugfs):
- /sys/kernel/debug/lustre/ec/benchmark
- 4 phases: FPU overhead, SIMD+FPU per-call,
SIMD+FPU amortized, C scalar baseline
- Input format (write to debugfs entry):
echo 1 > .../ec/benchmark (defaults)
echo "k=5 p=2 s=128" > .../ec/benchmark
cat .../ec/benchmark
Parameters (key=value, space separated):
k data stripes (default 5, max k+p=16)
p parity stripes (default 2)
s stripe size in KB (default 128, max 65536)
"echo 1" runs with all defaults. Unrecognized
input returns -EINVAL. cat shows results or
usage instructions if not yet run.
Performance (5+2, 1 thread, QEMU VM, AVX2):
Stripe | Kernel | Kernel | Userspace | SIMD | Kernel vs
Size | SIMD | Scalar | SIMD | Speedup | Userspace
| | | | in Kernel|
-------|---------|---------|-----------|----------|----------
64 KB | 17099 | 540 | 404345* | 31.7x | 0.04x*
128 KB | 16888 | 531 | 197511* | 31.8x | 0.09x*
256 KB | 17032 | 539 | 98401* | 31.6x | 0.17x*
1 MB | 17142 | 538 | 24830 | 31.9x | 0.69x
4 MB | 10309 | 533 | 5385 | 19.3x | 1.91x
16 MB | 10954 | 534 | 1048 | 20.5x | 10.5x
All throughput in MB/s. (*) Userspace numbers at
small stripes are inflated by cache residency.
At 4-16MB the kernel is faster -- its kvmalloc
pages are physically scattered, matching real
Lustre page cache behavior. Replicating this in
userspace is non-trivial and not worth the effort;
the kernel benchmark is authoritative.
FPU save/restore overhead was measured at 25-54 ns
on this platform (Ryzen 3700X) -- negligible vs
encode time (~39us at 128KB). The benchmark
measures this separately (Phase 1) so it can be
characterized on other hardware where it may be
more significant.
Generated with Claude Code + Tools
Test-Parameters: testlist=sanity-ec
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id37586461376ab266c61ed541473279f83c790ff
LU-19744 doc: bulk man page review Bulk review done by Augment and Claude Code. Completed systematic review of Section 3 library function man pages (llapi_*), fixing grammar, formatting, function signature errors, and SEE ALSO ordering throughout all 123 pages. Major sections reviewed: - Changelog API (8 pages) - File Operations (13 pages) - Filesystem Info (9 pages) - Locking (2 pages) - Heat/IO Hints (3 pages) - HSM (11 pages) - Layout API (30 pages) - Misc API (8 pages) - PCC (12 pages) - Project Quota (8 pages) - Quota (1 page) - Remove by FID (2 pages) - Search/Discovery (6 pages) - Foreign Files (1 page) Critical signature fixes found during careful review: PCC (Persistent Client Cache): - llapi_pcc_detach_fid_fd.3: removed documentation for non-existent llapi_pcc_detach_fid_fd() function - llapi_pccdev_get.3: fixed parameter name (path->mntpath) - llapi_pccdev_set.3: fixed parameter name (path->mntpath) Project Quota: - llapi_project_fgetprjid.3: added missing 'struct' keyword - llapi_project_get.3: added missing 'struct' keyword - llapi_project_getprjid.3: fixed parameter type (const unsigned int->__u32) - llapi_project_open.3: fixed double pointer (*hdl->**hdl) - llapi_project_put.3: added missing 'struct' keyword Search/Discovery: - llapi_root_path_open.3: fixed parameter name (fd->outfd) - llapi_search_rootpath.3: added missing semicolons - llapi_search_tgt.3: fixed parameter names (pool_name->poolname, tgt_name->tgtname/mdtname/ostname) Foreign Files: - llapi_unlink_foreign.3: fixed parameter name (name->dname) Other improvements: - Fixed grammar, capitalization, and formatting throughout - Corrected function parameter types and names - Fixed missing error codes in ERRORS sections - Improved consistency in RETURN VALUES sections - Fixed typos and awkward phrasing - Sorted SEE ALSO sections alphabetically (31 files) All signature changes verified against source code in lustre/utils/liblustreapi*.c and lustre/include/lustre/lustreapi.h Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I643e4b2ad2d00b5ebf58d4136d1524bd92b7cb5d
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-12187 llite: enable FLR EC support unconditionally Remove module parameter guards and enable FLR EC support unconditionally. This patch is intended to be applied after the guarded development phase is complete. Changes: - Remove mdt_enable_flr_ec module parameter - Remove llite_enable_flr_ec module parameter - Add OBD_CONNECT2_FLR_EC unconditionally to client connect flags - Change ll_enable_erasure_coding default from 0 to 1 - Remove MODOPTS_MDT and MODOPTS_LLITE settings from test framework Test-Parameters: ignore Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I431d309f85c8964e13a11eaaa729d944e9228c29
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-subtest-change crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-subtest-change crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-dne-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
LU-19566 tests: test EC with lfsck EC parity mirror components do not contain regular file data, so it's essential they be recognized as parity components after an lfsck repair. This test verifies this functionality. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0384d7676043cc23f52da7df41385dc61809114a
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
(typo) field is actually named `ff_comp_flags`
(style) rather than duplicating this code each time, it should just incrementally decode the new parts:
```
} else /* if (size >= sizeof(struct filter_fid_217) */ {
struct filter_fid_217 *ff_old = (struct filter_fid_217 *)dst;
ost_layout_cpu_to_le(&ff_old->ff_layout, &src->ff_layout);
ff_old->ff_layout_version = cpu_to_le32(src->ff_layout_version);
ff_old->ff_range = cpu_to_le32(src->ff_range);
}
if (size >= offsetof(dst->ff_comp_flags) + sizeof(dst->ff_comp_flags)) {
dst->ff_comp_flags = cpu_to_le32(src->ff_comp_flags);
}
```
(style) same as above - process each chunk incrementally instead of duplicating code
(defect) this comment should not be removed
Oh, hmm. There was another patch looking to use one of the uid_h fields:
https://review.whamcloud.com/64468 ("LU-18847 mdt: version layout checking")
but it is using the `cr_fsuid_h` field in `struct mdt_rec_create`, so this should be fine.
(style) better to use `enum lov_comp_md_entry_flags` to make it more clear which "flags" these are, and they can be found more easily. The wirecheck.c code will ensure that the enum size remains consistent.
(minor) I think this is a misnomer to write "for filter_fid", since it is primarily used for components. Maybe `layout flags for components and filter_fid`?
(style) this shouldn't be in the comment here, as it will invariably become outdated in the future
(style) should there be a helper function for this, like:
```
/* return minimum size of in-use filter_fid to maximize interop with old tools */
size_t filter_fid_sizeof(struct filter_fid *ff)
{
BUILD_BUG_ON(sizeof(*ff) >
sizeof(struct filter_fid_217) + sizeof(ff->ff_comp_flags));
/* duplicate next chunk when new fields are added to filter_fid */
/* if (!ff->ff_new_field && ff->ff_previous_last_field)
return sizeof(struct filter_fid_next); */
if (!ff->ff_comp_flags /* && ff->ff_layout_version */)
return sizeof(struct filter_fid_217);
return sizeof(*ff);
}
```
(minor) `ff_size = filter_fid_sizeof(ff);`
(minor) `ff_size = filter_fid_sizeof(ff);`
(minor) It isn't clear if there is any value to printing this field, especially *always* printing it (maybe vs. only printing it if it is non-zero for some reason)
(style) 'version_code 2.17' may be misspelled - perhaps 'version 2.16.x should be used'?
(minor) update to 2.17.52.52 at least
LU-19566 lustre: add layout flags to lfsck EC parity mirror components do not contain regular file data, so it's essential they be recognized as parity components after an lfsck repair. They are distinguished by a component flag, so we must add component flag support to lfsck. This support can be landed without the rest of EC support, so this is based on master. There is a test for lfsck + EC in: https://review.whamcloud.com/c/62489 Details: Extended struct filter_fid from 52 to 56 bytes by adding __u32 ff_flags field to store component flags (like LCME_FL_INIT, LCME_FL_PARITY) in OST objects. Created versioned structure (filter_fid_217) for backward compatibility with old 52-byte filter_fid. Modified lu_orphan_rec_v3 to add lor_comp_flags field (replacing lor_padding_1) to store component flags from filter_fid. This allows LFSCK to preserve component flags during reconstruction. Extended wire protocol by adding o_comp_flags field to struct obdo (replacing o_padding_4) and added OBD_MD_FLCOMPFLAGS flag to indicate when component flags are valid in the obdo. Updated client to set o_comp_flags in the obdo. Modified OFD to read o_comp_flags from obdo and store it in ff_flags during writes, setattr, punch, and fallocate operations. Updated LFSCK to read ff_flags from filter_fid and restore component flags when reconstructing layouts from orphan OST objects. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I716dbf24db48cc50385a2fa9bcc446d291380814
This should actually run the test script to see that it is working: ``` Test-Parameters: testlist=lfru-performance ``` I've submitted a manual test session via the `Test Results` page to see if it works. https://testing.whamcloud.com/test_sessions/related?jobs=lustre-reviews&builds=122012#redirect Ideally this testing could be run regularly via one of the test scripts (e.g. performance-sanity.sh or sanity-benchmark.sh) to monitor performance over time. Putting it in a separate test script means that the test framework needs to be modified to run this.
Ping
(style) It is better if comments do not contain the actual values in the constants, since that makes it more likely the comments become incorrect over time.
LU-11509 misc: add script lfru-performance.sh LFRU was introduced to provide scan-resistant, which was validated in sanity-test-124g. Furthermore, it ensures that high-priority locks are more likely to remain in the cache, improving overall system stability and performance under mixed workloads. 1. A new benchmark test, `lfru-performance.sh`, is introduced to compare LFRU against LRU. This test simulates a workload where both hot and cold files are accessed with a 50:50 ratio, involving 800 hot files and 16,000 cold files, and fixed sized cache size. The LFRU algorithm reduced the number of lock-RPCs (measured by `ldlm-enqueue` calls) by ~8% compared to the LRU policy. | Test Run | LFRU Enqueues | LRU Enqueues | Improvement | | 1 | 160075 | 174250 | 8% | | 2 | 159372 | 175925 | 9% | | 3 | 159488 | 174533 | 8% | | 4 | 159712 | 175481 | 8% | | 5 | 159986 | 174493 | 8% | The benchmark results showed that LFRU outperforms LRU in this mixed-access scenario. 2. Update ldlm_lfru_priv_too_many() so that the eviction trigger for priv_lock is now primarily based on its ratio to total_lock_counts. The use of LDLM_DEFAULT_LRU_SIZE is removed. Signed-off-by: Keguang Xu <squalfof@gmail.com> Change-Id: I60903947180fb8c2b4e1a74e94cb2c5bb387d9d5
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 | CentOS 8.3/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
LU-15069 llite: Clean up bit shift for assert There is a bizarre comment which says we're not converting pages to bytes yet to save cost in checking an assert, but we do that conversion immediately after the assert *and* it requires doing a similar shift in the assert. This is quite strange - clean it up. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id2bb4e6910276537d807828a9a33439dea054c3b
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56ab | seen in 5 other reviews |
this 88 of 500 sounds strange to me.. is it still failing?
I guess the intent is that the patch passed 5x as many iterations as without the patch, so it is improving the situation to some extent.
if this was the real problem introducing commit, it should be Fixes: header below.
The pieces of this patch don't add up. While I understand it improves the situation, now that that was demonstrated, it's time for a real human to look into why and make a patch that actually makes sense.
is this the actual weight bearing "fix" of this patch?
this comment seems wrong. We can never get here via unevict-clear (osc_unevict_cache_shrink call I guess?) because it sets reason to SK_REASON_UNEVICT_LRU, but that case is already handled above. That leaves a call for normal reclaim from osc_lru_reclaim?
I am not sure how this statement makes any sense?
LU-19487 osc: fix shrinker loop in osc_lru_list_shrink
Commit 109e32dc23 ("LU-19223 osc: stop after scanning")
changed the scan-limit tracking in osc_lru_list_shrink()
from a countdown to a count-up, and added a
--pages_scanned adjustment in the cl_object-switch path
to avoid counting object-switch iterations as scans.
When LRU pages belong to many different cl_objects, the
decrement causes pages_scanned to stagnate near zero,
making the loop run far longer than max_pages_to_scan
allows. The kernel's do_shrink_slab then re-calls the
shrinker because nr_scanned stays low, creating a
CPU-bound infinite loop that hangs the system.
Remove the --pages_scanned adjustment. The cl_object
switch does real work (dropping the spinlock, calling
cl_io_init) and should count toward the scan limit.
For forced scans (unevict-clear, cache-limit reduction,
etc.), increase the scan budget to target<<1 without
capping at lru_in_list so that object-switch overhead
does not prevent the shrinker from reaching its target.
Reproduced on a single-node test setup: sanityn test_16g
hung at iteration 17 of 100 without the fix (system
required hard reboot), passed 88 of 500 iterations
cleanly with the fix applied.
Generated with Claude Code + Tools
Test-Parameters: testlist=sanityn env=ONLY=16g,ONLY_REPEAT=10
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I7c007a1c021067e0b166b00045223ad8a927b5aa
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-20112 tests: verify drop_caches evicts client cache pages Add sanity test_101k that verifies drop_caches actually evicts Lustre client page cache pages once any in-flight bulk RPC pin has been released. Read pages reap their bulk pin immediately on RPC completion. Write pages keep the pin until the OST commits, so the test forces a commit via dd conv=fsync. Both paths assert the post-drop read shows ost_read activity (cache miss). Note: global sync(2) does NOT drain Lustre's bulk pins -- only fsync(fd) does, because only fsync goes through ll_fsync -> OST_SYNC. cancel_lru_locks osc also drains the pins by canceling the DLM extent locks, which is what the read path in this test uses. Generated with Claude Code + Tools Test-Parameters: testlist=sanity env=ONLY=101k Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Iafa8f5986fed38d98964a3ebea929e0b412a8e49
LU-16488 utils: fix help message for 'lctl interface_list' The help message for 'lctl interface_list' command was incorrect. It was saying: "You must run 'interface_list <network>' command before 'network'" But the correct syntax is: "lctl --net tcp0 interface_list" This patch updates the help message in lctl.c and the man page to reflect the correct usage. It also updates the error message in portals.c to be more accurate. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7be1c0189d4177ce0c6ed67d77c98ab723684191
This is pretty similar to `ll_filemap_get_folios`, but I guess with all the #define involved it is not easy to factorize. Maybe introduce `ll_split_folio` to do the split loop (and which would be a no-op in case folio_order is not defined).
LU-20069 osc: pass page index explicitly for DIO encrypt DIO encrypted file corruption during migration. The encrypt path in osc_brw_prep_request() writes cp_page_index into page_folio(page)->index, then osc_encrypt_pagecache_blocks() reads it back via folio->index + page offset. For DIO pages that are part of compound pages (order > 0), the page offset within the folio is added to the index, producing a wrong AES-XTS tweak. The decrypt path correctly uses cp_page_index directly. Compound pages can appear for DIO allocations on any kernel -- reproduced on RHEL 9.6 and SLES 15.6. Fix: pass cp_page_index explicitly to osc_encrypt_pagecache_blocks() for DIO, mirroring the decrypt path. Eliminates the fragile round-trip through folio->index that breaks for compound pages. Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Iba44faefa61a7da6377e9d906b0453ef4ded5d13
| unique failing test | history |
|---|---|
| recovery-small@ldiskfs+DNE:test_131 | seen in 64 other reviews |
Ah, interesting. We have a mistake here - twice. Or maybe four times, have to check llite. These parameters should all be defaulting to disabled.
Also we should have a trap set before the 2.18 release to check if we should remove these parameters, as discussed
LU-19989 llite: add FLR IWM connect flag and switches Add OBD_CONNECT2_FLR_IMMED_MIRROR connect flag support, LCME_FL_IMMEDIATE layout flag, and enable_immediate_mirror parameter to control immediate write mirror layouts. Add module parameters to gate presentation of connection flag. Define LCME_FL_IMMEDIATE (0x800) and add it to LCME_KNOWN_FLAGS, LCME_USER_COMP_FLAGS, LCME_CL_COMP_FLAGS, and LCME_TEMPLATE_FLAGS. LCME_FL_IMMEDIATE and LCME_FL_PARITY are mutually exclusive. Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I173c890d899a52cd57719cc89cc0629e54c86e51
| unique failing test | history |
|---|---|
| conf-sanity2@ldiskfs+DNE:test_73c | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_73e | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_74 | seen in 12 other reviews |
| conf-sanity2@ldiskfs+DNE:test_75 | seen in 9 other reviews |
| conf-sanity3@ldiskfs+DNE:test_87 | seen in 7 other reviews |
| conf-sanity3@ldiskfs+DNE:test_88 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_89 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90a | seen in 12 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90b | seen in 15 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90c | seen in 21 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90d | seen in 24 other reviews |
| conf-sanity3@ldiskfs+DNE:test_91 | seen in 29 other reviews |
| conf-sanity3@ldiskfs+DNE:test_98 | seen in 34 other reviews |
| conf-sanity3@ldiskfs+DNE:test_99 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_120 | seen in 13 other reviews |
| conf-sanity4@ldiskfs+DNE:test_161 | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_91 | seen in 1 other review |
| sanity-quota@ldiskfs+DNE:test_92 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_93 | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_94 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_95a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_95b | seen in 4 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
Should this set PARALLEL_MKFS='mdt ost mgt' or are we just trying to get the old behavior to work at this point?
This should add "fortestonly" until it is passing testing, so it doesn't run a score of other test sessions.
It looks like the problem being hit by conf-sanity.sh is that lustre-initialization has already formatted mounted the MDTs and OSTs, but then it tries to format them again:
mkfs.lustre --mgs --fsname=lustre --mdt --index=0 --param=sys.timeout=20 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --mkfsoptions=\"-b 4096 -E lazy_itable_init\" --reformat /dev/vg_Role_MDS/mdt1
mkfs.lustre FATAL: Unable to build fs /dev/vg_Role_MDS/mdt1 (256)
Permanent disk data:
Target: lustre:MDT0000
Index: 0
Lustre FS: lustre
Mount type: ldiskfs
Flags: 0x65
(MDT MGS first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: sys.timeout=20 mdt.identity_upcall=/usr/sbin/l_getidentity
device size = 1888MB
formatting backing filesystem ldiskfs on /dev/vg_Role_MDS/mdt1
target name lustre:MDT0000
kilobytes 200000
options -b 4096 -I 1024 -i 2560 -q -O uninit_bg,^extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L lustre:MDT0000 -b 4096 -I 1024 -i 2560 -q -O
/dev/vg_Role_MDS/mdt1 is apparently in use by the system; will not make a filesystem here!
Possibly there is new state in test-framework.sh to track the parallel formatting, but this is lost between lustre-initialization and the instance of test-framework.sh that is starting up?
At this point in the conf-sanity.sh "reformat_and_config->formatall->stop mds1" chain, this is called and does not detect the MDT as mounted:
CMD: trevis-130vm6 [ -e "/dev/vg_Role_MDS/mdt1" ]
CMD: trevis-130vm6 grep -c /mnt/lustre-mds1' ' /proc/mounts || true
CMD: trevis-130vm6 lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' || true
so something is going wrong during this stage and some debugging should be added here. It would probably be the same to debug locally by running "llmount.sh" and then run "conf-sanity.sh" afterward.
(style) prefer `[[...]]` for bash
(style) it would be better to declare this before usage in `stopall()` above
LU-17240 tests: add parallel format/mount/unmount support Add support for parallel format, mount, and unmount operations in the test framework. These operations are controlled by a hierarchy of variables that allow fine-grained control over which operations run in parallel, which target types are parallelized, and whether different target types can overlap. Variable hierarchy: - PARALLEL_OPS: Master switch controlling all operations - PARALLEL_FORMAT, PARALLEL_MOUNT, PARALLEL_UMOUNT: Enable parallelism for specific operations (default to PARALLEL_OPS) - PARALLEL_FORMAT_TARGETS, PARALLEL_MOUNT_TARGETS, PARALLEL_UMOUNT_TARGETS: Space-separated list of target types to parallelize within each operation (e.g., "mdt ost") - PARALLEL_FORMAT_TYPES, PARALLEL_MOUNT_TYPES, PARALLEL_UMOUNT_TYPES: Whether to overlap different target types during operations (inter-type parallelism) Implementation: - New helper function parallel_enabled_for(operation, target_type) checks if parallelism is enabled for a specific combination - Updated formatall(), mountmds(), mountoss(), stopall(), unmountoss() to support parallel execution with proper wait barriers between target types when overlap is disabled - MGS excluded from parallel operations since there's only ever one MGS target All parallel operations are disabled by default (PARALLEL_OPS=0) for backward compatibility. Future patches will enable these features gradually after validation. Test results show significant performance improvements when enabled: - Parallel format with type overlap: 19% faster than baseline - Parallel mount within types: 7% faster - Parallel unmount within types: 21% faster Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I51af959a4f20644d9bd5166c1c1c8ebcfbef69f3
| unique failing test | history |
|---|---|
| sanity1@zfs:test_27D | seen in 80 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_45 | seen in 14 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
[Sashiko] The label "lcme_flags:" is used on the line above to print the hex flags value. The new line reuses the same label for an "immediate mirror" string, so two consecutive log lines with that label would have different types of content. Would something like "lcme_immediate:" be clearer?
[Sashiko] test40: The third argument to llapi_layout_sanity() is the flr flag. With flr=false, the sanity callback checks flags against the non-FLR allowed set, which does not include LCME_FL_IMMEDIATE. So the call returns LSE_FLAGS rather than LSE_IMMEDIATE_MIRROR_COUNT. The assertion rc != 0 passes, but the path the test description describes -- "immediate needs >= 2 mirrors" -- is never reached. Would passing flr=true and asserting rc == LSE_IMMEDIATE_MIRROR_COUNT better match the stated intent?
[Sashiko] llapi_layout_comp_flags_set() enforces that LCME_FL_IMMEDIATE and LCME_FL_PARITY are mutually exclusive, but the sanity callback doesn't appear to check that combination independently. In the FLR case (lsa_flr=true), both flags are in LCME_USER_COMP_FLAGS and would pass the flags validation. Can a layout with both LCME_FL_IMMEDIATE and LCME_FL_PARITY on the same component pass llapi_layout_sanity() without error -- for example, one read back via llapi_layout_get_by_fd()?
LU-19991 llapi: immediate mirror layout support Add LCME_FL_IMMEDIATE kernel helpers, llapi validation, and layout passthrough support. Add lsme_is_immediate() and lsm_entry_is_immediate() helpers in lov_internal.h. Add debug print for LCME_FL_IMMEDIATE in pack_generic.c. Add LCME_FL_IMMEDIATE + LCME_FL_PARITY mutual exclusion validation, >= 2 mirror requirement, and per-mirror all-or-nothing enforcement in liblustreapi_layout.c sanity checks. Fix llapi_layout_set_by_fd to pass correct FLR flag to sanity check. Add llapi_layout_test cases: flag round-trip (test38), mutual exclusion (test39), mirror count validation (test40), kernel setstripe/getstripe round-trip (test41), directory default layout inheritance (test42), multi-component mirror flag consistency (test43), re-open by path (test44), flag clearing (test45), both-mirrors IMMEDIATE (test46), 3-mirror isolation (test47), partial IMMEDIATE rejection (test48). Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I42f17a6bba89c6c5c1f1a3e9d71ab3a0fb7c7f1e
(defect) This should use a proper LU number
```suggestion LU-930 doc: man page improvements ``` Could just use the default "man page improvement" ticket...
It looks like there are patches under LU-19744 that are conflicting with this one. It might be that this "LU-0000" patch was intended as a test, but should have been marked with "fortestonly" or "ignore" to avoid attention...
The other patch indeed has similar changes as this one (and the same issues around nodemap_add vs. nodemap_new). Since I already fixed that here, it may make sense to fold the changes here into the other patch. Either way, this patch looks good. Let's see what Patrick says.
Rather than adding sub-commands to lctl, it is better to add a separate manage for it. That allows proper formatting, subsections, examples, etc.
LU-930 doc: man page improvements Various man page improvements - adding missing commands and options, fixing typos, fixing format issues, etc. Includes one trivial fix to lctl.c. Focuses on man8. This code was generated by Augment. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I222ea824b25b0a3104dc4a8bc8f56c10449045e4
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
And look, we can finally get rid of this terrifying (and correct!) TODO ....... eek
LU-18553 llite: ensure layout refresh on fast read It is essential to refresh the layout before doing a fast read, otherwise we could read stale data if the layout has changed and, eg, the mirror our data is from is now stale. Today, we do this refresh incidentally in file_read_confine_iter->cl_io_init->vvp_io_init, but this is obviously fragile to future changes, since it's not directly associated with the fast read path. Add dedicated code in the fast read path to refresh the layout. Also opportunistically rename a few functions to make clear they are Lustre functions and not kernel functions. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ifbd827b79847309fe8d798963774ccd6650ad22f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
Since this is just a delta, why not use jiffies instead? u64 kstart = get_jiffies_64(); ... ll_stats_ops_tally(ll_i2sbi(file_inode(file)), LLPROC_LL_READ, jiffies_to_usecs(get_jiffies_64() - kstart));
LU-19344 llite: use ktime_get_coarse for read stats Replace ktime_get() with ktime_get_coarse() for read timing stats in do_file_read_iter. ktime_get() reads the hardware clocksource on every call, which is expensive on virtualized guests - pvclock_clocksource_read was the #1 CPU consumer at 9.16% of the tiny read profile on a KVM guest. Virtualization is not niche; all major cloud providers run KVM or similar hypervisors, so this cost is broadly relevant. ktime_get_coarse() reads a cached jiffies-granularity timestamp (~1-4ms resolution), avoiding the hardware read entirely. The coarse granularity is sufficient for the aggregate min/max/sum/count stats collected by ll_stats_ops_tally - these stats summarize thousands of ops, so per-op microsecond precision is not needed. Also move kstart assignment after the zero-count early return to avoid unnecessary work. perf profile with ktime_get_coarse (KVM, pvclock): pvclock_clocksource_read: 0.02% (from other callers) ktime_get_coarse_ts64: 0.74% Benchmark (8-byte sequential reads, 2M iterations): Before (ktime_get): ~3,600k reads/sec After (ktime_get_coarse): ~4,100k reads/sec (+14%) Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1c25e14b2fbb9a6f7f4f6f999aa9f2c6e2a0bb8c
LU-19744 doc: bulk man page review Bulk review done by Augment and Claude Code. Complete systematic review of Lustre man pages (sections 5, 7, 8) with fixes for documentation errors, missing options, formatting issues, and cross-references. Key fixes include: - Add missing options: --nomgs, --nidsfile, -q/--quiet, --catalog, -l, and others documented in usage but missing from man pages - Fix formatting errors: .TH syntax, incomplete sentences, SEE ALSO sections - Add missing cross-references to related man pages - Fix typos and grammar errors - Improve consistency across man pages - Add missing NAME section to lnetctl.8 - Fix alphabetical ordering in SEE ALSO sections (18 nodemap files) Files modified: 60+ man pages across sections 5, 7, and 8 Total changes: systematic improvements to documentation accuracy and completeness Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4896b64767e58c2b939b5a70db394a6f920949d5
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-19344 llite: inline fast path of ll_stats_ops_tally Inline the common case of ll_stats_ops_tally() for the default STATS_TRACK_ALL mode with per-CPU stats. This eliminates the out-of-line function call chain through ll_stats_ops_tally -> lprocfs_counter_add -> lprocfs_stats_lock/unlock on every read and write. The inlined version directly accesses the per-CPU counter struct under get_cpu()/put_cpu(), updating count, sum, sumsquare, min, and max in place. The slow path (__ll_stats_ops_tally) handles filtered tracking modes (PID/PPID/GID) and the NOPERCPU case. perf profile before inlining (KVM tiny writes): lprocfs_counter_add: 1.41% lprocfs_stats_lock: 0.64% ll_stats_ops_tally: 0.14% lprocfs_stats_unlock: 0.05% Total stats overhead: 2.24% After inlining: all four functions gone from profile. Benchmark (8-byte sequential I/O, 2M iterations): Writes: ~2,179k/sec -> ~2,307k/sec (+6%) Reads: ~4,100k/sec -> ~4,335k/sec (+6%) Combined with ktime_get_coarse (patches 1-2): Writes: ~1,970k/sec -> ~2,307k/sec (+17% total) Reads: ~3,600k/sec -> ~4,335k/sec (+20% total) Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie9e79d466fee8401dfc08485896124f918c0c9d4
| unique failing test | history |
|---|---|
| sanity-quota@zfs+DNE:test_12b | seen in 19 other reviews |
| sanity-quota@zfs+DNE:test_17 | seen in 23 other reviews |
| sanity-quota@zfs+DNE:test_25 | seen in 17 other reviews |
| sanity-quota@zfs+DNE:test_33 | seen in 23 other reviews |
| sanity-quota@zfs+DNE:test_34 | seen in 23 other reviews |
| sanity-quota@zfs+DNE:test_37 | seen in 23 other reviews |
| sanity-quota@zfs+DNE:test_38 | seen in 23 other reviews |
| sanity-sec@zfs:test_25b | seen in 31 other reviews |
This patch could run with `Test-Parameters: trivial` if it is refreshed.
LU-17000 lnet: refactor lnet_net_show_dump Refactor lnet_net_show_dump() to improve readability and reduce function size from ~360 lines to ~160 lines. Changes: - Add lnet_ni_dump_ctx struct to hold shared dump state - Extract lnet_ni_dump_one_msg_stats() helper for message stats (send/recv/drop) - eliminates code duplication - Extract lnet_ni_dump_health_stats() helper for health statistics - Extract lnet_ni_dump_tunables() helper for net tunables - Extract lnet_ni_format_cpts() helper for CPT list formatting with proper buffer overflow checking - Extract lnet_ni_dump_extended() to handle all verbosity-dependent NI dumping with linear control flow instead of goto-based skipping The refactoring eliminates goto labels (skip_msg_stats, skip_udsp) by restructuring the verbosity-based logic into clear conditional blocks. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I210f78fdf201a416f16733bc5ba4afdf45e92065
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 15 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-8 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 5 tests failed: sanity-pfl, lnet-selftest, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: replay-ost-single. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-17240 tests: enable parallel mounting by default Enable parallel mounting of targets within types by default. This allows multiple OSTs (or MDTs in multi-MDT configurations) to be mounted simultaneously, but maintains sequential ordering between different target types (MDT before OST). Configuration: - PARALLEL_MOUNT now defaults to 1 (enabled) - PARALLEL_MOUNT_TYPES explicitly set to 0 (no inter-type overlap) - PARALLEL_MOUNT_TARGETS="mdt ost" (MGS excluded) Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results: - Baseline (serial mounting): 29.7s - Parallel mounting within types: 27.3s - Improvement: 8% faster (2.4s savings) Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I71efad13359e3490129ce90dece5fee1d2513c75
| unique failing test | history |
|---|---|
| conf-sanity3@ldiskfs+DNE:test_87 | seen in 7 other reviews |
| conf-sanity3@ldiskfs+DNE:test_88 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_89 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90a | seen in 12 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90b | seen in 15 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90c | seen in 21 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90d | seen in 24 other reviews |
| conf-sanity3@ldiskfs+DNE:test_91 | seen in 29 other reviews |
| conf-sanity3@ldiskfs+DNE:test_98 | seen in 34 other reviews |
| conf-sanity3@ldiskfs+DNE:test_99 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_120 | seen in 13 other reviews |
| conf-sanity4@ldiskfs+DNE:test_161 | seen in 7 other reviews |
| sanity-quota@ldiskfs+DNE:test_91 | seen in 1 other review |
| sanity-quota@ldiskfs+DNE:test_92 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_93 | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_94 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_95a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_95b | seen in 4 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 13 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
LU-17240 tests: enable parallel formatting by default Enable parallel formatting of targets by default. This includes both intra-type parallelism (multiple MDTs or OSTs formatted simultaneously) and inter-type parallelism (MDT and OST formatting overlapping). Configuration: - PARALLEL_FORMAT now defaults to 1 (enabled) - PARALLEL_FORMAT_TYPES inherits from PARALLEL_FORMAT (=1) - PARALLEL_FORMAT_TARGETS="mdt ost" (MGS excluded) - Other operations (mount, unmount) remain disabled Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results: - Baseline (serial formatting): 31.1s - Parallel formatting: 17.3-25.7s - Improvement: 17-44% faster Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7cac5b4a677e3fee97edf57bb9a7cf3469bab92b
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 15 tests. 5 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck, runtests. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 17 tests. 5 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck, runtests. This build will be s | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: sanity-scrub, sanity-pcc. | session |
| review-dne-part-8 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 15 tests. 6 tests failed: sanity-pfl, lnet-selftest, pjdfstest, sanity-sec, sanity-lfsck, runtests. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: replay-ost-single. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-pcc. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 8 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-17240 tests: enable parallel unmounting across types Enable inter-type parallelism for unmount operations, allowing clients, MDTs, and OSTs to unmount simultaneously rather than waiting for each type to complete sequentially. WARNING: This feature is known to have stability issues and can cause hangs during unmount. It is enabled here for testing purposes only and should not be used in production environments. Configuration: - PARALLEL_UMOUNT_TYPES now inherits from PARALLEL_UMOUNT (=1) - Allows client, MDT, and OST unmounts to overlap Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results (when it works): - Baseline (sequential types): 24.6s - With inter-type overlap: 11.9s - Improvement: 51% faster (12.7s savings) Known issues: - Intermittent hangs during unmount operations - Race conditions in cleanup ordering - Resource deadlocks between target types Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I413c5be0437049c88a6c1066681ed258fa41fa06
| unique failing test | history |
|---|---|
| conf-sanity2@ldiskfs+DNE:test_73c | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_73e | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_74 | seen in 12 other reviews |
| conf-sanity2@ldiskfs+DNE:test_75 | seen in 9 other reviews |
| conf-sanity3@ldiskfs+DNE:test_87 | seen in 7 other reviews |
| conf-sanity3@ldiskfs+DNE:test_88 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_89 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90a | seen in 12 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90b | seen in 15 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90c | seen in 21 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90d | seen in 24 other reviews |
| conf-sanity3@ldiskfs+DNE:test_91 | seen in 29 other reviews |
| conf-sanity3@ldiskfs+DNE:test_98 | seen in 34 other reviews |
| conf-sanity3@ldiskfs+DNE:test_99 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_103 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| conf-sanity3@ldiskfs+DNE:test_120 | seen in 13 other reviews |
| conf-sanity4@ldiskfs+DNE:test_154 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| conf-sanity4@ldiskfs+DNE:test_161 | seen in 5 other reviews |
| sanity2@ldiskfs+DNE:test_130i | seen in 3 other reviews |
| sanity-lfsck@zfs:test_18a | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18c | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18d | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18e | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18f | seen in 3 other reviews |
| sanity-lfsck@zfs:test_18g | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18h | seen in 3 other reviews |
| sanity-lfsck@zfs:test_20a | seen in 2 other reviews |
| sanity-lfsck@zfs:test_20b | seen in 2 other reviews |
| sanity-lfsck@zfs:test_45 | seen in 1 other review |
| sanity-quota@ldiskfs+DNE:test_59 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_60 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_62 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_64 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_66 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_67 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_68 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_69 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_70a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_70b | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_71a | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_71b | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_72 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_73a | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_73b | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_74 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_75 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_76 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_77 | seen in 7 other reviews |
| sanity-quota@ldiskfs+DNE:test_79 | seen in 6 other reviews |
| sanity-quota@ldiskfs+DNE:test_80 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_81 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_82 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_83 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_84 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_85 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_86 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_87 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_89 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_90a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_90b | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_1b | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_1c | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_2 | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4a | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4b | seen in 10 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4c | seen in 12 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4d | seen in 16 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4e | seen in 13 other reviews |
| sanity-scrub@ldiskfs+DNE:test_5 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_6 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_7 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_8 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_9 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_10a | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_11 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_14 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_15 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_17a | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_17b | seen in 5 other reviews |
| sanity-sec@ldiskfs+DNE:test_75b | seen in 3 other reviews |
| sanityn@ldiskfs+DNE:test_104 | seen in 3 other reviews |
| sanityn@ldiskfs+DNE:test_106a | seen in 2 other reviews |
| sanityn@ldiskfs+DNE:test_115 | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. %% THIS TEST SESSION CRASHED % | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 15 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-8 crashed | RHEL 8.10 / x86_64 | ran 2 tests. 1 tests failed: lustre-initialization. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 5 tests failed: sanity-pfl, lnet-selftest, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: replay-ost-single. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-17240 tests: enable parallel unmounting by default Enable parallel unmounting of targets within types by default. This allows multiple clients, OSTs, or MDTs to be unmounted simultaneously within their respective type groups, but maintains sequential ordering between different target types (clients, then MDTs, then OSTs). Configuration: - PARALLEL_UMOUNT now defaults to 1 (enabled) - PARALLEL_UMOUNT_TYPES remains 0 (no inter-type overlap) - PARALLEL_UMOUNT_TARGETS="client mdt ost" Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results: - Baseline (serial unmounting): 30.9s - Parallel unmounting within types: 26.5s - Improvement: 14% faster (4.4s savings) Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Icbd5fa4fc79259225b03a9feab0bc50b4b48b09d
| unique failing test | history |
|---|---|
| conf-sanity2@ldiskfs+DNE:test_73c | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_73e | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_74 | seen in 12 other reviews |
| conf-sanity2@ldiskfs+DNE:test_75 | seen in 9 other reviews |
| conf-sanity4@ldiskfs+DNE:test_154 | seen in 1 other review |
| conf-sanity4@ldiskfs+DNE:test_161 | seen in 5 other reviews |
| sanity2@ldiskfs+DNE:test_130i | seen in 3 other reviews |
| sanity3@zfs:test_271f | seen in 10 other reviews |
| sanity-lfsck@zfs:test_18a | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18c | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18d | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18e | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18f | seen in 4 other reviews |
| sanity-lfsck@zfs:test_18g | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18h | seen in 3 other reviews |
| sanity-lfsck@zfs:test_20a | seen in 2 other reviews |
| sanity-lfsck@zfs:test_20b | seen in 2 other reviews |
| sanity-lfsck@zfs:test_45 | seen in 1 other review |
| sanity-quota@ldiskfs+DNE:test_59 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_60 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_62 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_64 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_66 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_67 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_68 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_69 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_70a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_70b | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_71a | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_71b | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_72 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_73a | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_73b | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_74 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_75 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_76 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_77 | seen in 7 other reviews |
| sanity-quota@ldiskfs+DNE:test_79 | seen in 6 other reviews |
| sanity-quota@ldiskfs+DNE:test_80 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_81 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_82 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_83 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_84 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_85 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_86 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_87 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_89 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_90a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_90b | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_1b | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_1c | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_2 | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4a | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4b | seen in 10 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4c | seen in 12 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4d | seen in 16 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4e | seen in 13 other reviews |
| sanity-scrub@ldiskfs+DNE:test_5 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_6 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_7 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_8 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_9 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_10a | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_11 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_14 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_15 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_17a | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_17b | seen in 5 other reviews |
| sanity-sec@ldiskfs+DNE:test_75b | seen in 3 other reviews |
| sanity-sec@ldiskfs+DNE:test_82 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_104 | seen in 3 other reviews |
| sanityn@ldiskfs+DNE:test_106a | seen in 2 other reviews |
| sanityn@ldiskfs+DNE:test_115 | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 15 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-8 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 5 tests failed: sanity-pfl, lnet-selftest, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 2 tests failed: sanity-quota, replay-ost-single. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-17240 tests: enable parallel mounting across types Enable inter-type parallelism for mount operations, allowing MDT and OST mounting to overlap rather than waiting for MDT to complete before starting OST mounts. Configuration: - PARALLEL_MOUNT_TYPES now inherits from PARALLEL_MOUNT (=1) - Allows MDT and OST mounts to run simultaneously Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results: - Baseline (sequential types): 24.9s - With inter-type overlap: 26.8s - Result: 8% slower (1.9s penalty) Note: Inter-type mount parallelism shows a small performance degradation in this configuration. The overhead of running MDT and OST mounts simultaneously appears to outweigh any parallelism benefits, likely due to resource contention or mount dependencies. This feature is enabled for completeness and may show benefits in larger configurations. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9981fc1a56d5196e311a95f9808cb14a163f09c3
(style) line length of 84 exceeds 80 columns
(style) line could be split after '||'
LU-17240 tests: fix dm-flakey device handling The recent parallel unmount commit attempted to fix device name functions to return the actual mounted device for dm-flakey support, but this created circular dependencies during initialization that caused hangs. The proper solution is to add a new facet_real_dev() function that returns the currently mounted device (which could be a dm-flakey device) when the facet is mounted, or falls back to the logical device name when unmounted. This is needed because direct device access tools like debugfs, tune2fs, dumpe2fs, and e2fsck need to access the actual mounted device (dm-flakey) to see current filesystem state, not the underlying physical device which may have stale data buffered in the dm layer. Updated all test code that uses these tools to call facet_real_dev() instead of ostdevname() or mdsdevname() to get the correct device for direct access. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8d9785351904fc4f21721f872a8cdce7958159a5
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-quota@zfs:test_48 | seen in 24 other reviews |
LU-15367 tests: Multiop allow mmap control Add the ability for multiop to specify where to access a file when it's mmaped by repurposing the 'w' and 'r' options when a file is mmapped. This makes multiop able to easily simulate application behavior using processed iotrace logs as input. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6ff455f15a59132018525410c7fcce840c5b6209
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: recovery-small. | session |
LU-15069 llite: remove ras_align ras_align is quite odd - it aligns to either RPC size, which has some justification, or to window size, which is totally strange. Window size has nothing to do with alignment and shouldn't be used for this at all. And ras_align rounds *down*, which results in extra misses because it's shrinking the readahead window selected by the rest of the readahead logic. Finally, although aligning readahead to RPC boundaries sounds nice, it makes readahead itself far more complicated by messing up the math for offsets and window sizes, for limited benefit: It is not very important for RPCs to be *aligned* so long as they are *large*, which is handled by the rest of the readahed logic. This significantly cleans up some of the readahead behavior and fixes the misses introduced by rounding down. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I267076a79120e145f49a4b2ffdeff97b4f2b158b
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-zfs | RHEL 8.9 / x86_64 | ran 8 tests. 1 tests failed: replay-single. | session |
LU-17473 llite: wait for partially successful aio For various reasons (notably conflicting buffered IO), we may need to fall back from DIO to buffered IO. This also affects AIO, and if it happens, we will sometimes submit only part of an AIO with the AIO path, completing the rest with the buffered path. Userspace doesn't expect this, expecting us to either do all or none of the IO with AIO, so it doesn't wait for completion in this case. To meet this expectation, we must recognize this case and wait for AIO to complete before returning to userspace. Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Iac7abac3bd01f027c353120483932a62c6475277
It would be even more useful if `i` was "number of iterations for the next command"?
The `J` option has already been used by madvise(HUGEPAGE), and 'j' is used by this patch, though `j` = "jump" would probably make more sense and flock can use something else, maybe `I`?. It would be good to get this patch landed, since a number of test cases appear to be using the
LU-12645 tests: Add read ahead tests The existing readahead tests are too lenient, not checking misses at all. They also do not cover enough cases, and use a complex special purpose utility, which takes in to account stripe size, etc. This is overly complex and not really correct - strided read patterns don't have to have any relation to stripe size. Instead, we can just modify multiop to support writing or reading a certain number of times, optionally jumping between each operation. This allows describing any possible strided pattern with just four arguments - iterations, size, jump, and starting offset. (It's also possible to use this for backwards reads.) This patch uses multiop to add a short but strict & varied set of tests for sequential and strided readahead. It also simplifies/replaces many of the existing readahead tests, which did odd things like disable the cache on the server, which is irrelevant to client side readahead. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Wang Shilong <wshilong@ddn.com> Change-Id: I774585a17deac8c0f3b25ddbe047617177f3caf3
LU-15367 scripts: Add iotrace to multiop script With the standardization of iotrace and multiop having the ability to take input from a file, we can start directly translating iotrace recordings to multiop input. This allows us to use multiop to simulate the I/O call sequences of an iotrace recording. There are a number of limitations currently, which we may choose to fix later (if this turns out very useful): 1. No support for multiple open files (multiop limitation) 2. Can only play-back one thread at a time We use the ability to go from command to iotrace recordin back to multiop command to test this functionality. Test-Parameters: trivial Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I2ec4f358c97ceb15b717342af5cc9854b9c60677
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 48 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 7 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 6 tests. 2 tests failed: lustre-initialization, replay-dual. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 13 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 1 tests. 1 tests failed: node-provisioning. %% NODE-PROVISIONING FAILED MULTIPLE TIMES FOR review-dne-zfs | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: recovery-small. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu failed 2× | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 3 tests failed: sanity-sec, sanity-lnet, sanity. | session |
LU-13814 osc: Move osc_page members to osc_async_page We're going to start using osc_async_page separately from osc_page, since it's used for both DIO and buffered, but osc_page is not. This moves all of the members which are needed for DIO. This commit deliberately ignores some packing related issues which will be handled *much* later, since the names and members of these structs will keep changing. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I994af9563142201a2c7193735ac02568735bd8d5
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 2 tests failed: sanity-lnet, sanity-sec. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-dom. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: recovery-small. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 2 tests failed: node-reset, sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 2 tests failed: sanity-lnet, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
Note to self: I am not 100% sure this is finding the right object, but it should blow up if not. Might need a cl_object_top() here if it does.
LU-13814 osc: remove use of cp_obj Since we're going to lose the cl_page, we need to remove usage of its members from the code which handles DIO pages. This removes cp_obj usage from the one place where it's used by the DIO code, by adding it to the BRW async args. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic53b872d930305c345a03c75dc21a613874bf3c8
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 12 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: recovery-small. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
LU-13814 clio: use osc_dio_completion everywhere The conversion to osc_dio_completion was incomplete because some other code wasn't ready. Finish that conversion. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id6500bfb55dc27e783a91f58498f9a13906056b8
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 8 other reviews |
| sanity-sec@zfs:test_59a | seen in 9 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 2 tests failed: sanity-lnet, sanity-sec. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: insanity. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
LU-13814 osc: Remove usage of cdp_cl_pages As part of eliminating cl_page in the DIO path, we need to clear out all uses of it. This is one more minor one - there's no need to clear this list before returning from this function, because if it fails we give up entirely. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9dc1053c542ce7a903a93f7b9a1fb0bfc6ac1641
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 7 other reviews |
| sanity-sec@zfs:test_59a | seen in 9 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 2 tests failed: sanity-lnet, sanity-sec. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: lustre-initialization, replay-single. | session |
| review-dne-part-8 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
So I think I figured out what's weird here, but check me if I'm wrong. I don't think it makes sense to both have padding bits AND the ((packed)) attribute? Basically the effect of the padding bits is to prevent ((packed)) from taking effect. Neighboring bitfields are combined without the ((packed)) attribute - at least in my observations - but the alignment requirements (the requirements for good performance, that is) are respected. When you add ((packed)), the alignment requirements are ignored. Adding the packing bits basically cancels that out again. Without packed and without the packing bits, we seem to get the desired packing while respecting recommended alignment. Thoughts?
I think "packed" is needed to combine fields across data structures. However, it is less useful for in-memory data structures so if it is no longer needed it could be removed. Did you check the structs without "packed" with pahole?
Does this struct need to be long aligned? Wondering if 4 byte alignment is an issue for some 64 bit arch
The alignment here more generally is a bit messy - because we chose packed to prefer memory efficiency, we're not padding. Much later, I have a portion of this series which removes every member of this struct and handles packing other stuff better. I'll see about integrating some of it earlier in the series.
These bitfields are handled by the compiler, this isn't the same as the "bitfield" macros used by the kernel that need to be "long" variables.
LU-17063 osc: remove duplicate info The from/to in osc_page replicate the info provided by oap_page_off and oap_count in osc_async_page, so just use those. Getting the full benefit of this requires removing the padding, but with that done, this gets us a full 8 bytes of size reduction in osc_page (and therefore cl_page) size. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If9f2ea5abfe1da6e586072f22c5e0758988b7760
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 47 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-hsm@zfs:test_254b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-13814 osc: rename osc_async_page osc_async_page isn't for async IO - it's for all data IO. Rename it osc_transfer_page so the name fits the usage. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I930aae585763f95d9085bea179765a0431bccf52
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-sec@zfs:test_59a | seen in 9 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-13814 osc: rename osc_prep_async_page This is another piece of renaming osc_async_page to osc_transfer_page. This is kept separate to make the previous patch as focused as possible. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ifd57eec46aeeb059ac836e09aa47322a69cc1493
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 7 other reviews |
| sanity-sec@zfs:test_59a | seen in 9 other reviews |
LU-13814 osc: rename osc_async_flags Async flags are used for all osc pages, rename them accordingly. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ibf808bcad5f03d950b4d2d05fb11f047fd28f311
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-flr@zfs:test_70a | seen in 75 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 5 other reviews |
LU-13814 clio: add cp_inode to page allocation cp_inode can be set correctly during page allocation, rather than after. This is a prelude to moving cp_inode to the osc_transfer_page, but that's better done in a separate patch. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I509f6cfbae8e5a6ec6b07c8253d68f6dd2794e59
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18f | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-lfsck@ldiskfs+DNE:test_18g | seen in 4 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18h | seen in 4 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 6 other reviews |
LU-13814 clio: move cp_inode to transfer page As part of moving DIO to use only the transfer page and not cl_page, we need to eliminate uses of cl_page where we have a transfer page available. That requires moving cp_inode to the transfer page. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If0312c7fa22501b789437479fadb023f09f341b8
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-flr@zfs:test_70a | seen in 74 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 7 other reviews |
LU-13814 osc: replace cra_page cra_page requires there to be a cl_page associated with an OSC transfer page. Since we're breaking that association, we replace it with the page index, which can do what's required. Test-Parameters: forjanitoronly Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1163dae32bce6ae9fcc458251c047f05ddfa6ec1
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 47 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
LU-13814 clio: move cp_page_index to transfer page cp_page_index is needed for both DIO and BIO, so it has to move to the OSC transfer page. Test-Parameters: forjanitoronly Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2eceda45e3ac0b92973362b5427948fd1163adc9
LU-13814 osc: add DIO/BIO related asserts These will be adjusted when cp_type is moved to the otp page, but for now, these asserts help ensure we don't have any DIO pages in unexpected locations. Test-Parameters: forjanitoronly Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9342a502e0195d86af5f6826e5677b96f6b57f20
LU-0000 tgt: async write commit A first and insufficient try at server side async. For hybrid, we should in fact do everything async except lock acquisition. That means splitting the handler in to a sync and async portions. Test-Parameters: forbuildonly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: Iba526e4d75b41992ea3ed77bb3c2a76fa9a1c41b
It would be nice to have a sanity test (or perhaps a small sanity-gds.sh) that exercises this using real hardware, for those that have it available. That's outside the scope of this patch, but this test case would be good to have in such a test suite.
LU-13805 llite: fail unaligned DIO for RDMA pages Unaligned DIO needs to directly access the page contents in order to copy to the buffer. This means it can't work with RDMA only (non-CPU accessible) pages. Implement that limitation. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I52bd1d4cc143e1018ddf6942403142f26be4430f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 13 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 13 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: recovery-small, lustre-rsync-test. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: recovery-small. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
LU-19223 shrinkers: Add nr_scanned to all shrinkers If the Lustre shrinkers can't free any pages but don't set nr_scanned, they may be called forever by the kernel - see do_shrink_slab() in the kernel. Add nr_scanned support to the remaining Lustre shrinkers: - LDLM pools server and client shrinkers - Lu site shrinker - Page pools shrinkers Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ica35a0dabe1dce78fd3cd29174ef142a965be824
| unique failing test | history |
|---|---|
| conf-sanity1@ldiskfs+DNE:test_30a | seen in 1 other review |
| recovery-small@ldiskfs+DNE:test_67 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@zfs:test_21 | seen in 2 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-1 crashed | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 crashed | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 crashed | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 crashed | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-5 crashed | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-zfs crashed | RHEL 8.10 / x86_64 | ran 10 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
LU-0000 osc: ghost index entries for deleted pages Replace OSC radix tree page entries with a ghost value that encodes the page index when pages are deleted. On insert, detect and replace such ghosts with the real page. This allows us to discard these 'ghost' pages after they've been removed. Add COIO_PCACHE_TRUNCATE and handle it in vvp to truncate pagecache by [start,end] page indices for a cl_object. Aggregate contiguous ghost indices in OSC discard paths and invoke cl_object_inode_ops() to drop pagecache efficiently. Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9f1d9a383e267fd6f1d8419d180f99bdb71d6667
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 crashed | RHEL 9.4 / x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-1 crashed | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 2 tests failed: sanity-flr, sanity-dom. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 9.4 / x86_64 | ran 13 tests. 2 tests failed: sanity-flr, sanity-dom. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 2 tests failed: sanity-flr, sanity-dom. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 9.4 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, RHEL 9.4 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu crashed | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
if want_composite is true, then we need to set mirror_count to 1, as there is an assertion to make sure mirror_count == 0 only for plain layout file in lod_fill_mirrors() as LU-18962 shows.
Instead of making this a
LU-17159 lod: mark file layouts with append striping Add LCM_FL_APPEND flag to file layouts when a file is opened with O_APPEND flag. This makes it easier to diagnose layout behavior when the MDS decides to use append-specific layout, which can override the default file layout. Add dah_flags to dt_allocation_hint structure to pass MDS_OPEN_APPEND flag from mdd_object_make_hint() to lod_ah_init(). The flag is set in mdd_object_make_hint() when MDS_OPEN_APPEND is present in the open flags, and is then used in lod_ah_init() to set the LCM_FL_APPEND flag in the layout. Also added BUILD_BUG_ON tests for LCM_FL flags in wiretest.c to ensure the flag values remain consistent. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I927faae3a385a9d0acf40a37ab1be0c0d4cbb82c
LU-0000 lod: initial implementation This is an initial and partially incorrect implementation, which I'll be tearing up a bit until we can actually get layout creation to work. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I229b5751b60f8078e22dbb4ed3e2e205d3e4c9a1
| unique failing test | history |
|---|---|
| conf-sanity3@ldiskfs+DNE:test_135 | seen in 20 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 2 other reviews |
| sanity-flr@zfs:test_200b | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs crashed | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. %% THIS TEST SESSION CRASHED %% | session |
LU-17433 osc: simplify osc_lock_set The presence of osc_lock_set_writer and osc_lock_set_reader and their complexity appears to be a holdover from the old CLIO, before the 2.7 era CLIO simplification. Most of the checks in the functions are unnecessary - we can't get here unless it's the same object and the range of the lock matches that of the IO. These may've been needed with the older more complicated CLIO locking, but they're not needed now. Clean all that up to make the code a bit more readable. Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I728039e18834eb08d9eb1f3492f8001c2a12f52b
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_398c | seen in 1 other review |
since it looks like this patch will get refreshed eventually, please also fix this typo -> divide.
truncate
I have a doubt here, if the start of the 1st page is not page aligned, since @to most of time is PAGE_SIZE, so at each loop, the page is always not page aligned as well, is it a glitch here?
Should @to be like this:
if (from != 0)
to = min(PAGE_SIZE - from, from + size);
else
to = min(PAGE, from + size);
and except the 1st/last page, all pages in the middle should always be aligned full page.
No changes here, code is correct as is.
from here we can see that the end of the 1st page is page aligned.
and the end of the last page could be page unaligned.
Can we do: unaligned_dio || skip "Need unaligned dio support" Instead of server version check?
More importantly, for older servers the unaligned DIO should be handled gracefully in some manner, since applications running on newer clients will try this and we shouldn't allow them to crash the servers. Just skipping the test doesn't solve the interop problem. One option for UDIO is to expose the DIO alignment via statx, as XFS does. That will allow userspace applications to know what the DIO alignment requirements are, and applications that don't follow them can return EIO or other error directly (as they did befoer UDIO existed, instead of trying to emulate their way through the issues.
LU-17993 debug: to reveal page count issue as title Test-Parameters: fortestonly Test-Parameters: testlist=sanity env=ONLY=119h,ONLY_REPEAT=500 clientdistro=el8.8 serverdistro=el8.8 Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com> Change-Id: Ia502400fb20603c369c34a3fb397a472cc3403c3
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 9.3 / x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
LU-0000 obdclass: Refactor cl_object_attr_update The object named "top" is not actually a top object - it's any cl_object and we navigate the list accordingly. Clarify. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6043f0e6c1705cfeeece62cd71f2208b2197f5cc
| unique failing test | history |
|---|---|
| replay-single@ldiskfs+DNE:test_65a | seen in 66 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-6 crashed | RHEL 9.4 / x86_64 | ran 6 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
LU-13814 llite: remove unnecessary smp_mb() This smp_mb() was added as part of: https://review.whamcloud.com/c/39542 To help protect the csi_sync_nr atomic. But it was not justified there, and is not in fact needed. IO submission has *several* memory barriers before another thread can see the IO. (Note there is a discussion on that Gerrit suggesting it was unneeded.) For example: osc_queue_sync_pages uses a spinlock to put the extent on the list(s) (after this point, it's available for ptlrpc to make an RPC). There are several other instances of spinlocks, AND the use of csi_sync_nr is under memory barriers (in atomic_dec_and_lock, or under a spinlock). Let's remove this. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8b81f8e02025cae801a980d2856993c6d4023716
| unique failing test | history |
|---|---|
| sanity1@zfs:test_24oa | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.4 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.4 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.4 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-18843 mdt: parallel rename in a single directory Testing... Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I920f9c0624ef25469ece046244d97623018acd63
LU-17831 osc: batch discard for write locks Batch page discard when cancelling a write lock. This roughly halves the time to cancel pages under a write lock. A future patch will do this for read locks as well. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I214f6babb69fc2117379490efe3d2d62b8122d90
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-3 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
LU-11532 tests: Improve cancel_lru_locks debug Cancel_lru_locks is almost always (always?) used to drop all locks in a namespace, not just the unused ones. A very common test failure is when a lock is not cleared because it is unexpectedly still in use. Improve the cancel_lru_locks function to report this case. A quick scan of the test-framework suggests nothing is relying on having null output from this function, so this should be safe even if some tests are expecting some locks to be in use (and so remain after this call). This should improve debugability of failures like LU-11532. Signed-off-by: Patrick Farrell <paf@cray.com> Change-Id: Idbb62b9a8881c19ae135bdb1cb22f366d236b43b
LU-12782 llite: Convert attr lock to rwlock Under some shared file workloads, the cl_object_attr_lock ends up 'hot'. Because it is a spinlock which is often used only for reading, it can easily be converted to an rwlock. This should show up in some shared file workloads, notably shared file reading. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I50d6f04f31eeea3ab5af58a1b6b56c1d4cfc7093
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_39j | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_64f | seen in 37 other reviews |
| sanity2@ldiskfs+DNE:test_133c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_398g | seen in 13 other reviews |
| sanity1@zfs:test_42e | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@zfs:test_45 | seen in 4 other reviews |
| sanity-benchmark@ldiskfs+DNE:test_fsx | seen in 17 other reviews |
| sanity-benchmark@ldiskfs+DNE:test_fsx_partial_punch | seen in 8 other reviews |
| sanity-dom@ldiskfs+DNE:test_fsx | seen in 7 other reviews |
| sanity-dom@zfs:test_fsx | seen in 7 other reviews |
| sanity-dom@zfs:test_42e | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-dom@zfs:test_4 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_70a | seen in 53 other reviews |
| sanity-hsm@zfs:test_3 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_16k | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_18 | seen in 1 other review |
| sanityn@zfs:test_16a | seen in 3 other reviews |
| sanityn@zfs:test_16b | seen in 3 other reviews |
| sanityn@zfs:test_16k | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@zfs:test_18 | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-4 | RHEL 9.3 / x86_64 | ran 9 tests. 2 tests failed: sanity-flr, sanity-dom. | session |
| review-dne-part-5 | RHEL 9.3 / x86_64 | ran 6 tests. 2 tests failed: sanityn, lustre-rsync-test. | session |
| review-dne-part-6 | RHEL 9.3 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.9 / x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.9 / x86_64 | ran 9 tests. 1 tests failed: sanity-dom. | session |
| review-dne-zfs-part-5 | RHEL 8.9 / x86_64 | ran 6 tests. 2 tests failed: sanityn, lustre-rsync-test. | session |
| review-dne-zfs-part-6 | RHEL 8.9 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-zfs-part-7 | RHEL 8.9 / x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. | session |
| review-ldiskfs | RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64, RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.9 / x86_64 | ran 10 tests. 2 tests failed: replay-single, sanity-flr. | session |
LU-13419 osc: Improve speed of enter_cache_try When doing writes to many files, one bottleneck on a client currently seems to be the grant code, specifically spinning in the lock around: osc_enter_cache_try The contention is *just* on osc_enter_cache_try, so there's no obvious way to refactor the lock, etc. Instead, we can look at where time is going in the function. Two things that stand out: obd_dirty_pages is an atomic, and expensive: In my perf tracing, the add_return to this is 50% of the time in this function. This can be replaced with a percpu_counter. These benchmark #s are with the earlier version of the patch, which mistakenly replaced the atomic with a bare unsigned long. I'm not currently able to benchmark the percpu_counter, but it should be similar. mpirun -np 36 $IOR -o $LUSTRE -w -t 1M -b 2G -i 1 -F That's 36 processes on one client, writing to separate files. Before patch: 5942 MiB/s After patch: 14950 MiB/s Looking in perf, the change is huge: I go from spending 60% of the time in osc_enter_cache_try to around 30%, but that's while moving 2.3x the amount of data per second. Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: If5a69b906c6b56786e6a06dccc723781591419e8
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-6 crashed | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %% | session |
LU-14838 ldlm: Disable lockless on contention The contention detection code is mostly broken, and if it ever returns -EUSERS to the client, the client will crash or corrupt user data. The code is being retained because a rewrite is in flight and it would be much harder to do that if the code were fully removed. But let's disable it at least. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I3128288df8ddd39d2875c817830f7a1884c0e763
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-1 | CentOS 8.3/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | CentOS 8.3/aarch64, CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-14882 tests: Fix S_NOSEC tests
The S_NOSEC tests were using incorrect fail_loc values, and
also did not work correctly with correct fail_loc values.
Correct the fail_locs and fix the tests.
Properly fixing the tests required an odd bit of behavior:
Userspace cannot normally tell if we took a lock in Lustre
or not. This was the problem with the earlier tests:
A successful tests was identical to normal operation, so
it was missed that the tests did not work.
The solution is to return an error when we detect the
*correct* behavior (using a fail loc). This allows the
test to clearly tell the difference between a successful
test and normal operation.
Fixes: 8bc4b26453 ("LU-8656 vvp: Add S_NOSEC support")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3650bb304b7548ba72d2c1812b30c0217883a441
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-part-4 | CentOS 8.3/x86_64 | ran 11 tests. 2 tests failed: sanity-flr, sanity-dom. | session |
| review-dne-selinux-ssk-part-1 | CentOS 8.3/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | CentOS 8.3/x86_64 | ran 11 tests. 1 tests failed: sanity-dom. | session |
| review-ldiskfs | CentOS 8.3/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | CentOS 8.3/aarch64, CentOS 8.3/x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | CentOS 8.3/x86_64, Ubuntu 20.04/x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-zfs crashed | CentOS 8.3/x86_64 | ran 10 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
LU-14887 llite: Add DIO splitting tunables The new faster DIO path is great, but benefit is only seen if I/O is split to multiple RPCs. Currently, this only happens if I/O size is > RPC size or if I/O crosses a stripe boundary. This requirement for splitting means that there is an inherent conflict between the desire to do large RPCs and doing single stream I/O at high speed. This patch adds a pair of tunables, turning on some I/O splitting by default while allowing users to control the degree. DIO parallelism, at the llite layer, specifying how many chunks we should try to split a DIO In to Minimum preferred I/O size, at the OSC layer, specifying the minimum size to which we should split I/O. Parallelism is a global control of how much splitting is desired for best performance, the overall preference between maximum RPC size and maximum single stream performance. The OSC level control is because some OSTs have dramatically different performance with synchronous I/O, so for a spinning OST, it may be desirable to enforce a higher minimum I/O size (and so less parallelism), and the reverse for a flash OST. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5ce010d97b0bf6d91d05d36a2c74268432b5f1f2
| unique failing test | history |
|---|---|
| sanity-pfl@zfs:test_15 | seen in 3 other reviews |
| sanity-pfl@zfs:test_16c | seen in 2 other reviews |
| sanity-pfl@zfs:test_17 | seen in 2 other reviews |
| sanity-pfl@zfs:test_18 | seen in 3 other reviews |
| sanity-pfl@zfs:test_19c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | CentOS 8.3/x86_64 | ran 7 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-2 | CentOS 8.3/x86_64 | ran 7 tests. 2 tests failed: sanity-lfsck, replay-dual. | session |
| review-dne-zfs-part-4 | CentOS 8.3/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-zfs crashed | CentOS 8.3/x86_64 | ran 10 tests. 1 tests failed: sanity-quota. %% THIS TEST SESSION CRASHED %% | session |
LU-15069 llite: Move most readahead code to ra.c The readahead algorithm/prediction code is mixed weirdly throughout rw.c, which is mostly code which actually moves data. Because the prediction/window management code is not clearly split from the page reading code, there's still some in rw.c - but this still makes things clearer. There are also a few trivial function renames where names were wrong (eg, RAS instead of RIA), and one extra debug print. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Icc17c057f23233f9f3be8bcf73b9730dfe6b4856
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-zfs-part-2 | CentOS 8.3/x86_64 | ran 7 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-6 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
LU-15483 tests: Reduce I/O sizes A lot of the LU-13799 tests use significantly more I/O than is required for the actual test. In particular, almost all of them use 64 MiB of data for every test, just because the tests started out by copy-pasting. Reduce the I/O size to closer to the minimum required for each test, which will save some time. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ife9a950c28ee09e2be1909cdd07c6f532a5f92bb
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_101j | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101j | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-4 | CentOS 8.3/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-selinux-ssk-part-1 | CentOS 8.3/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | CentOS 8.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | CentOS 8.3/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15663 llite: Simplify readahead stats Readahead stats should be a user readable collection of stats, not a debug dump. Also, even as debug, many of the stats aren't useful. Let's clean it up. RA_STAT_FAILED_FAST_READ is confusing, and it is recorded for every miss. Rename it, and move it to be recorded only for *hits* where we decide not to do fast read for other reasons. 'zero page window' is an almost useless internal detail that makes no sense to users, and has little use even as debug. Let's just remove it. zero file size isn't an interesting readahead stat, it's just a fact about a file. Remove it. 'failed reach end' is also meaningless for users and not useful debug. Remove it. 'readahead to eof' is simply not interesting - readahead reaching the end of the file is a normal part of operation and not a useful stat. RA_STAT_FAILED_MATCH is unused. NB: This is not marked trivial because the stats are used in various tests and we need to verify nothing is broken by these changes. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I24dd543f9703fe5883d774f5e9b3152579494c30
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | CentOS 8.5/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-3 | CentOS 8.5/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-zfs-part-4 | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-ldiskfs-arm crashed | RHEL 8.6/aarch64, RHEL 8.6/x86_64 | ran 8 tests. 1 tests failed: lnet-selftest. %% THIS TEST SESSION CRASHED %% | session |
| review-zfs | CentOS 8.5/x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
LU-15822 ldlm: Add debug in lock_matches 'lock_matches' is a core LDLM function and entirely opaque from a debugging perspective, giving no info on why a lock did not match existing locks. Let's fix this. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: Icd95a6ba0cd99f689d83758d7576ff45e681d49d
| unique failing test | history |
|---|---|
| lustre-rsync-test@ldiskfs+DNE:test_2c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| lustre-rsync-test@zfs:test_2c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_70 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@zfs:test_70 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@ldiskfs+DNE:test_51 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@zfs:test_51 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.6/x86_64 | ran 7 tests. 2 tests failed: sanity-sec, sanity-lfsck. | session |
| review-dne-part-4 | RHEL 8.6/x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-selinux-ssk-part-2 | CentOS 8.5/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | CentOS 8.5/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | CentOS 8.5/x86_64 | ran 7 tests. 2 tests failed: sanity-sec, sanity-lfsck. | session |
| review-dne-zfs-part-4 | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-ldiskfs-arm | RHEL 8.6/aarch64, RHEL 8.6/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs-ubuntu | CentOS 8.5/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs | CentOS 8.5/x86_64 | ran 8 tests. 2 tests failed: sanity-flr, sanity-quota. | session |
LU-15979 llite: restore S_NOSEC in ll_update_inode()
ll_update_inode() is to set S_NOSEC (if needed) after permission
update with help of inode_has_no_xattr() having inode->i_rwsem held.
Without that { creat(); fstat(); write(); } executes to eviction
favorable case:
cl_io_loop
cl_io_lock <- LDLM lock is taken here
cl_io_start
vvp_io_write_start
...
__generic_file_aio_write
file_remove_privs
security_inode_need_killpriv
...
ll_xattr_get_common
...
mdc_intent_lock <- enqueue RPC is sent here
If enqueue rpc is delayed, the client may get evicted as not
cancelling lock taken in cl_io_lock.
ll_update_inode() is called without inode->i_rwsem locked for regular
file but one case:
vfs_setxattr()
inode_lock(inode);
..
ll_xattr_set()
ll_setstripe_ea()
ll_lov_setstripe_ea_info()
ll_intent_file_open()
ll_prep_inode()
ll_update_inode()
where ll_update_inode() is called with inode->i_rwsem locked.
In order to be able to detect this case MDS_OPEN_SETXATTR flag is added.
Test to illustrate the issue is added.
The fix does not help for not NOSEC files.
Uncommenting chmod command in the test makes it to fail with eviction.
Test-Parameters: testlist=replay-dual env=ONLY=34,ONLY_REPEAT=100
Change-Id: Ie9e32d03402027f47381edddbd5cb3fb75023d59
HPE-bug-id: LUS-10989
Signed-off-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-1 | CentOS 8.5/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-ldiskfs | RHEL 8.6/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | CentOS 8.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.6/aarch64, RHEL 8.6/x86_64 | ran 6 tests. 2 tests failed: sanity-lnet, sanity. | session |
| review-ldiskfs-ubuntu | CentOS 8.5/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs | CentOS 8.5/x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-16564 ldlm: Remove cancel on block The LDLM_FL_CANCEL_ON_BLOCK flag was used by liblustre, but isn't used now. The comment on it explains why - it's for clients that can't reply reliably to BL callbacks, which is a disaster waiting to happen (or, not waiting, as the case may be). This should be removed - itss continued presence is confusing (at least to me!). Note: It turns out this is used by lease locks. I think that use may still be irrelevant - let's test and find out. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I608ef0e48999ac3c43d6a395c5351cf3151d897b
LU-16624 tests: Add log scan Add a dmesg log scan as a final test to catch warnings that may have occured in the logs. Let's see how this works and get some feedback, then I'll look at adding it to other test sets. Test-parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1b1e967b515a737c54702a7048429db79746d632
| unique failing test | history |
|---|---|
| recovery-small@ldiskfs+DNE:test_110m | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| replay-single@ldiskfs+DNE:test_70c | seen in 4 other reviews |
| sanity2@ldiskfs+DNE:test_154g | seen in 2 other reviews |
| sanity-pcc@ldiskfs+DNE:test_1d | seen in 3 other reviews |
| sanity-pcc@zfs:test_1f | seen in 2 other reviews |
| sanity-pcc@zfs:test_1g | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-1 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.7/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.7/x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-16656 llite: Improve 'out:' in ll_file_io_generic The location of the 'out:' label in ll_file_io_generic is a bit scary, because it skips part of the setup for io restart, where partial io is taken in to account. This is safe today because 'out' is only used before calls to cl_io_loop, so IO hasn't started yet, but if 'out' is ever used later in the function, it will be incorrect. Let's move it now rather than leave a trap for the unwary. Note that until cl_io_loop is called "io->ci_nob" is 0, so this shouldn't change current behavior. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I89e29708a6d17c5eecdf4f86261dfa013c7a5ec3
LU-16845 obd: rename imp_connect_flags_orig The imp_connect_flags_orig and imp_connect_flags2_orig values both end with "_orig", but there is no corresponding "updated" or other value for them to be original relative to. They seem to be named mirroring ns_connect_flags_orig, where there is also a ns_connect_flags, but this naming makes no sense for the import flags since they're not modified in this way. test-parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1551447b2e92f8cff665dd75c43dd4dde6da9a09
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-1 | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
LU-16858 build: Remove pinger config option Lustre is never built or tested without the pinger, so the config option to build without it should be removed. Note if anyone does want to disable the pinger this can be done at runtime. (This was done historically by Cray, but is not done any more. I suspect no one actually does this, but the option is present if needed.) Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I40aa68fbcee8f68a78316da844951b13bdcb4ffe
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.7/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 crashed | RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-zfs crashed | RHEL 8.7/x86_64 | ran 10 tests. 2 tests failed: replay-single, sanity-quota. %% THIS TEST SESSION CRASHED %% | session |
LU-14639 build: Remove disable-lru-resize config The lru-resize feature is disabled at runtime by setting lru_resize to 0. If it is compiled out or disabled at mount time, then it can never be enabled without recompiling or remounting. Disabling it like this doesn't offer any advantages, and and all customers disabling it are setting lru_size manually already. Let's remove the extra ways to disable lru-resize to avoid confusion and possible bugs. This patch also removes checks in the tests which verify the client and server have lru-resize support. Servers have had lru-resize support for several years, and with this patch, it becomes impossible to build clients without it either. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I801b9eb8cf280e37bc81b3adade7973e295d151f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.8/x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
LU-17210 llite: add kernel readahead asserts Add a set of asserts which confirm kernel readahead is disabled and wasn't used for mmap. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0924f2b1a2bc78e44b9a2082c3dad93a51b4d000
LU-0000 osc: batch osc_consume_write_grant Do osc_consume_write_grant on blocks of transfer pages. Test-Parameters: ignore Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Ieb211c4f7930da1ba431902a9703c22114d8829e
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.4 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
LU-9834 tests: fix loop condition in llapi_layout_test The loop condition in test29 of llapi_layout_test.c was incorrect, causing the loop to never execute. The condition was: for (i = LOV_MAX_STRIPE_COUNT-1; i <= 0; i--) Since LOV_MAX_STRIPE_COUNT-1 is a large positive number and the condition checks if i <= 0, the loop body was never entered. This patch fixes the condition to i >= 0 so that the loop properly executes as intended, starting from LOV_MAX_STRIPE_COUNT-1 and decrementing until it reaches 0. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If96063227542f79004688786f0aeb42e3b27b9d5
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_398k | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity2@ldiskfs+DNE:test_398l | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | CentOS 8.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15483 tests: Use fallocate to fill OSTs Rather than using dd, we can use fallocate to fill the OSTs in 398k and 398l, which should make them much faster. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id65726ccafb3a6e57581b3b14c846ad6b3757c35
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_39r | seen in 19 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.8/x86_64 | ran 7 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-zfs-part-2 crashed | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-zfs-part-6 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
These commands are piling up in a stack, which means they are going to be executed in reverse order IIUC, which gives: 1. wait_mds_ost_sync 2. wait_delete_completed 3. rm -f $DIR/$tfile* Is this what we want?
LU-16704 tests: cleanup after 398l Test 398l fills an OST, so we should have it delete the files it created and wait for delete/sync before the end of the test. Otherwise it can cause ENOSPC on tests that run after it. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5b5916046689c4a016893bc44cdfe9012bd4e987
LU-17433 osc: Make unaligned DIO async Because unaligned DIO is using a copy of the data from userspace, we can make the writes async on the OST side. Because DIO expects that data be 'safe', we cannot just go entirely async and not wait for RPC completion, but we can wait only for RPC completion and not force a commit sync on the server. This is because having our own copy of the data lets us replay the RPC after write() has completed, which is not possible for regular DIO. This reduces the DIO 'O_SYNC' guarantee slightly for unaligned DIO (and other small DIO if we change this). Before this patch, after a DIO write(), the data is fully committed to disk and cannot be lost even if the client and server crash. With this patch, after write() returns, if the server crashes, the client can replay the RPC, and if the client crashes, the server will finish writing out the data. However, if the client and server both crash, the data will be lost. For this reason, we make this behavior tunable: llite.*.dio_full_sync With the default to '0'. Todo: Run through tests Get perf #s for this Write a test verifying this behavior? Test-parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6ab6da2844010df209f219eb9df75c4bbb6e2042
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-4 | RHEL 8.9 / x86_64 | ran 9 tests. 2 tests failed: sanity-hsm, sanity-flr. | session |
LU-17831 osc: discard all if discard one When we cancel a read lock, we check if another read lock also protects the same pages. This is surprisingly expensive. So don't do this for every page: If we find a page which is not covered by a second lock (and therefore must be discarded), we discard all pages after that one. This cuts the time to discard 8 GiB of data under a read lock from 2.2 seconds to 1.2 seconds on a small VM system, so nearly a 50% reduction in time required. We may also do batch discards in the future, which this will permit. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5b5a63c9fcae246fd3db35e613df1cd882544946
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs | RHEL 9.4 / x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
LU-17843 build: correctly create lustre-devel.files The lustre-devel.files file is currently 'created' in the %description section of rpmbuild (the :> command), but that command doesn't do anything in %description. Move this to %install and add a blank line so the file is not empty. Otherwise, the lustre-devel.files file is only populated when we build with --shared, not static. If the lustre-devel.files file is not present OR if it's entirely empty (0 size), this results in an error in rpmbuild. Note the lustre-devel package still has other contents added in the %files directive, so lustre-devel is sound - the issue is just that you can't build RPMs when compiling statically due to the spec file issues. Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: Ic598531e376ce1cd356330023b74ec624b9adea8
| unique failing test | history |
|---|---|
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 5 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-3 | RHEL 9.3/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 9.3/x86_64 | ran 9 tests. 4 tests failed: sanity-quota, sanity-hsm, sanity-flr, replay-ost-single. | session |
| review-dne-part-5 | RHEL 9.3/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-part-7 crashed | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.9/x86_64 | ran 5 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.9/x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-2 | RHEL 8.9/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-3 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. This build will be scored -1 since fortestonly was specified. | session |
| review-dne-zfs-part-4 | RHEL 8.9/x86_64 | ran 9 tests. 4 tests failed: sanity-quota, sanity-hsm, sanity-flr, replay-ost-single. | session |
| review-dne-zfs-part-5 | RHEL 8.9/x86_64 | ran 6 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-dne-zfs-part-6 | RHEL 8.9/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-zfs-part-7 | RHEL 8.9/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. | session |
| review-ldiskfs crashed | RHEL 8.9/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.9/x86_64, SLES 15.5/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-zfs | RHEL 8.9/x86_64 | ran 8 tests. 3 tests failed: replay-ost-single, replay-single, sanity-quota. | session |
LU-0000 tgt: true async write commits This implements async write commit. TODO: - Testing - Add handling of failed commits (not too hard - just store error in export and make next commit forced to sync, like how the client handles the analogous) This is intended to attach to hybrid IO. Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I83e80bfea57bd9780ff5fec10cc4c3e992690584
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 8 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 7 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 10 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 10 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 10 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 10 other reviews |
| sanity1@zfs:test_56x | seen in 40 other reviews |
| sanity1@zfs:test_56xB | seen in 14 other reviews |
| sanity1@zfs:test_56xa | seen in 40 other reviews |
| sanity1@zfs:test_56xab | seen in 11 other reviews |
| sanity1@zfs:test_56xc | seen in 36 other reviews |
| sanity1@zfs:test_56ej | seen in 14 other reviews |
| sanity2@zfs:test_119e | seen in 21 other reviews |
| sanity2@zfs:test_119f | seen in 20 other reviews |
| sanity2@zfs:test_119g | seen in 20 other reviews |
| sanity2@zfs:test_119h | seen in 20 other reviews |
| sanity2@zfs:test_119p | seen in 11 other reviews |
| sanity2@zfs:test_119q | seen in 11 other reviews |
| sanity2@zfs:test_398o | seen in 25 other reviews |
| sanity2@zfs:test_398s | seen in 11 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 13 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 72 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 31 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 33 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_38 | seen in 34 other reviews |
| sanity-flr@zfs:test_44b | seen in 11 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 61 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 12 other reviews |
| sanity-hsm@zfs:test_607b | seen in 12 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 30 other reviews |
| sanityn@zfs:test_16j | seen in 33 other reviews |
LU-13814 osc: call transfer_page_init for DIO The transfer pages for DIO need to be set up, this does the necessary wrangling to call osc_transfer_page_init from the DIO path. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0fdc3340cfcecb1dc524c55f480961d36cabdedc
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 24 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 23 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_63b | seen in 61 other reviews |
| sanity2@ldiskfs+DNE:test_64a | seen in 61 other reviews |
| sanity2@ldiskfs+DNE:test_64c | seen in 61 other reviews |
| sanity2@ldiskfs+DNE:test_64d | seen in 100 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 39 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 29 other reviews |
| sanity1@zfs:test_56x | seen in 44 other reviews |
| sanity1@zfs:test_56xB | seen in 35 other reviews |
| sanity1@zfs:test_56xa | seen in 44 other reviews |
| sanity1@zfs:test_56xab | seen in 32 other reviews |
| sanity1@zfs:test_56xc | seen in 40 other reviews |
| sanity1@zfs:test_56ej | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 78 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 36 other reviews |
| sanity-flr@zfs:test_0g | seen in 37 other reviews |
| sanity-flr@zfs:test_0h | seen in 39 other reviews |
| sanity-flr@zfs:test_0j | seen in 38 other reviews |
| sanity-flr@zfs:test_36d | seen in 37 other reviews |
| sanity-flr@zfs:test_37 | seen in 40 other reviews |
| sanity-flr@zfs:test_38 | seen in 38 other reviews |
| sanity-flr@zfs:test_44b | seen in 32 other reviews |
| sanity-flr@zfs:test_61a | seen in 44 other reviews |
| sanity-flr@zfs:test_61c | seen in 37 other reviews |
| sanity-flr@zfs:test_70a | seen in 68 other reviews |
| sanity-flr@zfs:test_200a | seen in 37 other reviews |
| sanity-flr@zfs:test_200b | seen in 37 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 33 other reviews |
| sanity-hsm@zfs:test_250 | seen in 11 other reviews |
| sanity-hsm@zfs:test_607b | seen in 33 other reviews |
| sanity-quota@zfs:test_90a | seen in 2 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 36 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-17885 osc: add oe_page_array for dio This is a very simple patch which just adds the array and a few asserts for places that never see DIO pages. The array is used in the next patch. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If1cb80167e409a8dc36711b46ed4b5459a88df75
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 25 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 24 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 27 other reviews |
| sanity1@zfs:test_56x | seen in 44 other reviews |
| sanity1@zfs:test_56xB | seen in 33 other reviews |
| sanity1@zfs:test_56xa | seen in 44 other reviews |
| sanity1@zfs:test_56xab | seen in 30 other reviews |
| sanity1@zfs:test_56xc | seen in 40 other reviews |
| sanity1@zfs:test_56ej | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 78 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 36 other reviews |
| sanity-flr@zfs:test_0g | seen in 37 other reviews |
| sanity-flr@zfs:test_0h | seen in 39 other reviews |
| sanity-flr@zfs:test_0j | seen in 38 other reviews |
| sanity-flr@zfs:test_36d | seen in 37 other reviews |
| sanity-flr@zfs:test_37 | seen in 40 other reviews |
| sanity-flr@zfs:test_38 | seen in 38 other reviews |
| sanity-flr@zfs:test_44b | seen in 31 other reviews |
| sanity-flr@zfs:test_61a | seen in 44 other reviews |
| sanity-flr@zfs:test_61c | seen in 37 other reviews |
| sanity-flr@zfs:test_70a | seen in 67 other reviews |
| sanity-flr@zfs:test_200a | seen in 37 other reviews |
| sanity-flr@zfs:test_200b | seen in 37 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 31 other reviews |
| sanity-hsm@zfs:test_607b | seen in 32 other reviews |
| sanity-pcc@zfs:test_1d | seen in 34 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-17885 osc: remove DIO otp list use in osc_build_rpc This removes the usage of the osc_transfer_page list for DIO in osc_build_rpc. The list is still created and used elsewhere, but this reduces the usage with an eye to removal. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie04ef14569d07f5dde01b223f0dced66c2af0094
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 23 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 22 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 25 other reviews |
| sanity1@zfs:test_56x | seen in 44 other reviews |
| sanity1@zfs:test_56xB | seen in 34 other reviews |
| sanity1@zfs:test_56xa | seen in 44 other reviews |
| sanity1@zfs:test_56xab | seen in 31 other reviews |
| sanity1@zfs:test_56xc | seen in 40 other reviews |
| sanity1@zfs:test_56ej | seen in 34 other reviews |
| sanity2@zfs:test_119e | seen in 35 other reviews |
| sanity2@zfs:test_119f | seen in 34 other reviews |
| sanity2@zfs:test_119g | seen in 34 other reviews |
| sanity2@zfs:test_119h | seen in 34 other reviews |
| sanity2@zfs:test_119p | seen in 30 other reviews |
| sanity2@zfs:test_119q | seen in 30 other reviews |
| sanity2@zfs:test_398o | seen in 39 other reviews |
| sanity2@zfs:test_398s | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 77 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 36 other reviews |
| sanity-flr@zfs:test_0g | seen in 37 other reviews |
| sanity-flr@zfs:test_0h | seen in 39 other reviews |
| sanity-flr@zfs:test_0j | seen in 38 other reviews |
| sanity-flr@zfs:test_36d | seen in 37 other reviews |
| sanity-flr@zfs:test_37 | seen in 40 other reviews |
| sanity-flr@zfs:test_38 | seen in 38 other reviews |
| sanity-flr@zfs:test_44b | seen in 30 other reviews |
| sanity-flr@zfs:test_61a | seen in 44 other reviews |
| sanity-flr@zfs:test_61c | seen in 37 other reviews |
| sanity-flr@zfs:test_70a | seen in 66 other reviews |
| sanity-flr@zfs:test_200b | seen in 37 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 32 other reviews |
| sanity-hsm@zfs:test_607b | seen in 31 other reviews |
| sanity-pcc@ldiskfs+DNE:test_1c | seen in 27 other reviews |
| sanity-quota@zfs:test_90b | seen in 5 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 36 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-17885 osc: remove list for dio in extent finish This removes the last usage of the page lists for DIO, and also stops creating them. This gets us most of the performance benefit, but not all of it - yet. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic2c700a6e09def0e0162ab567d9a0af321fa7e87
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 21 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 20 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 39 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 28 other reviews |
| sanity1@zfs:test_56x | seen in 43 other reviews |
| sanity1@zfs:test_56xB | seen in 30 other reviews |
| sanity1@zfs:test_56xa | seen in 43 other reviews |
| sanity1@zfs:test_56xab | seen in 27 other reviews |
| sanity1@zfs:test_56xc | seen in 39 other reviews |
| sanity1@zfs:test_56ej | seen in 30 other reviews |
| sanity2@zfs:test_119e | seen in 34 other reviews |
| sanity2@zfs:test_119f | seen in 33 other reviews |
| sanity2@zfs:test_119g | seen in 33 other reviews |
| sanity2@zfs:test_119h | seen in 33 other reviews |
| sanity2@zfs:test_119p | seen in 27 other reviews |
| sanity2@zfs:test_119q | seen in 27 other reviews |
| sanity2@zfs:test_398o | seen in 38 other reviews |
| sanity2@zfs:test_398s | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 28 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 27 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 28 other reviews |
| sanity-hsm@zfs:test_607b | seen in 28 other reviews |
| sanity-quota@zfs:test_90b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 33 other reviews |
| sanityn@zfs:test_16j | seen in 36 other reviews |
LU-13814 clio: remove cp_type from vvp, mdc, ll cp_type appears only a little in the vvp, mdc, and ll code, so remove it all in one patch. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9dc3dc4e4d35322fbe4428d4d8ffd624baada693
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_119e | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 38 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 27 other reviews |
| sanity1@zfs:test_56x | seen in 42 other reviews |
| sanity1@zfs:test_56xB | seen in 27 other reviews |
| sanity1@zfs:test_56xa | seen in 42 other reviews |
| sanity1@zfs:test_56xab | seen in 24 other reviews |
| sanity1@zfs:test_56xc | seen in 38 other reviews |
| sanity1@zfs:test_56ej | seen in 27 other reviews |
| sanity2@zfs:test_119e | seen in 30 other reviews |
| sanity2@zfs:test_119f | seen in 29 other reviews |
| sanity2@zfs:test_119g | seen in 29 other reviews |
| sanity2@zfs:test_119h | seen in 29 other reviews |
| sanity2@zfs:test_119p | seen in 23 other reviews |
| sanity2@zfs:test_119q | seen in 23 other reviews |
| sanity2@zfs:test_398o | seen in 34 other reviews |
| sanity2@zfs:test_398s | seen in 23 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 24 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 26 other reviews |
| sanity-hsm@zfs:test_607b | seen in 24 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 33 other reviews |
| sanityn@zfs:test_16j | seen in 36 other reviews |
LU-13814 clio: remove type from cl_page_find type is no longer used in cl_page_find/alloc, remove it. Test-Parameters: forjanitoronly Test-Parameters: fortestonly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5168f5d34e24727d45d1d0910d8b90cec9429b35
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 37 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 26 other reviews |
| sanity1@zfs:test_56x | seen in 44 other reviews |
| sanity1@zfs:test_56xB | seen in 32 other reviews |
| sanity1@zfs:test_56xa | seen in 44 other reviews |
| sanity1@zfs:test_56xab | seen in 29 other reviews |
| sanity1@zfs:test_56xc | seen in 40 other reviews |
| sanity1@zfs:test_56ej | seen in 32 other reviews |
| sanity2@zfs:test_119e | seen in 35 other reviews |
| sanity2@zfs:test_119f | seen in 34 other reviews |
| sanity2@zfs:test_119g | seen in 34 other reviews |
| sanity2@zfs:test_119h | seen in 34 other reviews |
| sanity2@zfs:test_119p | seen in 29 other reviews |
| sanity2@zfs:test_119q | seen in 29 other reviews |
| sanity2@zfs:test_398o | seen in 39 other reviews |
| sanity2@zfs:test_398s | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 76 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 36 other reviews |
| sanity-flr@zfs:test_0g | seen in 37 other reviews |
| sanity-flr@zfs:test_0h | seen in 39 other reviews |
| sanity-flr@zfs:test_0j | seen in 38 other reviews |
| sanity-flr@zfs:test_36d | seen in 37 other reviews |
| sanity-flr@zfs:test_37 | seen in 40 other reviews |
| sanity-flr@zfs:test_38 | seen in 38 other reviews |
| sanity-flr@zfs:test_44b | seen in 29 other reviews |
| sanity-flr@zfs:test_61a | seen in 44 other reviews |
| sanity-flr@zfs:test_61c | seen in 37 other reviews |
| sanity-flr@zfs:test_70a | seen in 65 other reviews |
| sanity-flr@zfs:test_200a | seen in 37 other reviews |
| sanity-flr@zfs:test_200b | seen in 37 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 30 other reviews |
| sanity-hsm@zfs:test_607b | seen in 30 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 35 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-13814 osc: change asserts to use OTP type cp_type is no longer set for transient pages since they don't use cl_page. Switch asserts to use otp_type. Test-Parameters: trivial Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I3ad8292972fad9669b00aac3e98d0b6ea12ef398
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 18 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 17 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 36 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 25 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 25 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 22 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 25 other reviews |
| sanity2@zfs:test_119e | seen in 27 other reviews |
| sanity2@zfs:test_119f | seen in 26 other reviews |
| sanity2@zfs:test_119g | seen in 26 other reviews |
| sanity2@zfs:test_119h | seen in 26 other reviews |
| sanity2@zfs:test_119p | seen in 20 other reviews |
| sanity2@zfs:test_119q | seen in 20 other reviews |
| sanity2@zfs:test_398o | seen in 31 other reviews |
| sanity2@zfs:test_398s | seen in 20 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 23 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 22 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 23 other reviews |
| sanity-hsm@zfs:test_607b | seen in 23 other reviews |
| sanity-quota@zfs:test_90b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: move otp_obj to osc page The object pointer is only used from the OSC page, so move it there. This reduces the size of the OTP page, which is useful because for DIO there is only an OTP page. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I97360277a9d686b7b78648762d45d20864355cbc
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 22 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 21 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 35 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 24 other reviews |
| sanity1@zfs:test_56x | seen in 43 other reviews |
| sanity1@zfs:test_56xB | seen in 31 other reviews |
| sanity1@zfs:test_56xa | seen in 43 other reviews |
| sanity1@zfs:test_56xab | seen in 28 other reviews |
| sanity1@zfs:test_56xc | seen in 39 other reviews |
| sanity1@zfs:test_56ej | seen in 31 other reviews |
| sanity2@zfs:test_119e | seen in 34 other reviews |
| sanity2@zfs:test_119f | seen in 33 other reviews |
| sanity2@zfs:test_119g | seen in 33 other reviews |
| sanity2@zfs:test_119h | seen in 33 other reviews |
| sanity2@zfs:test_119p | seen in 28 other reviews |
| sanity2@zfs:test_119q | seen in 28 other reviews |
| sanity2@zfs:test_398o | seen in 38 other reviews |
| sanity2@zfs:test_398s | seen in 28 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 28 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 29 other reviews |
| sanity-hsm@zfs:test_607b | seen in 29 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 34 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-13814 osc: remove cp_type in osc Now that we no longer have cl_page for transient pages, cp_type is going away, so remove it in the OSC layer. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2ea93f46d8a99ba6c0e04af373827b4c8b8fd2be
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 20 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 19 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 34 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 23 other reviews |
| sanity1@zfs:test_56x | seen in 43 other reviews |
| sanity1@zfs:test_56xB | seen in 29 other reviews |
| sanity1@zfs:test_56xa | seen in 43 other reviews |
| sanity1@zfs:test_56xab | seen in 26 other reviews |
| sanity1@zfs:test_56xc | seen in 39 other reviews |
| sanity1@zfs:test_56ej | seen in 29 other reviews |
| sanity2@zfs:test_119e | seen in 33 other reviews |
| sanity2@zfs:test_119f | seen in 32 other reviews |
| sanity2@zfs:test_119g | seen in 32 other reviews |
| sanity2@zfs:test_119h | seen in 32 other reviews |
| sanity2@zfs:test_119p | seen in 26 other reviews |
| sanity2@zfs:test_119q | seen in 26 other reviews |
| sanity2@zfs:test_398o | seen in 37 other reviews |
| sanity2@zfs:test_398s | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 26 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 27 other reviews |
| sanity-hsm@zfs:test_607b | seen in 27 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 36 other reviews |
LU-13814 lov: remove cp_type from lov Removing cp_type from lov involves removing the LOV stripe information caching, since that was just for DIO. Other removals here are trivial. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia60397639e9499e69c17f8d549806f17b4000d05
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_119e | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 22 other reviews |
| sanity1@zfs:test_56x | seen in 43 other reviews |
| sanity1@zfs:test_56xB | seen in 28 other reviews |
| sanity1@zfs:test_56xa | seen in 43 other reviews |
| sanity1@zfs:test_56xab | seen in 25 other reviews |
| sanity1@zfs:test_56xc | seen in 39 other reviews |
| sanity1@zfs:test_56ej | seen in 28 other reviews |
| sanity2@zfs:test_119e | seen in 31 other reviews |
| sanity2@zfs:test_119f | seen in 30 other reviews |
| sanity2@zfs:test_119g | seen in 30 other reviews |
| sanity2@zfs:test_119h | seen in 30 other reviews |
| sanity2@zfs:test_119p | seen in 24 other reviews |
| sanity2@zfs:test_119q | seen in 24 other reviews |
| sanity2@zfs:test_398o | seen in 35 other reviews |
| sanity2@zfs:test_398s | seen in 24 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 25 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 25 other reviews |
| sanity-hsm@zfs:test_607b | seen in 26 other reviews |
| sanity-quota@zfs:test_90b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 36 other reviews |
LU-13814 clio: remove cp_type Page->cp_type is no longer needed and can be removed entirely. Two notes: This makes 'inode' in coo_page_init irrelevant since it's only used for DIO. This will be fixed in a future patch. The packing of cl_page is NOT correct currently and will be fixed in a future patch. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I660bb956b2d85fcff98b8e1726d60b51fd4f8ac5
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 17 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 16 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 19 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 24 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 21 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 24 other reviews |
| sanity2@zfs:test_119e | seen in 32 other reviews |
| sanity2@zfs:test_119f | seen in 31 other reviews |
| sanity2@zfs:test_119g | seen in 31 other reviews |
| sanity2@zfs:test_119h | seen in 31 other reviews |
| sanity2@zfs:test_119p | seen in 25 other reviews |
| sanity2@zfs:test_119q | seen in 25 other reviews |
| sanity2@zfs:test_398o | seen in 36 other reviews |
| sanity2@zfs:test_398s | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 22 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 21 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 22 other reviews |
| sanity-hsm@zfs:test_607b | seen in 22 other reviews |
| sanity-quota@zfs:test_90b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: remove otp_srvlock The srvlock information stored in the otp_srvlock flag is always available elsewhere, so remove it. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7a8669b8a3ae9014a9971386c316d514079a0ae1
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 14 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 13 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 21 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 20 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 17 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 20 other reviews |
| sanity2@zfs:test_119e | seen in 23 other reviews |
| sanity2@zfs:test_119f | seen in 22 other reviews |
| sanity2@zfs:test_119g | seen in 22 other reviews |
| sanity2@zfs:test_119h | seen in 22 other reviews |
| sanity2@zfs:test_119p | seen in 15 other reviews |
| sanity2@zfs:test_119q | seen in 15 other reviews |
| sanity2@zfs:test_398o | seen in 27 other reviews |
| sanity2@zfs:test_398s | seen in 15 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 28 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 18 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 73 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 32 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 17 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 18 other reviews |
| sanity-hsm@zfs:test_607b | seen in 18 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 31 other reviews |
| sanityn@zfs:test_16j | seen in 34 other reviews |
LU-13814 osc: remove aa_otps The aa_oops list isn't really used for anything, so let's remove it. This will let us remove the otp_rpc_item list, because the only use of that list was to go on the aa_otps list. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I491f66857786dd9ed13657d4211e774232cb6e22
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 19 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 18 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 20 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 26 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 23 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 26 other reviews |
| sanity2@zfs:test_119e | seen in 28 other reviews |
| sanity2@zfs:test_119f | seen in 27 other reviews |
| sanity2@zfs:test_119g | seen in 27 other reviews |
| sanity2@zfs:test_119h | seen in 27 other reviews |
| sanity2@zfs:test_119p | seen in 21 other reviews |
| sanity2@zfs:test_119q | seen in 21 other reviews |
| sanity2@zfs:test_398o | seen in 32 other reviews |
| sanity2@zfs:test_398s | seen in 21 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 24 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 23 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 24 other reviews |
| sanity-hsm@zfs:test_607b | seen in 25 other reviews |
| sanity-pcc@ldiskfs+DNE:test_100 | seen in 33 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: minor function relocation Two functions are essentially in the wrong file, and one of them is a trivial wrapper. Move the actual function and just remove the wrapper. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie408734f02a8047621f3447b13f5d1786b070801
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 12 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 11 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 14 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 18 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 18 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 15 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 18 other reviews |
| sanity2@zfs:test_119e | seen in 23 other reviews |
| sanity2@zfs:test_119f | seen in 22 other reviews |
| sanity2@zfs:test_119g | seen in 22 other reviews |
| sanity2@zfs:test_119h | seen in 22 other reviews |
| sanity2@zfs:test_119p | seen in 14 other reviews |
| sanity2@zfs:test_119q | seen in 14 other reviews |
| sanity2@zfs:test_398o | seen in 27 other reviews |
| sanity2@zfs:test_398s | seen in 14 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 28 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 17 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 73 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 32 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 16 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 17 other reviews |
| sanity-hsm@zfs:test_607b | seen in 17 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 31 other reviews |
| sanityn@zfs:test_16j | seen in 34 other reviews |
LU-13814 osc: remove otp_rpc_item list The otp_rpc_item list is only used to recognize if a page is in an RPC, which an error condition that should only occur if there's a bug in the code. We can remove it, and save ourselves two pointers in every page. With this, this the osc_transfer_page is now 61 bytes in size, fitting it inside a single cacheline(!). This has a huge impact on DIO performance. The net effect of these reduction patches is about a 40-50% boost in single threaded DIO performance beyond that achieved by cl_page removal. 1 GiB transfer size IOR performance (single threaded) Without the reduction patches: Read: 57 GiB/s Write: 48 GiB/s With: Read: 88 GiB/s Write: 65 GiB/s (Write performance is expected to be closer to read, but it may have been affected by a network issue.) Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Icb83298da1bdc797a33dae6ca3357b1ad3b3c848
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 16 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 15 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 16 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 21 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 18 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 21 other reviews |
| sanity2@zfs:test_119e | seen in 29 other reviews |
| sanity2@zfs:test_119f | seen in 28 other reviews |
| sanity2@zfs:test_119g | seen in 28 other reviews |
| sanity2@zfs:test_119h | seen in 28 other reviews |
| sanity2@zfs:test_119p | seen in 22 other reviews |
| sanity2@zfs:test_119q | seen in 22 other reviews |
| sanity2@zfs:test_398o | seen in 33 other reviews |
| sanity2@zfs:test_398s | seen in 22 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 20 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 19 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 20 other reviews |
| sanity-hsm@zfs:test_607b | seen in 20 other reviews |
| sanity-pcc@ldiskfs+DNE:test_1c | seen in 26 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: remove otp_page_off otp_page_off can be determined from otp_obj_off, so we can remove it. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4ab9e71a8d6e79544241021b72eff2a95c592cb9
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 15 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 14 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 17 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 23 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 20 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 23 other reviews |
| sanity2@zfs:test_119e | seen in 25 other reviews |
| sanity2@zfs:test_119f | seen in 24 other reviews |
| sanity2@zfs:test_119g | seen in 24 other reviews |
| sanity2@zfs:test_119h | seen in 24 other reviews |
| sanity2@zfs:test_119p | seen in 18 other reviews |
| sanity2@zfs:test_119q | seen in 18 other reviews |
| sanity2@zfs:test_398o | seen in 29 other reviews |
| sanity2@zfs:test_398s | seen in 18 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 21 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 20 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 21 other reviews |
| sanity-hsm@zfs:test_607b | seen in 21 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: remove otp_cmd The otp_cmd information is only need in the OSC page, so move it there so it's not allocated for DIO. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2c32126e72d45dad5a2104ded0b55cdb1150adce
| unique failing test | history |
|---|---|
| racer@zfs:test_1 | seen in 6 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 14 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 14 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 14 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 19 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 16 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 19 other reviews |
| sanity2@zfs:test_119e | seen in 26 other reviews |
| sanity2@zfs:test_119f | seen in 25 other reviews |
| sanity2@zfs:test_119g | seen in 25 other reviews |
| sanity2@zfs:test_119h | seen in 25 other reviews |
| sanity2@zfs:test_119p | seen in 19 other reviews |
| sanity2@zfs:test_119q | seen in 19 other reviews |
| sanity2@zfs:test_398o | seen in 30 other reviews |
| sanity2@zfs:test_398s | seen in 19 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 16 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 72 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 31 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 34 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 15 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 62 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 16 other reviews |
| sanity-hsm@zfs:test_607b | seen in 16 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 31 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: add osc_transfer_page comments Add comments highlighting the critical nature of the OSC transfer page for performance. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ifb282c0a5332fac3bf0a054a9e27b2e596fce8f0
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 13 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 12 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 15 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 15 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 15 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 15 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 22 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 19 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 22 other reviews |
| sanity2@zfs:test_119e | seen in 24 other reviews |
| sanity2@zfs:test_119f | seen in 23 other reviews |
| sanity2@zfs:test_119g | seen in 23 other reviews |
| sanity2@zfs:test_119h | seen in 23 other reviews |
| sanity2@zfs:test_119p | seen in 16 other reviews |
| sanity2@zfs:test_119q | seen in 16 other reviews |
| sanity2@zfs:test_398o | seen in 28 other reviews |
| sanity2@zfs:test_398s | seen in 16 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 19 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 73 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 33 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 18 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 19 other reviews |
| sanity-hsm@zfs:test_607b | seen in 19 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: remove otp_inode We can add the inode to the osc object, which allows us to remove the inode from the osc_transfer_page. This removes a pointer from the OTP struct, which is allocated for every page in Lustre. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ida9c030bf085ad9606e19522714497c4adfa33de
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 10 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 9 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 12 other reviews |
| sanity1@zfs:test_56x | seen in 40 other reviews |
| sanity1@zfs:test_56xB | seen in 16 other reviews |
| sanity1@zfs:test_56xa | seen in 40 other reviews |
| sanity1@zfs:test_56xab | seen in 13 other reviews |
| sanity1@zfs:test_56xc | seen in 36 other reviews |
| sanity1@zfs:test_56ej | seen in 16 other reviews |
| sanity2@zfs:test_119e | seen in 24 other reviews |
| sanity2@zfs:test_119f | seen in 23 other reviews |
| sanity2@zfs:test_119g | seen in 23 other reviews |
| sanity2@zfs:test_119h | seen in 23 other reviews |
| sanity2@zfs:test_119p | seen in 17 other reviews |
| sanity2@zfs:test_119q | seen in 17 other reviews |
| sanity2@zfs:test_398o | seen in 28 other reviews |
| sanity2@zfs:test_398s | seen in 17 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 14 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 72 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 31 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 33 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_38 | seen in 34 other reviews |
| sanity-flr@zfs:test_44b | seen in 13 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 62 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 13 other reviews |
| sanity-hsm@zfs:test_607b | seen in 14 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 30 other reviews |
| sanityn@zfs:test_16j | seen in 33 other reviews |
LU-13814 osc: remove otp_ll_index otp_ll_index is used to preserve the index for direct IO pages, but direct IO pages do not use the index value, so we can just store the index in the vmpage. This reduces the osc_transfer_page size by a further 8 bytes, to 43 bytes. This puts us tantalizingly close to a single cacheline, but this can only be achieved by removing the otp_pending_item list, which is challenging. That will require converting osc extents to use an array of pointers instead of a linked list. This can be done, and will improve performance because of that change by itself. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I621388fb36a2a792525e07f35c1c948c4ae4292e
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 32 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 5 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 32 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 4 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 33 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 7 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 6 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 6 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 6 other reviews |
| sanity1@zfs:test_56x | seen in 39 other reviews |
| sanity1@zfs:test_56xB | seen in 10 other reviews |
| sanity1@zfs:test_56xa | seen in 39 other reviews |
| sanity1@zfs:test_56xab | seen in 7 other reviews |
| sanity1@zfs:test_56xc | seen in 35 other reviews |
| sanity1@zfs:test_56ej | seen in 10 other reviews |
| sanity2@zfs:test_119e | seen in 22 other reviews |
| sanity2@zfs:test_119f | seen in 21 other reviews |
| sanity2@zfs:test_119g | seen in 21 other reviews |
| sanity2@zfs:test_119h | seen in 21 other reviews |
| sanity2@zfs:test_119p | seen in 13 other reviews |
| sanity2@zfs:test_119q | seen in 13 other reviews |
| sanity2@zfs:test_398o | seen in 26 other reviews |
| sanity2@zfs:test_398s | seen in 13 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 9 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 33 other reviews |
| sanity-flr@zfs:test_0h | seen in 35 other reviews |
| sanity-flr@zfs:test_0j | seen in 34 other reviews |
| sanity-flr@zfs:test_36d | seen in 32 other reviews |
| sanity-flr@zfs:test_37 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 8 other reviews |
| sanity-flr@zfs:test_61a | seen in 40 other reviews |
| sanity-flr@zfs:test_61c | seen in 33 other reviews |
| sanity-flr@zfs:test_70a | seen in 60 other reviews |
| sanity-flr@zfs:test_200a | seen in 33 other reviews |
| sanity-flr@zfs:test_200b | seen in 33 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 9 other reviews |
| sanity-hsm@zfs:test_607b | seen in 8 other reviews |
| sanity-pcc@zfs:test_100 | seen in 19 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 29 other reviews |
| sanityn@zfs:test_16j | seen in 31 other reviews |
LU-13814 clio: add args to cl_dio_pages_init The inode and osc index are needed to set up the transfer pages, so though they're unused here, they will be used in the next patch. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic41d1ca6db05c53ac86840379abcab2e62220b81
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 11 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 10 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 13 other reviews |
| sanity1@zfs:test_56x | seen in 40 other reviews |
| sanity1@zfs:test_56xB | seen in 17 other reviews |
| sanity1@zfs:test_56xa | seen in 40 other reviews |
| sanity1@zfs:test_56xab | seen in 14 other reviews |
| sanity1@zfs:test_56xc | seen in 36 other reviews |
| sanity1@zfs:test_56ej | seen in 17 other reviews |
| sanity2@zfs:test_119e | seen in 21 other reviews |
| sanity2@zfs:test_119f | seen in 20 other reviews |
| sanity2@zfs:test_119g | seen in 20 other reviews |
| sanity2@zfs:test_119h | seen in 20 other reviews |
| sanity2@zfs:test_119p | seen in 12 other reviews |
| sanity2@zfs:test_119q | seen in 12 other reviews |
| sanity2@zfs:test_398o | seen in 25 other reviews |
| sanity2@zfs:test_398s | seen in 12 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 15 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 72 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 31 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 33 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_38 | seen in 35 other reviews |
| sanity-flr@zfs:test_44b | seen in 14 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 62 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 15 other reviews |
| sanity-hsm@zfs:test_607b | seen in 15 other reviews |
| sanity-quota@zfs:test_90b | seen in 1 other review |
| sanityn@ldiskfs+DNE:test_16j | seen in 30 other reviews |
| sanityn@zfs:test_16j | seen in 33 other reviews |
LU-13814 osc: remove dedicated otp_obj_off member otp_obj_off and the offset stored in the brw page are the same, and the osc transfer page always contains a brw page. So we can just always use the bp_off offset. This saves 4 bytes in the osc_transfer_page, which is a meaningful reduction in size. Total size of osc_transfer_page (on x86_64) is now down to 51 bytes. This does not have a huge performance impact because the struct still occupies two cachelines. (Cachelines are 32 bytes.) Getting the size to 32 bytes will be challenging, but may be possible. For example, the otp_ll_index field may be removable, and it should be possible - with effort - to remove the otp_pending_list. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie38e8154e184c4c9bec38f1ddd482b02cac29ab9
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 9 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 8 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 11 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 11 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 11 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 11 other reviews |
| sanity1@zfs:test_56x | seen in 40 other reviews |
| sanity1@zfs:test_56xB | seen in 15 other reviews |
| sanity1@zfs:test_56xa | seen in 40 other reviews |
| sanity1@zfs:test_56xab | seen in 12 other reviews |
| sanity1@zfs:test_56xc | seen in 36 other reviews |
| sanity1@zfs:test_56ej | seen in 15 other reviews |
| sanity2@zfs:test_119e | seen in 20 other reviews |
| sanity2@zfs:test_119f | seen in 19 other reviews |
| sanity2@zfs:test_119g | seen in 19 other reviews |
| sanity2@zfs:test_119h | seen in 19 other reviews |
| sanity2@zfs:test_119p | seen in 10 other reviews |
| sanity2@zfs:test_119q | seen in 10 other reviews |
| sanity2@zfs:test_398o | seen in 24 other reviews |
| sanity2@zfs:test_398s | seen in 10 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 12 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 33 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_38 | seen in 34 other reviews |
| sanity-flr@zfs:test_44b | seen in 12 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 62 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 14 other reviews |
| sanity-hsm@zfs:test_607b | seen in 13 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 33 other reviews |
LU-17885 osc: move list to queue_dio_pages Move list handling to queue_dio_pages. This is a precursor to removing list usage and this one actually makes things less efficient, but only briefly. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2f9671bf276b03e3ce5e9bae2d4239649ae25cec
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_119e | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 8 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 8 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 8 other reviews |
| sanity1@zfs:test_56x | seen in 39 other reviews |
| sanity1@zfs:test_56xB | seen in 13 other reviews |
| sanity1@zfs:test_56xa | seen in 39 other reviews |
| sanity1@zfs:test_56xab | seen in 10 other reviews |
| sanity1@zfs:test_56xc | seen in 35 other reviews |
| sanity1@zfs:test_56ej | seen in 13 other reviews |
| sanity2@zfs:test_119e | seen in 20 other reviews |
| sanity2@zfs:test_119f | seen in 19 other reviews |
| sanity2@zfs:test_119g | seen in 19 other reviews |
| sanity2@zfs:test_119h | seen in 19 other reviews |
| sanity2@zfs:test_119p | seen in 9 other reviews |
| sanity2@zfs:test_119q | seen in 9 other reviews |
| sanity2@zfs:test_398o | seen in 24 other reviews |
| sanity2@zfs:test_398s | seen in 9 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 11 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 33 other reviews |
| sanity-flr@zfs:test_0h | seen in 35 other reviews |
| sanity-flr@zfs:test_0j | seen in 34 other reviews |
| sanity-flr@zfs:test_36d | seen in 32 other reviews |
| sanity-flr@zfs:test_37 | seen in 36 other reviews |
| sanity-flr@zfs:test_38 | seen in 33 other reviews |
| sanity-flr@zfs:test_44b | seen in 10 other reviews |
| sanity-flr@zfs:test_61a | seen in 40 other reviews |
| sanity-flr@zfs:test_61c | seen in 33 other reviews |
| sanity-flr@zfs:test_70a | seen in 60 other reviews |
| sanity-flr@zfs:test_200a | seen in 33 other reviews |
| sanity-flr@zfs:test_200b | seen in 33 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 10 other reviews |
| sanity-hsm@zfs:test_607b | seen in 9 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 29 other reviews |
| sanityn@zfs:test_16j | seen in 32 other reviews |
LU-13814 osc: drop cl_page structs for DIO This is the big one - this flips the switch and switches DIO from using osc transfer pages which are part of cl_pages and are initialized the same for buffered or DIO, to using bare OSC transfer pages - no associated cl_page. This patch is the primary goal of this series, but it stops halfway, leaving out removing the cl_page allocation for DIO, because that is better done in a separate patch. Once that is done in the next patch, we'll see the performance jump. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7f664f0ffee97c9f2c778996423cef5ae79c3460
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 33 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 6 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 33 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 5 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 34 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 8 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 9 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 9 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 9 other reviews |
| sanity1@zfs:test_56x | seen in 39 other reviews |
| sanity1@zfs:test_56xB | seen in 12 other reviews |
| sanity1@zfs:test_56xa | seen in 39 other reviews |
| sanity1@zfs:test_56xab | seen in 9 other reviews |
| sanity1@zfs:test_56xc | seen in 35 other reviews |
| sanity1@zfs:test_56ej | seen in 12 other reviews |
| sanity2@zfs:test_119e | seen in 19 other reviews |
| sanity2@zfs:test_119f | seen in 18 other reviews |
| sanity2@zfs:test_119g | seen in 18 other reviews |
| sanity2@zfs:test_119h | seen in 18 other reviews |
| sanity2@zfs:test_119p | seen in 8 other reviews |
| sanity2@zfs:test_119q | seen in 8 other reviews |
| sanity2@zfs:test_398o | seen in 23 other reviews |
| sanity2@zfs:test_398s | seen in 8 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 8 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 33 other reviews |
| sanity-flr@zfs:test_0h | seen in 35 other reviews |
| sanity-flr@zfs:test_0j | seen in 34 other reviews |
| sanity-flr@zfs:test_36d | seen in 32 other reviews |
| sanity-flr@zfs:test_37 | seen in 36 other reviews |
| sanity-flr@zfs:test_38 | seen in 33 other reviews |
| sanity-flr@zfs:test_44b | seen in 7 other reviews |
| sanity-flr@zfs:test_61a | seen in 40 other reviews |
| sanity-flr@zfs:test_61c | seen in 33 other reviews |
| sanity-flr@zfs:test_70a | seen in 60 other reviews |
| sanity-flr@zfs:test_200a | seen in 33 other reviews |
| sanity-flr@zfs:test_200b | seen in 33 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 8 other reviews |
| sanity-hsm@zfs:test_607b | seen in 10 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 29 other reviews |
| sanityn@zfs:test_16j | seen in 31 other reviews |
LU-13814 clio: remove cl_page allocation for DIO This removes the - now unused - cl_page allocation for DIO. This will be followed by further patches cleaning up various checks associated with transient cl pages, which no longer exist. With this patch, the performance benefits are realized. This reduces the time to submit DIO pages by about 85%. This results in about a 2.5-3x improvement in DIO performance, because of other overheads and hardware limitations. Without this patch, IOR at 1 GiB transfer size: Read: 22 GiB/s Write: 20 GiB/s With this patch, IOR at 1 GiB transfer size: Read: 57 GiB/s Write: 48 GiB/s There's still some substantial overhead in the transfer page allocations which is removed by the rest of this series. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I406c69d69049484e477a671ada6b4e95357d9f39
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 1 other review |
| runtests@zfs:test_1 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-17885 osc: move pending_list to OSC page Move the pending list to the osc page. This reduces the size of osc_transfer_page by two full pointers, which is quite significant - the entire struct is around 48 bytes, so the removal of two pointers has a significant impact. There's now little more than a BRW page in here. It wouldn't be too difficult to replace the flags in here with BRW flags, which would make the transfer page just a BRW page. This would give minimal benefit since it doesn't actually reduce the allocated size, but might be nice. Next, it is in theory possible to remove all per-page data for DIO except the vmpage pointer. For DIO, all per-page data (even size and offset, even for encryption and compression) can be inferred from a header associated with the extent. Still, at this point, one thread can submit DIO at over 100 GiB/s, so there's little point to further improvements. Time is better spent in other areas. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9c752f3dc2bb71c5ca2ed8bf6b3cb8db1756ad6c
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 1 other review |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 1 other review |
| runtests@zfs:test_1 | seen in 1 other review |
LU-17885 osc: inline osc_transfer_page_init Inlining osc_transfer_page_init notably reduces CPU time used in initializing pages - from 27% to 21% of total time. In this microbenchmark, this reduces CPU time for DIO overall by 10%, which should drive a performance improvement of up to 11%. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1193ce9f6a930aa07f2cc7f5bb8f4c4484ab2d03
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 31 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 2 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 31 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 1 other review |
| sanity1@ldiskfs+DNE:test_56xc | seen in 32 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 4 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 3 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 3 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 3 other reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 7 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 4 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 7 other reviews |
| sanity2@zfs:test_119e | seen in 16 other reviews |
| sanity2@zfs:test_119f | seen in 15 other reviews |
| sanity2@zfs:test_119g | seen in 15 other reviews |
| sanity2@zfs:test_119h | seen in 15 other reviews |
| sanity2@zfs:test_119p | seen in 4 other reviews |
| sanity2@zfs:test_119q | seen in 4 other reviews |
| sanity2@zfs:test_398o | seen in 20 other reviews |
| sanity2@zfs:test_398s | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 5 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 32 other reviews |
| sanity-flr@zfs:test_44b | seen in 4 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 5 other reviews |
| sanity-hsm@zfs:test_607b | seen in 5 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 clio: add cdp_bytes This is one of several things needed to do prep_transfer_page in the DIO path, so add it. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I622b43ba0cee5fb3124fc30dd177f47df3aef3c0
| unique failing test | history |
|---|---|
| sanity-quota@zfs:test_1b | seen in 38 other reviews |
LU-13814 clio: add cdp_osc_tpages array This adds the OSC transfer pages array to the cl_dio_pages struct, which will soon replace cl_pages for DIO. Test-Parameters: trivial Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2986bfc1b54b3e9e6c2e5517f130ace8ff2e4ef2
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 30 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 1 other review |
| sanity1@ldiskfs+DNE:test_56xa | seen in 30 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 31 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 3 other reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 6 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 3 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 6 other reviews |
| sanity2@zfs:test_119e | seen in 15 other reviews |
| sanity2@zfs:test_119f | seen in 14 other reviews |
| sanity2@zfs:test_119g | seen in 14 other reviews |
| sanity2@zfs:test_119h | seen in 14 other reviews |
| sanity2@zfs:test_119p | seen in 3 other reviews |
| sanity2@zfs:test_119q | seen in 3 other reviews |
| sanity2@zfs:test_398o | seen in 19 other reviews |
| sanity2@zfs:test_398s | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 3 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 4 other reviews |
| sanity-hsm@zfs:test_607b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 osc: move from/to to otp prep Everything in the otp needs to be init inside the OTP preparation function, so move from and to. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Id18a7ddbaf1c2f46b32b7d927df93792126ef5b3
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 6 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 3 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 6 other reviews |
| sanity2@zfs:test_119e | seen in 15 other reviews |
| sanity2@zfs:test_119f | seen in 14 other reviews |
| sanity2@zfs:test_119g | seen in 14 other reviews |
| sanity2@zfs:test_119h | seen in 14 other reviews |
| sanity2@zfs:test_119p | seen in 3 other reviews |
| sanity2@zfs:test_119q | seen in 3 other reviews |
| sanity2@zfs:test_398o | seen in 19 other reviews |
| sanity2@zfs:test_398s | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 3 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 4 other reviews |
| sanity-hsm@zfs:test_607b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 osc: move srvlock to prep otp Everything in the otp needs to be init inside the OTP preparation function, so move srvlock. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I1940101c9e21e345d20d93820951fe26eab95cf6
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 6 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 3 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 6 other reviews |
| sanity2@zfs:test_119e | seen in 15 other reviews |
| sanity2@zfs:test_119f | seen in 14 other reviews |
| sanity2@zfs:test_119g | seen in 14 other reviews |
| sanity2@zfs:test_119h | seen in 14 other reviews |
| sanity2@zfs:test_119p | seen in 3 other reviews |
| sanity2@zfs:test_119q | seen in 3 other reviews |
| sanity2@zfs:test_398o | seen in 19 other reviews |
| sanity2@zfs:test_398s | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 3 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 4 other reviews |
| sanity-hsm@zfs:test_607b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 osc: rename osc_prep_transfer_page osc_transfer_page_init is a better match for osc_page_init. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id12eec932eb4aab00139f08532b218c12e6ffba4
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 2 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 2 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 2 other reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 5 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 2 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 5 other reviews |
| sanity2@zfs:test_119e | seen in 14 other reviews |
| sanity2@zfs:test_119f | seen in 13 other reviews |
| sanity2@zfs:test_119g | seen in 13 other reviews |
| sanity2@zfs:test_119h | seen in 13 other reviews |
| sanity2@zfs:test_119p | seen in 2 other reviews |
| sanity2@zfs:test_119q | seen in 2 other reviews |
| sanity2@zfs:test_398o | seen in 18 other reviews |
| sanity2@zfs:test_398s | seen in 2 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 28 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 2 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 30 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 3 other reviews |
| sanity-hsm@zfs:test_607b | seen in 3 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 llite: fix RDMA only check for DIO pages We need to add the check for RDMA only pages and unaligned DIO for DIO pages as we move to them. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I987fecac7874b933a1b558ef442c3ef17120740a
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 26 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 26 other reviews |
| runtests@zfs:test_1 | seen in 20 other reviews |
LU-17885 osc: remove otp from extent_finish + Remove otp usage in osc_extent_finish and osc_flush_async_page. Slow progress. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I9a3a0fe525c93ca264fd60b9597f8ae76ec53398
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 25 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 26 other reviews |
| runtests@zfs:test_1 | seen in 19 other reviews |
LU-17885 osc: remove otp from osc_queue_async_io Remove otp usage in osc_queue_async_io Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I527228cc69b9a976c40cafc43e4ead322106df3f
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 21 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 24 other reviews |
| runtests@zfs:test_1 | seen in 15 other reviews |
LU-17885 osc: remove otp_page_off and brw_page2otp Remove simple functions to simplify otp->brw_page change Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Ie72724db96ae4e8ed7924952ca8652d426d7926b
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 23 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 24 other reviews |
| runtests@zfs:test_1 | seen in 17 other reviews |
LU-17885 osc: remove otp2osc Remove these simple functions to make transition to brw page easier. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Icef7e9998ed7261c0a0d3963684915c6d7d2bc26
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 24 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 24 other reviews |
| runtests@zfs:test_1 | seen in 18 other reviews |
LU-17885 osc: remove otp_index and otp2osc_page Remove these simple functions to make transition to brw page easier. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I7ab38fe8e1797fdfc783f9437d35bd36e54942d8
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 22 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 24 other reviews |
| runtests@zfs:test_1 | seen in 16 other reviews |
LU-17885 osc: remove otp2cl_page Remove these simple functions to make transition to brw page easier. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Iad78799a010ee818c802326c2c40958de98daadd
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 20 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 21 other reviews |
| runtests@zfs:test_1 | seen in 14 other reviews |
LU-17885 osc: remove otp_count macro Remove macro to simplify otp->brw_page change Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I17f41f7e49fb502d21c30727945b62532887bd68
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 19 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 21 other reviews |
| runtests@zfs:test_1 | seen in 13 other reviews |
LU-17885 osc: remove otp_obj_off macro Remove macro to simplify otp->brw_page change Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Id08e579da486c8d727d38f401d234ded0d66efa7
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 21 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 21 other reviews |
| runtests@zfs:test_1 | seen in 15 other reviews |
LU-17885 osc: remove otp_page and otp_flags macros Remove macros to simplify otp->brw_page change Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I500c1d37433a9b69eb1568caccf729a47d0a2d79
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 19 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 20 other reviews |
| runtests@zfs:test_1 | seen in 13 other reviews |
LU-17885 osc: remove otp flags Remove usage of otp flags, move entirely in to BRW page. And now OTP is just another name for a brw page. The next patch is clear enough. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Ic90118cbe622b8d777b5f162d0b8784892db1495
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 21 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 21 other reviews |
| runtests@zfs:test_1 | seen in 15 other reviews |
LU-17885 osc: begin move to brw_page only The osc_transfer_page is now a thin wrapper around the BRW page, so let's work on removing it entirely. This patch pushes cp_type in to the brw page flags, and preps the rest of the OTP flags in the BRW flags. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I045dc064a44ee098bcfc6c9b2dd4f627a03a32d8
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_119s | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_119t | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_119u | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_119s | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_119t | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_119u | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-subtest-change failed 3× | RHEL 9.3 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.9 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-subtest-change failed 3× | RHEL 8.9 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64, RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-17473 tests: add racing tests of aio This patch adds several racing tests for aio. This has been separated from the other patches in the unaligned DIO series because the aio issue uncovered by these tests is pre-existing and I want to separate solving it from the unaligned DIO patches. Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I3571a9a620299137624318e503ab901470f97823
| unique failing test | history |
|---|---|
| replay-vbr@zfs:test_5b | seen in 9 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3/x86_64 | ran 6 tests. 2 tests failed: sanity-pfl, sanity. | session |
| review-dne-part-2 crashed | RHEL 9.3/x86_64 | ran 11 tests. 3 tests failed: sanity-sec, sanity-lfsck, replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-3 | RHEL 9.3/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 crashed | RHEL 9.3/x86_64 | ran 11 tests. 5 tests failed: sanity-quota, sanity-hsm, sanity-flr, sanity-dom, replay-ost-single. %% THIS TE | session |
| review-dne-part-5 | RHEL 9.3/x86_64 | ran 10 tests. 3 tests failed: sanityn, sanity-scrub, recovery-small. | session |
| review-dne-part-6 | RHEL 9.3/x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. | session |
| review-dne-part-7 | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-part-8 crashed | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.9/x86_64 | ran 9 tests. 3 tests failed: sanity-selinux, sanity-sec, recovery-small. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.5/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-zfs crashed | RHEL 8.9/x86_64 | ran 16 tests. 5 tests failed: sanity-lsnapshot, replay-ost-single, replay-single, sanity-flr, sanity-quota. % | session |
LU-17831 osc: batch discard for read locks POC patch Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: If22f7d535eef620bbf01c7d738d447e304ca7ad6
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64, RHEL 9.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.9/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-13802 tests: hybrid IO consistency test Hybrid IO is an IO path change, and we should make sure it produces consistent data. Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I926e7cf23c61148b86b9492ed07138ab9d09a103
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.9/x86_64 | ran 4 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64, RHEL 9.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.9/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-13802 llite: add ZFS check for hybrid IO Because by default ZFS only does one DIO operation per commit interval and commit intervals are in seconds, ZFS performance for DIO is extremely poor. This means we should basically never do hybrid IO switching when using ZFS. Implement this. Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I633799cd080e4f8bbab758c972de592d7ff28725
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 12 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 12 other reviews |
| runtests@zfs:test_1 | seen in 5 other reviews |
LU-17433 llite: move dio range locking to DIO For async DIO writes, we need to do the range unlocking after the IO has completed, not before, otherwise we could get write reordering. So we move the unlocking to the cl_dio_aio_end. Also rename lli_write_tree, because it's not just used for writes. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I8ed0f9e41089f82260ec33e47637b785242240dc
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 12 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 12 other reviews |
| runtests@zfs:test_1 | seen in 5 other reviews |
LU-17433 osc: add dlmlock handling to DIO In order to safely do parallel DIO with dlmlocks, we need to take and put a reference on the dlmlock used for each DIO. With the new cl_dio_pages init and free code, this is straightforward. With this change, we can allow parallel DIO even when using dlmlocks. This will also allow async hybrid writes. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I27b2d57e90178e0b9bbfd3942eec02ec74d3f022
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 12 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 12 other reviews |
| runtests@zfs:test_1 | seen in 5 other reviews |
LU-17433 llite: implement async hybrid writes This patch implements async hybrid writes. This hugely boosts the performance of hybrid writes at smaller sizes. For example, on my local VM system, 2M hybrid writes normally go at 230 MiB/s (roughly the speed of storage), and buffered writes at 1.4 GiB/s. With this patch, 2M hybrid writes go at ~6.8 GiB/s. Note this does not include parallel data copies or page pool usage, which are also in flight and should increase this substantially. This should cause us to re-evaluate when we do hybrid IO vs regular buffered writes, since hybrid will now be faster in all cases except for write sizes less than one page. *However*, currently hybrid will not aggregate async writes, so it will result in a stream of small RPCs to the server. This is in fact something that can be resolved, but for now this problem can be avoided by not changing the switching threshold. Note the current form is based on an incomplete version of hybrid IO and not suitable for landing, but this is just a minor thing which will be resolved when the main hybrid patches are complete. This patch series *does* depend on the DIO simplification series, because of some changes to DIO tracking that code makes. This dependency could be removed at the cost of duplicating a decent chunk of the DIO simplification series, so I'd rather not. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I009bc37c92f391c0304b60fa207d220fa3172fa6
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56od | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15033 llite: Strengthen miss checking For most of our readahead tests, we can predict precisely the expected number of misses. Different read patterns require different numbers of misses to detect, and behavior like whole file read can also reduce the number of misses - but if we are trying to test a specific readahead pattern, using a different one renders the test invalid. If we are trying to verify our ability to handle a strided pattern but we instead read the whole file, there will be fewer misses, but the test is useless. This means as much as possible we should precisely specify the number of misses in our tests, so we ensure they are testing what we think they are testing. With the various fixes in place to remove random misses and generally tighten things up, this is now practical. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7a800db1a6a8fdf91f714366e6fe8147b7269656
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101ac | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101i | seen in 3 other reviews |
| sanity2@zfs:test_101l | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 llite: add strict discard checking Now all the tests try to enforce reasonable bounds on the number of discards. This is part of the ongoing effort to tightly characterize readahead in the individual tests, so we know for sure they are testing the right thing. Historically, many readahead tests have passed but have not actually tested the intended readahead behavior, eg, strided RA tests passing when the whole file is read in. This helps make those cases harder. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Iddc23d0228bdb813dda108455134a8afbe17785b
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15274 llite: Loose reverse readahead Some applications read files in reverse, we should support this in readahead. This adds the concept of "loose reverse readahead", where we will match 'loose' reverse reads, and do readahead for them in a similar manner to forward reads. For sequential and 'loose' (semi-sequential), this makes performance roughly the same for forward and reverse readahead. For one case, a 63 MiB backwards mmap read, it improved performance by ~98% - Reducing read time from 226 seconds to 5 seconds. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2da196f441edcacd18d834958bd30413b1675f02
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_101aa | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_101i | seen in 7 other reviews |
| sanity2@ldiskfs+DNE:test_101k | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_101l | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_101n | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15194 llite: Fix page size readahead window The first page of a read does not use the readahead code, so it doesn't learn the actual RPC size data from the OSC layer. But this is used to set the initial readahead window size. That means if the first read is a single page read and the next read triggers readahead, the size of the readahead is MAX_BRW_PAGES - so the readahead window is set to 64 MiB, even if the RPC size is set to 1 MiB or 4 MiB! It's tricky to get this information from the OSC layer before starting readahead, so we just default the starting RPC size to 1 MiB. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I65812d59f25135d769bf6c855de4f8631b458fc4
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 llite: whole file readahead test There is no test for whole file readahead. Add one. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia4426682139753588d6536b6b1afa2e95c896c34
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101i | seen in 4 other reviews |
| sanity2@zfs:test_101k | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101l | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101n | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_123i | seen in 4 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-3 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-ldiskfs crashed | RHEL 8.8/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15100 llite: Add loose read pages tunables Add tunables for loose forward and reverse loose matching. This allows tuning the number of pages which are considered a valid match when doing sequential I/O, for purpose of controlling the window. If pages are in the "loose" range, the window will still grow even if they were not actually matched. This lets readahead handle 'loose forward read' patterns, where it jumps ahead a small(-ish) but random number of pages. These are not strided patterns - because the jumps are random - and they are not 'simple' sequential, but they are forward read patterns which benefit hugely from doing readahead. Because the time to read 1 MiB of data is only a few times the time required to read 4K of data, it makes sense to be aggressive. We only need to hit a few pages per MiB to gain performance with this behavior. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie31f71b4380c8c384107eb5db416be106f088d9f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Miss counting for readahead The miss counting for the readahead tests is much too lax and includes a large fudge factor of +10 misses. This is unnecessary because these tests are deterministic and we should be able to explain and count all of the misses seen. This patch tightens the margins considerably on allowed misses, which should help avoid problems creeping in in the future. There are a few unexplained misses here - those will need to be debugged later, but we should get these tests in first, then fix the remaining strange behavior. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia876b7ed807fa1d101a8fd02bed4db7823d923dc
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Add rpc counting to readahead One of the key goals of readahead is to generate large RPCs regardless of how large the reads from userspace are. We currently don't test this at all, which is not ideal. Add RPC counting to the readahead tests where it applies. Note this patch includes margin for the RPC count being off due to various minor bugs. Fixing these is left for later - they have been present for a long time. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic4b28c936bc0e2b42339f2b00219177f77b8d85f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Add rpc stats to readahead tests RPC generation behavior is a key component of readahead, and the plan is to add tests of rpc generation to all the readahead tests. In the meantime, we can at least add output of rpc_stats whenever we output read_ahead_stats to make the tests more informative. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If18ed6bb43a5afb2a7b0c9267f2f056cd33c66d7
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Improve test 101a Test 101a is very old and uses some odd methods to control caching. Remove those, and switch to a simple cache flush to ensure data is read from disk. Shrink the test size to make it more consistent (previously it was hitting cache a lot, which made the RPC stats harder to predict). Also add RPC count checking - this will be added to the other readahead tests as well. Test-Parameters: trivial testlist=sanity env=ONLY=101a,ONLY_REPEAT=50 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I39b1ed23e4c080af9e3689d32ac60701c6c5a812
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Add ra after seek test Confirm read ahead can restart cleanly after seeking in a file. Do this by reading the back half and then the front half of the file. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I30c0515c95af0c35fc38ed612f6409c15d55c3a8
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101i | seen in 5 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Stricter readahead tests A few small test tweaks to make particular readahead tests stricter and more broad ranging or improve output. Didn't have a clear home in other patches in the series. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic8164f6cc2db09492574a247705bd55bc1f60989
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_63b | seen in 34 other reviews |
| sanity2@ldiskfs+DNE:test_64a | seen in 34 other reviews |
| sanity2@ldiskfs+DNE:test_64c | seen in 34 other reviews |
| sanity2@ldiskfs+DNE:test_64d | seen in 78 other reviews |
| sanity2@ldiskfs+DNE:test_101i | seen in 6 other reviews |
| sanity2@zfs:test_101i | seen in 2 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-6 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15516 llite: unify readahead logic The mmap readahead logic is *almost* the same as regular readahead logic, but for what appear to be historical or accidental reasons, it is initialized differently and has several special cases as a result. There's no clear need for separate mmap and regular readahead logic, and the existing differences are harmful, as mmap read takes more misses than a regular read of the same pattern. But more importantly, having the special case makes it much harder to write and test improvements to the readahead code, since they must be written carefully to hit both paths and then tested both ways as well. There may be some application for separate tunings, but none has been persuasively shown so far (and none are in place currently). Clean up and unify the logic. We can't unify as much as we'd like, since mmap needs to do ras_enter only after we've locked the page - because if we can't lock the page, mmap doesn't proceed to do IO. This means we must wait to call ras_enter for mmap until we're in the readpage code, unlike for regular file reads, which call it at the llite layer. This is OK for mmap because mmap reads are a single page, but it's important for regular reads to call ras_enter only once per read, not once per page. Note even without other changes this reduces the number of misses taken in the simple mmap read test. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic6c33a2714a256072ef15d56c40f2bd1e39a1e6f
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_101f | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_101i | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101f | seen in 1 other review |
| sanity2@zfs:test_101i | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15516 llite: Remove clustered read code The clustered read code changes mmap readahead behavior so it ignores patterns, and reads a set of pages around each new read. This works well if you have a random read pattern and are going to use most of the file, but it's bad if there is a pattern, which there almost always is. The result is extremely harmful for basic patterns like 'read a whole file from beginning to end'. It also appears that the described clustered behavior (where a bunch of data is read semi-randomly in an area and then there's a jump to a new area) is not real application behavior, instead it was a stopgap for the inability to support certain common database read patterns, like loose forward and sequential or loose sequential reverse. The clustered code, instead, breaks the ability to properly handle simple patterns like sequential forward read in mmap. It also creates a large section of 'mmap only' readahead logic, which makes it impossible for mmap reads to benefit from most readahead improvements. Having this separate logic path also makes testing and verification of readahead much more difficult. A 'read a cluster of pages during random read' feature is a reasonable one and could be helpful in some scenarios. Parts of this code can serve as a reference for that, but the existing code should be removed. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I15b203bc0692098614b691344d474a226ba3df4a
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_101i | seen in 5 other reviews |
| sanity-pcc@ldiskfs+DNE:test_7b | seen in 10 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15069 llite: improve ras usage The readahead state is file level and shared, the ria is a per-IO version of that state. Don't access the ras when the ria is sufficient. Also rename one function. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4de4e093d7d8e690b49852f26e2f44ae4474671a
| unique failing test | history |
|---|---|
| sanity-pcc@ldiskfs+DNE:test_5 | seen in 13 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm crashed | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
LU-15033 llite: rework end_idx handling There is a separate ra_end_idx argument passed to ll_read_ahead_pages, which captures the last page read and is then compared to the contents of ra_io_arg, which is also passed to ll_read_ahead_pages. This results in comparisons like: if (ria->ria_end_idx == ra_end_idx) which are a little tricky to understand. Instead, we put this in ra_io_arg and name it ria_last_read_idx. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I774e69c6a6334d2f56a4bc0c85ed00a5f2757e35
| unique failing test | history |
|---|---|
| sanity-sec@zfs:test_21 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-6 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-selinux. | session |
| review-dne-zfs-part-2 crashed | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
LU-16741 ptlrpc: remove unnecessary asserts ptlrpc_free_committed is at no risk of being called with either a null import or the import lock unlocked. These don't have a significant cost, but they're not adding much either. Removed 'trivial' from this patch to ensure the LU-16741 refactoring series gets at least one full run. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I90d3ddae2cfbb9c748e3ba004982bca3ef465667
| unique failing test | history |
|---|---|
| sanity-pcc@ldiskfs+DNE:test_18 | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne failed 2× | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-16741 ptlrpc: rename ptlrpc_free_request ptlrpc_free_request doesn't free requests, it commits them. Rename it accordingly. One nag: there is one call to rq_commit_cb() outside this function, in after_reply(). It is not clear to me how/why that call is different, so I am leaving it untouched. But ideally we would only call rq_commit_cb() from this function. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I52e81076a1d4e55fb83f9d4d6c86df64393004b0
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101ab | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 | RHEL 8.8/x86_64 | ran 11 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.8/x86_64 | ran 11 tests. 1 tests failed: sanity-flr. | session |
| review-dne-zfs-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.8/x86_64 | ran 10 tests. 1 tests failed: sanity-flr. | session |
LU-15155 llite: Make readahead request locks Currently, readahead will not request an LDLM lock if it encounters a region without one. This causes it to take misses and can confuse the readahead state as well. Not requesting locks for readahead is an artifact of the idea that readahead is an optional optimization, but it's almost as important as full reads/writes from userspace, and we should request locks for it. This will help cut down misses when starting to read a new file, which is particularly helpful in tests, where total I/O is small and the extra misses make it hard to predict behavior. However, to give better behavior under conflicting workloads, we make the lock requests from readahead nonblocking. This means it will get a lock if there is no conflicting lock, but otherwise will not. We also limit it to one lock request per stripe per readahead invocation, since otherwise it would ask for every page. The benefit to requesting locks can be seen in the test changes - miss counts are reduced because stripe count no longer factors in, and we can reenable async readahead because it no longer fails due to this (leading to unpredictable numbers of misses). Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie62a282245d036308ab0c6f8392af1098a74befc
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101ab | seen in 2 other reviews |
| sanity2@zfs:test_101n | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15178 llite: Clarify async vs nowait The existing code mixes the concepts of 'async' lock requests and 'nowait' (nonblocking) lock requests in to one term: 'speculative' lock requests. This prevents us from creating synchronous nonblocking/nowait lock requests, for no good reason. This patch clarifies the code, separating these concepts and allowing synchronous non-blocking lock requests. This is important because it allows readahead to make 'optional' lock requests, where it will wait for the request, but it does not want to cancel other locks. In essence, readahead would like to have the lock for immediate use so it can complete the readahead request, but it would prefer not to cancel other locks. Thus, synchronous non-blocking lock requests. This is implemented for readahead in another patch in this series. It also renames the badly named LDLM_FL_SPECULATIVE flag, because this flag actually implements 'NOWAIT' behavior. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0cbd44fe07e0c0206ba56eaef62590b19ba082eb
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_124c | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity2@ldiskfs+DNE:test_124d | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity2@ldiskfs+DNE:test_134a | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity2@ldiskfs+DNE:test_812a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_812b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_816 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_398a | seen in 3 other reviews |
| sanity2@zfs:test_812a | seen in 1 other review |
| sanity2@zfs:test_812b | seen in 1 other review |
| sanity2@zfs:test_816 | seen in 1 other review |
| sanityn@zfs:test_109 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@zfs:test_113 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.7/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-6 | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.7/x86_64 | ran 5 tests. 2 tests failed: sanity-selinux, sanity-sec. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.7/x86_64 | ran 4 tests. 2 tests failed: sanity-pfl, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-3 crashed | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 3 tests failed: sanity-quota, sanity-hsm, sanity-flr. | session |
| review-dne-zfs-part-5 crashed | RHEL 8.7/x86_64 | ran 8 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-6 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity-lnet. | session |
| review-ldiskfs-ubuntu | RHEL 8.7/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs | RHEL 8.7/x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
RE: rq_commit_cb and imp_lock This snippet of code strongly suggests no. (And this looks to me like it gets called regularly and isn't some weird stub or corner case, so it seemed solid evidence.) I actually considered doing this for the rq_commit_cb in the free_committed code, but you can't easily do so because you're walking the lists on the import. But more to the point, since that could be worked around, I'm skeptical dropping and taking the lock for *every* rq_commit_cb() in free_committed is a good idea. (If the rq_commit_cbs were the large majority of the work, dropping the lock and re-attacking the list in free_committed might make sense, but I don't think they are. It seems more an invitation to thrashing the lock and the memory for the lists between many CPUs.)
I may want to reflect on the code organization here - this seems a bit confusing. Let's see if it's correct first and then I'll try to think about that.
I'm guessing we're sometimes missing calling this section of code (And maybe not calling this whole function), so we're still referencing the import, since I've seen both OSC and OFD modules getting stuck.
stale comment, need to remove
Will probably just undo this to show I'm not changing this area...
(defect?) one thing that concerned me about *only* allowing the "now > start + 3" exit condition is that if the replay list was very long and filled with unfreeable RPCs (a million file opens?) then it seems possible the thread could loop through requests for a few seconds and not find anything, then exit with no RPCs to free. Then the next thread enters this code and does the same thing, since it restarts the scanning at the beginning of the list. So the exit condition (before this change) was: - accumulate at least 128 RPC and have been scanning > 3s - or walk the whole list (until reqs with transno > last_committed are found) without finding 128 freeable RPCs That way, each call here cleans up at least 128 RPCs (if there are that many), or hopefully finishes list walking much sooner. That amortizes the list walking over more RPCs. The main question is how much of the list walking does not produce "useful work"? With the reduction in lock contention, this could always reduce the number of entries per call. I also appreciate that if other CPUs are stuck on a spinlock then they are also burning cycles unproductively, so in theory as long as each pass made *some* forward progress (a handful of RPCs) then it would be OK to exit and leave it to the next lock waiter.
Hmm, so I think the big threat here is if we have an *extremely long* but unfreeable list (except for maybe a few at the far end which we don't reach), we could loop forever as we hand the problem off to a new thread that retraces our steps. Or at least, that is the problem taken to an extreme, where we get no work done so we make no progress. But isn't it the case that the worst case scenario is we essentially end up hung until the *next* commit comes through and that huge pile of RPCs is now freeable? Because they have to be freeable some time. I'm trying to decide how plausible it is as well - If we have, say, 10 million RPCs on the list (being generous) and, we say we've got 1 second, we have 100 nanoseconds (300 nanoseconds per for 3 seconds) for each RPC (to do the list walk + checks). Given that a cold DRAM access can be on the order of 90 ns (thanks, Google), yeah, that's not crazy. So the amounts of time are plausible. But I think we still have the backstop of "the next commit arrives and they become freeable", don't we? So the worst case is that we could get stuck looping (handing from thread to thread without real forward progress) until the next commit arrives. So I think this is OK. Does that wash? Have I missed/misunderstood something?
I _think_ that open-but-committed requests are still kept in this list in order to ensure the open is replayed before any later requests that may unlink the file without preserving the refcount. That said, I've long wanted to divorce open-committed requests from RPC replay so that they don't clog up the replay queue and also so that we don't need to keep exact copies of RPCs in memory for hours/days/weeks, since that complicates ever changing the RPC format over an upgrade. This allows a few different improvements to be implemented: - LU-5703 "Quiesce client mountpoints from the server" - LU-3290 "disallow ptlrpc RPCs with old client XIDs" - LU-15250 "RPC Replay Signature" and also simplifies the RPC handling code, since we don't need to preserve the close RPC replay after the open RPC has committed.
Style: Don't need braces here...
Note this was wrong in the previous patchset, but wasn't affecting behavior.
LU-16741 ptlrpc: defer & parallelize free_committed ptlrpc_free_committed can be extremely time consuming when there are many async requests outstanding, such as with small async DIO as potentially created by LU-13805, or in other unusual circumstances. Most of the work (in terms of time consumed) in ptlrpc_free_committed can be deferred and moved out from under the imp_lock. Additionally, if there is a process waiting to do the ptlrpc_free_committed work, the current thread can grab a 'batch' of requests to process, then drop the import lock and allow the waiting thread to grab the next batch. This splits the work across waiting threads. In cases where the lock is highly contended, it has the effect of having each thread do one 'batch' of work, but mostly in parallel. This has the effect of parallelizing this work and speeds it up enormously when there are many threads (the most important case, as they can generate a lot of work). If there are only a small number of threads working on the import, the load can be split unevenly - Consider the case of two threads. The first to arrive defers one batch of requests, notices the second thread is waiting, and leaves all the remaining requests for that thread. This is uneven, but doesn't really matter for a few reasons: 1. In the 'small number of threads' case, there isn't much work to do anyway, so it's not very impactful if it is unevenly distributed. 2. If it really takes a long time, the first thread will arrive again as a 'waiter', and will pick up work. A good example is this IO500 ior-hard-write test with async DIO; this is a bit of an extreme example, and this also helps IO500 hard write without async DIO, but that hits other limitations so it is harder to see the benefit. mpirun -n 80 ior -k -e -o $file -t 47008 -b 47008 -s 11990 -w -a POSIX --posix.odirect Performance without the patch: 2018.19 MiB/s Improves to: 4800 MiB/s with this patch (This was done with fake_io to reduce server side limits and better show the contention.) And the time spent in the imp spinlock drops from 40-50% of all client CPU time to ~0% (imp_lock spinning no longer shows up in the perf traces). Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5e9b8d556770dc4a33dce0ceb50f745201328c5e
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-1 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: sanity-pfl, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-2 crashed | RHEL 8.7/x86_64 | ran 13 tests. 4 tests failed: sanity-sec, sanity-lfsck, runtests, replay-dual. %% THIS TEST SESSION CRASHED % | session |
| review-dne-part-3 crashed | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 8.7/x86_64 | ran 11 tests. 2 tests failed: sanity-quota, sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 crashed | RHEL 8.7/x86_64 | ran 10 tests. 3 tests failed: sanityn, sanity-scrub, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-6 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-7 crashed | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-2 crashed | RHEL 8.7/x86_64 | ran 7 tests. 2 tests failed: sanity-selinux, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-1 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: sanity-pfl, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-2 crashed | RHEL 8.7/x86_64 | ran 13 tests. 4 tests failed: sanity-sec, sanity-lfsck, runtests, replay-dual. %% THIS TEST SESSION CRASHED % | session |
| review-dne-zfs-part-3 crashed | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-5 crashed | RHEL 8.7/x86_64 | ran 10 tests. 3 tests failed: sanityn, sanity-scrub, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-6 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-7 crashed | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs | RHEL 8.7/x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
| review-ldiskfs-arm crashed | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 8 tests. 2 tests failed: sanity-sec, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu | RHEL 8.7/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs crashed | RHEL 8.7/x86_64 | ran 12 tests. 2 tests failed: replay-single, sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
LU-16741 ptlrpc: check logic in osp_request_commit_cb osp_request_commit_cb code states that rq_commit_cb can be called on uncommitted requests. That seems wrong, in both the specific sense of "I doubt we do that" and the sense that "we should not do that". Note: This is NOT for landing; this is just exploratory. Test-Parameters: testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I09a20286bb3fde9a77396cebfce0028fd996bfa1
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
LU-16741 ptlrpc: add 'locked' to rq_commit_cb osp_request_commit_cb is unique among rq_commit_cbs in requiring the import lock be held. Add a parameter so we know when we need to take the lock in that callback. Note: This is NOT for landing; if this works it will be integrated in to the parent change. Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6c1b681eafeb2f805b41dc5654ab2e2c6e020f73
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
LU-16741 tests: Test locking around commit_cb Seeing if taking imp_lock() around commit_cb when doing the deferred work avoids the crash. Just for learning purposes - it's possible one of the commit callbacks has a hidden dependence on the import lock. Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie098251bb3ec78325bc813e7dd6946d503582894
| unique failing test | history |
|---|---|
| sanity2@zfs:test_398a | seen in 3 other reviews |
| sanity2@zfs:test_812a | seen in 1 other review |
| sanity2@zfs:test_812b | seen in 1 other review |
| sanity2@zfs:test_816 | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| review-ldiskfs-dne-arm | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
LU-16741 tests: Test to confirm replay-dual hits This is testing LU-16741 with no changes to confirm the specified replay-dual testing hits the bug reliably. Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ibc1a46df653cb9ce98b99ee3e5aef7705bfca111
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| review-ldiskfs-dne-arm | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
LU-16741 tests: Test locking around deferred work Testing to see if hold imp_lock() here avoids the crash. Just for learning purposes - this is NOT a proposed change since it would remove all benefit from the patch. Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0fdc1191a6e39f85715b5a1ae676dc916e70c8aa
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-2 retesting | RHEL 10.1 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-zfs-part-5 failed 2× crashed | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
Going forward, all of these lines should be replaced with a label: Assisted-by: ClaudeCode:MODEL_VERSION [TOOLNAME ...] https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution
Why is this for `fortestonly`? Should this be consolidated with another patch, or that annotation be removed?
Probably it was generated initially via AI, and nobody removed that designation as the patch was being updated?
(typo) The third path is `lov_io_lseek_end()`, which is the `.cio_end` entry in `lov_io_ops[CIT_LSEEK]` and runs from `cl_io_end()`, not from unlock. `lov_io_unlock()` is a separate op. Should this read "during sub-IO end"?
[Marc Bot] (style) This attribution line was flagged on patchset 34 and is still unresolved: it should use the `Assisted-by:` label format described at https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution instead of the free-form line.
"three independent paths" does not seem to hold for the third one.
The LSEEK sub-lock enqueue happens in cl_lockset_lock(), which cl_io_lock() runs *after* every cio_lock(), so the stripe is already marked by the time lov_io_call(cl_io_start) runs and the sub-IO is skipped there. A sub-IO that never started still has:
sub_io->ci_result = 0 /* lov_io_sub_init() */
sub_io->u.ci_lseek.ls_result = -ENXIO /* inherited from the parent in lov_io_sub_inherit(); ll_lseek() seeds it */
lov_io_lseek_end() already ignores both (`ci_result == 0` is a no-op, `sub_off == -ENXIO` hits the existing continue), so there is nothing for the third hunk to catch.
Also, ci_result propagation happens in .cio_end (lov_io_lseek_end), not during unlock.
(style) This was raised on an earlier patchset and the line is unchanged: tool attribution should use the `Assisted-by:` trailer format documented at https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution rather than a free-form sentence.
This guard sits in `lov_io_call()`, which is the shared dispatcher for four different ops:
lov_io_lock() -> lov_io_call(cl_io_lock)
lov_io_start() -> lov_io_call(cl_io_start)
lov_io_iter_fini() -> lov_io_call(lov_io_iter_fini_wrapper)
lov_io_unlock() -> lov_io_call(lov_io_unlock_wrapper)
All four are registered for CIT_LSEEK, so a stripe marked LSS_READ_ERR also skips `cl_io_unlock()` and `cl_io_iter_fini()` on its sub-IO, not just `cl_io_start()`. Its `ci_state` then goes CIS_LOCKED -> CIS_IO_FINISHED (set by `lov_io_end_wrapper()` in `lov_io_lseek_end()`) -> CIS_FINI, never passing through CIS_UNLOCKED/CIS_IT_ENDED.
Nothing leaks today because `osc_io_ops[CIT_LSEEK]` registers only cio_start/cio_end/cio_fini and the LSEEK DLM lock is taken on the top IO by `vvp_io_lseek_lock()`. But the intent is only to skip the data-fetch pass -- would putting the check in `lov_io_start()` (or keying it on `iofunc == cl_io_start`) keep the cleanup passes balanced?
(style) This isn't a bug, but `str` reads like a string; the rest of this file spells it `stripe` (see `lov_io_lseek_end()` a few hundred lines down, which uses `index`/`stripe` for the same two values). Worth matching if the patch is refreshed.
[Marc Bot] (defect) Skipping the stripe treats its extents as holes, but on an EC file that data is still readable via parity reconstruction. If the only data between ls_start and the next healthy-stripe data lives on the degraded stripe, SEEK_DATA returns the later offset, or -ENXIO if none, so sparse-aware tools (cp, tar) silently drop data that read() would return. Also, if every sub-IO in lis_active is skipped (e.g. single-stripe data component), offset stays -ENXIO and SEEK_HOLE fails with -ENXIO even though ls_start < file size, which breaks the SEEK_HOLE contract of a virtual hole at EOF. Would it be safer to treat a degraded stripe's covered range as data instead of skipping it, or to return an error rather than a misleading data map? The commit message claim that the remaining healthy stripes provide valid seek results does not hold in these cases.
(defect) Skipping cl_io_start() for the degraded stripe means its allocation map never contributes to the seek result, and lseek does not reconstruct anything from parity the way CIT_EC_RD does. So the answer is not "the remaining healthy stripes are sufficient" -- it is an answer computed from an incomplete extent map.
Concrete case, 4+2 EC, one data OST deactivated, data written only in the range that maps to that stripe:
lseek(fd, 0, SEEK_DATA)
-> degraded sub skipped, others report -ENXIO
-> offset stays -ENXIO, lseek fails
and with data further out on a healthy stripe it returns that later offset instead. Sparse-aware copies (cp --sparse, tar, rsync) would silently drop the bytes that a plain read() still returns via parity.
SEEK_HOLE has the mirror problem: if every sub-IO covering ls_start is skipped, `offset` stays -ENXIO and ll_lseek() returns -ENXIO for an offset below i_size, which breaks the "there is always a virtual hole at EOF" contract.
Is returning an error preferable to returning a wrong offset here? Alternatively, could the degraded stripe's range be reported as data (conservative) rather than dropped?
+1; error should be preferable compared with a wrong offset
(typo) This comment uses a non-ASCII em dash; the rest of the tree is plain ASCII. Plain "-" or "--" instead.
This hunk looks like it has no effect. Any stripe marked LSS_READ_ERR was already skipped by the new check in lov_io_call(), so its sub-IO never reached cl_io_start(): ci_result is still 0 from lov_io_sub_init(), and ls_result is still the -ENXIO that lov_io_sub_inherit() copied from the parent. The `if (io->ci_result == 0)` assignment and the `sub_off == -ENXIO` continue below both already handle that. Is there a path where a sub-IO is marked LSS_READ_ERR but still ran? If not, dropping this hunk would keep the two skip conditions from having to stay in sync.
LU-12668 lov: handle ESHUTDOWN for LSEEK on EC files When an OST hosting a data stripe of an EC file is deactivated, SEEK_DATA/SEEK_HOLE fails with ESHUTDOWN because the error propagates through the LOV layer during sub-lock enqueue. Fix this by marking the degraded stripe LSS_READ_ERR in lov_lock_enqueue (same mechanism used by CIT_EC_RD for parity recovery), then skipping those stripes in lov_io_call and lov_io_lseek_end. This avoids ESHUTDOWN leaking through three independent paths: sub-lock enqueue, sub-IO function dispatch, and sub-IO ci_result propagation during unlock. Add lov_lsm_has_parity() helper to check if any layout entry has parity, needed because CIT_LSEEK locks the data component while parity lives in a separate entry. This fixes lfs mirror verify failing on EC files when any data OST is deactivated. Generated with Claude Code + Tools Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5cce4e0ea51c68b0c6fda1d83b694af19cad57bd
| unique failing test | history |
|---|---|
| sanity-lfsck@ldiskfs+DNE:test_18g | seen in 13 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18h | seen in 13 other reviews |
(minor) LU-19536 in JIRA is titled "fault_in_iov_iter_readable() with a spinlock held", and the other changes landed under it (DIO pool ENOMEM drain/retry, DIO buffer double-free, wait queue for DIO copy sync) are all in the unaligned-DIO copy machinery. Grant/dirty accounting for DIO writes reads like a separate topic. Would a dedicated ticket be a better home, or is LU-19536 intended as an umbrella for all of the DIO work?
(minor) The comment doesn't quite match what happens. osc_wake_cache_waiters() is only `wake_up(&cli->cl_cache_waiters)`, so nothing re-enters this function - there is no recursion. The actual hazard is that osc_enter_cache() evaluates this as the condition of wait_event_idle_exclusive_timeout_cmd() on that same queue, so the wake would land on the evaluating task itself (and osc_enter_cache() already wakes the queue once it succeeds). Worth rewording so the rationale for the no-wake variant is accurate.
(minor) This bakes in an assumption that any `cl_dio_pages` reaching the OSC is embedded in a `cl_sub_dio`, which isn't part of the `cio_dio_submit`/osc_queue_dio_pages() contract.
A few lines down the same sdio is already derived from the page anchor:
ext->oe_csd = anchor->csi_dio_aio;
Since `is_aio` is only needed inside the `!ext->oe_rw` branch, could it just read `ext->oe_csd->csd_ll_aio->cda_is_aio` there and drop the container_of()?
Please do this improvement.
LU-19536 osc: enforce dirty limits for regular DIO writes
osc_queue_dio_pages() checked only server grant before queuing a
write. For regular non-AIO DIO, parallel submission could therefore
bypass the per-OSC max_dirty_mb and global obd_max_dirty_pages limits.
Add osc_reserve_dio_grant() to validate the local dirty limit,
atomically reserve global dirty pages, and reserve server grant under
the client lock. Check dirty pressure first so a simultaneous grant
shortage cannot hide the dirty limit.
When a regular DIO extent cannot enter dirty accounting, submit it
synchronously and wait for that extent before forming more unaccounted
RPCs. Preserve existing no-grant and true AIO paths. Keep buffered
osc_enter_cache_try() accounting unchanged.
Remove the obsolete DIO grant branch from osc_queue_sync_pages().
Since LU-13814, DIO requests use osc_queue_dio_pages(). Rename
__osc_unreserve_grant() to osc_unreserve_grant_no_wake() to document
the no-wakeup rollback needed by waitqueue callers. Add __must_hold
annotations to document the locking requirements of grant and cache
helpers.
Add sanity test_398v using one exact OSC, warming its grant, and
pausing OST bulk completion. Verify that four DIO RPCs do not all
overlap at the dirty limit. Check final dirty counters and compare the
client and server grant changes around the tested DIO.
Fixes: 8efbad8ff4ed ("LU-13814 osc: add osc_queue_dio_pages")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I115232216ae77740f0a779994e8a020eedeebfef
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 crashed | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. %% THIS TEST SESSION CRASHED %% | session |
| custom-1002 crashed | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-subtest-change failed 30× | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-zfs-subtest-change failed 29× | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
(minor) The range here starts at 49b, but 49a "test concurrent reads during EC recovery" is added by this patch too. Should it read 49a-50b? Every other test added is accounted for by one of the ranges.
(minor) Two hunks are not accounted for by any of the ranges above: the new `[[ "$SLOW" == "no" ]] && EXCEPT_SLOW="74a 75b"` line, and the `head -n1` fix in `enable_ec()` for the multi-mount `get_param` output. The `EXCEPT_SLOW` one also changes an existing test: 74a is not added by this patch and is not named in any of the range lists, so a reader can't tell from the message that it is being moved onto the SLOW list. Worth a sentence for each.
(defect) With 3 of 4 data stripes unreadable and pcount=2, -EIO is the only correct outcome; returning reconstructed-but-wrong bytes is silent corruption. Treating it as an acceptable pass means a real reconstruction bug in this path would go unnoticed.
48a, 48c and 64b in this same patch take the opposite position ("handing back silently wrong data would be a corruption bug, so assert against it"). Should 52b assert rc != 0 instead?
(minor) These eight files (2M..16M) are never removed. The same applies to the extra files in 71b/71c/71d, 73b's .pure, 73c's .zero and the directories in 75e/75f; 73e is the only new test that registers a cleanup. Added up, the new tests write on the order of 1.5G and leave most of it in place for the rest of the run, which on a small test filesystem can push later subtests into ENOSPC. A `stack_trap "rm -f ..."` next to each creation would keep the footprint bounded.
(suggestion) Both of these are parked against the feature ticket itself. 53a in particular is described as intermittently reconstructing wrong data, which is a silent-corruption symptom rather than a test problem, and 41d is an OSC retry loop that never terminates. Would separate LU tickets referenced here keep them from being lost when LU-12668 is closed? The convention elsewhere in this file (12a -> LU-19631, 5b/12b -> LU-20435) is one ticket per known failure.
(minor) A fresh `stack_trap` is pushed on every call, including repeat calls for the same index, so loop-driven tests accumulate identical cleanup entries: 75b registers 50, 74a 20, and 58a/63b/68b one per pair. Each entry re-runs `ec_apply_fault`, which is a `do_nodes` to every OSS plus a `cancel_lru_locks osc`, so teardown does that work dozens of times over. Registering the trap only when the bit was not already set would make it one entry per OST.
(minor) When no stripe list is given this walks every data stripe and returns the first parity-free one, which can be a stripe that holds no data. For a sub-raid-set file that is the vacuous-pass mode this helper is documented as preventing: 42a writes 512K into a 1M-stripe 4+2 layout, so only stripe 0 has data. If stripe 0's OST happens to double as parity, the fault is armed on stripe 1's (empty) object and the checksum comparison succeeds without reconstructing anything. `ec_data_stripe_osts()` handles the analogous case by calling `skip_env`. Would returning non-zero (so `ec_start_read_fault()` skips) be safer than falling through to a later stripe?
(minor) 44c and 44a are the same test - same layout, same `ec_start_all_reads_fail`, same EIO check, same clear-and-reread - differing only in the error strings. 43c is that body minus the reread. Could these collapse into one? While here, 43c's description says "too many OST failures (3+ OSTs)" but `ec_start_all_reads_fail` uses `fail_val=0`, which fails every OST, not three.
(minor) The comment says "an OST that is a safe data OST for all files", but only `$f2` is classified. `$f1` (2+1, 3 objects) and `$f3` (2+2, 4 objects) get whatever the allocator gave them, so on an 8-OST config the victim frequently holds none of their objects and the `$s1`/`$s3` comparisons pass without any recovery running. 71a, 71c, 71d and 75e/75f have the same shape (classify one file, assert on all of them). That is defensible for a batch test, but here the comment claims something stronger than the code does.
(defect) `safe_osts` here still comes from the `ec_classify_osts $tf` above, but the SEEK_DATA/SEEK_HOLE checks below run on `$tfs`, which was created separately and gets its own object placement from the allocator.
So the OST taken out need not hold any of `$tfs`'s stripes, and on a run where it doesn't, the two `lseek_test` assertions execute against a fully healthy file.
73c already documents and avoids exactly this ("safe_osts still describes $tf; $tf2 has its own object placement") by calling `ec_data_stripe_osts $tf2 0` first. Should 75d do the same for `$tfs`?
Related: the comment says the seek "crosses the degraded stripe", but the data lives at 5M, i.e. stripe 1 with `-c 4 -S 1M`, while `safe_osts[0]` is just the lowest-numbered data OST.
(minor) The negative-index handling, and the "Index -1 is the last stripe" note in the header comment, appear to be unreachable: all eleven callers pass 0..4. Worth dropping the branch and the doc line unless a caller is coming.
(style) The suite convention is a `#define` comment naming the fault right above the line that arms it, so a reader does not have to look up the bare hex. `ec_start_all_reads_fail()` and 41d both do this; this call site and the one in `ec_apply_fault()` do not.
#define OBD_FAIL_OST_BRW_READ_BULK 0x20f
ec_ost_fail_loc 0x20f $(( 0x10000 | mask ))
The value itself is right (obd_support.h has 0x20f), it is only the annotation that is missing.
LU-12668 tests: add EC recovery tests Add sanity-ec coverage for erasure-coding recovery. Each test writes an EC file, resyncs parity, fails one or more OSTs, and verifies the client reconstructs the data from parity (CIT_EC_RD) against the pre-failure checksum. Failure is injected with OBD_FAIL_OST_BRW_READ_BULK so the OSC import stays active and only bulk reads fail, which drives genuine parity reconstruction. The fault is set on the OSS nodes, where tgt_brw_read() evaluates it, and osc.*.resend_count is dropped to 1 for the duration so the injected -EIO reaches the LOV layer instead of being absorbed by an OSC resend. Victims are chosen by stripe rather than by OST index. ec_pick_data_ost() walks a file's data stripes in order and takes the first whose OST does not also carry parity: a file smaller than one raid set holds data on stripe 0 alone, so picking the lowest OST index instead would arm the fault on an object the read never reaches and the test would pass without exercising recovery. ec_check_fault_index() skips when a target OST index is >= 16, which cfs_fail_index() cannot express in its 16-bit fail_val bitmask. ec_mirror_victims() fails one parity-free OST in every data mirror, since a file with more than one data copy would otherwise answer the read from an intact mirror rather than reconstructing anything. Reads that check a sub-range cancel their locks first: a range re-read after a whole-file read is otherwise served from the page cache, issues no BRW RPC, and so never reaches the injected fault. Geometry and I/O patterns (40b-44c): - 2+1, 2+2 and 4+1 EC; partial, offset, mmap, direct and async reads; single, maximum and progressive OST failure; graceful failure when too many OSTs are gone Failure placement and layout (45a-48c): - consecutive, non-consecutive, boundary and parity-only OST failures; 64K, 256K and 4M stripe sizes; multiple EC and mixed EC/non-EC PFL components; stale and partially stale parity Concurrency and multi-mount (49a-50b): - concurrent readers over a file with two failed data OSTs; background writes, mirror resync and OST reactivation during recovery; both mounts reading the same EC file Core recovery (51a-53a): - EOF boundary recovery at RAID set / stripe set boundaries; degradation limits and mixed parity+data failure; recovery at non-zero read offsets Layout patterns (55a): - file-size boundaries (1 byte .. multi-stripe) Edge cases (58a-62d): - parity_used combinations and stripe rotation; sparse files with holes; recovery after truncate; sub-stripe files; append writes Multi-target (63a-66d): - OST failure cycling and multi-mount coordination; 3-4 component PFL with per-component EC geometry; multiple EC mirror pairs Write patterns (67a-71d): - writes to healthy stripes during degraded mode and overwrite cycles; varied geometries; O_DIRECT writes; partial and mid-file overwrites; batch recovery of many files Stress and admin (73a-73e, 74a, 75a-75f): - random reads, fallocate, truncate-extend, O_APPEND, and large (128M) file recovery; reads racing with OST deactivation; stat/getattr, stress loops, lfs mirror verify, directory-inherited layout, and stripe rotation Tests 41d and 53a are added but listed in always_except. A degraded mmap read never completes: the OSC alternates between "too many resent retries" and osc_brw_redo_request() forever, so -EIO never reaches the LOV layer and CIT_FAULT never switches to CIT_EC_RD. A recovery read at a non-zero offset intermittently reconstructs wrong data. Both stay off until those are fixed. Test 65c and the sparse half of 75d are skipped on ZFS. Both build a sparse file and resync it, which needs lseek to report the holes so resync knows which stripe sets to skip, and ZFS does not report them reliably for dirty data. Test 12b describes the same problem. Assisted-by: ClaudeCode:opus llm_code_and_review_tools Test-Parameters: trivial testlist=sanity-ec ostcount=8 Test-Parameters: trivial testlist=sanity-ec ostcount=8 fstype=zfs Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Max Dilger <mdilger@whamcloud.com> Change-Id: I5a06cd166487e0bff7bfdb6a39414af3f12c4326
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: recovery-small, lustre-rsync-test. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 7 tests. 2 tests failed: recovery-small, lustre-rsync-test. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: recovery-small, lustre-rsync-test. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
How is this better than just incrementing rpcs_in_flight under cl_loi_list_lock and decrementing it on failure? Does this mean we are now double counting some RPCs until the new counter is decremented?
In the upstream kernel this is expressed as:
Reported-by:
Or:
Suggested-by:
why remove this comment?
(style)
for ((i = 0; i < 16; i++)); do
(style) can this just grep for `5:` in the output?
$LCTL get_param -n osc.$osc.rpc_stats | grep "^5:" &&
error "found more than 4 RPCs in flight" || true
I guess this needs to only follow the `rpcs in flight:` section, so possibly:
```
$LCTL get_param -n osc.$osc.rpc_stats | grep -A 8 "rpcs in flight:" |
grep "^[5-9]:" && error ...`
```
This also detects the case where 5 RPCs-in-fight are somehow all skipped...
LU-19755 osc: fix race in max_rpcs_in_flight check When multiple ptlrpcd threads process RPCs concurrently, they can all pass the osc_max_rpc_in_flight() check before any of them has incremented the in-flight counter. This happens because the check is done under cl_loi_list_lock but the counter is incremented later in osc_send_*_rpc() after the lock is released. Fix this by adding a cl_pending_in_flight counter that reserves a slot while still holding the lock. This counter is included in rpcs_in_flight() so concurrent threads see the reservation and wait appropriately. Add OBD_FAIL_OSC_DELAY_RPC to allow testing this fix by injecting a delay in the race window, and sanity test 55d to verify the limit is respected under concurrency. Thanks to Jinshan Xiong of Google for reporting this issue and suggesting the fix. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8ceacf3d9040d94fc89ab54a39bd98a4fb35ae1d
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 crashed | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
LU-19956 osc: fix race from direct ops_transfer_pinned clear The ops_transfer_pinned flag in osc_page is paired with a cl_page reference -- the flag and the ref must always be managed together through the osc_page_transfer_get/put() accessors. osc_completion() violated this by clearing the flag directly and dropping the ref with a separate cl_page_put(). This decoupled the flag from the ref, opening a race on weakly-ordered architectures. The initial fix (complete first, then transfer_put) introduced a re-submission crash: once cl_page_complete() transitions the page to CPS_CACHED, a concurrent write can re-submit it via osc_page_cache_add -> osc_page_transfer_get, which asserts the pin is clear. If the old completion has not yet called transfer_put, the assert fires. Fix osc_completion to release the transfer pin BEFORE cl_page_complete(). While still in CPS_PAGEOUT, the state machine prevents any other transition, so the transfer_put is race-free. Take a temporary cl_page ref first: transfer_put drops the pin's ref (cp_ref 2->1), and cl_page_complete's end_page_writeback can make the page reclaimable on another CPU. A TLA+ formal model (formal_models/clio/TransferPin.tla) with the NoPinWhileCached invariant verifies this fix catches the re-submission crash in all three buggy variants (original, LASSERT-in-delete, complete-first). Generated with Claude Code + Tools Test-Parameters: testlist=sanity env=ONLY=80a Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ib8416c753e13fe6eb8e11790b63687fbc2c8a26d
| unique failing test | history |
|---|---|
| conf-sanity1@ldiskfs+DNE:test_25 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| conf-sanity2@ldiskfs+DNE:test_50a | seen in 1 other review |
| conf-sanity3@ldiskfs+DNE:test_101a | seen in 6 other reviews |
| conf-sanity4@ldiskfs+DNE:test_151a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| conf-sanity1@zfs:test_30a | seen in 3 other reviews |
| conf-sanity2@zfs:test_51 | seen in 2 other reviews |
| recovery-small@ldiskfs+DNE:test_52 | seen in 8 other reviews |
| recovery-small@zfs:test_52 | seen in 6 other reviews |
| replay-dual@ldiskfs+DNE:test_28 | seen in 1 other review |
| sanity2@ldiskfs+DNE:test_123aa | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity3@ldiskfs+DNE:test_398m | seen in 2 other reviews |
| sanity-slow@ldiskfs+DNE:test_255a | seen in 17 other reviews |
| sanity1@zfs:test_56xb | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_123aa | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-quota@ldiskfs+DNE:test_33 | seen in 1 other review |
| sanity-quota@zfs:test_13 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@ldiskfs+DNE:test_18 | seen in 4 other reviews |
| sanity-sec@zfs:test_18 | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_43k | seen in 2 other reviews |
| sanityn@zfs:test_45j | seen in 4 other reviews |
(minor) duplicate code block? I can't see any difference
Doh, "readable" vs. "writeable". I was confused that they were both checking HAVE_FAULT_IN_IOV_ITER_READABLE...
(style) this would be more clear if the same #ifdef was not checked twice:
```
#ifdef HAVE_FAULT_IN_IOV_ITER_READABLE
#define ll_iov_iter_fault_in_readable(iov, bytes) \
fault_in_iov_iter_readable(iov, bytes)
#define ll_iov_iter_fault_in_writeable(iov, bytes) \
fault_in_iov_iter_writeable(iov, bytes)
#else
#define ll_iov_iter_fault_in_readable(iov, bytes) \
iov_iter_fault_in_readable(iov, bytes)
#define ll_iov_iter_fault_in_writeable(iov, bytes) \
iov_iter_fault_in_writeable(iov, bytes)
#endif
```
(defect) it looks like this will leak cdp->cdp_pages if an error is returned.
(defect) same
(minor) this should be moved to the end and cleaned up in one place:
```
if (unlikely(result != page_count)) {
CDEBUG(D_PAGE, "ll_release_user_pages() result=%ld, page_count=%ld\n", result, page_count);
if (result >= 0)
- return -EFAULT;
+ result = -EFAULT;
+ GOTO(out_free, size = result);
}
pvec->ldp_count = page_count;
+out_free:
+ if (size < 0) {
+ ll_release_user_pages(pvec->ldp_pages, page_count);
+ pvec->ldp_pages = NULL;
+ }
return size;
```
LU-0000 llite: fault in pages before get_user_pages We must fault in the user pages before get_user_pages, otherwise we can livelock with the mmap sem. Not sure about the requirement for the other case, on newer kernels - but it's worth a try. Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8a9a8094101e37a12d59482efb6a788231233837
| unique failing test | history |
|---|---|
| sanity2@zfs:test_119l | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 10.1 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-subtest-change | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
(minor) ... so it could also be set and checked via `chattr +t FILE` and `lsattr FILE` commands from e2fsprogs.
The "Changes:" list mentions adding LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE, but the same hunk also adds LUSTRE_COMPR_FL to that mask and removes the duplicate LUSTRE_NOATIME_FL entry. Neither is explained. The COMPR change is an independent server-side behavior change - should it be split into its own patch? The lustre/utils/lfs.c hunk (skipping the range validation for nohybrid) isn't described either.
This changes UAPI flag definitions that the MDT interprets, so interop with an older server is worth an explicit test run. Consider adding something like:
Test-Parameters: testlist=sanity serverversion=2.16.0
New wire flag, but wirecheck.c and the two wiretest.c copies don't look updated - the neighbouring LUSTRE_*_FL values have CHECK_VALUE_X entries. Same for LU_LADVISE_NOHYBRID, which needs a CHECK_VALUE next to the other LU_LADVISE_* ones (LU_LADVISE_AHEAD seems to have been missed earlier too).
The compatibility claim points at the wrong side. Older clients aren't the problem; older servers are.
On a pre-patch MDS, LUSTRE_FL_USER_VISIBLE has no NOTAIL bit, so mdt_setattr_unpack() hits
if (rec->sa_attr_flags & ~LUSTRE_FL_USER_VISIBLE)
RETURN(-EOPNOTSUPP);
and lfs ladvise -a nohybrid fails with EOPNOTSUPP. Clearing the flag still "succeeds" as a no-op, so the two directions behave differently. Worth stating the required server version here.
This was asked on patchset 7 and looks unaddressed: adding LUSTRE_COMPR_FL here is unrelated to nohybrid and changes server behavior. mdt_setattr_unpack() masks with LUSTRE_FL_USER_MODIFIABLE, and osd_attr_set() replaces the whole masked set, so this makes the compression flag both settable and clearable on the MDT inode by any client. Is that intended ahead of the compression work landing?
(defect) why is NOATIME being removed?
Is the COMPR flag really user modifiable or just visible? And does it make sense to allow this to be set before CSDC is landed to master?
Should this use NOHYBRID?
Does this also set the flag directly on the inode, or is the inode here the root or parent directory?
parse ll_file_ioctl():error: Function too hairy. Giving up. 4 seconds warn: ll_file_ioctl():Function too hairy. No more merges.
ll_inode2ext_flags() is not a full picture of the file's flags - it rebuilds them from inode->i_flags via ll_inode_to_ext_flags(), which only knows SYNC/NOATIME/APPEND/DIRSYNC/IMMUTABLE/ENCRYPT, plus the PROJINHERIT and (new) NOHYBRID lli_flags bits.
LUSTRE_NODUMP_FL and LUSTRE_NOCOMPR_FL are in LUSTRE_FL_USER_MODIFIABLE but have no i_flags or lli_flags representation, so they come back as 0 here. osd_attr_set() then does a wholesale replace:
ei->i_flags = (ei->i_flags & ~LDISKFS_OSD_USER_MODIFIABLE) |
(attr->la_flags & LDISKFS_OSD_USER_MODIFIABLE);
so those bits get cleared on disk. chattr +d FILE followed by lfs ladvise -a nohybrid FILE should lose the 'd' flag.
The FS_IOC_SETFLAGS path avoids this by calling fileattr_get() first, which fetches body->mbo_flags from the MDT. Should this do the same before OR-ing in LUSTRE_NOHYBRID_FL?
Related: ll_set_project() builds op_attr_flags from ll_xflags_to_ext_flags() and also sets OP_XVALID_FLAGS, so lfs project on a file appears to clear the nohybrid flag for the same reason.
LU-19839 llite: add persistent nohybrid I/O flag Add a persistent flag to prevent hybrid I/O switching for specific files. This uses the NOTAIL flag (0x00008000) which is not used by Lustre/ext4 and unlikely to be used in the future. The flag is set via ladvise interface for discoverability but uses FS_IOC_SETFLAGS internally for implementation. When set, hybrid I/O will not switch the file from buffered to direct I/O regardless of I/O size. Changes: - Define LUSTRE_NOTAIL_FL and LUSTRE_NOHYBRID_FL - Add LUSTRE_NOTAIL_FL to LUSTRE_FL_USER_MODIFIABLE - Implement LU_LADVISE_NOHYBRID using FS_IOC_SETFLAGS - Update ll_update_inode_flags to handle NOTAIL flag - Add test_119l to verify nohybrid flag functionality Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6a69293801114e2a3015ed87f2258828922ab767
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 34 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 7 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 34 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 6 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 9 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 7 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 7 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 39 other reviews |
| sanity1@zfs:test_56xB | seen in 11 other reviews |
| sanity1@zfs:test_56xa | seen in 39 other reviews |
| sanity1@zfs:test_56xab | seen in 8 other reviews |
| sanity1@zfs:test_56xc | seen in 35 other reviews |
| sanity1@zfs:test_56ej | seen in 11 other reviews |
| sanity2@zfs:test_119e | seen in 18 other reviews |
| sanity2@zfs:test_119f | seen in 17 other reviews |
| sanity2@zfs:test_119g | seen in 17 other reviews |
| sanity2@zfs:test_119h | seen in 17 other reviews |
| sanity2@zfs:test_119p | seen in 7 other reviews |
| sanity2@zfs:test_119q | seen in 7 other reviews |
| sanity2@zfs:test_398o | seen in 22 other reviews |
| sanity2@zfs:test_398s | seen in 7 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 10 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 33 other reviews |
| sanity-flr@zfs:test_0h | seen in 35 other reviews |
| sanity-flr@zfs:test_0j | seen in 34 other reviews |
| sanity-flr@zfs:test_36d | seen in 32 other reviews |
| sanity-flr@zfs:test_37 | seen in 36 other reviews |
| sanity-flr@zfs:test_38 | seen in 33 other reviews |
| sanity-flr@zfs:test_44b | seen in 9 other reviews |
| sanity-flr@zfs:test_61a | seen in 40 other reviews |
| sanity-flr@zfs:test_61c | seen in 33 other reviews |
| sanity-flr@zfs:test_70a | seen in 60 other reviews |
| sanity-flr@zfs:test_200a | seen in 33 other reviews |
| sanity-flr@zfs:test_200b | seen in 33 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 11 other reviews |
| sanity-hsm@zfs:test_607b | seen in 11 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 29 other reviews |
| sanityn@zfs:test_16j | seen in 32 other reviews |
LU-13814 osc: assert transfer pages identical At this point, we're about to stop using cl_page for DIO, and the transfer pages created with the cl_page and those created separately for DIO should be identical. Let's assert that for every value. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6c71f6c2a3e48a65f5abb18d7beb0699f7577d92
We aggregate attributes from multiple objects for other reasons (e.g. maxbytes, timestamps, size, etc.). The current implementation looks "mostly OK" and will work for basic linear IO patterns, where the file transitions from non-rotational to rotational. However, any other kind of IO pattern (e.g. HDF5 where some writes are at the beginning of the file and others at the end) might have a serious issue? IMHO, it would be better to aggregate this information once from the object (maybe from the OSC's the object is allocated on) and then be done with it? Storing a threshold when the file transitions from non-rotational to rotational would be best, as this could be computed once based on the layout. It is fairly unlikely that a file would have HDD stripes in the middle and flash at the beginning and end. Alternately (probably better) is to store the nonrot state in each layout component (which is a perfect 1:1 mapping), and then this can be checked at IO submission time to see what type of storage it is covering.
Hm, OK. I don't think we can do this as easily as we'd like - the issue is we have to have this information very early. I can consider this, though, your points are good ones. The good thing is this isn't essential to merging the core feature, particularly since we're leaving it off by default for 2.16.
Actually, an OST is (generally) only going to be rotational or non-rotational, and this is already returned to the client via statfs, so the clients should have full knowledge whether any IO is on flash or disk.
LU-13802 llite: add file nonrotational check This patch adds the ability to note whether or not the last IO to a file hit a rotational or a non-rotational device. This will be used in selecting the cutover thresholds for hybrid IO, since DIO is synchronous and much slower on spinning disk. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I75a9970f91b1776ed6f04ac0d000a9ba576df75a
error: 'io' may be used uninitialized in this function [-Werror=maybe-uninitialized]
I was wondering if the Janitor flags this. Technically, I don't think we can get here since `rc` would be -ENOMEM if we jumped out in line :2006 where `io` would not be set yet. Looks like that still needs to be fixed (compile failed). I guess it'd be easiest to keep the first call to `vvp_env_new_io()` and drop the second one?
LU-19109 llite: remove extra vvp_env_new_io call vvp_env_new_io is called twice in ll_file_io_generic, which is confusing since the second call overwrites the first. Fix this and a cleanup path mistake, where we assume the IO was set up after the call to vvp_env_new_io, which is wrong. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6186e17db8b04f01fd37ea7ac5d4b69b30d0258c
It would make sense to put an LLM to work on converting the NASM .asm files int GCC .S files. It should be possible to have it iterate on getting the GCC-compiled code to generate the same x86 byte code, or at least compile and run correctly to pass whatever correctness tests exist for this code.
In fact, a quick search shows that NASM already has a tool to do this:
Automatically convert Intel NASM assembly to GNU Assembler (GAS/GASM)
syntax using the intel2gas tool, which converts between NASM and AT&T
syntax, or by utilizing NASM's built-in output capabilities to generate
GAS-compatible objects.
$ intel2gas -i input.asm -o output.s
These generated/converted .S files should be stored alongside the original .asm files in Git. If NASM is available the originals can be compiled, but in the common case where NASM is not available the .S files would be compiled and linked into the kernel modules.
I think it is worthwhile to keep both, since it would be easier to update the .asm files from upstream ISA-L, but it might be some time before the .S files are update to match.
Are these bugs in the upstream ISA-L code that should be pushed back to them?
Similarly, is this code from upstream ISA-L, or a test wrapper that we developed?
This should probably use LU-19905, or change the description of LU-20016?
This table is the same between the x86 and aarch64 patches. It would be useful to fix one or the other to have the correct data for that CPU architecture.
(minor) this should have a warning that the C version may only be 1/20-1/30th as fast, at least until the .S versions are available.
(minor) It would be better to specify which kernel version is non-functional, so that this can become conditional upon a newer kernel (assuming there is a benefit to use SVE-optimized versions).
(minor) Again, please specify kernel versions so that this can be fixed in the future for newer kernel versions.
(style) `bool`?
(style) externs should be avoided in .c files
These warnings are correct and the prototypes should be moved to a .h file that is included here and into the file where these functions are implemented. Having the function prototype in a local .c file prevents the compiler from checking argument type/count against the implementation, and can result in hard-to-find bugs in the future.
(defect?) According to the comments for `kernel_neon_begin()`: ``` * Unless called from non-preemptible task context, @state must point to a * caller provided buffer that will be used to preserve the task's kernel mode * FPSIMD context when it is scheduled out, or if it is interrupted by kernel * mode FPSIMD occurring in softirq context. May be %NULL otherwise. */ void kernel_neon_begin(struct user_fpsimd_state *state) ``` I'm thinking that this would be declared on the stack in e.g. `gf_vect_dot_prod()` and passed to `ec_neon_usable()->kernel_neon_begin()` so that it can also be passed (as required) to `kernel_neon_end()`: ``` * The value of @state must match the value passed to the preceding call to * kernel_neon_begin(). */ void kernel_neon_end(struct user_fpsimd_state *state) ``` The `struct user_fpsimd_state` is 528 bytes, so a _bit_ heavy to put on the stack, but at the same time this is (very likely) to be on a PAGE_SIZE=64KiB system so stack space should be available for this, and it will not have a deep call chain below this point. I think doing a `kmalloc()` and `kfree()` for each call would add noticeable overhead and should be avoided it possible. Alternatives would include having a dedicated slab for this, so that there is a per-CPU cache (528 does not fit into standard slabs well) and it likely has local allocations objects cached in the slab.
(style) should this be changed to `rows >= 6` and remove the `case 6:` in the switch, as is done in `ec_encode_data_neon()` above?
LU-20016 ec: ISA-L SIMD for userspace and kernel
Integrate Intel ISA-L optimized assembly into Lustre's
erasure coding library for both userspace and kernel,
on x86_64 and aarch64. Add debugfs kernel benchmark
for measuring FPU save/restore overhead.
Userspace x86_64 (libec.a):
- 72 NASM assembly files (SSE/AVX/AVX2/AVX-512/GFNI)
- ec_multibinary.asm: runtime CPUID dispatch
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar when NASM unavailable
Userspace aarch64 (libec.a):
- NEON + SVE .S assembly + SVE C intrinsics
- ec_aarch64_dispatcher.c: getauxval() dispatch
- ec_aarch64_highlevel_func.c: N-vector dispatch
Kernel x86_64 (ec.ko):
- Pre-assemble ISA-L NASM .asm files into .o,
link into ec.ko (71 assembly objects)
- ec_dispatch.c: boot_cpu_has() selects AVX2/AVX/SSE
at module init, kernel_fpu_begin/end wrapping
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar in interrupt context
Kernel aarch64 (ec.ko):
- NEON .S assembly files (GAS format, direct kbuild)
- ec_aarch64_neon.c: kernel_neon_begin/end wrapping
with may_use_simd() check, hwcap detection
- SVE excluded from kernel (toolchain portability)
Build system:
- lustre-erasurecode.m4: detect NASM, aarch64
- erasurecode/autoMakefile.am: three-way dispatch
- utils/Makefile.am: link against libec.a
- ec/Makefile.in: NASM pre-assembly for kernel
Bug fixes in ec_perf_bench.c:
- -p flag was setting k instead of p
- frag_ptrs allocated stripe_size pointers not m
- Inverted exit code and stale rc from getopt
- Use aligned_alloc(64) for data buffers
Kernel benchmark (debugfs):
- /sys/kernel/debug/lustre/ec/benchmark
- 4 phases: FPU overhead, SIMD+FPU per-call,
SIMD+FPU amortized, C scalar baseline
- Input format (write to debugfs entry):
echo 1 > .../ec/benchmark (defaults)
echo "k=5 p=2 s=128" > .../ec/benchmark
cat .../ec/benchmark
Parameters (key=value, space separated):
k data stripes (default 5, max k+p=16)
p parity stripes (default 2)
s stripe size in KB (default 128, max 65536)
"echo 1" runs with all defaults. Unrecognized
input returns -EINVAL. cat shows results or
usage instructions if not yet run.
Fix gf_vect_mul_init naming to use _base suffix
consistently (aligns with ISA-L convention) in
both userspace and kernel ec_base.c.
Performance (5+2, 1 thread, QEMU VM, AVX2):
Stripe | Kernel | Kernel | Userspace | SIMD | Kernel vs
Size | SIMD | Scalar | SIMD | Speedup | Userspace
| | | | in Kernel|
-------|---------|---------|-----------|----------|----------
64 KB | 17099 | 540 | 404345* | 31.7x | 0.04x*
128 KB | 16888 | 531 | 197511* | 31.8x | 0.09x*
256 KB | 17032 | 539 | 98401* | 31.6x | 0.17x*
1 MB | 17142 | 538 | 24830 | 31.9x | 0.69x
4 MB | 10309 | 533 | 5385 | 19.3x | 1.91x
16 MB | 10954 | 534 | 1048 | 20.5x | 10.5x
All throughput in MB/s. (*) Userspace numbers at
small stripes are inflated by cache residency.
At 4-16MB the kernel is faster -- its kvmalloc
pages are physically scattered, matching real
Lustre page cache behavior. Replicating this in
userspace is non-trivial and not worth the effort;
the kernel benchmark is authoritative.
FPU save/restore overhead was measured at 25-54 ns
on this platform (Ryzen 3700X) -- negligible vs
encode time (~39us at 128KB). The benchmark
measures this separately (Phase 1) so it can be
characterized on other hardware where it may be
more significant.
Generated with Claude Code + Tools
Test-Parameters: testlist=sanity-ec
Change-Id: If4c57c328c0e23bb6769dcbc45d52d43b03a7837
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
This should use LU-19905.
LU-20016 ec: kernel SIMD, aarch64 support, benchmark
Add kernel-space SIMD erasure coding via ISA-L's
proven NASM assembly (pre-assembled at build time)
and aarch64 NEON assembly. Add debugfs benchmark
for measuring FPU save/restore overhead.
Kernel x86_64 (ec.ko):
- Pre-assemble ISA-L NASM .asm files into .o,
link into ec.ko (71 assembly objects)
- ec_dispatch.c: boot_cpu_has() selects AVX2/AVX/SSE
at module init, kernel_fpu_begin/end wrapping
- ec_highlevel_func.c: N-vector dispatch layer
- Falls back to C scalar in interrupt context
Kernel aarch64 (ec.ko):
- NEON .S assembly files (GAS format, direct kbuild)
- ec_aarch64_neon.c: kernel_neon_begin/end wrapping
with may_use_simd() check, hwcap detection
- SVE excluded from kernel (toolchain portability)
Userspace aarch64 (libec.a):
- NEON + SVE .S assembly + SVE C intrinsics
- ec_aarch64_dispatcher.c: getauxval() dispatch
- ec_aarch64_highlevel_func.c: N-vector dispatch
Kernel benchmark (debugfs):
- /sys/kernel/debug/lustre/ec/benchmark
- 4 phases: FPU overhead, SIMD+FPU per-call,
SIMD+FPU amortized, C scalar baseline
- Input format (write to debugfs entry):
echo 1 > .../ec/benchmark (defaults)
echo "k=5 p=2 s=128" > .../ec/benchmark
cat .../ec/benchmark
Parameters (key=value, space separated):
k data stripes (default 5, max k+p=16)
p parity stripes (default 2)
s stripe size in KB (default 128, max 65536)
"echo 1" runs with all defaults. Unrecognized
input returns -EINVAL. cat shows results or
usage instructions if not yet run.
Performance (5+2, 1 thread, QEMU VM, AVX2):
Stripe | Kernel | Kernel | Userspace | SIMD | Kernel vs
Size | SIMD | Scalar | SIMD | Speedup | Userspace
| | | | in Kernel|
-------|---------|---------|-----------|----------|----------
64 KB | 17099 | 540 | 404345* | 31.7x | 0.04x*
128 KB | 16888 | 531 | 197511* | 31.8x | 0.09x*
256 KB | 17032 | 539 | 98401* | 31.6x | 0.17x*
1 MB | 17142 | 538 | 24830 | 31.9x | 0.69x
4 MB | 10309 | 533 | 5385 | 19.3x | 1.91x
16 MB | 10954 | 534 | 1048 | 20.5x | 10.5x
All throughput in MB/s. (*) Userspace numbers at
small stripes are inflated by cache residency.
At 4-16MB the kernel is faster -- its kvmalloc
pages are physically scattered, matching real
Lustre page cache behavior. Replicating this in
userspace is non-trivial and not worth the effort;
the kernel benchmark is authoritative.
FPU save/restore overhead was measured at 25-54 ns
on this platform (Ryzen 3700X) -- negligible vs
encode time (~39us at 128KB). The benchmark
measures this separately (Phase 1) so it can be
characterized on other hardware where it may be
more significant.
Generated with Claude Code + Tools
Test-Parameters: testlist=sanity-ec
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id37586461376ab266c61ed541473279f83c790ff
LU-19744 doc: bulk man page review Bulk review done by Augment and Claude Code. Completed systematic review of Section 3 library function man pages (llapi_*), fixing grammar, formatting, function signature errors, and SEE ALSO ordering throughout all 123 pages. Major sections reviewed: - Changelog API (8 pages) - File Operations (13 pages) - Filesystem Info (9 pages) - Locking (2 pages) - Heat/IO Hints (3 pages) - HSM (11 pages) - Layout API (30 pages) - Misc API (8 pages) - PCC (12 pages) - Project Quota (8 pages) - Quota (1 page) - Remove by FID (2 pages) - Search/Discovery (6 pages) - Foreign Files (1 page) Critical signature fixes found during careful review: PCC (Persistent Client Cache): - llapi_pcc_detach_fid_fd.3: removed documentation for non-existent llapi_pcc_detach_fid_fd() function - llapi_pccdev_get.3: fixed parameter name (path->mntpath) - llapi_pccdev_set.3: fixed parameter name (path->mntpath) Project Quota: - llapi_project_fgetprjid.3: added missing 'struct' keyword - llapi_project_get.3: added missing 'struct' keyword - llapi_project_getprjid.3: fixed parameter type (const unsigned int->__u32) - llapi_project_open.3: fixed double pointer (*hdl->**hdl) - llapi_project_put.3: added missing 'struct' keyword Search/Discovery: - llapi_root_path_open.3: fixed parameter name (fd->outfd) - llapi_search_rootpath.3: added missing semicolons - llapi_search_tgt.3: fixed parameter names (pool_name->poolname, tgt_name->tgtname/mdtname/ostname) Foreign Files: - llapi_unlink_foreign.3: fixed parameter name (name->dname) Other improvements: - Fixed grammar, capitalization, and formatting throughout - Corrected function parameter types and names - Fixed missing error codes in ERRORS sections - Improved consistency in RETURN VALUES sections - Fixed typos and awkward phrasing - Sorted SEE ALSO sections alphabetically (31 files) All signature changes verified against source code in lustre/utils/liblustreapi*.c and lustre/include/lustre/lustreapi.h Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I643e4b2ad2d00b5ebf58d4136d1524bd92b7cb5d
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-12187 llite: enable FLR EC support unconditionally Remove module parameter guards and enable FLR EC support unconditionally. This patch is intended to be applied after the guarded development phase is complete. Changes: - Remove mdt_enable_flr_ec module parameter - Remove llite_enable_flr_ec module parameter - Add OBD_CONNECT2_FLR_EC unconditionally to client connect flags - Change ll_enable_erasure_coding default from 0 to 1 - Remove MODOPTS_MDT and MODOPTS_LLITE settings from test framework Test-Parameters: ignore Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I431d309f85c8964e13a11eaaa729d944e9228c29
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-subtest-change crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-subtest-change crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-dne-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
LU-19566 tests: test EC with lfsck EC parity mirror components do not contain regular file data, so it's essential they be recognized as parity components after an lfsck repair. This test verifies this functionality. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0384d7676043cc23f52da7df41385dc61809114a
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
(typo) field is actually named `ff_comp_flags`
(style) rather than duplicating this code each time, it should just incrementally decode the new parts:
```
} else /* if (size >= sizeof(struct filter_fid_217) */ {
struct filter_fid_217 *ff_old = (struct filter_fid_217 *)dst;
ost_layout_cpu_to_le(&ff_old->ff_layout, &src->ff_layout);
ff_old->ff_layout_version = cpu_to_le32(src->ff_layout_version);
ff_old->ff_range = cpu_to_le32(src->ff_range);
}
if (size >= offsetof(dst->ff_comp_flags) + sizeof(dst->ff_comp_flags)) {
dst->ff_comp_flags = cpu_to_le32(src->ff_comp_flags);
}
```
(style) same as above - process each chunk incrementally instead of duplicating code
(defect) this comment should not be removed
Oh, hmm. There was another patch looking to use one of the uid_h fields:
https://review.whamcloud.com/64468 ("LU-18847 mdt: version layout checking")
but it is using the `cr_fsuid_h` field in `struct mdt_rec_create`, so this should be fine.
(style) better to use `enum lov_comp_md_entry_flags` to make it more clear which "flags" these are, and they can be found more easily. The wirecheck.c code will ensure that the enum size remains consistent.
(minor) I think this is a misnomer to write "for filter_fid", since it is primarily used for components. Maybe `layout flags for components and filter_fid`?
(style) this shouldn't be in the comment here, as it will invariably become outdated in the future
(style) should there be a helper function for this, like:
```
/* return minimum size of in-use filter_fid to maximize interop with old tools */
size_t filter_fid_sizeof(struct filter_fid *ff)
{
BUILD_BUG_ON(sizeof(*ff) >
sizeof(struct filter_fid_217) + sizeof(ff->ff_comp_flags));
/* duplicate next chunk when new fields are added to filter_fid */
/* if (!ff->ff_new_field && ff->ff_previous_last_field)
return sizeof(struct filter_fid_next); */
if (!ff->ff_comp_flags /* && ff->ff_layout_version */)
return sizeof(struct filter_fid_217);
return sizeof(*ff);
}
```
(minor) `ff_size = filter_fid_sizeof(ff);`
(minor) `ff_size = filter_fid_sizeof(ff);`
(minor) It isn't clear if there is any value to printing this field, especially *always* printing it (maybe vs. only printing it if it is non-zero for some reason)
(style) 'version_code 2.17' may be misspelled - perhaps 'version 2.16.x should be used'?
(minor) update to 2.17.52.52 at least
LU-19566 lustre: add layout flags to lfsck EC parity mirror components do not contain regular file data, so it's essential they be recognized as parity components after an lfsck repair. They are distinguished by a component flag, so we must add component flag support to lfsck. This support can be landed without the rest of EC support, so this is based on master. There is a test for lfsck + EC in: https://review.whamcloud.com/c/62489 Details: Extended struct filter_fid from 52 to 56 bytes by adding __u32 ff_flags field to store component flags (like LCME_FL_INIT, LCME_FL_PARITY) in OST objects. Created versioned structure (filter_fid_217) for backward compatibility with old 52-byte filter_fid. Modified lu_orphan_rec_v3 to add lor_comp_flags field (replacing lor_padding_1) to store component flags from filter_fid. This allows LFSCK to preserve component flags during reconstruction. Extended wire protocol by adding o_comp_flags field to struct obdo (replacing o_padding_4) and added OBD_MD_FLCOMPFLAGS flag to indicate when component flags are valid in the obdo. Updated client to set o_comp_flags in the obdo. Modified OFD to read o_comp_flags from obdo and store it in ff_flags during writes, setattr, punch, and fallocate operations. Updated LFSCK to read ff_flags from filter_fid and restore component flags when reconstructing layouts from orphan OST objects. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I716dbf24db48cc50385a2fa9bcc446d291380814
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 | CentOS 8.3/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
LU-15069 llite: Clean up bit shift for assert There is a bizarre comment which says we're not converting pages to bytes yet to save cost in checking an assert, but we do that conversion immediately after the assert *and* it requires doing a similar shift in the assert. This is quite strange - clean it up. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id2bb4e6910276537d807828a9a33439dea054c3b
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-20112 tests: verify drop_caches evicts client cache pages Add sanity test_101k that verifies drop_caches actually evicts Lustre client page cache pages once any in-flight bulk RPC pin has been released. Read pages reap their bulk pin immediately on RPC completion. Write pages keep the pin until the OST commits, so the test forces a commit via dd conv=fsync. Both paths assert the post-drop read shows ost_read activity (cache miss). Note: global sync(2) does NOT drain Lustre's bulk pins -- only fsync(fd) does, because only fsync goes through ll_fsync -> OST_SYNC. cancel_lru_locks osc also drains the pins by canceling the DLM extent locks, which is what the read path in this test uses. Generated with Claude Code + Tools Test-Parameters: testlist=sanity env=ONLY=101k Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Iafa8f5986fed38d98964a3ebea929e0b412a8e49
LU-16488 utils: fix help message for 'lctl interface_list' The help message for 'lctl interface_list' command was incorrect. It was saying: "You must run 'interface_list <network>' command before 'network'" But the correct syntax is: "lctl --net tcp0 interface_list" This patch updates the help message in lctl.c and the man page to reflect the correct usage. It also updates the error message in portals.c to be more accurate. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7be1c0189d4177ce0c6ed67d77c98ab723684191
| unique failing test | history |
|---|---|
| recovery-small@ldiskfs+DNE:test_131 | seen in 64 other reviews |
Ah, interesting. We have a mistake here - twice. Or maybe four times, have to check llite. These parameters should all be defaulting to disabled.
Also we should have a trap set before the 2.18 release to check if we should remove these parameters, as discussed
LU-19989 llite: add FLR IWM connect flag and switches Add OBD_CONNECT2_FLR_IMMED_MIRROR connect flag support, LCME_FL_IMMEDIATE layout flag, and enable_immediate_mirror parameter to control immediate write mirror layouts. Add module parameters to gate presentation of connection flag. Define LCME_FL_IMMEDIATE (0x800) and add it to LCME_KNOWN_FLAGS, LCME_USER_COMP_FLAGS, LCME_CL_COMP_FLAGS, and LCME_TEMPLATE_FLAGS. LCME_FL_IMMEDIATE and LCME_FL_PARITY are mutually exclusive. Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I173c890d899a52cd57719cc89cc0629e54c86e51
| unique failing test | history |
|---|---|
| sanity1@zfs:test_27D | seen in 80 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_45 | seen in 14 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
[Sashiko] The label "lcme_flags:" is used on the line above to print the hex flags value. The new line reuses the same label for an "immediate mirror" string, so two consecutive log lines with that label would have different types of content. Would something like "lcme_immediate:" be clearer?
[Sashiko] test40: The third argument to llapi_layout_sanity() is the flr flag. With flr=false, the sanity callback checks flags against the non-FLR allowed set, which does not include LCME_FL_IMMEDIATE. So the call returns LSE_FLAGS rather than LSE_IMMEDIATE_MIRROR_COUNT. The assertion rc != 0 passes, but the path the test description describes -- "immediate needs >= 2 mirrors" -- is never reached. Would passing flr=true and asserting rc == LSE_IMMEDIATE_MIRROR_COUNT better match the stated intent?
[Sashiko] llapi_layout_comp_flags_set() enforces that LCME_FL_IMMEDIATE and LCME_FL_PARITY are mutually exclusive, but the sanity callback doesn't appear to check that combination independently. In the FLR case (lsa_flr=true), both flags are in LCME_USER_COMP_FLAGS and would pass the flags validation. Can a layout with both LCME_FL_IMMEDIATE and LCME_FL_PARITY on the same component pass llapi_layout_sanity() without error -- for example, one read back via llapi_layout_get_by_fd()?
LU-19991 llapi: immediate mirror layout support Add LCME_FL_IMMEDIATE kernel helpers, llapi validation, and layout passthrough support. Add lsme_is_immediate() and lsm_entry_is_immediate() helpers in lov_internal.h. Add debug print for LCME_FL_IMMEDIATE in pack_generic.c. Add LCME_FL_IMMEDIATE + LCME_FL_PARITY mutual exclusion validation, >= 2 mirror requirement, and per-mirror all-or-nothing enforcement in liblustreapi_layout.c sanity checks. Fix llapi_layout_set_by_fd to pass correct FLR flag to sanity check. Add llapi_layout_test cases: flag round-trip (test38), mutual exclusion (test39), mirror count validation (test40), kernel setstripe/getstripe round-trip (test41), directory default layout inheritance (test42), multi-component mirror flag consistency (test43), re-open by path (test44), flag clearing (test45), both-mirrors IMMEDIATE (test46), 3-mirror isolation (test47), partial IMMEDIATE rejection (test48). Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I42f17a6bba89c6c5c1f1a3e9d71ab3a0fb7c7f1e
(defect) This should use a proper LU number
```suggestion LU-930 doc: man page improvements ``` Could just use the default "man page improvement" ticket...
It looks like there are patches under LU-19744 that are conflicting with this one. It might be that this "LU-0000" patch was intended as a test, but should have been marked with "fortestonly" or "ignore" to avoid attention...
The other patch indeed has similar changes as this one (and the same issues around nodemap_add vs. nodemap_new). Since I already fixed that here, it may make sense to fold the changes here into the other patch. Either way, this patch looks good. Let's see what Patrick says.
Rather than adding sub-commands to lctl, it is better to add a separate manage for it. That allows proper formatting, subsections, examples, etc.
LU-930 doc: man page improvements Various man page improvements - adding missing commands and options, fixing typos, fixing format issues, etc. Includes one trivial fix to lctl.c. Focuses on man8. This code was generated by Augment. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I222ea824b25b0a3104dc4a8bc8f56c10449045e4
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
And look, we can finally get rid of this terrifying (and correct!) TODO ....... eek
LU-18553 llite: ensure layout refresh on fast read It is essential to refresh the layout before doing a fast read, otherwise we could read stale data if the layout has changed and, eg, the mirror our data is from is now stale. Today, we do this refresh incidentally in file_read_confine_iter->cl_io_init->vvp_io_init, but this is obviously fragile to future changes, since it's not directly associated with the fast read path. Add dedicated code in the fast read path to refresh the layout. Also opportunistically rename a few functions to make clear they are Lustre functions and not kernel functions. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ifbd827b79847309fe8d798963774ccd6650ad22f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
Since this is just a delta, why not use jiffies instead? u64 kstart = get_jiffies_64(); ... ll_stats_ops_tally(ll_i2sbi(file_inode(file)), LLPROC_LL_READ, jiffies_to_usecs(get_jiffies_64() - kstart));
LU-19344 llite: use ktime_get_coarse for read stats Replace ktime_get() with ktime_get_coarse() for read timing stats in do_file_read_iter. ktime_get() reads the hardware clocksource on every call, which is expensive on virtualized guests - pvclock_clocksource_read was the #1 CPU consumer at 9.16% of the tiny read profile on a KVM guest. Virtualization is not niche; all major cloud providers run KVM or similar hypervisors, so this cost is broadly relevant. ktime_get_coarse() reads a cached jiffies-granularity timestamp (~1-4ms resolution), avoiding the hardware read entirely. The coarse granularity is sufficient for the aggregate min/max/sum/count stats collected by ll_stats_ops_tally - these stats summarize thousands of ops, so per-op microsecond precision is not needed. Also move kstart assignment after the zero-count early return to avoid unnecessary work. perf profile with ktime_get_coarse (KVM, pvclock): pvclock_clocksource_read: 0.02% (from other callers) ktime_get_coarse_ts64: 0.74% Benchmark (8-byte sequential reads, 2M iterations): Before (ktime_get): ~3,600k reads/sec After (ktime_get_coarse): ~4,100k reads/sec (+14%) Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1c25e14b2fbb9a6f7f4f6f999aa9f2c6e2a0bb8c
LU-19744 doc: bulk man page review Bulk review done by Augment and Claude Code. Complete systematic review of Lustre man pages (sections 5, 7, 8) with fixes for documentation errors, missing options, formatting issues, and cross-references. Key fixes include: - Add missing options: --nomgs, --nidsfile, -q/--quiet, --catalog, -l, and others documented in usage but missing from man pages - Fix formatting errors: .TH syntax, incomplete sentences, SEE ALSO sections - Add missing cross-references to related man pages - Fix typos and grammar errors - Improve consistency across man pages - Add missing NAME section to lnetctl.8 - Fix alphabetical ordering in SEE ALSO sections (18 nodemap files) Files modified: 60+ man pages across sections 5, 7, and 8 Total changes: systematic improvements to documentation accuracy and completeness Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4896b64767e58c2b939b5a70db394a6f920949d5
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-19344 llite: inline fast path of ll_stats_ops_tally Inline the common case of ll_stats_ops_tally() for the default STATS_TRACK_ALL mode with per-CPU stats. This eliminates the out-of-line function call chain through ll_stats_ops_tally -> lprocfs_counter_add -> lprocfs_stats_lock/unlock on every read and write. The inlined version directly accesses the per-CPU counter struct under get_cpu()/put_cpu(), updating count, sum, sumsquare, min, and max in place. The slow path (__ll_stats_ops_tally) handles filtered tracking modes (PID/PPID/GID) and the NOPERCPU case. perf profile before inlining (KVM tiny writes): lprocfs_counter_add: 1.41% lprocfs_stats_lock: 0.64% ll_stats_ops_tally: 0.14% lprocfs_stats_unlock: 0.05% Total stats overhead: 2.24% After inlining: all four functions gone from profile. Benchmark (8-byte sequential I/O, 2M iterations): Writes: ~2,179k/sec -> ~2,307k/sec (+6%) Reads: ~4,100k/sec -> ~4,335k/sec (+6%) Combined with ktime_get_coarse (patches 1-2): Writes: ~1,970k/sec -> ~2,307k/sec (+17% total) Reads: ~3,600k/sec -> ~4,335k/sec (+20% total) Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie9e79d466fee8401dfc08485896124f918c0c9d4
| unique failing test | history |
|---|---|
| sanity-quota@zfs+DNE:test_12b | seen in 19 other reviews |
| sanity-quota@zfs+DNE:test_17 | seen in 23 other reviews |
| sanity-quota@zfs+DNE:test_25 | seen in 17 other reviews |
| sanity-quota@zfs+DNE:test_33 | seen in 23 other reviews |
| sanity-quota@zfs+DNE:test_34 | seen in 23 other reviews |
| sanity-quota@zfs+DNE:test_37 | seen in 23 other reviews |
| sanity-quota@zfs+DNE:test_38 | seen in 23 other reviews |
| sanity-sec@zfs:test_25b | seen in 31 other reviews |
This patch could run with `Test-Parameters: trivial` if it is refreshed.
LU-17000 lnet: refactor lnet_net_show_dump Refactor lnet_net_show_dump() to improve readability and reduce function size from ~360 lines to ~160 lines. Changes: - Add lnet_ni_dump_ctx struct to hold shared dump state - Extract lnet_ni_dump_one_msg_stats() helper for message stats (send/recv/drop) - eliminates code duplication - Extract lnet_ni_dump_health_stats() helper for health statistics - Extract lnet_ni_dump_tunables() helper for net tunables - Extract lnet_ni_format_cpts() helper for CPT list formatting with proper buffer overflow checking - Extract lnet_ni_dump_extended() to handle all verbosity-dependent NI dumping with linear control flow instead of goto-based skipping The refactoring eliminates goto labels (skip_msg_stats, skip_udsp) by restructuring the verbosity-based logic into clear conditional blocks. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I210f78fdf201a416f16733bc5ba4afdf45e92065
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 15 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-8 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 5 tests failed: sanity-pfl, lnet-selftest, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: replay-ost-single. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-17240 tests: enable parallel mounting by default Enable parallel mounting of targets within types by default. This allows multiple OSTs (or MDTs in multi-MDT configurations) to be mounted simultaneously, but maintains sequential ordering between different target types (MDT before OST). Configuration: - PARALLEL_MOUNT now defaults to 1 (enabled) - PARALLEL_MOUNT_TYPES explicitly set to 0 (no inter-type overlap) - PARALLEL_MOUNT_TARGETS="mdt ost" (MGS excluded) Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results: - Baseline (serial mounting): 29.7s - Parallel mounting within types: 27.3s - Improvement: 8% faster (2.4s savings) Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I71efad13359e3490129ce90dece5fee1d2513c75
| unique failing test | history |
|---|---|
| conf-sanity3@ldiskfs+DNE:test_87 | seen in 7 other reviews |
| conf-sanity3@ldiskfs+DNE:test_88 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_89 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90a | seen in 12 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90b | seen in 15 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90c | seen in 21 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90d | seen in 24 other reviews |
| conf-sanity3@ldiskfs+DNE:test_91 | seen in 29 other reviews |
| conf-sanity3@ldiskfs+DNE:test_98 | seen in 34 other reviews |
| conf-sanity3@ldiskfs+DNE:test_99 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_120 | seen in 13 other reviews |
| conf-sanity4@ldiskfs+DNE:test_161 | seen in 7 other reviews |
| sanity-quota@ldiskfs+DNE:test_91 | seen in 1 other review |
| sanity-quota@ldiskfs+DNE:test_92 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_93 | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_94 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_95a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_95b | seen in 4 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 13 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
LU-17240 tests: enable parallel formatting by default Enable parallel formatting of targets by default. This includes both intra-type parallelism (multiple MDTs or OSTs formatted simultaneously) and inter-type parallelism (MDT and OST formatting overlapping). Configuration: - PARALLEL_FORMAT now defaults to 1 (enabled) - PARALLEL_FORMAT_TYPES inherits from PARALLEL_FORMAT (=1) - PARALLEL_FORMAT_TARGETS="mdt ost" (MGS excluded) - Other operations (mount, unmount) remain disabled Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results: - Baseline (serial formatting): 31.1s - Parallel formatting: 17.3-25.7s - Improvement: 17-44% faster Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7cac5b4a677e3fee97edf57bb9a7cf3469bab92b
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 15 tests. 5 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck, runtests. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 17 tests. 5 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck, runtests. This build will be s | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: sanity-scrub, sanity-pcc. | session |
| review-dne-part-8 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 15 tests. 6 tests failed: sanity-pfl, lnet-selftest, pjdfstest, sanity-sec, sanity-lfsck, runtests. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: replay-ost-single. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-pcc. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 8 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-17240 tests: enable parallel unmounting across types Enable inter-type parallelism for unmount operations, allowing clients, MDTs, and OSTs to unmount simultaneously rather than waiting for each type to complete sequentially. WARNING: This feature is known to have stability issues and can cause hangs during unmount. It is enabled here for testing purposes only and should not be used in production environments. Configuration: - PARALLEL_UMOUNT_TYPES now inherits from PARALLEL_UMOUNT (=1) - Allows client, MDT, and OST unmounts to overlap Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results (when it works): - Baseline (sequential types): 24.6s - With inter-type overlap: 11.9s - Improvement: 51% faster (12.7s savings) Known issues: - Intermittent hangs during unmount operations - Race conditions in cleanup ordering - Resource deadlocks between target types Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I413c5be0437049c88a6c1066681ed258fa41fa06
| unique failing test | history |
|---|---|
| conf-sanity2@ldiskfs+DNE:test_73c | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_73e | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_74 | seen in 12 other reviews |
| conf-sanity2@ldiskfs+DNE:test_75 | seen in 9 other reviews |
| conf-sanity3@ldiskfs+DNE:test_87 | seen in 7 other reviews |
| conf-sanity3@ldiskfs+DNE:test_88 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_89 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90a | seen in 12 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90b | seen in 15 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90c | seen in 21 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90d | seen in 24 other reviews |
| conf-sanity3@ldiskfs+DNE:test_91 | seen in 29 other reviews |
| conf-sanity3@ldiskfs+DNE:test_98 | seen in 34 other reviews |
| conf-sanity3@ldiskfs+DNE:test_99 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_103 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| conf-sanity3@ldiskfs+DNE:test_120 | seen in 13 other reviews |
| conf-sanity4@ldiskfs+DNE:test_154 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| conf-sanity4@ldiskfs+DNE:test_161 | seen in 5 other reviews |
| sanity2@ldiskfs+DNE:test_130i | seen in 3 other reviews |
| sanity-lfsck@zfs:test_18a | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18c | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18d | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18e | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18f | seen in 3 other reviews |
| sanity-lfsck@zfs:test_18g | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18h | seen in 3 other reviews |
| sanity-lfsck@zfs:test_20a | seen in 2 other reviews |
| sanity-lfsck@zfs:test_20b | seen in 2 other reviews |
| sanity-lfsck@zfs:test_45 | seen in 1 other review |
| sanity-quota@ldiskfs+DNE:test_59 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_60 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_62 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_64 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_66 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_67 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_68 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_69 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_70a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_70b | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_71a | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_71b | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_72 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_73a | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_73b | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_74 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_75 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_76 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_77 | seen in 7 other reviews |
| sanity-quota@ldiskfs+DNE:test_79 | seen in 6 other reviews |
| sanity-quota@ldiskfs+DNE:test_80 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_81 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_82 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_83 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_84 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_85 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_86 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_87 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_89 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_90a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_90b | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_1b | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_1c | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_2 | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4a | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4b | seen in 10 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4c | seen in 12 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4d | seen in 16 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4e | seen in 13 other reviews |
| sanity-scrub@ldiskfs+DNE:test_5 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_6 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_7 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_8 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_9 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_10a | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_11 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_14 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_15 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_17a | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_17b | seen in 5 other reviews |
| sanity-sec@ldiskfs+DNE:test_75b | seen in 3 other reviews |
| sanityn@ldiskfs+DNE:test_104 | seen in 3 other reviews |
| sanityn@ldiskfs+DNE:test_106a | seen in 2 other reviews |
| sanityn@ldiskfs+DNE:test_115 | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. %% THIS TEST SESSION CRASHED % | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 15 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-8 crashed | RHEL 8.10 / x86_64 | ran 2 tests. 1 tests failed: lustre-initialization. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 5 tests failed: sanity-pfl, lnet-selftest, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: replay-ost-single. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-17240 tests: enable parallel unmounting by default Enable parallel unmounting of targets within types by default. This allows multiple clients, OSTs, or MDTs to be unmounted simultaneously within their respective type groups, but maintains sequential ordering between different target types (clients, then MDTs, then OSTs). Configuration: - PARALLEL_UMOUNT now defaults to 1 (enabled) - PARALLEL_UMOUNT_TYPES remains 0 (no inter-type overlap) - PARALLEL_UMOUNT_TARGETS="client mdt ost" Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results: - Baseline (serial unmounting): 30.9s - Parallel unmounting within types: 26.5s - Improvement: 14% faster (4.4s savings) Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Icbd5fa4fc79259225b03a9feab0bc50b4b48b09d
| unique failing test | history |
|---|---|
| conf-sanity2@ldiskfs+DNE:test_73c | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_73e | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_74 | seen in 12 other reviews |
| conf-sanity2@ldiskfs+DNE:test_75 | seen in 9 other reviews |
| conf-sanity4@ldiskfs+DNE:test_154 | seen in 1 other review |
| conf-sanity4@ldiskfs+DNE:test_161 | seen in 5 other reviews |
| sanity2@ldiskfs+DNE:test_130i | seen in 3 other reviews |
| sanity3@zfs:test_271f | seen in 10 other reviews |
| sanity-lfsck@zfs:test_18a | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18c | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18d | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18e | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18f | seen in 4 other reviews |
| sanity-lfsck@zfs:test_18g | seen in 2 other reviews |
| sanity-lfsck@zfs:test_18h | seen in 3 other reviews |
| sanity-lfsck@zfs:test_20a | seen in 2 other reviews |
| sanity-lfsck@zfs:test_20b | seen in 2 other reviews |
| sanity-lfsck@zfs:test_45 | seen in 1 other review |
| sanity-quota@ldiskfs+DNE:test_59 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_60 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_62 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_64 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_66 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_67 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_68 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_69 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_70a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_70b | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_71a | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_71b | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_72 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_73a | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_73b | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_74 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_75 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_76 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_77 | seen in 7 other reviews |
| sanity-quota@ldiskfs+DNE:test_79 | seen in 6 other reviews |
| sanity-quota@ldiskfs+DNE:test_80 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_81 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_82 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_83 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_84 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_85 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_86 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_87 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_89 | seen in 3 other reviews |
| sanity-quota@ldiskfs+DNE:test_90a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_90b | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_1b | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_1c | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_2 | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4a | seen in 6 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4b | seen in 10 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4c | seen in 12 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4d | seen in 16 other reviews |
| sanity-scrub@ldiskfs+DNE:test_4e | seen in 13 other reviews |
| sanity-scrub@ldiskfs+DNE:test_5 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_6 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_7 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_8 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_9 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_10a | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_11 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_14 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_15 | seen in 2 other reviews |
| sanity-scrub@ldiskfs+DNE:test_17a | seen in 5 other reviews |
| sanity-scrub@ldiskfs+DNE:test_17b | seen in 5 other reviews |
| sanity-sec@ldiskfs+DNE:test_75b | seen in 3 other reviews |
| sanity-sec@ldiskfs+DNE:test_82 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_104 | seen in 3 other reviews |
| sanityn@ldiskfs+DNE:test_106a | seen in 2 other reviews |
| sanityn@ldiskfs+DNE:test_115 | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 15 tests. 4 tests failed: sanity-pfl, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 2 tests failed: sanity-quota, sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-part-8 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 5 tests failed: sanity-pfl, lnet-selftest, pjdfstest, sanity-sec, sanity-lfsck. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 2 tests failed: sanity-quota, replay-ost-single. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-17240 tests: enable parallel mounting across types Enable inter-type parallelism for mount operations, allowing MDT and OST mounting to overlap rather than waiting for MDT to complete before starting OST mounts. Configuration: - PARALLEL_MOUNT_TYPES now inherits from PARALLEL_MOUNT (=1) - Allows MDT and OST mounts to run simultaneously Test environment: - 1 MGS (combined with MDS) - 1 MDT (245MB) - 2 OSTs (391MB each) Performance results: - Baseline (sequential types): 24.9s - With inter-type overlap: 26.8s - Result: 8% slower (1.9s penalty) Note: Inter-type mount parallelism shows a small performance degradation in this configuration. The overhead of running MDT and OST mounts simultaneously appears to outweigh any parallelism benefits, likely due to resource contention or mount dependencies. This feature is enabled for completeness and may show benefits in larger configurations. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9981fc1a56d5196e311a95f9808cb14a163f09c3
(style) line length of 84 exceeds 80 columns
(style) line could be split after '||'
LU-17240 tests: fix dm-flakey device handling The recent parallel unmount commit attempted to fix device name functions to return the actual mounted device for dm-flakey support, but this created circular dependencies during initialization that caused hangs. The proper solution is to add a new facet_real_dev() function that returns the currently mounted device (which could be a dm-flakey device) when the facet is mounted, or falls back to the logical device name when unmounted. This is needed because direct device access tools like debugfs, tune2fs, dumpe2fs, and e2fsck need to access the actual mounted device (dm-flakey) to see current filesystem state, not the underlying physical device which may have stale data buffered in the dm layer. Updated all test code that uses these tools to call facet_real_dev() instead of ostdevname() or mdsdevname() to get the correct device for direct access. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8d9785351904fc4f21721f872a8cdce7958159a5
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-quota@zfs:test_48 | seen in 24 other reviews |
LU-15367 tests: Multiop allow mmap control Add the ability for multiop to specify where to access a file when it's mmaped by repurposing the 'w' and 'r' options when a file is mmapped. This makes multiop able to easily simulate application behavior using processed iotrace logs as input. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6ff455f15a59132018525410c7fcce840c5b6209
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: recovery-small. | session |
LU-15069 llite: remove ras_align ras_align is quite odd - it aligns to either RPC size, which has some justification, or to window size, which is totally strange. Window size has nothing to do with alignment and shouldn't be used for this at all. And ras_align rounds *down*, which results in extra misses because it's shrinking the readahead window selected by the rest of the readahead logic. Finally, although aligning readahead to RPC boundaries sounds nice, it makes readahead itself far more complicated by messing up the math for offsets and window sizes, for limited benefit: It is not very important for RPCs to be *aligned* so long as they are *large*, which is handled by the rest of the readahed logic. This significantly cleans up some of the readahead behavior and fixes the misses introduced by rounding down. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I267076a79120e145f49a4b2ffdeff97b4f2b158b
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-zfs | RHEL 8.9 / x86_64 | ran 8 tests. 1 tests failed: replay-single. | session |
LU-17473 llite: wait for partially successful aio For various reasons (notably conflicting buffered IO), we may need to fall back from DIO to buffered IO. This also affects AIO, and if it happens, we will sometimes submit only part of an AIO with the AIO path, completing the rest with the buffered path. Userspace doesn't expect this, expecting us to either do all or none of the IO with AIO, so it doesn't wait for completion in this case. To meet this expectation, we must recognize this case and wait for AIO to complete before returning to userspace. Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Iac7abac3bd01f027c353120483932a62c6475277
LU-15367 scripts: Add iotrace to multiop script With the standardization of iotrace and multiop having the ability to take input from a file, we can start directly translating iotrace recordings to multiop input. This allows us to use multiop to simulate the I/O call sequences of an iotrace recording. There are a number of limitations currently, which we may choose to fix later (if this turns out very useful): 1. No support for multiple open files (multiop limitation) 2. Can only play-back one thread at a time We use the ability to go from command to iotrace recordin back to multiop command to test this functionality. Test-Parameters: trivial Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I2ec4f358c97ceb15b717342af5cc9854b9c60677
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 48 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 7 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 6 tests. 2 tests failed: lustre-initialization, replay-dual. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 13 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 1 tests. 1 tests failed: node-provisioning. %% NODE-PROVISIONING FAILED MULTIPLE TIMES FOR review-dne-zfs | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: recovery-small. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu failed 2× | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 3 tests failed: sanity-sec, sanity-lnet, sanity. | session |
LU-13814 osc: Move osc_page members to osc_async_page We're going to start using osc_async_page separately from osc_page, since it's used for both DIO and buffered, but osc_page is not. This moves all of the members which are needed for DIO. This commit deliberately ignores some packing related issues which will be handled *much* later, since the names and members of these structs will keep changing. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I994af9563142201a2c7193735ac02568735bd8d5
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 2 tests failed: sanity-lnet, sanity-sec. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-dom. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: recovery-small. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 2 tests failed: node-reset, sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 2 tests failed: sanity-lnet, sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
Note to self: I am not 100% sure this is finding the right object, but it should blow up if not. Might need a cl_object_top() here if it does.
LU-13814 osc: remove use of cp_obj Since we're going to lose the cl_page, we need to remove usage of its members from the code which handles DIO pages. This removes cp_obj usage from the one place where it's used by the DIO code, by adding it to the BRW async args. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic53b872d930305c345a03c75dc21a613874bf3c8
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 12 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: recovery-small. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
LU-13814 clio: use osc_dio_completion everywhere The conversion to osc_dio_completion was incomplete because some other code wasn't ready. Finish that conversion. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id6500bfb55dc27e783a91f58498f9a13906056b8
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 8 other reviews |
| sanity-sec@zfs:test_59a | seen in 9 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 2 tests failed: sanity-lnet, sanity-sec. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: insanity. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
LU-13814 osc: Remove usage of cdp_cl_pages As part of eliminating cl_page in the DIO path, we need to clear out all uses of it. This is one more minor one - there's no need to clear this list before returning from this function, because if it fails we give up entirely. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9dc1053c542ce7a903a93f7b9a1fb0bfc6ac1641
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 7 other reviews |
| sanity-sec@zfs:test_59a | seen in 9 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 2 tests failed: sanity-lnet, sanity-sec. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: lustre-initialization, replay-single. | session |
| review-dne-part-8 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-zfs-part-6 | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
So I think I figured out what's weird here, but check me if I'm wrong. I don't think it makes sense to both have padding bits AND the ((packed)) attribute? Basically the effect of the padding bits is to prevent ((packed)) from taking effect. Neighboring bitfields are combined without the ((packed)) attribute - at least in my observations - but the alignment requirements (the requirements for good performance, that is) are respected. When you add ((packed)), the alignment requirements are ignored. Adding the packing bits basically cancels that out again. Without packed and without the packing bits, we seem to get the desired packing while respecting recommended alignment. Thoughts?
I think "packed" is needed to combine fields across data structures. However, it is less useful for in-memory data structures so if it is no longer needed it could be removed. Did you check the structs without "packed" with pahole?
Does this struct need to be long aligned? Wondering if 4 byte alignment is an issue for some 64 bit arch
The alignment here more generally is a bit messy - because we chose packed to prefer memory efficiency, we're not padding. Much later, I have a portion of this series which removes every member of this struct and handles packing other stuff better. I'll see about integrating some of it earlier in the series.
These bitfields are handled by the compiler, this isn't the same as the "bitfield" macros used by the kernel that need to be "long" variables.
LU-17063 osc: remove duplicate info The from/to in osc_page replicate the info provided by oap_page_off and oap_count in osc_async_page, so just use those. Getting the full benefit of this requires removing the padding, but with that done, this gets us a full 8 bytes of size reduction in osc_page (and therefore cl_page) size. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If9f2ea5abfe1da6e586072f22c5e0758988b7760
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 47 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-hsm@zfs:test_254b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-13814 osc: rename osc_async_page osc_async_page isn't for async IO - it's for all data IO. Rename it osc_transfer_page so the name fits the usage. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I930aae585763f95d9085bea179765a0431bccf52
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-sec@zfs:test_59a | seen in 9 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-13814 osc: rename osc_prep_async_page This is another piece of renaming osc_async_page to osc_transfer_page. This is kept separate to make the previous patch as focused as possible. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ifd57eec46aeeb059ac836e09aa47322a69cc1493
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 7 other reviews |
| sanity-sec@zfs:test_59a | seen in 9 other reviews |
LU-13814 osc: rename osc_async_flags Async flags are used for all osc pages, rename them accordingly. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ibf808bcad5f03d950b4d2d05fb11f047fd28f311
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-flr@zfs:test_70a | seen in 75 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 5 other reviews |
LU-13814 clio: add cp_inode to page allocation cp_inode can be set correctly during page allocation, rather than after. This is a prelude to moving cp_inode to the osc_transfer_page, but that's better done in a separate patch. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I509f6cfbae8e5a6ec6b07c8253d68f6dd2794e59
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18f | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-lfsck@ldiskfs+DNE:test_18g | seen in 4 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18h | seen in 4 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 6 other reviews |
LU-13814 clio: move cp_inode to transfer page As part of moving DIO to use only the transfer page and not cl_page, we need to eliminate uses of cl_page where we have a transfer page available. That requires moving cp_inode to the transfer page. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If0312c7fa22501b789437479fadb023f09f341b8
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity3@ldiskfs+DNE:test_230n | seen in 7 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
| sanity-flr@zfs:test_70a | seen in 74 other reviews |
| sanity-sec@ldiskfs+DNE:test_59a | seen in 7 other reviews |
LU-13814 osc: replace cra_page cra_page requires there to be a cl_page associated with an OSC transfer page. Since we're breaking that association, we replace it with the page index, which can do what's required. Test-Parameters: forjanitoronly Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1163dae32bce6ae9fcc458251c047f05ddfa6ec1
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@zfs:test_56x | seen in 36 other reviews |
| sanity1@zfs:test_56xa | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 47 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 90 other reviews |
LU-13814 clio: move cp_page_index to transfer page cp_page_index is needed for both DIO and BIO, so it has to move to the OSC transfer page. Test-Parameters: forjanitoronly Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2eceda45e3ac0b92973362b5427948fd1163adc9
LU-13814 osc: add DIO/BIO related asserts These will be adjusted when cp_type is moved to the otp page, but for now, these asserts help ensure we don't have any DIO pages in unexpected locations. Test-Parameters: forjanitoronly Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9342a502e0195d86af5f6826e5677b96f6b57f20
LU-0000 tgt: async write commit A first and insufficient try at server side async. For hybrid, we should in fact do everything async except lock acquisition. That means splitting the handler in to a sync and async portions. Test-Parameters: forbuildonly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: Iba526e4d75b41992ea3ed77bb3c2a76fa9a1c41b
It would be nice to have a sanity test (or perhaps a small sanity-gds.sh) that exercises this using real hardware, for those that have it available. That's outside the scope of this patch, but this test case would be good to have in such a test suite.
LU-13805 llite: fail unaligned DIO for RDMA pages Unaligned DIO needs to directly access the page contents in order to copy to the buffer. This means it can't work with RDMA only (non-CPU accessible) pages. Implement that limitation. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I52bd1d4cc143e1018ddf6942403142f26be4430f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 13 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 13 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: recovery-small, lustre-rsync-test. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: recovery-small. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
LU-19223 shrinkers: Add nr_scanned to all shrinkers If the Lustre shrinkers can't free any pages but don't set nr_scanned, they may be called forever by the kernel - see do_shrink_slab() in the kernel. Add nr_scanned support to the remaining Lustre shrinkers: - LDLM pools server and client shrinkers - Lu site shrinker - Page pools shrinkers Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ica35a0dabe1dce78fd3cd29174ef142a965be824
| unique failing test | history |
|---|---|
| conf-sanity1@ldiskfs+DNE:test_30a | seen in 1 other review |
| recovery-small@ldiskfs+DNE:test_67 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@zfs:test_21 | seen in 2 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-1 crashed | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 crashed | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 crashed | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 crashed | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-5 crashed | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-zfs crashed | RHEL 8.10 / x86_64 | ran 10 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
LU-0000 osc: ghost index entries for deleted pages Replace OSC radix tree page entries with a ghost value that encodes the page index when pages are deleted. On insert, detect and replace such ghosts with the real page. This allows us to discard these 'ghost' pages after they've been removed. Add COIO_PCACHE_TRUNCATE and handle it in vvp to truncate pagecache by [start,end] page indices for a cl_object. Aggregate contiguous ghost indices in OSC discard paths and invoke cl_object_inode_ops() to drop pagecache efficiently. Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9f1d9a383e267fd6f1d8419d180f99bdb71d6667
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 crashed | RHEL 9.4 / x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-1 crashed | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 2 tests failed: sanity-flr, sanity-dom. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 9.4 / x86_64 | ran 13 tests. 2 tests failed: sanity-flr, sanity-dom. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-6 | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 2 tests failed: sanity-flr, sanity-dom. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 9.4 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, RHEL 9.4 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu crashed | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
if want_composite is true, then we need to set mirror_count to 1, as there is an assertion to make sure mirror_count == 0 only for plain layout file in lod_fill_mirrors() as LU-18962 shows.
Instead of making this a
LU-17159 lod: mark file layouts with append striping Add LCM_FL_APPEND flag to file layouts when a file is opened with O_APPEND flag. This makes it easier to diagnose layout behavior when the MDS decides to use append-specific layout, which can override the default file layout. Add dah_flags to dt_allocation_hint structure to pass MDS_OPEN_APPEND flag from mdd_object_make_hint() to lod_ah_init(). The flag is set in mdd_object_make_hint() when MDS_OPEN_APPEND is present in the open flags, and is then used in lod_ah_init() to set the LCM_FL_APPEND flag in the layout. Also added BUILD_BUG_ON tests for LCM_FL flags in wiretest.c to ensure the flag values remain consistent. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I927faae3a385a9d0acf40a37ab1be0c0d4cbb82c
LU-0000 lod: initial implementation This is an initial and partially incorrect implementation, which I'll be tearing up a bit until we can actually get layout creation to work. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I229b5751b60f8078e22dbb4ed3e2e205d3e4c9a1
| unique failing test | history |
|---|---|
| conf-sanity3@ldiskfs+DNE:test_135 | seen in 20 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 2 other reviews |
| sanity-flr@zfs:test_200b | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs crashed | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. %% THIS TEST SESSION CRASHED %% | session |
LU-17433 osc: simplify osc_lock_set The presence of osc_lock_set_writer and osc_lock_set_reader and their complexity appears to be a holdover from the old CLIO, before the 2.7 era CLIO simplification. Most of the checks in the functions are unnecessary - we can't get here unless it's the same object and the range of the lock matches that of the IO. These may've been needed with the older more complicated CLIO locking, but they're not needed now. Clean all that up to make the code a bit more readable. Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I728039e18834eb08d9eb1f3492f8001c2a12f52b
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 9.3 / x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
LU-0000 obdclass: Refactor cl_object_attr_update The object named "top" is not actually a top object - it's any cl_object and we navigate the list accordingly. Clarify. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6043f0e6c1705cfeeece62cd71f2208b2197f5cc
| unique failing test | history |
|---|---|
| replay-single@ldiskfs+DNE:test_65a | seen in 66 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-6 crashed | RHEL 9.4 / x86_64 | ran 6 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
LU-13814 llite: remove unnecessary smp_mb() This smp_mb() was added as part of: https://review.whamcloud.com/c/39542 To help protect the csi_sync_nr atomic. But it was not justified there, and is not in fact needed. IO submission has *several* memory barriers before another thread can see the IO. (Note there is a discussion on that Gerrit suggesting it was unneeded.) For example: osc_queue_sync_pages uses a spinlock to put the extent on the list(s) (after this point, it's available for ptlrpc to make an RPC). There are several other instances of spinlocks, AND the use of csi_sync_nr is under memory barriers (in atomic_dec_and_lock, or under a spinlock). Let's remove this. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8b81f8e02025cae801a980d2856993c6d4023716
| unique failing test | history |
|---|---|
| sanity1@zfs:test_24oa | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.4 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.4 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.4 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
LU-18843 mdt: parallel rename in a single directory Testing... Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I920f9c0624ef25469ece046244d97623018acd63
LU-17831 osc: batch discard for write locks Batch page discard when cancelling a write lock. This roughly halves the time to cancel pages under a write lock. A future patch will do this for read locks as well. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I214f6babb69fc2117379490efe3d2d62b8122d90
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-6 crashed | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %% | session |
LU-14838 ldlm: Disable lockless on contention The contention detection code is mostly broken, and if it ever returns -EUSERS to the client, the client will crash or corrupt user data. The code is being retained because a rewrite is in flight and it would be much harder to do that if the code were fully removed. But let's disable it at least. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I3128288df8ddd39d2875c817830f7a1884c0e763
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-1 | CentOS 8.3/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | CentOS 8.3/aarch64, CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-14882 tests: Fix S_NOSEC tests
The S_NOSEC tests were using incorrect fail_loc values, and
also did not work correctly with correct fail_loc values.
Correct the fail_locs and fix the tests.
Properly fixing the tests required an odd bit of behavior:
Userspace cannot normally tell if we took a lock in Lustre
or not. This was the problem with the earlier tests:
A successful tests was identical to normal operation, so
it was missed that the tests did not work.
The solution is to return an error when we detect the
*correct* behavior (using a fail loc). This allows the
test to clearly tell the difference between a successful
test and normal operation.
Fixes: 8bc4b26453 ("LU-8656 vvp: Add S_NOSEC support")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3650bb304b7548ba72d2c1812b30c0217883a441
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-part-4 | CentOS 8.3/x86_64 | ran 11 tests. 2 tests failed: sanity-flr, sanity-dom. | session |
| review-dne-selinux-ssk-part-1 | CentOS 8.3/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | CentOS 8.3/x86_64 | ran 11 tests. 1 tests failed: sanity-dom. | session |
| review-ldiskfs | CentOS 8.3/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | CentOS 8.3/aarch64, CentOS 8.3/x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | CentOS 8.3/x86_64, Ubuntu 20.04/x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-zfs crashed | CentOS 8.3/x86_64 | ran 10 tests. 1 tests failed: sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
LU-14887 llite: Add DIO splitting tunables The new faster DIO path is great, but benefit is only seen if I/O is split to multiple RPCs. Currently, this only happens if I/O size is > RPC size or if I/O crosses a stripe boundary. This requirement for splitting means that there is an inherent conflict between the desire to do large RPCs and doing single stream I/O at high speed. This patch adds a pair of tunables, turning on some I/O splitting by default while allowing users to control the degree. DIO parallelism, at the llite layer, specifying how many chunks we should try to split a DIO In to Minimum preferred I/O size, at the OSC layer, specifying the minimum size to which we should split I/O. Parallelism is a global control of how much splitting is desired for best performance, the overall preference between maximum RPC size and maximum single stream performance. The OSC level control is because some OSTs have dramatically different performance with synchronous I/O, so for a spinning OST, it may be desirable to enforce a higher minimum I/O size (and so less parallelism), and the reverse for a flash OST. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5ce010d97b0bf6d91d05d36a2c74268432b5f1f2
| unique failing test | history |
|---|---|
| sanity-pfl@zfs:test_15 | seen in 3 other reviews |
| sanity-pfl@zfs:test_16c | seen in 2 other reviews |
| sanity-pfl@zfs:test_17 | seen in 2 other reviews |
| sanity-pfl@zfs:test_18 | seen in 3 other reviews |
| sanity-pfl@zfs:test_19c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | CentOS 8.3/x86_64 | ran 7 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-2 | CentOS 8.3/x86_64 | ran 7 tests. 2 tests failed: sanity-lfsck, replay-dual. | session |
| review-dne-zfs-part-4 | CentOS 8.3/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-zfs crashed | CentOS 8.3/x86_64 | ran 10 tests. 1 tests failed: sanity-quota. %% THIS TEST SESSION CRASHED %% | session |
LU-15069 llite: Move most readahead code to ra.c The readahead algorithm/prediction code is mixed weirdly throughout rw.c, which is mostly code which actually moves data. Because the prediction/window management code is not clearly split from the page reading code, there's still some in rw.c - but this still makes things clearer. There are also a few trivial function renames where names were wrong (eg, RAS instead of RIA), and one extra debug print. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Icc17c057f23233f9f3be8bcf73b9730dfe6b4856
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity-scrub. | session |
| review-dne-zfs-part-2 | CentOS 8.3/x86_64 | ran 7 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-6 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
LU-15483 tests: Reduce I/O sizes A lot of the LU-13799 tests use significantly more I/O than is required for the actual test. In particular, almost all of them use 64 MiB of data for every test, just because the tests started out by copy-pasting. Reduce the I/O size to closer to the minimum required for each test, which will save some time. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ife9a950c28ee09e2be1909cdd07c6f532a5f92bb
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_101j | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101j | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-4 | CentOS 8.3/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-selinux-ssk-part-1 | CentOS 8.3/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | CentOS 8.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | CentOS 8.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | CentOS 8.3/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15663 llite: Simplify readahead stats Readahead stats should be a user readable collection of stats, not a debug dump. Also, even as debug, many of the stats aren't useful. Let's clean it up. RA_STAT_FAILED_FAST_READ is confusing, and it is recorded for every miss. Rename it, and move it to be recorded only for *hits* where we decide not to do fast read for other reasons. 'zero page window' is an almost useless internal detail that makes no sense to users, and has little use even as debug. Let's just remove it. zero file size isn't an interesting readahead stat, it's just a fact about a file. Remove it. 'failed reach end' is also meaningless for users and not useful debug. Remove it. 'readahead to eof' is simply not interesting - readahead reaching the end of the file is a normal part of operation and not a useful stat. RA_STAT_FAILED_MATCH is unused. NB: This is not marked trivial because the stats are used in various tests and we need to verify nothing is broken by these changes. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I24dd543f9703fe5883d774f5e9b3152579494c30
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | CentOS 8.5/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-3 | CentOS 8.5/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-zfs-part-4 | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-ldiskfs-arm crashed | RHEL 8.6/aarch64, RHEL 8.6/x86_64 | ran 8 tests. 1 tests failed: lnet-selftest. %% THIS TEST SESSION CRASHED %% | session |
| review-zfs | CentOS 8.5/x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
LU-15822 ldlm: Add debug in lock_matches 'lock_matches' is a core LDLM function and entirely opaque from a debugging perspective, giving no info on why a lock did not match existing locks. Let's fix this. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: Icd95a6ba0cd99f689d83758d7576ff45e681d49d
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-1 | CentOS 8.5/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-ldiskfs | RHEL 8.6/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | CentOS 8.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.6/aarch64, RHEL 8.6/x86_64 | ran 6 tests. 2 tests failed: sanity-lnet, sanity. | session |
| review-ldiskfs-ubuntu | CentOS 8.5/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs | CentOS 8.5/x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-16564 ldlm: Remove cancel on block The LDLM_FL_CANCEL_ON_BLOCK flag was used by liblustre, but isn't used now. The comment on it explains why - it's for clients that can't reply reliably to BL callbacks, which is a disaster waiting to happen (or, not waiting, as the case may be). This should be removed - itss continued presence is confusing (at least to me!). Note: It turns out this is used by lease locks. I think that use may still be irrelevant - let's test and find out. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I608ef0e48999ac3c43d6a395c5351cf3151d897b
| unique failing test | history |
|---|---|
| recovery-small@ldiskfs+DNE:test_110m | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| replay-single@ldiskfs+DNE:test_70c | seen in 4 other reviews |
| sanity2@ldiskfs+DNE:test_154g | seen in 2 other reviews |
| sanity-pcc@ldiskfs+DNE:test_1d | seen in 3 other reviews |
| sanity-pcc@zfs:test_1f | seen in 2 other reviews |
| sanity-pcc@zfs:test_1g | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-1 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.7/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.7/x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-16656 llite: Improve 'out:' in ll_file_io_generic The location of the 'out:' label in ll_file_io_generic is a bit scary, because it skips part of the setup for io restart, where partial io is taken in to account. This is safe today because 'out' is only used before calls to cl_io_loop, so IO hasn't started yet, but if 'out' is ever used later in the function, it will be incorrect. Let's move it now rather than leave a trap for the unwary. Note that until cl_io_loop is called "io->ci_nob" is 0, so this shouldn't change current behavior. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I89e29708a6d17c5eecdf4f86261dfa013c7a5ec3
LU-16845 obd: rename imp_connect_flags_orig The imp_connect_flags_orig and imp_connect_flags2_orig values both end with "_orig", but there is no corresponding "updated" or other value for them to be original relative to. They seem to be named mirroring ns_connect_flags_orig, where there is also a ns_connect_flags, but this naming makes no sense for the import flags since they're not modified in this way. test-parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1551447b2e92f8cff665dd75c43dd4dde6da9a09
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-1 | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
LU-16858 build: Remove pinger config option Lustre is never built or tested without the pinger, so the config option to build without it should be removed. Note if anyone does want to disable the pinger this can be done at runtime. (This was done historically by Cray, but is not done any more. I suspect no one actually does this, but the option is present if needed.) Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I40aa68fbcee8f68a78316da844951b13bdcb4ffe
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.7/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 crashed | RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-zfs crashed | RHEL 8.7/x86_64 | ran 10 tests. 2 tests failed: replay-single, sanity-quota. %% THIS TEST SESSION CRASHED %% | session |
LU-14639 build: Remove disable-lru-resize config The lru-resize feature is disabled at runtime by setting lru_resize to 0. If it is compiled out or disabled at mount time, then it can never be enabled without recompiling or remounting. Disabling it like this doesn't offer any advantages, and and all customers disabling it are setting lru_size manually already. Let's remove the extra ways to disable lru-resize to avoid confusion and possible bugs. This patch also removes checks in the tests which verify the client and server have lru-resize support. Servers have had lru-resize support for several years, and with this patch, it becomes impossible to build clients without it either. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I801b9eb8cf280e37bc81b3adade7973e295d151f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.8/x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
LU-17210 llite: add kernel readahead asserts Add a set of asserts which confirm kernel readahead is disabled and wasn't used for mmap. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0924f2b1a2bc78e44b9a2082c3dad93a51b4d000
LU-0000 osc: batch osc_consume_write_grant Do osc_consume_write_grant on blocks of transfer pages. Test-Parameters: ignore Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Ieb211c4f7930da1ba431902a9703c22114d8829e
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.4 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
LU-9834 tests: fix loop condition in llapi_layout_test The loop condition in test29 of llapi_layout_test.c was incorrect, causing the loop to never execute. The condition was: for (i = LOV_MAX_STRIPE_COUNT-1; i <= 0; i--) Since LOV_MAX_STRIPE_COUNT-1 is a large positive number and the condition checks if i <= 0, the loop body was never entered. This patch fixes the condition to i >= 0 so that the loop properly executes as intended, starting from LOV_MAX_STRIPE_COUNT-1 and decrementing until it reaches 0. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If96063227542f79004688786f0aeb42e3b27b9d5
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_398k | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity2@ldiskfs+DNE:test_398l | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | CentOS 8.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15483 tests: Use fallocate to fill OSTs Rather than using dd, we can use fallocate to fill the OSTs in 398k and 398l, which should make them much faster. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id65726ccafb3a6e57581b3b14c846ad6b3757c35
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_39r | seen in 19 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.8/x86_64 | ran 7 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-zfs-part-2 crashed | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-zfs-part-6 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
These commands are piling up in a stack, which means they are going to be executed in reverse order IIUC, which gives: 1. wait_mds_ost_sync 2. wait_delete_completed 3. rm -f $DIR/$tfile* Is this what we want?
LU-16704 tests: cleanup after 398l Test 398l fills an OST, so we should have it delete the files it created and wait for delete/sync before the end of the test. Otherwise it can cause ENOSPC on tests that run after it. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5b5916046689c4a016893bc44cdfe9012bd4e987
LU-17433 osc: Make unaligned DIO async Because unaligned DIO is using a copy of the data from userspace, we can make the writes async on the OST side. Because DIO expects that data be 'safe', we cannot just go entirely async and not wait for RPC completion, but we can wait only for RPC completion and not force a commit sync on the server. This is because having our own copy of the data lets us replay the RPC after write() has completed, which is not possible for regular DIO. This reduces the DIO 'O_SYNC' guarantee slightly for unaligned DIO (and other small DIO if we change this). Before this patch, after a DIO write(), the data is fully committed to disk and cannot be lost even if the client and server crash. With this patch, after write() returns, if the server crashes, the client can replay the RPC, and if the client crashes, the server will finish writing out the data. However, if the client and server both crash, the data will be lost. For this reason, we make this behavior tunable: llite.*.dio_full_sync With the default to '0'. Todo: Run through tests Get perf #s for this Write a test verifying this behavior? Test-parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6ab6da2844010df209f219eb9df75c4bbb6e2042
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-4 | RHEL 8.9 / x86_64 | ran 9 tests. 2 tests failed: sanity-hsm, sanity-flr. | session |
LU-17831 osc: discard all if discard one When we cancel a read lock, we check if another read lock also protects the same pages. This is surprisingly expensive. So don't do this for every page: If we find a page which is not covered by a second lock (and therefore must be discarded), we discard all pages after that one. This cuts the time to discard 8 GiB of data under a read lock from 2.2 seconds to 1.2 seconds on a small VM system, so nearly a 50% reduction in time required. We may also do batch discards in the future, which this will permit. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5b5a63c9fcae246fd3db35e613df1cd882544946
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs | RHEL 9.4 / x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
LU-17843 build: correctly create lustre-devel.files The lustre-devel.files file is currently 'created' in the %description section of rpmbuild (the :> command), but that command doesn't do anything in %description. Move this to %install and add a blank line so the file is not empty. Otherwise, the lustre-devel.files file is only populated when we build with --shared, not static. If the lustre-devel.files file is not present OR if it's entirely empty (0 size), this results in an error in rpmbuild. Note the lustre-devel package still has other contents added in the %files directive, so lustre-devel is sound - the issue is just that you can't build RPMs when compiling statically due to the spec file issues. Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: Ic598531e376ce1cd356330023b74ec624b9adea8
| unique failing test | history |
|---|---|
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 5 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-3 | RHEL 9.3/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 9.3/x86_64 | ran 9 tests. 4 tests failed: sanity-quota, sanity-hsm, sanity-flr, replay-ost-single. | session |
| review-dne-part-5 | RHEL 9.3/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-part-6 | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-part-7 crashed | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.9/x86_64 | ran 5 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.9/x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-2 | RHEL 8.9/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-3 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. This build will be scored -1 since fortestonly was specified. | session |
| review-dne-zfs-part-4 | RHEL 8.9/x86_64 | ran 9 tests. 4 tests failed: sanity-quota, sanity-hsm, sanity-flr, replay-ost-single. | session |
| review-dne-zfs-part-5 | RHEL 8.9/x86_64 | ran 6 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-dne-zfs-part-6 | RHEL 8.9/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-zfs-part-7 | RHEL 8.9/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. | session |
| review-ldiskfs crashed | RHEL 8.9/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.9/x86_64, SLES 15.5/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-zfs | RHEL 8.9/x86_64 | ran 8 tests. 3 tests failed: replay-ost-single, replay-single, sanity-quota. | session |
LU-0000 tgt: true async write commits This implements async write commit. TODO: - Testing - Add handling of failed commits (not too hard - just store error in export and make next commit forced to sync, like how the client handles the analogous) This is intended to attach to hybrid IO. Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I83e80bfea57bd9780ff5fec10cc4c3e992690584
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 8 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 7 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 10 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 10 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 10 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 10 other reviews |
| sanity1@zfs:test_56x | seen in 40 other reviews |
| sanity1@zfs:test_56xB | seen in 14 other reviews |
| sanity1@zfs:test_56xa | seen in 40 other reviews |
| sanity1@zfs:test_56xab | seen in 11 other reviews |
| sanity1@zfs:test_56xc | seen in 36 other reviews |
| sanity1@zfs:test_56ej | seen in 14 other reviews |
| sanity2@zfs:test_119e | seen in 21 other reviews |
| sanity2@zfs:test_119f | seen in 20 other reviews |
| sanity2@zfs:test_119g | seen in 20 other reviews |
| sanity2@zfs:test_119h | seen in 20 other reviews |
| sanity2@zfs:test_119p | seen in 11 other reviews |
| sanity2@zfs:test_119q | seen in 11 other reviews |
| sanity2@zfs:test_398o | seen in 25 other reviews |
| sanity2@zfs:test_398s | seen in 11 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 13 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 72 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 31 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 33 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_38 | seen in 34 other reviews |
| sanity-flr@zfs:test_44b | seen in 11 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 61 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 12 other reviews |
| sanity-hsm@zfs:test_607b | seen in 12 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 30 other reviews |
| sanityn@zfs:test_16j | seen in 33 other reviews |
LU-13814 osc: call transfer_page_init for DIO The transfer pages for DIO need to be set up, this does the necessary wrangling to call osc_transfer_page_init from the DIO path. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0fdc3340cfcecb1dc524c55f480961d36cabdedc
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 24 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 23 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_63b | seen in 61 other reviews |
| sanity2@ldiskfs+DNE:test_64a | seen in 61 other reviews |
| sanity2@ldiskfs+DNE:test_64c | seen in 61 other reviews |
| sanity2@ldiskfs+DNE:test_64d | seen in 100 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 39 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 29 other reviews |
| sanity1@zfs:test_56x | seen in 44 other reviews |
| sanity1@zfs:test_56xB | seen in 35 other reviews |
| sanity1@zfs:test_56xa | seen in 44 other reviews |
| sanity1@zfs:test_56xab | seen in 32 other reviews |
| sanity1@zfs:test_56xc | seen in 40 other reviews |
| sanity1@zfs:test_56ej | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 78 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 36 other reviews |
| sanity-flr@zfs:test_0g | seen in 37 other reviews |
| sanity-flr@zfs:test_0h | seen in 39 other reviews |
| sanity-flr@zfs:test_0j | seen in 38 other reviews |
| sanity-flr@zfs:test_36d | seen in 37 other reviews |
| sanity-flr@zfs:test_37 | seen in 40 other reviews |
| sanity-flr@zfs:test_38 | seen in 38 other reviews |
| sanity-flr@zfs:test_44b | seen in 32 other reviews |
| sanity-flr@zfs:test_61a | seen in 44 other reviews |
| sanity-flr@zfs:test_61c | seen in 37 other reviews |
| sanity-flr@zfs:test_70a | seen in 68 other reviews |
| sanity-flr@zfs:test_200a | seen in 37 other reviews |
| sanity-flr@zfs:test_200b | seen in 37 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 33 other reviews |
| sanity-hsm@zfs:test_250 | seen in 11 other reviews |
| sanity-hsm@zfs:test_607b | seen in 33 other reviews |
| sanity-quota@zfs:test_90a | seen in 2 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 36 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-17885 osc: add oe_page_array for dio This is a very simple patch which just adds the array and a few asserts for places that never see DIO pages. The array is used in the next patch. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If1cb80167e409a8dc36711b46ed4b5459a88df75
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 25 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 24 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 27 other reviews |
| sanity1@zfs:test_56x | seen in 44 other reviews |
| sanity1@zfs:test_56xB | seen in 33 other reviews |
| sanity1@zfs:test_56xa | seen in 44 other reviews |
| sanity1@zfs:test_56xab | seen in 30 other reviews |
| sanity1@zfs:test_56xc | seen in 40 other reviews |
| sanity1@zfs:test_56ej | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 78 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 36 other reviews |
| sanity-flr@zfs:test_0g | seen in 37 other reviews |
| sanity-flr@zfs:test_0h | seen in 39 other reviews |
| sanity-flr@zfs:test_0j | seen in 38 other reviews |
| sanity-flr@zfs:test_36d | seen in 37 other reviews |
| sanity-flr@zfs:test_37 | seen in 40 other reviews |
| sanity-flr@zfs:test_38 | seen in 38 other reviews |
| sanity-flr@zfs:test_44b | seen in 31 other reviews |
| sanity-flr@zfs:test_61a | seen in 44 other reviews |
| sanity-flr@zfs:test_61c | seen in 37 other reviews |
| sanity-flr@zfs:test_70a | seen in 67 other reviews |
| sanity-flr@zfs:test_200a | seen in 37 other reviews |
| sanity-flr@zfs:test_200b | seen in 37 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 31 other reviews |
| sanity-hsm@zfs:test_607b | seen in 32 other reviews |
| sanity-pcc@zfs:test_1d | seen in 34 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-17885 osc: remove DIO otp list use in osc_build_rpc This removes the usage of the osc_transfer_page list for DIO in osc_build_rpc. The list is still created and used elsewhere, but this reduces the usage with an eye to removal. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie04ef14569d07f5dde01b223f0dced66c2af0094
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 23 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 22 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 25 other reviews |
| sanity1@zfs:test_56x | seen in 44 other reviews |
| sanity1@zfs:test_56xB | seen in 34 other reviews |
| sanity1@zfs:test_56xa | seen in 44 other reviews |
| sanity1@zfs:test_56xab | seen in 31 other reviews |
| sanity1@zfs:test_56xc | seen in 40 other reviews |
| sanity1@zfs:test_56ej | seen in 34 other reviews |
| sanity2@zfs:test_119e | seen in 35 other reviews |
| sanity2@zfs:test_119f | seen in 34 other reviews |
| sanity2@zfs:test_119g | seen in 34 other reviews |
| sanity2@zfs:test_119h | seen in 34 other reviews |
| sanity2@zfs:test_119p | seen in 30 other reviews |
| sanity2@zfs:test_119q | seen in 30 other reviews |
| sanity2@zfs:test_398o | seen in 39 other reviews |
| sanity2@zfs:test_398s | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 77 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 36 other reviews |
| sanity-flr@zfs:test_0g | seen in 37 other reviews |
| sanity-flr@zfs:test_0h | seen in 39 other reviews |
| sanity-flr@zfs:test_0j | seen in 38 other reviews |
| sanity-flr@zfs:test_36d | seen in 37 other reviews |
| sanity-flr@zfs:test_37 | seen in 40 other reviews |
| sanity-flr@zfs:test_38 | seen in 38 other reviews |
| sanity-flr@zfs:test_44b | seen in 30 other reviews |
| sanity-flr@zfs:test_61a | seen in 44 other reviews |
| sanity-flr@zfs:test_61c | seen in 37 other reviews |
| sanity-flr@zfs:test_70a | seen in 66 other reviews |
| sanity-flr@zfs:test_200b | seen in 37 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 32 other reviews |
| sanity-hsm@zfs:test_607b | seen in 31 other reviews |
| sanity-pcc@ldiskfs+DNE:test_1c | seen in 27 other reviews |
| sanity-quota@zfs:test_90b | seen in 5 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 36 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-17885 osc: remove list for dio in extent finish This removes the last usage of the page lists for DIO, and also stops creating them. This gets us most of the performance benefit, but not all of it - yet. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic2c700a6e09def0e0162ab567d9a0af321fa7e87
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 21 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 20 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 39 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 28 other reviews |
| sanity1@zfs:test_56x | seen in 43 other reviews |
| sanity1@zfs:test_56xB | seen in 30 other reviews |
| sanity1@zfs:test_56xa | seen in 43 other reviews |
| sanity1@zfs:test_56xab | seen in 27 other reviews |
| sanity1@zfs:test_56xc | seen in 39 other reviews |
| sanity1@zfs:test_56ej | seen in 30 other reviews |
| sanity2@zfs:test_119e | seen in 34 other reviews |
| sanity2@zfs:test_119f | seen in 33 other reviews |
| sanity2@zfs:test_119g | seen in 33 other reviews |
| sanity2@zfs:test_119h | seen in 33 other reviews |
| sanity2@zfs:test_119p | seen in 27 other reviews |
| sanity2@zfs:test_119q | seen in 27 other reviews |
| sanity2@zfs:test_398o | seen in 38 other reviews |
| sanity2@zfs:test_398s | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 28 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 27 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 28 other reviews |
| sanity-hsm@zfs:test_607b | seen in 28 other reviews |
| sanity-quota@zfs:test_90b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 33 other reviews |
| sanityn@zfs:test_16j | seen in 36 other reviews |
LU-13814 clio: remove cp_type from vvp, mdc, ll cp_type appears only a little in the vvp, mdc, and ll code, so remove it all in one patch. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9dc3dc4e4d35322fbe4428d4d8ffd624baada693
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_119e | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 38 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 27 other reviews |
| sanity1@zfs:test_56x | seen in 42 other reviews |
| sanity1@zfs:test_56xB | seen in 27 other reviews |
| sanity1@zfs:test_56xa | seen in 42 other reviews |
| sanity1@zfs:test_56xab | seen in 24 other reviews |
| sanity1@zfs:test_56xc | seen in 38 other reviews |
| sanity1@zfs:test_56ej | seen in 27 other reviews |
| sanity2@zfs:test_119e | seen in 30 other reviews |
| sanity2@zfs:test_119f | seen in 29 other reviews |
| sanity2@zfs:test_119g | seen in 29 other reviews |
| sanity2@zfs:test_119h | seen in 29 other reviews |
| sanity2@zfs:test_119p | seen in 23 other reviews |
| sanity2@zfs:test_119q | seen in 23 other reviews |
| sanity2@zfs:test_398o | seen in 34 other reviews |
| sanity2@zfs:test_398s | seen in 23 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 24 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 26 other reviews |
| sanity-hsm@zfs:test_607b | seen in 24 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 33 other reviews |
| sanityn@zfs:test_16j | seen in 36 other reviews |
LU-13814 clio: remove type from cl_page_find type is no longer used in cl_page_find/alloc, remove it. Test-Parameters: forjanitoronly Test-Parameters: fortestonly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5168f5d34e24727d45d1d0910d8b90cec9429b35
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 37 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 26 other reviews |
| sanity1@zfs:test_56x | seen in 44 other reviews |
| sanity1@zfs:test_56xB | seen in 32 other reviews |
| sanity1@zfs:test_56xa | seen in 44 other reviews |
| sanity1@zfs:test_56xab | seen in 29 other reviews |
| sanity1@zfs:test_56xc | seen in 40 other reviews |
| sanity1@zfs:test_56ej | seen in 32 other reviews |
| sanity2@zfs:test_119e | seen in 35 other reviews |
| sanity2@zfs:test_119f | seen in 34 other reviews |
| sanity2@zfs:test_119g | seen in 34 other reviews |
| sanity2@zfs:test_119h | seen in 34 other reviews |
| sanity2@zfs:test_119p | seen in 29 other reviews |
| sanity2@zfs:test_119q | seen in 29 other reviews |
| sanity2@zfs:test_398o | seen in 39 other reviews |
| sanity2@zfs:test_398s | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 44 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 76 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 36 other reviews |
| sanity-flr@zfs:test_0g | seen in 37 other reviews |
| sanity-flr@zfs:test_0h | seen in 39 other reviews |
| sanity-flr@zfs:test_0j | seen in 38 other reviews |
| sanity-flr@zfs:test_36d | seen in 37 other reviews |
| sanity-flr@zfs:test_37 | seen in 40 other reviews |
| sanity-flr@zfs:test_38 | seen in 38 other reviews |
| sanity-flr@zfs:test_44b | seen in 29 other reviews |
| sanity-flr@zfs:test_61a | seen in 44 other reviews |
| sanity-flr@zfs:test_61c | seen in 37 other reviews |
| sanity-flr@zfs:test_70a | seen in 65 other reviews |
| sanity-flr@zfs:test_200a | seen in 37 other reviews |
| sanity-flr@zfs:test_200b | seen in 37 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 30 other reviews |
| sanity-hsm@zfs:test_607b | seen in 30 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 35 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-13814 osc: change asserts to use OTP type cp_type is no longer set for transient pages since they don't use cl_page. Switch asserts to use otp_type. Test-Parameters: trivial Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I3ad8292972fad9669b00aac3e98d0b6ea12ef398
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 18 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 17 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 36 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 25 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 25 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 22 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 25 other reviews |
| sanity2@zfs:test_119e | seen in 27 other reviews |
| sanity2@zfs:test_119f | seen in 26 other reviews |
| sanity2@zfs:test_119g | seen in 26 other reviews |
| sanity2@zfs:test_119h | seen in 26 other reviews |
| sanity2@zfs:test_119p | seen in 20 other reviews |
| sanity2@zfs:test_119q | seen in 20 other reviews |
| sanity2@zfs:test_398o | seen in 31 other reviews |
| sanity2@zfs:test_398s | seen in 20 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 23 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 22 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 23 other reviews |
| sanity-hsm@zfs:test_607b | seen in 23 other reviews |
| sanity-quota@zfs:test_90b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: move otp_obj to osc page The object pointer is only used from the OSC page, so move it there. This reduces the size of the OTP page, which is useful because for DIO there is only an OTP page. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I97360277a9d686b7b78648762d45d20864355cbc
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 22 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 21 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 39 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 35 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 24 other reviews |
| sanity1@zfs:test_56x | seen in 43 other reviews |
| sanity1@zfs:test_56xB | seen in 31 other reviews |
| sanity1@zfs:test_56xa | seen in 43 other reviews |
| sanity1@zfs:test_56xab | seen in 28 other reviews |
| sanity1@zfs:test_56xc | seen in 39 other reviews |
| sanity1@zfs:test_56ej | seen in 31 other reviews |
| sanity2@zfs:test_119e | seen in 34 other reviews |
| sanity2@zfs:test_119f | seen in 33 other reviews |
| sanity2@zfs:test_119g | seen in 33 other reviews |
| sanity2@zfs:test_119h | seen in 33 other reviews |
| sanity2@zfs:test_119p | seen in 28 other reviews |
| sanity2@zfs:test_119q | seen in 28 other reviews |
| sanity2@zfs:test_398o | seen in 38 other reviews |
| sanity2@zfs:test_398s | seen in 28 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 28 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 29 other reviews |
| sanity-hsm@zfs:test_607b | seen in 29 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 34 other reviews |
| sanityn@zfs:test_16j | seen in 37 other reviews |
LU-13814 osc: remove cp_type in osc Now that we no longer have cl_page for transient pages, cp_type is going away, so remove it in the OSC layer. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2ea93f46d8a99ba6c0e04af373827b4c8b8fd2be
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 20 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 19 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 34 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 23 other reviews |
| sanity1@zfs:test_56x | seen in 43 other reviews |
| sanity1@zfs:test_56xB | seen in 29 other reviews |
| sanity1@zfs:test_56xa | seen in 43 other reviews |
| sanity1@zfs:test_56xab | seen in 26 other reviews |
| sanity1@zfs:test_56xc | seen in 39 other reviews |
| sanity1@zfs:test_56ej | seen in 29 other reviews |
| sanity2@zfs:test_119e | seen in 33 other reviews |
| sanity2@zfs:test_119f | seen in 32 other reviews |
| sanity2@zfs:test_119g | seen in 32 other reviews |
| sanity2@zfs:test_119h | seen in 32 other reviews |
| sanity2@zfs:test_119p | seen in 26 other reviews |
| sanity2@zfs:test_119q | seen in 26 other reviews |
| sanity2@zfs:test_398o | seen in 37 other reviews |
| sanity2@zfs:test_398s | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 26 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 27 other reviews |
| sanity-hsm@zfs:test_607b | seen in 27 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 36 other reviews |
LU-13814 lov: remove cp_type from lov Removing cp_type from lov involves removing the LOV stripe information caching, since that was just for DIO. Other removals here are trivial. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia60397639e9499e69c17f8d549806f17b4000d05
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_119e | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 33 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 22 other reviews |
| sanity1@zfs:test_56x | seen in 43 other reviews |
| sanity1@zfs:test_56xB | seen in 28 other reviews |
| sanity1@zfs:test_56xa | seen in 43 other reviews |
| sanity1@zfs:test_56xab | seen in 25 other reviews |
| sanity1@zfs:test_56xc | seen in 39 other reviews |
| sanity1@zfs:test_56ej | seen in 28 other reviews |
| sanity2@zfs:test_119e | seen in 31 other reviews |
| sanity2@zfs:test_119f | seen in 30 other reviews |
| sanity2@zfs:test_119g | seen in 30 other reviews |
| sanity2@zfs:test_119h | seen in 30 other reviews |
| sanity2@zfs:test_119p | seen in 24 other reviews |
| sanity2@zfs:test_119q | seen in 24 other reviews |
| sanity2@zfs:test_398o | seen in 35 other reviews |
| sanity2@zfs:test_398s | seen in 24 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 43 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 75 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 35 other reviews |
| sanity-flr@zfs:test_0g | seen in 36 other reviews |
| sanity-flr@zfs:test_0h | seen in 38 other reviews |
| sanity-flr@zfs:test_0j | seen in 37 other reviews |
| sanity-flr@zfs:test_36d | seen in 36 other reviews |
| sanity-flr@zfs:test_37 | seen in 39 other reviews |
| sanity-flr@zfs:test_38 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 25 other reviews |
| sanity-flr@zfs:test_61a | seen in 43 other reviews |
| sanity-flr@zfs:test_61c | seen in 36 other reviews |
| sanity-flr@zfs:test_70a | seen in 64 other reviews |
| sanity-flr@zfs:test_200a | seen in 36 other reviews |
| sanity-flr@zfs:test_200b | seen in 36 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 25 other reviews |
| sanity-hsm@zfs:test_607b | seen in 26 other reviews |
| sanity-quota@zfs:test_90b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 36 other reviews |
LU-13814 clio: remove cp_type Page->cp_type is no longer needed and can be removed entirely. Two notes: This makes 'inode' in coo_page_init irrelevant since it's only used for DIO. This will be fixed in a future patch. The packing of cl_page is NOT correct currently and will be fixed in a future patch. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I660bb956b2d85fcff98b8e1726d60b51fd4f8ac5
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 17 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 16 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 31 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 19 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 24 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 21 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 24 other reviews |
| sanity2@zfs:test_119e | seen in 32 other reviews |
| sanity2@zfs:test_119f | seen in 31 other reviews |
| sanity2@zfs:test_119g | seen in 31 other reviews |
| sanity2@zfs:test_119h | seen in 31 other reviews |
| sanity2@zfs:test_119p | seen in 25 other reviews |
| sanity2@zfs:test_119q | seen in 25 other reviews |
| sanity2@zfs:test_398o | seen in 36 other reviews |
| sanity2@zfs:test_398s | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 22 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 21 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 22 other reviews |
| sanity-hsm@zfs:test_607b | seen in 22 other reviews |
| sanity-quota@zfs:test_90b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: remove otp_srvlock The srvlock information stored in the otp_srvlock flag is always available elsewhere, so remove it. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7a8669b8a3ae9014a9971386c316d514079a0ae1
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 14 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 13 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 21 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 20 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 17 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 20 other reviews |
| sanity2@zfs:test_119e | seen in 23 other reviews |
| sanity2@zfs:test_119f | seen in 22 other reviews |
| sanity2@zfs:test_119g | seen in 22 other reviews |
| sanity2@zfs:test_119h | seen in 22 other reviews |
| sanity2@zfs:test_119p | seen in 15 other reviews |
| sanity2@zfs:test_119q | seen in 15 other reviews |
| sanity2@zfs:test_398o | seen in 27 other reviews |
| sanity2@zfs:test_398s | seen in 15 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 28 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 18 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 73 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 32 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 17 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 18 other reviews |
| sanity-hsm@zfs:test_607b | seen in 18 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 31 other reviews |
| sanityn@zfs:test_16j | seen in 34 other reviews |
LU-13814 osc: remove aa_otps The aa_oops list isn't really used for anything, so let's remove it. This will let us remove the otp_rpc_item list, because the only use of that list was to go on the aa_otps list. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I491f66857786dd9ed13657d4211e774232cb6e22
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 19 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 18 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 38 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 32 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 20 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 26 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 23 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 26 other reviews |
| sanity2@zfs:test_119e | seen in 28 other reviews |
| sanity2@zfs:test_119f | seen in 27 other reviews |
| sanity2@zfs:test_119g | seen in 27 other reviews |
| sanity2@zfs:test_119h | seen in 27 other reviews |
| sanity2@zfs:test_119p | seen in 21 other reviews |
| sanity2@zfs:test_119q | seen in 21 other reviews |
| sanity2@zfs:test_398o | seen in 32 other reviews |
| sanity2@zfs:test_398s | seen in 21 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 24 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 23 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 24 other reviews |
| sanity-hsm@zfs:test_607b | seen in 25 other reviews |
| sanity-pcc@ldiskfs+DNE:test_100 | seen in 33 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: minor function relocation Two functions are essentially in the wrong file, and one of them is a trivial wrapper. Move the actual function and just remove the wrapper. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie408734f02a8047621f3447b13f5d1786b070801
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 12 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 11 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 14 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 30 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 18 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 18 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 15 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 18 other reviews |
| sanity2@zfs:test_119e | seen in 23 other reviews |
| sanity2@zfs:test_119f | seen in 22 other reviews |
| sanity2@zfs:test_119g | seen in 22 other reviews |
| sanity2@zfs:test_119h | seen in 22 other reviews |
| sanity2@zfs:test_119p | seen in 14 other reviews |
| sanity2@zfs:test_119q | seen in 14 other reviews |
| sanity2@zfs:test_398o | seen in 27 other reviews |
| sanity2@zfs:test_398s | seen in 14 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 28 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 17 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 73 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 32 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 16 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 17 other reviews |
| sanity-hsm@zfs:test_607b | seen in 17 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 31 other reviews |
| sanityn@zfs:test_16j | seen in 34 other reviews |
LU-13814 osc: remove otp_rpc_item list The otp_rpc_item list is only used to recognize if a page is in an RPC, which an error condition that should only occur if there's a bug in the code. We can remove it, and save ourselves two pointers in every page. With this, this the osc_transfer_page is now 61 bytes in size, fitting it inside a single cacheline(!). This has a huge impact on DIO performance. The net effect of these reduction patches is about a 40-50% boost in single threaded DIO performance beyond that achieved by cl_page removal. 1 GiB transfer size IOR performance (single threaded) Without the reduction patches: Read: 57 GiB/s Write: 48 GiB/s With: Read: 88 GiB/s Write: 65 GiB/s (Write performance is expected to be closer to read, but it may have been affected by a network issue.) Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Icb83298da1bdc797a33dae6ca3357b1ad3b3c848
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 16 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 15 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 28 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 16 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 21 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 18 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 21 other reviews |
| sanity2@zfs:test_119e | seen in 29 other reviews |
| sanity2@zfs:test_119f | seen in 28 other reviews |
| sanity2@zfs:test_119g | seen in 28 other reviews |
| sanity2@zfs:test_119h | seen in 28 other reviews |
| sanity2@zfs:test_119p | seen in 22 other reviews |
| sanity2@zfs:test_119q | seen in 22 other reviews |
| sanity2@zfs:test_398o | seen in 33 other reviews |
| sanity2@zfs:test_398s | seen in 22 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 20 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 19 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 20 other reviews |
| sanity-hsm@zfs:test_607b | seen in 20 other reviews |
| sanity-pcc@ldiskfs+DNE:test_1c | seen in 26 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: remove otp_page_off otp_page_off can be determined from otp_obj_off, so we can remove it. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4ab9e71a8d6e79544241021b72eff2a95c592cb9
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 15 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 14 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 23 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 17 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 29 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 17 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 23 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 20 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 23 other reviews |
| sanity2@zfs:test_119e | seen in 25 other reviews |
| sanity2@zfs:test_119f | seen in 24 other reviews |
| sanity2@zfs:test_119g | seen in 24 other reviews |
| sanity2@zfs:test_119h | seen in 24 other reviews |
| sanity2@zfs:test_119p | seen in 18 other reviews |
| sanity2@zfs:test_119q | seen in 18 other reviews |
| sanity2@zfs:test_398o | seen in 29 other reviews |
| sanity2@zfs:test_398s | seen in 18 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 21 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 42 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 74 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 34 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 20 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 21 other reviews |
| sanity-hsm@zfs:test_607b | seen in 21 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: remove otp_cmd The otp_cmd information is only need in the OSC page, so move it there so it's not allocated for DIO. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2c32126e72d45dad5a2104ded0b55cdb1150adce
| unique failing test | history |
|---|---|
| racer@zfs:test_1 | seen in 6 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 14 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 14 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 14 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 19 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 16 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 19 other reviews |
| sanity2@zfs:test_119e | seen in 26 other reviews |
| sanity2@zfs:test_119f | seen in 25 other reviews |
| sanity2@zfs:test_119g | seen in 25 other reviews |
| sanity2@zfs:test_119h | seen in 25 other reviews |
| sanity2@zfs:test_119p | seen in 19 other reviews |
| sanity2@zfs:test_119q | seen in 19 other reviews |
| sanity2@zfs:test_398o | seen in 30 other reviews |
| sanity2@zfs:test_398s | seen in 19 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 16 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 72 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 31 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 34 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_44b | seen in 15 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 62 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 16 other reviews |
| sanity-hsm@zfs:test_607b | seen in 16 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 31 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: add osc_transfer_page comments Add comments highlighting the critical nature of the OSC transfer page for performance. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ifb282c0a5332fac3bf0a054a9e27b2e596fce8f0
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 13 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 12 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 37 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 15 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 21 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 15 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 15 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 27 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 15 other reviews |
| sanity1@zfs:test_56x | seen in 41 other reviews |
| sanity1@zfs:test_56xB | seen in 22 other reviews |
| sanity1@zfs:test_56xa | seen in 41 other reviews |
| sanity1@zfs:test_56xab | seen in 19 other reviews |
| sanity1@zfs:test_56xc | seen in 37 other reviews |
| sanity1@zfs:test_56ej | seen in 22 other reviews |
| sanity2@zfs:test_119e | seen in 24 other reviews |
| sanity2@zfs:test_119f | seen in 23 other reviews |
| sanity2@zfs:test_119g | seen in 23 other reviews |
| sanity2@zfs:test_119h | seen in 23 other reviews |
| sanity2@zfs:test_119p | seen in 16 other reviews |
| sanity2@zfs:test_119q | seen in 16 other reviews |
| sanity2@zfs:test_398o | seen in 28 other reviews |
| sanity2@zfs:test_398s | seen in 16 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 19 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 41 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 73 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 33 other reviews |
| sanity-flr@zfs:test_0g | seen in 35 other reviews |
| sanity-flr@zfs:test_0h | seen in 37 other reviews |
| sanity-flr@zfs:test_0j | seen in 36 other reviews |
| sanity-flr@zfs:test_36d | seen in 35 other reviews |
| sanity-flr@zfs:test_37 | seen in 38 other reviews |
| sanity-flr@zfs:test_38 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 18 other reviews |
| sanity-flr@zfs:test_61a | seen in 42 other reviews |
| sanity-flr@zfs:test_61c | seen in 35 other reviews |
| sanity-flr@zfs:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_200a | seen in 35 other reviews |
| sanity-flr@zfs:test_200b | seen in 35 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 19 other reviews |
| sanity-hsm@zfs:test_607b | seen in 19 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 35 other reviews |
LU-13814 osc: remove otp_inode We can add the inode to the osc object, which allows us to remove the inode from the osc_transfer_page. This removes a pointer from the OTP struct, which is allocated for every page in Lustre. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ida9c030bf085ad9606e19522714497c4adfa33de
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 10 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 9 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 12 other reviews |
| sanity1@zfs:test_56x | seen in 40 other reviews |
| sanity1@zfs:test_56xB | seen in 16 other reviews |
| sanity1@zfs:test_56xa | seen in 40 other reviews |
| sanity1@zfs:test_56xab | seen in 13 other reviews |
| sanity1@zfs:test_56xc | seen in 36 other reviews |
| sanity1@zfs:test_56ej | seen in 16 other reviews |
| sanity2@zfs:test_119e | seen in 24 other reviews |
| sanity2@zfs:test_119f | seen in 23 other reviews |
| sanity2@zfs:test_119g | seen in 23 other reviews |
| sanity2@zfs:test_119h | seen in 23 other reviews |
| sanity2@zfs:test_119p | seen in 17 other reviews |
| sanity2@zfs:test_119q | seen in 17 other reviews |
| sanity2@zfs:test_398o | seen in 28 other reviews |
| sanity2@zfs:test_398s | seen in 17 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 14 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 72 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 31 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 33 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_38 | seen in 34 other reviews |
| sanity-flr@zfs:test_44b | seen in 13 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 62 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 13 other reviews |
| sanity-hsm@zfs:test_607b | seen in 14 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 30 other reviews |
| sanityn@zfs:test_16j | seen in 33 other reviews |
LU-13814 osc: remove otp_ll_index otp_ll_index is used to preserve the index for direct IO pages, but direct IO pages do not use the index value, so we can just store the index in the vmpage. This reduces the osc_transfer_page size by a further 8 bytes, to 43 bytes. This puts us tantalizingly close to a single cacheline, but this can only be achieved by removing the otp_pending_item list, which is challenging. That will require converting osc extents to use an array of pointers instead of a linked list. This can be done, and will improve performance because of that change by itself. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I621388fb36a2a792525e07f35c1c948c4ae4292e
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 32 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 5 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 32 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 4 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 33 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 7 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 16 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 6 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 6 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 22 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 6 other reviews |
| sanity1@zfs:test_56x | seen in 39 other reviews |
| sanity1@zfs:test_56xB | seen in 10 other reviews |
| sanity1@zfs:test_56xa | seen in 39 other reviews |
| sanity1@zfs:test_56xab | seen in 7 other reviews |
| sanity1@zfs:test_56xc | seen in 35 other reviews |
| sanity1@zfs:test_56ej | seen in 10 other reviews |
| sanity2@zfs:test_119e | seen in 22 other reviews |
| sanity2@zfs:test_119f | seen in 21 other reviews |
| sanity2@zfs:test_119g | seen in 21 other reviews |
| sanity2@zfs:test_119h | seen in 21 other reviews |
| sanity2@zfs:test_119p | seen in 13 other reviews |
| sanity2@zfs:test_119q | seen in 13 other reviews |
| sanity2@zfs:test_398o | seen in 26 other reviews |
| sanity2@zfs:test_398s | seen in 13 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 9 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 33 other reviews |
| sanity-flr@zfs:test_0h | seen in 35 other reviews |
| sanity-flr@zfs:test_0j | seen in 34 other reviews |
| sanity-flr@zfs:test_36d | seen in 32 other reviews |
| sanity-flr@zfs:test_37 | seen in 36 other reviews |
| sanity-flr@zfs:test_44b | seen in 8 other reviews |
| sanity-flr@zfs:test_61a | seen in 40 other reviews |
| sanity-flr@zfs:test_61c | seen in 33 other reviews |
| sanity-flr@zfs:test_70a | seen in 60 other reviews |
| sanity-flr@zfs:test_200a | seen in 33 other reviews |
| sanity-flr@zfs:test_200b | seen in 33 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 9 other reviews |
| sanity-hsm@zfs:test_607b | seen in 8 other reviews |
| sanity-pcc@zfs:test_100 | seen in 19 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 29 other reviews |
| sanityn@zfs:test_16j | seen in 31 other reviews |
LU-13814 clio: add args to cl_dio_pages_init The inode and osc index are needed to set up the transfer pages, so though they're unused here, they will be used in the next patch. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic41d1ca6db05c53ac86840379abcab2e62220b81
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 11 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 10 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 13 other reviews |
| sanity1@zfs:test_56x | seen in 40 other reviews |
| sanity1@zfs:test_56xB | seen in 17 other reviews |
| sanity1@zfs:test_56xa | seen in 40 other reviews |
| sanity1@zfs:test_56xab | seen in 14 other reviews |
| sanity1@zfs:test_56xc | seen in 36 other reviews |
| sanity1@zfs:test_56ej | seen in 17 other reviews |
| sanity2@zfs:test_119e | seen in 21 other reviews |
| sanity2@zfs:test_119f | seen in 20 other reviews |
| sanity2@zfs:test_119g | seen in 20 other reviews |
| sanity2@zfs:test_119h | seen in 20 other reviews |
| sanity2@zfs:test_119p | seen in 12 other reviews |
| sanity2@zfs:test_119q | seen in 12 other reviews |
| sanity2@zfs:test_398o | seen in 25 other reviews |
| sanity2@zfs:test_398s | seen in 12 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 27 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 15 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 37 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 40 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 72 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 31 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 33 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_38 | seen in 35 other reviews |
| sanity-flr@zfs:test_44b | seen in 14 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 62 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 15 other reviews |
| sanity-hsm@zfs:test_607b | seen in 15 other reviews |
| sanity-quota@zfs:test_90b | seen in 1 other review |
| sanityn@ldiskfs+DNE:test_16j | seen in 30 other reviews |
| sanityn@zfs:test_16j | seen in 33 other reviews |
LU-13814 osc: remove dedicated otp_obj_off member otp_obj_off and the offset stored in the brw page are the same, and the osc transfer page always contains a brw page. So we can just always use the bp_off offset. This saves 4 bytes in the osc_transfer_page, which is a meaningful reduction in size. Total size of osc_transfer_page (on x86_64) is now down to 51 bytes. This does not have a huge performance impact because the struct still occupies two cachelines. (Cachelines are 32 bytes.) Getting the size to 32 bytes will be challenging, but may be possible. For example, the otp_ll_index field may be removable, and it should be possible - with effort - to remove the otp_pending_list. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie38e8154e184c4c9bec38f1ddd482b02cac29ab9
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56x | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 9 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 35 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 8 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 36 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 11 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 20 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 11 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 11 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 26 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 11 other reviews |
| sanity1@zfs:test_56x | seen in 40 other reviews |
| sanity1@zfs:test_56xB | seen in 15 other reviews |
| sanity1@zfs:test_56xa | seen in 40 other reviews |
| sanity1@zfs:test_56xab | seen in 12 other reviews |
| sanity1@zfs:test_56xc | seen in 36 other reviews |
| sanity1@zfs:test_56ej | seen in 15 other reviews |
| sanity2@zfs:test_119e | seen in 20 other reviews |
| sanity2@zfs:test_119f | seen in 19 other reviews |
| sanity2@zfs:test_119g | seen in 19 other reviews |
| sanity2@zfs:test_119h | seen in 19 other reviews |
| sanity2@zfs:test_119p | seen in 10 other reviews |
| sanity2@zfs:test_119q | seen in 10 other reviews |
| sanity2@zfs:test_398o | seen in 24 other reviews |
| sanity2@zfs:test_398s | seen in 10 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 12 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 34 other reviews |
| sanity-flr@zfs:test_0h | seen in 36 other reviews |
| sanity-flr@zfs:test_0j | seen in 35 other reviews |
| sanity-flr@zfs:test_36d | seen in 33 other reviews |
| sanity-flr@zfs:test_37 | seen in 37 other reviews |
| sanity-flr@zfs:test_38 | seen in 34 other reviews |
| sanity-flr@zfs:test_44b | seen in 12 other reviews |
| sanity-flr@zfs:test_61a | seen in 41 other reviews |
| sanity-flr@zfs:test_61c | seen in 34 other reviews |
| sanity-flr@zfs:test_70a | seen in 62 other reviews |
| sanity-flr@zfs:test_200a | seen in 34 other reviews |
| sanity-flr@zfs:test_200b | seen in 34 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 14 other reviews |
| sanity-hsm@zfs:test_607b | seen in 13 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 32 other reviews |
| sanityn@zfs:test_16j | seen in 33 other reviews |
LU-17885 osc: move list to queue_dio_pages Move list handling to queue_dio_pages. This is a precursor to removing list usage and this one actually makes things less efficient, but only briefly. Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2f9671bf276b03e3ce5e9bae2d4239649ae25cec
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_119e | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 8 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 8 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 24 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 8 other reviews |
| sanity1@zfs:test_56x | seen in 39 other reviews |
| sanity1@zfs:test_56xB | seen in 13 other reviews |
| sanity1@zfs:test_56xa | seen in 39 other reviews |
| sanity1@zfs:test_56xab | seen in 10 other reviews |
| sanity1@zfs:test_56xc | seen in 35 other reviews |
| sanity1@zfs:test_56ej | seen in 13 other reviews |
| sanity2@zfs:test_119e | seen in 20 other reviews |
| sanity2@zfs:test_119f | seen in 19 other reviews |
| sanity2@zfs:test_119g | seen in 19 other reviews |
| sanity2@zfs:test_119h | seen in 19 other reviews |
| sanity2@zfs:test_119p | seen in 9 other reviews |
| sanity2@zfs:test_119q | seen in 9 other reviews |
| sanity2@zfs:test_398o | seen in 24 other reviews |
| sanity2@zfs:test_398s | seen in 9 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 11 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 33 other reviews |
| sanity-flr@zfs:test_0h | seen in 35 other reviews |
| sanity-flr@zfs:test_0j | seen in 34 other reviews |
| sanity-flr@zfs:test_36d | seen in 32 other reviews |
| sanity-flr@zfs:test_37 | seen in 36 other reviews |
| sanity-flr@zfs:test_38 | seen in 33 other reviews |
| sanity-flr@zfs:test_44b | seen in 10 other reviews |
| sanity-flr@zfs:test_61a | seen in 40 other reviews |
| sanity-flr@zfs:test_61c | seen in 33 other reviews |
| sanity-flr@zfs:test_70a | seen in 60 other reviews |
| sanity-flr@zfs:test_200a | seen in 33 other reviews |
| sanity-flr@zfs:test_200b | seen in 33 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 10 other reviews |
| sanity-hsm@zfs:test_607b | seen in 9 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 29 other reviews |
| sanityn@zfs:test_16j | seen in 32 other reviews |
LU-13814 osc: drop cl_page structs for DIO This is the big one - this flips the switch and switches DIO from using osc transfer pages which are part of cl_pages and are initialized the same for buffered or DIO, to using bare OSC transfer pages - no associated cl_page. This patch is the primary goal of this series, but it stops halfway, leaving out removing the cl_page allocation for DIO, because that is better done in a separate patch. Once that is done in the next patch, we'll see the performance jump. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7f664f0ffee97c9f2c778996423cef5ae79c3460
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 33 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 6 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 33 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 5 other reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 34 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 8 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 9 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 9 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 25 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 9 other reviews |
| sanity1@zfs:test_56x | seen in 39 other reviews |
| sanity1@zfs:test_56xB | seen in 12 other reviews |
| sanity1@zfs:test_56xa | seen in 39 other reviews |
| sanity1@zfs:test_56xab | seen in 9 other reviews |
| sanity1@zfs:test_56xc | seen in 35 other reviews |
| sanity1@zfs:test_56ej | seen in 12 other reviews |
| sanity2@zfs:test_119e | seen in 19 other reviews |
| sanity2@zfs:test_119f | seen in 18 other reviews |
| sanity2@zfs:test_119g | seen in 18 other reviews |
| sanity2@zfs:test_119h | seen in 18 other reviews |
| sanity2@zfs:test_119p | seen in 8 other reviews |
| sanity2@zfs:test_119q | seen in 8 other reviews |
| sanity2@zfs:test_398o | seen in 23 other reviews |
| sanity2@zfs:test_398s | seen in 8 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 26 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 8 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 36 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 39 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 71 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 30 other reviews |
| sanity-flr@zfs:test_0g | seen in 33 other reviews |
| sanity-flr@zfs:test_0h | seen in 35 other reviews |
| sanity-flr@zfs:test_0j | seen in 34 other reviews |
| sanity-flr@zfs:test_36d | seen in 32 other reviews |
| sanity-flr@zfs:test_37 | seen in 36 other reviews |
| sanity-flr@zfs:test_38 | seen in 33 other reviews |
| sanity-flr@zfs:test_44b | seen in 7 other reviews |
| sanity-flr@zfs:test_61a | seen in 40 other reviews |
| sanity-flr@zfs:test_61c | seen in 33 other reviews |
| sanity-flr@zfs:test_70a | seen in 60 other reviews |
| sanity-flr@zfs:test_200a | seen in 33 other reviews |
| sanity-flr@zfs:test_200b | seen in 33 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 8 other reviews |
| sanity-hsm@zfs:test_607b | seen in 10 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 29 other reviews |
| sanityn@zfs:test_16j | seen in 31 other reviews |
LU-13814 clio: remove cl_page allocation for DIO This removes the - now unused - cl_page allocation for DIO. This will be followed by further patches cleaning up various checks associated with transient cl pages, which no longer exist. With this patch, the performance benefits are realized. This reduces the time to submit DIO pages by about 85%. This results in about a 2.5-3x improvement in DIO performance, because of other overheads and hardware limitations. Without this patch, IOR at 1 GiB transfer size: Read: 22 GiB/s Write: 20 GiB/s With this patch, IOR at 1 GiB transfer size: Read: 57 GiB/s Write: 48 GiB/s There's still some substantial overhead in the transfer page allocations which is removed by the rest of this series. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I406c69d69049484e477a671ada6b4e95357d9f39
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 1 other review |
| runtests@zfs:test_1 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-17885 osc: move pending_list to OSC page Move the pending list to the osc page. This reduces the size of osc_transfer_page by two full pointers, which is quite significant - the entire struct is around 48 bytes, so the removal of two pointers has a significant impact. There's now little more than a BRW page in here. It wouldn't be too difficult to replace the flags in here with BRW flags, which would make the transfer page just a BRW page. This would give minimal benefit since it doesn't actually reduce the allocated size, but might be nice. Next, it is in theory possible to remove all per-page data for DIO except the vmpage pointer. For DIO, all per-page data (even size and offset, even for encryption and compression) can be inferred from a header associated with the extent. Still, at this point, one thread can submit DIO at over 100 GiB/s, so there's little point to further improvements. Time is better spent in other areas. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9c752f3dc2bb71c5ca2ed8bf6b3cb8db1756ad6c
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 1 other review |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 1 other review |
| runtests@zfs:test_1 | seen in 1 other review |
LU-17885 osc: inline osc_transfer_page_init Inlining osc_transfer_page_init notably reduces CPU time used in initializing pages - from 27% to 21% of total time. In this microbenchmark, this reduces CPU time for DIO overall by 10%, which should drive a performance improvement of up to 11%. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1193ce9f6a930aa07f2cc7f5bb8f4c4484ab2d03
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 31 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 2 other reviews |
| sanity1@ldiskfs+DNE:test_56xa | seen in 31 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | seen in 1 other review |
| sanity1@ldiskfs+DNE:test_56xc | seen in 32 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 4 other reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 13 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 3 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 3 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 19 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 3 other reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 7 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 4 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 7 other reviews |
| sanity2@zfs:test_119e | seen in 16 other reviews |
| sanity2@zfs:test_119f | seen in 15 other reviews |
| sanity2@zfs:test_119g | seen in 15 other reviews |
| sanity2@zfs:test_119h | seen in 15 other reviews |
| sanity2@zfs:test_119p | seen in 4 other reviews |
| sanity2@zfs:test_119q | seen in 4 other reviews |
| sanity2@zfs:test_398o | seen in 20 other reviews |
| sanity2@zfs:test_398s | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 5 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 32 other reviews |
| sanity-flr@zfs:test_44b | seen in 4 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 5 other reviews |
| sanity-hsm@zfs:test_607b | seen in 5 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 clio: add cdp_bytes This is one of several things needed to do prep_transfer_page in the DIO path, so add it. Test-Parameters: fortestonly Test-Parameters: forjanitoronly Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I622b43ba0cee5fb3124fc30dd177f47df3aef3c0
| unique failing test | history |
|---|---|
| sanity-quota@zfs:test_1b | seen in 38 other reviews |
LU-13814 clio: add cdp_osc_tpages array This adds the OSC transfer pages array to the cl_dio_pages struct, which will soon replace cl_pages for DIO. Test-Parameters: trivial Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2986bfc1b54b3e9e6c2e5517f130ace8ff2e4ef2
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56x | seen in 30 other reviews |
| sanity1@ldiskfs+DNE:test_56xB | seen in 1 other review |
| sanity1@ldiskfs+DNE:test_56xa | seen in 30 other reviews |
| sanity1@ldiskfs+DNE:test_56xab | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_56xc | seen in 31 other reviews |
| sanity1@ldiskfs+DNE:test_56ej | seen in 3 other reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 6 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 3 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 6 other reviews |
| sanity2@zfs:test_119e | seen in 15 other reviews |
| sanity2@zfs:test_119f | seen in 14 other reviews |
| sanity2@zfs:test_119g | seen in 14 other reviews |
| sanity2@zfs:test_119h | seen in 14 other reviews |
| sanity2@zfs:test_119p | seen in 3 other reviews |
| sanity2@zfs:test_119q | seen in 3 other reviews |
| sanity2@zfs:test_398o | seen in 19 other reviews |
| sanity2@zfs:test_398s | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 3 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 4 other reviews |
| sanity-hsm@zfs:test_607b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 osc: move from/to to otp prep Everything in the otp needs to be init inside the OTP preparation function, so move from and to. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Id18a7ddbaf1c2f46b32b7d927df93792126ef5b3
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 6 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 3 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 6 other reviews |
| sanity2@zfs:test_119e | seen in 15 other reviews |
| sanity2@zfs:test_119f | seen in 14 other reviews |
| sanity2@zfs:test_119g | seen in 14 other reviews |
| sanity2@zfs:test_119h | seen in 14 other reviews |
| sanity2@zfs:test_119p | seen in 3 other reviews |
| sanity2@zfs:test_119q | seen in 3 other reviews |
| sanity2@zfs:test_398o | seen in 19 other reviews |
| sanity2@zfs:test_398s | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 3 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 4 other reviews |
| sanity-hsm@zfs:test_607b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 osc: move srvlock to prep otp Everything in the otp needs to be init inside the OTP preparation function, so move srvlock. Test-Parameters: fortestonly Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I1940101c9e21e345d20d93820951fe26eab95cf6
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 6 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 3 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 6 other reviews |
| sanity2@zfs:test_119e | seen in 15 other reviews |
| sanity2@zfs:test_119f | seen in 14 other reviews |
| sanity2@zfs:test_119g | seen in 14 other reviews |
| sanity2@zfs:test_119h | seen in 14 other reviews |
| sanity2@zfs:test_119p | seen in 3 other reviews |
| sanity2@zfs:test_119q | seen in 3 other reviews |
| sanity2@zfs:test_398o | seen in 19 other reviews |
| sanity2@zfs:test_398s | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 29 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 3 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 31 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 4 other reviews |
| sanity-hsm@zfs:test_607b | seen in 4 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 osc: rename osc_prep_transfer_page osc_transfer_page_init is a better match for osc_page_init. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id12eec932eb4aab00139f08532b218c12e6ffba4
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_119e | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119f | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119g | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119h | seen in 12 other reviews |
| sanity2@ldiskfs+DNE:test_119p | seen in 2 other reviews |
| sanity2@ldiskfs+DNE:test_119q | seen in 2 other reviews |
| sanity2@ldiskfs+DNE:test_398o | seen in 18 other reviews |
| sanity2@ldiskfs+DNE:test_398s | seen in 2 other reviews |
| sanity1@zfs:test_56x | seen in 38 other reviews |
| sanity1@zfs:test_56xB | seen in 5 other reviews |
| sanity1@zfs:test_56xa | seen in 38 other reviews |
| sanity1@zfs:test_56xab | seen in 2 other reviews |
| sanity1@zfs:test_56xc | seen in 34 other reviews |
| sanity1@zfs:test_56ej | seen in 5 other reviews |
| sanity2@zfs:test_119e | seen in 14 other reviews |
| sanity2@zfs:test_119f | seen in 13 other reviews |
| sanity2@zfs:test_119g | seen in 13 other reviews |
| sanity2@zfs:test_119h | seen in 13 other reviews |
| sanity2@zfs:test_119p | seen in 2 other reviews |
| sanity2@zfs:test_119q | seen in 2 other reviews |
| sanity2@zfs:test_398o | seen in 18 other reviews |
| sanity2@zfs:test_398s | seen in 2 other reviews |
| sanity-flr@ldiskfs+DNE:test_0g | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_0h | seen in 33 other reviews |
| sanity-flr@ldiskfs+DNE:test_0j | seen in 32 other reviews |
| sanity-flr@ldiskfs+DNE:test_36d | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_37 | seen in 31 other reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 25 other reviews |
| sanity-flr@ldiskfs+DNE:test_44b | seen in 3 other reviews |
| sanity-flr@ldiskfs+DNE:test_50a | seen in 35 other reviews |
| sanity-flr@ldiskfs+DNE:test_50b | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_50d | seen in 34 other reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 38 other reviews |
| sanity-flr@ldiskfs+DNE:test_61c | seen in 29 other reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 70 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 30 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 28 other reviews |
| sanity-flr@zfs:test_0g | seen in 32 other reviews |
| sanity-flr@zfs:test_0h | seen in 34 other reviews |
| sanity-flr@zfs:test_0j | seen in 33 other reviews |
| sanity-flr@zfs:test_36d | seen in 31 other reviews |
| sanity-flr@zfs:test_37 | seen in 35 other reviews |
| sanity-flr@zfs:test_38 | seen in 31 other reviews |
| sanity-flr@zfs:test_44b | seen in 2 other reviews |
| sanity-flr@zfs:test_61a | seen in 39 other reviews |
| sanity-flr@zfs:test_61c | seen in 32 other reviews |
| sanity-flr@zfs:test_70a | seen in 59 other reviews |
| sanity-flr@zfs:test_200a | seen in 30 other reviews |
| sanity-flr@zfs:test_200b | seen in 32 other reviews |
| sanity-hsm@ldiskfs+DNE:test_607b | seen in 3 other reviews |
| sanity-hsm@zfs:test_607b | seen in 3 other reviews |
| sanityn@ldiskfs+DNE:test_16j | seen in 27 other reviews |
| sanityn@zfs:test_16j | seen in 30 other reviews |
LU-13814 llite: fix RDMA only check for DIO pages We need to add the check for RDMA only pages and unaligned DIO for DIO pages as we move to them. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I987fecac7874b933a1b558ef442c3ef17120740a
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 26 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 26 other reviews |
| runtests@zfs:test_1 | seen in 20 other reviews |
LU-17885 osc: remove otp from extent_finish + Remove otp usage in osc_extent_finish and osc_flush_async_page. Slow progress. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I9a3a0fe525c93ca264fd60b9597f8ae76ec53398
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 25 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 26 other reviews |
| runtests@zfs:test_1 | seen in 19 other reviews |
LU-17885 osc: remove otp from osc_queue_async_io Remove otp usage in osc_queue_async_io Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I527228cc69b9a976c40cafc43e4ead322106df3f
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 21 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 24 other reviews |
| runtests@zfs:test_1 | seen in 15 other reviews |
LU-17885 osc: remove otp_page_off and brw_page2otp Remove simple functions to simplify otp->brw_page change Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Ie72724db96ae4e8ed7924952ca8652d426d7926b
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 23 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 24 other reviews |
| runtests@zfs:test_1 | seen in 17 other reviews |
LU-17885 osc: remove otp2osc Remove these simple functions to make transition to brw page easier. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Icef7e9998ed7261c0a0d3963684915c6d7d2bc26
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 24 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 24 other reviews |
| runtests@zfs:test_1 | seen in 18 other reviews |
LU-17885 osc: remove otp_index and otp2osc_page Remove these simple functions to make transition to brw page easier. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I7ab38fe8e1797fdfc783f9437d35bd36e54942d8
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 22 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 24 other reviews |
| runtests@zfs:test_1 | seen in 16 other reviews |
LU-17885 osc: remove otp2cl_page Remove these simple functions to make transition to brw page easier. Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Iad78799a010ee818c802326c2c40958de98daadd
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 20 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 21 other reviews |
| runtests@zfs:test_1 | seen in 14 other reviews |
LU-17885 osc: remove otp_count macro Remove macro to simplify otp->brw_page change Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I17f41f7e49fb502d21c30727945b62532887bd68
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 19 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 21 other reviews |
| runtests@zfs:test_1 | seen in 13 other reviews |
LU-17885 osc: remove otp_obj_off macro Remove macro to simplify otp->brw_page change Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Id08e579da486c8d727d38f401d234ded0d66efa7
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 21 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 21 other reviews |
| runtests@zfs:test_1 | seen in 15 other reviews |
LU-17885 osc: remove otp_page and otp_flags macros Remove macros to simplify otp->brw_page change Test-Parameters: trivial Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I500c1d37433a9b69eb1568caccf729a47d0a2d79
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 19 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 20 other reviews |
| runtests@zfs:test_1 | seen in 13 other reviews |
LU-17885 osc: remove otp flags Remove usage of otp flags, move entirely in to BRW page. And now OTP is just another name for a brw page. The next patch is clear enough. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: Ic90118cbe622b8d777b5f162d0b8784892db1495
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 21 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 21 other reviews |
| runtests@zfs:test_1 | seen in 15 other reviews |
LU-17885 osc: begin move to brw_page only The osc_transfer_page is now a thin wrapper around the BRW page, so let's work on removing it entirely. This patch pushes cp_type in to the brw page flags, and preps the rest of the OTP flags in the BRW flags. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I045dc064a44ee098bcfc6c9b2dd4f627a03a32d8
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_119s | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_119t | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_119u | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_119s | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_119t | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_119u | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-subtest-change failed 3× | RHEL 9.3 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.9 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-subtest-change failed 3× | RHEL 8.9 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64, RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-17473 tests: add racing tests of aio This patch adds several racing tests for aio. This has been separated from the other patches in the unaligned DIO series because the aio issue uncovered by these tests is pre-existing and I want to separate solving it from the unaligned DIO patches. Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: I3571a9a620299137624318e503ab901470f97823
| unique failing test | history |
|---|---|
| replay-vbr@zfs:test_5b | seen in 9 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3/x86_64 | ran 6 tests. 2 tests failed: sanity-pfl, sanity. | session |
| review-dne-part-2 crashed | RHEL 9.3/x86_64 | ran 11 tests. 3 tests failed: sanity-sec, sanity-lfsck, replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-3 | RHEL 9.3/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 crashed | RHEL 9.3/x86_64 | ran 11 tests. 5 tests failed: sanity-quota, sanity-hsm, sanity-flr, sanity-dom, replay-ost-single. %% THIS TE | session |
| review-dne-part-5 | RHEL 9.3/x86_64 | ran 10 tests. 3 tests failed: sanityn, sanity-scrub, recovery-small. | session |
| review-dne-part-6 | RHEL 9.3/x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. | session |
| review-dne-part-7 | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-part-8 crashed | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.9/x86_64 | ran 9 tests. 3 tests failed: sanity-selinux, sanity-sec, recovery-small. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.5/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-zfs crashed | RHEL 8.9/x86_64 | ran 16 tests. 5 tests failed: sanity-lsnapshot, replay-ost-single, replay-single, sanity-flr, sanity-quota. % | session |
LU-17831 osc: batch discard for read locks POC patch Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: If22f7d535eef620bbf01c7d738d447e304ca7ad6
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64, RHEL 9.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.9/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-13802 tests: hybrid IO consistency test Hybrid IO is an IO path change, and we should make sure it produces consistent data. Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I926e7cf23c61148b86b9492ed07138ab9d09a103
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.9/x86_64 | ran 4 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64, RHEL 9.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.9/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-13802 llite: add ZFS check for hybrid IO Because by default ZFS only does one DIO operation per commit interval and commit intervals are in seconds, ZFS performance for DIO is extremely poor. This means we should basically never do hybrid IO switching when using ZFS. Implement this. Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I633799cd080e4f8bbab758c972de592d7ff28725
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 12 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 12 other reviews |
| runtests@zfs:test_1 | seen in 5 other reviews |
LU-17433 llite: move dio range locking to DIO For async DIO writes, we need to do the range unlocking after the IO has completed, not before, otherwise we could get write reordering. So we move the unlocking to the cl_dio_aio_end. Also rename lli_write_tree, because it's not just used for writes. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I8ed0f9e41089f82260ec33e47637b785242240dc
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 12 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 12 other reviews |
| runtests@zfs:test_1 | seen in 5 other reviews |
LU-17433 osc: add dlmlock handling to DIO In order to safely do parallel DIO with dlmlocks, we need to take and put a reference on the dlmlock used for each DIO. With the new cl_dio_pages init and free code, this is straightforward. With this change, we can allow parallel DIO even when using dlmlocks. This will also allow async hybrid writes. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I27b2d57e90178e0b9bbfd3942eec02ec74d3f022
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 12 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 12 other reviews |
| runtests@zfs:test_1 | seen in 5 other reviews |
LU-17433 llite: implement async hybrid writes This patch implements async hybrid writes. This hugely boosts the performance of hybrid writes at smaller sizes. For example, on my local VM system, 2M hybrid writes normally go at 230 MiB/s (roughly the speed of storage), and buffered writes at 1.4 GiB/s. With this patch, 2M hybrid writes go at ~6.8 GiB/s. Note this does not include parallel data copies or page pool usage, which are also in flight and should increase this substantially. This should cause us to re-evaluate when we do hybrid IO vs regular buffered writes, since hybrid will now be faster in all cases except for write sizes less than one page. *However*, currently hybrid will not aggregate async writes, so it will result in a stream of small RPCs to the server. This is in fact something that can be resolved, but for now this problem can be avoided by not changing the switching threshold. Note the current form is based on an incomplete version of hybrid IO and not suitable for landing, but this is just a minor thing which will be resolved when the main hybrid patches are complete. This patch series *does* depend on the DIO simplification series, because of some changes to DIO tracking that code makes. This dependency could be removed at the cost of duplicating a decent chunk of the DIO simplification series, so I'd rather not. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I009bc37c92f391c0304b60fa207d220fa3172fa6
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_56od | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15033 llite: Strengthen miss checking For most of our readahead tests, we can predict precisely the expected number of misses. Different read patterns require different numbers of misses to detect, and behavior like whole file read can also reduce the number of misses - but if we are trying to test a specific readahead pattern, using a different one renders the test invalid. If we are trying to verify our ability to handle a strided pattern but we instead read the whole file, there will be fewer misses, but the test is useless. This means as much as possible we should precisely specify the number of misses in our tests, so we ensure they are testing what we think they are testing. With the various fixes in place to remove random misses and generally tighten things up, this is now practical. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I7a800db1a6a8fdf91f714366e6fe8147b7269656
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101ac | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101i | seen in 3 other reviews |
| sanity2@zfs:test_101l | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 llite: add strict discard checking Now all the tests try to enforce reasonable bounds on the number of discards. This is part of the ongoing effort to tightly characterize readahead in the individual tests, so we know for sure they are testing the right thing. Historically, many readahead tests have passed but have not actually tested the intended readahead behavior, eg, strided RA tests passing when the whole file is read in. This helps make those cases harder. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Iddc23d0228bdb813dda108455134a8afbe17785b
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15274 llite: Loose reverse readahead Some applications read files in reverse, we should support this in readahead. This adds the concept of "loose reverse readahead", where we will match 'loose' reverse reads, and do readahead for them in a similar manner to forward reads. For sequential and 'loose' (semi-sequential), this makes performance roughly the same for forward and reverse readahead. For one case, a 63 MiB backwards mmap read, it improved performance by ~98% - Reducing read time from 226 seconds to 5 seconds. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2da196f441edcacd18d834958bd30413b1675f02
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_101aa | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_101i | seen in 7 other reviews |
| sanity2@ldiskfs+DNE:test_101k | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_101l | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_101n | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15194 llite: Fix page size readahead window The first page of a read does not use the readahead code, so it doesn't learn the actual RPC size data from the OSC layer. But this is used to set the initial readahead window size. That means if the first read is a single page read and the next read triggers readahead, the size of the readahead is MAX_BRW_PAGES - so the readahead window is set to 64 MiB, even if the RPC size is set to 1 MiB or 4 MiB! It's tricky to get this information from the OSC layer before starting readahead, so we just default the starting RPC size to 1 MiB. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I65812d59f25135d769bf6c855de4f8631b458fc4
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 llite: whole file readahead test There is no test for whole file readahead. Add one. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia4426682139753588d6536b6b1afa2e95c896c34
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101i | seen in 4 other reviews |
| sanity2@zfs:test_101k | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101l | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101n | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_123i | seen in 4 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-3 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-ldiskfs crashed | RHEL 8.8/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15100 llite: Add loose read pages tunables Add tunables for loose forward and reverse loose matching. This allows tuning the number of pages which are considered a valid match when doing sequential I/O, for purpose of controlling the window. If pages are in the "loose" range, the window will still grow even if they were not actually matched. This lets readahead handle 'loose forward read' patterns, where it jumps ahead a small(-ish) but random number of pages. These are not strided patterns - because the jumps are random - and they are not 'simple' sequential, but they are forward read patterns which benefit hugely from doing readahead. Because the time to read 1 MiB of data is only a few times the time required to read 4K of data, it makes sense to be aggressive. We only need to hit a few pages per MiB to gain performance with this behavior. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie31f71b4380c8c384107eb5db416be106f088d9f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Miss counting for readahead The miss counting for the readahead tests is much too lax and includes a large fudge factor of +10 misses. This is unnecessary because these tests are deterministic and we should be able to explain and count all of the misses seen. This patch tightens the margins considerably on allowed misses, which should help avoid problems creeping in in the future. There are a few unexplained misses here - those will need to be debugged later, but we should get these tests in first, then fix the remaining strange behavior. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia876b7ed807fa1d101a8fd02bed4db7823d923dc
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Add rpc counting to readahead One of the key goals of readahead is to generate large RPCs regardless of how large the reads from userspace are. We currently don't test this at all, which is not ideal. Add RPC counting to the readahead tests where it applies. Note this patch includes margin for the RPC count being off due to various minor bugs. Fixing these is left for later - they have been present for a long time. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic4b28c936bc0e2b42339f2b00219177f77b8d85f
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Add rpc stats to readahead tests RPC generation behavior is a key component of readahead, and the plan is to add tests of rpc generation to all the readahead tests. In the meantime, we can at least add output of rpc_stats whenever we output read_ahead_stats to make the tests more informative. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If18ed6bb43a5afb2a7b0c9267f2f056cd33c66d7
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Improve test 101a Test 101a is very old and uses some odd methods to control caching. Remove those, and switch to a simple cache flush to ensure data is read from disk. Shrink the test size to make it more consistent (previously it was hitting cache a lot, which made the RPC stats harder to predict). Also add RPC count checking - this will be added to the other readahead tests as well. Test-Parameters: trivial testlist=sanity env=ONLY=101a,ONLY_REPEAT=50 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I39b1ed23e4c080af9e3689d32ac60701c6c5a812
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Add ra after seek test Confirm read ahead can restart cleanly after seeking in a file. Do this by reading the back half and then the front half of the file. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I30c0515c95af0c35fc38ed612f6409c15d55c3a8
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101i | seen in 5 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15033 tests: Stricter readahead tests A few small test tweaks to make particular readahead tests stricter and more broad ranging or improve output. Didn't have a clear home in other patches in the series. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic8164f6cc2db09492574a247705bd55bc1f60989
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_63b | seen in 34 other reviews |
| sanity2@ldiskfs+DNE:test_64a | seen in 34 other reviews |
| sanity2@ldiskfs+DNE:test_64c | seen in 34 other reviews |
| sanity2@ldiskfs+DNE:test_64d | seen in 78 other reviews |
| sanity2@ldiskfs+DNE:test_101i | seen in 6 other reviews |
| sanity2@zfs:test_101i | seen in 2 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-6 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15516 llite: unify readahead logic The mmap readahead logic is *almost* the same as regular readahead logic, but for what appear to be historical or accidental reasons, it is initialized differently and has several special cases as a result. There's no clear need for separate mmap and regular readahead logic, and the existing differences are harmful, as mmap read takes more misses than a regular read of the same pattern. But more importantly, having the special case makes it much harder to write and test improvements to the readahead code, since they must be written carefully to hit both paths and then tested both ways as well. There may be some application for separate tunings, but none has been persuasively shown so far (and none are in place currently). Clean up and unify the logic. We can't unify as much as we'd like, since mmap needs to do ras_enter only after we've locked the page - because if we can't lock the page, mmap doesn't proceed to do IO. This means we must wait to call ras_enter for mmap until we're in the readpage code, unlike for regular file reads, which call it at the llite layer. This is OK for mmap because mmap reads are a single page, but it's important for regular reads to call ras_enter only once per read, not once per page. Note even without other changes this reduces the number of misses taken in the simple mmap read test. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic6c33a2714a256072ef15d56c40f2bd1e39a1e6f
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_101f | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_101i | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_101f | seen in 1 other review |
| sanity2@zfs:test_101i | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15516 llite: Remove clustered read code The clustered read code changes mmap readahead behavior so it ignores patterns, and reads a set of pages around each new read. This works well if you have a random read pattern and are going to use most of the file, but it's bad if there is a pattern, which there almost always is. The result is extremely harmful for basic patterns like 'read a whole file from beginning to end'. It also appears that the described clustered behavior (where a bunch of data is read semi-randomly in an area and then there's a jump to a new area) is not real application behavior, instead it was a stopgap for the inability to support certain common database read patterns, like loose forward and sequential or loose sequential reverse. The clustered code, instead, breaks the ability to properly handle simple patterns like sequential forward read in mmap. It also creates a large section of 'mmap only' readahead logic, which makes it impossible for mmap reads to benefit from most readahead improvements. Having this separate logic path also makes testing and verification of readahead much more difficult. A 'read a cluster of pages during random read' feature is a reasonable one and could be helpful in some scenarios. Parts of this code can serve as a reference for that, but the existing code should be removed. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I15b203bc0692098614b691344d474a226ba3df4a
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_101i | seen in 5 other reviews |
| sanity-pcc@ldiskfs+DNE:test_7b | seen in 10 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-15069 llite: improve ras usage The readahead state is file level and shared, the ria is a per-IO version of that state. Don't access the ras when the ria is sufficient. Also rename one function. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4de4e093d7d8e690b49852f26e2f44ae4474671a
| unique failing test | history |
|---|---|
| sanity-pcc@ldiskfs+DNE:test_5 | seen in 13 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm crashed | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
LU-15033 llite: rework end_idx handling There is a separate ra_end_idx argument passed to ll_read_ahead_pages, which captures the last page read and is then compared to the contents of ra_io_arg, which is also passed to ll_read_ahead_pages. This results in comparisons like: if (ria->ria_end_idx == ra_end_idx) which are a little tricky to understand. Instead, we put this in ra_io_arg and name it ria_last_read_idx. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I774e69c6a6334d2f56a4bc0c85ed00a5f2757e35
| unique failing test | history |
|---|---|
| sanity-sec@zfs:test_21 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-6 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-selinux. | session |
| review-dne-zfs-part-2 crashed | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
LU-16741 ptlrpc: remove unnecessary asserts ptlrpc_free_committed is at no risk of being called with either a null import or the import lock unlocked. These don't have a significant cost, but they're not adding much either. Removed 'trivial' from this patch to ensure the LU-16741 refactoring series gets at least one full run. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I90d3ddae2cfbb9c748e3ba004982bca3ef465667
| unique failing test | history |
|---|---|
| sanity-pcc@ldiskfs+DNE:test_18 | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne failed 2× | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-16741 ptlrpc: rename ptlrpc_free_request ptlrpc_free_request doesn't free requests, it commits them. Rename it accordingly. One nag: there is one call to rq_commit_cb() outside this function, in after_reply(). It is not clear to me how/why that call is different, so I am leaving it untouched. But ideally we would only call rq_commit_cb() from this function. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I52e81076a1d4e55fb83f9d4d6c86df64393004b0
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101ab | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 | RHEL 8.8/x86_64 | ran 11 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.8/x86_64 | ran 11 tests. 1 tests failed: sanity-flr. | session |
| review-dne-zfs-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.8/x86_64 | ran 10 tests. 1 tests failed: sanity-flr. | session |
LU-15155 llite: Make readahead request locks Currently, readahead will not request an LDLM lock if it encounters a region without one. This causes it to take misses and can confuse the readahead state as well. Not requesting locks for readahead is an artifact of the idea that readahead is an optional optimization, but it's almost as important as full reads/writes from userspace, and we should request locks for it. This will help cut down misses when starting to read a new file, which is particularly helpful in tests, where total I/O is small and the extra misses make it hard to predict behavior. However, to give better behavior under conflicting workloads, we make the lock requests from readahead nonblocking. This means it will get a lock if there is no conflicting lock, but otherwise will not. We also limit it to one lock request per stripe per readahead invocation, since otherwise it would ask for every page. The benefit to requesting locks can be seen in the test changes - miss counts are reduced because stripe count no longer factors in, and we can reenable async readahead because it no longer fails due to this (leading to unpredictable numbers of misses). Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie62a282245d036308ab0c6f8392af1098a74befc
| unique failing test | history |
|---|---|
| sanity2@zfs:test_101ab | seen in 2 other reviews |
| sanity2@zfs:test_101n | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-15178 llite: Clarify async vs nowait The existing code mixes the concepts of 'async' lock requests and 'nowait' (nonblocking) lock requests in to one term: 'speculative' lock requests. This prevents us from creating synchronous nonblocking/nowait lock requests, for no good reason. This patch clarifies the code, separating these concepts and allowing synchronous non-blocking lock requests. This is important because it allows readahead to make 'optional' lock requests, where it will wait for the request, but it does not want to cancel other locks. In essence, readahead would like to have the lock for immediate use so it can complete the readahead request, but it would prefer not to cancel other locks. Thus, synchronous non-blocking lock requests. This is implemented for readahead in another patch in this series. It also renames the badly named LDLM_FL_SPECULATIVE flag, because this flag actually implements 'NOWAIT' behavior. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0cbd44fe07e0c0206ba56eaef62590b19ba082eb
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_124c | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity2@ldiskfs+DNE:test_124d | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity2@ldiskfs+DNE:test_134a | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity2@ldiskfs+DNE:test_812a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_812b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_816 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@zfs:test_398a | seen in 3 other reviews |
| sanity2@zfs:test_812a | seen in 1 other review |
| sanity2@zfs:test_812b | seen in 1 other review |
| sanity2@zfs:test_816 | seen in 1 other review |
| sanityn@zfs:test_109 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@zfs:test_113 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.7/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-6 | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.7/x86_64 | ran 5 tests. 2 tests failed: sanity-selinux, sanity-sec. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.7/x86_64 | ran 4 tests. 2 tests failed: sanity-pfl, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-3 crashed | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 3 tests failed: sanity-quota, sanity-hsm, sanity-flr. | session |
| review-dne-zfs-part-5 crashed | RHEL 8.7/x86_64 | ran 8 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-6 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity-lnet. | session |
| review-ldiskfs-ubuntu | RHEL 8.7/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs | RHEL 8.7/x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
RE: rq_commit_cb and imp_lock This snippet of code strongly suggests no. (And this looks to me like it gets called regularly and isn't some weird stub or corner case, so it seemed solid evidence.) I actually considered doing this for the rq_commit_cb in the free_committed code, but you can't easily do so because you're walking the lists on the import. But more to the point, since that could be worked around, I'm skeptical dropping and taking the lock for *every* rq_commit_cb() in free_committed is a good idea. (If the rq_commit_cbs were the large majority of the work, dropping the lock and re-attacking the list in free_committed might make sense, but I don't think they are. It seems more an invitation to thrashing the lock and the memory for the lists between many CPUs.)
I may want to reflect on the code organization here - this seems a bit confusing. Let's see if it's correct first and then I'll try to think about that.
I'm guessing we're sometimes missing calling this section of code (And maybe not calling this whole function), so we're still referencing the import, since I've seen both OSC and OFD modules getting stuck.
stale comment, need to remove
Will probably just undo this to show I'm not changing this area...
(defect?) one thing that concerned me about *only* allowing the "now > start + 3" exit condition is that if the replay list was very long and filled with unfreeable RPCs (a million file opens?) then it seems possible the thread could loop through requests for a few seconds and not find anything, then exit with no RPCs to free. Then the next thread enters this code and does the same thing, since it restarts the scanning at the beginning of the list. So the exit condition (before this change) was: - accumulate at least 128 RPC and have been scanning > 3s - or walk the whole list (until reqs with transno > last_committed are found) without finding 128 freeable RPCs That way, each call here cleans up at least 128 RPCs (if there are that many), or hopefully finishes list walking much sooner. That amortizes the list walking over more RPCs. The main question is how much of the list walking does not produce "useful work"? With the reduction in lock contention, this could always reduce the number of entries per call. I also appreciate that if other CPUs are stuck on a spinlock then they are also burning cycles unproductively, so in theory as long as each pass made *some* forward progress (a handful of RPCs) then it would be OK to exit and leave it to the next lock waiter.
Hmm, so I think the big threat here is if we have an *extremely long* but unfreeable list (except for maybe a few at the far end which we don't reach), we could loop forever as we hand the problem off to a new thread that retraces our steps. Or at least, that is the problem taken to an extreme, where we get no work done so we make no progress. But isn't it the case that the worst case scenario is we essentially end up hung until the *next* commit comes through and that huge pile of RPCs is now freeable? Because they have to be freeable some time. I'm trying to decide how plausible it is as well - If we have, say, 10 million RPCs on the list (being generous) and, we say we've got 1 second, we have 100 nanoseconds (300 nanoseconds per for 3 seconds) for each RPC (to do the list walk + checks). Given that a cold DRAM access can be on the order of 90 ns (thanks, Google), yeah, that's not crazy. So the amounts of time are plausible. But I think we still have the backstop of "the next commit arrives and they become freeable", don't we? So the worst case is that we could get stuck looping (handing from thread to thread without real forward progress) until the next commit arrives. So I think this is OK. Does that wash? Have I missed/misunderstood something?
I _think_ that open-but-committed requests are still kept in this list in order to ensure the open is replayed before any later requests that may unlink the file without preserving the refcount. That said, I've long wanted to divorce open-committed requests from RPC replay so that they don't clog up the replay queue and also so that we don't need to keep exact copies of RPCs in memory for hours/days/weeks, since that complicates ever changing the RPC format over an upgrade. This allows a few different improvements to be implemented: - LU-5703 "Quiesce client mountpoints from the server" - LU-3290 "disallow ptlrpc RPCs with old client XIDs" - LU-15250 "RPC Replay Signature" and also simplifies the RPC handling code, since we don't need to preserve the close RPC replay after the open RPC has committed.
Style: Don't need braces here...
Note this was wrong in the previous patchset, but wasn't affecting behavior.
LU-16741 ptlrpc: defer & parallelize free_committed ptlrpc_free_committed can be extremely time consuming when there are many async requests outstanding, such as with small async DIO as potentially created by LU-13805, or in other unusual circumstances. Most of the work (in terms of time consumed) in ptlrpc_free_committed can be deferred and moved out from under the imp_lock. Additionally, if there is a process waiting to do the ptlrpc_free_committed work, the current thread can grab a 'batch' of requests to process, then drop the import lock and allow the waiting thread to grab the next batch. This splits the work across waiting threads. In cases where the lock is highly contended, it has the effect of having each thread do one 'batch' of work, but mostly in parallel. This has the effect of parallelizing this work and speeds it up enormously when there are many threads (the most important case, as they can generate a lot of work). If there are only a small number of threads working on the import, the load can be split unevenly - Consider the case of two threads. The first to arrive defers one batch of requests, notices the second thread is waiting, and leaves all the remaining requests for that thread. This is uneven, but doesn't really matter for a few reasons: 1. In the 'small number of threads' case, there isn't much work to do anyway, so it's not very impactful if it is unevenly distributed. 2. If it really takes a long time, the first thread will arrive again as a 'waiter', and will pick up work. A good example is this IO500 ior-hard-write test with async DIO; this is a bit of an extreme example, and this also helps IO500 hard write without async DIO, but that hits other limitations so it is harder to see the benefit. mpirun -n 80 ior -k -e -o $file -t 47008 -b 47008 -s 11990 -w -a POSIX --posix.odirect Performance without the patch: 2018.19 MiB/s Improves to: 4800 MiB/s with this patch (This was done with fake_io to reduce server side limits and better show the contention.) And the time spent in the imp spinlock drops from 40-50% of all client CPU time to ~0% (imp_lock spinning no longer shows up in the perf traces). Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5e9b8d556770dc4a33dce0ceb50f745201328c5e
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| review-dne-part-1 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: sanity-pfl, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-2 crashed | RHEL 8.7/x86_64 | ran 13 tests. 4 tests failed: sanity-sec, sanity-lfsck, runtests, replay-dual. %% THIS TEST SESSION CRASHED % | session |
| review-dne-part-3 crashed | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 8.7/x86_64 | ran 11 tests. 2 tests failed: sanity-quota, sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 crashed | RHEL 8.7/x86_64 | ran 10 tests. 3 tests failed: sanityn, sanity-scrub, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-6 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-7 crashed | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-2 crashed | RHEL 8.7/x86_64 | ran 7 tests. 2 tests failed: sanity-selinux, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-1 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: sanity-pfl, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-2 crashed | RHEL 8.7/x86_64 | ran 13 tests. 4 tests failed: sanity-sec, sanity-lfsck, runtests, replay-dual. %% THIS TEST SESSION CRASHED % | session |
| review-dne-zfs-part-3 crashed | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-5 crashed | RHEL 8.7/x86_64 | ran 10 tests. 3 tests failed: sanityn, sanity-scrub, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-6 crashed | RHEL 8.7/x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-7 crashed | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs | RHEL 8.7/x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
| review-ldiskfs-arm crashed | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 8 tests. 2 tests failed: sanity-sec, sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu | RHEL 8.7/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs crashed | RHEL 8.7/x86_64 | ran 12 tests. 2 tests failed: replay-single, sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
LU-16741 ptlrpc: check logic in osp_request_commit_cb osp_request_commit_cb code states that rq_commit_cb can be called on uncommitted requests. That seems wrong, in both the specific sense of "I doubt we do that" and the sense that "we should not do that". Note: This is NOT for landing; this is just exploratory. Test-Parameters: testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I09a20286bb3fde9a77396cebfce0028fd996bfa1
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
LU-16741 ptlrpc: add 'locked' to rq_commit_cb osp_request_commit_cb is unique among rq_commit_cbs in requiring the import lock be held. Add a parameter so we know when we need to take the lock in that callback. Note: This is NOT for landing; if this works it will be integrated in to the parent change. Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6c1b681eafeb2f805b41dc5654ab2e2c6e020f73
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
LU-16741 tests: Test locking around commit_cb Seeing if taking imp_lock() around commit_cb when doing the deferred work avoids the crash. Just for learning purposes - it's possible one of the commit callbacks has a hidden dependence on the import lock. Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie098251bb3ec78325bc813e7dd6946d503582894
| unique failing test | history |
|---|---|
| sanity2@zfs:test_398a | seen in 3 other reviews |
| sanity2@zfs:test_812a | seen in 1 other review |
| sanity2@zfs:test_812b | seen in 1 other review |
| sanity2@zfs:test_816 | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| review-ldiskfs-dne-arm | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
LU-16741 tests: Test to confirm replay-dual hits This is testing LU-16741 with no changes to confirm the specified replay-dual testing hits the bug reliably. Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ibc1a46df653cb9ce98b99ee3e5aef7705bfca111
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-101 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-102 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-103 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| custom-104 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: replay-dual. | session |
| review-ldiskfs-dne-arm | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 5 tests. 2 tests failed: sanity-lnet, sanity. | session |
LU-16741 tests: Test locking around deferred work Testing to see if hold imp_lock() here avoids the crash. Just for learning purposes - this is NOT a proposed change since it would remove all benefit from the patch. Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Test-Parameters: trivial testlist=replay-dual env=ONLY=26,ONLY_REPEAT=10 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0fdc1191a6e39f85715b5a1ae676dc916e70c8aa
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-ubuntu retesting | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
That seems like a pretty big hammer, basically breaking AIO completely for EC files, rather than just the recovery path? Or is the comment wrong and this is triggered only for EC recovery when `-EIOCBQUEUED` is returned? Would it be better to push **all** EC recovery to a workqueue instead of keeping it directly in the IO path, then it can fire the AIO completion when the reconstruction is complete, and synchronous readers would wait on the completion?
Andreas, yes, correct. This is essentially making AIO synchronous for all reads on EC files because we can't tell at submission time if we need reconstruction. So, this patch is just a stopgap to allow AIO to work for degraded reads, albeit with collateral damage. I'd agree that a workqueue is the better long-term approach, however, I'm not familiar enough with the code path at the minute here to gauge the effort required. So, I'd keep this patch as a short-term fix for AIO for degraded reads, and open a ticket so we address this properly with a workqueue. Do you agree? If yes, I open the ticket
(minor) aio_ec_sync deliberately survives the goto restart at the end of the function, and it has to. cda_no_aio_complete and cda_creator_free stay set on the cl_dio_aio across restarts, so a later pass that ends with anything other than -EIOCBQUEUED (cl_io_rw_init() failing, or cl_io_loop() returning a hard error) still needs both !is_aio || aio_ec_sync tests to be true. If it were cleared at restart, __cl_sync_io_note() would skip the free because creator_free is 1, and the creator branch would skip it too, leaking the cl_dio_aio and its cda_obj reference. This isn't a bug as written, but the coupling between a stack bool and two flags on a heap struct isn't obvious. Could the comment above say the flag is intentionally sticky, so it doesn't get reset in a later cleanup?
on refresh
LU-12669 llite: make AIO reads on EC file synchronous For sync DIO, EC recovery on read failure runs inline in ll_file_io_generic after cl_sync_io_wait_recycle. AIO has the same recovery requirement but cannot run recovery from the sub-DIO completion path -- ll_file_io_generic returns -EIOCBQUEUED to the VFS before the BRW completion fires, so the syscall context recovery needs (the lu_env, the user's iov_iter, the range lock) is gone by the time the read error is known. Rather than building a workqueue-based async recovery path, make AIO reads on EC files behave as sync DIO: after cl_io_loop, if an AIO read on an EC layout (io->ci_cross_ec, set by lov_io_mirror_init during cl_io_rw_init) got -EIOCBQUEUED, set cda_no_aio_complete and cda_creator_free to disarm the async ki_complete path, then drop into the same wait + restart code sync DIO uses. The VFS calls ki_complete itself when we return. The flag flip is race-free because the submission ref on cda_sync is still held -- end_io cannot fire until we drop it in cl_sync_io_wait_recycle. This disables async semantics for every AIO read on an EC file, not only those that need recovery -- we cannot tell at submission time whether reconstruction will be needed, so the sync conversion fires unconditionally on -EIOCBQUEUED for an EC layout. As a side effect, the range_lock taken by ll_file_io_generic is now held until all sub-DIOs have drained, instead of being dropped while they are still in flight. Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Assisted-by: Opus:4.8 llm_code_and_review_tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Marc Vef <mvef@whamcloud.com> Change-Id: I394a3610e33b29ead8f5adb52dfa21db6b721944
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_63c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
(suggestion) This is the last DIO assumption left in lov_io_submit(), and the patch's own premise says it can no longer hold: every page that reaches here now comes from a cl_page_list, and the only CPT_TRANSIENT pages in the tree are built by ll_direct_rw_pages() into a cl_dio_pages, which goes to lov_dio_submit() instead. So an empty page arriving here would be CPT_CACHEABLE and this would LBUG.
In practice the branch looks unreachable too - lov_page_init_empty() does SetPageUptodate(), and all four submit paths (ll_readpage(), ll_read_ahead_page(), ll_prepare_partial_page(), ll_io_zero_page()) skip uptodate pages before queueing. If so the whole block is dead and could go with the rest of the DIO leftovers.
The comment is stale either way: cl_page_prep() here was removed by 169f076ae0 ("LU-13814 clio: remove cl_page_prep for transients").
(minor) Not a bug, but with `dio` dropped the only remaining value in this message is `ext->oe_srvlock`, and the enclosing condition is `dlmlock == NULL && !ext->oe_srvlock` - so it always prints `srvlock: 0`. If the patch is refreshed, either drop that field or print something that varies, e.g. `oe_rw`.
LU-19536 osc: remove dead DIO handling from sync path Since LU-13814 routed DIO through cio_dio_submit() and cl_dio_pages, transient DIO pages no longer reach lov_io_submit(), osc_io_submit(), or osc_queue_sync_pages(). The DIO-specific branches and setup left in the ordinary page-list path are therefore dead. Remove the stale DIO handling from the LOV and OSC submit paths, including OBD_BRW_NOCACHE setup, the oe_dio assignment, and the sync_io anchor setup. The ordinary LOV path now always groups pages by stripe, while osc_io_submit() always prepares its pages. OBD_BRW_NOCACHE is set only by osc_dio_submit() before it calls osc_queue_dio_pages(). Since oe_dio is always false in the sync path, simplify extent priority handling and omit oe_dio from the NOLCK diagnostic. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com> Change-Id: I63711a7f3bc3699680e1c09d3349291ccebb75ec
This paragraph describes a change that does not seem to be in the diff. lustre/lov/lov_page.c isn't touched, and lov_page_init_composite() has no append exception either before or after this commit:
stripe_cached = lio->lis_cached_entry != LIS_CACHE_ENTRY_NONE &&
page->cp_type == CPT_TRANSIENT;
Was this paragraph left over from an earlier version of the patch?
This fixes wrong-stripe routing and a wrong file size, so a Fixes: tag would help decide which maintenance branches need it. The "DIO is already split by stripe" assumption that this patch repairs was introduced by:
Fixes: d31647c017a3 ("LU-13799 lov: Improve DIO submit")
That commit made lov_io_submit() splice every DIO page onto the stripe of the first page, and 14db1faa0fbe ("LU-13799 lov: Cache stripe offset calculation") then extended the same assumption to the per-page stripe cache. The assumption never held for the append path.
The clamp to the component extent lives inside this branch, so nothing bounds the batch when the component at `start` has a single stripe. For a layout like `-E 1M -c 1 -E -1 -c 4`, a 4MB DIO append from offset 0 gets no split at all and the batch crosses the component boundary; lov_page_init_composite() then hits
if (!lov_io_layout_at_confirm(lio, entry, offset))
return -ENODATA;
for the pages past 1M and the write fails.
The non-append path below clamps unconditionally - it starts at MAX_LFS_FILESIZE and applies min(next, lse->lsme_extent.e_end) outside the stripe_count test. Should this do the same, so the commit message's "matching what the non-append path already does" actually holds?
lio->lis_io_endpos is an absolute file offset set once in lov_io_init():
lio->lis_io_endpos = crw_pos + crw_bytes;
For append that crw_pos is only the estimate taken in ll_file_io_generic(); the offset the write actually lands at is chosen later by vvp_io_write_start() from i_size_read(). After the first iteration `start` tracks that real position, so if the file grew in the meantime `start` can run past lis_io_endpos.
Two consequences: the loop stops once next reaches lis_io_endpos even though bytes remain in the iov_iter (short write from an O_APPEND write(), where before the patch the whole count went out in one iteration), and on the next line `next - start` goes negative into the size_t crw_bytes.
Should the append path clamp against the number of bytes still outstanding rather than an absolute endpos, and guard next >= start?
Setting ci_continue here turns a single append iteration into several, and cl_io_loop() drops the lock between them:
cl_io_lock() -> cl_io_start() -> cl_io_end() -> cl_io_unlock()
vvp_io_write_lock() takes [0, wr_append_lockpos] PW for append precisely so the whole write lands atomically at EOF, and vvp_io_write_start() re-reads i_size_read() under that lock on every iteration. With the split, another client can revoke the lock and append between our iterations, so a single 4MB O_APPEND write() can end up with a foreign client's data interleaved in the middle of it.
Is there a way to keep the append as one locked iteration and instead bound the cl_dio_pages batch itself (in ll_direct_IO()/cl_dio_pages_init()) at the stripe boundary?
This contradicts the assertion 14 lines above, which says the opposite about the same queue:
/* it could only be mirror read to get here therefore
* the pages will be transient. */
LASSERT(page->cp_type == CPT_TRANSIENT);
If transient pages genuinely never reach lov_io_submit(), that earlier LASSERT can never be satisfied and would LBUG the client on any buffered read that lands on an uninstantiated component. Worth resolving one way or the other while this function is being touched.
I read that `drop_caches` is not considered reliable. It might be better to use `cancel_lru_locks osc`.
Andreas, out of interest, do you have some more info on this as to why?
There is no OST count precondition, so on a single-OST setup `-c 2` silently yields one stripe and the test passes without exercising anything. Consider:
(( OSTCOUNT >= 2 )) || skip_env "needs at least 2 OSTs"
Also, the file is empty here, so the append starts at offset 0 and is already stripe aligned. Appending to a file that is not a multiple of the stripe size (say seed it with 512KB first) is the case where the split has to get the first partial stripe right.
Both md5sums read the same OST data - the first one populates the page cache from the OSTs after the DIO write - so this compares the file against itself rather than against what was written. Writing from a temp file (or `dd`ing a known pattern) and comparing that checksum would actually catch mis-routed data; right now only the size check can fail. On a related note, the earlier suggestion on this hunk to use `cancel_lru_locks osc` instead of drop_caches doesn't appear to have been picked up in the current revision.
LU-19900 lov: fix DIO with O_APPEND stripe routing When a file is opened with O_APPEND and written via direct IO, all data is incorrectly routed to stripe 0 instead of being distributed across stripes. The root cause is that lov_io_rw_iter_init() does not split append writes at stripe boundaries. Each DIO iteration processes a cl_dio_pages batch, which lov_dio_pages_init_composite() assigns to a single stripe. Without splitting, the entire write goes to one stripe, causing incorrect file sizes (e.g. 7MB instead of 4MB for a 4MB write to a 2-stripe file) and data corruption. Fix by adding stripe boundary splitting to the append path of lov_io_rw_iter_init(), matching what the non-append path already does. Each iteration now covers at most one stripe_size worth of data, so lov_dio_pages_init_composite() correctly assigns each batch to a single stripe. Also add an LASSERT in lov_io_submit() confirming that DIO pages (CPT_TRANSIENT) never reach that path - they use lov_dio_submit() instead. Also revert the append exception for the DIO stripe cache in lov_page_init_composite() since it is no longer needed now that iterations are single-stripe. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I66304a6692eaab9c68a03159e51cf3d27c465a83
| unique failing test | history |
|---|---|
| sanity1@zfs:test_56ab | seen in 5 other reviews |
this 88 of 500 sounds strange to me.. is it still failing?
I guess the intent is that the patch passed 5x as many iterations as without the patch, so it is improving the situation to some extent.
if this was the real problem introducing commit, it should be Fixes: header below.
The pieces of this patch don't add up. While I understand it improves the situation, now that that was demonstrated, it's time for a real human to look into why and make a patch that actually makes sense.
is this the actual weight bearing "fix" of this patch?
this comment seems wrong. We can never get here via unevict-clear (osc_unevict_cache_shrink call I guess?) because it sets reason to SK_REASON_UNEVICT_LRU, but that case is already handled above. That leaves a call for normal reclaim from osc_lru_reclaim?
I am not sure how this statement makes any sense?
LU-19487 osc: fix shrinker loop in osc_lru_list_shrink
Commit 109e32dc23 ("LU-19223 osc: stop after scanning")
changed the scan-limit tracking in osc_lru_list_shrink()
from a countdown to a count-up, and added a
--pages_scanned adjustment in the cl_object-switch path
to avoid counting object-switch iterations as scans.
When LRU pages belong to many different cl_objects, the
decrement causes pages_scanned to stagnate near zero,
making the loop run far longer than max_pages_to_scan
allows. The kernel's do_shrink_slab then re-calls the
shrinker because nr_scanned stays low, creating a
CPU-bound infinite loop that hangs the system.
Remove the --pages_scanned adjustment. The cl_object
switch does real work (dropping the spinlock, calling
cl_io_init) and should count toward the scan limit.
For forced scans (unevict-clear, cache-limit reduction,
etc.), increase the scan budget to target<<1 without
capping at lru_in_list so that object-switch overhead
does not prevent the shrinker from reaching its target.
Reproduced on a single-node test setup: sanityn test_16g
hung at iteration 17 of 100 without the fix (system
required hard reboot), passed 88 of 500 iterations
cleanly with the fix applied.
Generated with Claude Code + Tools
Test-Parameters: testlist=sanityn env=ONLY=16g,ONLY_REPEAT=10
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I7c007a1c021067e0b166b00045223ad8a927b5aa
This is pretty similar to `ll_filemap_get_folios`, but I guess with all the #define involved it is not easy to factorize. Maybe introduce `ll_split_folio` to do the split loop (and which would be a no-op in case folio_order is not defined).
LU-20069 osc: pass page index explicitly for DIO encrypt DIO encrypted file corruption during migration. The encrypt path in osc_brw_prep_request() writes cp_page_index into page_folio(page)->index, then osc_encrypt_pagecache_blocks() reads it back via folio->index + page offset. For DIO pages that are part of compound pages (order > 0), the page offset within the folio is added to the index, producing a wrong AES-XTS tweak. The decrypt path correctly uses cp_page_index directly. Compound pages can appear for DIO allocations on any kernel -- reproduced on RHEL 9.6 and SLES 15.6. Fix: pass cp_page_index explicitly to osc_encrypt_pagecache_blocks() for DIO, mirroring the decrypt path. Eliminates the fragile round-trip through folio->index that breaks for compound pages. Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Iba44faefa61a7da6377e9d906b0453ef4ded5d13
LU-16624 tests: Add log scan Add a dmesg log scan as a final test to catch warnings that may have occured in the logs. Let's see how this works and get some feedback, then I'll look at adding it to other test sets. Test-parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I1b1e967b515a737c54702a7048429db79746d632
LU-11962 mdt: lazy as strict som Tiny test for lazy-as-strict SOM. Test-Parameters: forbuildonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Sohei Koyama <skoyama@ddn.com> Change-Id: I3ce247c7d255512c979fbb5c5286e56a4c218c45
LU-15367 llite: Add iotrace debug logs for delete operations Add iotrace debug logs to track delete operations in ll_unlink()/ll_rmdir(). Also drop the dead dchild->d_inode NULL check in ll_rmdir(): ll_rmdir() is only reachable as .rmdir, and the VFS rejects negative dentries before calling ->rmdir. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Xiao Yang <xyang@ddn.com> Change-Id: I30ef291a63d23ed1051f288b955ef6a81eb073de
LU-15367 llite: Add iotrace debug logs for link operations Add iotrace debug logs to track link operations in ll_link()/ll_symlink(). A long symlink target may overflow the debug page and get printed to the console unthrottled, so cap the target in the traces. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Xiao Yang <xyang@ddn.com> Change-Id: Ifc252381b81472510b2fa523dc7d482cb989bc76
| unique failing test | history |
|---|---|
| replay-single1@ldiskfs+DNE:test_80c | seen in 6 other reviews |
LU-15367 llite: Add iotrace debug logs for rename operations Add iotrace debug logs to track rename operations in ll_rename(). Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Xiao Yang <xyang@ddn.com> Change-Id: I54ce741aafa802384cfe81979adea851c55e13f3
LU-15367 llite: Add iotrace debug logs for lock operations Add iotrace debug logs to track lock operations in ll_file_flock(). Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Xiao Yang <xyang@ddn.com> Change-Id: I110d4051df9e534410809a79353520c2e91e7491
LU-19939 llite: reorder tiny write before PCC/hybrid Move the tiny write attempt before PCC and hybrid DIO checks in do_file_write_iter. When a tiny write succeeds (page already dirty in cache), the function returns immediately without touching PCC, hybrid switch, or the CLIO path. Replace ktime_get() with ktime_get_coarse() (same approach as companion read patch LU-19344) to avoid the expensive hardware clocksource read. On KVM guests, pvclock_clocksource_read was the #1 CPU consumer at 9.16% in the tiny write profile. Virtualization is not a niche scenario - all major cloud providers run KVM or similar hypervisors. ktime_get_coarse() reads a cached jiffies-granularity timestamp (~1-4ms resolution), reducing clock overhead from 9.16% to 0.54% of CPU. The coarse granularity is sufficient for the aggregate min/max/sum/count stats - keeping stats on all I/O paths is important for observability, so we pay the remaining stats cost rather than skipping collection on fast paths. perf profile with ktime_get_coarse (KVM, pvclock): pvclock_clocksource_read: 0.02% (from other callers) ktime_get_coarse_ts64: 0.54% (was: pvclock 7.32% + 1.84% = 9.16%) Benchmark (8-byte sequential writes, 2M iterations): Before (ktime_get): ~1,970k writes/sec After (ktime_get_coarse): ~2,179k writes/sec (+11%) Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ib73b36a9c588bb6ac1a78f77b49bef160af4ee76
LU-0000 doc: man page cleanups This is a large collection of minor man page cleanups, covering missing options, incorrectly specified APIs, and many more minor issues. This code was generated by Augment. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I9a3269b30945e2821819ef8e03998d856a27a5ed
LU-13814 osc: simplify osc_dio_page_submit Remove more code from osc_dio_page_submit, with the goal of eliminating it entirely. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Id408aad01c2e81a126dd04807819aed9d29fca6b
LU-13814 osc: add osc_dio_completion Specialize osc_completion to add the osc_dio_completion version. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4d202165ee6c4f50338b6a9381aa476b969d3593
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-part-8 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lnet. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-13814 osc: add osc_dio_page_submit This patch adds the osc_dio_page_submit and specializes osc_page_submit to take only BIO pages. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I048bbcd25ffee28fbbd8fee26017796b5e3586bd
LU-0000 lov: initial client support Initial client support for EC layouts. A step towards allowing actually creating EC layouts. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I0e2a5d59df04ba6f7053d889cb5f204048edf8d7
| unique failing test | history |
|---|---|
| sanity-lfsck@ldiskfs+DNE:test_18c | seen in 7 other reviews |
LU-15367 llite: Add iotrace debug logs for allocate operations Add iotrace debug logs to track allocate operations in ll_fallocate(). Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Xiao Yang <xyang@ddn.com> Change-Id: Ib983223f5b3aac3c6c31aa317f93bff44a87beb6
LU-16741 ptlrpc: refactor __ptlrpc_req_put Remove ptlrpc_req_put_with_imp_lock, since it's used inconsistently. Change parameter 'locked' from int to bool. Return value is never used; make it void. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie33dd6e3b6153a7ec80a05dbc5dc275ae4024360
no Fixes: line for the patch that introduced this?
this patch was pushed 8 month ago and not refreshed since. that means no ai review (this is a note to myself to run it before pushing)
(defect) This drops the `if (skip_index)` condition but keeps its body, so the assignment two lines above is now dead - it is always overwritten. Removing skip_index should mean the code behaves as it did when skip_index was 0, i.e. the window end stays anchored at ras_window_start_idx:
if (ras->ras_window_pages > 0)
end_idx = ras->ras_window_start_idx + ras->ras_window_pages - 1;
end_idx = *start_idx + ras->ras_window_pages - 1; /* was skip_index only */
As written the RA window end is now anchored at ras_next_readahead_idx for every read, not just the mmap range case, which is a behaviour change the commit message describes as a simplification. Is that intended? If so it deserves a sentence in the message and the dead lines above should go.
The `ras->ras_window_pages > 0` guard also disappears with it. When the window is zero the new expression underflows to `*start_idx - 1`, and with `*start_idx == 0` that wraps to ULONG_MAX; the EOF clamp below then turns it into `[0, eof_index]`, i.e. a whole-file readahead window, where the old code fell through to the `end_idx == 0` RA_STAT_ZERO_WINDOW return. ras_detect_read_pattern() reaches that state - the stride-invalidation path sets `ras_window_pages = 0; ras_next_readahead_idx = index;` with index 0, and the mmap hit path in ras_update() sets `ras_window_pages = 0` without touching ras_next_readahead_idx.
this... sounds somewhat unintentional? I guess forcing whole file readahead would do wonders to tests that expect great readhead, but what about actual workloads? do we have any actual tests?
(style) prefer `((...))` for numeric comparisons
(style) this isn't a bug, but the numeric comparison was asked to move to `(( ... ))` on an earlier patchset and the line is still using `[ ... -eq ... ]`:
(( miss == 2 )) || error "expected misses 2 but got $miss"
LU-15069 llite: remove skip_index Skip_index seems to have been a confused attempt to change mmap readahead behavior, but basically just turns off a number of readahead optimizations in a weird way and sometimes causes us not to read a page for - as far as I can tell - no reason. This causes misses in normal read patterns. It also skips the minimum page reservation logic, again for no reason I can see - mmap reads should use the same logic here as everything else. Remove it entirely to simplify the code a little. This also reduces the number of misses we take in a simple readahead test, which is a nice indication of how this code is affecting readahead. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia953c8a8cb6dc175255ac316b4b3f02d78effae9
LU-13805 llite: udio and encryption Encryption requires the client always send complete pages, which unaligned DIO does not do. So when encryption is enabled, and we would get a partial page from UDIO, we fall back to buffered IO. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic81f0338d26c86321e5ffd74ef6ed6c8ddb5cdc6
| unique failing test | history |
|---|---|
| sanity2@zfs:test_104c | seen in 19 other reviews |
LU-17478 clio: rename ll_* dio functions In the code reorganizations for the IO path rewrite (parallel DIO, UDIO, Hybrid) these functions all moved from being ll_* functions to being cl_* functions, declared and implemented in the cl layer. Rename them accordingly. OCI-bug-id: LFS-288 Test-Parameters: trivial Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I867f7db1d9cede573f7dd840b80641544d8575b7
LU-18541 doc: Update man page see also Update man page see also for lfs.1 and lctl.8. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I42b07b19ab6f821537a61c9f7fc8e663b455e91a
| unique failing test | history |
|---|---|
| sanity-quota@ldiskfs+DNE:test_1b | seen in 11 other reviews |
I had to manually change the email in the Author line for Gerrit to accept the patch.
this seems at least somewhat redundant? Either it is set and then the original data is NULL (or data and original data are equal) or it is not set and then they differ of course. Also we used to have an assertion at least in llite for when you try to set the ast data that's already set to something else, did not we?
I think what Oleg is trying to point out is that it would be more useful to print the current l_ast_data and data, since the previous l_ast_data is not interesting (either NULL if set=1, or equal to current l_ast_data).
LU-16567 ldlm: Add l_ast_data to debugging Knowing the value of l_ast_data, if it's set, and if it has changed is useful for debugging some issues. Add it to the lock debug output. Also add debug in OSC where l_ast_data is changed. Test-Parameters: trivial Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I60290d03f024d8e990ff9cffe326225753c8a37e
LU-16858 build: remove flock config option flock'ing can be turned on and off at mount time, so a configure option to change the default behavior is rather odd and is not actually used. So let's remove it. Note despite the slightly confusing phrasing in the code, flock is *on* by default and this lets you change the mount time default to off. So this patch is not changing the 'normal' runtime flock behavior. test-parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: If2706506135264f501c6cbc6243ed449f9792605
LU-16858 build: Remove checksum config option Lustre is never built without checksum support and it can be disabled at runtime if needed. Remove the config option. Unlike some of the config removals, this one required almost no changes, so I've marked it trivial. test-parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I04ea24a8a8ae1d41e70113438b8a7abaa6826231
LU-16741 ptlrpc: rename __ functions Some functions are named with __ but aren't the internal implementation of something else; rename them accordingly. Use ptlrpc_request_free in ptlrpc_req_fini(). Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I790aa71fc87bf218930091c5c8c618d987ab1401
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 12 other reviews |
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 12 other reviews |
| runtests@zfs:test_1 | seen in 5 other reviews |
LU-17433 llite: make hybrid writes locked Hybrid IO writes should default to locked, so they can be async. Once we have the dlmlock contention checking for hybrid, we can switch them OFF this mode when appropriate. Test-Parameters: forjanitoronly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I12fd403eaf54dfecb98b8867aff1e0c46e266b5e
LU-17433 clio: add and put ref on dlmlock for sdio Add the add and putref on the dlmlock associated with an SDIO. The problem here is you haven't figured out how to get that lock yet, which is a little tricky. You also haven't considered when or how to force hybrid over to being locked. Realistically, hybrid write should start locked and only go lockless if there's contention. Test-Parameters: ignore Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ie330d072d45f4222691a0150a1c7ab3664a03c81
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 crashed | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-2 crashed | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-pfl. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-2 crashed | RHEL 9.5 / x86_64 | ran 13 tests. 1 tests failed: runtests. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-3 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-3 crashed | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 8.10 / x86_64 | ran 17 tests. 5 tests failed: sanity-quota, sanity-hsm, sanity-flr, sanity-dom, replay-ost-single. %% THIS TE | session |
| review-dne-part-4 crashed | RHEL 9.5 / x86_64 | ran 15 tests. 4 tests failed: sanity-quota, sanity-hsm, sanity-dom, replay-ost-single. %% THIS TEST SESSION C | session |
| review-dne-part-5 crashed | RHEL 8.10 / x86_64 | ran 7 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 crashed | RHEL 9.5 / x86_64 | ran 7 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-6 crashed | RHEL 8.10 / x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-6 crashed | RHEL 9.5 / x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-7 crashed | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-8 crashed | RHEL 8.10 / x86_64 | ran 4 tests. 1 tests failed: replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-2 crashed | RHEL 8.10 / x86_64 | ran 9 tests. 3 tests failed: sanity-selinux, sanity-sec, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-2 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 1 tests failed: replay-dual. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 crashed | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-hsm. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-5 crashed | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-6 crashed | RHEL 8.10 / x86_64 | ran 6 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-7 crashed | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-scrub. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-zfs crashed | RHEL 8.10 / x86_64 | ran 10 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %% | session |
(style) single space between variable type and name
LU-12782 osc: Do not touch object attrs for every page osc_io_commit_async is currently updating the object attributes for every page it handles, but this is extremely wasteful - the attributes being updated are size and mtime, both of which are linked to the IO and not to the specific page. Just do it for the last page in each osc IO. I'm sticking this patch under LU-12782 temporarily, if it works OK in early testing I will create a new LU for it. Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ida150b9b2a1087de787bd7d18dcddfec3401b900
LU-19255 utils: fix write round up in mirror ops The mirror functions: lfs_mirror_write llapi_mirror_copy llapi_mirror_copy_many llapi_mirror_resync_many_params all rounded up the write size to page size, which causes them to write empty bytes at the end of files which are not page aligned. Add sanity tests to validate mirror copy operations. Tests by Augment Agent. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I63dcff1434b09b42d569a23b692fc58481781fbf
| unique failing test | history |
|---|---|
| runtests-ssk@ldiskfs+SharedKey:test_1 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-4 | CentOS 8.3/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-5 crashed | CentOS 8.3/x86_64 | ran 8 tests. 1 tests failed: recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-arm | CentOS 8.3/aarch64, CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs | CentOS 8.3/x86_64 | ran 8 tests. 1 tests failed: replay-single. | session |
LU-15069 llite: Tie readahead proc values The readahead proc values have a hierarchy of limitations, where some cannot be increased until others have. Change the code to do this automatically rather than force users to figure it out. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Iad8f8e2ad5e9a8e69db0f5c3511c8f14c18507b2
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | CentOS 8.3/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 failed 2× | RHEL 8.3/x86_64 | ran 1 tests. 1 tests failed: node-provisioning. %% NODE-PROVISIONING AND/OR LUSTRE-INIT FAILED MULTIPLE TIMES | session |
| review-dne-part-4 failed 2× | CentOS 8.3/x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
LU-14917 llite: Switch mmap readahead logic The mmap readahead logic has shown to be badly suited for certain workloads (kdb). Experimentation showed that for these workloads, the standard readahead algorithm works better. This patch allows switching the readahead algorithm for mmap to the 'standard' readahead algorithm. The added tunable is: mmap_read_ahead_logic in llite (lctl get_param llite.*.mmap_read_ahead_logic) It defaults to '1', which is the special mmap readahead logic. Setting it to 0 switches mmap readahead to use the standard readahead logic. This patch also fixes the existing mmap readahead test, which was not running. Note the results for the existing test are better with the standard readahead logic, ie, with mmap readahead logic disabled. This suggests we should default to non-mmap readahead logic. However, the mmap readahead logic was carefully tuned for certain workloads, and this would be a large change. So, this question is deferred until we can do a larger look at readahead behavior. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ib34297f1dd0498356b21580224f4b954bb10cb14
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.6/x86_64 | ran 9 tests. 1 tests failed: replay-dual. | session |
| review-dne-selinux-ssk-part-2 | CentOS 8.5/x86_64 | ran 5 tests. 1 tests failed: recovery-small. | session |
| review-zfs | CentOS 8.5/x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-16653 llite: Add aio to rw stats The read/write stats in proc miss IO done via AIO, because we don't record them when completing an aio. Add them. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Icaf32ab90f3be2c38fa1dcd6ed4c1fdd168d8482
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_413b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.7/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.7/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.7/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-5 crashed | RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs-ubuntu | RHEL 8.7/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs | RHEL 8.7/x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
Except I didn't put this one in order, so I'll do that.
LU-16714 utils: Improve DIO/BIO choice in migrate We default to direct IO in migrate_copy_data, but this is harmful for performance on small sizes (Roughly < 32 MiB). Switch to BIO for these sizes, and make this tunable, because the correct cutover size between DIO and BIO can differ substantially from system to system. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I90db1f097d13f75cc4a89fd186e2ed824e70c0c6
LU-13371 api: add llapi_getdirstripe function Add a new llapi_getdirstripe() function to the Lustre API to retrieve directory striping information. This function is similar to llapi_file_get_stripe() but specifically for directory striping information. This function will make it easier for applications to retrieve directory striping information without having to directly use the LL_IOC_LMV_GETSTRIPE ioctl. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I41da7e83c6d67bc182d008524d08a9088405cc05
| unique failing test | history |
|---|---|
| sanity-pcc@ldiskfs+DNE:test_1b | seen in 2 other reviews |
| sanity-scrub@ldiskfs:test_17b | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-zfs-part-5 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
LU-14919 osd-ldiskfs: Fix fake i/o page unlocking The fake i/o code incorrectly unlocks pages used for direct i/o. This causes an assert when unloading & freeing the pages. Add direct i/o tests to the existing fake_rw tests. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ia456905c1819705b447d3957eaa17c46cccbae6a
LU-13802 llite: hybrid IO HDD thresholds Sync time is a huge factor in DIO performance, so this means the cutover point for hybrid IO is very different. Test-Parameters: fortestonly Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com> Change-Id: I11e74536117ce3fa0c5e297640b1a074f4517074
| unique failing test | history |
|---|---|
| conf-sanity2@ldiskfs+DNE:test_73c | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_73e | seen in 8 other reviews |
| conf-sanity2@ldiskfs+DNE:test_74 | seen in 12 other reviews |
| conf-sanity2@ldiskfs+DNE:test_75 | seen in 9 other reviews |
| conf-sanity3@ldiskfs+DNE:test_87 | seen in 7 other reviews |
| conf-sanity3@ldiskfs+DNE:test_88 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_89 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90a | seen in 12 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90b | seen in 15 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90c | seen in 21 other reviews |
| conf-sanity3@ldiskfs+DNE:test_90d | seen in 24 other reviews |
| conf-sanity3@ldiskfs+DNE:test_91 | seen in 29 other reviews |
| conf-sanity3@ldiskfs+DNE:test_98 | seen in 34 other reviews |
| conf-sanity3@ldiskfs+DNE:test_99 | seen in 8 other reviews |
| conf-sanity3@ldiskfs+DNE:test_120 | seen in 13 other reviews |
| conf-sanity4@ldiskfs+DNE:test_161 | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_91 | seen in 1 other review |
| sanity-quota@ldiskfs+DNE:test_92 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_93 | seen in 5 other reviews |
| sanity-quota@ldiskfs+DNE:test_94 | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_95a | seen in 4 other reviews |
| sanity-quota@ldiskfs+DNE:test_95b | seen in 4 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-4 | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
Should this set PARALLEL_MKFS='mdt ost mgt' or are we just trying to get the old behavior to work at this point?
This should add "fortestonly" until it is passing testing, so it doesn't run a score of other test sessions.
It looks like the problem being hit by conf-sanity.sh is that lustre-initialization has already formatted mounted the MDTs and OSTs, but then it tries to format them again:
mkfs.lustre --mgs --fsname=lustre --mdt --index=0 --param=sys.timeout=20 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --mkfsoptions=\"-b 4096 -E lazy_itable_init\" --reformat /dev/vg_Role_MDS/mdt1
mkfs.lustre FATAL: Unable to build fs /dev/vg_Role_MDS/mdt1 (256)
Permanent disk data:
Target: lustre:MDT0000
Index: 0
Lustre FS: lustre
Mount type: ldiskfs
Flags: 0x65
(MDT MGS first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: sys.timeout=20 mdt.identity_upcall=/usr/sbin/l_getidentity
device size = 1888MB
formatting backing filesystem ldiskfs on /dev/vg_Role_MDS/mdt1
target name lustre:MDT0000
kilobytes 200000
options -b 4096 -I 1024 -i 2560 -q -O uninit_bg,^extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L lustre:MDT0000 -b 4096 -I 1024 -i 2560 -q -O
/dev/vg_Role_MDS/mdt1 is apparently in use by the system; will not make a filesystem here!
Possibly there is new state in test-framework.sh to track the parallel formatting, but this is lost between lustre-initialization and the instance of test-framework.sh that is starting up?
At this point in the conf-sanity.sh "reformat_and_config->formatall->stop mds1" chain, this is called and does not detect the MDT as mounted:
CMD: trevis-130vm6 [ -e "/dev/vg_Role_MDS/mdt1" ]
CMD: trevis-130vm6 grep -c /mnt/lustre-mds1' ' /proc/mounts || true
CMD: trevis-130vm6 lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' || true
so something is going wrong during this stage and some debugging should be added here. It would probably be the same to debug locally by running "llmount.sh" and then run "conf-sanity.sh" afterward.
(style) prefer `[[...]]` for bash
(style) it would be better to declare this before usage in `stopall()` above
LU-17240 tests: add parallel format/mount/unmount support Add support for parallel format, mount, and unmount operations in the test framework. These operations are controlled by a hierarchy of variables that allow fine-grained control over which operations run in parallel, which target types are parallelized, and whether different target types can overlap. Variable hierarchy: - PARALLEL_OPS: Master switch controlling all operations - PARALLEL_FORMAT, PARALLEL_MOUNT, PARALLEL_UMOUNT: Enable parallelism for specific operations (default to PARALLEL_OPS) - PARALLEL_FORMAT_TARGETS, PARALLEL_MOUNT_TARGETS, PARALLEL_UMOUNT_TARGETS: Space-separated list of target types to parallelize within each operation (e.g., "mdt ost") - PARALLEL_FORMAT_TYPES, PARALLEL_MOUNT_TYPES, PARALLEL_UMOUNT_TYPES: Whether to overlap different target types during operations (inter-type parallelism) Implementation: - New helper function parallel_enabled_for(operation, target_type) checks if parallelism is enabled for a specific combination - Updated formatall(), mountmds(), mountoss(), stopall(), unmountoss() to support parallel execution with proper wait barriers between target types when overlap is disabled - MGS excluded from parallel operations since there's only ever one MGS target All parallel operations are disabled by default (PARALLEL_OPS=0) for backward compatibility. Future patches will enable these features gradually after validation. Test results show significant performance improvements when enabled: - Parallel format with type overlap: 19% faster than baseline - Parallel mount within types: 7% faster - Parallel unmount within types: 21% faster Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I51af959a4f20644d9bd5166c1c1c8ebcfbef69f3
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-3 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
LU-11532 tests: Improve cancel_lru_locks debug Cancel_lru_locks is almost always (always?) used to drop all locks in a namespace, not just the unused ones. A very common test failure is when a lock is not cleared because it is unexpectedly still in use. Improve the cancel_lru_locks function to report this case. A quick scan of the test-framework suggests nothing is relying on having null output from this function, so this should be safe even if some tests are expecting some locks to be in use (and so remain after this call). This should improve debugability of failures like LU-11532. Signed-off-by: Patrick Farrell <paf@cray.com> Change-Id: Idbb62b9a8881c19ae135bdb1cb22f366d236b43b
LU-12782 llite: Convert attr lock to rwlock Under some shared file workloads, the cl_object_attr_lock ends up 'hot'. Because it is a spinlock which is often used only for reading, it can easily be converted to an rwlock. This should show up in some shared file workloads, notably shared file reading. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I50d6f04f31eeea3ab5af58a1b6b56c1d4cfc7093
| unique failing test | history |
|---|---|
| sanity1@ldiskfs+DNE:test_39j | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_64f | seen in 37 other reviews |
| sanity2@ldiskfs+DNE:test_133c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity2@ldiskfs+DNE:test_398g | seen in 13 other reviews |
| sanity1@zfs:test_42e | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@zfs:test_45 | seen in 4 other reviews |
| sanity-benchmark@ldiskfs+DNE:test_fsx | seen in 17 other reviews |
| sanity-benchmark@ldiskfs+DNE:test_fsx_partial_punch | seen in 8 other reviews |
| sanity-dom@ldiskfs+DNE:test_fsx | seen in 7 other reviews |
| sanity-dom@zfs:test_fsx | seen in 7 other reviews |
| sanity-dom@zfs:test_42e | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-dom@zfs:test_4 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_70a | seen in 63 other reviews |
| sanity-flr@zfs:test_70a | seen in 53 other reviews |
| sanity-hsm@zfs:test_3 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_16k | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_18 | seen in 1 other review |
| sanityn@zfs:test_16a | seen in 3 other reviews |
| sanityn@zfs:test_16b | seen in 3 other reviews |
| sanityn@zfs:test_16k | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@zfs:test_18 | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-4 | RHEL 9.3 / x86_64 | ran 9 tests. 2 tests failed: sanity-flr, sanity-dom. | session |
| review-dne-part-5 | RHEL 9.3 / x86_64 | ran 6 tests. 2 tests failed: sanityn, lustre-rsync-test. | session |
| review-dne-part-6 | RHEL 9.3 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.9 / x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.9 / x86_64 | ran 9 tests. 1 tests failed: sanity-dom. | session |
| review-dne-zfs-part-5 | RHEL 8.9 / x86_64 | ran 6 tests. 2 tests failed: sanityn, lustre-rsync-test. | session |
| review-dne-zfs-part-6 | RHEL 8.9 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-zfs-part-7 | RHEL 8.9 / x86_64 | ran 4 tests. 1 tests failed: sanity-pcc. | session |
| review-ldiskfs | RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9 / x86_64, RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.9 / x86_64 | ran 10 tests. 2 tests failed: replay-single, sanity-flr. | session |
LU-13419 osc: Improve speed of enter_cache_try When doing writes to many files, one bottleneck on a client currently seems to be the grant code, specifically spinning in the lock around: osc_enter_cache_try The contention is *just* on osc_enter_cache_try, so there's no obvious way to refactor the lock, etc. Instead, we can look at where time is going in the function. Two things that stand out: obd_dirty_pages is an atomic, and expensive: In my perf tracing, the add_return to this is 50% of the time in this function. This can be replaced with a percpu_counter. These benchmark #s are with the earlier version of the patch, which mistakenly replaced the atomic with a bare unsigned long. I'm not currently able to benchmark the percpu_counter, but it should be similar. mpirun -np 36 $IOR -o $LUSTRE -w -t 1M -b 2G -i 1 -F That's 36 processes on one client, writing to separate files. Before patch: 5942 MiB/s After patch: 14950 MiB/s Looking in perf, the change is huge: I go from spending 60% of the time in osc_enter_cache_try to around 30%, but that's while moving 2.3x the amount of data per second. Signed-off-by: Patrick Farrell <paf0187@gmail.com> Change-Id: If5a69b906c6b56786e6a06dccc723781591419e8
It would be even more useful if `i` was "number of iterations for the next command"?
The `J` option has already been used by madvise(HUGEPAGE), and 'j' is used by this patch, though `j` = "jump" would probably make more sense and flock can use something else, maybe `I`?. It would be good to get this patch landed, since a number of test cases appear to be using the
LU-12645 tests: Add read ahead tests The existing readahead tests are too lenient, not checking misses at all. They also do not cover enough cases, and use a complex special purpose utility, which takes in to account stripe size, etc. This is overly complex and not really correct - strided read patterns don't have to have any relation to stripe size. Instead, we can just modify multiop to support writing or reading a certain number of times, optionally jumping between each operation. This allows describing any possible strided pattern with just four arguments - iterations, size, jump, and starting offset. (It's also possible to use this for backwards reads.) This patch uses multiop to add a short but strict & varied set of tests for sequential and strided readahead. It also simplifies/replaces many of the existing readahead tests, which did odd things like disable the cache on the server, which is irrelevant to client side readahead. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Wang Shilong <wshilong@ddn.com> Change-Id: I774585a17deac8c0f3b25ddbe047617177f3caf3
LU-18664 target: set proper kthread state The function distribute_txn_commit_batchid_update() reports a __might_sleep() issue due to distribute_txn_commit_thread() potentially calling it while in an idle state. Get the current state before distribute_txn_commit_batchid_update(), switch to TASK_RUNNING, and restore the state [likely TASK_IDLE] to avoid skipping the subsequent schedule() and allowing other threads to proceed. In the case where distribute_txn_commit_batchid_update() failed it will retry when the kernel scheduler runs the thread again. To ensure the task is rescheduled use an increasing timeout, max at 30s. Test-Parameters: testlist=sanity env=ONLY=60g,ONLY_MINUTES=30 Signed-off-by: James Simmons <jsimmons@infradead.org> Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: If9ae5ac47ec4eb24f87bc2a6bf78d3134136be6f
LU-15248 tests: don't let racer leak fs users that hang the run racer can time out with the client drained but the harness stuck. The racer.sh wrapper blocks in "wait $rpids", the tee/sed/grep logging pipeline sits in pipe_wait, and no racer worker is running. Two janitor crashdumps show the cause -- an orphaned process (a copied binary that file_exec.sh exec'd, e.g. "8"/"14", reparented to init) still holding the racer output pipe open, so the pipeline never sees EOF and the enclosing "wait" never returns. file_exec.sh copies /bin/sleep to $DIR/$file and executes it. Because racer's other tasks concurrently overwrite those same names, the copy is often corrupt and can hang or spin instead of exiting; and racer_cleanup's "killall $P.sh" kills the script but not the foreground process it exec'd, which is orphaned holding the pipe. Bound the exec'd copy with "timeout -s KILL 5" so it cannot outlive the worker and hold the pipe/fs open. As a backstop for any other straggler that keeps the mount busy (e.g. before unmount), also sweep leftover fs users with "fuser -k -m -M" over the racer dirs. Test-Parameters: trivial Test-Parameters: optional testlist=racer Signed-off-by: Oleg Drokin <green@whamcloud.com> Assisted-by: ClaudeCode:Opus-4.8 llm_code_and_review_tools Change-Id: I0fde591d306bc0026382c931a7441f182c73aaea
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
A few hunks aren't covered by the description, so it's hard to tell which are intentional:
- obd.h reorders op_code/op_xvalid/op_bias/op_cli_flags (a packing cleanup, unrelated to stale merges);
- obd.h converts `bool op_new_layout` into a bitfield, which touches a field this feature never uses;
- llite/file.c changes the LL_IOC_LEASE unlock error paths from `rc` to `rc2` and folds it in at the end.
That last one is a real user-visible fix, not a cleanup: today `GOTO(out_lease_close, rc = -EINVAL)` (and -EFAULT/-EBADF/-EPERM/-ENOMEM) is immediately overwritten by `rc = ll_lease_close_intent()`, so LL_LEASE_RESYNC_DONE / LAYOUT_MERGE / LAYOUT_SPLIT return the lease type instead of the error. Would it be better as its own patch with
Fixes: f172b116885 ("LU-10092 llite: Add persistent cache on client")
so it can be reviewed and backported on its own?
This adds a new on-wire field (cd_merge_flags) and a new client-set flag value, so a Test-Parameters: line requesting interop coverage would be useful here, e.g. an older serverversion and an older clientversion run.
LL_LEASE_ALLOW_STALE is a modifier bit but the switch still matches lil_flags exactly, so every future combination has to be enumerated as its own case. Anything unenumerated (LL_LEASE_LAYOUT_SPLIT | LL_LEASE_ALLOW_STALE, or the bit on its own) falls into `default:` and quietly releases the lease with bias = 0, returning the lease type as if it had worked. Would `switch (ioc->lil_flags & ~LL_LEASE_ALLOW_STALE)` plus a check that the modifier is only accepted for MERGE read better?
All of these bool -> 1 are unnecessary and in files not otherwise touched - let's leave them out?
I'm not sure whether coverity would complain about the transform bitwise from/to boolean value kinda of warning, so I made this change.
This isn't a bug that I can trigger today, but now that merge can mark every component of the new mirror stale, should this pass LVF_ALL_STALE like lod_declare_layout_split() does at the equivalent point? That's the flag that turns on the "can not set all stale mirrors" check in lod_parse_striping(), and merge is currently the one stale-producing path that skips it.
There doesn't seem to be any negotiation for this. An MDT that predates the patch never looks at cd_merge_flags - mdt_close_handle_layouts() just calls mo_xattr_set(..., LU_XATTR_MERGE) - so a new client asking for a stale merge against an older server gets the mirror merged with no LCME_FL_STALE set, and rc = 0 back. The result is a mirror that was never written being advertised as up to date, which reads can then be served from. Should this be gated on a new OBD_CONNECT2_* flag, with the client either failing with -EOPNOTSUPP or falling back to the sync path when the server doesn't advertise it?
The declare above passes LU_XATTR_MERGE_STALE, but the execute phase here still passes plain LU_XATTR_MERGE. It works only because lod_declare_layout_merge() builds the merged layout at declare time, and because LU_XATTR_MERGE_STALE would actually break the execute path - lod_xattr_set() has
if (!(fl & LU_XATTR_MERGE))
LASSERT(equi(...));
which would fire for an existing file if MERGE_STALE were passed instead.
Would it be safer to define LU_XATTR_MERGE_STALE as an extra bit used together with LU_XATTR_MERGE rather than in place of it? Then every existing `fl & LU_XATTR_MERGE` test (lod_object.c) and `fl == LU_XATTR_MERGE` test (mdd_xattr_set()) stays correct, and declare/execute can use the same value.
This is the only caller of mirror_extend_layout() and it hardcodes sync=true, so the `if (sync)` branch is always taken and `data->lil_flags |= LL_LEASE_ALLOW_STALE` is never executed. Grepping the tree, LL_LEASE_ALLOW_STALE is only referenced here and in ll_file_unlock_lease(). So as it stands `lfs mirror extend` can't create a stale mirror and the entire mdc/mdt/mdd/lod path added by this patch is unreachable from any shipped tool. Is the intent to add an `lfs mirror extend` option (plus the matching Documentation/man8/lfs-mirror-extend.8 text) in this patch, or is that coming in a follow-on?
check_close_data() is called inside the CHECK_COND_START(CONFIG_LUSTRE_FS_SERVER) block, so the generated assertions in both wiretest.c copies end up under `#ifdef CONFIG_LUSTRE_FS_SERVER`. struct close_data is packed by the client in mdc_close_intent_pack(), so a client-only build never verifies the layout of a struct it puts on the wire. The other client-visible structs (check_swap_layout(), check_hsm_*()) are called outside that block - should this one be too?
LU-18746 flr: allow merge stale mirror This patch makes it possible to merge a stale mirror to an existing file, so that we can append several mirrors w/o writing data to those mirrors first, and after we've finished the merge we'd do the mirror resync later. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: I0d730566ea9b238aeac5d2e27e77c719a1c73308
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 failed 2× | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
I'm sort of confused about the logic here - why do we have to replace it? What does that do? And how can this page already be in the radix tree? I guess it's not "this" page, it's just another page at the same index. So this seems like it might be a race with removal in cl_page_delete/osc_page_delete? If it is, then how do we avoid the other thread removing *this* page from the tree, since it's done by index? It looks to me like vvp_page_delete() makes the page inaccessible in the page cache(?) by resetting the private pointer, but I'm not 100% sure. Then osc_page_delete() is called after. I guess my point is I'm not sure this is solving rather than hiding the problem. I guess perhaps if the page is in the process of being removed (which is why we're getting a new page), we can just do that removal ourselves first? Is that the logic here? I guess that seems sound, but what about the deleting thread possibly removing this page?
I think we must investigate why the page is already in the tree?
yes, I agree that it's strange to find an osc_page didn't successfully deleted from the radix tree, I'm just working on a workaround here as the customer find out certain version kernel does not have this issue while some does, so I'd guess that certain version kernel has some bug in the radix tree implementation.
LU-19037 osc: initialize osc_object::oo_tree Initialize the radix tree of osc_object::oo_tree. Lustre-change: https://review.whamcloud.com/59340 Lustre-commit: 1f4d2013929bd61f775e6c98a4122fe143b4242b Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Change-Id: I77a19cb08ccc52f4eb3457cd3367884b84624054
This should actually run the test script to see that it is working: ``` Test-Parameters: testlist=lfru-performance ``` I've submitted a manual test session via the `Test Results` page to see if it works. https://testing.whamcloud.com/test_sessions/related?jobs=lustre-reviews&builds=122012#redirect Ideally this testing could be run regularly via one of the test scripts (e.g. performance-sanity.sh or sanity-benchmark.sh) to monitor performance over time. Putting it in a separate test script means that the test framework needs to be modified to run this.
Ping
(style) It is better if comments do not contain the actual values in the constants, since that makes it more likely the comments become incorrect over time.
LU-11509 misc: add script lfru-performance.sh LFRU was introduced to provide scan-resistant, which was validated in sanity-test-124g. Furthermore, it ensures that high-priority locks are more likely to remain in the cache, improving overall system stability and performance under mixed workloads. 1. A new benchmark test, `lfru-performance.sh`, is introduced to compare LFRU against LRU. This test simulates a workload where both hot and cold files are accessed with a 50:50 ratio, involving 800 hot files and 16,000 cold files, and fixed sized cache size. The LFRU algorithm reduced the number of lock-RPCs (measured by `ldlm-enqueue` calls) by ~8% compared to the LRU policy. | Test Run | LFRU Enqueues | LRU Enqueues | Improvement | | 1 | 160075 | 174250 | 8% | | 2 | 159372 | 175925 | 9% | | 3 | 159488 | 174533 | 8% | | 4 | 159712 | 175481 | 8% | | 5 | 159986 | 174493 | 8% | The benchmark results showed that LFRU outperforms LRU in this mixed-access scenario. 2. Update ldlm_lfru_priv_too_many() so that the eviction trigger for priv_lock is now primarily based on its ratio to total_lock_counts. The use of LDLM_DEFAULT_LRU_SIZE is removed. Signed-off-by: Keguang Xu <squalfof@gmail.com> Change-Id: I60903947180fb8c2b4e1a74e94cb2c5bb387d9d5
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_398c | seen in 1 other review |
since it looks like this patch will get refreshed eventually, please also fix this typo -> divide.
truncate
I have a doubt here, if the start of the 1st page is not page aligned, since @to most of time is PAGE_SIZE, so at each loop, the page is always not page aligned as well, is it a glitch here?
Should @to be like this:
if (from != 0)
to = min(PAGE_SIZE - from, from + size);
else
to = min(PAGE, from + size);
and except the 1st/last page, all pages in the middle should always be aligned full page.
No changes here, code is correct as is.
from here we can see that the end of the 1st page is page aligned.
and the end of the last page could be page unaligned.
Can we do: unaligned_dio || skip "Need unaligned dio support" Instead of server version check?
More importantly, for older servers the unaligned DIO should be handled gracefully in some manner, since applications running on newer clients will try this and we shouldn't allow them to crash the servers. Just skipping the test doesn't solve the interop problem. One option for UDIO is to expose the DIO alignment via statx, as XFS does. That will allow userspace applications to know what the DIO alignment requirements are, and applications that don't follow them can return EIO or other error directly (as they did befoer UDIO existed, instead of trying to emulate their way through the issues.
LU-17993 debug: to reveal page count issue as title Test-Parameters: fortestonly Test-Parameters: testlist=sanity env=ONLY=119h,ONLY_REPEAT=500 clientdistro=el8.8 serverdistro=el8.8 Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com> Change-Id: Ia502400fb20603c369c34a3fb397a472cc3403c3
| unique failing test | history |
|---|---|
| lustre-rsync-test@ldiskfs+DNE:test_2c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| lustre-rsync-test@zfs:test_2c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_70 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@zfs:test_70 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@ldiskfs+DNE:test_51 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-sec@zfs:test_51 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.6/x86_64 | ran 7 tests. 2 tests failed: sanity-sec, sanity-lfsck. | session |
| review-dne-part-4 | RHEL 8.6/x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-selinux-ssk-part-2 | CentOS 8.5/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 | CentOS 8.5/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | CentOS 8.5/x86_64 | ran 7 tests. 2 tests failed: sanity-sec, sanity-lfsck. | session |
| review-dne-zfs-part-4 | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-ldiskfs-arm | RHEL 8.6/aarch64, RHEL 8.6/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs-ubuntu | CentOS 8.5/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-zfs | CentOS 8.5/x86_64 | ran 8 tests. 2 tests failed: sanity-flr, sanity-quota. | session |
LU-15979 llite: restore S_NOSEC in ll_update_inode()
ll_update_inode() is to set S_NOSEC (if needed) after permission
update with help of inode_has_no_xattr() having inode->i_rwsem held.
Without that { creat(); fstat(); write(); } executes to eviction
favorable case:
cl_io_loop
cl_io_lock <- LDLM lock is taken here
cl_io_start
vvp_io_write_start
...
__generic_file_aio_write
file_remove_privs
security_inode_need_killpriv
...
ll_xattr_get_common
...
mdc_intent_lock <- enqueue RPC is sent here
If enqueue rpc is delayed, the client may get evicted as not
cancelling lock taken in cl_io_lock.
ll_update_inode() is called without inode->i_rwsem locked for regular
file but one case:
vfs_setxattr()
inode_lock(inode);
..
ll_xattr_set()
ll_setstripe_ea()
ll_lov_setstripe_ea_info()
ll_intent_file_open()
ll_prep_inode()
ll_update_inode()
where ll_update_inode() is called with inode->i_rwsem locked.
In order to be able to detect this case MDS_OPEN_SETXATTR flag is added.
Test to illustrate the issue is added.
The fix does not help for not NOSEC files.
Uncommenting chmod command in the test makes it to fail with eviction.
Test-Parameters: testlist=replay-dual env=ONLY=34,ONLY_REPEAT=100
Change-Id: Ie9e32d03402027f47381edddbd5cb3fb75023d59
HPE-bug-id: LUS-10989
Signed-off-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-4 | RHEL 8.7/x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-zfs | RHEL 8.7/x86_64 | ran 8 tests. 1 tests failed: sanity-flr. | session |
[minor] I'm pretty sure 'rm $DIR/$tfile' is unnecessary. A lot of tests don't do it. I would remove if patch is refreshed.
Right, only when a subtest is creating hundreds/thousands of files, or the files are MBs in size do they need to be cleaned up explicitly.
This looks good... I suppose testing the lock_no_expand API would require writing a test program to do it. It can't be tested from ladvise since it's just on a single file descriptor. So the IO has to come from the program which calls the API. Well, that is not a new limitation. I could've written that test when the API was added and I didn't :)
LU-16669 llite: add LOCK_NO_EXPAND fail_loc on a client Lustre advise IOCTL interface can set the CEF_LOCK_NO_EXPAND flag, which tells the OSC to set LDLM_FL_NO_EXPANSION on any lock requests. This lock flag tells the server (OST) not to expand the lock extent. However, It is inconvenient to use and requires programming to call Lustre advise API. To set this parameter much easier for benchmark or debug purpose, we add a fail_loc (0x1425) to control whether to expand the lock extent for I/O on a client. Add sanity/test_255d to verify it works as expected. Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I0d7462665cea37baf8ecbb8ab8f609a4b7f1f317
| unique failing test | history |
|---|---|
| obdfilter-survey@ldiskfs+DNE:test_1a | seen in 10 other reviews |
| obdfilter-survey@ldiskfs+DNE:test_1b | seen in 10 other reviews |
| obdfilter-survey@ldiskfs+DNE:test_1c | seen in 10 other reviews |
| obdfilter-survey@zfs:test_1a | seen in 10 other reviews |
| obdfilter-survey@zfs:test_1b | seen in 10 other reviews |
| obdfilter-survey@zfs:test_1c | seen in 10 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 failed 2× | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-3 failed 2× | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-5 failed 2× | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-selinux-ssk-part-1 failed 2× | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-1 failed 2× | RHEL 8.8/x86_64 | ran 6 tests. 2 tests failed: sanity-pfl, sanity. | session |
| review-dne-zfs-part-3 failed 2× | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-zfs-part-5 failed 2× | RHEL 8.8/x86_64 | ran 6 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-dne-zfs-part-6 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-ldiskfs failed 2× | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs failed 2× | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm failed 2× | RHEL 8.7/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs failed 2× | RHEL 8.8/x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
LU-6142 obdfilter: obdfilter script improvements This patch: - replaces 'lctl' with '$LCTL' wherever applicable. - Returns true/false instead of 1/0 for function is_local_addr() - replaces "! local_node" call with more natural remote_node Test-Parameters: testlist=obdfilter-survey Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Change-Id: Id4ba601fd8f31cde79c099581575a5390bb9515c
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.3/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.9/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.9/x86_64, RHEL 9.3/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
Is the LUSTRE environment variable also inherited by the system() command that is run? At first I was thinking "you can't just use '$LUSTRE' in the string!" but since the system command is executed by the shell it should work OK? Alternately, since this is already calling getenv() it could save the returned pointer and print it into the cmd string directly.
It works. For example: [centos@ip-172-31-20-187 lustre-release]$ LUSTRE=fizzbuzz ./lustre/utils/lfs help migrate man: fizzbuzz/doc/lfs-migrate.*: No such file or directory No manual entry for fizzbuzz/doc/lfs-migrate.* You can do the same thing in python, which is super helpful for integrating with shell scripts.
Does this interfere with tab completion of commands?
This code is only invoked when you run the `help` command, so it shouldn't interfere with shell completion.
This block doesn't seem to work, causing sanity.sh to fail. Even locally, this seems wrong. Also, this will need to be updated once the man pages are relocated.
LU-4959 parser: make help work like git The git help subcommand displays the man page for the given subcommand. Currently, the Lustre tools (lfs, lctl, lnetctl, lst) that use the parser in libcfs only return a small help text. This patch changes the parser to first make an attempt to display a related man page before falling back to the old help text. The implementation is inspired by git, but differs in a few ways. It is best-effort, it assumes that the user has 'man' available, and has some Lustre specific parsing. Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I912e7c7e8439a822da0fb2e82f160adb9eaac589
Ping, reviewers
(style) more natural abbreviation would be `priv_thresh`, or just `thresh` to fit into 89 columns below
This tests the window size is being adjusted, which is fine, but I was more wondering if there is a way to effectively test whether the dynamic window size is improving the LRU behavior and lock retention effectiveness?
Hi Andreas, honestly I treat this as an intermediate patch: it fixes issues we can address quickly here -- per-CPU counter inaccuracy and threshold-adjustment lag on large-core hosts (e.g. 640 window size on a 64-core system). A better evaluation would likely need timing-aware? behavior and better observability. My plan is to add more metrics in the follow-up patch so we can see retention effectiveness in production-like workloads (will try to simulate some...), then use that data to tune the dynamic window (and with additional timing factors maybe) more suitably.
Has any work been done to measure the LFRU retention behavior?
LU-11509 ldlm: scale LFRU sample window from lru_size lfru_sample_window_size is the number of lock insertions sampled before LFRU recomputes priv_score_threshold from max_freq. It controls how the promotion gate adapts after metadata bursts without filling the priv list with one-touch locks. Previously the window was LDLM_DEFAULT_LRU_SIZE / 10 (10 * num_cpus), set only at namespace creation. On large clients the sample period was too long to promote valuable locks before they were displaced; on small clients it was too short and tending to chased transient churn. It also ignored the lru_size and was not refreshed at runtime. This patch addresses the issue by periodically recalculating the window size based on current lru size, setting it to ns_nr_unused / 10 and clamping it between 32 and 160, the promotion gate now dynamically adapts to changing workloads. Use LDLM_LFRU_PRIV_DEMOTE_THRESH (64) to avoid batch priv demotion oscillation while the cache is still filling. sanity/124e/124f/124h, set llite.*.enable_statahead_fname=0 with the intention to disable `stat()` randomness, to reduce flakiness. Test-Parameters: testlist=sanity env=ONLY=124e,124f,ONLY_REPEAT=100 Test-Parameters: testlist=sanity env=ONLY=124g,ONLY_REPEAT=200 Test-Parameters: clientdistro=el10.1 serverdistro=el10.1 testlist=sanity env=ONLY=124g,ONLY_REPEAT=200 Signed-off-by: Keguang Xu <kxu@ddn.com> Change-Id: I2dfdca5a58c71a2d9ab9fe60795c5db60587b4ce
(minor) The struct brw_page helpers earlier in this file come as a set: brw_pgno(), brw_kmap_local() and brw_folio_page(). The lnb set stops at the two kmap variants, so the folio+fpgno pairing ends up open-coded wherever a struct page is needed:
folio_page(local_nb[i].lnb_folio, local_nb[i].lnb_fpgno)
That appears in tgt_checksum_niobuf(), tgt_brw_read() and tgt_brw_write(), plus twice in osd_integrity.c.
If the patch is refreshed, would an lnb_folio_page() mirroring brw_folio_page() keep the pairing in one place and match the accessor set already established for brw_page?
Seems like a reasonable improvement if patch is refreshed, or in a follow-on patch.
The blank line separating this from echo_put_object() got deleted. Two more unrelated blank-line deletions come in with the patch, at echo.c:309 and osd-wbcfs/osd_io.c:341.
Acknowledged
Could be fixed if patch is refreshed.
LU-17916 osd: prefer folio of order 0 Switch from struct page and page based API to struct folio and the folio API while keeping the PAGE_SIZE logic by only allocating folio(s) of order 0. Provide lnb_fpgno to track the page number within an lnb_folio where kmem_to_folio() in osd-zfs can be a large folio. Propagate lnb_folio to the other osd implementations. HPE-bug-id: LUS-12384 Signed-off-by: Shaun Tancheff <shaun@tancheff.com> Change-Id: I87b37ce94f2f717e8767785e3620e4a30bc22e88
The body explains why lod_layout_mutex is needed, but two hunks aren't accounted for: - `lo->ldo_flr_state = 0;` added to the plain-layout branch of lod_parse_striping() - the extra `!lo->ldo_is_composite ||` term in the lod_declare_update_extents() assertion Could the body say what each of those is for? As written the text reads as if the assertion should stop firing for plain layouts, but the code makes it fire more often (see the inline comment).
This fixes an LBUG, so it would help to carry a Fixes: tag. The assertion being hit was added by:
Fixes: ff5eb304fa37 ("LU-10070 lod: SEL: Implement basic spillover space")
That commit introduced lod_declare_update_extents() together with `if (lo->ldo_flr_state == LCM_FL_NONE) LASSERT(start_index == 0 && max_comp == lo->ldo_comp_cnt);` and the unlocked access to ldo_mirrors[]/ldo_comp_cnt that this change now serializes.
The same reset looks missing in the other two places that build a plain in-core layout after freeing the old one: - lod_use_defined_striping() (lod_qos.c:2217) sets `mo->ldo_is_composite = 0` for LOV_MAGIC_V1/V3 with no ldo_flr_state reset - lod_qos_parse_config() (lod_qos.c:2557) does the same after lod_free_comp_entries() lod_free_comp_entries() clears ldo_mirrors, ldo_mirror_count and ldo_is_composite but not ldo_flr_state, so a composite-to-plain transition through either of those still leaves a stale value. lod_declare_layout_change() then dispatches on it, e.g. into lod_declare_update_rdonly() whose `LASSERT(lo->ldo_mirror_count > 0)` cannot hold for a plain layout. Should those get the same fix?
Adding `!lo->ldo_is_composite ||` widens the guard rather than narrowing it: `A || B` is true whenever `B` is true, so the LASSERT is now evaluated in strictly more cases than before, and the plain-layout case (`ldo_is_composite == 0`) now always evaluates it.
The commit message says ldo_flr_state is only meaningful for a composite layout and that a plain file "could get into lod_declare_update_plain()->lod_declare_update_extents() and bump into this assertion", which reads like the intent was to exempt plain layouts:
if (lo->ldo_is_composite && lo->ldo_flr_state == LCM_FL_NONE)
With `||`, a plain file in the state that produced the LU-18592 LBUG (flr_state == LCM_FL_NONE, stale ldo_mirrors) still hits the same assertion. Which of the two is intended?
Changing `RETURN(-EALREADY)` into `GOTO(out_unlock, rc = -EALREADY)` also routes this through `out:`, so the -EALREADY case now calls lod_striping_free() and drops the cached layout. Previously it returned with the cache intact. mdd_layout_instantiate_component() turns -EALREADY into success, so every layout-write intent on an already-instantiated component now throws away the in-memory striping and forces a re-read of the LOV EA. Is that intended, or should this path skip the free?
This isn't a bug today, but making the unlock conditional on `rc` is fragile: `out:` is reached both by fall-through with the mutex already released (rc == 0) and by `GOTO(out, ...)` with it still held. It only works because every goto to `out:` happens to set rc != 0 -- a future `GOTO(out, rc = 0)`, or a `lod_declare_update_extents()` return of 0 being propagated, silently leaks the mutex. A dedicated `out_unlock:` label above `out_free:` (as was done in lod_declare_update_plain()) would make the lock state structural instead of value-dependent. Same shape in lod_declare_update_write_pending() and lod_declare_update_sync_pending().
This assignment to ldo_flr_state is still outside ldo_layout_mutex, but lod_declare_update_write_pending() now takes the mutex and then asserts on the value:
lod_declare_update_sync_pending() lod_striping_reload()
lo->ldo_flr_state = WRITE_PENDING mutex_lock(ldo_layout_mutex)
lod_parse_striping()
lo->ldo_flr_state = <from disk>
mutex_unlock()
lod_declare_update_write_pending()
mutex_lock(ldo_layout_mutex)
LASSERT(ldo_flr_state == LCM_FL_WRITE_PENDING) <- fires
Since this is the same class of problem the patch is fixing, should the state transition move inside the mutex (or the mutex be taken here and passed down)?
This isn't a bug, but `layout_lcked` is spelled `layout_locked` in every sibling function (lod_check_and_reserve_ost(), lod_ost_alloc_rr(), lod_ost_alloc_specific(), lod_qos_prep_create()). Worth making it consistent if the patch is refreshed.
This is the one remaining caller passing `false`, and it leaves the two lock orders divergent:
lod_prepare_create() -> lod_ost_alloc_{qos,rr}()
lq_rw_sem / ltd_rw_sem held
-> lod_comp_is_ost_used(..., false)
-> mutex_lock(ldo_layout_mutex)
lod_declare_update_plain() mutex_lock(ldo_layout_mutex)
-> lod_declare_instantiate_components()
-> lod_qos_prep_create(..., true)
-> lod_ost_alloc_qos() down_write(lq_rw_sem)
So ldo_layout_mutex is now above lq_rw_sem on the new paths and below it here. This isn't reachable as a hang today (the MDT holds the layout LDLM lock EX for both entry points on a given object), but it makes the ordering depend on an unrelated lock. Would it be cleaner to have lod_prepare_create() take ldo_layout_mutex too and pass `true` everywhere, so lod_obj_for_each_stripe() never has to acquire it as an inner lock?
Related: the new critical section now spans OST allocation, which includes a `down_write_killable()` with a 2s timer in lod_ost_alloc_qos() and lod_sub_declare_create() on OSP objects, so the per-object layout mutex is held far longer than before.
LU-18592 lod: protect lod in-memory layout's change The lod::ldo_flr_state is only meaningful for composite layout, while a plain file could get into lod_declare_update_plain()-> lod_declare_update_extents() and bump into this assertion. lod_declare_update_extents() ) ASSERTION(start_index == 0 && max_comp == lo->ldo_comp_cnt) And lod_declare_update_extents() would modify lod's layout, it needs lod_layout_mutex's protection for consistence. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: I2b3d32b32efb88317c124a7975dfd9889333bc8a
| unique failing test | history |
|---|---|
| runtests-ssk@ldiskfs+SharedKey:test_1 | seen in 100 other reviews |
(style) Several hunks aren't accounted for by the description, and naming the new pieces would make the change findable via git log later: - the new resend queue itself (osc_extent_resend(), cl_resend_list, oo_resend_read/write, oe_resends) is never named. - brw_interpret() now rewrites rc to -EBADF on an import generation change, and the old -EAGAIN/-EINPROGRESS to -EIO mapping is gone. - the discard path in osc_cache_writeback_range() switches from 0 to -EREMOTEIO. - osc_free_grant() switching to OSC_DUMP_GRANT(), plus the trailing-space fix in that macro. - get_write_extents() rewritten around the new osc_collect_rpc(). - the sanity test_118c/118d lt-multiop hunks, which have nothing to do with resend and would be better as their own change. The wording is also hard to follow: "use an osc extents to resend a data" and "it covers a problems ... caused a data lost".
(minor) This is described as a data-loss fix, so a Fixes: tag would help whoever backports it. The osc_build_rpc() error path that completed extents on a local ENOMEM dates to:
Fixes: 9fe4b52ad2ff ("LU-1030 osc: new IO engine implementation")
If this is really a rework rather than a fix for one specific commit, saying that in the body would settle it.
(style) The body carried the space indentation over from osc_internal.h. New/moved code should use tabs.
The comparison also flipped from `>` to `>=` here, which gives one extra attempt compared with the old code for the same cl_resends value. Is that intentional? It isn't mentioned in the commit message, and mdc_getpage() (which increments before calling) and brw_interpret() (which passes resends + 1) count differently, so it's worth being explicit about which one the tunable is supposed to mean.
This call site is only reached when rc == -ETIMEDOUT (the lines just above return early for every other error). The new client_should_resend() returns
(rc == -EINPROGRESS) || (bulk_recoverable_error(rc) && _resend)
and bulk_recoverable_error() covers only EIO/EROFS/ENOMEM/EAGAIN/EINPROGRESS. So -ETIMEDOUT now always yields false, and mdc_getpage() returns -EIO on the very first bulk timeout instead of retrying up to cl_resends times.
That makes resends++, msleep_interruptible() and the goto restart_bulk below dead code, and turns a transient readdir bulk timeout into a failed readdir. Should this caller keep the old count-only semantics, or should -ETIMEDOUT be added to the recoverable set?
do not comment code
oo_resend_item is linked onto the client-wide cl_resend_list without taking a reference on the object, and osc_next_obj() is the only thing that ever unlinks it. __osc_list_maint() manages oo_ready_item/oo_hp_ready_item/oo_write_item/oo_read_item (which is what makes the matching LASSERTs in osc_object_free() hold), but it doesn't know about oo_resend_item, so nothing removes it on teardown. There is also a window right here: the object lock was dropped on the line above, so another thread already inside osc_resend_write() can splice oo_resend_write empty before this list_move_tail() runs. The object then sits on cl_resend_list with both resend lists empty and nothing pending to pull it off, and the new LASSERT(list_empty(&osc->oo_resend_item)) in osc_object_free() trips if the object is destroyed first. Would it be safer to queue the object under the same critical section as the extent, and to clear oo_resend_item from __osc_list_maint()?
(minor) The function returns void now, so "Return %0 always" is stale. Same for the "Return: %0 on success / %negative on failure" block left on osc_extent_make_ready() below.
(minor) This loop no longer stops early. The loops it replaces returned as soon as try_to_add_extent_for_io() failed or the RPC was full; this one walks the whole list even after erd_max_extents has hit 0 or erd_max_pages is reached, and get_write_extents() runs under osc_object_lock(), a spinlock. On an object with a long oo_urgent_exts/oo_full_exts list that is an O(n) spin per RPC build. Breaking out once data->erd_page_count == data->erd_max_pages would keep the old cost. While here, `x == y ? true : false` below can just be `x == y`, and the comment above get_write_extents() about full extents differing from the others is now stale since all three lists go through the same helper.
(style) These two new doc blocks use the old \param/\return doxygen form while the tree is being converted to kernel-doc; they document a parameter named obj when it is called osc, and they list return values for a void function (osc_resend_read()'s block has no closing indent either). Both functions also end with EXIT but have no matching ENTRY.
This changes a successful discard into a reported error. osc_extent_finish() passes it to osc_completion() -> cl_page_complete() -> vvp_vmpage_error(), which calls mapping_set_error(inode->i_mapping, -EREMOTEIO), so the next fsync()/close() on the file returns EREMOTEIO where it previously returned 0. Since osc_extent_finish() never resends, what does the "unrecoverable error" buy here? If the intent is to report discarded dirty data, -EIO would at least also trigger ll_dirty_page_discard_warn(), which vvp_vmpage_error() skips for -EREMOTEIO.
This replaces rc for any non-zero error, not just the recoverable ones the old code handled, and imp_generation is bumped by ptlrpc_deactivate_import_nolock() on ordinary disconnect/failover as well as on eviction. So after a failover an -ENOSPC, -EDQUOT or -ETIMEDOUT BRW failure now reaches the application as EBADF via mapping_set_error(). It also silences ll_dirty_page_discard_warn(), which vvp_vmpage_error() only calls for -ESHUTDOWN/-EINTR/-EIO, so the "dirty page discarded" console warning disappears for this case. Was the intent just to suppress the resend, rather than to overwrite the error the caller sees?
LU-17604 osc: rework resend logic use an osc extents to resend a data after recoverable error hit. it covers a problems with local ENOMEM errors caused a data lost. HPe-bug-id: LUS-11863 Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com> Change-Id: I0f53f992c54d472c3af99d0608cf031720afa626
(minor) ll_dom_finish_open() was the last caller of ll_read_cache_page() and it now calls ll_read_cache_folio() directly, so this wrapper is dead. Same for wbe_folio_folio() added below - all three arms of it are unreferenced. Worth removing both while the header is being reworked?
(minor) This was raised on patchset 41 with the ask to avoid growing cl_page, and the field did move down to sit next to the cp_kmem_index/cp_kmem_size union - but the growth is still there, because a 4-byte field cannot live in the 2-byte hole that follows the union.
Working the offsets on x86-64:
cp_batch ends at 56
bitfield unit 56..59 (cp_layer_offset[] + the :2/:4/:1 fields)
kmem union 60..61
cp_pgno 64..67 (4-byte aligned, so 62..63 stays a hole)
cp_owner 72..79
cp_sync_io 80..87
sizeof(struct cl_page) goes 80 -> 88, of which 6 bytes are new padding.
A 16-bit field would fit the hole, but it cannot hold the index: these folios come from user memory, and a 1GiB hugetlb buffer gives folio_page_idx() values up to 262143. So if the 8 bytes really are unavoidable here, would it be worth saying so in the commit message so the point can be closed, rather than leaving it looking fixed?
(minor) When @index is not the folio's own index this quietly returns NULL rather than an error, and cl_page_find() treats that the same as "no cl_page yet":
cl_page_find() -> cl_page_from_folio(folio, idx, true) /* NULL */
-> cl_page_alloc() -> vvp_page_init()
-> folio_attach_private(folio, cl_page)
So a differing index would attach a second cl_page over the first, stranding the reference folio_attach_private() took for it. Every caller passes folio->index today and everything is order 0, so it cannot happen yet - but this is the helper the multi-page work will build on. Would a LASSERT(index == folio->index) here (or dropping the parameter until sub-page lookup is real) pin the contract down?
(minor) This was the only user of folio_test_mlocked_page(), so LC_HAVE_FOLIO_TEST_MLOCKED in config/lustre-core.m4 now generates a macro nothing references. Should the autoconf test go with it?
LU-17916 clio: switch to struct folio Switch cl_page from struct page and page based API to struct folio and the folio API. Allocate order 0 folios on the buffered I/O path and for the DIO bounce buffer. User pages used for DIO can be embedded in large folios so add support for identifying and tracking the page number inside a large folio from user pages. Introduce cl_page_batch* and clarify oti_cl_batch as pending cl_page objects available for cl_commit_cbt to release additional grant space in osc_queue_async_io() HPE-bug-id: LUS-12384 Signed-off-by: Shaun Tancheff <shaun@tancheff.com> Change-Id: I2c2554d3f5feaf7ec205f4b239bebf688140d874
LU-17482 llite: short read could mess up next read offset
When read reaches EOF, it could read data from stale pagecache, but
we need to restore the iocb->ki_pos so that next read could continue
from the correct offset.
Lustre-change: https://review.whamcloud.com/53827
Lustre-commit: 35fb413683affe9d41f3521259c260de6caf81d9
Fixes: 4468f6c9d9 ("LU-16025 llite: adjust read count as file got truncated")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ib8b62c41bf65f8efec82dda53fcfbdb68ad08b38
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Patrick Farrell <patrick.farrell@oracle.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
| unique failing test | history |
|---|---|
| recovery-small@ldiskfs:test_155 | seen in 7 other reviews |
(minor) Can a cached read really mark a DIO folio uptodate? A PagePrivate2 folio is allocated by folio_alloc() in osd_get_page() and never inserted into inode->i_mapping, so get_folio_create()/get_folio_lock() can never return it, and osd_get_page() calls folio_clear_uptodate() on every DIO folio just before handing it out. The only other places osd-ldiskfs sets PG_uptodate are dio_complete_routine() and osd_write_commit(), both of which run in an earlier RPC and are undone by that folio_clear_uptodate(). That leaves the OBD_FAIL_OST_FAKE_RW SetPageUptodate() two lines above the cache-hit test as the only in-tree way to reach this branch with a DIO folio. If there is another path, it would be good to name it here, because in that case skipping the read is itself wrong (see the comment on osd_io.c).
We can just remove that sentence instead...
(minor) This reproducer does not seem to match the mechanism above. Neither sanity test_155g nor test_63a sets fail_loc=0x238, and read_cache_enable=0 on its own only selects the DIO path - it does not make a DIO folio uptodate. LU-19778 is filed as "sanity test_428: osd_write_commit() ASSERTION( PageLocked(lnb[i].lnb_page) ) failed".
A sequence that does hold together, and matches the ticket:
test_399b (test_fake_rw read) truncates $tfile to up to 1GB.
fsize > od_readcache_max_filesize (default totalram/64), so
osd_bufs_get() picks cache=false and every lnb gets a DIO folio.
fail_loc=0x238 then makes every folio uptodate, so all of them take
the cache-hit branch and get unlocked; iobuf->dr_npages stays 0 so
the guarded early-release loop never runs.
test_428 later writes 128MB files (also over the filesize limit) on
the same service thread and trips the write-side LASSERT.
That also explains why this survived since LU-13309: it needs fail_loc=0x238 *and* the non-cached path, and test_399a/399b only take the non-cached path when the test file exceeds readcache_max_filesize.
Again, too much LLM-generated explanation is just causing more issues than it solves.
LU-19778 osd-ldiskfs: don't unlock DIO folios in osd_read_prep cache-hit path
DIO folios (PagePrivate2) are per-thread state kept locked in
oti_dio_folios[] for their lifetime. osd_get_page() only calls
folio_lock() on first allocation; reused slots skip the lock.
When any path marks a DIO folio uptodate before osd_read_prep() sees
it — e.g. OBD_FAIL_OST_FAKE_RW (0x238) calling SetPageUptodate(), or
a cached read hitting the same folio — the cache-hit branch fires and
unlock_page() is called on a folio that must remain locked.
On the next RPC the same thread reuses the (now unlocked) folio without
re-locking it. Two crash sites follow:
- Read path: if the file offset maps to a sparse region, osd_do_bio()
submits no bio, the early-release loop runs, and
LASSERT(PageLocked(page)) panics the server.
- Write path: if a subsequent write RPC obtains the same DIO folio,
osd_write_commit() hits LASSERT(PageLocked(lnb[i].lnb_page)).
This is the scenario reproduced by toggling read_cache_enable=0
while concurrent writes are in flight (test_155g + test_63a).
Guard the unlock_page() call with !PagePrivate2(), mirroring the
identical guard already present in the early-release loop below.
Fixes: e790df5fa38d ("LU-13309 osd-ldiskfs: speedup osd_bufs_get/put")
TLC-bug-id: TLU-184
Signed-off-by: Minh Diep <mdiep@thelustrecollective.com>
Assisted-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Change-Id: I58cecf2cdf89f41a01fdeb14e2e304e9b6588f94
LU-20078 osc: call osc_extent_tree_dump() without object lock held
The panic of next_extent() in osc_extent_tree_dump() is caused by:
- osc_extent_tree_dump() walks the extent rbtree without holding
object lock: osc_object_lock().
- LU-19014 added a new debug call to osc_extent_tree_dump() in
osc_cache_writeback_range() for the IO_PRIO_DIRTY_EXCEEDED path.
- next_extent() assumes the passed exctent is still in the tree
and asserts on RB_EMPTY_NODE(); This is reasonable for normal
tree walkers under the object lock, but unsafe for an unlocked
debug dump.
This patch fixes it by removing the debug code added by LU-19014:
osc_extent_tree_dump().
This patch also fixes the similar call of osc_extent_tree_dump() in
osc_enter_cache() by adding object lock.
Fixes: c413d2ede5 ("LU-19014 memcg: fix client hang in
balance_dirty_page()")
Change-Id: I7802d951f1c01d92a9240e7b4d395104b52c8969
Signed-off-by: Yingjin Qian <qian@ddn.com>
(typo) "aother" -> "another".
The body only describes the oe_hp linkage check in osc_cache_writeback_range(). It does not mention the second change in the diff: osc_extent_release() no longer does a synchronous osc_io_unplug() for high-priority I/O, plus the matching kernel-doc edit.
That hunk reverts part of c413d2ede5 ("LU-19014 memcg: fix client hang in balance_dirty_pages()") on the same ticket. Is it intentional, and should it be split into its own change with its own Change-Id so it can be reviewed and landed separately?
It would also help to state the user-visible symptom (writeback of the extent stalls, and osc_lock_flush()->osc_cache_wait_range() then waits on it) rather than only "inconsistent state".
This fixes a hang/stall, so it should carry a Fixes: tag. The skip-if-oe_hp branch this patch rewrites was added by 61a01fd9b689, which also introduced oo_hp_exts and the oe_hp bookkeeping; before it the code asserted !oe_hp instead of returning early.
Fixes: 61a01fd9b689 ("LU-17190 osc: client-side high prio I/O under blocking AST")
(minor) Since the function is unconditionally asynchronous again after this change, the word "asynchronously" is now accurate - it reads as if the removal went the wrong way.
Dropping the synchronous unplug for high-priority I/O reverses what c413d2ede5 added under this same ticket. On the ll_write_end() path, wb->dirty_exceeded sets prio = IO_PRIO_URGENT, which reaches here via vvp_io_write_commit()->cl_io_commit_async()->lov_io_commit_async()->cl_io_extent_release(). With osc_io_unplug() the writer ran osc_check_rpcs() in its own context and issued the RPC before returning to balance_dirty_pages(); now it only does schedule_work(&cli->cl_writeback_work). What makes the async unplug sufficient here now? If it is a fix for something (a lock-order or latency problem in the synchronous path), that reasoning is worth capturing in the commit message.
(typo) "could has been" -> "could have been".
list_empty() only distinguishes "on no list at all" from "on some list" - it does not say the extent is on oo_hp_exts. A cached extent can already be linked on oo_urgent_exts (set by an earlier non-hp writeback, or by osc_cache_truncate_end()) or on oo_full_exts (osc_extent_release() when oe_nr_pages == oe_mppr). For those, list_empty() is false, so list stays NULL and the extent never reaches oo_hp_exts even though hp was requested.
Before this patch the hp branch set list = &obj->oo_hp_exts unconditionally and list_move_tail() pulled the extent off whichever list it was on.
The caller that loses out is the blocking-AST path osc_lock_flush()->osc_cache_writeback_range(..., hp=1, ...). With no entry on oo_hp_exts, osc_makes_hprpc() is false and osc_check_rpcs() takes
if (osc_max_rpc_in_flight(cli, osc) &&
list_empty(&osc->oo_hp_exts) &&
list_empty(&osc->oo_hp_read_exts)) { ...; break; }
so the flush no longer bypasses cl_max_rpcs_in_flight - the case LU-17190 added the HP list for. osc_lock_flush() then blocks in osc_cache_wait_range() waiting for that extent.
Would keeping the hp request unconditional work, with the linkage test only for the "already marked by another thread" case?
if (hp) {
ext->oe_hp = 1;
list = &obj->oo_hp_exts;
} else if (ext->oe_hp) {
if (list_empty(&ext->oe_link))
list = &obj->oo_hp_exts;
} else if (!ext->oe_urgent) {
As written, setting ext->oe_hp = 1 before the test also creates a state the patch is trying to avoid: oe_hp set while the extent sits on the urgent or full list. Since osc_extent_find() then skips it, it can no longer be reopened for writes either.
LU-19014 osc: check the linkage of extent while oe_hp set An extent must be linked to its object when the oe_hp was set. However, it could be set before writeback. Meantime, aother thread might remove it from the list (e.g. truncate). So we would leave an extent in an inconsistent state. Signed-off-by: Yang Sheng <ys@whamcloud.com> Change-Id: Ie6f9cefcbf50c02c2fdf7a26fcde874675609d10
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-5 failed 3× | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
what's zerooffset? we have zero hits in the entirety of the codebase and commit messages.
Did we ever get a confirmation this is not going to affect application api / old/new library linked apps problems?
This should be giving a checkpatch error.
I would assume that the old `LLAPI_LAYOUT_RAID0 = 0` value would also be accepted? Otherwise, this will introduce a compatibility issue. Even so, this means that applications using the new library would break with ones compiled using the old headers. Maybe a better option would be to add `LLAPI_LAYOUT_RAID0_SET = 0x0000040ULL` that can be set and checked in the code, but still keep the `LLAPI_LAYOUT_RAID0 = 0` value for compatibility for a few years.
(style) should use `#ifndef BIT`
(style) spurious blank line
(style) line length of 95 exceeds 80 columns
It isn't clear what these lines are for? They should be removed.
my bad. some debuging.
(style) line length of 89 exceeds 80 columns
This can be wrapped easily
it's in 80 chars i think.
Is this change correct? Swaps from overstriping to & raid0? Definitely a semantic shift?
(defect?) this should be in the same order as lov_pattern_names[], since `setstripe` doesn't produce any output, so the `getstripe` output is what matters.
(minor) there is now `llapi_convert_str2mask()` that implements this in userspace, like `cfs_str2mask()` in the kernel. It needs a simple `bit2str()` helper function, instead of reimplementing it for each set it bitmasks. It would probably make sense for `llapi_lov_string_pattern()` and `llapi_lov_pattern_string()' to be converted over to use that as well.
LU-18461 llapi: replace pattern with bitmask use a bitmask as llapi_pattern, to simplify overstripe / zerooffset handing. Fix usage a LOV_PATTERN_PARITY on the userspace (llapi) code. Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com> Change-Id: I29a6943af1f58ed3834076d3bb3d618dd6b83983
LU-20446 man3: add llapi_find.3 and llapi_find_with_cb.3 Add llapi_find.3 and llapi_find_with_cb.3 man pages to document APIs llapi_find() and llapi_find_with_cb(), including traversal algorithms, matching conditions, work-stealing queue architecture for parallel mode, and usage examples. Test-Parameters: forbuildonly Signed-off-by: Emoly Liu <emoly@whamcloud.com> Change-Id: I043051307671449987d0511c7ad9a123bab1cafe
cfs_fail_val isn't used in the added tests, fwiw
the braces aren't necessary and could be removed; same for the CFS_FAIL_CHECK() below
Could this be cpu_to_le32(0xffffffff), same as below?
well, no, I wanted to make it looking "mostly fine"
Should the skip() use ">=" or maybe the version_code() should use ">"? Same in test_210d()
The version seems to have moved to 2.17.52
The version seems to have moved to 2.17.53
LU-19687 lov: skip foreign layouts for regular file IO If the client sees a bad layout in lsm_unpackmd_comp_md_v1(), for example if the OST index is invalid after the retry loop in lsme_unpack_comp()->lsme_unpack() waits for newly-added OSTs to appear in the client configuration (LU-17334) and fails, then lsm_unpackmd_comp_md_v1() changes the component type to use LOV_MAGIC_FOREIGN (in memory only). If a regular IO (such a write) meets a foreign component layout in lov_io_layout_at(), then just skip it instead of LASSERT(). Add sanity-flr.sh test_210c and test_210d to verify this case. Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Change-Id: I51c6dbc196d70d8d03a165b9877f7617b85dfac1
(minor) it is confusing to me that this is using `p` = parity and `m` = total stripe count. I thought the standard was `k` = data, `m` = parity, as with patch https://review.whamcloud.com/61965 ("LU-12187 lfs: add EC setstripe support with k+m notation").
It would be useful if this printed some information about the local CPU architecture (e.g. `model name` and `cpu MHz` from `/proc/cpuinfo` or equivalent from `lscpu`) and the EC implementation that was selected, so that we have some way to identify the results that are generated. Otherwise, we get a bunch of numbers and no way to identify how they relate.
LU-20016: Simple benchmark tool for EC computations Note that in the ISL-L library, computing parities and recovering lost data stripes is the same operation: ec_encode_data(). The difference is not in the computations but rather the content of the coefficient matrix. The cost of these operations are primarily bound by the number of stripes to compute which amusingly means that recovering a single lost stripe should be FASTER than generating two parities. Test-Parameters: trivial testlist=sanity-ec Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I61f409b631a8f4aefe0ceb2628e56f58f4c3a07c
LU-19009 llite: auto revise ra_page with read_ahead_kb=N settting Lustre explicitly disables the kernel readahead for the bdi at the mount time because it is incompatible with the Lustre-internal readahead (which takes into account the file striping, data consistency locking, etc). Setting read_ahead_kb explicitly will trigger the kernel readahead which is out the control of Lustre and result in the checking in ->readpage() failed: echo 1024 > /sys/devices/virtual/bdi/$FSNAME-*/read_ahead_kb. In the original PCC codes, we have already fixed the ra_pages of file read-ahead state for the read path and mmap I/O. In this patch, we improve it to also revise the bdi ra_pages automatically (which may set with non-zero value wrongly by user) once detect that the ra_pages is not zero. Add sanity/test_853 to verify it works as expected. Signed-off-by: Yingjin Qian <qian@ddn.com> Change-Id: I119ef6c6ce542e70f4ed59069c2756d5d39d1785
| unique failing test | history |
|---|---|
| runtests@ldiskfs+DNE:test_1 | seen in 11 other reviews |
| sanityn@zfs:test_51c | seen in 1 other review |
LU-17364 llite: don't use stale page. using stale page for write might confuse a read path, which expect any IO page have PG_uptodate flag set, and it caused an panic with removing from IO. Lustre-commit: dad3bed7617fba895db169facde91856e89c2b08 Lustre-change: https://review.whamcloud.com/53550 Test-Parameters: testlist=sanityn env=SLOW=yes,ONLY=16k,ONLY_REPEAT=10 Test-Parameters: testlist=sanityn env=SLOW=yes,ONLY=16k,ONLY_REPEAT=10 Test-Parameters: testlist=sanityn env=SLOW=yes,ONLY=16k,ONLY_REPEAT=10 Test-Parameters: testlist=sanityn env=SLOW=yes,ONLY=16k,ONLY_REPEAT=10 Test-Parameters: testlist=sanityn env=SLOW=yes,ONLY=16k,ONLY_REPEAT=10 Test-Parameters: testlist=sanityn env=SLOW=yes,ONLY=16k,ONLY_REPEAT=10 Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com> Change-Id: Ia01129ceaecf53d8d9f301c26cd2d65122f6a267 Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com> Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
| unique failing test | history |
|---|---|
| recovery-small@ldiskfs+DNE:test_24b | seen in 1 other review |
| replay-single1@ldiskfs+DNE:test_90 | seen in 1 other review |
LU-19113 llite: cfs_delete_from_page_cache() keep page locked
Like in other places where generic_error_remove_folio() is
also being called, in both Lustre and Kernel, page should not
be unlocked prior to call it in cfs_delete_from_page_cache().
This was also allowing a race where page->mapping may become
NULL.
Taking an extra reference is also useless if page not unlocked
anymore.
Lustre-change: https://review.whamcloud.com/59829
Lustre-commit: 3dec26990738ad3f5008d46ee9d3d03df7454925
Fixes: 738e69d4b9 ("LU-16292 llite: delete_from_page_cache not exported")
Signed-off-by: Bruno Faccini <bfaccini@nvidia.com>
Change-Id: If39575f4339afe460b3b1c955201e8f9cdfeb871
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@whamcloud.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
LU-16612 llite: protect cp_state with vmpage lock cl_page_make_ready() calls cl_page_io_start() without vmpage lock protection, and that could mess up cl_page's cp_state/cp_owner. Lustre-change: https://review.whamcloud.com/50180 Lustre-commit: d03b038d0dd8360dc896ceb7f3cee99245551cb8 Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: Id0df7e14246aa561494a9b6e581cebc55241c4b9 Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
LU-12325 ldlm: mode downgrade, wire changes Wire protocol changes to pass conflicting lock mode from server, so client may use it for better lock convert decisions. Second part of lock convert with mode: - add l_conflict_mode field in ldlm_lock. This is combined mode of conflicting server locks. - add l_convert_mode field in ldlm_lock. This is lock mode to convert to during lock convert. - split inodebits policy to in-memory and on-wire structures, some fields are not needed on-wire and in-memory structure shouldn't combine try_bits and cancel_bits in union. - use liw_ prefix for wire policy struct members - re-organize ldlm_bl_desc2lock(), add ldlm_bl_lock2desc() to pass lock convert info between lock and descriptor and allow mode convert for other lock types - update swabber and wiretest according with changes Test-Parameters: testlist=dom-performance Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Change-Id: Ib001216aa1812623871cb167d6cd6fcabdf38cb3
This struct contains data more than just state, and since it's for WBC files, is wbc_file_data better?
The name is bit misleading, maybe wbc_inode_info?
We already have pcc_inode, pcc_dentry naming for PCC. To keep the name consistency with PCC, IMHO, these names is minor, should be okey.
ditto, wbc_dentry_data?
LU-13047 wbc: embeded MemFS for the basic MetaWBC framework In the MetaWBC design, a memory file system (called MemFS for short, which is based on VFS and similar to ramfs or tmpfs) is embededded into the main Lustre file system which is based on the persistent storage backend. - I/O (data/metadata) first tries to write into MemFS; - Using writeback mechanism in Linux kernel, delay writing back dirty data from MemFS into the main file system Lustre; It can not only obtain the fast access speed of the embedded MemFS but also can maintain the data persistence. This is the MemFS code part of the basic MetaWBC framework. Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: If2c7acd7f3c3fdbf6ba721ef658b5db43f2e83e6
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_39d | seen in 23 other reviews |
LU-17583 mdt: don't fetch LOOKUP lock for getattr/open by fid LOOKUP lock is used to revalidate client dentry so we should only return one if the MDT does a directory lookup. This causes the sanity-hsm 408 failed because there are no longer conflicts between OPEN_RW|LOOKUP cached lock and the copytool LOOKUP|OPEN_RO lock. A client close request for a write open handle can be sent after the file is archived and marked the file as dirty. The patch fixes this server-side by revoking client open write locks when handling an archive request. Client-side, we use early cancel for local open write locks when sending an archive request. Test-Parameters: testlist=sanity-hsm env=ONLY=408,ONLY_REPEAT=20 Signed-off-by: Etienne AUJAMES <eaujames@ddn.com> Change-Id: Id5807a1ba3c8b563405c89325083247f6d666102
LU-16789 utils: use AIO for 'lfs migrate' Create a mini async IO framework with ll_aio_ prefix. The framework is a wrapper layer of libaio to run asynchronous IO. However, it can also run traditional synchronous IO with POSIX APIs without any code change if io queue is initialized with sync_mode=true. Or if libaio is not availble on the running system, the framework will fallback to synchronous mode automatically. Create function ll_aio_copy() to copy data from one fd to another. The function can work in async/sync mode, with different buffer size and different number of async IO tasks. The function also integrates checking file, stats report and IO throttling. Due to LU-18032, the last chunk of file may not be aligned so cannot be read/written by libaio. So the last partial chunk will be read/written in sync mode with POSIX API in sync mode even if ll_aio_copy() is called in async mode. In async mode, 2 more fields are appended to stats report: read_tasks and write_tasks, which indicate how many tasks are pending on reading/writing data when the stats is printed. Replace migrate_copy_data() with an AIO version of migrate_copy_data_aio() based on ll_aio_copy(). Command 'lfs migrate' and 'lfs mirror extend' will benefit from it. Add --aio-tasks=TASKS parameter for 'lfs migrate' and 'lfs mirror extend' command. It tells how many AIO tasks will be started to transfer data. But if --aio-tasks=0, the IO will fallback to synchronous mode. All the read/write will be performed in synchronous mode sequencially. Add --chunk-size=SIZE_MB parameter to specify the buffer chunk size in MB used to transfer data. Signed-off-by: Feng Lei <flei@whamcloud.com> Test-Parameters: trivial Test-Parameters: testlist=sanity-flr Change-Id: I50521db8e84d745edaaad791cb44fc3107ae411d
LU-13047 wbc: foundation framework for WBC This patch refined the foundation framework for WBC based on Oleg's implementation. Also it implements the simple lazy flush mode for WBC. Test-Parameters: testlist=sanity-wbc clientdistro=el9.3 Test-Parameters: testlist=sanity-wbc clientdistro=ubuntu2204 Test-Parameters: testlist=sanity-wbc,sanity-wbc Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I612136b8297ccfdbfdeca1e4d271fe94f511f645
(style) prefer (( ... )) for numeric comparisons
(style) prefer (( ... ))
Do we have equivalent nodemap-based root squash tests somewhere? It would also be useful to update the LOM to indicate that this set_param/conf_param mechanism is deprecated and to use nodemap instead (preferably referencing a section that explains the details of how to do it).
We do. See for instance sanity-sec test_15 and test_17. Regarding LOM, let's discuss in dedicated patch #46740.
Is it possible to create a test to test that we can still modify legacy root squash settings? Perhaps using a fail_loc to disable the new check? I get this slight nervous feeling testing only one direction that we might break this in some distant future :)
To be clear 'using a fail loc to disable the new check' so we can create the legacy setting for testing
It makes sense, I will add this fail_loc and update the test.
LU-15656 sec: deprecate legacy root squash The legacy root squash mechanism relies on 2 parameters: - the root_squash parameter specifies the UID and GID to which the root user is squashed when accessing the Lustre file system; - the nosquash_nids parameter specifies the set of clients to which root squash does not apply. This legacy mechanism has become redundant with the ability provided by the nodemap feature to implement root squash, in a much more flexible way. Sor for new file systems, prevent setting the root_squash and nosquash_nids parameters. For upgraded file systems, maintain the ability to change the already set values. Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I439f2ad2272592da73066c7f8edf8b7a360a2286
| unique failing test | history |
|---|---|
| conf-sanity2@zfs:test_38 | seen in 3 other reviews |
| conf-sanity2@zfs:test_39 | seen in 7 other reviews |
| conf-sanity2@zfs:test_40 | seen in 7 other reviews |
| conf-sanity2@zfs:test_41a | seen in 5 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18e | seen in 3 other reviews |
LU-16033 llite: refine ll_use_fast_io() for stride mode This patch refines ll_use_fast_io() to detect whether current readahead window allows issuing readahead RPC for stride mode. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: Id49d947a0433f94bc7b63fe4d528d7c32b7b789f
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_418 | seen in 30 other reviews |
| sanity-pcc@ldiskfs+DNE:test_1a | seen in 1 other review |
| sanity-pcc@ldiskfs+DNE:test_1b | seen in 3 other reviews |
| sanity-pcc@zfs:test_1a | seen in 2 other reviews |
| sanity-pcc@zfs:test_1b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-16285 ldlm: output the conflict lock Print out the detail info while lock BL timout so we know which lock conflict with it. Signed-off-by: Yang Sheng <ys@whamcloud.com> Change-Id: I5698341a694d64aabea0f28f97216ccbd57f5641
LU-16538 llite: use lock flags to distingush OPEN ibits lock In the original Lustre design, an OPEN ibits lock returned to a client is mandatorily bound to a specific lock mode: - FMODE_WRITE CW - FMODE_EXEC PR - other CR For DoM files, it would better to return DOM bits combined with OPEN bits (DOM|OPEN) in an ibits lock to a client during open(). It can reduce the lock traffic for the possible subsquent read (PR lock mode) or write (PW lock mode). Thus, three lock flags are defined to distingush the OPEN ibits lock with different open modes: open_mode lock_flag lock_mode - FMODE_WRITE LDLM_FL_OPEN_WRITE CW or PW - FMODE_EXEC LDLM_FL_OPEN_EXEC PR - other LDLM_FL_OPEN_READ CR or PR These new lock flags are for internal use on the client only and declared in "lustre/include/lustre_dlm_flags" and store in @ldlm_lock->l_flags. Thus, this is a client-side patch. Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I921c42a557760d5485b4c28156c74a2b5bda2f91
I thought it made sense to have the OST inode checking be its own test, rather than add it into check_seq_oid(), so that it can easily test multiple settings for the parameter. Right now I've duplicated the logic of parsing "lfs getstripe" output to get the object's filename -- do you think would it be worthwhile to try to refactor this so that the 2 tests call some common function do to this?
does this need "skip parallel run"? I saw that the check_seq_oid() test (27z) has it. I wasn't sure if that's because the usage of "sync" might interfere with other tests, or something.
LU-13031 ofd: add jobid xattr to ost object
This stores the jobid of the process that creates an object in an
extended attribute in the OST inode for that object. The name of the
extended attribute is determined by a new sysfs parameter
"obdfilter.*.job_xattr". The default value is "user.job". A value of
"NONE" means that the jobid will not be stored.
This builds on 23a2db28dcf1 ("LU-13031 jobstats: store jobid in xattr
when files are created"), which stores the jobid in the MDT inode
for the file. Note that the jobid need not be the same on the MDT
inode and the OST inodes, because the process that creates a file
might not be the same process that performs the initial write to a
given stripe.
Signed-off-by: Thomas Bertschinger <bertschinger@lanl.gov>
Change-Id: I9ab1aa394def3869c8d87a252bd9f95962f214f6
| unique failing test | history |
|---|---|
| sanity1@zfs:test_27T | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
LU-13214 llite: pass error to upper layers vvp_io_write_start ignores error when some bytes were written. For a stripe case this could lead to a wrong data and wrong assumptions at user level. For example write at two stripes, first is DOM was written 1 page from 16, seocond is OST and was fully written. Data is [0:4096]-hole-[64K:1M], user level see successfull write for 4096 bytes, and file position at the 1MB. The patch passes error to the upper layer to notify caller about write error. Also sanity test 281 is added. Test simulate MDT enospace, this leads to no grants on a client, and partial fail write data-hole-data. HPE-bug-id: LUS-8414 Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com> Change-Id: I56ff460ca536549009ff2105669f4eed37021dde
LU-18169 llite: wait unstable pages to be committed during umount The client must wait unstable pages to be committed to the stable storage during umount. If a data OBD is in disconnected state during umount, the client will wait for a certain time (30 seconds, by default) at most, and then force to deactive the OBD to cleanup the uncommitted unstable pages. Otherwise, it will fail the replay-single.sh/test_89. Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I307b5e6063f6d726e5f84018ebeecd58bac860ab
LU-18169 osc: add accounting for WB_WRITEBACK for memcg This patch adds accounting of unstable pages for WB_WRITEBACK per inode. Moreover, we only need to check unstable pages for buffered I/O as an I/O requests for direct I/O is already committed to the stable storage on the server when the client receives the reply. Thus, there is no need to do unstable pages accounting for direct I/O. Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: Ibcc0c02d6e1a0d3d678edd5fa2941a31a08343e9
LU-10499 pcc: asynchronous PCCRO attach command support
Currently PCCRO attach via the command "lfs pcc attach" will block
during the data copying.
There is a requirement that this command can also do data copy
asynchronously. Thus we add an option "--async|-A" to the command
which will not block while the file data is being fetched.
Add sanity-pcc/test_{103, 104} to verify that it works correctly.
EX-bug-id: EX-6373
Change-Id: I6f31190c8b9e9b9876b34f8e484c6c8b7f16b6db
Signed-off-by: Qian Yingjin <qian@ddn.com>
cl2osc() is container_of_safe() on oo_cl, which sits at offset 0, so IS_ERR_OR_NULL(cl2osc(obj)) is the same test as IS_ERR_OR_NULL(obj). It checks the cl_object pointer that was passed in, not whether the osc_object still exists; an object that was freed while still referenced is an ordinary pointer, not NULL or an ERR_PTR. Could the message name the pointer that actually becomes NULL or an error, and where it is set that way?
ldlm/ldlm_resource.c: warn: ldlm_resource_get():inconsistent indenting
This paragraph explains the bucket-array sizing change, but not that the bucket selection function changed shape as well. ldlm_res_hop_hash() summed all four words of the res_id; ldlm_res_hop_fid_hash() hashes only the FID in name[0] and name[1]. Worth a sentence, since it changes which resources share an ldlm_ns_bucket. The new LDLM_FL_KUNIT_TESTING flag in lustre_dlm_flags.h isn't mentioned anywhere in the body either - is that meant to be part of this patch, or a separate cleanup of the bare BIT(63) uses?
ldlm/ldlm_resource.c: warn: ldlm_resource_get():inconsistent indenting
lr_hash and lr_rcu share storage, so call_rcu(&res->lr_rcu, ...) writes over lr_hash.next. That was safe with cfs_hash because every chain walk held the bucket lock; rhashtable readers walk the chain under rcu_read_lock() alone. ldlm_resource_putref() -> __ldlm_resource_putref_final() does rhashtable_remove_fast(), then ldlm_resource_free(res, true) -> call_rcu(), and call_rcu() sets head->next = NULL right away. RCU removal requires the removed node's next pointer to stay intact for a grace period, because a reader can already be sitting on that node when it is unlinked. A concurrent ldlm_resource_get() walking that bucket then loads NULL as the next pointer. rht_is_a_nulls() only tests bit 0, so NULL is not recognised as end-of-chain: the loop body runs with pos == NULL, rht_obj() subtracts head_offset, and rs_cmp() dereferences the result. struct lu_object_header keeps loh_hash and loh_rcu as separate fields, with a comment about exactly this requirement. Should lr_rcu be split out of the union the same way?
This isn't a bug, but BIT() is unsigned long while l_flags is __u64, so BIT(63) is out of range on a 32-bit build. Every other flag in this file spells the value out as 0x...ULL with a /* bit N */ comment and provides ldlm_is_/ldlm_set_ helpers - consider matching that:
#define LDLM_FL_KUNIT_TESTING 0x8000000000000000ULL /* bit 63 */
#define ldlm_is_kunit_testing(_l) LDLM_TEST_FLAG((_l), 1ULL << 63)
#define ldlm_set_kunit_testing(_l) LDLM_SET_FLAG((_l), 1ULL << 63)
which would also let the two ldlm_lock.c users read like the surrounding flag tests.
This gives lock_res_and_lock() a new failure return, but only 8 of the 111 call sites look at the result; the rest go straight on to unlock_res_and_lock(), which is unlock_res(lock->l_resource) on the very pointer that produced the error. As far as I can tell the error can't fire today: l_resource is only cleared by the rcu_assign_pointer(lock->l_resource, NULL) in ldlm_lock_put(), which runs after h_ref has already reached zero, so no caller that holds a lock reference can observe it, and nothing assigns an ERR_PTR to l_resource at all. So either it is unreachable, and the error plumbing added through ldlm_lock_enqueue(), ldlm_callback_handler(), osc_ldlm_glimpse_ast() and osc_extent_release() could go, or it is reachable and the other 103 callers need converting in the same patch. Which is it?
ns_reclaim_start is now only ever assigned 0 in ldlm_namespace_new(); the ldlm_res_to_ns(res)->ns_reclaim_start++ that used to advance it went away with the per-bucket bookkeeping in ldlm_reclaim_lock_cb(). rcd_start is therefore always 0, the rcd_skip/rcd_cursor fast-forward below can never trigger, and every reclaim pass restarts on the same resources. nsb_reclaim_start is incremented but no longer read either. That drops the round-robin scan the ldlm_reclaim_res() comment still describes, and it is not mentioned in the commit message. Is the intent to keep the round-robin, or to remove it and the now-dead rcd_skip/rcd_cursor/rcd_start fields with it? Also worth noting the units no longer match: rcd_cursor counts resources while tbl->size is a bucket count.
(minor) This comment doesn't match the new scheme. With the rhashtable the table holds no reference of its own: an entry is removed exactly when `lr_refcount` hits zero in `__ldlm_resource_putref_final()`, and `rs_cmp()` hides zero-refcount entries from lookups. During the walk the only extra reference is the `refcount_inc_not_zero()` in `ldlm_resource_for_each()`. So the count printed below is one lower than the number of stray references, and a resource holding exactly one leaked reference no longer produces the message at all.
(style) This is a new exported API and it has no kerneldoc, unlike `ldlm_resource_get()` right below. Worth documenting the callback contract, in particular that the callback owns no reference (it must not putref), that a non-zero return stops the walk, and that a callback can be invoked more than once for the same resource - `rhashtable_walk_start()` here is the void variant, so the `-EAGAIN` a resize produces is discarded and the walk restarts from slot 0. That last property is what forced the `l_pending_chain` check in `ldlm_chain_lock_for_replay()`, and every future callback author needs to know about it.
The comment says the remaining reference is the resource being in the hashtable, but the table holds no reference: ldlm_resource_putref() removes the entry exactly when lr_refcount reaches 0. The extra count seen here is the one ldlm_resource_for_each() took before calling the callback, so a resource with a single leaked reference reads 2 and is skipped, which is the case this function exists to report, and the printed count is one low. The old cfs_hash iterator also held a reference during the callback, which is why the previous code used -1. Should this be <= 1 and refcount - 1?
Changing this in the past broke things. Not ready to try this change.
Hashing only the extracted FID drops name[2] and name[3] from the bucket choice, and those are significant for some resource types:
fid_build_quota_res_name(): name[2]/name[3] = quota id
mdt_handler.c: name[LUSTRE_RES_ID_HSH_OFF] = lh->mlh_pdo_hash
lfsck_lib.c: name[LUSTRE_RES_ID_HSH_OFF] = full_name_hash(...)
Every per-ID quota resource under one global-index FID now lands in a single ns_rs_buckets[] entry, and likewise every PDO/LFSCK name lock under one parent FID. The lookup hash is unaffected (rhashtable keys on the full ldlm_res_id), but nsb_count and nsb_at_estimate are per-bucket, and nsb_at_estimate feeds obd_at_get()/obd_at_measure() via ldlm_lock_to_ns_at(). Was collapsing those onto one bucket intended?
LU-8130 ldlm: convert ldlm_resource hash to rhashtable Using an rhashtable allows lockless lookup at the cost of rcu freeing of entries. When we find an entry, we need to atomically check the reference hasn't dropped to zero. When adding an entry, we might find an existing entry which is in the process of being removed - with a zero refcount. In that case we loop around and repeat the lookup. To ensure this doesn't spin, the 'cmp' function will fail any comparison with a resource which has a zero refcount. Now that we are using resizing hash tables, we don't need to preconfig suitable sizes for each namespace. We can just use the default and let it grow as needed. We keep the pre-configured sizes for the bucket array. Previously the size of the bucket array was the difference between nsd_all_bits and nsd_bkt_bits. As we don't need nsd_all_bits any more, nsd_bkt_bits is changed to the number of bits used to choose a bucket. Walking an rhashtable requires that we manage refcounts ourself, so a new function, ldlm_resource_for_each() is added to do that. Note that with this patch we now update a per-table counter on every insert/remove, which might cause more contention between CPUs on a busy system. Hopefully rhashtable will be enhanced in the near future to support a per-CPU counter for nelems. Only use call_rcu() to free slab resources that have been removed from the rhashtable, other cases can be free immediately. This change exposes new race conditions in the osc layer so we add code to check if osc_objects still exist. Signed-off-by: NeilBrown <neil@brown.name> Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Signed-off-by: James Simmons <jsimmons@infradead.org> Change-Id: Ie65f6c5f6e246ed6684ade05ddab8740ac8137dd
Fixes: ...?
The body only covers the fault-in return value, but the diff also restructures the copy loop: `while (true)` becomes `while (count)`, `short_copies` is now reset after every full-page copy, and the trailing LASSERTF changes from `cdp_page_count - 1` to `cdp_page_count`. The `short_copies` reset in particular looks load-bearing - without it a multi-page copy that needs one re-fault per page would trip the `short_copies > 2` limit. Could the message describe these changes too, so a reader can tell they are intentional?
A Fixes: tag was asked for on patchset 7 and is still missing.
The incorrect handling of the return value came in with the compat macro and the check itself:
Fixes: 7194eb6431d2 ("LU-13805 clio: bounce buffer for unaligned DIO")
I guess "left" is misleading here because just liek Claude says, count is what's left to copy, and original_count - count is "copied"?
From Claude:
(defect) left is set to original_count - count, which is the number of bytes already copied, but the re-fault wants the bytes still to copy, i.e. count. The top-of-function fault-in at the WRITE check uses count for exactly this.
With left = original_count - count, the first loop iteration is broken: if folio_from_iter() returns 0 on the first page (atomic copy of a non-resident page under memory pressure, the case this patch targets), count is still original_count, so left == 0. Then ll_iov_iter_fault_in_readable(iter, 0) returns 0, ret == left holds (0 == 0), and it breaks with -EFAULT before ever retrying. Since nothing was copied, out: returns status, so the write fails with -14 - the symptom LU-19458 is meant to fix.
Did you mean:
ret = ll_iov_iter_fault_in_readable(iter, count);
if (ret == count) {
status = -EFAULT;
break;
}
This isn't a bug, but the comment no longer matches: a partial fault-in is now accepted and only a completely failed one bails out. Maybe "try to fault in the userspace iovec; a partial fault-in is fine"?
`ll_iov_iter_fault_in_readable()` has two definitions with different return conventions:
#ifdef HAVE_FAULT_IN_IOV_ITER_READABLE
-> fault_in_iov_iter_readable() /* size_t, bytes NOT faulted in */
#else
-> iov_iter_fault_in_readable() /* int, 0 or -EFAULT */
The `== count` test only makes sense for the first one. On kernels taking the `#else` branch (config/lustre-core.m4 puts the cut at 5.15, so el8 and its 4.18 kernel land there, and lustre.spec.in still requires kernel >= 4.18) the return is 0 or -EFAULT; -EFAULT promoted to `size_t` can never equal `count`, so this check and the one in the loop body are both dead there and a genuine -EFAULT is dropped.
Would it work to normalise in the macro instead - have the pre-5.15 variant return `bytes` on -EFAULT and 0 otherwise - so both call sites mean the same thing on every supported kernel?
LU-19458 obdclass: fault_in_readable() can be short kernel's fault_in_iov_iter_readable() can be short due to memory limitations (e.g. close to OOM), but we can still make progress by smaller chunks. Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Change-Id: I20487a8766823281100cf26fbf121c71380ab66c
LU-12669 ec: recover data from parity
Restore read data when some OSTs are unavailable from updated parity
codes.
When normal read fails, the read would switch to CIT_EC_RD, and the
ec read inner IO would expand to cover the whole raid set stripes, the
extent lock of the ec_rd would cover the available data OSTs, then
it reads pages from available data objects and parity objects,
calculates and fill in the missing data pages so that the original
outer IO can pick them up and copy to user buffer.
The ec read outer IO is to keep track of the original read position
and count, and the ec read inner IO is to expand the IO to cover the
whole raid set stripes.
lov_ec_read_stripe_pages() would read all pages for one data stripe
across all page positions in a recovery group in a single
cl_io_submit_sync call. And lov_ec_read_parity_stripe() would read
all pages for one parity stripe in a single cl_io_submit_sync call.
Also catch DIO read failure and switch to CIT_EC_RD to recover data
from parity (buffered IO).
For EC recovery reads, the page beyong the end_index has no DLM lock
(EC recovery only locks the actual read range), so return -EIO to stop
the kernel from retrying. And for EC recovery read, verify DLM lock
coverage for every page, pages on good stripes need to be read from
OSTs and have locks, pages on deactivated stripes should already be in
the page cache from EC reconstruction. If a page without lock coverage
reaches ll_readpage(), it maps to a failed stripe -- return -EIO to
prevent LBUG in osc_req_attr_set().
Introduce a new cl_page_alloc_sub() to allocate a cl_page for a
sub-object. The sub-object is the lovsub object for a parity stripe,
the difference it's from cl_page_alloc() is that the page index of the
sub-object is relative to the parity object, not to the file.
This patch also fixes a resync crash on overstriped parity mirrors. If
a file's parity mirror was set overstriped ("-C, --overstripe-count"),
the lov_pattern_supported/available() check failed causing that mirror
not being created and the write to fail.
Fixes: e90c6a428e27 ("LU-12188 uapi: add LOV_PATTERN_PARITY")
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I74977a9148256e2604d948979b5f0b944786e4e0
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: lustre-rsync-test. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: lustre-rsync-test. | session |
| review-dne-zfs-part-5 | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: lustre-rsync-test. | session |
The body only describes the grant/@tmp reset in osc_queue_async_io(). Three other pieces of the diff aren't mentioned: the new OBD_FAIL_OSC_EXTENT_RESTART (0x41a) definition, the two CFS_FAIL_CHECK() calls added to the extent state check in osc_queue_async_io(), and sanity test_64k. The fault-injection hunks change a production code path, so could the message say what the injection point is for and that a test was added?
(typo) "accrodingly" -> "accordingly".
I am not comfortable with this goto label skipping the if condition... Ah, I see you're setting grants == 0 below - so, just move it up above the if and it's good
(defect) The label is placed below the write-commit folio_batch flush, but the comment right above it says osc_enter_cache() must not run while page locks are held:
/* We must not hold a page lock while we do osc_enter_cache()
* or osc_extent_find(), ... */
if (folio_batch_count(fbatch)) { cb(env, io, fbatch); ... }
When the restart is taken on an extent that came from `oio->oi_active` (the osc_extent_expand() case named in the commit message), the `if (ext == NULL)` block was never entered on the way in, so the batch was never flushed and still holds pages owned by this io. write_commit_callback() is what disowns them, and cl_page_make_ready() takes lock_page() on exactly those pages when the extent is turned into an RPC. Jumping to a label below the flush means the newly added osc_enter_cache() now sleeps waiting for writeback grant with those pages still locked. Should the flush move above restart_find?
(minor) CFS_FAIL_CHECK() is not a pure predicate - it bumps the fail counter and can set CFS_FAILED/CFS_FAIL_ONCE - and it is evaluated twice here for what is a single decision. Which of the two fires depends on which branch short-circuits (`ext->oe_state != OES_ACTIVE` skips the first, `ext->oe_state == OES_CACHE` skips the second), so with CFS_FAIL_SOME the counter is consumed differently depending on the extent state. Evaluating it once into a local bool would make the fail_val semantics predictable.
(minor) osc_exit_cache() already does exactly this (take cl_loi_list_lock, osc_release_write_grant(), drop it) and is used a few lines up on the osc_extent_find() error path. Reusing it here would keep the two grant-release sites identical.
(defect) `ext` is not cleared before jumping back, and the label now sits above the osc_enter_cache() call, so a failed re-entry leaves the released extent live:
restart_find:
if (grants == 0) {
rc = osc_enter_cache(env, cli, osc, oap, tmp);
...
}
tmp = grants;
if (rc == 0) {
ext = osc_extent_find(...); /* skipped when rc != 0 */
}
osc_enter_cache() returns -EDQUOT on forced sync i/o (`cl_dirty_max_pages == 0`, `ar_force_sync`, OBD_FAIL_OSC_NO_GRANT) and also after the grant wait times out. When that happens osc_extent_find() is skipped, `ext` still points at the extent that was just released, waited on with osc_extent_wait() and dropped with osc_extent_put(), so the `if (ext != NULL)` block below runs on it: EASSERTF() reads a possibly freed osc_extent, and LASSERT((oap->oap_brw_flags & OBD_BRW_FROM_GRANT) != 0) is guaranteed to fire because the credit was released just above and never reacquired.
Before this change the label was below the osc_enter_cache() block and osc_extent_wait() only returns <= 0, so `rc` was always 0 at the label and `ext` was always reassigned. Should this set `ext = NULL;` before the goto so a failed osc_enter_cache() just returns rc?
Does this case actually fail without the fix? This dd is a fresh io, so `oio->oi_active` is NULL and osc_queue_async_io() takes the `ext == NULL` path: osc_enter_cache() succeeds and `grants` is chunksize + cl_grant_extent_tax when the injected restart is taken. The old label sat below that block, so the retry called osc_extent_find() with *grants already >= chunksize + tax and the assertion in the ticket could not fire. The 0/4096/24576 LBUG needs `grants == 0`, which only happens on the `oio->oi_active` / osc_extent_expand() path - i.e. two or more pages committed in one io so a prior page has already installed oi_active. The cur_dirty_grant_bytes check below looks like it would pass either way too: on the old code the extra osc_unreserve_grant() subtracts from cl_reserved_grant, while cl_dirty_grant is still balanced by the matching osc_free_grant() when the extent completes.
LU-19709 osc: fix LASSERT failure on osc_extent_find()
The customer hits the following LBUG on a client:
(osc_cache.c:735:osc_extent_find()) ASSERTION( *grants >=
chunksize + cli->cl_grant_extent_tax ) failed: 0/4096/24576.
The reason is that an OES_ACTIVE extent being used by a writer
may be written back and changed into the other state at any
time.
If the original extent (@ext) comes from @osc_extent_expand(),
the value of @grants will be set with 0.
This may result in the panic at the above assertion when try to
restart finding a new extent in @osc_extent_find().
The patch resets @tmp value and @grants value by calling
@osc_enter_cache() and compensate the accounted grant, fixing the
panic accrodingly.
Fixes: c413d2ede5df ("LU-19014 memcg: fix client hang in balance_dirty_pages()")
Signed-off-by: Chris Horn <chorn@ddn.com>
Signed-off-by: Yingjin Qian <qian@ddn.com>
Change-Id: I83b750b2a35d97a52072a4cee0dd4097edec6242
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-4 | RHEL 10.1 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-6 | RHEL 9.7 / x86_64 | ran 4 tests. 1 tests failed: ost-pools. | session |
| review-ldiskfs | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 6 tests. 1 tests failed: sanity. | session |
I haven't yet looked into the details of this patch, but I'm wondering if the same effect could be approximated by changing the existing SLV mechanism and values returned to the client without implementing a new protocol? I'm not against a new protocol, but having a "partial" implementation that turns on (or potentially *off*) based on the number of clients could be quite difficult to test and measure the effects. I read the concern about the new mechanism imposing more lock pressure on a small number of new clients, but it seems possible that this could still co-exist with older clients that use the existing SLV mechanism (with fixes).
Unfortunately this comment was marked resolved without an answer.
hhh, answer back. > Can the same effect be approximated by changing the existing SLV values, without a new protocol? Partially yes, on the server side, we can try to drive the legacy scalar SLV from pool occupancy (pressure high → SLV lower), still returning a scalar value rather than a ratio, so existing clients can trigger cancellation without any wire-format change. However, this is only a best-effort approximation. As Wenzhao noted in the earlier discussion, the fundamental limitation of the legacy path is that each client compares its own CLV against a `single server-provided threshold`. There is no notion of reclaiming a fraction of the client's unused cache. In practice, clients with similar lock ages tend to keep similar numbers of locks, regardless of how large or active they are — a small client and a large client receive the same SLV and apply the same cutoff logic locally. Conclusion We do need the new contract: one OBD_CONNECT2 flag and reuse of the existing 64-bit slv field to carry packed pool pressure, with client-side eviction driven by a configurable curve over the client's own unused locks. That is what gives bounded, paced, proportional reclamation. At the same time, the old clients could/should participate in backpressure without waiting for a rollout threshold: - clients that negotiate OBD_CONNECT2_POOL_SLV_PACKED always receive packed pressure; - legacy clients continue to receive a scalar SLV, optionally occupancy-driven as a fallback; - both paths respond to the same underlying pool pressure, without a coverage-based switch. We accept the known limitations of the legacy fallback (equal-threshold behavior across clients of different sizes). The new contract addresses those limitations; the legacy path is there so mixed-version clusters still get timely cancellation, not to replace the pressure mechanism.
Review note: The patch is large because of negotiation, dual SLV, rollout, and tests. The new logic is small: server pressure is essentially granted/limit; the client maps pressure to an evict quota and uses a quota-based LRU policy instead of per-lock CLV arithmetic. Suggested review order: 1) lustre/include/lustre_dlm.h, lustre_idl.h — packed SLV layout and pool fields 2) ldlm_pool_calc_slv_by_pressure(), ldlm_pool_pressure_evict_*() — core math (~100 lines) 3) ldlm_cancel_pressure_policy(), ldlm_cancel_lru_policy() — reclaim behavior 4) connect, rollout, sysfs, tests — compatibility and rollout (bulk of the diff)
Not a bug, but the MDT name in this sample output is missing the UUID suffix. mdt_init0() builds the namespace name as "%s-%s" of LUSTRE_MDT_NAME and obd_uuid.uuid, so it reads mdt-testfs-MDT0000_UUID, matching the filter-testfs-OST0000_UUID line just below.
Currently, `ldlm_pool` tracks metrics at the per-target level (e.g., 'fsname-target'). I propose aggregating these metrics by 'fsname' instead. Furthermore, we should discuss whether resources should be allocated equally across all file systems, or if we need a weighted policy.
(style) Not a bug, but with `pressure_adjust == true` this just returns `L` unchanged, and the only caller passing true is the CDEBUG in ldlm_pool_calc_slv_by_pressure(). A boolean that turns the function into the identity is hard to follow - would printing `limit` directly there be clearer?
(style) These six new helpers are the only EXPORT_SYMBOL()s in this file; ldlm_pool_get_slv(), ldlm_pool_init() and friends aren't exported. ldlm_lib.c and ldlm_request.c build into the same ptlrpc module (lustre/ldlm/Makefile), so none of them need exporting, and this one is used only inside ldlm_pool.c so it could be static.
Once the pool is pressure-ready these two limits diverge by roughly 5x, and the legacy scalar looks like it collapses to the floor.
ldlm_pools_recalc_task() sets pl_limit = ldlm_ratio2locknr(30)/nr_ns for a ready pool but pl_legacy_limit = LDLM_POOL_HOST_L/nr_ns. HOST_L is 50 locks per MB of RAM, i.e. RAM/20971 locks; ldlm_ratio2locknr(30) is 0.30*RAM/LDLM_LOCK_MEM_OVERHEAD. Packed clients only start evicting at 50% of pl_limit (default curve), so granted settles well above pl_legacy_limit.
With granted > limit here:
grant_usage = max_t(int, limit - (granted - grant_plan), 1); /* -> 1 */
slv_factor = (grant_usage << 10); do_div(slv_factor, limit); /* -> 0 */
slv = slv * slv_factor; /* -> 0 */
and the clamp below lifts it to ldlm_pool_slv_min() == 1. In ldlm_cancel_lrur_policy() slv == 1 means every lock with lv >= 1 (any lock idle for a second or more) is cancelled, so a non-packed client drops essentially its whole LRU on each recalc, and the drop also trips the ns_recalc_pct urgent-recalc path in ldlm_cli_update_pool().
That hits up to 30% of client exports at the default threshold, plus every MDT OSC on an OST since lod never negotiates OBD_CONNECT2_POOL_SLV_PACKED. Does "keep legacy behavior for old clients" still hold once the pool flips, or should the legacy scalar be derived from the limit the pool is actually being filled to?
pl_slv_mode is set to pressure_rollout two lines up, so this branch always runs and a brand new server pool gets the pressure budget before any client has negotiated packed SLV and before ldlm_pool_fallback_legacy_on_multi_fs_register() has had a chance to force the pool back to legacy_slv. It is also the whole-node budget: ldlm_pools_recalc_task() divides ldlm_ratio2locknr() by ldlm_namespace_nr_read(LDLM_NAMESPACE_SERVER), this does not. The neighbouring initialisers (pl_legacy_limit, pl_grant_plan) still use LDLM_POOL_HOST_L, so pl_server_lock_volume is seeded from one basis and decayed against another until the first recalc pass. Would it be simpler to seed with LDLM_POOL_HOST_L here and let ldlm_pools_recalc_task() apply the pressure budget when the readiness gate says so?
This could be: for ((batch = 0; batch < BATCH_COUNT; batch++)) ...
(style) This commented-out curve set/restore block looks like leftover scaffolding - drop it, or enable it if the suite is meant to cover a non-default curve?
(style) Carried over from patchset 9 and still open: this reads more naturally as `for ((batch = 0; batch < BATCH_COUNT; batch++))`, with subdir_idx computed from batch directly.
LU-7266 ldlm: introduce pressure-based SLV Legacy SLV/CLV reacts to server pressure indirectly and can delay lock reclaim. This change introduces a pressure-based SLV path: server sends packed pressure in SLV, and client converts it to evict quota. Packed SLV Format: * Bits 63..60: format version (current 0x1). Legacy SLV leaves these bits clear. * Bits 59..16: reserved for future use. * Bits 15..0: pressure in hundredths of a percent, where 10000 means 100.00%. Here pressure is total-grant-lock/total-limit. Unlike the previous grant-plan-based scalar SLV. Rationale: - Percentage-based eviction scales with client lock footprint, so larger lock holders evict more locks under pressure while smaller clients are not over-penalized. - Gradual rollout is gated by packed-capable export coverage, avoiding a mixed-upgrade case where only a small subset of new clients absorbs repeated reclaim pressure. - Client eviction uses segmented pressure-to-evict mapping, so reclaim is mild at low pressure and stronger at high pressure. At relatively low pressure, clients react faster to keep pressure in a healthy range; at higher pressure, clients use a longer evict interval because each round evicts more locks, reducing oscillation and avoiding over-evicting locks that should be retained; at or above the last curve point the interval shortens so high-pressure reclaim stays frequent. Key updates: - add OBD_CONNECT2_POOL_SLV_PACKED negotiation and packed SLV helpers - add slv_mode (legacy_slv / pressure_rollout), rollout threshold, and client evict-curve sysfs controls - pressure_rollout pools use ldlm.lock_pool_limit_mem_pct (default 30% of RAM / LDLM_LOCK_MEM_OVERHEAD) instead of LDLM_POOL_HOST_L - keep dual SLV state (legacy scalar + pressure scalar) for compatibility, where in pressure path; add quota-based client reclaim policy: ldlm_cancel_pressure_policy - fall back to legacy_slv on mixed-filesystem server nodes - keep legacy behavior for old clients and pre-threshold rollout stages - add sanity-ldlm-pool.sh, sanity 124aa, and man pages for the new pool parameters Performance impact: CPU overhead should be negligible. - The server recomputes pressure once per second per pool. On the client, pool recalc runs on the ~10s timer and when unused locks are decrefed. - On the packed pressure path, reclaim no longer performs per-lock arithmetic during LRU scanning, so lock-heavy workloads should see unchanged or slightly lower CPU usage. Assisted-by: Cursor:auto/codex5.5 Signed-off-by: Keguang Xu <squalfof@gmail.com> Signed-off-by: Keguang Xu <kxu@ddn.com> Change-Id: If8a6f247127576965ae45c349a4ab04fb286615a
| unique failing test | history |
|---|---|
| sanity-flr@ldiskfs+DNE:test_0m | seen in 1 other review |
| sanity-flr@ldiskfs+DNE:test_205b | seen in 4 other reviews |
| sanity-flr@zfs:test_0m | seen in 1 other review |
| sanity-flr@zfs:test_205b | seen in 4 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 | RHEL 9.7 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-4 | RHEL 10.1 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-5 | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-zfs-part-4 | RHEL 9.7 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-zfs-part-4 | RHEL 10.1 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 8 tests. 1 tests failed: sanity-flr. | session |
(typo) "without sync its content before hand" reads awkwardly; maybe "without syncing its content beforehand". It would also help to spell out here that the newly-added mirrors are marked stale and stay unusable until `lfs mirror resync` runs, and that this needs an MDS with the stale-merge support.
(minor) The description doesn't mention that the option has no effect when the mirror comes from `-f VICTIM_FILE`, which is how the code behaves today. Carried over from patch set 2 and still open: it would help to say why this is useful, e.g. "This can allow an interactive command to configure the new mirrors on a file, while deferring the work of data resync to another node." (typo) "and be unusable" should be "and will be unusable", and "by mirror resync command" reads better as "by the \fBlfs mirror resync\fR command".
(defect) This version gate lets the test run on servers that can't do a stale merge. The MDT side (LU-18746 "flr: allow merge stale mirror" and LU-18771 "flr: allow merge mirrored layout") landed after the 2.17.57 tag, so every server from 2.16.52-70 up to 2.17.57 passes this check and will fail the `verify_comp_attr ... stale` assertions on an interop run. The gate should name the version the stale merge actually landed in, e.g. `$(version_code v2_17_57-2-gc4b410f7e3)` or the next interim version. (style) The skip text should also say why the version is needed rather than repeat it, e.g. skip "need MDS >= ... for stale mirror merge".
(suggestion) The test only covers the layout-generated path. A case for `--no-resync -f VICTIM_FILE`, and one asserting that `lfs setstripe --no-resync` / `lfs migrate --no-resync` are rejected, would cover the rest of the new option. Also, the mirror reads here are piped straight into md5sum, so a failing `lfs mirror read` just yields the checksum of empty input and the `!=` assertions still pass. Capturing the exit status would make those checks meaningful.
(minor) The kernel-doc block above this enum documents every other flag (@MF_NO_VERIFY, @MF_DESTROY, @MF_COMP_ID, @MF_COMP_POOL, @MF_FOREIGN) but gets no entry for the new one. While adding it, is MF_MERGE_STALE the best name? Every other flag here is named after its option, so MF_NO_RESYNC would be easier to connect back to `--no-resync`.
(defect) `--no-resync` is dropped on the floor in the victim-file path. mirror_extend_file() only looks at MF_NO_VERIFY, and it sends `lil_flags = LL_LEASE_LAYOUT_MERGE` with no LL_LEASE_ALLOW_STALE.
So `lfs mirror extend --no-resync -f victim file` still runs the full content comparison and attaches the mirror as up-to-date, with no diagnostic. That is the opposite of what the man page promises.
Either honour the flag here, or reject the combination the way --no-verify is validated at the bottom of lfs_setstripe_internal():
if (mirror_flags & MF_NO_VERIFY) {
if (opc != SO_MIRROR_EXTEND) ...
else if (!has_m_file) ...
}
(defect) Is there anything that stops this from being used against an MDS that doesn't understand a stale merge?
The stale marking is entirely server-side: mdc_close_intent_pack() sets `cd_merge_flags = CD_MERGE_STALE`, mdt_close_handle_layouts() turns that into `mrd_merge_stale`, and lod_declare_layout_merge() is what ORs in `LCME_FL_STALE`. There is no OBD_CONNECT bit and no version check for any of it. An MDS predating LU-18746 just ignores `cd_merge_flags` and merges normally.
So with a single new mirror on such a server:
lfs mirror extend -N --no-resync file
skips migrate_nonblock(), the MDT merges the empty volatile layout as a *non-stale* mirror, and the file ends up with two mirrors declared in sync where one contains only zeroes. `lfs mirror resync` then sees nothing stale and is a no-op, so reads can legitimately be served from the empty mirror.
The `-N2` path added by LU-18771 doesn't have this exposure because it always resyncs afterwards; this option is the first one that leaves the file permanently unsynced. Should this refuse (or fall back to syncing) when the MDS can't honour the flag?
The multi-mirror case is safe by accident: an old LOD rejects a victim layout with `lcm_mirror_count != 0` with -EBUSY.
(style) Not a bug, but this lands after LFS_MIRROR_STATE_OPT and breaks the sorting the rest of this hunk just introduced - alphabetically it belongs between LFS_MIRROR_INDEX_OPT and LFS_MIRROR_STATE_OPT.
(style) `!(opc == SO_MIRROR_EXTEND)` is more simply written `opc != SO_MIRROR_EXTEND`. The message also doesn't match the rest of this function, which uses `"%s %s: ...", progname, argv[0]`. Something closer to the --no-verify wording would read better, e.g. "--no-resync is valid only for lfs mirror extend command". The braces around the case body aren't needed either since nothing is declared in it.
LU-17531 lfs: add --no-resync option in mirror extend So that we can add a mirror without sync its content before hand, it can be resync'd later. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: I1b43fb0299c2aaea72587eff9168e4cdee075624
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
30% improvement for buffered writes seems worthwhile to pursue?
LU-17916 osc: early switch to sync write When running up to grant or dirty page limits switch to sync write to speedup freeing additional grant and clear dirty pages. This improves single and multi-threaded throughput for buffered writes by about 30% HPE-bug-id: LUS-12384 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I2d34b4aedafdf22392d81bce1cd03b58302381d6
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
parameters -> parameter
sysfs_memparse() returns Bytes (not KiB) so `val >> PAGE_SHIFT` is correct.
LU-16897 ptlrpc: network layer sparse read optimization This patch adds support for sparse file read in Lustre. It is a solution mainly in PtlRPC and LNET layer. To avoid increasing the message size of the other LNET messages, it extends LNET header with variable size to store the hole bitmap information. This just needs the detailed KLNDs do some small changes. It adds a new message type LNET_MSG_SPUT which is similar to LNET_MSG_PUT message but with a IOV page hole bitmap (32 bytes, 256 bits) which is stored in the extended LNET header. When OFD target reports the read containing hole pages for bulk I/O, the server will remap the KIOV page to filter out the hole pages. And then the server send a LNET_MSG_SPUT message to the server which contains the page hole bitmap information. Once a client receives a SPUT message, it first zeros out the corresponding hole pages and then filters out hole pages and re-maps the previous prepared KIOV. And then receive the data (via RDMA) if any. TODO: 1. disable the sparse file read for rdma-only (GPU) data. 2. special handle for a heterogeneous cluster that the PAGE_SIZE between the client and the server are different (in the current case, the client will just fallback to the normal read if the PAGE SIZEs between client and server are different). Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I9c3b4c74fd8762a8040741e77f11efd031e6145a
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.7 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-5 | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: lustre-rsync-test. | session |
LU-20238 ec: branch-free byte-indexed scalar base path Replace the per-byte gf_mul() in the scalar *_base hot paths with a byte-indexed per-coefficient region table: tbl[b] = c * b in GF(2^8), built once per coefficient gf_region_mul(tbl, x) = tbl[x] (single load, inner loop) ec_encode_data_base() at k=6/p=2, gcc -O2: 0.79x at 256 B, 1.08x at 1 KiB, 1.32x at 64 KiB. gf_mul() alone is ~20% slower than the 64 KiB direct lookup it replaces, so regions below GF_REGION_TBL_MIN_LEN (1 KiB) keep a scalar fallback. Byte-indexed lookup (gf-nishida-16 "FullByte"-style) is also patent-clean: no PSHUFB, no nibble split, no 16-byte gftbl, so it is structurally distinct from the ISA-L SIMD pattern that maps onto US 8,683,296 claim 21. ec_base.c carried an unconditional "#define GF_LARGE_TABLES", so both ec.ko and libec.a compiled the 64 KiB gf_mul_table_base and the small-table branch was dead code. With that define and the inert ec_base_o_CPPFLAGS in ec/Makefile both gone nothing could set it, so the block is deleted outright: .rodata 65792 -> 1536, .text 2217 -> 3591 for the fallbacks. gf_vect_mul_init is renamed gf_vect_mul_init_base, with a wrapper in ec_base_aliases.c; its 64-bit path now selects on BITS_PER_LONG for kernel builds, since __WORDSIZE is glibc-only. No existing entry point changes behaviour. Validated on RHEL 10.1 (AVX2+GFNI): exhaustive gf_mul + gf_inv, k=6/p=2 round-trip KAT, and gfni_ec_test byte-identical parity. Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity env=ONLY=910 Signed-off-by: Hiroshi Nishida <hnishida@thelustrecollective.com> Change-Id: I905e365d145d0018cbcd2bbc919eb4e516fafe8e
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs failed 2× | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.8/x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity. | session |
| review-zfs | RHEL 8.8/x86_64 | ran 11 tests. 3 tests failed: test-groups/review-zfs, sanity-lsnapshot, sanity-quota. | session |
So I’ve looked at this before and this isn’t safe - we can’t change file flags in the kernel, or they stay changed in user space. We just have to not do this on older kernels, I think. Changing the ki_flags is fine though
We have restored the file flags (unmask O_DIRECT flags) at code line 1987. Is it okey and enough?
LU-16964 llite: auto switch from BIO to DIO We design a hybird I/O path engine to perform buffered I/O as direct I/O. It can switch from the default buffered I/O to direct I/O and allow the buffered I/O which meets the requirements and conditions to perform I/O in direct mode with much more efficient way appropirately. Switching to use direct I/O may provide some performance benefits in the following cases: - There is no access locality; - The I/O size is large enough; - The system is under memory pressure or high CPU usage; - A file is shared access under high conflict contention from many clients; In this patch, we implement auto switch from BIO to DIO when a file is under high lock contention. Once enough conflicting lock requests are seen within a certain time window (a tunable, 4 secondes by default), the resource contention is reported to the client. When informed the lock contention, the corresponding inode on the client is marked as contended and this state lasts for a time period (a tunable, 30 seconds by default). In this time period, the subsequent I/O will be performed in direct I/O mode by using lockless I/O. Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I9288d3049671ee67e829409b1fe28ca3ab45cbfb
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
(minor) this should only be needed once before the start of the loop? After that, the return value from atomic_cmpxchg() can be used instead of doing another read.
LU-19721 llite: multiple lock cb hold invalidate_lock
On the newer kernel such as Rocky9.5 or SLES15sp5, the test
case sanity-pcc/test_99b deadlock as follows:
A client is granted a PW extent lock L1;
TD1 - TD8: read direct I/O
On client side (OSC): use out all available RPC slots of 8.
On server side (OST):
- Waiting for the server side locking for direct read I/O;
- Lock blocking callback to revoke the conflict granted L1;
<== Waiting for the completion of T2.
T2: Client side lock blocking AST for L1
osc_dlm_blocking_ast0()
->osc_lock_flush()
->osc_lock_discard_pages()
- Acquire execlusive invalidate_lock
<== Waiting for T3 which is holding shared invalidate_lock
T3: generic buffered read:
->ll_file_io_generic()
- Acquire shared invalidate_lock
->ll_readpage()
->lov_io_submit()->osc_io_submit()
- Waiting for I/O finished
- I/O extent is waiting for RPC slots
<== Waiting for TD1-TD8
The deadlock is: TD1-TD8=>T2=>T3=>TD1-TD8
This patch solves the deadlock using trylock for invalidate_lock
in lock blocking AST.
If trylock failed, it will check the pending read extents in the
OSC object and move them into high priority list to make them fire
into wire as soon as possible. Thus the read holding the shared
invalidate_lock can release as quickly as possible.
And lock BAST can acquire the invalidate_lock finally.
Was-Change-Id: I6800c998cf0ce92de11d846d4715cec81cb9a513
Allow multiple lock blocking callbacks from multiple threads and
different OSC objects enter into the critical section of write
@invalidate_lock of the inode mapping to discard pages in
parallel.
This means once a thread from the context of lock blocking AST
held the @invalidate_lock, all other BAST thread can enter into
the critical section simultaneously to discard pages in parallel.
This can avoid the possible BAST timeout and client evictiond as
in the old way the exclusive wirte lock @invalidate_lock makes
the BAST threads perform serially and this increases the overall
delay of BAST from multiple stripes.
Add sanityn/test_122.
Fixes: bba59b1287 ("LU-16651 llite: hold invalidate_lock when invalidate cache pages")
Fixes: 56e59cbb11d8 ("LU-19427 llite: fix invalidate_lock deadlock for mixed BIO/DIO")
Test-Parameters: clientdistro=sles15sp7 testlist=sanity-pcc env=ONLY=99b,ONLY_REPEAT=50
Test-Parameters: clientdistro=rocky9.5 testlist=sanity-pcc env=ONLY=99b,ONLY_REPEAT=50
Signed-off-by: Yingjin Qian <qian@ddn.com>
Signed-off-by: Andreas Dilger <adilger@thelustrecollective.com>
Change-Id: I70a88fcf895e1f313f6a5ded204fa5d6750bba70
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
LU-20323 llite: enable async readahead for stride read async readahead should also work for stride read, this patch enables it. with a 2.19TB file, single thread stride benchmark reads 47008 bytes, and then seeks 1M, looping until EOF. Without patch 156.72s 628.7MB/s With patch 68.42s 1440.5MB/s However for multithread stride io, with each thread starting with an offset of thread_id * 47008 bytes, there's performance regression due to extra contention on adding/removing the page from address space in vfs. It makes sense for single thread read, we could use multiple async work items to fetch pages in and keep fast read going, but for multi thread read, it's not a good idea to bring extra contention. Let's limit async readahead to single thread, by checking lli_open_fd_read_count and lli_open_fd_write_count. write_count is also checked because when a file is opened with O_RDWR, only lli_open_fd_write_count is increased, same logic as the kernel's i_readcount and i_writecount. Change-Id: Iab7ee9455f10b1d22b0a1f2cc15d40825fd1f105 Signed-off-by: Li Dongyang <dongyangli@ddn.com>
| unique failing test | history |
|---|---|
| sanity-pfl@ldiskfs+DNE:test_101 | seen in 1 other review |
| sanity-pfl@zfs:test_101 | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-pfl. | session |
| review-dne-part-2 | RHEL 9.7 / x86_64 | ran 11 tests. 1 tests failed: sanity-pfl. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-pfl. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-pfl. | session |
| review-dne-zfs-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-pfl. | session |
For patches using AI assistance during development, the upstream kernel has requested to add `Assisted-by:` to the commit message. Please see the following page for details: https://wiki.lustre.org/Commit_Comments#AI/LLM/Tool_Attribution
Hi @patrick@thelustrecollective.com, This patch has been created by Justin with AI tools. It solves LU-20289 problem described in EXA6. It also passes the test you added in https://review.whamcloud.com/c/fs/lustre-release/+/64127. So in case of EXA6 it works fot both cases. If speak about master, this patch passes sanity_389t from 64217, but fails LU-20289 reproducer(sanity-pfl_101). As far as I can see it fails because the size of the file is smaller for 9 pages then it should be. At the same time, it successfully created the 2nd PFL component and wrote several DIO buffers there. It seems there is a difference between master in exa6 that causes to pass in on case and fail in another.
@scherementsev@ddn.com it could be that the difference on master is the addition of patch https://review.whamcloud.com/61688 ("LU-12738 pfl: do not instantiate full PFL layout on append")
LU-19900 lov: DIO+APPEND corruption on PFL extent boundaries
Data corruption occurs when O_DIRECT|O_APPEND writes cross PFL
component or stripe boundaries. The root cause is the DIO stripe
cache in lov_page_init_composite(): once the first page resolves a
(component, stripe) pair, every subsequent page in the same DIO batch
blindly reuses that cached value, even when the page offset falls in a
different PFL component or a different stripe within the same
component.
For O_APPEND the LOV iter_init phase runs before vvp_io_write_start()
resolves the real file-end position (crw_pos is still 0 at that
point), so the batch is never pre-split at component/stripe
boundaries. A single DIO batch can therefore span multiple components
and stripes.
lov_page.c - lov_page_init_composite():
Invalidate the DIO stripe cache when the page offset crosses:
1) A PFL component boundary (offset >= lsme_extent.e_end), or
2) A stripe boundary within a multi-stripe component (the
stripe-size-relative chunk index differs from the cached one).
On invalidation the full lov_io_layout_at / lov_stripe_number /
lov_stripe_offset lookup is re-executed for the new page.
lov_io.c - lov_io_submit():
Remove the CPT_TRANSIENT shortcut that spliced all remaining DIO
pages into a single sub-IO batch regardless of cp_lov_index. With
the cache fix, DIO pages in an append batch can now carry different
cp_lov_index values (different components/stripes). Always group
pages by cp_lov_index so each group is submitted to the correct
sub-object.
Add sanity-pfl_101 which reproduces the problem without the fix.
Test-Parameters: fortestonly
Signed-off-by: Justin Oberdorf <joberdorf@ddn.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I110ccc8ed5e114e87c784e330b0ef08149d3ddba
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_27D | seen in 92 other reviews |
| sanity1@zfs:test_27D | seen in 96 other reviews |
| sanity-hsm@zfs:test_12q | seen in 60 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-19823 lod: Make the stripe allocators failure domain aware. Add failure domain awareness to the RR and QOS allocators when/if configured. A failure domain value of 0 means not configured/not used. For now we only implement this for EC. Later, we might enable this for FLR Mirroring too. Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I7bf09636939915b866eb97d9952bab2968d7718e
| unique failing test | history |
|---|---|
| lnet-selftest@zfs:test_smoke | seen in 3 other reviews |
| recovery-small@zfs:test_19a | seen in 3 other reviews |
| sanity3@ldiskfs+DNE:test_300g | seen in 2 other reviews |
| sanity3@zfs:test_300m | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity-lfsck@ldiskfs+DNE:test_18a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-lfsck@zfs:test_18a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@ldiskfs+DNE:test_29a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@zfs:test_29a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-2 | RHEL 9.7 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-part-4 | RHEL 9.7 / x86_64 | ran 11 tests. 2 tests failed: sanity-quota, sanity-hsm. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-part-7 | RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-pcc. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.7 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 3 tests failed: sanity-sec, sanity-lnet, sanity. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
Is this something that can continue to be improved in the future?
It would make sense to fix those cases on the MDS that you know about, so they request the needed bits, even if the client still cancels all bits on LAOUT lock for now. That makes the code "more correct" and is a step toward fixing the problem properly.
I am trying to move forward incrementally covering more cases.
LU-11284 ldlm: enable lock convert for all bits Patch enables lock convert for all inodebits. Several issues appeared during that due to changed lock behaviors which weren't seen previously. - most issues are related to EX LAYOUT lock taken on server MDT often assumes that taking EX LAYOUT bit will cancel whole lock with all other bits. That is hard to track and fix all such cases, so current patch just keep that behavior, if blocking lock is EX LAYOUT then lock convert is skipped - llapi_layout_test 31 failure. Test adds new component and read new layout via fgetxattr() but it contains old layout. The problem is that at the moment of fgetxattr() call the local layout is still in 'invalid' because lock has gone during setxattr. Problem is hidden without lock convert because LAYOUT lock cancel also cancels PERM lock which causes inode dentries invalidate, so new object is created each time with fresh xattr from server. With lock convert that PERM bit stays and test read obsoleted xattr. Solution is to do ll_layout_refresh() in ll_getxattr_lov() prior getting layout from LOV, so obsoleted one will be updated. Test-Parameters: testlist=performance-sanity,dom-performance,sanity-benchmark Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Change-Id: I7e8c915e43841e10f21d2df0106c1bb783b322cc
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_27D | seen in 92 other reviews |
| sanity1@zfs:test_27D | seen in 96 other reviews |
| sanity-hsm@zfs:test_12q | seen in 60 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-20139 lod: Add failure-domains to FLR allocations Add failure-domain awareness to FLR mirrored files. If overstriping is requested we allow the same failure domain to be re-used within a component but not across components in different mirrors. Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: Id8f4beb23ef20f805cbf47ba743b2291c8d4a9ae
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_127f | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-5 crashed | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-subtest-change | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanityn. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
(typo) ".patch" Writes for fscrypt files *should* always be full-block writes? Why/how would writes to fscrypt files be partial blocks? The client should be sending a full encrypted 4KiB block, but then set the size on the inode without truncating it.
Please note this patch is using buffered I/O path on ldiskfs. I think Lustre fscrypt does not handle the buffered I/O case, only patched ext4 for fscypt with write through (direct I/O) mode. Although writes from clients are always full-block writes, but the size calls via lnb offset + len may be not full-block aligned, we use this size for the actual size of the file?
LU-12916 osd: use writeback for small writes in ldiskfs This patch implements the writeback caching for small writes on Lustre OSD (ldiskfs). The writes are not synchronous any more on OST. It marks the page as dirty and create buffer heads mapping pages to the physical blocks. The kernel writeback mechanism will handle the actual I/O. This patch should improve the performance for I/O cases such as small writes from many files (i.e. mdtest-hard-write) or unaligned I/O on a shared file (i.e. ior-hard-write). Add a ldiskfs patch "ext4-writeback.patch" to not zero out non-full page during writeback for a encrypted file. Without this ldiskfs patch, it will fail sanity-sec/test_37. According to the benchmark results, this patch can improve the IO500 mdtest-hard performance and the performance of the real application Nek5000 turbulent pipe with more than 10%. Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I9b5aa568a49d5b39d04656fdacb94c3a6ec5a7ff
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 7 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 8 tests. 1 tests failed: sanity. | session |
LU-20186 llite: fix libaio metrics in llite.*.stats When using asynchronous I/O (e.g., libaio), operations return -EIOCBQUEUED upon submission, bypassing the standard stats tallying in ll_file_io_generic(). This results in missing I/O metrics in llite.*.stats and extents_stats_per_process. This patch fixes the issue by deferring AIO/DIO stats tallying to the completion path: - Extend `struct cl_dio_aio` to track submission context (start time, pid, file data, and I/O type). - Update `cl_dio_aio_alloc()` to accept a custom end_io callback. - Introduce `ll_cl_dio_aio_end()` in llite to tally latency and bytes upon AIO completion. - Skip synchronous tallying for IOCB_DIRECT to avoid double-counting. Note: This patch is AI generated and reviewed by human. Signed-off-by: Jinshan Xiong <jinshanx@google.com> Change-Id: I9fe38cc64ed50548a71dee7cc303d129329f191c
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-1 crashed | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-3 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-part-7 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-subtest-change failed 2× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 crashed | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-6 crashed | RHEL 8.10 / x86_64 | ran 6 tests. 2 tests failed: ost-pools, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-7 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-zfs-subtest-change failed 2× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity-lnet. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-9 lod: improve qos stripe allocation Replace random OST selection with deterministic weighted round-robin allocation that ensures proportional distribution based on OST capacity and distributing the load across MDSs. The previous QoS method relied on RNG to approximately distribute stripes proportionally to OST weights, however this lead to occasional unbalancing due to RNG variance. This new approach uses a cursor-based system that increments by the average weight, ensuring that OSTs with a higher weight will be hit more often and OSTs with a lower weight will be skipped more often. Key improvements: - Deterministic allocation eliminates RNG variance - Better load distribution across unbalanced OSTs - Enhanced anti-aliasing to prevent clustering - New aggressive rebalancing (prio_freee > 100%) - prio_free <= 100% is now a linear slider between RR and QoS - Comprehensive test coverage for allocation patterns These features are now enabled by default through qos_threshold_rr=0 qos_prio_free=100 which means that QoS is active if there is a difference of 0% or more across target devices, and will balance directly proportional to the target weights. Test-Parameters: testlist=sanity env=ONLY=413a,56xe Test-Parameters: testlist=sanity env=ONLY=123ac,ONLY_REPEAT=10 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=3 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=4 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=5 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=6 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=7 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=8 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=9 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=10 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=11 Test-Parameters: testlist=sanity env=ONLY=51d,OSTCOUNT=12 Signed-off-by: Frederick Dilger <fdilger@whamcloud.com> Change-Id: Ie314438e03f867fe978adc37d62b504532961cd2
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 crashed | RHEL 9.5 / x86_64 | ran 13 tests. 2 tests failed: sanity-pfl, sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-2 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 2 tests failed: sanity-pfl, sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 8.10 / x86_64 | ran 11 tests. 2 tests failed: sanity-quota, sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-4 crashed | RHEL 9.5 / x86_64 | ran 11 tests. 2 tests failed: sanity-quota, sanity-flr. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-7 crashed | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-7 crashed | RHEL 9.5 / x86_64 | ran 7 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-2 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 2 tests failed: sanity-pfl, sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-flr. | session |
| review-dne-zfs-part-7 crashed | RHEL 8.10 / x86_64 | ran 7 tests. 1 tests failed: sanity-pcc. %% THIS TEST SESSION CRASHED %% | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 10 tests. 1 tests failed: sanity-flr. | session |
James, the lack of ability to remove other components than the last one is a deliberate choice to prevent the creation of incomplete layouts. There is nothing about the current approach that can’t support removing other components, it’s just a sanity restriction.
Think about file join work with 2 mirrors each with 3 components and another file with also 2 mirrors also with 3 components. Its going to be merged right. The current approach would require allocating a new larger array. Then copying items from file 1 static array and then copying items from file 2 static array. Then deleting the old array. Some slicing is needed due the 2 mirrors. Compared to Xarray its erasing the reference to pointers in file 2 xarray and adding a reference of that same pointer into file 1 Xarray. Do you feel the former approach is way better? Honestly the biggest impact is for the file join project so the really important opinion I need is from Alexey. Alexey what do you think?
Yes, I do - it's very simple. You allocate a larger array and write the layout to it. Very simple. It requires no complex data structures and no locking. The idea that we should have references across two xarrays is very concerning - that means the layout components need reference counting then, I guess? They do not today. There is no particular impact on file join - it is just as easy to do file join via layout copy as any other operation.
James, file layout join/migrate/mirror happen _maybe_ only once or twice in the lifetime of a file, so it doesn't make sense to add a complex data structure and overhead to optimize a 0.00001% usage case. That is doubly a problem if it means many small allocations and/or additional locking needs to be added to handle this extremely rare use case.
LU-19298 lod: use Xarray instead of static arrays for comp entires The LOD layer implements very large static arrays to manage the component entries for each layout. This works well for the case that all the components are for block I/O that are expected to be aligned to each other in a specific order. In such cases the API works to add or delete components at the end of such layouts. With the upcoming introducting of parity and foreign components the rules change in that you can delete or add a component located in the middle of the layout. This would be very complicated when using static arrays. To make life easier for upcoming projects we move away from static arrays to using Xarrays. We no longer requires creating a whole new array and copying components over. Instead we just insert or delete components into the Xarray. This current implementation just does the change from static arrary to Xarray without truly introducing gaps in the Xarray. This can be done in follow on patches and can be used to greatly simplify the code. For example since "gaps" are allowed in the Xarray we can place components in the Xarray using the mirror id as apart of the index which mirrors the component ID. This removes the need to manage the new mirror ranges in the Xarray. Change-Id: I67020f899ad89bfa6095c4b22f7f1b07b6de3e86 Signed-off-by: James Simmons <jsimmons@infradead.org>
| unique failing test | history |
|---|---|
| sanity-pfl@ldiskfs+DNE:test_1c | NEW unique failure for this branch in the last 30 days, and was seen 2 times across 1 other branches 1 reviews |
| sanity-pfl@zfs:test_1c | NEW unique failure for this branch in the last 30 days, and was seen 2 times across 1 other branches 1 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-pfl. | session |
| review-dne-part-2 | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-pfl. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-pfl. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
LU-19298 lod: merge different lmm handling togther. For PFL handling the LOD has developed very similar code independently. Besides code duplication we end up not having consistent PFL handling. For example lod_layout_add() handling doesn't support specific OST indexes being requested by the user for the new component. We merge all the code into lod_comp_for_each_lcm(). In the future any change here will impact many code paths at the same time. For EC developement we don't need to touch any many places in the code. Signed-off-by: James Simmons <jsimmons@infradead.org> Change-Id: Ic2392dd84eb397e87e384d6916ccb2de04b6d083
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-zfs | RHEL 8.10 / x86_64 | ran 8 tests. 1 tests failed: sanity-quota. | session |
(style) wrap lines <= 80 columns
(style) prefer to show YAML-formatted layout in example (minor) this shouldn't need root access to work? Ah, I see that it is accessing `trusted.lov` (which does need root access), when it could be using `lustre.lov` (which allows non-root users to read the xattr). See my later comments about this.
(minor) it would be better to reference `lfs-getstripe (1)` here, instead of the generic `lfs (1)` page
(style) wrap lines at 80 columns, preferably at natural breaks in the sentence (e.g. after '.' or ',')
If writing a new tool, it would be better to always use the YAML-style output like "lfs getstripe --yaml" so that it can be parsed and read more easily. The "old style" output here has a lot of historical baggage (e.g. "group" doesn't really exist anymore).
(minor) This should verify that the output format is valid YAML:
verify_yaml_available &&
$LL_DECODE_LOV --yaml $DIR/$tdir/$tfile | verify_yaml ||
error "ll_decode_lov --yaml does not produce valid YAML output"
(style) line length of 86 exceeds 80 columns
(style) don't need `local` again here, since `local lov` was already declared above
(minor) This should also verify that the output format is valid YAML. Since this is a new utility, is there any benefit to *not* producing YAML output by default from the beginning?
(style) Missing a blank line after declarations
(style) This could be fixed if patch is refreshed.
(style) 'sprintf' may be misspelled - perhaps 'snprintf'?
(minor) this warning is valid
(defect?) why re-declare this function, which is available in `liblustreapi.c`? It could be declared only in `lustreapi_internal.h`
(defect?) the `trusted.lov` xattr is only available to the root user. If this read the `lustre.lov` alias then regular users could use this tool as well. However, that wouldn't work with a local ldiskfs mount, so it probably makes sense to try `lustre.lov` and if `-ENODATA` is returned then try `trusted.lov` before returning an error (or try `trusted.lov` and if `-EACCES` is returned then try `lustre.lov`).
LU-18725 utils: create ll_decode_lov Like ll_decode_linkea, create ll_decode_lov. This allows us to directly scan the MDT for information that can be obtained with lfs getstripe. The LOV printing functions are moved from liblustreapi.c to a new file liblustreapi_lovea.c. This includes both raw printing functions (for xattr buffer decoding) and wrapper functions (with path lookups). Signed-off-by: Sohei Koyama <skoyama@ddn.com> Change-Id: I50fcb0be067a4cda80aea19fa4b64b8ccf3a9667
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity1@ldiskfs+DNE:test_27D | seen in 50 other reviews |
| sanity1@zfs:test_27D | seen in 48 other reviews |
| sanity-hsm@zfs:test_12q | seen in 17 other reviews |
| sanity-hsm@zfs:test_254b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-lfsck@ldiskfs+DNE:test_20a | seen in 31 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_20b | seen in 31 other reviews |
| sanity-lfsck@zfs:test_20a | seen in 31 other reviews |
| sanity-lfsck@zfs:test_20b | seen in 31 other reviews |
| sanity-pcc@zfs:test_20 | seen in 15 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-dne | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm crashed | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
LU-19823 lod: move flagging ost for avoidance into a helper function Test-Parameters: trivial Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: I0838b1271d48c2e157a164472870d650a9e125d8
| unique failing test | history |
|---|---|
| sanity-lnet@zfs:test_260 | seen in 6 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-3 crashed | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: lustre-rsync-test. | session |
| review-dne-zfs-part-3 crashed | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. %% THIS TEST SESSION CRASHED %% | session |
please clarify how it's better than the existing approach. notice we use the same mechanism for other OSD API devices like OSP
I've been looking for ways to make the Lustre start/stop flows simpler and more robust. I've already done a bunch of work towards standardizing the individual device start/stop flows (i.e. https://review.whamcloud.com/c/fs/lustre-release/+/58831/3). Lustre uses a global mutex on the server start/stop (i.e. server_start_lock). I think it would be better if we could move away from that. To get there, I think we need to stop sharing devices globally as much and keep more stuff private to the superblock of each mount. This patch is an attempt to do that for OSD. I'm not 100% convinced yet. My main concern is that we likely want the OSD to lifetime to exist independently of the lustre_tgt superblock. I don't think anything in Lustre requires this today. But I'd like to see some kind of online OSD reader, which would require this. But even for that, I think this obd_connect()/disconnect() approach is a bit heavy. A lighter weight ref count (perhaps the lu_device ref count? Or a new refcount?) would be better.
(style) It would be better to remove the `lod_init0()` name and use something more meaningful, since James is just trying to remove the `0` usage.
An llog_ctxt contains a disk export - so this needs a bit more reworking.
Need to add check for dt device or similar.
LU-17848 osd: don't use exports for ref counting OSDs currently use OBD exports to track users. OSD consumers obd_connect()/obd_disconnect() to hold and release references on the OSD. This is an abuse of the OBD API and needlessly verbose. Instead, attach the lifetime of the OSD device to the lifetime of the lustre_tgt superblock - start the OSD first and stop it last. Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: Id14bc14881a3b694b214c121c7e00bfbb9b1f896
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 failed 2× | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: pjdfstest. | session |
LU-19223 osc: stop after scanning Currently, the Lustre shrinker tries to scan until it has freed the number of pages requested for scanning. If no pages can be freed, this can take an extremely long time, manifesting as an almost hung system. Also set nr_scanned so the kernel can understand what we've done. Lustre-change: https://review.whamcloud.com/60489 Lustre-commit: 109e32dc23c53baf0d5b8e11d34a9f5f2fce70aa Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: Ic3ed85d27a785c2f722b6529b79beb3757e15549 Reviewed-by: Zhenyu Xu <bobijam@whamcloud.com> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
| review-dne-zfs-part-2 | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
I don't think the DLM lock association could be optional? Otherwise, the client could cache holes that are not revoked when another client writes into that extent.
We purge the hole extent conflicting with lock extent during the BAST of the lock in osc_dlm_blocking_ast0().
(minor) it should be enough to check e.g. `cra_hole_end != 0` instead of a separate `cra_hole_found`?
done
(style) maybe 'che_' would be a better prefix
The client should not know about the backing filesystem type.
Is there a reason not to enable hole extents by default?
It is not stable. Once it is stable, it will enable by default.
(style) align "bool" arguments to avoid making struct larger
done
(style) better not to have "magic" numbers in different places. Prefer some kind of constant, like `RA_PAGE_HOLE = 2` and then use that here and below.
done
(style) can fit on previous line
(style) line length of 165 exceeds 80 columns
(style) this warning can be quieted by putting `DFID` at the end of the previous line
(defect?) Somehow this change has caused a lot of test timeouts.
To clarify here, if there is a write to a hole extent, is the hole split/shrunk or is it removed entirely?
(defect) this functionality shouldn't depend on the backing filesystem type. The ZFS hole size is always going to be the same alignment or larger than ldiskfs (multiple of 4KiB), so I don't understand why this is here.
LU-19469 llite: hole-aware read optimization for truncate extend
A client can cache hole extent information locally and serve
zero-filled pages for hole regions avoiding long client I/O path,
without network round-trips and bulk data transfer, significantly
improving performance for sparse file read operations while
maintainint data consistency through Lustre's existing DLM
infrastructure.
This patch implements hole-aware read optimization for the holes
generated by extended truncate().
The hole extents are managed using the existing @osc_extent data
structure with the following enhancements:
- hole marker: Add a flag (oe_hole) to distinguish hole extents
from regular data extents;
- Rbtree integration: Store hole extents in the same osc_object->
oo_root tree alongside regular extents;
- State management: define specific states for hole extent
lifecycle (e.g., OES_CACHE with oe_hole=1);
- Lock association (optional): Maintain association between hole
extents and protecting DLM locks;
The hole extent will be inserted into per-object extent rbtree
when a client executed extended truncate operation and extended
the file beyond current size.
Page read or readahead will try to check the OSC object's extent
tree. If found a matched hole extent, directly zero-fill pages
for the hole regions and mark pages as uptodate.
The hole extent will be removed when detect the write operations
overlapping with the cached hole extent.
Lock blocking AST will also remove conflicting hole extents from
the rbtree of the object.
Add sanity/test_856{a/b/c/d/e} to verify it.
Signed-off-by: Yingjin Qian <qian@ddn.com>
Change-Id: I53ffaded1bc92fb60ceecf6b3acc78e979713953
This patch is a improvement even if it does do "There are a lot of places in Lustre where we use more than one atomic operation where one will do".
IMO, to change read + check for zero then increment like below... ASSERT(atomic_read(&lp->lpni_refcount) > 0); atomic_inc(&lp->lpni_refcount); is to change to LASSERT(atomic_inc_not_zero(&hdev->ibh_ref)); since we do not want to always increment. But only increment if the value is not zero. Else it is error condition
same...
this looks correct
this is definitely a win... others also
LU-10013 llite: Atomic usage cleanups There are a lot of places in Lustre where we use more than one atomic operation where one will do. The compiler can't fix this for us, since usage of atomics is mostly exempt from optimization. This patch cleans up many of those cases. In a few places, this has a measurable benefit. (lu_object_get can use atomic_inc_return for the ASSERT, and this nets a few % improvement on single process reads). The change to osc_update_pending is expected to net a few % on shared file workloads. Signed-off-by: Patrick Farrell <paf@cray.com> Change-Id: Ie2b0b4a7fac4e544a66746c95e496071765070a3
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_127e | NEW unique failure for this branch in the last 30 days, and was seen 5 times across 1 other branches 1 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
This really needs a longer description of how the batch IO is implemented. Do merged writes all have to be from the same UID/GID/PROJID to manage quota properly? Do all of the writes have independent timestamps and other attributes? How are checksums, transno, etc. handled? Out of space for some of the writes? Are all of the writes completely independent, but just batched into a single RPC over the wire?
I don't think that discontiguous short IO is a priority to implement. Virtually all small-file writes will be contiguous.
(defect) I would expect a new `OBD_CONNECT2_BATCH_WRITE` or similar connect flag is needed to verify protocol compatibility for this on the server...
LU-16355 osc: batch dirty buffered write of small files This patch implements dirty write batch I/O for small files to improve the small write I/O performance using the existed short I/O mechanism via batching. TODO: discontinuous short I/O support. Signed-off-by: Qian Yingjin <qian@ddn.com> Signed-off-by: Sohei Koyama <skoyama@ddn.com> Change-Id: Ib63710afec65fa17c58186a793cae4442df95af3 Test-Parameters: fortestonly
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-dne-part-5 | RHEL 9.5 / x86_64 | ran 5 tests. 2 tests failed: sanityn, recovery-small. | session |
| review-dne-part-6 | RHEL 9.5 / x86_64 | ran 6 tests. 1 tests failed: replay-single. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: recovery-small. | session |
| review-dne-zfs-part-1 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-3 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 3 tests failed: sanityn, recovery-small, lustre-rsync-test. | session |
| review-ldiskfs | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs failed 2× | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
(minor) up to .59 now
(style) no need for linefeed escape '\' after "||" or "&&" at end of line
LU-17493 mdc: restore LDLM cancel on blocking callback In highly contended directories like ROOT/, /home, /projects, abnormal clients holds LCK_PR executing 'ls' may block all other operations. To prevent this, we can prioritize directory modification responsiveness over strict readdir() cache coherency. This approach is supported by the POSIX standard, which does not require full cache coherency for readdir() across processes on the same node (discussed in LU-3308). A directory is considered 'contended' if it falls into one of the following three categories: 1. Critical directories: essential system directories like ROOT/, /home, and /projects. 2. Directories with racing locks: directories experiencing significant lock contention can be identified by checking ldlm_res_check_contention(). To improve performance in these situations, we can yield the CPU during ll_iterate() -> xxx -> mdc_read_page() using the lock flags LDLM_FL_CANCEL_ON_BLOCK and LDLM_FL_CANCEL_ON_CONTEND. - Client-side: The client checks for conditions 1 and sets the LDLM_FL_CANCEL_ON_BLOCK flag. - MDS-side: MDS has complete lock contention information, this could be used to set the LDLM_FL_CANCEL_ON_CONTEND flag on locks. By setting these flags, the LDLM will release the 'ls' locks and quickly grant other pending locks for modifications, ensuring a more responsive system. Note.1: ll_getattr() that satifies condiction 1&2 has been updated accordingly as well. Since its lock:LCK_PR(UPDATE|PERM) encompasses the lock:LCK_PR(UPDATE) required by ll_iterate(), it's preferrable to set CANCEL_ON_BLOCK at the first place in ll_getattr(). Failing to do so would result in this flag being absent from ll_iterate()'s lock. Note.2: we've leave ll_lookup as is. When ll_lookup() is invoked, it lacks the necessary information (specifically, dir_depth) to verify condition 1. Signed-off-by: Keguang Xu <squalfof@gmail.com> Change-Id: I90163f2bba64f7955cec0d538724ad363eed9abb
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 2 tests failed: sanity-sec, sanity-lnet. | session |
LU-12325 ldlm: mode downgrade, server handler changes Extend lock convert with mode downgrade ability. It can be used in various scenarios when lock is not needed in some strict mode anymore but still can be useful in cache with lower mode Patch contains: - sanity check for new mode - server convert handler allows mode downgrade - enable related connection flag on server Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Change-Id: I7de64736fed40761cf7b528687ae9a4ffa9ad40e
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_64d | seen in 3 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-5 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanityn. | session |
LU-12325 ldlm: mode convert client changes Client part for mode convert: - ldlm_cli_inodebits_convert() support for mode - ldlm_cli_convert_req() is updated - enable connection flag on client side and add interoperability checks - implicit mode convert when DoM bit is dropped to keep remaining bits in less strict PR mode Test-Parameters: testlist=dom-performance Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Change-Id: I50abde0f14adbf8bfe162c1dcd77fb973ae0cc11
| unique failing test | history |
|---|---|
| sanity-sec@ldiskfs+DNE:test_59b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 failed 4× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 failed 2× | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: pjdfstest. | session |
LU-17454 nodemap: allow mapping for root Allow an id mapping for root, to match what is implemented for regular users, with the following behavior: - if admin property is set, root remains root. - if admin property is not set, the idmap for '0' is taken into account. - if admin property is not set and there is no idmap for '0' and deny_unknown property is not set, root is squashed to the squash uid/gid. - if admin property is not set and there is no idmap for '0' and deny_unknown property is set, root is blocked. Note that map_mode remains ignored for root. Also, capabilities are not dropped for root when mapped, just like it is done for regular users. If admins want to drop root capabilities, root must be squashed. sanity-sec test_15 is updated to test root mapping. Lustre-change: https://review.whamcloud.com/53870 Lustre-commit: b4a336d0ce91c05ae48544b3fd2e56f0bcb0a8cf Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: Id2e950b99e3b3ba27179408c647e1f7b7c49e32e Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-6 | CentOS 8.5/x86_64 | ran 4 tests. 2 tests failed: ost-pools, replay-single. | session |
LU-16495 libcfs: expand debug mask
Introduce debug mask expansion option with additional 32 bits to use.
This allows for more custom debug message types per subsystem.
To take advantage of the expanded mask, subsystem needs to define
its own interpretation of the expanded mask bits and manage it
via corresponding lctl option. For example:
lctl set_param debug_lnet=+msgtrk
CDEBUG(D_NET | (D_LNET_MSGTRK >> 32), "message");
The default behavior remains unchanged: the expanded mask is optional.
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I41481566530b21574db0990798b803544d802a6c
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-ldiskfs | RHEL 8.9 / x86_64, RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 9.3 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-17663 osc: extent find only care for write
Don't use read lock in extent, read with different regions have
different compatible read locks.
Fixes: 67aca1fcc6 ("LU-16160 osc: take ldlm lock when queue sync pages")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Iceb05eb22744debc2e8122ef3ee451d149d79195
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 9.4 / x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
Perhaps we could use an already existing request format? Curious if anyone else has thoughts.
I don't think we should have a dedicated RPC just to set this one bit.
LU-18615 mdt: setup protocal for super hide This is the first patch to implement "lfs hide [path]". With this cmd, [path] will be hidden from directory listing "ls -a". Similar to .lustre. We mark a file/directory as hidden by setting a one-bit flag in the lma_compact of trusted.lma. Here in this patch, we reset this bit as LMAC_HIDDEN = 0x00000080 in enum lma_compat. The "lfs hide [path]" cmd will be run from a lustre client. Therefore, this patch also reserves the new ptlrpc req_format to be sent between clients and MDSs. 1. We reserve "MDS_HIDE = 65" in "enum mds_cmd" 2. We "#define OBD_FAIL_MDS_HIDE_NET 0x2410" We also provide the new req_format "RQF_MDS_HIDE" in this patch. Signed-off-by: Zanhua Huang <zanhua@amazon.com> Change-Id: I797ae49d74c95fac7aa765ed83cda0c4603a7aa3
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-zfs | CentOS 8.5/x86_64 | ran 10 tests. 1 tests failed: replay-single. | session |
LU-14901 cli: consume grants if async write fallback sync io In vvp_io_write_commit() if out of quota, it will try sync write. However in osc_queue_sync_pages(), space grant will be only consumed with DIO, introduce a new flag to indicate consuming grants needed. Signed-off-by: Wang Shilong <wangshilong1991@gmail.com> Change-Id: I6f4389704dc549d266f8fdcc0a5dbe17bf1fb37c
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.6/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-selinux-ssk-part-2 | CentOS 8.5/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-2 | CentOS 8.5/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-4 | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-zfs-part-5 crashed | CentOS 8.5/x86_64 | ran 6 tests. 1 tests failed: sanityn. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-arm | RHEL 8.6/aarch64, RHEL 8.6/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs-ubuntu | CentOS 8.5/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
This seems like a long-winded way to say "not the first page"? But of course what's a shorter way to do it... Huh.
LU-15578 readahead: hold the lock reference for readahead
While the readahead is working as separate thread, we should
hold the lock reference of ldlm lock to avoid it was canceled.
LustreError: (osc_object.c:396:osc_req_attr_set()) uncovered page!
BUG: unable to handle kernel NULL pointer dereference at 000000016
IP: ldlm_resource_dump+0x86/0x420 [ptlrpc]
Oops: 0000 [#1] SMP
Call Trace:
osc_req_attr_set+0x3ec/0x620 [osc]
cl_req_attr_set+0x63/0x160 [obdclass]
osc_build_rpc+0x483/0x1070 [osc]
osc_io_unplug0+0xc22/0x1910 [osc]
brw_interpret+0x34b/0xea0 [osc]
ptlrpc_check_set.part.23+0x481/0x1df0 [ptlrpc]
ptlrpc_check_set+0x5b/0xe0 [ptlrpc]
ptlrpcd_check+0x4ab/0x590 [ptlrpc]
ptlrpcd+0x29b/0x560 [ptlrpc]
Fixes: c2791674260 ("LU-12043 llite: improve single-thread read performance")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Id8c3b8544ef5652693746471f41c938566875857
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 crashed | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-2 | RHEL 8.8/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-4 crashed | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-5 crashed | RHEL 8.8/x86_64 | ran 8 tests. 2 tests failed: sanityn, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-part-6 crashed | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-1 crashed | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-selinux-ssk-part-2 crashed | RHEL 8.8/x86_64 | ran 7 tests. 2 tests failed: sanity-sec, recovery-small. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs crashed | RHEL 8.8/x86_64, SLES 15.4/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.8/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 9.2/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.8/x86_64, SLES 15.5/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs crashed | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 7 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-arm crashed | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 8 tests. 2 tests failed: sanity-sec, sanity. %% THIS TEST SESSION CRASHED %% | session |
LU-16847 ldiskfs: reduce a memory usage by ost IO threads sectors array don't needs at modern time but large IO generate a large memory consumption. BIO creation code simplified dramatically. Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com> Change-Id: I7da65929f10f1439ef9f3c80094c2300a1df9822
| unique failing test | history |
|---|---|
| sanity2@zfs:test_398c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-3 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-part-4 | RHEL 8.8/x86_64 | ran 9 tests. 1 tests failed: sanity-hsm. | session |
| review-dne-zfs-part-1 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.8/x86_64, RHEL 9.2/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-17190 osc: force I/O when all RPC slots used out by DIO Once all RPC slots are used out by parallel DIOs, force to send I/O RPCs to OST even it exceeds the limit of @max_rpcs_in_flight. This should be a temporary solution for the deadlock caused by parallel DIO. It will send DIO to OST without any throttle when all I/O processes are doing direct I/O. In this case, the RPCs in flight are out of the control of @max_rpcs_in_flight. It may overwhelm the OSS with lots of requests. Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I0c4346b3f14e70a11226997a2664f197066bcbbe
| unique failing test | history |
|---|---|
| sanity2@zfs:test_441 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 | RHEL 8.8/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne | RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-dne-arm | RHEL 8.8/aarch64, RHEL 8.8/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-17190 test: parallel DIO should not cause deadlock This patch adds sanity/test_441 to reproduce the deadlock problem caused by parallel lockless DIO. Test-Parameters: trivial Test-Parameters: testlist=sanity env=ONLY=441,ONLY_REPEAT=20 Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: Ie7e51d79d46bb8cb7e1cc275211066d1386ad6f8
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 crashed | CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. %% THIS TEST SESSION CRASHED %% | session |
| review-ldiskfs-arm | CentOS 8.3/aarch64, CentOS 8.3/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-13669 readahead: increase ra window progressively For KX applications(https://kx.com), use default read ahead per window will hurt performance, we only get good numbers with 1M window size. The reason behind that is it is not really sequenetial mmap read, however sequential read RPC size increased as one RPC size even first time, this potentially cause a lot of page discard. To overcome problem like above, RPC size will be started as 32K and increased twice every time before it reach RPC size(16M etc). Once we reach RPC size, it will grow one RPC size each time as before. With this patch, we got same performance numbers with 1M and 64M read_ahead_per_file_mb. Signed-off-by: Wang Shilong <wshilong@ddn.com> Change-Id: I147aafefb66b574a9b130786597018d079553727
| unique failing test | history |
|---|---|
| sanity-quota@zfs:test_7a | seen in 13 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 8.6/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-part-5 | RHEL 8.6/x86_64 | ran 8 tests. 1 tests failed: recovery-small. | session |
| review-dne-selinux-ssk-part-2 | CentOS 8.5/x86_64 | ran 9 tests. 2 tests failed: sanity-sec, recovery-small. | session |
| review-dne-zfs-part-2 | CentOS 8.5/x86_64 | ran 7 tests. 1 tests failed: sanity-sec. | session |
| review-dne-zfs-part-5 | CentOS 8.5/x86_64 | ran 8 tests. 1 tests failed: recovery-small. | session |
| review-dne-zfs-part-6 | CentOS 8.5/x86_64 | ran 4 tests. 2 tests failed: ost-pools, replay-single. | session |
| review-ldiskfs-arm | RHEL 8.6/aarch64, RHEL 8.6/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
| review-ldiskfs-ubuntu | CentOS 8.5/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
(style) one fewer tabs indentation
(defect) I really don't think that exp_last_request_time can be used to prolong a lock, since even a ping will be enough to keep updating it. It should only be the write RPCs that are prolonging the lock in this way, with a new "exp_last_write_time" field.
I am not so clear why only write RPCs can be considered? In fact, we should wait BL callback so long as client still alive(e.g some locks cover big file queued up before us and they located on other OSTs). Of course, we should cease a upper limitation since the BL ack could be lost. But it should be rare.
My concern here is that some client has a bug and the page flush or lock cancellation is broken (e.g. client-side deadlock, LBUG, etc) and no progress is made flushing pages to the OST, but the thread sending OBD_PING RPCs is still working. If *only* ping or read or statfs RPCs are being sent, then the client is broken and no longer making progress to flush dirty pages and release the DLM lock. The current lock timeout extension code is only doing this for *writes* under the *same* DLM lock, but clearly this is not enough in this case. The next step to relax this would be to do lock timeout extension for writes under *any* DLM lock, since that at least makes it clear that the client is still working to clear up the dirty pages under the DLM locks. In the client debug logs that you looked at, do you know how many locks on the client were seeing BL callbacks? Were there a large number of such blocked locks in the HP list in osc_check_rpcs(), or RPCss that are taking a long time to complete? Looking at osc_check_rpcs() it looks like it is already sending round-robin RPCs for each object in the high priority list, so if the OST is not getting any RPCs from the BL object, then there must be a lot of objects in the HP list. Also, do you know why BL locks were sent to the client? My understanding is that this is a single-client workload, so there shouldn't be lock contention from other clients, and there shouldn't be blocking callbacks on the lock. One possibility is that there are two threads enqueuing locks at the same time on the same object?
I was been working for client side patch, will submit a initial patch shortly as a discussion base. From 10.12 log, more than 500000 locks in the bl queue. But not clear how many in priority queue, since not count it separately. But about 2316 locks added to priority queue in 24s. Also not found obvious long time RPCs. I was also tried to find the reason that BL lock was sent. But even still no luck since limit logs. It need a longer time to collect since the lock timeout can be prolong.
(style) there is no need to check "match == 0" here, since it isn't harmful to set "match = 1" twice
Will do.
I would rather no mess with the timeouts down here. This will be confusing because the caller is using "ldlm_bl_timeout()" or "ldlm_bl_timeout_by_rpc()" to calculate "delay", but the code is essentially ignoring "delay" and working out its own timeout. It would be more clear IMHO to further improve ldlm_bl_timeout() and/or ldlm_bl_timeout_by_rpc() from LU-16062 to take into account the last time the export was sending or processing a write RPC (with a new exp_last_write_time), and then using that to extend the timeout value returned. That keeps the lock timeout calculation consistent in one place, and immediately works wherever ldlm_bl_timeout() is called.
We have two call path relate to expand the timeout. The ldlm_lock_prolong_one and ldlm_refresh_waiting_lock. I add code in there can be unify those two. We also can move it to ldlm_refresh_waiting_lock.
(defect?) the old code set the timer on the lock exactly at "l_callback_timestamp", but now l_callback_timestamp is "delay" seconds in the future, but the timer is set at "delay + scrap / 8", which is strange.
It is my fault, I will keep l_callback_timestamp & timeout sync.
(style) "scrap" is always >= 0, so this check is not needed.
Will do.
OBD_RECOVERY_TIME_HARD is 900s by default, which might be OK by itself (though the same could be done by setting "echo 900 > /sys/modules/ptlrpc/parameters/ldlm_enqueue_min" or similar). However, this is also only adding scrap/8 of the BL AST age, so the client eviction could be extended as much as 7200s (2h)? That seems far too long.
Keep sync l_callback_timestamp & timeout should avoid such issue.
(defect) floating point does not work in the kernel. It might be that CPP is handling this itself because OBD_RECOVERY_TIME_HARD is a constant, but it would be better to use "* 3 / 4" (which will work in all cases).
LU-16285 ldlm: prolong the bl timeout Don't evict the client when BL timeout if the client still busy to work. Also accumulate the time while the lock was refreshed many times. Stop to expand the value if it big than certain limitation. Signed-off-by: Yang Sheng <ys@whamcloud.com> Change-Id: I6fb5b4499259ef993bd8fd7e369889ea9fe5be4d
| unique failing test | history |
|---|---|
| sanityn@ldiskfs+DNE:test_80b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
LU-9839 clio: lov active ios accounting fix ASSERT(atomic_read(&lov->lo_active_ios)==0) is triggered due to a bug in active_ios accounting. For some cl_io_init(,CIT_MISC,,) calls increment the lov_active_ios counter is not protected by the layout lock. So the checks for active_ios != 0 are racy and not preventing another thread from starting new cl_io and incrementing the active_ios counter after any check but before the assertion. The lov_active_ios counter increment should be done under the same condition as taking the layout type lock. The ci_type=CIT_MISC and ci_ignore_layout=1 should not be used in ll_dom_finish_open() as the I/O doesn't come "from the osc layer" and may race with a layout change. Lustre-change: https://review.whamcloud.com/51638 Lustre-commit: 5bc1dd825b700677b002a43463a463c3ccb665ec HPE-bug-id: LUS-11628 Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com> Change-Id: I35fda85b968b847a87e73dd36bbb1648c744d62c Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-3 | RHEL 8.9 / x86_64 | ran 3 tests. 1 tests failed: conf-sanity. | session |
| review-dne-zfs-part-5 | RHEL 8.9 / x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
LU-18452 osc: configuration of checksums on recovery
Enable mount and sysfs flags to enable or disable the feature
to fixup checksums during recovery.
Check:
lctl get_param osc.*.checksum_fix mdc.*.checksum_fix \
llite.*.checksum_fix
Enable:
lctl set_param -P osc.*.checksum_fix=1 mdc.*.checksum_fix=1 \
llite.*.checksum_fix=1
Disable:
lctl set_param -P osc.*.checksum_fix=0 mdc.*.checksum_fix=0 \
llite.*.checksum_fix=0
HPE-bug-id: LUS-12605
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ia42f740012dd829f455fef09b37d824df994a2c8
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-4 | RHEL 8.9/x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
LU-17833 ptlrpc: Check lru_resize during connection Since the parameter log processing might finish before connection is established, so it should check if lru size has been disabled by parameters log in ptlrpc_connect_set_flags(). Lustre-change: https://review.whamcloud.com/55060 Lustre-commit: d79bbae7a6e22b576e0d06f1d4eba28daf11456e OCI-bug-id: LFS-229 Signed-off-by: Di Wang <di.d.wang@oracle.com> Change-Id: I246fcbcd17aa201f80b6950d8eff57489dc81645 Reviewed-by: Patrick Farrell <patrick.farrell@oracle.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
| unique failing test | history |
|---|---|
| sanity2@ldiskfs+DNE:test_160n | seen in 1 other review |
| sanity2@ldiskfs+DNE:test_271a | seen in 1 other review |
| sanity2@ldiskfs+DNE:test_271c | seen in 3 other reviews |
| sanity2@zfs:test_271a | seen in 1 other review |
| sanity2@zfs:test_271c | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-dne-part-5 | RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-selinux-ssk-part-1 | RHEL 8.7/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 | RHEL 8.7/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | RHEL 8.7/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.7/aarch64, RHEL 8.7/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.7/x86_64, Ubuntu 20.04/x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
LU-12325 mdc: reduce DoM lock mode on file close When file is closed and there are no reader/writer then downgrade DoM lock mode to less strict PR mode. That can help the other client to read file data at open and also that keeps any combined bits also in friendly PR mode MDT logic to choose DOM lock mode is changed, now it try to get PR lock on open if there are other PR locks Test-Parameters: testlist=dom-performance Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Change-Id: I949a804d1429181dc80a29c5e2021cbc0c08d4f1
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | CentOS 8.5/x86_64 | ran 5 tests. 1 tests failed: recovery-small. | session |
| review-dne-zfs-part-4 | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
Can you please expand the description of what this patch is doing. It looks like there is a separate thread or timer that is generating RPCs for high priority locks, but I don't understand why this is much different from the current client RPC generation engine? I'm wondering if we are missing something simple in the current RPC engine that it doesn't keep on sending new RPCs for blocked locks when the current RPCs have completed. Maybe something like the high-priority locks/object are being blocked by rpcs_in_flight or similar?
Yes, I set a timer if the bl queue depth has reached a limitation while a blwi insert. Then start flushing out the pages after the timer fired if the blwi still in queue. The different than usual RPC engine is that current engine only consider flush whole range of lock but we need only flush part pages(only one RPC in this patch).
This is a good explanation, but it needs to be added into the commit message so that it can be found in the future, not in a Gerrit comment on an old version of the patch. My main concern is that having two separate RPC engines may cause other issues (eg. overflow of max_rpcs_in_flight, fragmented RPCs, etc). My preference would be to update the current RPC generation to round-robin over HP objects/locks when there is a large HP object/lock that is taking too long to flush. I'm thinking something like limiting the number of RPCs for each object at one time to max(1, max_rpcs_in_flight / 4), and doing a pass over all HP locks and sending an RPC for those that have not had any RPCs sent in some time (obd_timeout / 4). This will ensure that there are no locks left idle for a long time, and if there are many small files that can be written with a few RPCs they will be finished quickly, and the large files that take thousands of RPCs to flush will still have occasional RPCs sent to keep the locks alive. If there are only the large files in the HP list then they will be processed more quickly.
Is there a danger that this could ever divide by zero? I don't think so, (blp_proc_time should always be <= jiffies, so I guess the "+1" is to handle the == case) but it isn't totally obvious.
I was thinking no possible to divide zero since jiffies should change rapidly. But in fact it really crash on divide zero.
(style) it would be more understandable if this was written like "list_for_each_entry()" or similar.
Since bl queue is accessed as FIFO in bl thread. But we need flush the pages as a reverse order(All of blwis before blp_echo_item are older than it). Since the start pointer is different than end pointer. Seem no stand api can be used. But i'll try to find it.
(defect?) I don't understand why this does not just lloop on the same blwi entry each time? The item is not being removed from the list or moved to the end (AFAICS), so it definitely has some danger.
It is my fault. I should set the pos as blp_echo_item->prev. So it was moved one step forward in worst case. Will update.
(style) remove extra blank space
(style) only one space between variable type and name for local variable declarations
LU-16285 ldlm: flush the page out for bl lock The client could be evicted if it is no chance to write out the dirty data and cancel the lock in timely manner. So we made some IO for locks which still in BL queue. The timeout can be prolong to avoid eviction. Signed-off-by: Yang Sheng <ys@whamcloud.com> Change-Id: I7349d45e565cade7581fb756eee448356c6d91f7
LU-17281 ofd: use correct lock end for ladvise In ofd_ladvise_hdl(), tgt_extent_lock() shouldn't use end - 1 as the end value of the range. When willread has no range argument, the ladivse end will be EOF, and this mistake will generate lock end with value EOF - 1. Change-Id: Ie5acdb7ef36a051a4c01da73e937c0b051d4864c Signed-off-by: Li Xi <lixi@ddn.com>
| unique failing test | history |
|---|---|
| recovery-small@zfs:test_155 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
does it need a protection at all?
LU-19335 llite: Convert lli_layout_lock to rwlock_t Convert lli_layout_lock to rwlock_t HPE-bug-id: LUS-13043 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I3db5aa21ac749c19dbd3816079fae66c4606ee43
(defect?) I tend to think both 'VM_RAND_READ' and 'VM_SEQ_READ' are just kernel hints. It's fine to have no hints and I don't think it is correct by implying the IO pattern to be sequential.
We probably need to move it out and revise the `ll_ras_enter()` to take the hints above into consideration.
LU-19622 llite: mmap read fault hint readahead Default to sequential reads when VM_RAND_READ / MADV_RANDOM is not advised. Read fault hint ll_ras_enter with RA_MIN_MMAP_RANGE_PAGES for sequential reads. HPE-bug-id: LUS-12841 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: Ic73ecb985b9a4360b693bc56a07d932e97c2f19c
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 | RHEL 9.4 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-dne-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
Is npages argument really needed here? cl_dio_pages has cdp_count? I think a small refactor could remove this argument.
LU-13814 llite: pass cl_dio_pages Rather than passing an array of struct page prefer the struct cl_dio_pages to ensure the page array is released and the state is up-to-date. Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: Idb958683d2e1197e70291e750721c470de6582a9
ldlm_lock_cancel() sets it to LCK_MINMODE, why NL here ?
LU-17446 ldlm: Do not wait for BL AST RPC completion on cancel If we have sent an AST RPC to the client and while it's in flight the client sent in the cancel, sometimes (esp. if AST or reply to it are lost) even though the lock is already cancelled, whoever is waiting on it is still stuck while trying to resend ASTs. And in the end the client is not even evicted because the lock cancel did come and all is fine, but it can add over a hundred seconds to lock granting process in some non-ideal circumstances. For simplicity we only treat Blocking ASTs like this, since we can only have a single one of this kind. This is adding additional pointer to struct ldlm_lock, but that is already 560 bytes so does not really mean much. Change-Id: Id2231bc3bfc3e094faae2872fe09f3c330d441df Signed-off-by: Oleg Drokin <green@whamcloud.com>
| unique failing test | history |
|---|---|
| sanity-lnet@zfs:test_219 | seen in 2 other reviews |
| sanity-pcc@ldiskfs+DNE:test_7b | seen in 9 other reviews |
| sanity-quota@ldiskfs+DNE:test_1g | seen in 10 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-5 | RHEL 8.8/x86_64 | ran 6 tests. 1 tests failed: sanityn. | session |
| review-dne-part-8 | RHEL 8.8/x86_64 | ran 4 tests. 1 tests failed: replay-vbr. | session |
| review-ldiskfs failed 2× crashed | RHEL 9.2/x86_64 | ran 2 tests. 1 tests failed: lustre-initialization. %% THIS TEST SESSION CRASHED %% | session |
LU-17292 osc: refine cl page references Page should be pinned for radix tree independed from a transfer. Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com> Change-Id: I9f4e07d909df86787ec8706aa781bb7fa5ad8066
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-4 failed 2× | CentOS 8.5/x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
LU-15866 llite: remove unstable page accounting Unstable page accounting is disabled by default due to its performance impact and has no test coverage. So remove it. Signed-off-by: John L. Hammond <jhammond@whamcloud.com> Change-Id: I073b7f9bb0fedf925605172accb597a5863cc709
| unique failing test | history |
|---|---|
| sanity-flr@zfs:test_200 | seen in 29 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-1 | CentOS 8.5/x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
A general thought - I think it would be better to fall back to slow read on error here, rather than add this cost to the fast read path every time. In fact, I think we should probably just fall back to slow read on *any* error from fast read.
Hmm, is tiny_write an issue as well? (see ll_file_write_iter)
don't we need to do this for mkwrite as well?
So we're going to do this for every page. If there are many readers moving many pages, I think this will get *expensive* quickly, because there will be several LCCs, and we will check all of them for every page. Basically, we're taking an inode level rwlock for every page we move, and then walking a list. That list won't have thousands of members, but it could have 10s or 100s of members, and if it's being updated a lot (which would happen with many readers), it would be taking the 'write' lock a lot. If we really want to go this route, it might make sense to cache the most recently used lcc and check it first?
That caching would be tricky, now that I think about it. Maybe not worth the trouble.
I remain worried about cost of this; I think it's correct but I'm worried about the cost...
LU-16665 llite: check whether page under I/O in releasepage() We use seqlock to check if a page has been deleted on this inode during the fault process, allowing us to catch an erronous short read or EIO and retry the I/O. However, the newer kernel (such as Unbutu 2204) introduces @mapping->invalidate_lock. By using this lock, we could get rid of all these seqlock check for the newer kernel which may have impact on the performance. A user can use drop_caches or DONTNEED fadvise to drop unused page cache. In kernel it calls invalidate_mapping_pages() to release pages that are not in use or under I/O. And this function is called without invalidate_lock held. Thus it needs to check wether the current page trying to release in ->releasepage() is under I/O or not. In ->releasepage(), we check whether the current page is under I/O by check whether it is in the I/O range of the read/fault I/O context in the list @lli_lccs(ll_cl_context). The page can only be released if it is not under I/O or in use. Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I281c0815e79ab800944700451c4a168ebddbc7c8
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-subtest-change | RHEL 9.3/x86_64 | ran 3 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-zfs-part-6 | RHEL 8.9/x86_64 | ran 4 tests. 1 tests failed: replay-single. | session |
LU-10499 pcc: Add dio support for data copy during attach PCC attach performance is bottlenecked by single threaded buffered I/O performance. We could do multi-threading, but multi-threaded buffered I/O to one file has a very low performance ceiling. In order to significantly speed up PCC attach performance, we need to switch to DIO. DIO cannot be done from kernel memory due to various restrictions, so we call out to a usermode helper. Note that the helper uses open by fid because given a file pointer, it's not possible to reliably generate the path to a file on Lustre due to container namespace issues. Specifically, the path used by the user may not work for our helper program due to namespace differences. So we must use open by fid for the Lustre side of the copy. This patch improves attach performance from about 1 GiB/s to about 5 GiB/s. This performance figure includes time to read the data from Lustre *and* to write it out to PCC. EX-5014 pcc: avoid deadlock during DIO open attach on rhel7 The Maloo testing fails with sanity-pcc/45 due to the following deadlock on rhel7 kernel: ll_fid_path_cop D ffff9a32db5eb180 0 10783 10782 0x00000080 Call Trace: schedule_preempt_disabled+0x29/0x70 __mutex_lock_slowpath+0xc7/0x1d0 mutex_lock+0x1f/0x2f lookup_slow+0x33/0xa7 link_path_walk+0x80f/0x8b0 path_openat+0xae/0x5a0 do_filp_open+0x4d/0xb0 do_sys_open+0x124/0x220 SyS_open+0x1e/0x20 dd D ffff9a32fb5b6300 0 10779 10755 0x00000080 Call Trace: wait_for_completion+0xfd/0x140 call_usermodehelper_exec+0x179/0x1a0 call_usermodehelper+0x40/0x60 pcc_copy_data_dio+0x267/0x340 [lustre] pcc_attach_data_archive+0x6ff/0xe80 [lustre] pcc_readonly_attach+0x3d2/0xad0 [lustre] pcc_readonly_attach_sync+0x205/0x260 [lustre] pcc_file_open+0x798/0xdd0 [lustre] ll_atomic_open+0xd80/0x1780 [lustre] do_last+0xa53/0x1340 path_openat+0xcd/0x5a0 do_filp_open+0x4d/0xb0 do_sys_open+0x124/0x220 SyS_open+0x1e/0x20 This bug only happened on el7 kernel which uses mutex for inode locking. During ->ll_atomic_open(), the kernel will take this mutex on the parent inode. However, when copy data via the user space helper program ll_fid_path_copy, it will also try to obtain this mutex lock on the parent inode during lookup, resulting in deadlock. Was-Change-Id: I384c7b1979d93183b86bbde311d29a50346a8d56 EX-5014 pcc: minor fixes for parameter checks Improve console message when out-of-range pcc_dio_attach_size_mb values are supplied. Fix sanity-pcc test_49b to allow future limit changes Was-Change-Id: I2bf7d0bf564c954318980f7a09d8713a70f37db9 Test-Parameters: clientdistro=el8.9 mdscount=2 mdtcount=4 testlist=sanity-pcc env=ONLY=45,ONLY_REPEAT=10 EX-bug-id: EX-5014 Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Qian Yingjin <qian@ddn.com> Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Change-Id: Idb2a12296c3e4778763c9b576bbb0ecd2570a458
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 | RHEL 8.6/x86_64 | ran 4 tests. 1 tests failed: sanity. | session |
| review-ldiskfs | CentOS 8.5/x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
This needs to be rebased - not urgently or anything, but I see it's missing the latest version of the patch which added this
[minor] Maybe split this comment to "clearpageuptodate" and "seqlock" parts and move the clearpageuptodate part above the #ifdef? It would be sad if we removed the whole comment when we (some day!) remove the lli_page_inv_lock code completely
LU-16665 llite: remove lli_page_inv_lock in newer kernel This patch remove @lli_page_inv_lock in newer kernel as @mapping->invalidate_lock can achieve the same effect. Test-Parameters: testlist=sanityn clientdistro=ubuntu2204 env=ONLY="16f 16g 95b",ONLY_REPEAT=10 Signed-off-by: Qian Yingjin <qian@ddn.com> Change-Id: I5a26ce366817b3a92f9fe1ca9724ab5a9e13f999
LU-14021 llite: don't touch vma after filemap_fault In case of error filemap_fault unlock mutex vma->vm_mm->mmap_sem, so touching vma is dangerous, it could be reused or freed. The patch uses local file variable to skip vma. Lustre-change: https://review.whamcloud.com/44558 Lustre-commit: 0f5d3c4b954da2f6b880da243dacec52cb4011a6 HPE-bug-id: LUS-10240 Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com> Change-Id: I72cd086645061819fab5b8595a880db64cfb9ff7 Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
LU-15608 sec: fix DIO for encrypted files
With Direct IO, we do not have proper page cache pages. So we need to
retrieve by ourselves the page mapping and the page index of the page
to be encrypted/decrypted.
For the index, we need to use the offset of the page within the file,
and not the object.
So we rename cl_page's cp_osc_index to cp_page_index for that purpose.
cp_osc_index is redundant with osc_async_page's oap_obj_off and only
used by osc_index(), so we also adapt this function.
cp_page_index is initialized in cl_page_alloc(), and accessed in
the OSC layer where the llcrypt primitives are called.
For the mapping, problem is page->mapping is not set to NULL on page
allocation, so it cannot safely be used to see if a page is a direct
I/O page.
Use cl_page for direct I/O and page->mapping for buffered
I/O. (clpage->cp_inode is only set for direct I/O and
cannot easily be always set.)
Without this, we sometimes get panics when page2inode is
used in the OSC layer. (Note the remaining use in dom is
safe because ll_dom_readpage is a page cache helper and
will never see DIO pages.)
Lustre-commit: 966ca46e4aa2eb39c70e49648ffe6fcaaf475536
Lustre-change: https://review.whamcloud.com/46664
Fixes: a71e0dd7f7 ("LU-14306 sec: get rid of bad rss-counter state messages")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Icb53a4e45463b8d3febc2e6212b39dc25719d866
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| custom-1002 | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-subtest-change failed 2× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-dne-zfs-subtest-change failed 7× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity-ec. | session |
| review-ldiskfs-dne-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 5 tests. 1 tests failed: sanity. | session |
LU-12668 ec: Add tests for computing the parity coverage When we write or verify the parities we no longer use the whole stripe, instead we compute the coverage of what ranges of parity is important and what can be ignored. This is based on SEEK_DATA/SEEK_HOLE and EOF. Add tests that we compute these ranges correctly. Test-Parameters: trivial Test-Parameters: testlist=sanity-ec Test-Parameters: testlist=sanity-ec fstype=zfs Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com> Change-Id: Iaee9d0ccb0875f6d515a8adc322fa3cd37bfe0f6
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 retesting | RHEL 9.7 / x86_64 | ran 11 tests. 1 tests failed: replay-dual. | session |
| review-dne-zfs-part-2 retesting | RHEL 10.1 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-ldiskfs-ubuntu retesting | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity-lnet. | session |
(minor) The ll_readahead_handle_work() piece fixes a pre-existing bug that has nothing to do with EC: ll_ra_count_get() reserves the pages and nothing puts them back when cl_io_rw_init() fails, and ll_ra_count_put() is the only decrement of ra_cur_pages. A Fixes: line would let the maintenance branches pick it up:
Fixes: c2791674260b ("LU-12043 llite: improve single-thread read performance")
(defect) The lov_io_set_range() hunk fixes an LBUG that 84c1a4a074 introduced, and that commit is three patches back in this same unlanded series. That leaves 84c1a4a074, 13af17f231 and b718cadc7b each panicking the client on a two-component EC layout, so the series is not bisectable and those revisions cannot be tested on their own.
Can the clamp be folded into 84c1a4a074 instead? If it has to stay a separate patch, it needs its own tag alongside the existing one:
Fixes: 84c1a4a07423 ("LU-12669 ec: recover data from parity")
(minor) This label bypasses cl_io_fini(). cl_io_init()'s contract is that the caller calls cl_io_fini() no matter what it returned, and the commit message points out that this exit stops being rare once a dead import can fail an EC read at init. Should the new path run cl_io_fini(env, io) before dropping the reservation?
(defect) On a component with lsme_dstripe_count == 0 this can push eoff well past lio->lis_endpos.
Neither place that sets the cycle end rounds it to a recovery group on that branch: lov_io_set_range() skips its end-rounding block when dstripe_count == 0, and lov_io_ec_rd_iter_init() only rounds when dstripe_count > 1. So lis_endpos is just the request end, while eoff becomes soff + RGs, and RGs there is ss * 4 (or ss * lo_nr).
lov_ec_read_stripe_pages() classifies pages against eoff alone, so the pages in [lis_endpos, soff + RGs) are grabbed and submitted. lov_io_lock() enqueued only [ec_inner.crw_pos, +crw_bytes) for this cycle, and lov_ecio_add_data_sub() skipped those stripes (lov_stripe_intersects() false against the cycle extent), so lov_sub_get() allocates a fresh sub-IO with no DLM lock -- the same "uncovered page!" LBUG in osc_req_attr_set() that the lov_io_set_range() hunk is fixing.
Worked example, layout "-E 128M --ec 4+2 -E 512M -c 4" (EC component followed by a plain one, as in the mirror layouts in this suite), read [0, 130M) with a dead OST:
cycle in the plain component: lis_pos 128M, lis_endpos 130M
dcount 4, ss 1M => RGs 4M, soff 128M
eoff was min(132M, 130M) = 130M, now min(132M, 512M) = 132M
stripes 2 and 3 cover [130M, 132M) -- outside the enqueued lock
Before the change those pages were EC_DPG_ZERO and never touched. Should the clamp stay bounded by lio->lis_endpos on the dcount == 0 branch, where there is no recovery group to complete?
(minor) The sibling switch below clears the retry count before returning:
io->ci_switch_ec_io = 1;
io->ci_need_restart = 1;
io->ci_ndelay_tried = 0;
RETURN(-ENODATA);
This one leaves ci_ndelay_tried alone, and ll_file_io_generic() carries it across the restart (retried = io->ci_ndelay_tried). So if the read had already restarted once before the import went invalid, the CIT_EC_RD pass lands on ndelay_tried: with a non-zero count and can still take the 10 ms schedule_timeout_interruptible() and set ci_tried_all_mirrors -- the backoff the commit message says this path avoids. Should it reset the count too?
error: lov_lock_enqueue():'osc' dereferencing possible ERR_PTR()
error: lov_lock_enqueue():'osc' dereferencing possible ERR_PTR()
[AI review - fable] (defect) Including CIT_READ here looks dangerous. After the continue, the top lock is granted without covering this stripe, and nothing in the normal read path checks lso_status (only the EC_RD and LSEEK paths do), so pages on the dead stripe are still generated and submitted, and osc_build_rpc() -> cl_req_attr_set() -> osc_req_attr_set() hits the "uncovered page!" LBUG in osc_object.c when osc_dlmlock_at_pgoff() finds nothing - the very crash the commit message wants to avoid. The lov_io_mirror_init() check does not guard this path when ci_cross_ec is unset (a read of a non-EC component of a file whose layout has parity entries elsewhere - lov_lsm_has_parity() is file-wide, and lov_io_ec_rd_start() explicitly supports dcount == 0 components), or when the import goes inactive after cl_io_init(). Before this change the enqueue failed fast with -ESHUTDOWN and the IO returned an error or restarted into EC_RD instead of crashing. Should the skip be limited to CIT_EC_RD, letting CIT_READ fail the enqueue so the existing restart logic switches to EC_RD? As a side effect the CIT_READ marking also leaves a stale LSS_READ_ERR on the lovsub (the normal read path never resets it), which lov_io_lseek_end() then treats as degraded even after the OST is reactivated.
error: lov_lock_enqueue():'osc' dereferencing possible ERR_PTR()
error: lov_lock_enqueue():'osc' dereferencing possible ERR_PTR()
(style) This isn't a bug, but the include groups are inverted here - `<lustre_osc.h>` is a lustre header and belongs before the local `"lov_cl_internal.h"`. The same include added to lov_io.c in this patch is ordered that way.
(minor) `lov_lsm_has_parity()` walks all `lsm_entry_count` entries and is loop-invariant, but it now runs once per sub-lock inside the `lls_nr` loop on every read enqueue. Previously it was only reached from the `rc != 0` LSEEK path. Worth hoisting it above the loop?
LU-12668 lov: proactive dead-OST detection for degraded reads
Instead of letting a read proceed to an inactive OST and fail deep in
the BRW/lock path, detect dead imports during mirror selection and
route to the EC recovery path.
- lov_io_mirror_init() rejects a candidate mirror that has a data
stripe on a deactivated or invalid import, so FLR rotation can still
pick an intact mirror. Only when no mirror is intact does it set
ci_switch_ec_io, which also skips the FLR backoff sleep -- we
already know the OSTs are dead. The scan covers the whole mirror
rather than just the stripes the I/O touches: see the comment on
lov_ec_has_inactive_stripe() for why bounding it to the I/O extent
is not safe until the size path handles unreachable stripes.
- lov_ecio_add_data_sub() marks such stripes errored up front, so the
recovery loop reconstructs them instead of building a sub-IO that
cl_io_iter_init() or the lock enqueue would reject.
lov_io_set_range() rounded the CIT_EC_RD range end up to a raid-set
boundary using the geometry of the component holding the read end, but
that rounded value can land past the component. The next component
anchors its raid sets at its own e_start, so the end fell mid-raid-set
there: lov_io_ec_rd_iter_init() cut the cycle short at it while
lov_io_ec_rd_start() still read a whole recovery group, leaving pages
outside the lock lov_io_lock() had enqueued. osc_req_attr_set() then
LBUGged ("uncovered page!") from brw_queue_work and panicked the
client. Clamp the rounded end at the component boundary; the read
cannot need data past it, since the request itself ended inside that
component and each component is erasure coded independently.
Reproduced with a -E 4M -c 3 --ec 2+1, -E -1 -c 5 --ec 4+1 layout.
ll_readahead_handle_work() leaked its ra_cur_pages reservation when
cl_io_rw_init() failed. That exit is rare today but becomes routine
once a dead import can fail an EC read at init, and ll_ra_count_put()
is the counter's only decrement, so readahead would stop for the whole
mount and never recover. Release the reservation from a single exit
label that every path past ll_ra_count_get() reaches, and call
cl_io_fini() on the failed init, which cl_io_init() requires no matter
what it returned.
Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools
Test-Parameters: testlist=sanity-ec ostcount=8
Test-Parameters: testlist=sanity-ec ostcount=8 fstype=zfs
Test-Parameters: testlist=sanity ostcount=8
Fixes: c2791674260b ("LU-12043 llite: improve single-thread read performance")
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I0913e1ee977c9850193c92835edb185b0aedc6d4
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | RHEL 10.1 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-arm | RHEL 8.10 / x86_64, Rocky 9.5 / aarch64 | ran 6 tests. 1 tests failed: sanity-sec. | session |
(minor) The body doesn't account for everything in the diff. Hunks I couldn't tie back to it: - lproc_llite.c adds `folios_order_min` and `folios_order_max` in addition to `large_folios`, but only `large_folios` is described. - obdclass/obd_sysfs.c adds a new read-only `lustre.dirty_pages` attribute. - llite_mmap.c wraps ll_page_mkwrite() in sb_start_pagefault()/sb_end_pagefault(). - pcc.c adds folio_order_adjust() and a new pcc_mmap_pages_convert() implementation. - obd_support.h adds OBD_FAIL_OSC_LRU_UNRESERVE_DELAY and sanity.sh test_277 is reworked. Could these be described, or split out into their own changes?
(style) The component tag says llite, but the bulk of the diff is under lustre/osc/ (osc_cache.c, osc_request.c and osc_io.c together are roughly 1900 of the changed lines, against about 670 in lustre/llite/). Since the change spans llite/vvp down to osc, would `clio:` be the more accurate tag?
(defect) bp_pgno was s32 in brw_page and is s16 here, but the value assigned to it is not narrowed anywhere upstream: cl_dio_pages::cdp_pgno and cl_page::cp_pgno are both still s32, and cl_page.c fills cdp_pgno[] from folio_page_idx(), which for a DIO buffer backed by a 1G hugetlb page reaches 262143. osc_prep_async_page() and osc_page_submit() then do `oap->oap_brw.bp_pgno = cl_page->cp_pgno`, so the index truncates and brw_folio_page()/brw_kmap_local() land on the wrong page - silently wrong data on O_DIRECT. Can bp_pgno stay s32, or does the DIO path need a bound check?
(style) This isn't a bug in itself, but WARN() taints the kernel and is not rate limited, and brw_bytes() is called once per brw_ext in several tight loops (osc_desc_add_brw(), osc_checksum_bulk*(), the short-io copy), so a single bad extent would flood the log. If the state is genuinely impossible an LASSERTF() reads better; if it is possible, the fall-through returns a negative byte count that callers feed straight into min_t() and memcpy() lengths.
(minor) Adding superblock freeze protection to the mkwrite path is a real behaviour change and doesn't seem related to multi-order folios. Would it be better as its own patch, so the interaction with the DLM locking and RPCs done under __ll_page_mkwrite() gets reviewed on its own?
(minor) New tunables need a man page in the same patch - Documentation/man4/ has llite.sync_on_close.4 and llite.enable_setstripe_gid.4 as the pattern to follow. Could llite.large_folios.4, llite.folios_order_min.4 and llite.folios_order_max.4 be added (and lustre.dirty_pages for the new obd_sysfs.c attribute)?
(defect) On the failure path filemap_add_folio() has already done __folio_clear_locked() before returning, so this unlocks a folio that is not locked - VM_BUG_ON_FOLIO() on a CONFIG_DEBUG_VM build, and a stray clear_bit_unlock() plus wakeup otherwise. Should the unlock move below the `if (rc) break;`?
rc = filemap_add_folio(...);
folio_put(folio);
if (rc)
break;
folio_unlock(folio);
warn: __use_fast_io():missing conversion: 'ras->ras_window_start_idx + ras->ras_window_pages' 'unit_byte + unit_page'
warn: __use_fast_io():comparing different units: 'ras->ras_window_start_idx + ras->ras_window_pages < ras->ras_next_readahead_idx + skip_pages' 'unit_byte < unit_page'
warn: __use_fast_io():missing conversion: 'ras->ras_window_start_idx + ras->ras_window_pages' 'unit_byte + unit_page'
warn: __use_fast_io():comparing different units: 'ras->ras_window_start_idx + ras->ras_window_pages < ras->ras_next_readahead_idx + skip_pages' 'unit_byte < unit_page'
(defect) This sleeps 1ms on every fault that lands on a multi-page folio, in the page-fault path. Combined with the comment ("induce a small delay for memory copy into folio") this reads as debug scaffolding rather than something intended to ship. Should it be dropped?
LU-19895 llite: prepare multi-order folios Handle multi-order folios from user pages (DIO) tracking one PAGE_SIZE chunk per cl_page. Prepare for multi-order folios allocation tracking each folio and number of pages though osc_extent to brw_ext. This enables allocation of large folios on buffered write path Large folio supported is available with Linux kernels 6.12 and later Enable large folio support via mount and get/set param: lctl get_param llite.*.large_folios lctl set_param llite.*.large_folios=[0|1] Default is enabled. Adjust folio order max: lctl get_param llite.*.folio_order_max lctl set_param llite.*.folio_order_max=[0-MAX], default is MAX View folio order min: lctl get_param llite.*.folio_order_min Maximum order equates to a 1M folio, explicitly (20 - PAGE_SHIFT) which is 8 on 4k page platforms and 4 on 64k page platforms. Test-Parameters: optional clientdistro=el10.0 Test-Parameters: clientdistro=el10.1 HPE-bug-id: LUS-13263 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I5c83d9555a2563c764ef6ec12d227b4890db3f5e
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | RHEL 9.7 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
(minor) "stop handing out EC layouts" doesn't quite match the code. mdt_pattern_types is only read in mdt_connect_internal(); nothing in mdt/ or lod/ consults it when a layout is created, so an MDT with parity removed will still create EC layouts on request. What it stops is clients *using* parity components, and only for clients that mount afterwards. Worth noting too that the negotiated mask does not gate creation on the client either: with lustre.enable_flr_ec=0 (the default) a client can still create an EC file via llite.*.enable_erasure_coding and then get -EIO writing it. Should layout creation be gated on the negotiated mask as well?
This baseline omits LOV_PATTERN_COMPRESS, but lov_pattern_supported() right below still lists `LOV_PATTERN_RAID0 | LOV_PATTERN_COMPRESS` as a pattern this client's IO stack handles.
So for a compressed component lov_lsme_usable() -> lov_pattern_available_mds() computes
pattern_base = 0x801
0x801 & ~LOV_MDS_PATTERN_SUPPORT_217 = 0x800 /* != 0 */
and returns false, on both branches: the compat branch uses this macro, and the negotiated branch uses the MDS mask, which is capped by MDT_PATTERN_TYPES_SUPPORTED = baseline | PARITY. There is no value of mdt.*.pattern_types that can put COMPRESS back (conf-sanity test_163a asserts `+compress` is silently dropped).
The result is that lov_init_composite() leaves every compressed component !lle_valid and skips lco_init(), so lov_io_rw_iter_init() returns -EAGAIN/-EIO for a plain compressed file. This is reachable today: lsme_unpack() accepts COMPRESS components when llite.*.enable_compression is set (lov_ea.c), and lod_generate_lovea() packs them.
Should the mask be derived from what the client actually supports (i.e. include COMPRESS in both the 2.17 baseline and MDT_PATTERN_TYPES_SUPPORTED), rather than a hard-coded RAID0/MDT/OVERSTRIPING set?
(style) F_HOLE and F_RELEASED are layout flags rather than pattern types, and every consumer strips them first (`pattern & ~LOV_PATTERN_F_MASK` in lov_pattern_available_mds(), `& ~(F_RELEASED | F_MASK)` in lov_pattern_supported()). lov_pattern_bit2str() also has no names for bits 30/31, so they never show up in mdt.*.pattern_types or lov.*.mds_pattern_support. They only put two bits nobody reads on the wire in ocd_pattern_support. Could the macro just be RAID0 | MDT | OVERSTRIPING?
(minor) `data` here comes from obd_get_info(KEY_CONN_DATA) on sbi->ll_md_exp, which is the LMV export, and lmv_get_info() forwards that key to `lmv_tgt(lmv, 0)` only. So on DNE the stored mask is whatever MDT index 0 negotiated; mdt.<fs>-MDT0001.pattern_types has no effect on any client, even though the parameter is per-MDT and the man page shows a single-MDT example. Should the client intersect the masks from all MDTs, or should the caveat be documented?
(minor) This version gate was flagged on an earlier patchset ("this version should be updated when the patch is refreshed") and answered "Done", but it is still 2.17.52.224 here and in test_163a/test_163b.
The things these tests check (`pattern_support` in the mdc/osc import, mdt.*.pattern_types) only exist from this patch, so any server in 2.17.53..2.17.57 passes the gate without the feature. The tests do fall through to a skip on the missing parameter, but the gate should be the version this lands in.
LU-12187 lov: MDS layout pattern negotiation
Add 'obd_connect_data::ocd_pattern_support' to allow the client
and MDS to negotiate the file layout patterns that they support.
This prevents clients from trying to use layout types not
recognized by the MDS, and allows the MDS to (potentially)
convert/filter existing file layouts to a format that the
client understands.
This will be used by FLR-EC to negotiate whether the client
and MDS support LOV_PATTERN_PARITY layouts, and others in
the future.
Add LOV_MDS_PATTERN_SUPPORT_217 for compat with pre-2.18
MDS that do not send ocd_pattern_support. Generalize
lov_pattern_available_mds() to check all patterns against
the MDS-negotiated mask. The client sends its supported patterns
to the MDS, which responds with the intersection of both sets.
The client stores the negotiated mask only when the server
grants OBD_CONNECT2_FLR_EC. target_handle_connect() echoes
un-negotiated ocd fields back from the request, so an ungated
store would let the client's own request value defeat the
pre-2.18 fallback.
Gate pattern usability at layout init rather than at parse time:
lov_init_composite() consults lov_lsme_usable() (pattern support
intersected with the MDS-negotiated mask), and leaves unusable
components un-initialized and !lle_valid, the same as components
with an unknown pattern. Parsing itself stays pattern-agnostic
to preserve lsme alloc/free symmetry.
Adds mdt.*.pattern_types which gives an administrator the
ability to enable or disable layout types at runtime, and a
read-only lov.*.mds_pattern_support which exposes the
effective negotiated mask on the client. The mdt_enable_flr_ec
module parameter now only sets the initial pattern_types value
when an MDT starts up, so mdt.*.pattern_types is what changes
the advertised types at runtime. Add man pages for both
parameters and for the renamed lustre.enable_flr_ec parameter.
Rename the 'llite_enable_flr_ec' module parameter to
'enable_flr_ec', so it is accessed as 'lustre.enable_flr_ec'
per LU-14144 convention. Add the 'lustre' module to the libcfs
parameter path list so the client module parameters are reachable
via 'lctl {get,set}_param --module' instead of a hard-coded
/sys/module path.
Assisted-by: ClaudeCode:Opus-5 llm_code_and_review_tools
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=conf-sanity env=ONLY="163 163a 163b"
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Change-Id: Iaef716e28014be5b91dd50b117dd0881f5c37f37
(defect) The body only describes removing unused variables, but the osc_page.c hunk also adds a new early return when the LRU wait is interrupted. That is a behaviour change, not a variable removal, and nothing in the message accounts for it. Could the message describe the new abort path (and why it is correct) so a reader isn't surprised by it? If the intent is really just to silence the warning, the alternative would be to drop the assignment entirely and leave the retry loop as-is.
(minor) If the abort-on-signal behaviour is intentional, this is a fix for a loop that could not be interrupted, and it would help to carry a Fixes: tag pointing at the commit that added the uninterruptible retry:
Fixes: e8b421531c16 ("LU-6271 osc: further OSC cleanup after eviction")
(minor) This sha doesn't look like the origin of the bug. osc_lru_reserve() as added by e8b421531c16 had no retry loop at all - a single best-effort cmpxchg, returning 0 when it could not reserve, and no `rc` variable.
The `again:` loop over an unchecked l_wait_event_abortable() came in later:
2a34dc95bd10 ("LU-12142 clio: fix hang on urgent cached pages")
and `rc` only became set-but-unused when 776e163cf542 ("LU-17180 ptlrpc: don't block ptlrpcd too long") replaced `rc = ptlrpcd_queue_work(cli->cl_writeback_work); if (rc) return 0;` with a plain schedule_work().
Since the substance of this hunk is making the wait abortable again, 2a34dc95bd10 looks like the right target:
Fixes: 2a34dc95bd10 ("LU-12142 clio: fix hang on urgent cached pages")
It isn't clear from this patch whether this is an "unused variable" or a bug that "rc" is not being checked when the wait is interrupted? One of the main reasons for `l_wait_event_abortable()` is to allow users/applications to gracefully CTRL-C some thread stuck in the syscall for a long time waiting for the server. If `rc` is not being checked here, then this just becomes a busy loop that cannot be interrupted. However, it doesn't look like this function allows an error return (and it is not checked in the caller), so maybe this is the best we can do. Maybe @bobijam@whamcloud.com or @pfarrell@whamcloud.com have an opinion here?
This is a good point. Either way, this patch would be wrong. If we don't check rc, then this shouldn't be abortable. I think l_wait_event_abortable should be marked __must_check to prevent this type of bug.
Agree about the __must_check. I think we should check rc and break like the above code.
Up here the `rc` is checked and the loop is aborted if the wait is interrupted.
(defect) osc_lru_reserve() returns `unsigned long`, and its value is the number of pages reserved - there is no error channel. Returning -EINTR here hands the caller (unsigned long)-4, i.e. 0xfffffffffffffffc.
osc_io_lru_reserve() stores it verbatim and discards nothing:
oio->oi_lru_reserved = osc_lru_reserve(osc_cli(osc), npages);
RETURN(0);
So the I/O keeps running with a bogus reservation, and two things follow.
osc_lru_alloc() takes the reserved-slot shortcut for every page of the I/O:
if (oio->oi_lru_reserved > 0) {
--oio->oi_lru_reserved;
goto out;
}
cl_lru_busy is incremented but cl_lru_left is never debited, so the LRU budget is bypassed for the rest of the I/O.
Then osc_io_rw_iter_fini() gives the remainder back:
if (oio->oi_lru_reserved > 0)
osc_lru_unreserve(osc_cli(osc), oio->oi_lru_reserved);
and osc_lru_unreserve() does atomic_long_add(npages, cli->cl_lru_left) - the huge unsigned value converts to a large negative long, so cl_lru_left is permanently reduced by roughly (4 + pages touched). cl_lru_left points into the mount-wide cl_cache->ccc_lru_left, so every OSC on the mount is affected, and once it goes negative the next unreserved allocation trips
LASSERT(atomic_long_read(cli->cl_lru_left) >= 0);
in osc_lru_alloc(). The trigger is a fatal signal delivered while a writer waits for LRU slots, which is exactly the case l_wait_event_abortable() exists for.
Would `return 0;` (nothing reserved) work here instead? osc_lru_alloc() then does its own per-page reservation and already handles the abort correctly with `rc = -EINTR; break;`. Alternatively, propagate the failure through osc_io_lru_reserve()'s int return so the I/O actually stops - but that needs the callers of cio_lru_reserve to honour it.
> (defect) osc_lru_reserve() returns unsigned long, and its value is the number of pages reserved - there is no error channel. Returning -EINTR here hands the caller (unsigned long)-4, i.e. 0xfffffffffffffffc. This look correct. I think even coverity will flag this. (-1 just to highlight this)
(style) Not a bug, but there's a stray `;` left after the comment:
return 0; /* reserve nothing */;
(Your thoughts?) Tim, looks like cksum_counter is already being calculated on successful read. Why not (debug)print it below along with client_cksum. Instead of deleting it completely.? This can give us number of successful read's inbetween failures where it is reset to 0.
I'm skeptical it would be useful. We could get similar info (i.e. number of successful reads vs failures) by comparing llite stats to the number of error/debug messages. Plus, static variables like this aren't thread safe. The stat may not be reliable anyway.
I think in the old days the cksum_counter was used to change the `CDEBUG(D_PAGE)` into `CDEBUG(D_ERROR)` when the number of errors exceeded some threshold? I see something similar below with `cksum_missed`. That said, I don't think there is much value to keeping this anymore, just overhead that doesn't provide any value.
LU-16518 osc: fix -Wunused-but-set-variable warnings
Remove unused variables in osc. Clang emits a warning on set
but unused variables, so remove them. For osc_lru_reserve() we don't
handle the rc returned by l_wait_event_abortable(). If we do get
an interruption abort and report we reserved nothing.
Test-Parameters: trivial
Fixes: e8b421531c16 ("LU-6271 osc: further OSC cleanup after eviction")
Signed-off-by: Timothy Day <timday@thelustrecollective.com>
Change-Id: I7cbfbb937ba5b9947bf685fbd3bbff2b6f3b461d
| unique failing test | history |
|---|---|
| racer@ldiskfs+DNE:test_2 | seen in 65 other reviews |
| sanity-lfsck@ldiskfs+DNE:test_18e | seen in 12 other reviews |
(defect) The description says the read-only state is not checked for page discard coming in as CIT_MISC+ignore_layout, but the code does not key on that at all - lsm_is_rdonly() is still called for that I/O type whenever `lo_lsm` happens to be non-NULL. The skip is keyed on the pointer, not on the I/O type. Could the body also say why `lo_lsm` can be NULL here? The OSC path runs without lo_type_guard, so lov_layout_change() can be sitting between lov_fini_composite() and the new llo_init(). Without that, a later reader cannot tell what the NULL case corresponds to.
(style) Change-Id: normally sits after Signed-off-by: - that is the order the Lustre commit-msg hook produces.
Should we skip this for any other cases as well?
You mean only check for type CIT_READ CIT_WRITE CIT_SETATTR CIT_FAULT and skip other types?
I think the more appropriate change is to check lo_lsm before checking the rdonly state, as CIT_MISC+ignore_layout is called from the OSC, and the object's lo_lsm could have already been freed by then.
done
(defect) This is a test and then a separate dereference of a field that nothing on this path serializes. LOV_2DISPATCH_MAYLOCK() in lov_io_init() deliberately skips lov_conf_freeze() for CIT_MISC+ci_ignore_layout, and lov_io_init_composite() does not bump lo_active_ios for it, so lov_layout_wait() never waits on this I/O.
lov_free_memmd() does:
*lsmp = NULL;
kref_put(&lsm->lsm_refc, lsm_free);
A read that lands just before that store still returns a pointer that lsm_free() can release before lsm_is_rdonly() gets to `lsm_is_rdonly`. Does this shrink the race window rather than close it?
The CIT_MISC+ignore_layout case never touches lo_lsm anywhere else - it returns at the ci_ignore_layout test further down, before the lo_lsm users. So keying the skip off the I/O type would keep that path from reading lo_lsm at all, and would match what the commit message describes:
if (!(io->ci_ignore_layout && io->ci_type == CIT_MISC))
rdonly = lsm_is_rdonly(obj->lo_lsm);
Maybe have a copy of the line `rdonly = lsm_is_rdonly(obj->lo_lsm);` in every case of the switch-case that needs it?
error: lov_io_slice_init():we previously assumed 'obj->lo_lsm' could be null (see line 727)
null obj->lo_lsm is usually coming from CIT_MISC
LU-20156 lov: dont check layout rdonly for OSC page discard
It should not check the read-only state of the LOV layout for
page discard from OSC layer via CIT_MISC+ignore_layout as it
does not care/access LOV layout related info.
Fixes: ce98bfe5f72 ("LU-10499 pcc: add readonly mode for PCC")
Change-Id: I18506033ed53fbc6376fc8db546efbc0bfc9e350
Signed-off-by: Yingjin Qian <qian@ddn.com>
| unique failing test | history |
|---|---|
| sanity-slow@ldiskfs+DNE:test_255c | seen in 2 other reviews |
| sanityn@ldiskfs+DNE:test_121 | seen in 5 other reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | RHEL 9.7 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 | RHEL 10.1 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
The body explains the postponement, but three parts of the diff are not accounted for. The biggest one is the MDT exclusion in obd_stale_export_get(): the fix is deliberately disabled for MDT exports, so DoM writes from an evicted client are still exposed. That limitation belongs in the message rather than only in an XXX comment. The new OBD_FAIL_OST_PAUSE_COMMITRW fault point in ofd_commitrw_write() and the two new replay-dual subtests are also unmentioned.
> > we can do what is enough to resolve the problem. to grab. or to mark individual locks as not to be cancelled on eviction - so that you know which ones to leave on the export in ldlm_bl_thread_exports. > this is what the patch does - do not cancel on eviction and wait for all the export's RPC to complete. no, the patch postpones all the locks on the export from being cancelled, what I suggested above is to grab/mark only involved locks in ptlrpc_server_request_add for the on-going IO. others can be cancelled immediately. > > > once IO (in ofd) started we don't check locks until the operation is done. > > we prolong locks twice in advance in ptlrpc_server_request_add() and at the end in ptlrpc_server_finish_request() > we do, but eviction can happen during bulk, for example which may take dozen on seconds and during this time the corresponding thread just sleeps being unable to check/prolong locks. bulk starts after ptlrpc_server_request_add() which prolongs locks where I suggested to grab to corresponding locks, thus at the time of the eviction it will be clear which one are not to be cancelled. 1 operation under 1 lock should not block for dozen of seconds other 1000 locks on the same export which can be cancelled immediately. btw, there is another thread which is able to prolong if there is an IO under an expired lock - see expired_lock_main->ldlm_lock_busy. this loop could be optimised here as well.
we can't grab a lock on both server and client. marking makes no sense as few RPCs can be covered by a single lock.
(style) `exp` is used bare here while every other reference in this macro is parenthesised as `(exp)`. Worth matching the rest of the macro so a non-trivial argument expression can't bind wrongly.
ldlm_bl_thread_wakeup() is only declared in lustre/ldlm/ldlm_internal.h, which is a module-private header, and it has no EXPORT_SYMBOL. obd_class.h is a public header included all over the tree. It builds today only because the two users of class_export_rpc_dec() (ptlrpc/service.c via ptlrpc_internal.h, and ldlm/ldlm_lib.c) both happen to pull in ldlm_internal.h and both live in ptlrpc.ko. The first caller added from ofd/mdt/obdclass gets an implicit-declaration error, or an unresolved symbol at modpost. Would it be better to declare the wakeup in a public header (or route this through a small obdclass helper) so the macro stands on its own?
Is this really just unique to the MDS? What about OSTs where we're running clients for hot pools?
Maybe I'm misunderstanding this, but isn't this making a very large change - We will now not do lock cancellation while there are any RPCs running? But on a busy system there will be RPCs ~all the time, so this seems like it would be a very big behavior change with possible performance impacts?
this way the thread cannot take another job to handle. this check should be done right in ldlm_bl_get_work
I tend to agree, but we'd block anyway? and we have few threads to handle this?
(minor) The loop always breaks immediately after list_del_init(), so nothing is iterated past a removal and `tmp` is never used for its purpose. Plain list_for_each_entry() would say what this does more clearly.
This carries forward an open question from patchset 29 that the current revision still doesn't answer, with some extra detail. The comment says an MDS can deadlock because RPC handling may need to evict its own client. The same shape looks reachable on an OST: a service thread holds exp_rpc_count on E and blocks in tgt_extent_lock()/ldlm_cli_enqueue_local() on a lock that conflicts with one of E's own granted locks; E's locks are only dropped by ldlm_bl_thread_exports(), which now can't get E because exp_rpc_count > 0. What structurally prevents that cycle on OST that doesn't hold on MDT? There is also no bound and no fallback wake here. If exp_rpc_count never returns to zero the export stays on obd_stale_exports forever, which keeps it on obd_unlinked_exports, and target cleanup then spins in obd_exports_barrier() printing "Is it stuck?" (and obd_zombie_impexp_stop() asserts the list is empty). Previously the export was always picked up.
Matching on obd_name means the fix silently turns itself off for the OSTs of any filesystem whose name contains "MDT" - `mkfs.lustre --fsname=testMDT` gives obd_name "testMDT-OST0000", and strstr() finds it.
The device type is what's actually being tested here, so comparing the type would be exact:
strcmp(exp->exp_obd->obd_type->typ_name, LUSTRE_MDT_NAME) != 0
That is the idiom used elsewhere, e.g. tgt_handler.c:973. It also avoids running strstr() over every stale export on every scan while holding obd_stale_export_lock.
(style) Test preconditions in this suite read as the positive condition that must hold:
[[ "$ost1_FSTYPE" == ldiskfs ]] || skip "needs ldiskfs backend"
The negated `&&` form is what the tree is moving away from.
(minor) The suggestion to use /dev/urandom instead of /dev/random was raised on patchsets 38 and 39 and marked resolved, but both dd calls here (and in test_34b) still read /dev/random.
LU-16064 ldlm: postpone lock cancellation until all export's RPCs are complete, otherwise lock can be granted to another client and then RPC processing from just evicted client can break data consistency. Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Change-Id: Id7944cb5583cbe5997e96ee413f7ec70d3faf287
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-selinux-ssk-part-2 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-sec. | session |
(minor) "Augment"?
Auggie is Augment's cli tool you can run in a terminal, much better than mucking around with vs code and such: https://docs.augmentcode.com/cli/overview
Not properly aligned.
Better would be
if (!lnd || !lnd->lnd_nl_get || !lnd->lnd_keys)
return rc;
It removed one level of indentation and the checkpatch issue below.
LU-19769 lnet: simplify lnet_net_show_dump This is 100% agent-refactoring using auggie and whatever default model they use underneath. 1. Reduced Function Size: The main function went from ~363 lines to ~142 lines (61% reduction) 2. Created 7 Helper Functions: • lnet_ni_format_cpts() - Formats CPT list as a string • lnet_ni_show_basic_info() - Outputs NID, status, and interface info • lnet_ni_show_stats() - Outputs basic send/recv/drop statistics • lnet_ni_show_msg_stats_type() - Outputs detailed message statistics • lnet_ni_show_health_stats() - Outputs health monitoring statistics • lnet_ni_show_tunables() - Outputs network and LND tunables • lnet_ni_show_extended() - Orchestrates all extended information output 3. Improved Code Quality: • Reduced nesting depth from 6 to 4 levels • Better separation of concerns • Each function has a single, clear responsibility • Comprehensive documentation for all helper functions • Improved error handling and propagation 4. Reusability Analysis: • Several helper functions (CPT formatting, basic info, stats, health stats) can be reused in other parts of the codebase • The message stats function is specific to network interface dump but follows a pattern that could be applied to similar functions like lnet_peer_ni_show_dump() Change-Id: I0843206c84da5f53c48550ef1ec047a3c50b39f9 Signed-off-by: Oleg Drokin <green@whamcloud.com>
(style) should be named `ll_d_ancestor()` so that it is more clear where it came from (minor) should have a configure check for d_ancestor() export in case that is changed in upstream kernels.
(defect) This is OK for testing, but it isn't helpful to print this on the server console every time that someone passes the wrong arguments to `mv`. This should *at the very most* be CWARN(), but probably be a CDEBUG() since it could happen during normal operation and there isn't anything "wrong" with the filesystem that the admin needs to fix.
note it's client console, but the usefullness is still questionable.
(defect) this is accessing "new" and "inode" after `dput(new)` and `iput(inode)` above. This should be printed first
ah yes. thanks!
seems not, new dentry (which is actually the "old" dentry found by the alias search, but the naming comes from d_splice_alias) has an extra ref from the search,
sorry. wrong reply :-) Andreas, your comment is right.
? We should fix this test?
I do not know, it is a way to create nested aliases to directories. I think maybe it is not a good idea to allow access to .lustre by its fid in ./lustre/fid/FID
That should be disabled as part of this patch? If you can do open-by-FID you shouldn't ever need to do that inside the `.lustre/fid/` directory.
Yes, I guess that's a good question, why do we even need .lustre/fid nowadays?
I once suggested removing .lustre/fid and boy did I get hate.
This really shows this patch breaks things.
yes, it is possible to fix as the Lustre root case, but really , is it important to have an access to .lustre/ and .lustre/fid using .lustre/fid/* ? > No modern kernel allows multiple alias to the same directory. not sure what do you mean, but VFS tries to eliminate the extra dir aliases since this issue https://bugzilla.kernel.org/show_bug.cgi?id=7178 in 2006.. the mentioned code fix now is in d_splice_alias(). I see in 7.0-rc the d_splice_alias_ops() still has the code to eliminate extra dir aliases. the problem might be that the my fix doesn't do it with the same level of atomicity as d_splice_alias() does. Well I think it is fixable I can try call d_splice_alias() for all dir dentries except the Lustre ROOT one (and probably .lustre & .lustre/fid).
James, it isn't clear if the sanity test_233b represents a valid use case or not. I can understand that "open-by-FID" should work for the `.lustre` and `./lustre/fid` *FIDs* (and should find the already-instantiated dentries/inodes for those FIDs), but it isn't clear that `$MOUNT/.lustre/fid/OBF_FID` should work via the pathname, so skipping test_233b seems reasonable. On the flip side, this is fixing a real issue that has nothing to do with .lustre/fid, but can be triggered with regular filesystem operations.
Yes its important. Its not just about test 233b. Its about open(/lustre/.lustre/fid/OBF_FID") which is done by HSM. This patch impacts the atomic open path. If 233b is broken I doubt HSM will survive this change.
`open("$MOUNT/.lustre/fid/OBF_FID")` doesn't make sense. You could just use `open("/lustre/.lustre/fid")` to get a handle on the open-by-fid directory, or `llapi_open_by_fid(OBF_FID)` (which would do `dfd = open("$MOUNT")` then `open_by_handle_at(dfd, OBF_FID)` to get a handle on the open-by-fid directory.
From a consistency point of view I guess it is good to be flexible, but this could also be special-cased in the code if `$MOUNT/.lustre/lustre/fid` is being used to lookup itself, then just return the parent directly?
James, no, it is only about test 233b. the test accesses .lustre/ by .lustre/fid/<FID of .lustre>. It creates a loop in dcache because .lustre is a parent of .lustre/fid/<FID of .lustre> and the result of the lookup gets rejected by the code I added. The same goes for .lustre/fid, but there are no more exceptions, other objects do not have "nested aliases" and can be accessed by .lustre/fid/<whatever Lustre object FID> .
If test 233b is not a problem then I can push what Neil did to fix this problem years ago. The only reason I didn't push the LU-11501 fix years ago was due to RHEL7 being to old to handle the dcache handling changes.
LU-20085 llite: client vfs rename hang ll_splice_alias doesn't try to reuse an existing dir alias if its name is not equal to the new one. It leads to a deadlock during rename when VFS tries to lock inodes to both names/dentries, the same inode getting locked twice. The code makes an exception for accessing Lustre ROOT inode by .lustre/fid/<ROOT_FID>, but doesn't allow the same for .lustre and .lustre/fid, these objects became not accessible by their FIDs using .lustre/fid/, the corresponding test case gets disabled. HPE-bug-id: LUS-13174 Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com> Change-Id: I280c016208ecdc529d7735abb36dba3dc3d15df9
(defect) the `connect_flags` parameter already prints `connect_flags2`
... interesting ...
OK, we've got some substantial formatting issues generating this huge diff, I see
Hmm, shouldn't this be "BEFORE encryption"?
LU-20178 osc: compression engine and BRW integration Implement the core client-side compression engine and integrate it into the OSC bulk read/write (BRW) path: - New osc_compress.c: page-based compression/decompression using lz4, lz4hc, and lzo with configurable chunk sizes from 64KB to 4MB. Handles multi-page chunk assembly, bounce page allocation, and partial-chunk edge cases. - New osc_compress.h: internal API for the compression engine including chunk iterators and helper macros. - osc_request.c refactor: split osc_brw_prep_request into modular helpers for page preparation, encryption, compression, NIO buffer packing, and short I/O. Write path compresses before sending; read path decompresses after receiving. Added compression statistics tracking. - osc_io.c: set compression parameters on cl_page from the layout's per-component compression settings. - cl_object.h/cl_object.c: add compression type and chunk size fields to cl_page for per-page compression context. Signed-off-by: Brian Barbisch <brbarbis@microsoft.com> Test-Parameters: Ignore Change-Id: I5321248314db9d9d4f9a3daa97fb34126b236753
| unique failing test | history |
|---|---|
| sanity3@ldiskfs+DNE:test_272c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity3@ldiskfs+DNE:test_272d | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity3@ldiskfs+DNE:test_430a | seen in 1 other review |
| sanity3@zfs:test_272c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity3@zfs:test_272d | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_36a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_38 | seen in 1 other review |
| sanity-flr@ldiskfs+DNE:test_41 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_44a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_50b | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_50d | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_61a | seen in 45 other reviews |
| sanity-flr@ldiskfs+DNE:test_200a | seen in 4 other reviews |
| sanity-flr@ldiskfs+DNE:test_200b | seen in 2 other reviews |
| sanity-flr@ldiskfs+DNE:test_204a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_204c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_204e | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@ldiskfs+DNE:test_204f | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@zfs:test_36a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@zfs:test_38 | seen in 1 other review |
| sanity-flr@zfs:test_41 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@zfs:test_61a | seen in 45 other reviews |
| sanity-flr@zfs:test_200a | seen in 1 other review |
| sanity-flr@zfs:test_200b | seen in 3 other reviews |
| sanity-flr@zfs:test_204a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@zfs:test_204c | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@zfs:test_204e | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-flr@zfs:test_204f | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanityn@ldiskfs+DNE:test_71a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-1 failed 7× | RHEL 9.5 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-1 failed 8× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-part-2 failed 4× | RHEL 9.5 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-2 failed 7× | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-part-4 failed 7× | RHEL 9.5 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-part-4 failed 8× | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-dne-selinux-ssk-part-1 failed 7× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-1 failed 8× | RHEL 8.10 / x86_64 | ran 3 tests. 1 tests failed: sanity. | session |
| review-dne-zfs-part-2 failed 7× | RHEL 8.10 / x86_64 | ran 11 tests. 1 tests failed: sanity-lfsck. | session |
| review-dne-zfs-part-4 failed 7× | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-flr. | session |
| review-ldiskfs failed 8× | RHEL 8.10 / x86_64, RHEL 9.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs failed 8× | RHEL 8.10 / x86_64, SLES 15.5 / x86_64 | ran 5 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 22.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
| review-zfs failed 7× | RHEL 8.10 / x86_64 | ran 8 tests. 1 tests failed: sanity-flr. | session |
It would be better to name this function more clearly, like `lov_pattern_supported_server()` or similar. Possibly this needs to take an export or `struct obd_connect_data` argument so that the supported layout types can be determined by the client's feature support?
The patch https://review.whamcloud.com/64441 ("LU-12187 lov: MDS layout pattern negotiation") is implementing proper layout pattern negotiation between the MDS and client.
This check on the client is useful to add for future incompatible layouts, but does not help *existing* clients that don't understand `LCME_FL_PARITY` or `LOV_PATTERN_PARITY` layouts.
LU-19520 ec: don't read parity comp on old clients * Add layout_pattern_supported() so that it allows clients to set new type of layout and server also use it to check the validity of the layout passed by clients. * Clients then use lov_pattern_supported() to check what type of layout pattern it understand to exercise IO upon it. * CLIO also checks whether it understand the component by lov_supported_comp_flags(), as parity component would also set LCME_FL_PARITY in its component flags. Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Change-Id: I173fc93b5ccb1d99c154d77a96a146c96bcab81a
| unique failing test | history |
|---|---|
| conf-sanity-slow@ldiskfs+DNE:test_69 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 crashed | RHEL 8.10 / x86_64 | ran 13 tests. 1 tests failed: sanity-lfsck. %% THIS TEST SESSION CRASHED %% | session |
| review-dne-zfs-part-4 | RHEL 8.10 / x86_64 | ran 9 tests. 1 tests failed: sanity-quota. | session |
| review-ldiskfs-ubuntu | RHEL 8.10 / x86_64, Ubuntu 24.04 / x86_64 | ran 6 tests. 1 tests failed: sanity. | session |
I've done this once (twice?) before and didn't notice any actual improvement in performance. Thoughts? I'd want to see something to make the change I think
With: https://review.whamcloud.com/c/fs/lustre-release/+/60942 https://review.whamcloud.com/c/fs/lustre-release/+/61061 https://review.whamcloud.com/c/fs/lustre-release/+/61211 We got 15% improvement of a specific workload [2 threads per core single shared file 8k reads from system cache].
Shaun, I said this before, but I'll say it again: I have written this exact patch in the past because I saw this contention, but when I actually benchmarked it I saw no improvement. The contention simply relocated to other nearby harder-to-improve structures with no overall benefit. So I'm very open to this change, but we will have to provide concrete numbers showing improvement.
Yes, this is incremental and there are other spinlock_t cases on this path that go hot (There are a couple other patches that address those separately). This does cumulatively have an effect when doing small reads from cache (~5% improvement) although the larger fix is: https://review.whamcloud.com/c/fs/lustre-release/+/61230 which is responsible for a 20% performance regression doing small (8k) reads from cache.
LU-19318 clob: convert coh_attr_guard to rwlock_t Move the spin_lock out of contention nn read heavy shared file workloads with lots of small reads. HPE-bug-id: LUS-13035 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I005b5cbb1f463b7898f84b6fe1cae92cfbc940e8
| unique failing test | history |
|---|---|
| sanity-pcc@ldiskfs+DNE:test_15 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@ldiskfs+DNE:test_16 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@ldiskfs+DNE:test_17 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@ldiskfs+DNE:test_40 | seen in 12 other reviews |
| sanity-pcc@ldiskfs+DNE:test_101a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@zfs:test_15 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@zfs:test_16 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@zfs:test_17 | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| sanity-pcc@zfs:test_101a | NEW unique failure for this branch in the last 30 days, and was seen 0 times across 0 other branches 0 reviews |
| failed enforced test | platform | detail | |
|---|---|---|---|
| custom-1001 | RHEL 8.10 / x86_64 | ran 5 tests. 1 tests failed: sanity-pcc. | session |
I do not understand the purpose of this patch... Does it solve any bug in PCC? IMHO, we do not need to initialize PCC inode for IOCTL call, this patch is useless...
Currently the ldlm shrinker math reserves a minimum of 99 'unused' locks, dropping this implicit reservation from: ldlm_cli_pool_shrink() ... `return (unused / 100) * sysctl_vfs_cache_pressure;` with the change to use: `return vfs_pressure_ratio(unused);` which is effectively avoiding the integer truncation (unused / 100): `(sysctl_vfs_cache_pressure * unused) / 100;` breaks sanity-pcc/20 due to `ll_i2pcci(inode)` being NULL. Alternatively we can deny the IOCTL on a directory?
Yingjin, the problem appears to be that "lfs pcc state" does not return useful information if the inode is not in cache on the client. It should read the inode from PCC storage to determine the state. Shaun, Yingjin, does the dcache store the FID for a dentry name that this could be used to look up the file in the PCC cache without having to do an extra RPC to the MDS?
This changes breaks sanity-pcc/20 and sanity-flr/33c and hangs on sanity-flr/34a Retaining 100 feels like a magic number
Why is this in this change?
This change is to fix: sysctl -w vm.drop_caches=3 Currently Lustre keeps 100 ... this logic is introduced in https://review.whamcloud.com/c/fs/lustre-release/+/59970 it is written to maintain compatibility with: return (unused / 100) * sysctl_vfs_cache_pressure; Which looks like a truncation. vfs_pressure_ratio() is equivalent to: return (unused * sysctl_vfs_cache_pressure) / 100; which minimizes truncation. In this PCC case the non-truncate logic does *not* evict the unused dentry being tested in sanity-pcc/20 where the test is not 'valid' because the test is verifying that "lfs pcc state <dir>" works when the dentry is not in cache. When the dentry is evicted the test fails.
I still do not understand Why we need PIT_IOCTL here? and not check "@cached" after pcc_io_init? For the problem: "Yingjin, the problem appears to be that "lfs pcc state" does not return useful information if the inode is not in cache on the client. It should read the inode from PCC storage to determine the state" We have a patch to display the enough info for a file in PCC: https://review.whamcloud.com/54485
sanity-pcc/20 fails when the cache is purged: https://review.whamcloud.com/c/fs/lustre-release/+/63637
(style) _this_ could go into previous patch
Yes, this should be included in v6.15 compat changes.
LU-19266 pcc: ensure pcci is available post cache eviction After inode evicted from cache pcc_ioctl_state() needs to call pcc_io_init() when ll_i2pcci(inode) is not available. Test-Parameters: trivial testlist=sanity-pcc,recovery-small,sanity-flr Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: Iae1ad85264ed8ebacb6481ca16af2408a29b19aa
| unique failing test | history |
|---|---|
| sanity-flr@zfs:test_204c | seen in 3 other reviews |
| sanity-pfl@ldiskfs+DNE:test_22c | seen in 1 other review |
LU-10026 csdc: DoM pattern could be a combined value
DoM pattern is LOV_PATTERN_MDT for now, and in the future it could
be combined with LOV_PATTERN_COMPRESS to represent a compressed
DoM component.
Fix a minor glitch for lov_getstripe_old code path (in
ll_lov_getstripe_ea_info), which intends to return the last component
stripe info but the commit abf04e7ea3 omits to correctly set the
last component stripe info before using it.
Lustre-change: https://review.whamcloud.com/51978
Lustre-commit: bb0cc84fbed51e006bfac230dada426bfac4f500
Fixes: abf04e7ea3 ("LU-14337 lov: return valid stripe_count/size for PFL files")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Id0779c30c004b6979f88bf96b7b7b74a8b8c26e4
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
| unique failing test | history |
|---|---|
| sanity-flr@ldiskfs+DNE:test_40 | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity-flr@zfs:test_40 | NEW unique failure for this branch in the last 30 days, and was seen 1 times across 1 other branches 1 reviews |
| sanity-pcc@ldiskfs+DNE:test_1c | seen in 1 other review |
| sanity-pcc@ldiskfs+DNE:test_1d | seen in 1 other review |
| sanity-pcc@ldiskfs+DNE:test_1f | seen in 1 other review |
| sanity-pcc@ldiskfs+DNE:test_2a | seen in 1 other review |
| sanity-pcc@ldiskfs+DNE:test_2c | seen in 1 other review |
| sanity-pcc@ldiskfs+DNE:test_4 | seen in 1 other review |
| sanity-pcc@ldiskfs+DNE:test_13c | seen in 1 other review |
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-part-2 | CentOS 7.0/x86_64 | ran 11 tests. 1 tests failed: sanity-pcc. | session |
| review-dne-part-4 | CentOS 7.0/x86_64 | ran 10 tests. 1 tests failed: sanity-flr. | session |
| review-dne-zfs-part-4 crashed | CentOS 7.0/x86_64 | ran 12 tests. 2 tests failed: sanity-flr, replay-single. %% THIS TEST SESSION CRASHED %% | session |
| review-zfs | CentOS 7.0/x86_64 | ran 7 tests. 1 tests failed: sanity-flr. | session |
LU-13058 lod: Intermediate component removal The classic PFL layout example is a three tier layout, generally DOM->SSD->HDD. In this case, it is possible for the SSD tier to be low on/out of space. Classic PFL has no provision for handling this - The SSD component is either instantiated (even if the OSTs are almost out of space) or an error is returned (if the OSTs are truly out of space). This is not desirable behavior, and self extending layouts improve on this by making it possible to change the layout dynamically to avoid this. They do this in two ways: 1. If the SSD tier is low on space, do not instantiate that component, instead, extend the HDD component "up" to cover that region 2. Dynamically assign the layout in chunks, only gradually giving more layout on the SSD tier. This handles the case where the SSD tier becomes low on space while in use. "2" requires a specialized self-extending layout, but there is nothing about "1" which actually requires this type of layout. It is possible to 'skip' a full tier in a normal PFL layout. This patch implements that for normal PFL layouts. Before instantiating an intermediate layout component, the stripe allocator is asked to assign striping, then the chosen OSTs are checked to see if they're low on space. If an OST is low on space, we simply remove this component from the layout and extend the next component downward instead. The assumption is that the later tiers are larger in size, and so most likely have space. This is identical to the behavior for SEL files, using the same basic check. Signed-off-by: Patrick Farrell <farr0186@gmail.com> Change-Id: I380db620903e795523c2d4a5554c8c56505db593
| failed enforced test | platform | detail | |
|---|---|---|---|
| review-dne-zfs-part-1 | CentOS 7.0/x86_64 | ran 6 tests. 3 tests failed: recovery-small, sanityn, sanity. | session |
| review-dne-zfs-part-2 | CentOS 7.0/x86_64 | ran 11 tests. 1 tests failed: sanity-hsm. | session |
LU-0000 osc: introduce delayed extents This might improve the possibility that aio request could be merged to reduce overhead of ptlrpc. Change-Id: I23c97624026213ea6e94f3767ccab1c0c29c9f91 Signed-off-by: Wang Shilong <wshilong@ddn.com>
This is just a copy of the lockahead tests right now because I lost the prefetch test while reorganizing... Which is an example of why I wanted to get this pushed to Gerrit. :x
LU-16485 llite: Add manual prefetch to ladvise It is extremely useful for some applications to be able to manually request that specific page ranges be prefetched. This is particularly interesting for databases. Database queries can often generate fairly random access patterns, which can make readahead difficult or wasteful (if accesses are sparse). But critically, the database knows which pages of the file it will need to answer the query. This interface makes it possible to request that an arbitrary set of pages (non-contiguous, etc) be prefetched in to client memory. This is incomplete prototype code with no expectation of further work currently, but I wanted to get this where it would not be lost. Test-parameters: fortestonly Signed-off-by: Patrick Farrell <farr0186@gmail.com> Change-Id: I6a7332bb8b4d649302a69e5283bd944aacd2f1e4
LU-20002 llite: add cached_read_bytes stat Add a new cached_read_bytes counter to llite stats that tracks bytes served from the page cache via the cached read path (ll_read_from_cache). This allows users to determine what fraction of their read I/O hits the page cache versus going through the full cl_io/DLM lock path. Rename ll_do_fast_read() to ll_read_from_cache() to clarify that this function handles all cached reads, not just small or "fast" ones. The stat uses LPROCFS_TYPE_BYTES_FULL, providing count, min, max, sum, and sumsq - matching the format of the existing read_bytes and hybrid_read_bytes counters. Comparing cached_read_bytes to read_bytes gives a cache hit ratio useful for performance analysis and tuning. A sanity test (127g) verifies the stat with four sub-tests: cold cache reads show zero cached bytes, warm cache reads show matching cached and total bytes, disabling fast_read zeroes the counter, and a multi-page cold read shows readahead feeding the cached read path (at most 2 out of 16 pages miss). Generated with Claude Code + Tools Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I6167f4afef5856a34831f16332c86683bcc055f9 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/64615 Reviewed-by: Timothy Day <timday@thelustrecollective.com> Tested-by: Maloo <maloo@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-13643 connect: reserve FLR_IMMED_MIRROR flag Reserve a new connect flag OBD_CONNECT2_FLR_IMMED_MIRROR for the immediate mirror feature in File Level Replication (FLR). This flag will be used to indicate that the client/server supports immediate mirror functionality, where mirror files are synchronized immediately rather than using the default lazy synchronization behavior. The flag value 0x20000000000ULL is the next available value in the OBD_CONNECT2 series after OBD_CONNECT2_FLR_EC. This patch only reserves the flag and updates the necessary infrastructure: - Add OBD_CONNECT2_FLR_IMMED_MIRROR flag definition - Update obd_connect_names[] array with "flr_immediate_mirror" name - Add wiretest assertions for both utils and ptlrpc wiretest files The actual implementation of the immediate mirror feature will be submitted in subsequent patches. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I4fb92940773db6a395ea3d5dc91cce2444185fac Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/59587 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-by: Marc Vef <mvef@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>
LU-12668 target: extend BRW_READ_BULK fail_loc bitmask
Make OBD_FAIL_OST_BRW_READ_BULK (0x20f) select which OST(s) to fail
through the shared cfs_fail_index() helper, so a single fail_loc can
fail multiple OSTs at once:
fail_val = 0 fail all OSTs (unchanged)
fail_val = 1..0xffff fail single OST at index
fail_val-1 (unchanged)
fail_val > 0xffff bitmask mode: bits 0-15 select
which OST indices to fail
This is needed for EC degraded read testing. EC recovery requires
multiple simultaneous OST read failures to exercise the reconstruction
path. The previous single-OST targeting could only fail one stripe at
a time.
The encoding is provided by cfs_fail_index() and is backwards
compatible -- no existing fail_loc user passes fail_val > 0xffff for
single-OST targeting since OST indices are 16-bit.
EC recovery tests use this with CFS_FAIL_SOME and osc.*.resend_count=1
to inject persistent read errors that propagate to the LOV layer
without causing infinite BRW retries.
Assisted-by: ClaudeCode:Opus llm_code_and_review_tools
Test-Parameters: trivial
Test-Parameters: testlist=sanity-ec
Test-Parameters: testlist=sanity-ec fstype=zfs
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I432a262520b28ca1c344a1ec9f01ed435ffe2805
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/65010
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Maximilian Dilger <mdilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Marc Vef <mvef@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@thelustrecollective.com>