diff --git a/docs/README-lanes.md b/docs/README-lanes.md index a6caed0a..7608e532 100644 --- a/docs/README-lanes.md +++ b/docs/README-lanes.md @@ -1225,7 +1225,7 @@ else. | 2 | refusal — bad arguments, the object is held, an unknown alias, a checkout that cannot be rebased, or `commit` refusing to publish local commits that are not register or handoff writes, from a checkout whose HEAD is off the branch, or that it cannot read (#194; it pushes none of them) | | 3 | the edit is never a permanent loss, but this attempt cannot confirm whether it also reached origin — a rebase conflict (not pushed by this attempt; a later write from this checkout may already carry it to origin, so look by content before you retry), a network timeout on a push or a pull (a hung one often already landed), or six attempts exhausted because a peer's uncommitted file blocks every rebase | | 4 | the mutex could not be taken within 60s | -| 5 | an edit moved more than one line and was refused | +| 5 | an edit moved more than one line and was refused — or a register rewrite whose copy, write or result could not be proved (#120 item 1). The copy every proof is made against is compared with the register, and what lands with what was built; where a write cut the register short, its earlier bytes are put back and proved first, and the refusal says which of the two the register holds — never "unchanged" about a register the write truncated | | 6 | `git add` / `commit` / `push` failed | | 7 | another act got there first and this one wrote nothing: `claim`'s `CLAIM-LOST` (another lane's claim landed first), and `set-lane-state` / `set-lane-tree`'s lifecycle fence (openRepoTools#91, below) | | 9 | `CLAIM-LOST` — issue #30's own dead-lane verdict could not be reconfirmed before a `--force` takeover's push landed: the source lane resumed, a live session now backs it up, or that could not be read at all (`claim` only). Never 7 — that code is a RIVAL's claim, and this is the same lane the takeover was granted over | @@ -3158,12 +3158,28 @@ had nowhere to keep its warnings about published logs it could not read, and it says so. Counting zero there would be "nothing to do" about rows nobody could class. +A log whose **last** lane-kind line is `ENDED` or `RETIRED` is refused too, +since a `RETIRED` after it would be one more line in an append-only log. The +refusal says the lane is **already CLOSED** and already hidden (19(b)). For +`RETIRED` it names `lanes-edit.sh archive-rows ` as the act that takes the +row out of the register. For `ENDED` it says nothing more is owed: the row stays +as the record, because `archive-rows` moves only rows whose cell says `RETIRED`. +A log whose last lane-kind line is `STARTED`, `PAUSED` or `RESUMED` is sent to +`lane-end ` instead (#120 item 3). The lane is called CLOSED only when +origin's copy of its log ends on that line. A closing line only this checkout's +log carries, left by an earlier act that stopped part-way or never pushed, is +refused as exactly that. + It refuses six more things for the reason every writer here refuses them — a half-written act is worse than no act: a checkout **behind** `origin` or dirty in anything but the register; a row whose ` | ` count makes **which text is the state cell** unknowable; a lane whose object log would need Amendment 15's **case-only rename**, which is a write of its own and not one this commit -smuggles; an object log that exists here and is **NOT TRACKED**, which is +smuggles — or whose log `origin` still publishes under another case while this +checkout already has the row's spelling, **a case-only rename whose commit +never landed**, which the scan refuses in `ensure_log`'s own words rather than +leaving it to `ensure_log` half-way through the write (#120 item 2); an object +log that exists here and is **NOT TRACKED**, which is somebody's uncommitted work and would land inside this commit under this act's message; a lane **named twice**, whose log would take the same line twice in a file nothing can correct; and a `--reason` carrying a `|`, a second ` · ` or a @@ -3188,11 +3204,17 @@ publishes.** Every refusal above is made in the scan, before a byte is written. That includes the commit of a peer's uncommitted register edit, which the act makes as its own `LANES(pre-existing@…)` commit: it is made only after the scan, so a refusal leaves HEAD where it was. A failure in the writes themselves — a log whose append could not be proved, a -row whose rewrite touched more than one line — leaves this checkout DIRTY and +row whose rewrite touched more than one line or could not be proved (the +register is put back where the write cut it short, #120 item 1) — leaves this checkout DIRTY and the register unpublished: that is what `git status` then shows, `git checkout -- lanes` undoes it whole, and the next run is REFUSED by the dirty-checkout guard rather than made twice. `migrate-state-cells` has the same shape for the same -reason. +reason. Such a refusal says this act has committed nothing (a peer's edit it +found and captured first is a commit of its own), and lists what the act had +already written in this checkout. Where the register may be truncated, it also +names where the exact copy of the register from before the edit is kept: in the +checkout's own git directory, as `lanes-edit-register-before-edit..` +with mode 0600, where a restart that empties the temporary directory leaves it. ### The archive — a second act on a second word @@ -3240,7 +3262,46 @@ fail closed rather than answer, and for the same reason. The first three refuse `add-row` refuses a lane name the archive holds, so `lane-start ` cannot reissue one by hand either. It also refuses (exit **1**) where the published register cannot be read, rather than read that as "no such row": -otherwise a row this checkout has not pulled would be added a second time. **Rule 9 holds either way** — a row is +otherwise a row this checkout has not pulled would be added a second time. +And it refuses (exit **1**) an archive that carries a `|` line it cannot take +apart, naming each such line by its number and the copy it is in, published +or this checkout's (#122). A line counts only if it is +the table's header (its first cell is `lane`), its `|---|` rule, or a row whose +first cell holds its lane in backticks; any other line may be the very lane +being added. The header is found by that first cell, because a ` | ` count +cannot tell it from a row (both carry six), and a hand edit can part it from its +rule. An archive scan that **failed** is a refusal too, never "no match". So is +an archive with CRLF line endings, or a table written without the spaces around +its `|`: `archive-rows` writes neither, and nothing in this tool reads CRLF. +Where add-row has already committed its row when a peer's archive arrives in its +`pull --rebase`, any of these refusals removes the row again in a compensating +local commit, as a peer's retirement does, so no later write publishes it. It +removes only the copy this checkout's register adds over origin's: where a peer +published the same row first, the rebase drops add-row's commit, and the copy +left is the peer's, so nothing is taken back. It finds that copy's line in a +diff it asks for with no context: `GIT_DIFF_OPTS` unset and +`diff.interHunkContext=0`, either of which would otherwise add context to +`-U0`. Any context that still arrives is counted where it stands. It reads +that diff as text (`--text`), so an attribute, a NUL byte or a +`core.bigFileThreshold` that classes the register as binary cannot hide the +row, and its proof that one line was removed counts lines whatever +`core.bigFileThreshold` says. Where the take-back itself cannot remove the row +— a copy or a write cut short, or a proof that refuses — the refusal says that +add-row has committed its row, locally and unpushed, and that it must be removed +before any other write in that checkout publishes it. Where the row is removed +but the removal cannot be committed, a note says whether the removal is staged, +that add-row's commit still holds the row, and the command that commits the +removal. + +**One such line refuses every lane, not only its own.** `lanes-edit.sh +retired-identity `, the check `lane-start` makes before it launches, +exits 1 on it as well, so every `lane-start` refuses while the line is there. +`lane-start` hides the reason: it reads that check with its stderr discarded and +says only that the archive could not be checked for the lane. Run +`lanes-edit.sh retired-identity ` to see the line, its number and its +copy. + +**Rule 9 holds either way** — a row is one `git show` away — so the archive is for a register a person wants shorter, and never a requirement. diff --git a/lanes-edit.sh b/lanes-edit.sh index 72b74e41..181fed36 100755 --- a/lanes-edit.sh +++ b/lanes-edit.sh @@ -493,7 +493,11 @@ # every rebase. # 4 the mutex could not be taken within 60s # 5 an edit moved more than one line and was refused — or, Amendment 15, a -# lane's object log could not be renamed to the row's own spelling +# lane's object log could not be renamed to the row's own spelling — or, +# #120 item 1, a register rewrite (`replace_line`, `delete_lines`) whose +# copy, write or result could not be proved; where the write had cut the +# register short its earlier bytes are put back first, and the refusal +# says which of the two the register holds # 6 git add / commit / push failed # 7 ANOTHER ACT GOT THERE FIRST, and this one wrote nothing. `claim`: # CLAIM-LOST, another lane's claim landed on main first. @@ -1589,6 +1593,103 @@ lane_from_text() { # --------------------------------------------------------------- file writes +# THE REGISTER IS REWRITTEN IN TWO PLACES, `replace_line` AND `delete_lines`, +# AND BOTH PROVE WHAT THEY COPIED AND WHAT LANDED (opensoft/openRepoTools#120 +# item 1, Copilot round 5 on #93). The appends below them prove their result +# with `cmp`; the rewrites proved only the file they built, and two steps of +# theirs were never checked at all: +# +# * THE COPY. Every proof of a rewrite (the line count, `numstat`, the no-op) +# is made against `$pre`, a `cat` of the register into a temporary file, and +# that `cat` had no status. A copy that came out short (a full temporary +# filesystem, a read that failed part-way) made a SHORT register look like a +# one-line edit of itself: every proof passed and the short copy was written +# back as the register and committed. Measured at 5e88a2c with the copy cut +# at half its lines: a 156,611-byte register rewritten as 78,306, exit 0. +# * THE WRITE. `cat -- > LANES.md` had its status read and not its +# result, and `>` TRUNCATES before `cat` writes a byte. A write that failed +# part-way left a half register behind a refusal that called it UNCHANGED, +# and one that came out short and said nothing was a half register +# committed. Even an uncommitted half register is one write from being +# published: the next writer in this checkout commits a dirty register as a +# peer's pre-existing edit (`handle_preexisting`). +# +# So the copy is compared with the register and the result with what was built. +# Where the result is not what was built, the copy's bytes are put back and +# proved before the refusal (5) says which of the two the register now holds. +# Both still write through `cat` and a redirection, which FOLLOW a symlinked +# register; a rename beside it would replace the link with a file. +# +# A REFUSAL SAYS WHAT IS ALREADY ON DISK (the review of #263, S2). The sweeps +# write their logs, their archive and their earlier rows before a row's +# rewrite, so a refusal that said "Nothing was written" was false there: `git +# status` showed those files, and a re-run read them back. What holds is that +# THIS ACT has committed nothing — not "nothing has been committed": a peer's +# uncommitted register edit found first is captured as a commit of its own +# (`handle_preexisting`) before the act's edit, and the one caller that HAS +# committed when it rewrites (add-row's after-rebase take-back) says so through +# `REG_COMMITTED_HERE` (the delta review of #263). Each sweep names its local +# writes with `reg_wrote`, which the refusals list. AND WHERE THE REGISTER MAY +# BE TRUNCATED the copy is KEPT (S2's NIT 6): it is the only exact copy of the +# register as it was, uncommitted edits included, and the exit trap removes +# `$TMPD`. So it is copied into the checkout's own git directory — mode 0600, +# under a name nothing else holds, proved byte for byte — where a restart that +# empties the temporary directory leaves it; only where that cannot be done is +# it renamed beside `$TMPD` instead (a rename needs no free space). +REG_WRITTEN_HERE="" +REG_COMMITTED_HERE="" +DL_SAYS_UNCHANGED="" +DL_SAYS_AFTER_WRITE="" +reg_wrote() { # + REG_WRITTEN_HERE="$REG_WRITTEN_HERE, $1" +} +reg_committed() { # — what this act has committed, as a refusal says it + printf '%s' "${REG_COMMITTED_HERE:-this act has committed nothing}" +} +reg_written_here() { # — the sentence a refusal ends with, or nothing + [ -n "$REG_WRITTEN_HERE" ] || return 0 + printf ' This act has already written, in this checkout and NOT committed: %s. `git -C %s status` shows them; commit or undo them before re-running.' \ + "${REG_WRITTEN_HERE#, }" "$LANES_REPO" +} +register_copy() { # + cat -- "$LANES_FILE" > "$1" && cmp -s -- "$LANES_FILE" "$1" || + die "the register at $LANES_FILE could not be copied whole to $1, and every proof of this edit is made against that copy: a short one would make a short register look like a one-line edit of itself. The register is as it was and $(reg_committed): $2.$(reg_written_here)" 5 +} +register_rewrite() { # + cat -- "$2" > "$LANES_FILE" && cmp -s -- "$LANES_FILE" "$2" && return 0 + if cmp -s -- "$LANES_FILE" "$1"; then + die "could not write the rewritten register $LANES_FILE: the write did not reach it, and it is byte for byte as it was before this edit. $3$(reg_written_here)" 5 + fi + if cat -- "$1" > "$LANES_FILE" && cmp -s -- "$LANES_FILE" "$1"; then + die "the rewritten register $LANES_FILE did not land as it was written: the write cut it short, and its earlier bytes have been put back and proved. $3$(reg_written_here)" 5 + fi + rgw_keep=""; rgw_where="" + rgw_git="$(git -C "$LANES_REPO" rev-parse --absolute-git-dir 2>/dev/null)" || rgw_git="" + if [ -n "$rgw_git" ] && [ -d "$rgw_git" ]; then + rgw_to="$rgw_git/lanes-edit-register-before-edit.$(date -u +%Y%m%dT%H%M%SZ).$$" + if [ ! -e "$rgw_to" ] && [ ! -L "$rgw_to" ]; then + if ( umask 077 && set -C && cat -- "$1" > "$rgw_to" ) 2>/dev/null && cmp -s -- "$1" "$rgw_to"; then + rgw_keep="$rgw_to" + else + rm -f -- "$rgw_to" 2>/dev/null || : + fi + fi + fi + if [ -z "$rgw_keep" ]; then + rgw_to="${1%/*}.register-before-edit" + if [ ! -e "$rgw_to" ] && [ ! -L "$rgw_to" ] && chmod 600 -- "$1" 2>/dev/null && mv -- "$1" "$rgw_to" 2>/dev/null; then + rgw_keep="$rgw_to" + rgw_where=" That is the temporary directory, because the checkout's git directory could not take the copy: a restart that empties it loses them, so restore them first." + fi + fi + if [ -n "$rgw_keep" ]; then + rgw_copy="Its exact bytes from before this edit, with any uncommitted edit they held, are kept in $rgw_keep (mode 0600; \`cat -- $rgw_keep > $LANES_FILE\` puts them back, and \`git -C $LANES_REPO diff -- $LANES_PATH\` shows what it lost meanwhile); \`git checkout\` would bring back only its last commit and drop any uncommitted edit.$rgw_where" + else + rgw_copy="Its bytes from before this edit could not be kept either: \`git -C $LANES_REPO diff -- $LANES_PATH\` shows what it lost, and \`git -C $LANES_REPO checkout -- $LANES_PATH\` brings back its last commit, dropping any uncommitted edit it held." + fi + die "could not write the rewritten register $LANES_FILE, and the bytes it had before this edit could not be put back either: IT MAY NOW BE TRUNCATED. Restore it before any other write in $LANES_REPO, because the next one would commit it as a pre-existing edit. $rgw_copy $3$(reg_written_here)" 5 +} + # Replace line N with $2, leaving every other byte untouched. Verified with # `git diff --no-index --numstat` (must be exactly 1 added / 1 deleted). replace_line() { @@ -1599,7 +1700,7 @@ replace_line() { REPLACE_LINE_NOOP=0 TMPD="$(mktemp -d)" pre="$TMPD/pre"; out="$TMPD/out" - cat -- "$LANES_FILE" > "$pre" + register_copy "$pre" "its line $n is as it was and this edit is not made" before="$(wc -l < "$pre" | tr -d ' ')" { [ "$n" -gt 1 ] && head -n "$((n - 1))" -- "$pre" @@ -1646,9 +1747,9 @@ replace_line() { # writable: the two proofs above pass (they are made against a temporary # file), the note says the line was rewritten, and the caller commits whatever # else it changed. For a rename that is the alias landing while the row it - # points at never moves. - cat -- "$out" > "$LANES_FILE" || # redirect FOLLOWS the symlink - die "the register at $LANES_FILE could not be written: its line $n is UNCHANGED and this edit is not made. Nothing else this write touched has been committed." 5 + # points at never moves. AND ITS RESULT IS PROVED, not only its status, with + # the register put back where the write cut it short (#120 item 1, above). + register_rewrite "$pre" "$out" "Its line $n is not rewritten, and $(reg_committed)." note "line $n rewritten in $LANES_FILE" } @@ -1668,7 +1769,22 @@ delete_lines() { # [ "$dl_n" -gt 0 ] || return 0 TMPD="$(mktemp -d)" dl_pre="$TMPD/pre"; dl_out="$TMPD/out" - cat -- "$LANES_FILE" > "$dl_pre" + # WHAT A REFUSAL HERE SAYS ABOUT THE ROWS IS THE CALLER'S (the delta review of + # #263, NIT). archive-rows' sentences are the default: its rows are already in + # the archive. add-row's take-back passes its own, because there the row is + # add-row's committed one and no archive is involved. + dl_unchanged="${DL_SAYS_UNCHANGED:-no row was removed, the rows are in $LANES_ARCH_PATH, and the register is whole}" + dl_after_write="${DL_SAYS_AFTER_WRITE:-The archive may have been appended locally, but $(reg_committed); resolve that partial move before retrying.}" + register_copy "$dl_pre" "$dl_unchanged" + # AND EVERY LATER REFUSAL HERE SAYS IT AS WELL (delta review 4 of #263, N1 + # and S4's N2). The pass, the line count and `numstat` below refused with no + # word of what the act had committed: add-row's take-back, which HAS committed + # its row, died saying only "the removal added or rewrote lines" or "that row + # is still in it and in that commit", with no row and no commit named, and the + # next write published the row. Each now ends as `register_copy`'s refusal + # does: the register as it was, what this act has committed, the caller's + # sentence, and what the act has already written in this checkout. + dl_as_was="The register is as it was and $(reg_committed): $dl_unchanged.$(reg_written_here)" dl_before="$(wc -l < "$dl_pre" | tr -d ' ')" # `awk -v` CARRIES ONE LINE, AND THIS LIST IS MANY — the finding `who_landing` # took in round 5 of A9 Addendum 4 (R-A9-11), re-made here and answered by @@ -1694,19 +1810,28 @@ delete_lines() { # BEGIN { k = split(ENVIRON["LANES_DROP_LINES"], a, "\n") for (i = 1; i <= k; i++) if (a[i] != "") drop[a[i] + 0] = 1 } !(NR in drop)' "$dl_pre" > "$dl_out"; then - die "the pass that removes $dl_n row(s) from $LANES_FILE failed, so what the register would become is not known — and that is not an empty register (Amendment 7(d)). Nothing was removed; the rows are in $LANES_ARCH_PATH and the register is whole." 5 + die "the pass that removes $dl_n row(s) from $LANES_FILE failed, so what the register would become is not known — and that is not an empty register (Amendment 7(d)). $dl_as_was" 5 fi dl_after="$(wc -l < "$dl_out" | tr -d ' ')" [ "$dl_after" -eq "$((dl_before - dl_n))" ] || - die "removing $dl_n row(s) changed the line count by $((dl_before - dl_after)); refusing" 5 - dl_stat="$(git --no-pager diff --no-index --numstat -- "$dl_pre" "$dl_out" 2>/dev/null | head -n1 | cut -f1,2)" + die "removing $dl_n row(s) changed the line count by $((dl_before - dl_after)); refusing. $dl_as_was" 5 + # AND IT COUNTS LINES WHATEVER THE REGISTER'S SIZE (delta review 4 of #263, + # S4). Under a `core.bigFileThreshold` below the register's size git classes + # both copies as binary, and `numstat` prints a `-` for each count, which + # `--text` does not change: add-row's take-back refused there, and its row + # stayed committed for the next write to publish. A `-c` on the command line + # outranks every config file and `GIT_CONFIG_*` in the environment (measured + # on git 2.43). An attribute or a NUL byte that classes the copies as binary + # still makes this proof refuse, in the words above. + dl_stat="$(git --no-pager -c core.bigFileThreshold=2g diff --no-index --numstat -- "$dl_pre" "$dl_out" 2>/dev/null | head -n1 | cut -f1,2)" [ "$dl_stat" = "$(printf '0\t%s' "$dl_n")" ] || - die "the removal added or rewrote lines (numstat: ${dl_stat:-none}); refusing" 5 + die "the removal added or rewrote lines (numstat: ${dl_stat:-none}); refusing. $dl_as_was" 5 # The redirection follows the register symlink, but its status is still a # write fence. An unwritable target must not let archive-rows commit the - # appended archive while the source rows remain in the register. - cat -- "$dl_out" > "$LANES_FILE" || - die "could not write the rewritten register $LANES_FILE. The archive may have been appended locally, but no commit was made; resolve that partial move before retrying." 5 + # appended archive while the source rows remain in the register. And its + # RESULT is the fence as well, with the register put back where the write cut + # it short (#120 item 1, above `replace_line`). + register_rewrite "$dl_pre" "$dl_out" "$dl_after_write" note "$dl_n row(s) removed from $LANES_FILE" } @@ -3558,49 +3683,208 @@ archive_local_text() { # A retired name OR its numeric position is unavailable, even when a recut # suffix changes the spelling. Inspect both published and unpushed archives. -retired_identity_hits() { # ; stdout: matching retired spellings +# +# A LINE THE SCAN CANNOT TAKE APART IS A REFUSAL, NOT A NON-ROW (#122, Copilot +# round 6 on #93). The scan skipped every `|` line with no backticked token, and +# took the first backtick ANYWHERE in a line as its lane, so a retired row whose +# lane cell a hand edit or a bad merge had mangled — `| repoX-4 | harness `` +# | …` — dropped out of the answer and add-row reissued its position: 19(b)'s +# guarantee failing open. Every `|` line of the archive is now one of three: +# +# * THE RULE: `|`, `-`, `:` and blanks only, with at least one dash; +# * THE HEADER: its first cell is the word `lane`, as in the header +# `archive_rows` writes. It is NOT found as "the line above the rule": the +# live register carried two rows between its header and its rule on +# 2026-10-19, so a hand edit can part the two, and refusing a header for it +# would make add-row refuse on the path that creates every lane. Nor is it +# found by counting ` | `: the header carries six, exactly as a row does; +# * A ROW: its first backticked token lies wholly inside its FIRST cell. That +# is the token every reader here takes as the lane, `*(unnamed)* ``` +# included (three live rows are spelled that way). +# +# Any other `|` line is printed with its line number, and this returns 3: a row +# nobody can read may be the very lane being added. AND THE SCAN'S OWN STATUS +# IS READ. The scan's status used to be that of the `sort -u` behind it, so an +# awk that FAILED read as "no archived row matched". +# +# EACH LINE NUMBER NAMES ITS COPY (the review of #263, NIT 4). Both copies are +# scanned, and where the published archive and this checkout's differ, one bad +# line had two numbers and no word on which file either was in. +RETIRED_ROWS_AWK=' + function key(n, h, p) { + p = n; sub(/^.*-/, "", p); sub(/[A-Za-z]$/, "", p) + h = n; sub(/-[^-]*$/, "", h) + if (h == n || p !~ /^[0-9]+$/) return "" + return tolower(h) "-" (p + 0) + } + BEGIN { wk = key(want) } + substr($0, 1, 1) == "|" { + if ($0 ~ /^[|][-:| \t]*$/ && index($0, "-")) next + c = substr($0, 2); s = index(c, " | "); if (s) c = substr(c, 1, s - 1) + h = c; sub(/^[ \t]+/, "", h); sub(/[ \t|]+$/, "", h) + if (tolower(h) == "lane") next + p1 = index(c, "`"); p2 = p1 ? index(substr(c, p1 + 1), "`") : 0 + if (!p1 || p2 < 2) { printf "M%s%d%s%s\n", sep, NR, sep, $0; next } + t = substr(c, p1 + 1, p2 - 1) + if (tolower(t) == tolower(want) || (wk != "" && key(t) == wk)) printf "H%s%s\n", sep, t + }' +retired_identity_hits() { # ; stdout: matching retired spellings · 1 not read · 3 a line not a row ri_lane="$1" ri_pub="$(archive_text 2>/dev/null)" || return 1 ri_local="" ri_local="$(archive_local_text 2>/dev/null)" || return 1 - printf '%s\n%s\n' "$ri_pub" "$ri_local" | awk -v want="$ri_lane" ' - function key(n, h, p) { - p = n; sub(/^.*-/, "", p); sub(/[A-Za-z]$/, "", p) - h = n; sub(/-[^-]*$/, "", h) - if (h == n || p !~ /^[0-9]+$/) return "" - return tolower(h) "-" (p + 0) - } - BEGIN { wk = key(want) } - substr($0,1,1) == "|" { - p1 = index($0, "`"); if (!p1) next - r = substr($0, p1 + 1); p2 = index(r, "`"); if (!p2) next - t = substr(r, 1, p2 - 1) - if (tolower(t) == tolower(want) || (wk != "" && key(t) == wk)) print t - }' | LC_ALL=C sort -u -} - + ri_scan="" + ri_scan="$(printf '%s\n' "$ri_pub" | awk -v want="$ri_lane" -v sep="$US" "$RETIRED_ROWS_AWK")" || return 1 + ri_scan2="" + ri_scan2="$(printf '%s\n' "$ri_local" | awk -v want="$ri_lane" -v sep="$US" "$RETIRED_ROWS_AWK")" || return 1 + # WHICH COPY `archive_text` ANSWERED WITH: the index's under `--index`, else + # origin's where the tree publishes one, else this checkout's (its own rungs, + # read the same way; a read that failed has already returned 1 above). + ri_from="this checkout" + if [ -n "$LANES_IDX_DIR" ]; then + [ -f "$LANES_IDX_DIR/archive" ] && ri_from="the index" + elif register_published "$LANES_ARCH_PATH" >/dev/null 2>&1; then + ri_from="origin/$LANES_BRANCH" + fi + ri_bad="" + ri_bad="$( { printf '%s\n' "$ri_scan" | awk -v sep="$US" -v w="$ri_from" 'substr($0, 1, 2) == "M" sep { print w sep substr($0, 3) }' + printf '%s\n' "$ri_scan2" | awk -v sep="$US" -v w="this checkout" 'substr($0, 1, 2) == "M" sep { print w sep substr($0, 3) }'; } | + awk -F"$US" '{ k = $2 FS $3 + if (!(k in w)) { o[++n] = k; w[k] = $1 } else if (w[k] != $1) w[k] = w[k] " and " $1 } + END { for (i = 1; i <= n; i++) { split(o[i], p, FS); print "line " p[1] " (" w[o[i]] "): " p[2] } }')" || return 1 + if [ -n "$ri_bad" ]; then + note "$LANES_ARCH_PATH carries a table line that is neither its header, its rule, nor a row whose first cell holds its lane in backticks (each named with the copy it is in):" + printf '%s\n' "$ri_bad" | sed -e 's/^/lanes-edit: /' >&2 + return 3 + fi + ri_hits="" + ri_hits="$(printf '%s\n%s\n' "$ri_scan" "$ri_scan2" | + awk -v sep="$US" 'substr($0, 1, 2) == "H" sep { print substr($0, 3) }')" || return 1 + [ -z "$ri_hits" ] || printf '%s\n' "$ri_hits" | LC_ALL=C sort -u +} + +# EVERY REFUSAL HERE TAKES ITS ROW BACK, not only the retired match (the review +# of #263, M1). This runs after `pull --rebase`, so the add-row is already a +# local commit on top of the peer's archive. A refusal that only returned left +# that commit at HEAD, and the next unrelated write from this checkout pushed +# it: measured, a peer's archive line this scan could not take apart during +# add-row's pull gave exit 1, then `set-row-state` published the row it had +# refused. A read that failed and a line that is not a row are refusals of the +# same row as a retired match is, so all three run the same compensation. add_row_retirement_rescan() { - ar_rebased_hits="$(retired_identity_hits "$ADD_ROW_LANE")" || { - note "the archive could not be read after rebase; refusing to push add-row for $ADD_ROW_LANE" + ar_rebased_rc=0 + ar_rebased_hits="$(retired_identity_hits "$ADD_ROW_LANE")" || ar_rebased_rc=$? + if [ "$ar_rebased_rc" != 0 ]; then + ar_why="the archive could not be read after rebase, or it carries a line that is not a row (above), so whether lane $ADD_ROW_LANE was retired meanwhile is not known" + ar_subject="cancel add-row whose retirement could not be checked after rebase" + ar_back=1 + elif [ -n "$ar_rebased_hits" ]; then + ar_why="lane $ADD_ROW_LANE was retired by a peer before this add-row could push (archive: $(printf '%s' "$ar_rebased_hits" | tr '\n' ' '))" + ar_subject="cancel add-row rejected by peer retirement" + ar_back=2 + else + return 0 + fi + # Leaving the rejected add-row at HEAD would let the next unrelated registry + # write push it. Remove ONLY this invocation's exact appended line and commit + # that compensation locally; any swept pre-existing edit stays intact. + # + # AND ONLY A ROW THIS CHECKOUT STILL ADDS IS ITS OWN TO TAKE BACK (the delta + # review of #263, D1). A peer that published the byte-identical row first + # makes `pull --rebase` DROP this add-row's commit as already upstream, and + # the one copy left in the register is then the peer's: the exact-line grep + # found it and deleted it, and the next write published that deletion + # (measured: the peer's row gone from origin). So the copy taken back is the + # one this checkout's register adds over origin/$LANES_BRANCH's, read from + # their diff by content — which is what a surviving add-row commit is, under + # whatever sha the rebase gave it, and whatever local commits sit beside it. + # None added: nothing of this add-row's is left, and nothing is taken back. + # One: that line, wherever an identical peer copy sits. More than one, or a + # diff that cannot be read: not this act's to guess, and said so. + # "origin already has the row" would be wrong the other way: an add-row + # commit that SURVIVED beside a peer's identical row at another place would + # be kept, and published by the next write as a second copy. + # + # AND THE LINE IS COUNTED WHATEVER CONTEXT THE DIFF CARRIES (the delta review + # of #263, S3). `-U0` does not hold on its own: a `GIT_DIFF_OPTS=-u3` in the + # environment overrides it outright, and a user's `diff.interHunkContext` + # fuses two nearby changes into one hunk with the lines between them as + # context. The parse counted only `+` lines, so a hunk carrying context put + # the row at the wrong line, the line check refused, nothing was taken back, + # and the next ordinary write published the row add-row had refused (M1 + # again; measured with each of the two). So both: this one diff runs with + # `GIT_DIFF_OPTS` unset and `diff.interHunkContext=0` (a `-c` on the command + # line outranks every config file and `GIT_CONFIG_*` in the environment), and + # the parse advances its new-file line on every line of a hunk but a removed + # one (`-`) or git's "\ No newline" note, so whatever context still arrives + # is counted where it stands. The `unset` is inside the command + # substitution's own subshell, and so touches nothing after it. + # + # AND IT IS READ AS TEXT (delta review 4 of #263, S4). Where git classes the + # register as binary, the diff is one line, "Binary files … differ", with no + # hunk to read: a `binary` or `-diff` attribute, published or local, a diff + # driver with `binary=true`, `core.attributesFile`, `GIT_ATTR_SOURCE`, + # `attr.tree`, a NUL byte in its first 8000 bytes, or a `core.bigFileThreshold` + # below its size (the live register is over 2 MB). add-row then said this + # checkout added no copy of the row, kept its commit, and the next write + # published the row (measured with each). `--text` gives the hunks whatever + # classes the register. + ar_diff="$(unset GIT_DIFF_OPTS; git --no-replace-objects --no-pager -C "$LANES_REPO" -c "$CP_EXACT" -c diff.interHunkContext=0 diff --no-color --no-ext-diff --no-textconv --text -U0 "refs/remotes/origin/$LANES_BRANCH" -- "$LANES_PATH" 2>/dev/null)" || { + note "$ar_why; and this checkout's register could not be compared with origin/$LANES_BRANCH's, so whether a copy of the rejected row here is this add-row's or a peer's is not known. Nothing was taken back: read \`git -C $LANES_REPO log -p origin/$LANES_BRANCH..HEAD -- $LANES_PATH\` and remove this add-row's row before any later push" return 1 } - if [ -n "$ar_rebased_hits" ]; then - # The rejected add-row is already a local commit on top of the peer's - # archive. Leaving it at HEAD would let the next unrelated registry write - # push it. Remove ONLY this invocation's exact appended line and commit - # that compensation locally; any swept pre-existing edit stays intact. - ar_lines="$(command grep -nFx -- "$ADD_ROW_TEXT" "$LANES_FILE" 2>/dev/null)" || { - note "the rejected row could not be found exactly in $LANES_FILE; the local add-row commit must be repaired before any later push" - return 1 - } - case "$ar_lines" in *$'\n'*) note "more than one exact copy of the rejected row exists; refusing automatic compensation"; return 1 ;; esac - ar_line="${ar_lines%%:*}" - delete_lines "$ar_line" - git -C "$LANES_REPO" -c "$CP_EXACT" add -- "$LANES_PATH" || return 1 - git -C "$LANES_REPO" -c "$CP_EXACT" commit -q -m "LANES($ADD_ROW_LANE@$WS): cancel add-row rejected by peer retirement" -- "$LANES_PATH" || return 1 - note "lane $ADD_ROW_LANE was retired by a peer before this add-row could push (archive: $(printf '%s' "$ar_rebased_hits" | tr '\n' ' ')); the rejected row was removed in a compensating LOCAL commit. Neither commit was pushed." - return 2 - fi + ar_lines="$(printf '%s\n' "$ar_diff" | AR_ROW="$ADD_ROW_TEXT" LC_ALL=C awk ' + /^@@ / { h = $0; sub(/^@@ -[0-9,]+ [+]/, "", h); sub(/[ ,].*$/, "", h); n = h + 0; inh = 1; next } + !inh { next } + { c = substr($0, 1, 1) } + c == "-" || c == "\\" { next } + c == "+" && substr($0, 2) == ENVIRON["AR_ROW"] { print n } + { n++ } + ')" || { + note "$ar_why; and the diff of this checkout's register against origin/$LANES_BRANCH's could not be read, so nothing was taken back: remove this add-row's row before any later push" + return 1 + } + case "$ar_lines" in + '') + if git --no-replace-objects --no-pager -C "$LANES_REPO" show "refs/remotes/origin/$LANES_BRANCH:$LANES_PATH" 2>/dev/null | command grep -qFx -- "$ADD_ROW_TEXT"; then + note "$ar_why. origin/$LANES_BRANCH already carries this exact row, published by a peer, and the rebase dropped this add-row's commit as already upstream: this checkout adds no copy of it, so there is nothing of this add-row's to take back and the peer's row is left as it is." + else + note "$ar_why. This checkout's register carries no copy of the row that origin/$LANES_BRANCH's does not, so there is nothing of this add-row's to take back." + fi + return "$ar_back" ;; + *$'\n'*) + note "$ar_why; and this checkout's register adds more than one exact copy of the rejected row over origin/$LANES_BRANCH's, so which is this add-row's is not known and none was taken back: the local add-row commit must be repaired before any later push" + return 1 ;; + esac + ar_line="$ar_lines" + [ "$(sed -n "${ar_line}p" "$LANES_FILE" 2>/dev/null)" = "$ADD_ROW_TEXT" ] || { + note "$ar_why; and line $ar_line of $LANES_FILE, where the diff puts the rejected row, does not hold it exactly, so nothing was taken back: the local add-row commit must be repaired before any later push" + return 1 + } + # A REFUSAL OF THE TAKE-BACK ITSELF SAYS ADD-ROW'S TRUTH (the delta review of + # #263, NIT): `delete_lines`' own sentences are archive-rows' — "the rows are + # in the archive", "the archive may have been appended" — and false here. + REG_COMMITTED_HERE="this act HAS committed its row for lane $ADD_ROW_LANE, locally and unpushed" + DL_SAYS_UNCHANGED="no row was removed, so that row is still in it and in that commit. Remove it from $LANES_PATH and commit that before any other write in $LANES_REPO, because the next one would publish it" + DL_SAYS_AFTER_WRITE="Its row for lane $ADD_ROW_LANE was not taken back, and this act HAS committed it, locally and unpushed: with the register whole, remove that row from $LANES_PATH and commit that before any other write in $LANES_REPO, because the next one would publish it." + delete_lines "$ar_line" + REG_COMMITTED_HERE=""; DL_SAYS_UNCHANGED=""; DL_SAYS_AFTER_WRITE="" + # A REMOVAL THAT CANNOT BE COMMITTED SAYS SO (delta review 4 of #263, N3). + # These returned in silence: the removal was left written or staged under + # add-row's commit, and the next write's own commit carried it, under its + # own caption, with no one told. + ar_msg="LANES($ADD_ROW_LANE@$WS): $ar_subject" + ar_fix="add-row's commit still holds the row. Commit the removal before any other write in $LANES_REPO, so that write does not carry it under its own caption: \`git -C $LANES_REPO commit -m '$ar_msg' -- $LANES_PATH\`" + git -C "$LANES_REPO" -c "$CP_EXACT" add -- "$LANES_PATH" || { + note "$ar_why; the rejected row was removed from $LANES_FILE, but staging that removal failed, so it is only in the working tree, and $ar_fix" + return 1 + } + git -C "$LANES_REPO" -c "$CP_EXACT" commit -q -m "$ar_msg" -- "$LANES_PATH" || { + note "$ar_why; the rejected row was removed from $LANES_FILE and that removal is staged, but its compensating commit failed, so $ar_fix" + return 1 + } + note "$ar_why; the rejected row was removed in a compensating LOCAL commit, so no later write publishes it. Neither commit was pushed." + return "$ar_back" } # Each lane's LAST line on , as @@ -11801,6 +12085,7 @@ EOF cat -- "$LANES_FILE" > "$msc_arch" || die "could not write the archive $msc_arch — nothing else has been touched" 6 cmp -s -- "$LANES_FILE" "$msc_arch" || die "the archive $msc_arch is not byte for byte the register it was copied from; nothing else has been touched" 5 note "archive written: $msc_arch_rel" + reg_wrote "$msc_arch_rel (the register as it was)" msc_paths=("$LANES_PATH" "$msc_arch_rel") while IFS="$US" read -r msc_n msc_lf msc_rel; do @@ -11816,10 +12101,14 @@ EOF done </dev/null || :)" die "lane $rr_l's object log $rr_lf could not be parsed, so whether it carries a lane-kind line is not known (Amendment 7(d)). Nothing was written." 1 fi rr_open="$(printf '%s\n%s\n' "$rr_ev" "$rr_lev" | awk -F"$US" '$3 == "STARTED" || $3 == "PAUSED" || $3 == "RESUMED" || $3 == "ENDED" || $3 == "RETIRED" { v = $3 } END { if (v != "") print v }')" - [ -z "$rr_open" ] || - die "lane $rr_l's object log carries a $rr_open line, so it is not a dormant row — a closed lane cannot be retired twice, and a running or parked lane must be ended as itself. Nothing was written." 2 + rr_pub_open="$(printf '%s\n' "$rr_ev" | awk -F"$US" '$3 == "STARTED" || $3 == "PAUSED" || $3 == "RESUMED" || $3 == "ENDED" || $3 == "RETIRED" { v = $3 } END { if (v != "") print v }')" + # THE RULE IS CLAUSE (c)'s AND STAYS: a log that CARRIES a lane-kind line is + # refused. THE SENTENCE FOLLOWS THE LAST ONE (#120 item 3, Copilot round 5 on + # #93). A log whose last lane-kind line is ENDED or RETIRED is a lane already + # CLOSED (19(a)), and `lane-end` is not its exit: it is told it is closed, + # already hidden, and — for RETIRED alone, the only cell `archive-rows` + # moves — which act takes the row out of the register. + # + # AND "CLOSED" IS SAID ONLY OF A CLOSING LINE ORIGIN CARRIES (the review of + # #263, S2). An earlier sweep that stopped part-way leaves its RETIRED line + # in this checkout's log and nowhere else; a re-run told the lane it was + # CLOSED, "nothing more is owed", about a row never retired and a line never + # published. That line is still refused, and is named as this checkout's. + case "$rr_open" in + '') : ;; + ENDED | RETIRED) + rr_art=a; [ "$rr_open" != ENDED ] || rr_art=an + [ "$rr_pub_open" = "$rr_open" ] || + die "lane $rr_l's object log in THIS CHECKOUT ends on $rr_art $rr_open line, and origin/$LANES_BRANCH's copy does not: it is a write made here and never published, by an earlier act that stopped part-way or a commit whose push has not landed (\`git -C $LANES_REPO status\` and \`git -C $LANES_REPO log origin/$LANES_BRANCH..HEAD -- $(log_path_for "$rr_l")\` say which). It is refused either way, since a second closing line would be one more line in an append-only log, and the lane is not called CLOSED, since nothing published says so. Publish that line or undo it, then re-run. Nothing was written." 2 + case "$rr_open" in + RETIRED) + die "lane $rr_l's object log ends on a RETIRED line, so the lane is already CLOSED and RETIRED (Amendment 19(a)), not a dormant row: \`lanes\` already hides it (19(b)), and its log is append-only, so it is not retired a second time. Its row leaves the register by \`lanes-edit.sh archive-rows ${rr_l%-*}\` (19(d)), which moves that repository's rows whose state cell says RETIRED. Nothing was written." 2 ;; + *) + die "lane $rr_l's object log ends on an ENDED line, so the lane is already CLOSED (Amendment 19(a)): it was ended, and a closed lane is not a dormant row. \`lanes\` already hides it (19(b)), and its log is append-only, so a RETIRED line after its ENDED would be one more line for a lane that is already finished. Nothing more is owed: its row stays in the register as the record (Rule 9), and \`archive-rows\` is not its exit, since it moves only rows whose state cell says RETIRED. Nothing was written." 2 ;; + esac ;; + *) + die "lane $rr_l's object log carries a $rr_open line as its last lane-kind line, so it is not a dormant row: a running or parked lane is a lane a person can still pick up. End it as itself when its work is done: lane-end $rr_l. Nothing was written." 2 ;; + esac # (c) REFUSAL 3 — THE ROW ITSELF SAYING THE LANE IS SOMEWHERE, which is the # same test the listing makes and is re-proved here rather than trusted @@ -12499,9 +12814,26 @@ $(session_ids_local_of_lane "$rr_l" 2>/dev/null || :)" # `log_path_ci` is a `ls-tree` of the whole log directory and this loop runs # per lane, so asking it twice for one lane is a second read of the same # tree for the same answer. + # + # AND THE QUESTION IS `ensure_log`'S OWN, NOT A PARAPHRASE OF IT (#120 item + # 2, Copilot round 5 on #93). The scan asked it only where this checkout had + # NO log, and beside that compared a BASENAME with a repo-relative PATH, a + # clause that could never be equal. So the state a case-only rename leaves + # when its commit has not been pushed — the row's spelling here, committed; + # the old spelling still in origin's tree; a checkout merely AHEAD, which the + # `behind` guard rightly lets through — passed the scan, and `ensure_log` + # refused it INSIDE THE WRITE LOOP, after the lanes before it in this one + # commit had had their logs created and appended. It is asked here as + # `ensure_log` asks it, in its words, because it is the same refusal; the + # refusal of a local file of another case above means `rr_from`, where + # there is one, is the row's own spelling. rr_pub="$rr_pubp" - if [ "${rr_pub##*/}" != "$(log_path_for "$rr_l")" ] && [ "$rr_pub" != "$(log_path_for "$rr_l")" ] && [ "$rr_ln2" = 0 ]; then - die "lane $rr_l's object log is published as $rr_pub and this checkout does not have it: writing one here now would leave TWO files for one lane (Amendment 15). Pull first — \`git -C $LANES_REPO pull --rebase\` — and re-run. Nothing was written." 2 + rr_from="" + [ "$rr_ln2" = 1 ] && rr_from="$rr_loc" + if [ "$rr_pub" != "$(log_path_for "$rr_l")" ] && [ "${rr_from##*/}" != "${rr_pub##*/}" ]; then + rr_here="and this checkout does not have it" + [ -n "$rr_from" ] && rr_here="while this checkout has ${LANES_LOG_PREFIX}${rr_from##*/} instead — a case-only rename whose commit never landed" + die "lane ${rr_l}'s object log is published as $rr_pub $rr_here: the row spells the lane $rr_l, so the log is $(log_path_for "$rr_l") (Amendment 15), and writing one here now would leave TWO files for one lane. Pull first — \`git -C $LANES_REPO pull --rebase\` — and re-run. Nothing was written." 2 fi # THE LINE AND THE CELL, BOTH BUILT AND BOTH CHECKED BEFORE ANYTHING IS @@ -12548,12 +12880,14 @@ $(session_ids_local_of_lane "$rr_l" 2>/dev/null || :)" ensure_log "$rr_l" append_text_block "$(log_file_for "$rr_l")" "$(cat -- "$rr_tmp/line.$rr_ln")" rr_paths+=("$(log_path_for "$rr_l")") + reg_wrote "$(log_path_for "$rr_l") (its RETIRED line)" done < "$LANES_ARCH_FILE" || die "could not write $LANES_ARCH_FILE — nothing has been removed from the register" 6 note "created $LANES_ARCH_FILE" + reg_wrote "$LANES_ARCH_PATH (created)" fi append_text_block "$LANES_ARCH_FILE" "$(cat -- "$ar_tmp/rows")" + reg_wrote "$LANES_ARCH_PATH ($ar_n row(s) appended)" delete_lines "$(cat -- "$ar_tmp/nums")" ar_msg="LANES(archive@$WS): Amendment 19(d) — $ar_n RETIRED row(s) of $ar_repo moved to $LANES_ARCH_PATH: $(tr '\n' ' ' < "$ar_tmp/names")" @@ -14947,9 +15283,16 @@ Nothing was written." 2 # AND A READ THAT FAILED IS A REFUSAL, not an empty archive (round 2): the # whole point of this check is that a retired identity is never reissued, # and "I could not read the file that says which ones are retired" is not - # "none of them are" (Amendment 7(d)). - ar_arch="$(retired_identity_hits "$lane_new")" || - die "$LANES_ARCH_PATH exists and could not be read, so whether '$lane_new' is a name this estate has RETIRED is not known — and a retired position is never reissued (Amendment 19(b)). A read that failed is not 'it is not retired' (Amendment 7(d)). Fix the read and re-run; nothing was written." 1 + # "none of them are" (Amendment 7(d)). NOR IS A LINE IT COULD NOT TAKE + # APART (#122): that row may be this lane's own, so it is the same refusal + # and the same 1, naming the line. + ar_arch_rc=0 + ar_arch="$(retired_identity_hits "$lane_new")" || ar_arch_rc=$? + case "$ar_arch_rc" in + 0) : ;; + 3) die "$LANES_ARCH_PATH carries a line this scan cannot take apart (above), so whether '$lane_new' is a name this estate has RETIRED is not known: a row this scan cannot take apart may be that lane's own, and a retired position is never reissued (Amendment 19(b)). A row that could not be read is not a row that is not there (Amendment 7(d)). Put the lane back in backticks in that row's first cell, or remove the line if it is no row; commit the archive (\`git -C $LANES_REPO commit -m \"\" -- $LANES_ARCH_PATH\`), publish it where the line is on origin (\`LANES_LANE= lanes-edit.sh commit \"\"\`), and re-run. Nothing was written." 1 ;; + *) die "$LANES_ARCH_PATH exists and could not be read or scanned, so whether '$lane_new' is a name this estate has RETIRED is not known — and a retired position is never reissued (Amendment 19(b)). A read that failed is not 'it is not retired' (Amendment 7(d)). Fix the read and re-run; nothing was written." 1 ;; + esac if [ -n "$ar_arch" ]; then die "lane '$lane_new' is RETIRED: its row is in $LANES_ARCH_PATH, spelled $(printf '%s\n' "$ar_arch" | tr '\n' ' ')— and a retired position is NEVER REISSUED (Amendment 19(b)). That lane's object log is $(log_path_for "$lane_new") and it is append-only, so a second lane under this name would write its life into the first one's file and every read of that log would answer for two lanes at once. Take a free position instead — \`lanes --prefix \` prints the next one and the line that takes it. Nothing was written." 2 fi @@ -17450,12 +17793,13 @@ EOF printf '%s\n' "$rr_row" ;; - # 0 with archived spellings, 8 absent, 1 when either archive cannot be read. + # 0 with archived spellings, 8 absent, 1 when either archive cannot be read + # or carries a line that is not a row (#122, named above the refusal). retired-identity) lane="${1-}"; [ -n "$lane" ] || die "usage: retired-identity " 2 check_lane_name "$lane" log_sync - rr_hits="$(retired_identity_hits "$lane")" || die "could not read the lane archive" 1 + rr_hits="$(retired_identity_hits "$lane")" || die "could not read the lane archive, or it carries a line that is not a row (above)" 1 [ -n "$rr_hits" ] || exit 8 printf '%s\n' "$rr_hits" ;; diff --git a/tests/test_register_hardening_120_122.py b/tests/test_register_hardening_120_122.py new file mode 100644 index 00000000..3d98b5db --- /dev/null +++ b/tests/test_register_hardening_120_122.py @@ -0,0 +1,778 @@ +# SPDX-License-Identifier: Apache-2.0 +"""The register's two rewrites prove what they copied and what landed, the +sweep's preflight asks `ensure_log`'s own question, a lane whose log already +closes it is told so, and add-row's archive scan refuses a line it cannot take +apart: opensoft/openRepoTools#120 (Copilot round 5 on #93) and #122 (round 6). + +Each case builds the throwaway register `test_register_scan_hardening_138.py` +builds — a bare repository and a clone of it in a temporary directory, with a +temporary `$HOME` — and runs the real `lanes-edit.sh` against it. A write that +has to come out short is cut by a `cat` first on `PATH` that cuts ONE exact +copy and hands every other call to the real one; a scan that has to fail, by +an `awk` that refuses ONE exact program. Nothing here reaches a network or the +operator's own register. +""" + +from __future__ import annotations + +import os +import re +import shutil +import stat +import subprocess +from pathlib import Path + +import pytest + +from test_register_scan_hardening_138 import ( # noqa: F401 (pytestmark is this module's too) + HEADER, OLD_ID, UNPRIVILEGED_ONLY, Register as SweepRegister, lane_line, log_text, pytestmark, row) + +NEW_ID = "cccc0122-0003-4000-8000-000000000003" +#: The archive exactly as `archive_rows` creates it: a title, a paragraph, then +#: the header (line 7) and its rule (line 8). Rows follow from line 9. +ARCHIVE_HEAD = ("# LANES-retired.md — rows RETIRED out of `lanes/LANES.md`\n\n" + "lane-collision-protocol Amendment 19(d). These lanes are finished. Their\n" + "positions are NEVER REISSUED: every reader of the next free position and\n" + "`lanes --closed` read this file beside the register.\n\n" + HEADER) +#: The copy seam reads where a `cat`'s output is going, which only Linux says. +PROC_FD = pytest.mark.skipif(not Path("/proc/self/fd/1").exists(), + reason="the copy seam reads /proc//fd/1") + + +class Register(SweepRegister): + """`test_register_scan_hardening_138`'s register, with thirty rows of another + repository below repoX's — so a copy cut at half its lines still holds every + row a case edits — and the seams these cases need.""" + + def __init__(self, tmp: Path): + super().__init__(tmp) + self.write("lanes/LANES.md", self.read("lanes/LANES.md") + "".join( + row(f"repoZ-{n}", OLD_ID, f"another repository's row {n}") for n in range(1, 31))) + self.publish("another repository's rows") + + def published(self, rel: str) -> str: + return self.raw_git("-C", str(self.origin), "show", f"main:{rel}") + + def shim(self, name: str, body: str) -> None: + """A `` first on PATH: `body`, then the real one.""" + real = shutil.which(name, path=os.environ.get("PATH")) + path = self.fakebin / name + path.write_text(f'#!/usr/bin/env bash\n{body}\nexec "{real}" "$@"\n') + path.chmod(0o755) + + def cut_cat(self, cut: str) -> None: + """A `cat` that cuts a copy at half its lines. `copy…`: the + register's copy into a file named `pre`, the copy every proof of a + rewrite is made against. `write…`: the rewrite itself, `cat -- /out` + into the register — and with `…and restore fail`, the copy's bytes put + back into the register as well. `…fail(s)` exits 1 after the half, as a + full disk does; `…says done` exits 0, a short write nothing reported.""" + rc = 0 if cut.endswith("says done") else 1 + goes_to = 'case "$(readlink "/proc/$$/fd/1" 2>/dev/null)" in {} ) true ;; *) false ;; esac' + if cut.startswith("copy"): + when = ('[ "${1-}" = -- ] && case "${2-}" in */lanes/LANES.md) true ;; *) false ;; esac && ' + + goes_to.format("*/pre")) + elif cut == "write and restore fail": + when = ('[ "${1-}" = -- ] && case "${2-}" in */out) true ;; ' + '*/pre) ' + goes_to.format("*/lanes/LANES.md") + ' ;; *) false ;; esac') + else: + when = '[ "${1-}" = -- ] && case "${2-}" in */out) true ;; *) false ;; esac' + self.shim("cat", f'if {when}; then\n' + f' n=$(wc -l < "$2"); head -n $((n / 2)) -- "$2"; exit {rc}\nfi') + + +@pytest.fixture +def reg(tmp_path: Path) -> Register: + return Register(tmp_path) + + +# ------------------------------------------- #120 item 1: the two rewrites + +def retire_repo_x_3(reg: Register) -> None: + reg.write("lanes/LANES.md", reg.read("lanes/LANES.md").replace( + "| more old words |", "| RETIRED · 2026-09-01T00:00:00Z · swept |")) + reg.publish("repoX-3 retired") + + +#: The one act per rewrite: `set-row-state` ends in `replace_line`, and +#: `archive-rows --yes` in `delete_lines`. +ACTS = { + "replace_line": ("set-row-state", "repoX-2", "PAUSED · cut short"), + "delete_lines": ("archive-rows", "repoX", "--yes"), +} +CUTS = ["copy fails", "copy says done", "write fails", "write says done", + "write and restore fail"] + + +@PROC_FD +@pytest.mark.parametrize("cut", CUTS) +@pytest.mark.parametrize("act", sorted(ACTS)) +def test_120_item_1_a_rewrite_cut_short_leaves_the_register_whole(reg, act, cut): + """#120 item 1: `cat -- > LANES.md` had its status read, but not + its result, and `>` truncates before `cat` writes. A write that failed + part-way left a HALF register behind a refusal that called it UNCHANGED; + one that was short and said nothing was a half register committed (exit + 0). And the COPY every proof is made against was read with no status at + all, so a short copy made a short register look like a one-line edit of + itself: rewritten, committed and pushed. Each refuses (5) now, and the + register on disk and on origin is byte for byte what it was — or, where + the bytes could not be put back either, the refusal says the register MAY + NOW BE TRUNCATED, keeps the exact copy it was made from and names where, + and nothing is committed (the review of #263, NIT 6). The copy is kept in + the checkout's own git directory, mode 0600, under a name of its own, where + a restart that empties the temporary directory leaves it (the delta + review's NIT). The copy refusal says THIS ACT has committed nothing and + names what the act had already written here, never "Nothing was written" + (S2).""" + if act == "delete_lines": + retire_repo_x_3(reg) + register = reg.read("lanes/LANES.md") + before = reg.head() + reg.cut_cat(cut) + result = reg.run(*ACTS[act], LANES_LANE="repoX-1") + assert result.returncode == 5, (result.stdout, result.stderr) + assert reg.head() == before + assert reg.published("lanes/LANES.md") == register + if cut == "write and restore fail": + assert "IT MAY NOW BE TRUNCATED" in result.stderr + assert "UNCHANGED" not in result.stderr + found = re.search(r"are kept in (\S+) \(", result.stderr) + assert found, result.stderr + kept = Path(found.group(1)) + assert kept.read_text(encoding="utf-8") == register + assert kept.parent.resolve() == Path(reg.git("rev-parse", "--absolute-git-dir").strip()).resolve() + assert re.fullmatch(r"lanes-edit-register-before-edit\.\d{8}T\d{6}Z\.\d+", kept.name), kept.name + assert stat.S_IMODE(kept.stat().st_mode) == 0o600 + return + assert reg.read("lanes/LANES.md") == register + if cut.startswith("copy"): + assert "could not be copied whole" in result.stderr + assert "this act has committed nothing" in result.stderr + assert "Nothing was written" not in result.stderr + if act == "delete_lines": + assert "NOT committed: lanes/archive/LANES-retired.md (created)" in result.stderr + else: + assert "have been put back and proved" in result.stderr + assert "UNCHANGED" not in result.stderr + + +@PROC_FD +def test_120_item_1_a_refusal_after_a_captured_peer_edit_says_this_act_committed_nothing(reg): + """The delta review of #263: a peer's uncommitted register edit found + before the act is captured as a commit of its own (`handle_preexisting`), + so a refusal after it that said "nothing has been committed" was false. It + says what holds: this act has committed nothing.""" + before = reg.head() + reg.write("lanes/LANES.md", reg.read("lanes/LANES.md").replace( + "another repository's row 30 |", "another repository's row 30, edited by a peer |")) + edited = reg.read("lanes/LANES.md") + reg.cut_cat("copy fails") + result = reg.run(*ACTS["replace_line"], LANES_LANE="repoX-1") + assert result.returncode == 5, (result.stdout, result.stderr) + assert "captured pre-existing edit" in result.stderr + assert reg.git("log", "--format=%s", f"{before}..HEAD").startswith("LANES(pre-existing@") + assert reg.git("rev-list", "--count", f"{before}..HEAD").strip() == "1" + assert "this act has committed nothing" in result.stderr + assert "nothing has been committed" not in result.stderr + assert reg.read("lanes/LANES.md") == edited + + +@UNPRIVILEGED_ONLY +@pytest.mark.parametrize("act", sorted(ACTS)) +def test_120_item_1_a_write_that_never_reached_the_register_says_it_is_as_it_was(reg, act): + """The review of #263, NIT 3 (its X4): a register the write could not even + open is byte for byte what it was, and the refusal says so in words of its + own, apart from the put-back's and the truncation's. archive-rows has + appended the archive by then, and the refusal names it (S2).""" + if act == "delete_lines": + retire_repo_x_3(reg) + register = reg.read("lanes/LANES.md") + before = reg.head() + path = reg.wip / "lanes/LANES.md" + path.chmod(0o444) + try: + result = reg.run(*ACTS[act], LANES_LANE="repoX-1") + finally: + path.chmod(0o644) + assert result.returncode == 5, (result.stdout, result.stderr) + assert "the write did not reach it, and it is byte for byte as it was before this edit" in result.stderr + assert "put back" not in result.stderr + assert "MAY NOW BE TRUNCATED" not in result.stderr + assert reg.read("lanes/LANES.md") == register + assert reg.head() == before + assert reg.published("lanes/LANES.md") == register + if act == "delete_lines": + assert "NOT committed: lanes/archive/LANES-retired.md (created)" in result.stderr + + +@PROC_FD +def test_120_item_1_a_sweep_refused_part_way_names_what_it_wrote_and_is_not_called_closed(reg): + """The review of #263, S2: retire-rows appends a lane's RETIRED line before + it rewrites the row, so a copy refusal that said "Nothing was written" left + `?? lanes/log/repoX-2.md` behind it. And a re-run then read that + uncommitted line and called the lane "already CLOSED … nothing more is + owed", about a row never retired and a line never published. The refusal + names the log, and the re-run refuses the line as this checkout's alone.""" + reg.cut_cat("copy fails") + result = reg.sweep("repoX-2") + assert result.returncode == 5, (result.stdout, result.stderr) + assert "this act has committed nothing" in result.stderr + assert "NOT committed: lanes/log/repoX-2.md (its RETIRED line)" in result.stderr + assert "Nothing was written" not in result.stderr + (reg.fakebin / "cat").unlink() + rerun = reg.sweep("repoX-2") + assert rerun.returncode == 2, (rerun.stdout, rerun.stderr) + assert "in THIS CHECKOUT ends on a RETIRED line, and origin/main's copy does not" in rerun.stderr + assert "already CLOSED" not in rerun.stderr + assert "`repoX-2`" in reg.published("lanes/LANES.md") + assert "RETIRED" not in reg.published("lanes/LANES.md").split("`repoX-2`")[1].splitlines()[0] + + +# ------------------------------- #120 item 2: the sweep's published-path scan + +def case_sensitive(path: Path) -> bool: + probe = path / "case-probe" + probe.write_text("") + try: + return not (path / "CASE-PROBE").exists() + finally: + probe.unlink() + + +def test_120_item_2_a_rename_whose_commit_never_landed_refuses_in_the_scan(reg): + """#120 item 2: the sweep's preflight asked whether origin publishes the + log under another case only where this checkout had NO log, and compared a + basename with a path besides. So the state a case-only rename leaves when + its commit is not pushed — the row's spelling here, committed; the old + spelling still on origin; a checkout merely AHEAD — passed the scan, and + `ensure_log` refused it inside the write loop, after the lane before it had + had its log created and appended. The scan refuses it now, with + `ensure_log`'s own words, and nothing is written.""" + if not case_sensitive(reg.tmp): + pytest.skip("two log names that differ only by case need a case-sensitive filesystem") + reg.write("lanes/log/repox-3.md", log_text("repox-3", lane_line( + "NOTED", "repoX-3", OLD_ID, "2026-09-13T00:00:00Z") + " — migrated; history kept here")) + reg.publish("repoX-3's log, under the spelling it had before Amendment 15") + reg.git("mv", "lanes/log/repox-3.md", "lanes/log/repoX-3.md") + reg.git("commit", "-q", "-m", "repoX-3's log takes the row's spelling (not pushed)") + before = reg.head() + result = reg.sweep("repoX-2", "repoX-3") + assert result.returncode == 2, (result.stdout, result.stderr) + assert "published as lanes/log/repox-3.md" in result.stderr + assert "a case-only rename whose commit never landed" in result.stderr + assert reg.head() == before + assert not (reg.wip / "lanes/log/repoX-2.md").exists() + assert reg.dirty() == "" + + +# ------------------------------- #120 item 3: a log that already closes the lane + +@pytest.mark.parametrize("last", ["ENDED", "RETIRED"]) +def test_120_item_3_a_lane_whose_log_closes_it_is_told_it_is_closed(reg, last): + """#120 item 3: the refusal for a lane whose log's LAST lane-kind line is + ENDED or RETIRED read the same as the one for a running lane, and named no + way out. It says the lane is already CLOSED and hidden (19(b)) — never + `lane-end`, which is not the exit for a lane that has ended. For RETIRED it + names `archive-rows ` (19(d)) exactly; for ENDED it says archive-rows + is not the row's exit, and never that the lane "is not retired a second + time" (the review of #263, NITs 3 and 5).""" + reg.write("lanes/log/repoX-2.md", log_text( + "repoX-2", lane_line("STARTED", "repoX-2", OLD_ID, "2026-09-11T00:00:00Z"), + lane_line(last, "repoX-2", OLD_ID, "2026-09-11T02:00:00Z"))) + reg.publish(f"repoX-2 started and {last.lower()}") + before = reg.head() + result = reg.sweep("repoX-2") + assert result.returncode == 2, (result.stdout, result.stderr) + assert f"{last} line" in result.stderr + assert "already CLOSED" in result.stderr + assert "lane-end" not in result.stderr + if last == "RETIRED": + assert "`lanes-edit.sh archive-rows repoX`" in result.stderr + else: + assert "`archive-rows` is not its exit" in result.stderr + assert "lanes-edit.sh archive-rows" not in result.stderr + assert "retired a second time" not in result.stderr + assert reg.head() == before + + +@pytest.mark.parametrize("last", ["ENDED", "RETIRED"]) +def test_120_item_3_a_closing_line_only_this_checkout_has_is_not_called_closed(reg, last): + """The review of #263, S2, from a commit rather than a cut sweep: a closing + line committed here and never pushed is refused as this checkout's alone, + never called CLOSED, and with its article right: "an ENDED line" (the + delta review's NIT).""" + started = lane_line("STARTED", "repoX-2", OLD_ID, "2026-09-11T00:00:00Z") + reg.write("lanes/log/repoX-2.md", log_text("repoX-2", started)) + reg.publish("repoX-2 started") + reg.write("lanes/log/repoX-2.md", log_text( + "repoX-2", started, lane_line(last, "repoX-2", OLD_ID, "2026-09-12T00:00:00Z"))) + reg.commit(f"repoX-2 {last.lower()}, not pushed") + before = reg.head() + result = reg.sweep("repoX-2") + assert result.returncode == 2, (result.stdout, result.stderr) + article = "an" if last == "ENDED" else "a" + assert (f"in THIS CHECKOUT ends on {article} {last} line, and origin/main's copy does not" + in result.stderr) + assert "already CLOSED" not in result.stderr + assert reg.head() == before + + +def test_120_item_3_a_lane_whose_log_says_it_is_parked_is_sent_to_lane_end(reg): + """…and the other branch: a log whose last lane-kind line is STARTED, + PAUSED or RESUMED is a lane a person can still pick up, so the refusal + sends it to `lane-end ` and not to the archive.""" + reg.write("lanes/log/repoX-2.md", log_text( + "repoX-2", lane_line("STARTED", "repoX-2", OLD_ID, "2026-09-11T00:00:00Z"), + lane_line("PAUSED", "repoX-2", OLD_ID, "2026-09-11T01:00:00Z"))) + reg.publish("repoX-2 is parked") + result = reg.sweep("repoX-2") + assert result.returncode == 2, (result.stdout, result.stderr) + assert "carries a PAUSED line" in result.stderr + assert "lane-end repoX-2" in result.stderr + assert "archive-rows" not in result.stderr + + +# --------------------------------------------- #122: add-row's archive scan + +RETIRED_CELL = "RETIRED · 2026-08-02T00:00:00Z · moved" +#: Three ways a hand edit or a bad merge leaves a row of repoX-4 that the scan +#: cannot take apart, each with what it does to the first cell. +MANGLED = { + "the lane lost its backticks": f"| repoX-4 | harness `{OLD_ID}` | Eagle / test / brett | 2026-08-01 | none | none | {RETIRED_CELL} |", + "no backtick at all": f"| repoX-4 | harness {OLD_ID} | Eagle / test / brett | 2026-08-01 | none | none | {RETIRED_CELL} |", + "one backtick lost": f"| `repoX-4 | harness {OLD_ID} | Eagle / test / brett | 2026-08-01 | none | none | {RETIRED_CELL} |", +} + + +#: The row `add_repo_x_4` adds, with its newline: a peer that publishes it +#: first publishes the very line add-row appends. +NEW_ROW = row("repoX-4", NEW_ID, "new here", "2026-10-19") +ARCHIVE = "lanes/archive/LANES-retired.md" + + +def add_repo_x_4(reg: Register) -> subprocess.CompletedProcess: + return reg.run("add-row", NEW_ROW.rstrip("\n"), LANES_LANE="repoX-4") + + +def peer_pushes_during_pull(reg: Register, tmp_path: Path, changes: dict[str, str], *, + diff_log: Path | None = None, diff_context: int | None = None, + extra_shell: str = "") -> Path: + """The shell suite's `repo19race` shape. A peer clone commits `changes` + (a path and its whole new text each, one commit each, in order) now, and a + `git` first on PATH pushes them the moment the writer's own checkout runs + its first `pull`: after add-row has committed its row, before it pushes. + Returns the mark the shim leaves; the caller removes the shim. + + The same shim sees add-row's take-back diff (the register's `-U0` diff + against origin's). `diff_log` records how each was called, the + `GIT_DIFF_OPTS` it saw and its arguments; `diff_context` runs it with that + much context instead, as any source of context the take-back cannot turn + off would. `extra_shell` runs in the shim before it hands each call to the + real `git`, with `@MARK@` standing for the mark's path.""" + peer = tmp_path / "peer" + reg.raw_git("clone", "-q", str(reg.origin), str(peer)) + for rel, text in changes.items(): + (peer / rel).parent.mkdir(parents=True, exist_ok=True) + (peer / rel).write_text(text, encoding="utf-8") + reg.raw_git("-C", str(peer), "add", "--", rel) + reg.raw_git("-C", str(peer), "commit", "-q", "-m", f"peer: {rel}") + real = shutil.which("git", path=os.environ.get("PATH")) + mark = tmp_path / "raced" + shim = reg.fakebin / "git" + take_back = "" + if diff_log is not None: + take_back += f""" printf '%s\\t%s\\n' "${{GIT_DIFF_OPTS-}}" "$*" >> '{diff_log}'\n""" + if diff_context is not None: + take_back += (f' a=(); for x in "$@"; do if [ "$x" = -U0 ]; then a+=(-U{diff_context}); ' + 'else a+=("$x"); fi; done; set -- "${a[@]}"\n') + if take_back: + take_back = ('case " $* " in *" -U0 refs/remotes/origin/main -- "*)\n' + + take_back + " ;;\nesac\n") + if extra_shell: + take_back += extra_shell.replace("@MARK@", str(mark)) + "\n" + shim.write_text(f"""#!/usr/bin/env bash +if [ "${{1-}}" = -C ] && [ "${{3-}}" = pull ] && [ ! -e '{mark}' ]; then + case "${{2-}}" in */wip) + : > '{mark}' + '{real}' -C '{peer}' push -q origin main || exit 1 ;; + esac +fi +{take_back}exec '{real}' "$@" +""") + shim.chmod(0o755) + return mark + + +def a_good_archive(reg: Register) -> str: + reg.write(ARCHIVE, ARCHIVE_HEAD + row("repoX-9", OLD_ID, RETIRED_CELL, "2026-08-01")) + reg.publish("a good archive") + return reg.read(ARCHIVE) + + +@pytest.mark.parametrize("where", ["published", "committed here, not pushed"]) +@pytest.mark.parametrize("mangle", sorted(MANGLED)) +def test_122_an_archive_row_the_scan_cannot_take_apart_reissues_no_position(reg, mangle, where): + """#122: the scan skipped any archive table line with no backticked lane + token — and read a first backtick in ANOTHER cell as the lane — so a + retired row whose lane cell a hand edit had mangled disappeared from the + answer, and add-row reissued its position (exit 0). Such a line is refused + by its line number and its copy now (1), in add-row's own words. The + archive's own header and its rule are `|` lines with no backtick on every + archive `archive_rows` writes, and they are not named.""" + mangled = MANGLED[mangle] + reg.write("lanes/archive/LANES-retired.md", ARCHIVE_HEAD + + row("repoX-9", OLD_ID, RETIRED_CELL, "2026-08-01") + mangled + "\n") + if where == "published": + reg.publish("the archive, with a hand-mangled row 10") + else: + reg.commit("the archive, with a hand-mangled row 10 (not pushed)") + before = reg.head() + result = add_repo_x_4(reg) + assert result.returncode == 1, (result.stdout, result.stderr) + copies = "origin/main and this checkout" if where == "published" else "this checkout" + assert f"line 10 ({copies}): {mangled}" in result.stderr + assert "carries a line this scan cannot take apart (above)" in result.stderr + assert "line 7 (" not in result.stderr + assert "line 8 (" not in result.stderr + assert reg.head() == before + assert "`repoX-4`" not in reg.published("lanes/LANES.md") + + +def test_122_an_archive_scan_that_failed_is_not_an_archive_with_no_match(reg): + """#122, the `|| :` half: the scan's status was the `sort -u` behind it, + so an awk that FAILED read as "no archived row matched" and the archived + repoX-4 was added again (exit 0). It is a refusal (1) now.""" + reg.write("lanes/archive/LANES-retired.md", ARCHIVE_HEAD + + row("repoX-4", OLD_ID, RETIRED_CELL, "2026-08-01")) + reg.publish("the archive holds repoX-4") + reg.shim("awk", 'case "$*" in *"function key("*) echo "test shim: awk refused" >&2; exit 2 ;; esac') + before = reg.head() + result = add_repo_x_4(reg) + assert result.returncode == 1, (result.stdout, result.stderr) + assert reg.head() == before + assert "`repoX-4`" not in reg.published("lanes/LANES.md") + + +def test_122_a_line_number_says_which_copy_it_is_in(reg): + """The review of #263, NIT 4: with the published archive and this + checkout's apart, one bad line was listed as "line 10" and "line 12" with + no word on which file either number belongs to.""" + mangled = MANGLED["the lane lost its backticks"] + reg.write("lanes/archive/LANES-retired.md", ARCHIVE_HEAD + + row("repoX-9", OLD_ID, RETIRED_CELL, "2026-08-01") + mangled + "\n") + reg.publish("the archive, with a hand-mangled row 10") + reg.write("lanes/archive/LANES-retired.md", ARCHIVE_HEAD + + row("repoX-7", OLD_ID, RETIRED_CELL, "2026-08-01") + + row("repoX-8", OLD_ID, RETIRED_CELL, "2026-08-01") + + row("repoX-9", OLD_ID, RETIRED_CELL, "2026-08-01") + mangled + "\n") + reg.commit("two more rows above it, not pushed") + result = add_repo_x_4(reg) + assert result.returncode == 1, (result.stdout, result.stderr) + assert f"line 10 (origin/main): {mangled}" in result.stderr + assert f"line 12 (this checkout): {mangled}" in result.stderr + + +def test_122_a_refusal_after_the_rebase_takes_its_row_back(reg, tmp_path): + """The review of #263, M1, the shell suite's `repo19race` shape: a peer + publishes an archive line this scan cannot take apart while add-row is in + its `pull --rebase`. The after-rebase rescan refused (1), but only the + retired-match branch took the committed row back, so the next unrelated + write from this checkout published the row add-row had refused. Every + refusal there now removes it in a compensating local commit.""" + archive = a_good_archive(reg) + mark = peer_pushes_during_pull(reg, tmp_path, { + ARCHIVE: archive + MANGLED["the lane lost its backticks"] + "\n"}) + result = add_repo_x_4(reg) + (reg.fakebin / "git").unlink() + assert mark.exists(), "the peer never raced the pull" + assert result.returncode == 1, (result.stdout, result.stderr) + assert "removed in a compensating LOCAL commit" in result.stderr + assert "`repoX-4`" not in reg.read("lanes/LANES.md") + assert "`repoX-4`" not in reg.git("diff", "origin/main", "HEAD", "--", "lanes/LANES.md") + assert "`repoX-4`" not in reg.published("lanes/LANES.md") + after = reg.run("set-row-state", "repoX-2", "PAUSED · the next unrelated write", LANES_LANE="repoX-1") + assert after.returncode == 0, (after.stdout, after.stderr) + assert "PAUSED · " in reg.published("lanes/LANES.md").split("`repoX-2`")[1].splitlines()[0] + assert "`repoX-4`" not in reg.published("lanes/LANES.md") + + +#: What the rescan after the rebase refuses, each with add-row's exit: a peer's +#: archive line it cannot take apart (another lane's), or a peer's retirement +#: of repoX-4 itself. +RESCAN_REFUSALS = { + "a line the scan cannot take apart": ( + MANGLED["the lane lost its backticks"].replace("repoX-4", "repoQ-7") + "\n", 1), + "a retirement": (row("repoX-4", OLD_ID, RETIRED_CELL, "2026-08-01"), 2), +} + + +@pytest.mark.parametrize("why", sorted(RESCAN_REFUSALS)) +def test_122_a_refusal_after_the_rebase_leaves_a_peers_identical_row_alone(reg, tmp_path, why): + """The delta review of #263, D1: a peer publishes the byte-identical + repoX-4 row while add-row is in its `pull --rebase`, so the rebase drops + add-row's commit as already upstream and the one copy left is the PEER's. + The take-back found it by its exact text and deleted it in a local commit, + and the next write published that deletion: the peer's row gone from + origin. Only a copy this checkout adds over origin's is taken back now, so + here none is, for either refusal of the rescan.""" + archive = a_good_archive(reg) + refusal, code = RESCAN_REFUSALS[why] + mark = peer_pushes_during_pull(reg, tmp_path, { + "lanes/LANES.md": reg.read("lanes/LANES.md") + NEW_ROW, ARCHIVE: archive + refusal}) + result = add_repo_x_4(reg) + (reg.fakebin / "git").unlink() + assert mark.exists(), "the peer never raced the pull" + assert result.returncode == code, (result.stdout, result.stderr) + ahead = reg.git("log", "--oneline", "origin/main..HEAD") + diff = reg.git("diff", "origin/main", "HEAD") + after = reg.run("set-row-state", "repoX-2", "PAUSED · the next unrelated write", LANES_LANE="repoX-1") + assert after.returncode == 0, (after.stdout, after.stderr) + assert reg.published("lanes/LANES.md").count("`repoX-4`") == 1, (ahead, result.stderr) + assert ahead == "" and diff == "" + assert "origin/main already carries this exact row" in result.stderr + assert "compensating LOCAL commit" not in result.stderr + + +def test_122_a_refusal_after_the_rebase_takes_back_its_own_row_beside_a_peers_copy(reg, tmp_path): + """…and the other direction. Where the peer put the identical row in + another place, the rebase KEEPS add-row's commit and the register holds + two copies. The one this checkout adds is taken back: not the peer's, and + not neither. "origin already has the row" would have kept add-row's + commit, and the next write would have published a second copy.""" + archive = a_good_archive(reg) + seeded = row("repoX-3", OLD_ID, "more old words") + peer_register = reg.read("lanes/LANES.md").replace(seeded, seeded + NEW_ROW) + assert peer_register != reg.read("lanes/LANES.md") + mark = peer_pushes_during_pull(reg, tmp_path, { + "lanes/LANES.md": peer_register, + ARCHIVE: archive + RESCAN_REFUSALS["a line the scan cannot take apart"][0]}) + result = add_repo_x_4(reg) + (reg.fakebin / "git").unlink() + assert mark.exists(), "the peer never raced the pull" + assert result.returncode == 1, (result.stdout, result.stderr) + register = reg.read("lanes/LANES.md") + diff = reg.git("diff", "origin/main", "HEAD", "--", "lanes/LANES.md") + after = reg.run("set-row-state", "repoX-2", "PAUSED · the next unrelated write", LANES_LANE="repoX-1") + assert after.returncode == 0, (after.stdout, after.stderr) + assert reg.published("lanes/LANES.md").count("`repoX-4`") == 1, result.stderr + assert register == peer_register + assert diff == "" + assert "removed in a compensating LOCAL commit" in result.stderr + + +#: The ways the take-back cannot remove its row once it has found it: its copy +#: of the register cut short, its write cut short (put back), the pass that +#: removes the line refused, a pass that dropped a line too many, and a +#: `numstat` proof that reads the copies as binary (an attribute on every +#: path, which no `core.bigFileThreshold` changes). +TAKE_BACK_CUTS = [pytest.param("copy", marks=PROC_FD), pytest.param("write", marks=PROC_FD), + "awk pass", "line count", "numstat"] + + +@pytest.mark.parametrize("cut", TAKE_BACK_CUTS) +def test_122_a_take_back_that_cannot_rewrite_says_add_row_has_committed(reg, tmp_path, cut): + """The delta review of #263, on the wording: where the take-back itself + cannot rewrite the register, add-row HAS committed its row (locally, not + pushed), and the refusal says so and how to remove it, never that this + act has committed nothing. And in add-row's own words (delta review 3's + NIT): `delete_lines`' sentences are archive-rows' ("the rows are in" the + archive, "the archive may have been appended"), and false here. And on + every refusal `delete_lines` makes after its copy (delta review 4, N1 and + S4's N2): the pass, the line count and `numstat` named no row and no + commit, so "the removal added or rewrote lines" was all add-row said.""" + archive = a_good_archive(reg) + mark = peer_pushes_during_pull(reg, tmp_path, { + ARCHIVE: archive + RESCAN_REFUSALS["a line the scan cannot take apart"][0]}) + real_awk = shutil.which("awk", path=os.environ.get("PATH")) + if cut == "awk pass": + reg.shim("awk", 'case "$*" in *LANES_DROP_LINES*) echo "test shim: the removal pass refused" >&2; exit 2 ;; esac') + elif cut == "line count": + reg.shim("awk", f'case "$*" in *LANES_DROP_LINES*) "{real_awk}" "$@" | sed 1d; exit 0 ;; esac') + elif cut == "numstat": + (reg.home / "attributes").write_text("* -diff\n", encoding="utf-8") + (reg.home / ".gitconfig").write_text(f"[core]\n\tattributesFile = {reg.home / 'attributes'}\n", + encoding="utf-8") + elif cut == "copy": + copies = tmp_path / "copies" + # The register's SECOND copy into a `pre` is cut: add-row's own append + # makes the first, the take-back's `delete_lines` the second. + reg.shim("cat", 'if [ "${1-}" = -- ] && case "${2-}" in */lanes/LANES.md) true ;; *) false ;; esac && ' + 'case "$(readlink "/proc/$$/fd/1" 2>/dev/null)" in */pre) true ;; *) false ;; esac; then\n' + f" echo copy >> '{copies}'\n" + f" if [ \"$(grep -c . '{copies}')\" -ge 2 ]; then n=$(wc -l < \"$2\"); head -n $((n / 2)) -- \"$2\"; exit 1; fi\n" + "fi") + else: + # The take-back's rewrite is the one write from an `out` add-row makes. + reg.cut_cat("write fails") + result = add_repo_x_4(reg) + for shim in ("git", "cat", "awk"): + if (reg.fakebin / shim).exists(): + (reg.fakebin / shim).unlink() + assert mark.exists(), "the peer never raced the pull" + assert result.returncode == 5, (result.stdout, result.stderr) + if cut == "write": + assert "have been put back and proved" in result.stderr + assert "Its row for lane repoX-4 was not taken back, and this act HAS committed it" in result.stderr + else: + assert ("The register is as it was and this act HAS committed its row for lane repoX-4, locally and " + "unpushed: no row was removed, so that row is still in it and in that commit.") in result.stderr + assert {"copy": "could not be copied whole", + "awk pass": "the pass that removes 1 row(s)", + "line count": "removing 1 row(s) changed the line count by 2; refusing. The register", + "numstat": "the removal added or rewrote lines (numstat: -\t-); refusing. The register", + }[cut] in result.stderr + assert "from lanes/LANES.md and commit that before any other write in " in result.stderr + for archive_rows_words in ("the rows are in", "The archive may have been appended", "resolve that partial move"): + assert archive_rows_words not in result.stderr + assert "this act has committed nothing" not in result.stderr + assert "add row" in reg.git("log", "--format=%s", "origin/main..HEAD") + assert "`repoX-4`" not in reg.published("lanes/LANES.md") + + +#: What is set up before add-row runs (the delta review of #263, S3): each is a +#: way context reaches the take-back's diff, or a hunk where the row is not the +#: first line of it. +CONTEXT_CASES = [ + "GIT_DIFF_OPTS=-u3 in add-row's environment", + "diff.interHunkContext=3 beside an unpushed edit one row above the end", + "context the diff carries anyway, beside that edit", + "another lane's unpushed row just above", +] + + +@pytest.mark.parametrize("case", CONTEXT_CASES) +def test_122_a_take_back_finds_its_row_whatever_context_the_diff_carries(reg, tmp_path, case): + """The delta review of #263, S3: `-U0` alone does not keep context out of + the take-back's diff. A `GIT_DIFF_OPTS=-u3` overrides it, and a user's + `diff.interHunkContext` fuses a nearby change's hunk with the row's, the + lines between them as context. The parse counted only `+` lines, so the + row was put at the wrong line, the line check refused, nothing was taken + back, and the next ordinary write published the row add-row had refused (M1 + again). The diff now runs with neither, and the parse counts every line of + a hunk but a removed one, so context that arrives anyway (a shim, here) is + counted where it stands, and so is a second added row in the same hunk.""" + archive = a_good_archive(reg) + extra = {"LANES_LANE": "repoX-4"} + diff_log = tmp_path / "take-back-diff" + context = None + if case.startswith("GIT_DIFF_OPTS"): + extra["GIT_DIFF_OPTS"] = "-u3" + elif case.startswith("another lane"): + reg.write("lanes/LANES.md", reg.read("lanes/LANES.md") + + row("repoX-5", OLD_ID, "another lane's row, not pushed", "2026-10-19")) + reg.commit("another lane's row, not pushed") + else: + if case.startswith("diff.interHunkContext"): + (reg.home / ".gitconfig").write_text("[diff]\n\tinterHunkContext = 3\n", encoding="utf-8") + else: + context = 3 + reg.write("lanes/LANES.md", reg.read("lanes/LANES.md").replace( + "another repository's row 29 |", "another repository's row 29, edited here |")) + reg.commit("an edit one row above the end, not pushed") + mark = peer_pushes_during_pull(reg, tmp_path, { + ARCHIVE: archive + RESCAN_REFUSALS["a line the scan cannot take apart"][0]}, + diff_log=diff_log, diff_context=context) + result = reg.run("add-row", NEW_ROW.rstrip("\n"), **extra) + (reg.fakebin / "git").unlink() + assert mark.exists(), "the peer never raced the pull" + assert result.returncode == 1, (result.stdout, result.stderr) + after = reg.run("set-row-state", "repoX-2", "PAUSED · the next unrelated write", LANES_LANE="repoX-1") + assert after.returncode == 0, (after.stdout, after.stderr) + assert reg.published("lanes/LANES.md").count("`repoX-4`") == 0, result.stderr + assert "removed in a compensating LOCAL commit" in result.stderr + # How the one take-back diff was asked for: with no GIT_DIFF_OPTS and no + # inter-hunk context, whatever add-row's environment and config held. + calls = diff_log.read_text(encoding="utf-8").splitlines() + assert len(calls) == 1, calls + seen_opts, args = calls[0].split("\t", 1) + assert seen_opts == "", calls + assert " -c diff.interHunkContext=0 diff " in f" {args} ", calls + + +#: The ways git classes the register as binary (delta review 4 of #263, S4), +#: each set up before add-row runs. +BINARY_CASES = [ + "`lanes/LANES.md -diff` in .git/info/attributes", + "a published .gitattributes marking it binary", + "core.bigFileThreshold below the register's size", +] + + +@pytest.mark.parametrize("case", BINARY_CASES) +def test_122_a_take_back_reads_its_diff_as_text_whatever_classes_the_register_binary(reg, tmp_path, case): + """Delta review 4 of #263, S4: where git classes the register as binary, + the take-back's diff is the one line "Binary files … differ", with no hunk + to read. add-row said this checkout added no copy of the row, kept its + commit, and the next write published the row add-row had refused (M1 + again). The diff is now read as text, and under a `core.bigFileThreshold` + the removal's `numstat` proof counts lines as well, so each takes its row + back. The next write is an `append-line`: a `set-row-state` refuses under + such a threshold on its own `numstat` proof, as it does on main.""" + archive = a_good_archive(reg) + if case.startswith("`lanes/LANES.md -diff`"): + (reg.wip / ".git" / "info").mkdir(parents=True, exist_ok=True) + (reg.wip / ".git" / "info" / "attributes").write_text("lanes/LANES.md -diff\n", encoding="utf-8") + elif case.startswith("a published .gitattributes"): + reg.write(".gitattributes", "lanes/LANES.md binary\n") + reg.publish("every clone reads the register as binary") + else: + (reg.home / ".gitconfig").write_text("[core]\n\tbigFileThreshold = 200\n", encoding="utf-8") + assert (reg.wip / "lanes/LANES.md").stat().st_size > 200 + mark = peer_pushes_during_pull(reg, tmp_path, { + ARCHIVE: archive + RESCAN_REFUSALS["a line the scan cannot take apart"][0]}) + result = add_repo_x_4(reg) + (reg.fakebin / "git").unlink() + assert mark.exists(), "the peer never raced the pull" + assert result.returncode == 1, (result.stdout, result.stderr) + after = reg.run("append-line", "a plain line, the next unrelated write", LANES_LANE="repoX-1") + assert after.returncode == 0, (after.stdout, after.stderr) + assert reg.published("lanes/LANES.md").count("`repoX-4`") == 0, result.stderr + assert "removed in a compensating LOCAL commit" in result.stderr + + +#: Where the take-back's removal is written but cannot be committed (delta +#: review 4 of #263, N3), each with what `git status` then shows. +COMPENSATION_FAILS = { + "staging": ('if [ -e \'@MARK@\' ]; then case " $* " in *" add -- lanes/LANES.md "*)\n' + ' echo "test shim: add refused" >&2; exit 1 ;; esac; fi', " M lanes/LANES.md", + "but staging that removal failed, so it is only in the working tree, and add-row's " + "commit still holds the row."), + "commit": ('case " $* " in *" commit "*"cancel add-row"*)\n' + ' echo "test shim: commit refused" >&2; exit 1 ;; esac', "M lanes/LANES.md", + "and that removal is staged, but its compensating commit failed, so add-row's commit " + "still holds the row."), +} + + +@pytest.mark.parametrize("step", sorted(COMPENSATION_FAILS)) +def test_122_a_removal_that_cannot_be_committed_says_so(reg, tmp_path, step): + """Delta review 4 of #263, N3: where the take-back's `git add` or `git + commit` failed, add-row returned with nothing of its own said. The removal + was left in the working tree or staged under add-row's commit, and the next + write's commit carried it under that write's caption. It now says what is + staged, that add-row's commit still holds the row, and the command that + commits the removal, which is run here: then the next write publishes no + repoX-4, and origin's history has the removal under its own caption.""" + archive = a_good_archive(reg) + shell, status, said = COMPENSATION_FAILS[step] + mark = peer_pushes_during_pull(reg, tmp_path, { + ARCHIVE: archive + RESCAN_REFUSALS["a line the scan cannot take apart"][0]}, extra_shell=shell) + result = add_repo_x_4(reg) + (reg.fakebin / "git").unlink() + assert mark.exists(), "the peer never raced the pull" + assert result.returncode == 1, (result.stdout, result.stderr) + assert reg.dirty().splitlines() == [status] + assert "add row" in reg.git("log", "--format=%s", "origin/main..HEAD") + assert said in result.stderr + caption = "LANES(repoX-4@Eagle): cancel add-row whose retirement could not be checked after rebase" + fix = re.search(r"Commit the removal before any other write in (\S+), so that write does not carry it " + r"under its own caption: `(git -C (\S+) commit -m '([^']+)' -- lanes/LANES\.md)`", result.stderr) + assert fix, result.stderr + assert fix.group(1) == fix.group(3) and os.path.samefile(fix.group(1), reg.wip), fix.group(0) + assert fix.group(4) == caption, fix.group(0) + done = subprocess.run(["bash", "-c", fix.group(2)], env=reg.env, text=True, capture_output=True, check=False) + assert done.returncode == 0, (done.stdout, done.stderr) + after = reg.run("set-row-state", "repoX-2", "PAUSED · the next unrelated write", LANES_LANE="repoX-1") + assert after.returncode == 0, (after.stdout, after.stderr) + assert reg.published("lanes/LANES.md").count("`repoX-4`") == 0 + assert caption in reg.raw_git("-C", str(reg.origin), "log", "--format=%s", "main")