diff --git a/docs/README-lanes.md b/docs/README-lanes.md index 6f36e492..3d5b8bf0 100644 --- a/docs/README-lanes.md +++ b/docs/README-lanes.md @@ -1,7 +1,7 @@ # `lanes/LANES.md` — how the estate lane register is stored, and the commands that write it **THE TOOLING LIVES HERE; THE DATA LIVES IN YOUR WORKSPACE REPOSITORY.** -Amendment 9, 2026-09-13. This manual and the four commands it describes — +Amendment 9, 2026-09-13. This manual and the commands it then described — `lanes-edit.sh`, `lane-start`, `lane-end` and `link-estates` — came to `opensoft/openRepoTools` from `opensoft/brett-wip` with their history, because a `-wip` is a person's DATA and its own rules file says so: "No secrets @@ -73,7 +73,7 @@ workstation**. | the object logs | `/lanes/log/.md` — one per lane (Amendment 7) | | the alias table | two layers: `repos.tsv` shipped by openRepoTools and installed beside the commands, then `/lanes/repos.tsv` where you keep an override (Amendment 9(b)) | | who places the symlinks | `link-estates`, an installed command. It places the handoffs links, the register link and the `lanes-edit.sh` link — **and no longer the two `~/.local/bin` ones**, which `--install` owns | -| who installs all of it | `openRepoTools --install` (thirteen files, idempotent, all-or-nothing) | +| who installs all of it | `openRepoTools --install` (every command and the alias table, idempotent, all-or-nothing) | | the lane alias table | `/lanes/aliases.tsv` — ``, written by `lane-rename` and by nothing else, resolved by every reader that takes a lane name (Amendment 16(e)) | Why `main` of the workspace repository and not the aggregation's: the @@ -286,7 +286,7 @@ a row exists. canonical** (Amendment 15, in force 2026-09-14T00:07:18Z). `lane-start openxfactory 2` and `lane-start openXfactory 2` are the same lane: every lookup of a name — the row, the object log, the tmux window, the session name, -`LANES_LANE`, and every `` argument of the four commands — compares +`LANES_LANE`, and every `` argument of the commands here — compares case-insensitively, and the row's own spelling is what the window is renamed to, what the launch is `--name`d with, what the log file is called, and what every line and stamp writes. A name typed in another case RESOLVES to it and is never @@ -298,9 +298,10 @@ register already holds two rows differing only by case, **every writer refuses, naming both**, until they are merged by hand — 15(d): the newer row's session id(s) appended to the older row's session cell, in order, and the newer row removed in the SAME commit, whose message names both spellings. `lanes-edit.sh -canon-lane ` is that resolver as a read, and the one all four commands -share: **0** with the canonical spelling (the typed one where no row matches, so -`add-row` can still create a lane), **2** for the pair, **64** usage — and that +canon-lane ` is that resolver as a read, and the one the commands share +rather than each keeping its own: **0** with the canonical spelling (the typed +one where no row matches, so `add-row` can still create a lane), **2** for the +pair, **64** usage — and that 2 is RELAYED by `lane-start`, `lane-end` and `lane` rather than carried past, because the pair may be on `origin/` while this checkout still holds one row. `add-row` asks the published register as well as this copy of it for the @@ -2789,7 +2790,7 @@ Historical `ready` intents do not control an ordinary resume. The pane is respawned with `lane-handoff --supervise --lane --operation `, **by absolute path**, because a respawned pane's `PATH` is whatever the person's shell profile makes of it. It is a MODE of a command that is already -installed and not a thirteenth word: Amendment 18 Addendum 2 (i-8) is explicit +installed and not one more word: Amendment 18 Addendum 2 (i-8) is explicit that *"no word is kept on `PATH` for it alone"*, which is the ground on which `restart` was taken off `PATH` in the first place. @@ -4128,8 +4129,8 @@ link-estates # repoints ~/projects/xFac ``` **You are not asked to remember it: `--install` refuses** (A9 Addendum 4, -R-A9-12). In its planning phase, before any of the twenty-seven artifacts is placed, -it walks all thirteen targets and dies naming every one that is not a regular file, +R-A9-12). In its planning phase, before any of its artifacts is placed, it walks +every target in the bin directory and dies naming every one that is not a regular file, what it is, and the one `rm` that clears them. `cp` FOLLOWS A SYMLINK, so an install over these would leave the two commands UNINSTALLED — the targets stay links — and would write the post-move bytes into `opensoft/brett-wip`'s working diff --git a/lane b/lane index 5105fc1c..65783a88 100755 --- a/lane +++ b/lane @@ -1218,9 +1218,9 @@ if [ -n "$name" ]; then # name is one name under any case"* (ratified 2026-09-14T00:07:18Z verbatim # "Ratify as drafted"), and `lanes-edit.sh canon-lane` is the ONE resolver # every command that takes a `` shares — for the reason clause (h) gives - # every other read here: four implementations of *which row is this name's* is - # how the four come to disagree about it. This is `restart`'s own call, moved - # here with the act under Amendment 18 Addendum 2. + # every other read here: an implementation of *which row is this name's* in + # each command is how they come to disagree about it. This is `restart`'s own + # call, moved here with the act under Amendment 18 Addendum 2. # # `2` IS TWO ANSWERS AND ONE OF THEM IS A REFUSAL. It is what a helper # predating Amendment 15 spends on an unknown subcommand — silent, expected, diff --git a/lane-end b/lane-end index cde8c280..b9cb7c30 100755 --- a/lane-end +++ b/lane-end @@ -858,10 +858,10 @@ if [ -z "${LANES_FILE:-}" ]; then die "$(lanes_workspace_why)" 1; fi # ------------------------- AMENDMENT 15: THE ROW'S SPELLING IS THE LANE'S NAME # # **A lane name is one name under any case** (ratified 2026-09-14T00:07:18Z), -# and `lanes-edit.sh canon-lane` is the ONE resolver all four commands share — -# for the reason Amendment 11 clause (h) gives every other read here: four -# implementations of "which row is this name's" is how the four would come to -# disagree about it. +# and `lanes-edit.sh canon-lane` is the ONE resolver the commands share rather +# than each keeping its own — for the reason Amendment 11 clause (h) gives every +# other read here: an implementation of "which row is this name's" in each +# command is how they would come to disagree about it. # # `2` IS TWO ANSWERS AND ONE OF THEM IS A REFUSAL (Copilot round 4 on # openRepoTools#41). It is what a helper predating this amendment spends on an diff --git a/lane-start b/lane-start index 810a5601..8aedb0c8 100755 --- a/lane-start +++ b/lane-start @@ -1066,8 +1066,8 @@ if [ -z "${LANES_FILE:-}" ]; then die "$(lanes_workspace_why)" 1; fi # # ONE RESOLVER, SHARED. `lanes-edit.sh canon-lane` is the same read `lane-end`, # `lanes` and `restart` make, for the reason clause (h) gives every other read -# in this toolset: four implementations of "which row is this name's" is how the -# four would come to disagree about it. +# in this toolset: an implementation of "which row is this name's" in each +# command is how they would come to disagree about it. # # 2 IS TWO ANSWERS AND THE REFUSAL IS THE ONE THAT MATTERS (Copilot round 4 on # openRepoTools#41). It is the code a helper predating this amendment spends on diff --git a/lanes-edit.sh b/lanes-edit.sh index b7232eca..8897dbc0 100755 --- a/lanes-edit.sh +++ b/lanes-edit.sh @@ -16242,9 +16242,10 @@ EOF # AMENDMENT 15 — THE RESOLVER, EXPOSED, because `lane-start`, `lane-end`, # `lanes` and `restart` all take a `` argument and the amendment gives - # all four the SAME resolver. Four implementations of "which row is this - # name's" is how the four would come to disagree about it — which is the whole - # reason clause (h) turned `window-lane` and `session-lane` into subcommands. + # them all the SAME resolver. An implementation of "which row is this name's" + # in each command is how they would come to disagree about it — which is the + # whole reason clause (h) turned `window-lane` and `session-lane` into + # subcommands. # # Read-only, out of `origin/` like every other state read (R19). # 0 the canonical spelling — the row's own where a row matches ignoring diff --git a/tests/conftest.py b/tests/conftest.py index 2d8a2410..83411240 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -592,7 +592,7 @@ def required_dependency(): #: but `park`, `resume` and `openRepoTools` are bash scripts that run `make` in #: a root which execs a shebang script, and a shebang means nothing to #: CreateProcess. Skipping is honest because there is no Windows twin to run -#: instead: these three files are bash, and on Windows the way in is WSL2, +#: instead for any shipped bash file, and on Windows the way in is WSL2, #: exactly as it is for openRepoShape's own `setup.sh`. What the Windows job #: still proves is the checkout and the hygiene tests, which need no shell. #: A shared `skipif` object rather than a named marker because there is no diff --git a/tests/test_lane_helpers.sh b/tests/test_lane_helpers.sh index b06a7cdd..b13adcc0 100755 --- a/tests/test_lane_helpers.sh +++ b/tests/test_lane_helpers.sh @@ -8831,7 +8831,7 @@ rm -f "$sessions_dir/a16i.json" # ============ Copilot round 4 on openRepoTools#81 ============================ # ---- THE ALIAS-READ REFUSAL HAS ITS OWN EXIT CODE, because 2 already carries -# two meanings to the four commands that read `canon-lane`. +# two meanings to the commands that read `canon-lane`. ren_unread3="$WIP/lanes/aliases-unread3.tsv" printf 'someLane-1\tsomeLane-2\t2026-09-15T00:00:00Z\n' > "$ren_unread3" chmod 0000 "$ren_unread3" diff --git a/tests/test_openrepotools_command.py b/tests/test_openrepotools_command.py index 0bdc7683..b5acf8ae 100644 --- a/tests/test_openrepotools_command.py +++ b/tests/test_openrepotools_command.py @@ -142,7 +142,7 @@ #: reason `AGENTS.md` gives for the submodule SKIP. #: #: The tests NOT marked are the ones that never reach the merge: the refusals, -#: `--help`, `--version`, and `test_install_places_all_nine_or_none`, which is +#: `--help`, `--version`, and `test_install_places_all_or_none`, which is #: refused at `collect_commands` before `plan_hook_merge` is called at all. #: `test_without_jq_it_refuses_and_places_nothing` is the positive case for the #: same fact and is deliberately unmarked. @@ -392,7 +392,7 @@ def test_install_refuses_a_second_argument_and_installs_nothing(tmp_path): @NEEDS_JQ def test_install_writes_an_executable_copy(tmp_path): - """All NINE files, each 755 and byte-identical to this checkout's. + """Every `INSTALLED` file, each 755 and byte-identical to this checkout's. A `park` that is not executable is not a command, and a `park` that is a near-copy is a command whose refusals nobody reviewed — so the bytes are @@ -2256,7 +2256,7 @@ def offline_github(tmp_path): return fake_github(tmp_path, FETCHED) -def test_install_places_all_nine_or_none(tmp_path): +def test_install_places_all_or_none(tmp_path): """ALL IN HAND BEFORE ANY IS PLACED (openRepoShape #82, F10 of the review on its #83). One file at a time, dying on the first fetch that failed, leaves a person with a NEW `openRepoTools` and no `park` — a half-install diff --git a/tests/test_repo_hygiene.py b/tests/test_repo_hygiene.py index 54a79acc..9f93a837 100644 --- a/tests/test_repo_hygiene.py +++ b/tests/test_repo_hygiene.py @@ -1089,8 +1089,8 @@ def test_the_readme_carries_the_two_line_onboarding_chain(): def test_no_document_offers_a_windows_powershell_twin(): """NATIVE WINDOWS PARKS NOTHING, and saying otherwise is worse than saying - nothing. All three files here are bash; there is no `Invoke-WebRequest` - two-liner and no PowerShell twin to point at, so on Windows the way in is + nothing. No command here has a PowerShell twin, and there is no + `Invoke-WebRequest` two-liner to point at, so on Windows the way in is WSL2 — which both documents say, once each.""" for name in ("README.md", "AGENTS.md"): text = doc(name).read_text(encoding="utf-8") @@ -3111,7 +3111,7 @@ def test_the_root_carries_the_line_ending_rule(): place against the bytes already there with `cmp`, and its own tests compare an installed copy to this checkout's file byte for byte. Git for Windows' installer default is `core.autocrlf=true`; a checkout made under it turns - every `\\n` in these four bash files into `\\r\\n`, and the comparison is + every `\\n` in the shipped bash files into `\\r\\n`, and the comparison is then about line endings rather than about content. The rule normalises text to LF in the object store on check-in and checks it out as LF on every platform, whatever `core.autocrlf` says — so nobody has to be told a git @@ -3174,35 +3174,142 @@ def test_the_python_files_compile(): } -def _shell_tokens(text: str) -> list[str]: - """Split one shell command into tokens, keeping a quoted run together. +#: THE TOKENS THAT END ONE COMMAND ON A LINE AND BEGIN THE NEXT, as +#: `_shell_tokens` hands them back: each is a token of its own, cut out of +#: the line only where the shell reads it as syntax, and never out of a +#: quoted run. +_COMMAND_SEPARATORS = frozenset({"|", ";", "&", "(", ")", "$(", "`"}) + + +def _shell_tokens(text: str, opened: str = "") -> tuple[list[str], bool]: + """Split one line of shell into tokens, keeping a quoted run together and + handing every command separator back as a token of its own. Returns the + tokens, and whether the line ENDS inside a quoted run. Deliberately small: it expands nothing and does not care what a token - MEANS. All it has to answer is "is this token exactly `--`, and did an - operand come before it". + MEANS. All it has to answer is "is this token exactly `--`, did an operand + come before it, and was that in the same command". + + `opened` starts the line inside a quoted run of that kind, `'` or `"`: the + second reading `_commands_on` gives a line that may close a quote an + earlier line opened. + + THE LINE IS TOKENIZED BEFORE IT IS CUT, AND NOT THE OTHER WAY ROUND + (opensoft/openRepoTools#54). The cut used to be a bare regular-expression + split of the raw text at every `|`, `;`, `&`, `(`, `)`, `$(` and backtick, + and a pattern is text. `grep -v 'not fetching; reading origin/' -- "$f"` + came apart at the `;` inside its own quotes, neither half held both the + operand and the `--` after it, and the check called a real BSD failure + clean — in a shipped command, the same line #53 had just fixed in a skill. + A quoted message that happened to hold `; grep x -- y` was the same cut + the other way: a finding in a string nothing runs. + + So a separator is one only OUTSIDE a quoted run, and only where the shell + reads it as one: `|`, `;`, `&`, `(` and `)` outside every quote, and the + two that open a nested command, `$(` and a backtick, inside a + double-quoted run as well — `out="$(sed -n 'p' -- "$f")"` is a command + inside an assignment, and its quotes start afresh, so the `$(` and the + `)` that closes it are tracked to know where the outer double quote + resumes. A single-quoted run holds no syntax at all, and a backslash makes + the next character plain text either way. """ tokens: list[str] = [] current: list[str] = [] - quote, started = "", False - for ch in text: - if quote: + started = False + # What encloses the character being read, innermost last: "'" or '"' for a + # quoted run, "$(", "(" or "`" for a command nested inside the line. + stack: list[str] = [opened] if opened else [] + + def end_token() -> None: + nonlocal current, started + if started: + tokens.append("".join(current)) + current, started = [], False + + index = 0 + while index < len(text): + ch = text[index] + inside = stack[-1] if stack else "" + step = 1 + if inside == "'": current.append(ch) - if ch == quote: - quote = "" - continue - if ch in "'\"": - quote, started = ch, True + if ch == "'": + stack.pop() + elif ch == "\\": + current.append(text[index:index + 2]) + started, step = True, 2 + elif inside == '"' and ch == '"': + current.append(ch) + stack.pop() + elif ch == "`" and inside == "`": + end_token() + tokens.append(ch) + stack.pop() + elif ch == "`" or text.startswith("$(", index): + opener = "`" if ch == "`" else "$(" + end_token() + tokens.append(opener) + stack.append(opener) + step = len(opener) + elif inside == '"': current.append(ch) + elif ch in "'\"": + stack.append(ch) + current.append(ch) + started = True elif ch.isspace(): - if started: - tokens.append("".join(current)) - current, started = [], False + end_token() + elif ch == "(": + end_token() + tokens.append(ch) + stack.append(ch) + elif ch == ")": + end_token() + tokens.append(ch) + if inside in ("$(", "("): + stack.pop() + elif ch in "|;&": + end_token() + tokens.append(ch) else: started = True current.append(ch) - if started: - tokens.append("".join(current)) - return tokens + index += step + end_token() + return tokens, bool(stack) and stack[-1] in ("'", '"') + + +def _commands_on(line: str) -> list[list[str]]: + """The tokens of `line`, cut into one list per command at every separator + token — so a quoted pattern stays whole in the command that passes it. + + A QUOTE CAN SPAN LINES, AND THIS READS ONE LINE AT A TIME. A line that + ends inside a quoted run is the first line of such a quote or its last, + and the line alone cannot say which. Read from outside every quote, a LAST + line's closing quote opens one instead, and the command after it goes into + a single token — `}' | LC_ALL=C sort | awk -v s="$US" '{ … }')"`, the + tail of a multi-line awk program in `lanes-edit.sh`, would lose its second + `awk` that way. So such a line is read again from inside a single-quoted + run and from inside a double-quoted one, and the commands of every reading + are checked; a line that closes every quote it opens is read once. + """ + def cut(tokens: list[str]) -> list[list[str]]: + commands: list[list[str]] = [[]] + for token in tokens: + if token in _COMMAND_SEPARATORS: + commands.append([]) + else: + commands[-1].append(token) + return commands + + tokens, ends_quoted = _shell_tokens(line) + commands = cut(tokens) + if ends_quoted: + for quote in ("'", '"'): + commands += [command + for command in cut(_shell_tokens(line, quote)[0]) + if command not in commands] + return commands def _end_of_options_after_an_operand(text: str) -> list[str]: @@ -3225,8 +3332,7 @@ def _end_of_options_after_an_operand(text: str) -> list[str]: for line in text.splitlines(): if line.lstrip().startswith("#"): continue - for piece in re.split(r"[|;&()]|\$\(|`", line): - tokens = _shell_tokens(piece) + for tokens in _commands_on(line): for start, token in enumerate(tokens): tool = token.rsplit("/", 1)[-1] if tool not in _OPTIONS_WITH_ARGUMENTS: @@ -3274,6 +3380,74 @@ def test_no_shipped_bash_ends_its_options_after_an_operand(name): f"script behind `-e` instead:\n " + "\n ".join(bad)) +#: LINES THE END-OF-OPTIONS RULE MUST REFUSE, each with a separator inside a +#: quoted run (opensoft/openRepoTools#54). The cut over the raw text took +#: every one of them apart at that separator and called it clean. Each row is +#: (why, line). +LATE_END_OF_OPTIONS_REFUSED = [ + ("#54's own line, a semicolon inside the pattern", + "grep -v 'not fetching; reading origin/' -- \"$local_err_file\" >&2 || :"), + ("a pipe inside an alternation", + "grep -E 'PARKED|RESUMED' -- \"$log\""), + ("an ampersand inside a sed replacement", + "sed -n 's/^lane: /[&]/p' -- \"$file\""), + ("parentheses inside a double-quoted pattern", + 'grep -n "^cut_to_line()" -- "$f"'), + ("a semicolon inside an awk program", + "awk '{ n++; print n }' -- \"$f\""), + ("a backtick inside a single-quoted pattern", + "grep -F 'run `lane`' -- \"$f\""), + ("a nested command inside double quotes, its pattern quoted afresh", + "out=\"$(grep 'a;b' -- \"$f\")\""), + ("the last line of a multi-line awk program, then a second call", + "}' | LC_ALL=C sort | awk '{ print; }' -- \"$f\")\""), +] + +#: LINES IT MUST ACCEPT: a separator inside a quoted run that the raw cut +#: turned into a call nothing makes. Each row is (why, line). +LATE_END_OF_OPTIONS_ACCEPTED = [ + ("a message that quotes the shape, after a semicolon", + "note \"the stderr filter is gone; grep 'pattern' -- file was its shape\""), + ("a single-quoted command line, after a double ampersand", + "printf '%s\\n' 'cd \"$dir\" && grep pattern -- file'"), + ("a single-quoted usage line, after a pipe", + "usage='lanes | grep \"x\" -- file'"), + ("parentheses inside a double-quoted message", + "echo \"(sed 's/x/y/' -- f)\""), + ("backticks inside a single-quoted message", + "printf '%s\\n' 'run `sed s/a/b/ -- f` by hand'"), + ("a correct call whose -e pattern holds a semicolon and a call", + "grep -v -e 'x; sed s/a/b/ -- y' -- \"$f\""), +] + + +@pytest.mark.parametrize("why,line", LATE_END_OF_OPTIONS_REFUSED, + ids=[why for why, _ in LATE_END_OF_OPTIONS_REFUSED]) +def test_the_end_of_options_rule_sees_a_call_whose_quotes_hold_a_separator( + why, line): + """RED: a quoted pattern, program or nested command keeps its separators + to itself (opensoft/openRepoTools#54). `lanes` shipped the first row's + line while this rule called it clean — the `;` inside the pattern cut the + call in two before the tokenizer saw its `--` — so every row here is one + the raw-text cut let through, and the line is tokenized before it is cut. + """ + assert _end_of_options_after_an_operand(line), ( + f"the rule let this through ({why}): {line!r}") + + +@pytest.mark.parametrize("why,line", LATE_END_OF_OPTIONS_ACCEPTED, + ids=[why for why, _ in LATE_END_OF_OPTIONS_ACCEPTED]) +def test_the_end_of_options_rule_finds_no_call_inside_a_quoted_string( + why, line): + """GREEN: text inside quotes is a string, not a command, whatever + separators it holds. The raw-text cut began a new command at each of them, + so the words after one read as a call with its `--` too late — a finding + in a line nothing runs, which teaches a reader to stop trusting the rule. + """ + assert _end_of_options_after_an_operand(line) == [], ( + f"the rule refused this ({why}): {line!r}") + + @pytest.mark.parametrize("name", ALL_BASH) def test_no_shipped_bash_leaves_a_variable_name_to_bash_3_2s_locale(name): """`bash -n` PARSES THIS AND macOS DIES ON IT (A9 Addendum 4, R-A9-11). @@ -3349,11 +3523,11 @@ def test_no_shipped_bash_opens_a_case_inside_a_command_substitution(name): not' ;; esac)` against `did not` - twice, and green on every GNU runner. A `bash -n` ON A GNU RUNNER PARSES IT, which is why this is a text rule - rather than a parse gate: all twelve files already pass `bash -n` in CI, - and the macOS job is the only place the defect exists. The fix is one line - each time - hoist the `case` above the assertion and read the variable it - sets - so the rule is held for every bash file this repository ships - rather than for the two that were found. + rather than a parse gate: every file in `ALL_BASH` already passes `bash -n` + under the Linux bash CI runs, and bash 3.2 is the only place the defect + exists. The fix is one line each time - hoist the `case` above the + assertion and read the variable it sets - so the rule is held for every + bash file this repository ships rather than for the two that were found. Only the same-line form is matched. A `case` inside a multi-line `$(` ... `)` is the same defect and wants the same hoist, but a line-level regex is