Skip to content

Derived index follow-ups from #164: one read snapshot for export; holds keyed by canonical repository; history --index stderr count #166

Open

Description

@brettheap

Follow-ups to #164, the derived index for Amendment 14 act 2 (#160). Copilot review is capped at two rounds per PR. Everything raised after that cap, or found in the final checks and not taken there, is filed here as one issue. It is kept as one issue across the rounds that followed.

1. export should read one snapshot (Copilot, review of 50594dc, lanes-index cmd_export)

cmd_export reads provenance, then source_files, lanes, register_lines, lane_aliases and log_lines, each as a separate statement:

  • On SQLite, each statement is its own read snapshot.
  • On Postgres, each rows() call is a separate psql session.

A sync that commits between two of those reads can make export write rows from two commits, while meta names the first sha. The --index read would then show a torn image until the next read.

The window is the export itself, a fraction of a second, and only a read that decides nothing sees it. It is still a break of the "one commit's image" the read: index … at register@<sha12> line asserts.

Suggested fix:

  • SQLite: BEGIN on the read-only connection before the provenance read and COMMIT after the last table. In rollback-journal mode, that holds the SHARED lock, so a writer waits. In WAL mode, it is a snapshot.
  • Postgres: send every read in ONE psql call, inside BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY; … COMMIT;, through _query_many.
  • Test: an export whose reads are interleaved with a sync's commit, for example by patching rows() to run a sync after the first table. The test checks that the export comes out as either the old commit's image or the new one, never a mix.

2. holds keys an object as written; who folds the repository spelling

The holds view computes holders_of's answer per object key as the log line wrote it. who <object> first folds its argument through lanes/repos.tsv (R20, canon_object) and then reads only the lines carrying the canonical spelling.

I compared the view with who, read-only on the live register (5,696 records at da4c9e06dd7b), for all 130 held objects plus 40 free ones. There was exactly one difference:

  • opensoft/codexFactory#492
  • Its lines 89 and 90 in openxfactory-4.md are LANDING and WITHDRAWN under the old spelling.
  • Lines 91 and 92 are LANDING and LANDED under codeXfactory/codexFactory#492.
  • The view lists the old key as held by openxfactory-4. who says FREE.

The manual's holds row documents this as of 50594dc. Nothing in #164 reads the view.

The fix needs lanes/repos.tsv indexed (a repo_aliases table, schema 2 or an additive change to schema 1 before anything consumes it). The view, or a canonical-object column on log_lines, then folds owner/repo#n the way canon_object does. The open question is whether a line under a legacy spelling should count toward the canonical object's holds at all. Today who ignores such lines, so folding them in would make the view and who disagree in the other direction. Whatever is chosen, who and the view should give the same answer.

3. history <lane> --index drops LOG_AWK's closing unreadable-line count on stderr (Copilot, review of 4bfa584, lanes-edit.sh lane_log_events)

In the index branch, lane_log_events keeps from the export's warn file only the lines prefixed unreadable: <that log>:. The source path runs LOG_AWK over the one file. LOG_AWK ends with N unreadable line(s) above — an append-only log is never rewritten, … whenever N > 0, and the index path never prints that line.

So for a lane whose log has unreadable lines, history <lane> --index prints the same STDOUT as the source read, which is the parity the tests hold it to. Its STDERR, however, lacks that one closing line.

Suggested fix: count the lines the filter keeps and print LOG_AWK's own sentence when the count is above zero. Extend test_the_index_read_is_byte_identical_to_the_source_read, or add a case, so that stderr is compared too, with the read: line removed, for history repoA-1. The fixture's NASTY_LOG has nine unreadable lines.

4. Make the fake psql decode \echo instead of hard-coding the marker byte (Copilot, review of 20e23a4)

Copilot claimed that \echo '\036LANES-INDEX-NEXT' prints the four characters \036 instead of byte 0x1E. That would make every real Postgres store look wiped. The claim is wrong. psql's exec_command_echo reads OT_NORMAL arguments, and psqlscanslash.l decodes xeoctesc in single-quoted text through strtol(..., 8) (lines 106 and 337–341 on master). The manual says so too.

The suite still cannot show this. FAKE_PSQL in tests/test_lanes_index.py prints \x1eLANES-INDEX-NEXT for any \echo line, whatever its argument.

Suggested fix: have the fake decode the argument as psql's lexer does (\n \t \b \r \f, octal, hex, ''). The marker the indexer writes is then what the parser reads back.

Refs #160, #164, brettheap/new-workstation#48

Lane: openRepoTools-3

Activity

  1. changed the title [-]Derived index follow-ups from #164: one read snapshot for export; holds keyed by the canonical repository spelling[/-] [+]Derived index follow-ups from #164: one read snapshot for export; holds keyed by canonical repository; history --index stderr count[/+] on Oct 5, 2026
  2. brettheap commented on Oct 5, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoTools-3, session 83619980-06bb-4469-ae73-2a8c5e61981a@Eagle, 2026-10-05T15:32:46Z, for #166

    Logged in lanes/log/openRepoTools-3.md at 94a25fdbe407700241bcf4bc1da8fa05ff4ce601.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoTools#166:
       none
    2. `gh pr list --repo opensoft/openRepoTools --state all --search "#166"`, kept where the row or its body names `#166`:
    164	Amendment 14 act 2: lanes-index derives a non-gating index of the lane register, nudged after each landed write and read only behind --index; history reads across lanes	feat/amendment-14-derived-index	OPEN
    3. `git ls-remote --heads git@github.com:opensoft/openRepoTools` branches naming `166` as a whole token:
       none
    
  3. brettheap commented on Oct 11, 2026

    @brettheap
    ContributorAuthor

    TAKEOVERopenRepoTools-3 — lane openRepoTools-4, session e545e636-93fd-4b80-81b1-48177346b887@Eagle, 2026-10-11T16:51:33Z, for #166

    Logged in lanes/log/openRepoTools-4.md at 9fe787cbdc86708efc614d842389f2e65f89dcc3 (takeover of the stale claim held by lane openRepoTools-3).

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoTools#166:
    6:CLAIMED — lane openRepoTools-3, session 83619980-06bb-4469-ae73-2a8c5e61981a@Eagle, 2026-10-05T15:32:46Z, for opensoft/openRepoTools#166
    13:1. existing `CLAIMED —` comments on opensoft/openRepoTools#166:
    2. `gh pr list --repo opensoft/openRepoTools --state all --search "#166"`, kept where the row or its body names `#166`:
    164	Amendment 14 act 2: lanes-index derives a non-gating index of the lane register, nudged after each landed write and read only behind --index; history reads across lanes	feat/amendment-14-derived-index	MERGED
    3. `git ls-remote --heads git@github.com:opensoft/openRepoTools` branches naming `166` as a whole token:
       none
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions