Skip to content

Report exit 5, not 4, when the content was never obtained - #6

Merged
jehrr merged 1 commit into
mainfrom
unify-transport-exit-code
Sep 21, 2026
Merged

jehrr merged 1 commit into
mainfrom
unify-transport-exit-code

Conversation

@jehrr

@jehrr jehrr commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

A run that never obtained its pages used to report exit 4 — the code
this family documents as "ran fine, found nothing". So a dead proxy and an
empty catalogue were one value to anything branching on the exit code, which
is what this family says exit codes are for.

That much was already known. What a family-wide measurement on 2026-09-21
found is that three different repos had fixed it three different ways,
none aware of the others. Measured by importing each repo's output_writer
and CALLING finish_run([], …, stop_reason="page_load_timeout"):

answer repos
exit 4 20 — unfixed
exit 5 2 — farfetch, amazon
exit 6 7 — fixed, with written reasoning, using EXIT_PARTIAL
neither 1 — stockx, not built on this core

This unifies all 30 on exit 5, and takes the better half from each camp.

The rule comes from the exit-6 camp

They keyed on if not complete. The exit-5 camp keyed on a named list of
stop reasons. A list cannot cover a reason nobody has added to it yet, so
a new stop_reason falls silently through to "the catalogue is empty" — the
exact defect all three camps set out to fix. So the list was the hole, and
the repos that shipped the fix first are the ones being simplified here.

The code comes from the exit-5 camp

EXIT_PARTIAL (6) means "some rows were gathered and the output is
incomplete". A run holding nothing writes no output at all, so a consumer
that reads the file on a 6 finds either nothing or the previous run's
good data — which save deliberately does not overwrite. Exit 5 promises no
file, and this family's contract already reserves 5 for a transport failure
(scraper_api_client has used it for a remote API error since it was
written). No new code, and no per-repo table for a caller driving more than
one of these scrapers.

A run that gathered rows and THEN failed is still exit 6, unchanged.

Verified by calling, not by reading

Every repo's finish_run was invoked after the patch and required to return
5 before the change was kept, and each repo's own suite had to stay green.
Where a repo's suite pinned the old answer by name, that check was
rewritten in the same commit — a test asserting exit 6 is that repo
stating its position, and changing the code while leaving the test would
make the suite red and read like a regression rather than a decision.

🤖 Generated with Claude Code

…inish

Exit 4 is a claim about the CATALOGUE — "we asked, and the answer was
nothing" — so handing it to a run that never reached the site tells a
pipeline the listing is empty when nothing was read at all.

Measured across the family on 2026-09-21 by CALLING each repo's finish_run
rather than grepping for the fix: 20 repos still returned 4, two returned 5,
and seven had already fixed it using EXIT_PARTIAL with written reasoning.
Three camps, none aware of the other two.

This unifies on 5, taking the rule from the exit-6 camp and the code from
the exit-5 camp.

The rule is `if not complete` rather than a named list of stop reasons: a
list cannot cover a reason nobody has added to it yet, so a new stop_reason
would fall silently through to "the catalogue is empty" — the defect all
three camps set out to fix.

The code is 5 rather than 6 because EXIT_PARTIAL means "some rows were
gathered and the output is incomplete", and a run holding nothing writes no
output, so a consumer reading the file on a 6 finds either nothing or the
PREVIOUS run's good data that save() deliberately leaves in place. 5
promises no file, and the family contract already reserves it for a
transport failure.

Rows gathered and THEN a failure is still exit 6, unchanged.

Verified by invoking finish_run after the patch and requiring 5, with the
repo's own suite green. Checks that pinned the old answer were rewritten in
the same commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jehrr
jehrr merged commit 0eb3020 into main Sep 21, 2026
7 checks passed
@jehrr
jehrr deleted the unify-transport-exit-code branch September 21, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants