Refresh every fleet number, and stop typing them into the markup - #73
Closed
realmarcin wants to merge 7 commits into
Closed
realmarcin wants to merge 7 commits into
realmarcin wants to merge 7 commits into
Conversation
The 22 columns were in the order they were first written down, so the table said nothing by its shape: a vocabulary every Mech uses sat next to one only ProteinTraitsMech has. They now sort left to right by the number of Mechs that ground anything in the vocabulary, ties broken by the total records citing it across the fleet, then by name so the order is stable when a vocabulary appears in no records at all. PMID and DOI are pinned to the right end instead of sorted. Every Mech cites literature, so both would rank first on Mech count and crowd out the vocabularies that actually say what the fleet shares; they also have no record counts to rank by, since build_subsets.py skips citation prefixes. The result runs NCBITaxon and ChEBI at nine Mechs each, through GO at six and a five-Mech tier of UniProt, ENVO and UBERON, down to RHEA, MIBiG, NPAtlas and GTDB at one. The order is computed in build_data.py, so a corpus refresh re-derives it rather than leaving the columns stale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXHrBTSU8fU4Zg7HfaMKs9
Review of #60 found the list of prefixes that name a paper rather than a concept written down twice, in scripts that cannot see each other: build_subsets.py skipped them with a literal tuple so they get no record lists, and build_data.py declared its own copy to pin them to the right of the heatmap. Adding a third prefix to one would have left the other sorting it to the far left on Mech count with no record total to break the tie, which is the failure the pin exists to prevent. Both now import CITATION from roots.py, which already owned the other shared fact about the corpora. build_data.py also stops redeclaring ORDER, which roots.py has derived from RECORD_GLOBS since #40. fleet_data.json rebuilds byte-identical, so this changes no published number. Closes #61 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXHrBTSU8fU4Zg7HfaMKs9
Four changes to the fleet page, all asked for: The stat strip gains a fifth tile, 2,097 merged pull requests summed across the nine Mech repositories, and each Mech card gains a line of small print under its headline number carrying that Mech's reviewed records and merged PRs. Both come from scripts/fleet/mech_stats.py and are substituted into the markup by assemble_page.py, so a recount moves the page instead of quietly disagreeing with it. Whether a Mech tracks review at all is read from its LinkML schema rather than from the values its records happen to carry. Six declare a status slot whose enum permits REVIEWED, so a count of zero there is a real zero. MediaIngredientMech and CultureMech have a mapping_status, but its enum runs UNMAPPED to AMBIGUOUS and never reaches REVIEWED, so it grades mapping rather than review; CommunityMech has no such slot. Those three print only their PR count, because "nobody has reviewed one" and "this Mech does not track review" are different claims and the page should not make the second look like the first. HabitatMech's headline drops its own "684 reviewed", now that every card carries the same figure in the same place. The Shared vocabulary heatmap moves above the graph, from the template into the fragment, since the graph is what it explains. Its heading is written as raw HTML with the id kramdown would have generated, because markdown is not processed inside the fragment. Graph nodes are shaded as lit spheres: one highlight and one limb gradient in objectBoundingBox units, so a single pair serves every node, neither carries a hue, and both multiply over whichever Mech colour is underneath. The weighting is a theme token because it has to change between themes -- light-theme Mech colours are far darker, carrying their contrast against white, and a limb tuned for the dark theme turned them to mud. The overlays take no pointer events, so hovering still hits the node. The hub keeps its flat fill and reads as the centre rather than a tenth planet. The stat strip steps 5 -> 3 -> 2 columns on explicit breakpoints. auto-fit picks four between roughly 700px and 1010px, which leaves the fifth tile alone on a row and stretched across it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXHrBTSU8fU4Zg7HfaMKs9
The Mech card offered only a GitHub button while the other eight offered Browse, because NaturalProductMech had no published site when the page was written. It has one now: the repository root redirects to pages/index.html, which serves a browser over all 3,115 structures, and the Pages deployment reports built. Verified with a request to the live URL, not from the repository settings alone. The card gains its Browse button, the ladder link and the graph's site link stop pointing at GitHub, and index.md stops doing so too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXHrBTSU8fU4Zg7HfaMKs9
Five findings from the review of #63: The graph's accessible title said "the eight Mech knowledge bases". It has drawn nine since NaturalProductMech joined, and that title is the whole description a non-sighted reader gets of the graph. Closes #67. Three cards show a merged-PR count with no reviewed count beside it, which reads as an oversight rather than as the deliberate refusal to print a misleading zero that it is. The cards intro now names those three and says their schemas have no field that records review. Closes #65. mech_stats.py called its PR count "the closest thing the fleet has to a measure of how much human review the corpus has passed through". It is not: seeding runs, regeneration and dependency updates merge the same way curation does. The docstring now says it measures development activity, not how much of it was human. Closes #66. mech_stats.py is the only pipeline script that needs PyYAML, and the repository declares no Python dependencies, so a clean interpreter met a traceback. It now stops with the install command instead, the way roots.py handles a missing checkout. Closes #68. _fleet/README.md gains the new pipeline step, its two external requirements and its --no-prs flag, and stops saying NaturalProductMech publishes no site. Left filed: #64, that the card record totals date from the 2026-09-08 census while the reviewed counts are from today. Closing it means rerunning both scanning passes and changing the card numbers, and CellStructureMech's 57 on the page against 108 files matched today is too wide a gap to adopt without first working out which of the two the card means. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXHrBTSU8fU4Zg7HfaMKs9
The page was claiming 448,624 curated records against corpora that now hold 449,114, and the drift was worst where curation is busiest: CellStructureMech showed 57 records against 338, TraitMech 477 against 618. Reported as #64, where the CellStructureMech gap looked like it might be a definitional mismatch. It was not. The corpora simply grew that much in a week. Reran the whole pipeline. Records, the curated-records total, the ontologies-and-databases tile, merged pull requests, the reviewed counts and the per-Mech secondary stat are now placeholders that assemble_page.py fills from _fleet/data, and the build fails rather than shipping an unfilled one. Nothing about a count is typed into the markup any more, including the "as of" date, which now comes from when the census file was written rather than from a month someone remembered to update. The four secondary card stats are derived too, in a small per-Mech table in mech_stats.py: they have to be per-Mech, since a percentage grounded in ChEBI means nothing for a Mech that mints its own identifiers and CultureMech's figure counts a directory its records do not live in. All four had gone stale: TraitMech 353 -> 374 causal graphs, AntibioticMech 92% -> 91% ChEBI-grounded, MediaIngredientMech 91% -> 89% mapped, CultureMech 15,877 -> 15,878 normalized. Hand-written prose corrected to match, since it is prose and cannot be substituted: the four node detail panels that repeat these figures, the suite sentence on index.md, and TraitMech's record count on resources.md, which also gained a tenth trait category. Merged pull requests moved most of all, 2,097 to 2,735, because the count was eight days old and the Mechs merge fast. Closes #64 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXHrBTSU8fU4Zg7HfaMKs9
This was referenced Sep 16, 2026
Review of #73 found two defects in that PR. The heatmap note's "as of" date was read from the mtime of prefix_census.json. Git neither records nor restores mtimes, so a fresh clone stamps the file with the checkout time and the page would have claimed the corpora were counted on whatever day someone cloned it. Reproduced: the same commit printed 15 September in the working copy where the scan ran and 16 September in a clone made a day later. That is worse than the hardcoded "September 2026" it replaced, because it is confidently wrong rather than vague, and it would be wrong on every CI build. prefix_census.py now writes its run date into the file it produces, and assemble_page.py reads that. mech_stats.py counted the record files a second time to get the denominators for its percentages, while the cards printed the count from the census. Two scans of one corpus at two moments, so a card could show a percentage whose denominator was not the number next to it. It now takes the total from the census and stops with a message if the two disagree, which is the signal that the corpus moved between passes and the earlier ones need rerunning. Closes #74 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BXHrBTSU8fU4Zg7HfaMKs9
Contributor
Author
|
Adversarial review done, read-only, against What I checked beyond reading the diff:
Addressed here: #74, and it was a real defect in this PR. The "as of" date was read from Also in #74: Left filed: #75, that the graph's per-Mech detail panels still carry hand-typed figures. Four were stale and are corrected here, which is the point: nothing stops them going stale again. The remaining figures on ProteinTraitsMech, AntibioticMech, NaturalProductMech and CommunityMech are unverified. Those strings are prose with numbers woven through, so fixing them properly means either a derivation per figure or rewriting the panels so the prose carries none, and both are more than a refresh. |
realmarcin
force-pushed
the
feat/mech-pr-and-review-stats
branch
from
September 16, 2026 17:29
c912e98 to
e306ce7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #63, which is stacked on #60. Merge those two first and this retargets to
mainon its own.Closes #64. The page was stating 448,624 curated records against corpora that now hold 449,114, and the drift was worst where curation is busiest.
#64 wondered whether CellStructureMech's gap meant the card counted something other than the record glob. It did not. The corpus grew from 57 to 338 in a week, and both repositories had commits from today.
The durable part. Every count is now a placeholder that
assemble_page.pyfills from_fleet/data, and the build fails rather than shipping an unfilled one. That covers the card record numbers, the graph's node subtitles, the curated-records total, the ontologies-and-databases tile, merged pull requests, reviewed counts, each card's secondary stat, and the "as of" date, which now comes from the census file's own mtime instead of a month someone remembered to edit. Nothing about a count is typed into the markup any more.Secondary card stats are derived too, through a small per-Mech table in
mech_stats.py. They have to be per-Mech: a percentage grounded in ChEBI means nothing for a Mech that mints its own identifiers, and CultureMech's figure counts a directory its records do not live in. All four were stale.Prose corrected by hand, since prose cannot be substituted: the four graph detail panels that repeat these figures, including HabitatMech's "174 attested by two or more sources" which is now 215, the suite sentence on
index.md, and TraitMech onresources.md, which also gained a tenth trait category.Merged pull requests moved most of all, 2,097 to 2,735, because that count was eight days old.
Other things the rerun shifted on their own: the strip's cited-vocabulary tile went 43 to 45, and RHEA moved from one Mech to two, which #60's ordering picked up and moved left.
Verified in Chrome against a local render: the strip, the nine cards and the nine graph node subtitles all agree, the heatmap has no horizontal overflow, and no placeholder survives into
mechs.md.🤖 Generated with Claude Code
https://claude.ai/code/session_01BXHrBTSU8fU4Zg7HfaMKs9