Repository navigation
Resolve the base image at build time instead of pinning a version - #4
Merged
Merged
Conversation
The template hard-coded a release tag, so the repo carried a version
somebody had to remember to bump, and every day after a release it
pointed at something stale.
Packer has a built-in http data source, and sfc_base_image already
publishes a release index, so the template can just ask: latest.json
for the current release, <tag>/manifest.json for a specific one. No
version in the repo.
The index lists each image's sha256 beside its URL, which turns out to
be worth more than dropping the pin. iso_checksum now comes from the
same document as iso_url, so the two cannot disagree, and the
file:SHA256SUMS indirection is gone along with the whole
base_image_checksum variable and its "=none" escape hatch -- which
existed only because a pinned tag could predate the checksum file.
Pinning stays available and is the documented answer for anyone who
needs two builds to produce the same bytes: -var base_image_tag=<tag>.
Tracking latest means next month's rebuild starts from a different
base, so the README says that plainly rather than leaving it implied.
One rough edge, documented in the troubleshooting table: plain
`packer validate` does not execute data sources, so it now fails with
`invalid checksum: encoding/hex: invalid byte: U+003C '<'`. The flag
is `packer validate -evaluate-datasources`. The other two new failure
modes are in the table too -- a tag with no index 404s from the data
source, and an unknown base_image_name fails on an empty collection
rather than downloading the wrong thing.
Verified on Linux with KVM. No tag passed, resolved the current
release and enforced its digest:
Trying .../v20260930.063326/ubuntu-24.04-cuda-13.2.qcow2
?checksum=sha256%3Aac30efbe2de11060802b12fa0f27e81a316f49d814ba...
Build 'qemu.sfc_base' finished after 3 minutes 55 seconds.
That build is also the first to inherit the base image's sshd
hardening, which only became possible with the release cut today:
ubuntu hardening inherited: PasswordAuthentication no
build override: removed
root: root:!:... random-seed: GONE
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Review changes with Changed Files
|
An end-user review walked the README literally. Four things it says are wrong, and in one case following it leaves a template that will not load. The -only warning was false. Packer exits 1 on a mistyped -only and lists the valid build names; it does not, as this README claimed, silently succeed having run nothing. That claim came from testing `packer validate`, which does behave that way, and generalizing to `packer build` without checking. Quick start spent a paragraph telling people to audit their terminal for a non-event, and the troubleshooting table taught a diagnosis for a symptom nobody will see. "Making it yours" told readers to delete three things to drop the stock-Ubuntu path, and omitted the fourth: the "source.qemu.ubuntu" entry in the build block's sources list. Doing exactly what it said produced `Error: Unknown source qemu.ubuntu`, with no row in the troubleshooting table to explain it. The README broke the template and then offered no way out. Now four bullets, re-tested by following it to the letter. "Pin the release if you need two builds to produce the same bytes" was false in a way that mattered, because it was the stated reason to keep pinning at all: customize.sh runs apt-get install, so pinning the base image does not pin the Ubuntu archive. Now describes what pinning actually buys -- a repeatable starting point, same driver, CUDA and kernel -- and says plainly that it is not bit-for-bit. The two network failures the new resolver introduces were undocumented and are the ones a laptop hits: DNS/offline, and a captive portal whose HTML lands in a jsondecode error. Both now have rows, as does the case where a 404 means the index is missing rather than a bad pin -- the old row told people to drop a flag they never passed. Also: builds now write output/sfc_base/manifest.json recording the release they resolved, which is what makes pinning usable (build once, read the tag, pin it). The output filename tracks base_image_name, which the upload command silently contradicted. The stock-Ubuntu path now says where its artifact lands. A qemu-img check before pushing 20 GiB. The quoted sample output was not what Packer prints. The CDN host is named as ours once, rather than appearing as a bare domain beside a sha256 the reader is asked to trust. "Image requirements" moved after the troubleshooting table, since it opens by addressing readers who are not following this README. Not taken: a request_timeout on the data source -- not a supported argument in Packer 1.15, verified. -var accelerator=hvf for Intel Macs is probably a large win there but is untested, so it stays unwritten. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
base_image_name defaulted to ubuntu-24.04-cuda-13.2, which was a version in disguise: the release matrix is two newest LTS by two newest CUDA, so the name drops out of latest.json the next time CUDA moves and the default build fails on an empty lookup. A hardcoded tag was removed for the same reason. Empty base_image_name now means the image with the newest CUDA for ubuntu_codename, ranked numerically so 13.10 would outrank 13.9. ubuntu_codename therefore applies to both starting points. An explicit name still wins, and the pinned-tag path is unchanged. The output file is now a fixed sfc-base-custom.raw. With the image chosen at build time, a filename that followed it would make the upload command in the README wrong for anyone who changed a variable; manifest.json already records what was picked. Verified against the live index by starting builds and reading the URL Packer chose: the default resolves ubuntu-24.04-cuda-13.4 from v20261008.012133, ubuntu_codename=resolute resolves ubuntu-26.04-cuda-13.4, an explicit name is honored, and base_image_tag=v20260930.063326 resolves that release's ubuntu-24.04-cuda-13.2. An unknown name or codename fails before any download with the max() error the troubleshooting table now documents. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Runs on pull requests, pushes to master, and daily. The daily run is the point: the template resolves the release index at build time, so the repo can stop building without a commit, and validate with -evaluate-datasources is exactly the resolution a reader's build does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…sions The variable description and the 404 troubleshooting row said older tags are "listed by the release index". They are not: latest.json carries one tag, and the repository that cuts releases is private. The only place a reader can get an earlier tag is a previous build's manifest.json, which both now say. The default candidate filter now keeps only the base variant, so a non-base build published into the same release cannot tie with the base image on CUDA and win on list order, and a cuda_version with no minor ranks as .0 instead of failing on an index error. Both checked against a crafted local index: on a 13.10 tie the base image wins over a -minimal one, 13.10 outranks 13.9, and a bare "14" resolves. CI drops the OVMF install: validate never checks firmware paths, so the step tested nothing and was the one most likely to flake (apt-get install with stale lists). Adds permissions: contents: read, and records GitHub's 60-day inactivity rule for schedules in a public repo, since this repo is meant to go quiet. README: the base_image_name example uses a placeholder rather than a name that will leave the index, and the customization section says what removing the base-image path involves, since data sources are evaluated for every build. Comments that still said jammy now say noble and resolute. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
#2 shipped
build.pkr.hclwith a hardcoded release tag and a hardcoded image name. Both are versions somebody has to remember to bump: the tag goes stale after every release, and the name leaveslatest.jsonwhenever the matrix moves (it is derived as the two newest Ubuntu LTS by the two newest CUDA, so names drop out by construction). This removes both and adds CI.What changed
The template resolves the base image at build time. Packer's
httpdata source fetcheslatest.json, or<tag>/manifest.jsonwhenbase_image_tagis set. The index lists each image's sha256 beside its URL, soiso_checksumcomes from the same document asiso_urland the two cannot disagree. That also deletes thefile:SHA256SUMSindirection and thebase_image_checksumvariable with its=noneescape hatch, which existed only because a pinned tag could predate the checksum file.Empty
base_image_namemeans the newest CUDA forubuntu_codename. Candidates are the release's base-variant images for that codename; the newest CUDA wins, ranked numerically so 13.10 outranks 13.9.ubuntu_codenametherefore applies to both starting points. An explicit name still wins. An unknown name or codename fails before any download.Pinning stays available and is opt-in.
-var base_image_tag=<tag>fixes the release. The README says plainly that tracking latest means next month's rebuild starts from a different base, and that pinning does not make the build bit-for-bit reproducible sincecustomize.shstill installs whatever the archive has that day. The index carries only the current tag and the release repo is private, so the only source of an earlier tag is a previous build'soutput/sfc_base/manifest.json, which records the tag, image name, URL and sha256 that were resolved.The output file is a fixed
sfc-base-custom.raw. With the image chosen at build time, a filename that followed it would make the README's upload command wrong for anyone who changed a variable.CI (
.github/workflows/ci.yml):packer fmt -check,packer validate -evaluate-datasourcesand shellcheck on pull requests, pushes to master, and daily. The daily run matters because the template resolves the index at build time, so the repo can stop building without a commit. GitHub disables schedules in a public repo after 60 days without activity; the workflow comment records that, and the release pipeline insfc_base_imagewill build this example on every release, which is immune to it.Rough edge, documented
Plain
packer validatedoes not execute data sources, so it fails withinvalid checksum: encoding/hex: invalid byte: U+003C '<', which does not sound like the real problem. The flag ispacker validate -evaluate-datasources .; the README's quick start and troubleshooting table both say so, and CI uses it.packer buildneeds no flag. Fixing this at the source would mean a CDN redirect in front of the index, which was considered and declined.Verification
Resolution checked against the live index by starting builds and reading the URL Packer chose:
v20261008.012133/ubuntu-24.04-cuda-13.4ubuntu_codename=resolutev20261008.012133/ubuntu-26.04-cuda-13.4base_image_name=ubuntu-26.04-cuda-13.3base_image_tag=v20260930.063326v20260930.063326/ubuntu-24.04-cuda-13.2, the newest 24.04 in that releaseCall to function "max" failed: must pass at least one number, before any downloadRanking checked against a crafted local index: on a CUDA tie the base variant wins over a
-minimalone, 13.10 outranks 13.9, and a bare14resolves.Real build of the final revision on Linux with KVM (Packer 1.15.3, QEMU 8.2), no variables passed:
qemu-img info:file format: raw,virtual size: 20 GiB, 7.02 GiB on disk.manifest.jsonrecords tagv20261008.012133, imageubuntu-24.04-cuda-13.4and the matching URL and sha256.Every error string in the troubleshooting table was reproduced verbatim.
packer fmt -check,packer validate -evaluate-datasources, actionlint and shellcheck are clean, and CI passed on the final commit.An adversarial review of the whole branch produced eight confirmed findings, all addressed: the false "tags are listed by the index" claim, variant ties on list order, a minor-less
cuda_version, a CI OVMF step that tested nothing and was the likeliest to flake, missingpermissions, the 60-day schedule rule, a README example that was itself a version in disguise, and what removing the base-image path involves.Two remaining example tags
One in the
base_image_tagdescription showing the tag format, and one in the sample of Packer's output above. Neither is copy-pasteable into a build; the one command a reader would copy uses a<tag from latest.json>placeholder.🤖 Generated with Claude Code