From 4173dbc93df82fbee5ec96fc7714688d6a1d2fe0 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Fri, 2 Oct 2026 23:12:36 +0000 Subject: [PATCH 01/15] T099: a release workflow that publishes openDox to PyPI by trusted publishing (plan 034) RULED openxFactory#656 comment 5962754358, item 1: "Publish to PyPI at the cut (Recommended)". Release 1's ruled install line is `pip install "opendox[local]"` (#1144 10.3, batch H's addendum, 5850003126), and today pypi.org/pypi/opendox/json answers 404. Release 1's version is 0.1.0 (RULED 5963162921), bumped in a separate last landing. .github/workflows/release.yml, workflow_dispatch only, with a `version` input that must equal pyproject.toml's version (normalized PEP 440, not the 0.0.0 placeholder): - build: refuses unless the dispatched commit is the one opensoft/openDox main's contracts/code-pin.yaml pins, and is on main (5963162921); refuses unless both environments, testpypi and pypi, name a required reviewer; installs the release tools from a hash lock; builds the sdist and the wheel with `python -m build`; runs `twine check --strict`; verifies the artifacts before any upload (exactly the two files, the metadata's name and version, the `local` extra, the `opendox` console script, every tracked web file, every tracked file under src/, the migrations); installs the wheel with its `local` extra into a fresh venv and runs `opendox --help`. It holds no token. - testpypi: the dry run, in the `testpypi` environment. - testpypi-install: TestPyPI serves the verified digests; then T099's falsifier, `opendox[local]` from TestPyPI into a fresh venv, and `opendox --help`. - pypi: the publish, in the `pypi` environment. Both uploads use pypa/gh-action-pypi-publish by trusted publishing (OIDC), with `id-token: write` on those two jobs only, and each checks the files' digests against the build job's first. Every action is pinned by full commit SHA. No token and no secret appear anywhere. .github/release-tools-cpython312-linux.txt: build 1.6.1, twine 7.0.0 and setuptools 84.0.0 with their dependencies, hash-locked by uv. pyproject.toml: `readme = "README.md"`, so the metadata carries a long description and `twine check --strict` passes. pyproject.toml's single-writer order puts this edit after T072 (openDox-code#69) and T075 (openDox-code#73). The version stays 0.0.0 here. The artifact checks pass only once openDox-code#69 (T072, the `local` extra and the migrations) and #73 (T075, the 42nd web file) have landed. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/release-tools-cpython312-linux.txt | 514 ++++++++++++++++++++ .github/workflows/release.yml | 535 +++++++++++++++++++++ pyproject.toml | 6 + 3 files changed, 1055 insertions(+) create mode 100644 .github/release-tools-cpython312-linux.txt create mode 100644 .github/workflows/release.yml diff --git a/.github/release-tools-cpython312-linux.txt b/.github/release-tools-cpython312-linux.txt new file mode 100644 index 00000000..dfca766c --- /dev/null +++ b/.github/release-tools-cpython312-linux.txt @@ -0,0 +1,514 @@ +# THE RELEASE WORKFLOW'S OWN TOOLS, LOCKED BY HASH (plan 034 T099; RULED +# openxFactory#656 comment 5962754358, item 1: "Publish to PyPI at the cut +# (Recommended)"). +# +# `.github/workflows/release.yml` installs exactly the files named here, with +# `--only-binary :all: --require-hashes`, into a virtual environment of its +# own, and builds with `python -m build --no-isolation` in that environment. +# So every program that touches the sdist and the wheel before they are +# uploaded (the `build` frontend, the `setuptools` backend and `twine`) is a +# file this list names by its digest. Nothing is resolved from an index on the +# day of the release (SonarCloud `githubactions:S8544` and `githubactions:S8541`, +# met in the one job whose output is published). +# +# NOT `constraints-cpython312-linux.txt`. That file is THIS PACKAGE's resolved +# dependency tree, and `tests_runtime/test_deploy_shape.py` holds every +# install of the package to it. These are the tools that BUILD the package, +# which it never imports. The release workflow's fresh-venv install IS an +# install of the package, and it reads that file with `-c`. +# +# `setuptools==84.0.0` is the version openDox-code#69 (plan 034 T072) pins in +# that file for the `test` extra, so a release is built by the same backend +# the suite's own wheel builds run. +# +# MEASURED, NOT WRITTEN BY HAND. Regenerate it on the platform the workflow +# runs on (cpython 3.12, x86_64 linux), then put this header back above the +# output, and commit the diff as its own act: +# +# printf 'build==1.6.1\ntwine==7.0.0\nsetuptools==84.0.0\n' \ +# | uv pip compile --generate-hashes --no-header --python-version 3.12 \ +# --python-platform x86_64-manylinux_2_28 \ +# --index-url https://pypi.org/simple - -o +# +# Resolved 2026-10-02 with uv 0.12.18. + +build==1.6.1 \ + --hash=sha256:51cc11666391ab6f092070437ac747002ff46f3e4113a3622177ee6b488bfc53 \ + --hash=sha256:ecd351a4be9d35a9eaaba244a7687143c9c7d4aea6ac964e7e7ddab20cbcf4e7 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 + # via requests +cffi==2.1.1 \ + --hash=sha256:046bfc24911b37851ee1b51aab8bffe713d89c68c6a057b09484ce9fd5f69b4e \ + --hash=sha256:06c72bb76605a4b0cd0aad6930b69d4baf7dd5d806cfc409b824191099700e66 \ + --hash=sha256:0beceaabe56af686895136a2de78db54ecd8e4046b236b8fd6d6cb61389e9bf2 \ + --hash=sha256:154852545011f779917b11c78db2358d095da62a9a172b78ad0a583ee5adc0d0 \ + --hash=sha256:194cffa889098ced9976c3fc6340305e43f6303657d298da55366907c05c22d6 \ + --hash=sha256:19ee6127ee34de7d83ce3d371ebc5ed91addbdcc39f9ab15ce4eb35a4e534971 \ + --hash=sha256:1a18a57b58cfb21fc28d72e876acf10eaed67a1ed96226f92af4df681d571c4c \ + --hash=sha256:1aa5645c30469b09530c4ebca77ebf8f17618293c58f8549cb1a543a50236e7d \ + --hash=sha256:1dea0e4d7d4f11f619fe8c1d76caf49e24405b4b5743c0e3be16a500ecd930c9 \ + --hash=sha256:208f941bb9d18e768138677f0a6d2ce01f590df56043dda1df1535ac57c88517 \ + --hash=sha256:210019b6c7cf07f081b4c54635c8cf744377001350e29cc0f81c4377b4797735 \ + --hash=sha256:246fa40ce8645a614ff682e0b70f37134e460eaf93a775e0cbe3cca585a67a80 \ + --hash=sha256:25792eac27877609e7bb06d42ff88278a6624fff2ba9bbb523c09616b117e80f \ + --hash=sha256:27350daa11d4f10c540e6e89dada4c54feb7256ad03e9a4dc075ebad7ba360d1 \ + --hash=sha256:28907ab9bfb6aa13184cfc17c6b8e1023c5ab6fd7076d8c20a35e59fe04f8f29 \ + --hash=sha256:2ae64be792b8966f2c69538199728b290e34726562896df1e5dc8ffd8d8188e8 \ + --hash=sha256:31348097ff5bbe827ccc41795d4dd099d9f0625e7def00ee653c137a490c2a6c \ + --hash=sha256:3143d81e29e1e20a9ce10901ec369012947876596f75a222235965f2b7ae832e \ + --hash=sha256:3222ba5d678f80a030e6afbcc33dc1ae5cb45facabb61cee2c7016b8432fde48 \ + --hash=sha256:3311ed60d36f83378794e1009ac6258bafbf81f7888b4caa7b35a521e3f95813 \ + --hash=sha256:334644fbac4eff73d985a17a91226df55d0f394160c4cfb880e084c8f7161cac \ + --hash=sha256:34e261f78cb6ceaaa36f42f2613f4380d94d9c759a9c73c769ee6e0247364632 \ + --hash=sha256:363e05fa78e15116c3c32c210ee36884fd6b9afa6d440e47112c3bd511d64cb6 \ + --hash=sha256:398aff33cee2767e3e781d2554c54bd0dff386bb437581e0d8011fde1a942ec1 \ + --hash=sha256:3d22a20b1fb1632cc72c22f95f7b0d2961c3e1c235f245ba4c606c4771035659 \ + --hash=sha256:42a494cee34437f05546455144f2b5d9ac09b1face62bcfce597d2e521066688 \ + --hash=sha256:42e2f76b9455f5a9a844f770bf3e200ed3da0e15f5df3db9c31fe80b04b3d004 \ + --hash=sha256:42f6930c31dc7f50732c9ae793c2786c7b6b044195967bbdde40bb9be81c4cc0 \ + --hash=sha256:456a61fa52d579ebf9df2e9552ead5129855dbaff6c1e5a9b1bc408809bdc062 \ + --hash=sha256:471cee653ae88de62096552e6d24ccb4a5adb8c8c9f10b5054d0122c15bf2779 \ + --hash=sha256:49cbc70e6542d4ccccb936558d1064a8012541e78f821f955cff24e357776c94 \ + --hash=sha256:4a7c934f7360e8cd64fe9efadcbd10c7c6364f531e432b9a4bf5ccbc9e0e8b50 \ + --hash=sha256:4be96343e422f2dfcd12ab5c9f5aebe03f82f737c6bffeca6830b3875cb44aab \ + --hash=sha256:4f42141fc14250de6dde5ee7ea4432be017252d91f19c5ad043c084cea629cac \ + --hash=sha256:507a24c282e0f42f8ed737cf048572cbf580468da5555764a8331735e9c736b6 \ + --hash=sha256:51b31d1c98274844cfd7838ce00bfc27c7423a4dc00fc0772fc3331c2cc90676 \ + --hash=sha256:58acb8ab8e295e6c5ea12f888cbb13cf21511ef2a3303a23f4325c29d17fe5c1 \ + --hash=sha256:5a59cc1c4442bc3d5c703bf720b51138d0bfc173618807c9ee2490a7541dd3d9 \ + --hash=sha256:5bb4e7ea95dcd6a014a6fef62e62467d67d8e582326443f3d68e71d6320a9fcf \ + --hash=sha256:5c58fe613dc5e5336357eff555824a314d8e43282600435c8d1cb6a7a2fedd13 \ + --hash=sha256:5e7cecbaadb83884793e05828cee59b210b24583b9c7425d0ba6a754fe22eb4e \ + --hash=sha256:616f097f2fe415bc92a247f02e11f634e1f9e9a83d327e3c915c15089c87869e \ + --hash=sha256:63bbfd5ded17c4840ac07cd8f1c21ba9d9708141f840b324f422f41b207e3973 \ + --hash=sha256:64faea20f4e2613363a1a9b9c7dd73058f3ecd00133a511e72ad7c511658f527 \ + --hash=sha256:661c298b4821edebead0c91edd2b00374d67ad7c5a1f7a91d4442633b79d6a72 \ + --hash=sha256:68e62fe11f30d5ca8289242866f0a5291402d8529ca2178ab8afc5c9694ae890 \ + --hash=sha256:6a8dddef476fab96d066d578fc88526767b836ab5ab21754e1d5bf3879c31c7c \ + --hash=sha256:6e192623c49c94421616a5778fba35cf0d5a8d000650c1967ef4448ee5cdd990 \ + --hash=sha256:7225e4514edb64eb6740324353e0da0711954fd8d7da4576755b1c6e09b697cd \ + --hash=sha256:75f80557d1389eddbd0de2681f6a390a0c5338c31ddaa821381c203fc3fd50d9 \ + --hash=sha256:770de9db11e84213beec501cfcaa013b019820ca881e03344dea5844f7876d94 \ + --hash=sha256:7750c6449dff7864bb9bb27ddfb0267756189201a3afc911d82b3caacd70dfc3 \ + --hash=sha256:7bde5e4cc5c10140859842b9d383af292b22639a4dffb725314baf45968cef80 \ + --hash=sha256:7ce713ace7c0e4520535b42b77eaa742c16dab813978064913e5a3cf82973b41 \ + --hash=sha256:7da0c5eff80f0197f3b3d1232ec5a682a9325f4ae9016a78f5f5ca35f9ced1f5 \ + --hash=sha256:7dbb61fe3a7699468030f71bbe5f8a0e326a151daa91beb11a6fc1f980c55e1c \ + --hash=sha256:811bd1e21d32de12efca32393a0ab3f5133b54fce9bd44b8bd77ab07da14bf6a \ + --hash=sha256:8ef53b2de9bcb9197d31854256575d59dbac0cba72ac627bb291ef5eceb74be4 \ + --hash=sha256:937c0052c05a31ca1daf18de3158eed4dbfcb9cc107adbea227728d647be701e \ + --hash=sha256:9d2055050ea716bd38b7f7f1579c275386646b4894c155a3e2f3cd62ed41b7c6 \ + --hash=sha256:9f8d177621de5cb38ee3e731eda45d421db093ec0739f46a5594babda7987a98 \ + --hash=sha256:a2d7755bef5a12ed488f4ef1f1b69ee9191d7396083b755a5d2295f6edb4768b \ + --hash=sha256:a48d62ab9d6f4f98c983223a547af44be6ca3691074c31cecced6facd3ba2dc1 \ + --hash=sha256:a4f00aa42f75d6e4595e8866e748cc1705adc0cddfeb2ca86d0d03993d63ba03 \ + --hash=sha256:a6e721d4b0e45d5b65e87534470e67b18dcd092c83f68fba09f152b9cbc061af \ + --hash=sha256:a730a083190634c65cca36ba5f489531576ebd79bcd5c8e172130f6453127231 \ + --hash=sha256:a931079504ecc49efed7744c476a5c343a92fabf66dec2db95edb1b2fdc770e2 \ + --hash=sha256:aa9511c62d14da7aacc9b4bf51f3f697a621e83b2d6919008243c3aad168eea3 \ + --hash=sha256:ab36d55f9ed2d067327667c2fea18dda018eb628dd6347aa01dda6cf1f5d3836 \ + --hash=sha256:ad2c86c495b899d862ea0f4b42891b8713a3bd45dd4105c7fd51c2a72f39f3a5 \ + --hash=sha256:aeae0e330c9f6acd681f647d46cefd30c29f93e3392882e792e82080c9691399 \ + --hash=sha256:b0431303acaea1089ad4b3e9ce4e6518193def1118d4073ca848635ee4ea2e96 \ + --hash=sha256:b5bdfd1c873d4e093aabc0ca84c4ca6dbc4f752afb5c86f146d9742580c9da2e \ + --hash=sha256:baed1e86cc735622097354b9d1281406caf42ff42a886d29faa8e8d1630333be \ + --hash=sha256:c1453022f490d2459a11819d83ad1d586e9ff65a12ac3e705ffebd46d3685dcf \ + --hash=sha256:c26608d2222fb1e94487e4a387d85f13eb55d5ed725cb25a0c589ac4ee60e7bc \ + --hash=sha256:c7659f22557c5a0bc4855cd635f55edec690cc008a40768527762cb9fb263455 \ + --hash=sha256:c8c69575568085ba0b1b10c0249d779a214aea6f6522e949a0fc9fb0fcb449d0 \ + --hash=sha256:c8d2c9fd1f2d16f780d15127abb050d13d1a76c03a4bd87d7e4980e45e511e12 \ + --hash=sha256:ca82be1a1d406ecfe1d25dc16cb33488e5a16bf4438c9fb590484ea29d92478b \ + --hash=sha256:cc572dace3f60ef98d7b12ff411d20f5362feb31a0439eab0085bbfd349982d7 \ + --hash=sha256:d18e5ac0f2f03f4f518d3e23db0f0cad7faa1da8620e9c09461d443bbf6e6692 \ + --hash=sha256:d28630f5854ab07ab1fd4aba756de52326c82e6be15d414b12793f1975048b54 \ + --hash=sha256:d9c275eaacd24aa73f94ffd6de08fc3f932424d8b6c376f4bed7cde376fe7bc3 \ + --hash=sha256:da0e573f9f97159390c89d9f1a9e41908b66d408cc5b58d08cf3847d844c531b \ + --hash=sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be \ + --hash=sha256:dddad92b554513a31f272570678ba307fb9f618f05e3d4a5eacafff9eae03e1d \ + --hash=sha256:df423d40ee8654634421812bc3b196da3f9bd7d32929da813f8394c4348a5358 \ + --hash=sha256:df913725b79db7bcf03448f36b7bf8815363417d5b58deecf9305e3e30f0f21a \ + --hash=sha256:e0bcb7e0f677f543555d2adff3bf19c05f66cdb4796e5ff602442ab2fe3c4ef7 \ + --hash=sha256:e2d65b31f36619cda3999b78b2aa9632e76b78448e7a56fc4240824200e7c4fc \ + --hash=sha256:e6e8cff14d6fb0be70a09c0bdc58096f501952d04624ebf867e0e56da2df8960 \ + --hash=sha256:f16c709686a78c727bbbf059f92b0bf41c6fc60deec706d2dc19f529175a6125 \ + --hash=sha256:f24fb43132a4c6b4cb4eb029492919b2db645be6808d738f244fd146c03c32cb \ + --hash=sha256:f53e442b08449d42821fa4a4fba000095af9f62742a500f978a9f557ec44339a \ + --hash=sha256:f5cfbc5fe74540d335175b656c725d74d90e3730c626d92575eea35029d9afaa \ + --hash=sha256:f81b3b8f3d4e343550fa4baa0e479bba9f2d29ce9c2e9b51d1ce1718d7442fcf \ + --hash=sha256:f8ec5e643a9a937f64e1999eb9f75d072263751912dc5cd06d3c85f8f44be7c3 \ + --hash=sha256:fb92203a88b3d3053034db775110081c49d28be6551923805e039924093761e4 \ + --hash=sha256:fcd22650c908d7b7da162bbfaab594a1227a15d1643a98c68b122ac642fa2264 + # via cryptography +charset-normalizer==3.5.2 \ + --hash=sha256:01077390b03f7988f11d700a2194e69b119741a86b1a638b1db88891e3eced8e \ + --hash=sha256:01b0c0d2262a9e28e8484a278c7e1b5d650e3ac8cf2683d2967e25899f208bdf \ + --hash=sha256:04851f73ae72b8413dddadb16a49dfee95263553741fd42d546f7d66907e6be5 \ + --hash=sha256:0521c5665880b33d603717defa76c094048900010897909952397feb3039da56 \ + --hash=sha256:0774bf9bf620249fee3e0b8b9fd3065de213be30f3aa94ce2494b3b638949e26 \ + --hash=sha256:0891b9d3903c5571c03771ca669a4b0ec5618ca722a5c957d3d29cd4e5062848 \ + --hash=sha256:0c951d5e6dd9c2ff60609476752bee49da4206adde960ebc247766937f72e718 \ + --hash=sha256:0fed1d06615f022ee3b13caf5e8b180cfea32bb2c5aded8a9d44277afc040f93 \ + --hash=sha256:114e4d0c92d618409ed82a99e22b5c5e768fe995f2973f78265f4524f49d4640 \ + --hash=sha256:11912e4bb14baae7c5d8791aa55ba0a3a03ec6729073307b0f57270abaa713d3 \ + --hash=sha256:11a4d68a6ecda3292cb1e50239e111543ba5d709bb62a6b4ea1afcfa729d8875 \ + --hash=sha256:124fbf1a8ff966d87ae05bb8bd45a71f966055ed8bba320d0c7cf450bc5f4d0e \ + --hash=sha256:1461ac396c4fdb983a675f20aa555624f0ee18ac83d832b9244ffff3d8055275 \ + --hash=sha256:1503bccbeb36d5527790c3930327704c39af22de3112f1b1666a9f3ce15ee204 \ + --hash=sha256:15bb4005af6320d259dc7593ca84a38d7fe06a421dbcf7b910ae23979101e787 \ + --hash=sha256:15c44f7edfd477b06f517a5cc317fc1707edb9de2c865f43d4b6513907473234 \ + --hash=sha256:16fa0eccf81304b79c5cd87f9271c3b85dd9dd99245e4422ae9c0dd45e0f99d3 \ + --hash=sha256:183b88127acdb4fabe59d951ab424faf1af7b63cdbb5f776186c1ea2ffcaed98 \ + --hash=sha256:195c26fb65950f8fce54e26349852b7bdd7c5f120aeefbcc440b8a20faaed4a3 \ + --hash=sha256:1afb975bd5d68d5ce9f6b6d44fdf2f7e34b895a35e95708a7a91b20a3b51d187 \ + --hash=sha256:1b4cbc7c3491ccb4aa17fcd8165649d01cf39f76de1696da8631b5f71b85401d \ + --hash=sha256:1bc0baf5ef96b6ede57d47f4b8fe4d9d84019c3bfcbeb20a41edc6a6ee341f1f \ + --hash=sha256:1c50fe28bbc2ced33386f298650d91218076c05420e6cbd790b913adc41659e7 \ + --hash=sha256:1db38f4c5496827c1a501846d64d14c3b80c7e6714e406cd7dc36a9899fa1011 \ + --hash=sha256:211d5a3eb6af8f513b8d4ca19a8c1b7accab1b5f0d3175f9826b03c1a920dc1f \ + --hash=sha256:23851fb4e1b85ed3f6c2a27b777cdfe2e19fb5b38429a8faf38c7542b7665869 \ + --hash=sha256:254eb48b9fa5ee9898a3c445825a1f340fe53712a098904b39b0bddba8ea3cb1 \ + --hash=sha256:2625388c6c754520c37abaf3b41eb34d1cc4a373f457898f08606c8e362b891d \ + --hash=sha256:281cb91036248400f4cc957495cccd44c275c2e0c5854f7e45ac5cf7dc193847 \ + --hash=sha256:28a15fdad492a99b6eccfaaed66ef3f74050680545ea61ec8b2f4c538f1f1320 \ + --hash=sha256:28b4f0d66fb834ff90f28209ac7bce77868c45d8c93e26f906709d9b7c2e1af9 \ + --hash=sha256:2a925889534b3748302dae5dead07cc13480de1dac3aea80a941b729b471ef93 \ + --hash=sha256:2b7b3bbfb4fe8ef40600792d762fbaa9057559f9d3fad209525b7a22b99e91fd \ + --hash=sha256:2c9ad19a6cfcd5ea5c0d41161d22f9df1dcc277e9bef2751391334546a314c00 \ + --hash=sha256:2cc961b171b3f3440f410489ab3573e86aea8736134ebbb40ea1338b7f0831bc \ + --hash=sha256:2ce45c6627b22c47e390bc91a41c3d13032192e699fa0bea96e9671b373d69b0 \ + --hash=sha256:2e06a3a98f916dd41d27f3105e02e7a40181c98c94b9158733d03a6f80506c09 \ + --hash=sha256:304d5463e65a35d7bb0850550e0780395395f6fcf452f04db7d5ca7cecc425ac \ + --hash=sha256:304d8e4d493af723536393eee0c689eb7813f4a474c8b479dee63f1fdd98f621 \ + --hash=sha256:30fcd120b732aa79317f08dee04d7de0847822e4cf7ee0e9f445bb958832252c \ + --hash=sha256:31f3930700408d211f13378ccbe1c40845d8da54bd0681fac3a9b5aae81c7aa8 \ + --hash=sha256:34276fd796040bf0993ab33a369aa572e6979c7aab225a88893667ad8eac8f7a \ + --hash=sha256:355ad8011081dec5412240c087a9a0c9d4d5039f3ed11a3f13e18c2b29b56c51 \ + --hash=sha256:38a873987f3be698494da8b2e3085e29da02da7b633dce73e79c699a113d7bf0 \ + --hash=sha256:39de2a259fc954455c57274dc94c79d5842774e1247a016aff30bc0efed0f4ef \ + --hash=sha256:3d14b50de6bf4d0edf857a9386836846f982b8f524e188e2e68b96d702bcf4aa \ + --hash=sha256:3d21b8b13c7592db2ac5e544a6d83187b995257472b0c9e8351b6d507ae37ed6 \ + --hash=sha256:3d31298449090ab8d47b7b1b2a555ff73cac7ed438a08b7ac160980c7ebed649 \ + --hash=sha256:3ddacd27458c45bdacd6bd6db644bfb730efbf9e830310186e3045c9c5be8fb2 \ + --hash=sha256:3df041de8887954562c9b261cba85ca0e9ded74048daf125f45edcfaa4832229 \ + --hash=sha256:40ab6bffa02ae10a0581e6c198be7d2d8ca5c2a0c64e4ed3465d766df457573e \ + --hash=sha256:4275811936e2f06feff5e598fb42a1b7ae852da8e39605211892b56b81a34efd \ + --hash=sha256:443eae2bf318abeaf6f15d785138f71fd6de770e99a92158b8b814265e079115 \ + --hash=sha256:447441e76ec720b15e64418d32e092297340387053047c7c694f579efb0ee1d9 \ + --hash=sha256:4495c5002a7b28557e7e222e77e0b661183e432b7d6d2e788101e3f240e05b8c \ + --hash=sha256:44bd4fbb29dfbeba60e7d2bd000c59e4b21ddb3cc53912b14048d37092706d7c \ + --hash=sha256:4685902cf26edf013ed7a3da0f426ebba7a00ebb9541386d835afbf002c11cab \ + --hash=sha256:498dc3188ca05a68231ac3fdbfc7f57eb67e1343c30e0fea17f8218c1599b253 \ + --hash=sha256:4c2b5031f63e331e3839b40aed2dd6f191e9c07edbde303e7876846ea1946995 \ + --hash=sha256:4d48f2d08b9de5864e2c8744d4461b862fb149a18274abc8b698c45975573438 \ + --hash=sha256:4f87960d57feabfb618e4e0af6e7371645fa26a277860739d6e5d6e0012c92f0 \ + --hash=sha256:50e3adfb96fc189eb27b1cf62d3b598b89b4bb0420d93a3d3e42e137409011be \ + --hash=sha256:51cf45226a9b588d0d2b4880c62d686934b63ab0bd79ca23ab0e9762eb27441b \ + --hash=sha256:52aa6992700996af31f375de0c6bacd402b0097fe40b53c426b9f51a90ebabc7 \ + --hash=sha256:55ea99acb17b9325618de155a0cd6a2e8f5d10be008113e1d433bbb58db543b2 \ + --hash=sha256:56bc200a365efb37383b7852e4cc5898d3b2da5987289b543956cf8cad71018a \ + --hash=sha256:588461c2e8384d309bd63e5826019b6977bc66d629b99ac8737bb795d7b2cb5a \ + --hash=sha256:58ca3755ee7ff7f59b57789ec9833c9de9ea275405cdd240eda1f193112e398a \ + --hash=sha256:58f361dcbab699cf8f42db3f47c8e7fd1036f138c23a5d08de9fde5f425a730c \ + --hash=sha256:598a11a2c7ebaa5334bf698bf29568c9c390abac6a154d8170fedecd1cea38c5 \ + --hash=sha256:59f63901b0031c3136cf64704dcb21de0bbae62ce2c9529bc39d27665463de37 \ + --hash=sha256:5cde776b7cc66e4f6c99612cea4aa7269aa65863f7a15841b2c264f103822f4e \ + --hash=sha256:5e2b6b57e9733d39f0c9fd3185efa6b8e29652c4cd8fe94180272cf6ed9a78c4 \ + --hash=sha256:5fb29fb8cd1a46c27a1bf9613ad5ec2599310d46b4025d9556404a6b6a292800 \ + --hash=sha256:6045373d5a89a5ec71afde535db987ca28e76dfa276c2d4c818265b375d4b055 \ + --hash=sha256:619799369eeef6366ed3e8755a5670f4f2f0fb6b30a0fd7264dc0fdc2357058e \ + --hash=sha256:62588a277bfb59def052abd940703fa35107152bf479781a878617d60faf8fb5 \ + --hash=sha256:62603db9a7caa0802eaa28c1c46fecd7b3a263a774069c24c3c28c302448721c \ + --hash=sha256:65cd72beeeca9d3aaea1201e5923859f308f952f9c71de93f06063c79f0f7a3b \ + --hash=sha256:68eb192d85ab8e5f6ec69c2bc6ac0179fbf04a5ac1569d12fbef74883fe102d0 \ + --hash=sha256:6bd128f206a7752ae1f2ab6c61bf8a24ba28913a10df8b14c2637b973ff97a80 \ + --hash=sha256:6be488a102b8cf28d0391d8c4ba7748938ae28b78ad901f8585520fca33ead1a \ + --hash=sha256:7218e8f32b0956cfcd048fd42d9d5779809745ca1d86113ca56f66e7ae1549c4 \ + --hash=sha256:7441d755b7ab94f8d4eb3e43ec05482d760842fd263d003a99102d742cd835e2 \ + --hash=sha256:749e97e1b32313717a565abbe321bc2190bc8b35f1a67e4cdbc7c56c8d8ffe58 \ + --hash=sha256:75a3ceed0724d625d64b86ca20aba182e4df462e04c2414fc941c0f523f06aac \ + --hash=sha256:780fbe7cab297b81dad9fb8dc5eb003c0468ffb0d9e5f65068c53a34661a96bc \ + --hash=sha256:78456a747de8dc58360ffa581f30a002baf5aa28cb262536545e91f113ed7639 \ + --hash=sha256:7967d08cf06dee78443b874f98c98036f624f3a4e73e11f9f64f5be4d25393cf \ + --hash=sha256:7a881931aa470808df94a8c380eed2bbbc76cd9dc622310f99665658c821eb6d \ + --hash=sha256:7dcd882da75ef9adf94903b1e3b9419e8aa8fb4c7396822b834b9ef7fb96954f \ + --hash=sha256:7e841fb9010836c992c9f12fcbd43a831de93a5f726fc1ccd8ca1d0268c5014c \ + --hash=sha256:7fdde2c9fd9e3eca40631e024664cf2584272cc8f96308cbe5fdfc930f51d8bc \ + --hash=sha256:8024d00c3faf3fc0c16e07a69f4405e8eac7cc0ab15f65fe6cf43827c4cf72b4 \ + --hash=sha256:80d02b6f04e92601a081dd97b23d3128033098bff5d35d392ddcc0476ea11253 \ + --hash=sha256:838dcc90063569a0448120554591a1d6c4a4ffe11babf048908793154ab86ade \ + --hash=sha256:849df64e889b2e17230d58410a03dba311a65b163508fd33679b2b737d4b7858 \ + --hash=sha256:87475fabc8d9996fd9c27debb395e642e8c838d78a00b6e932227a0e06b81e26 \ + --hash=sha256:87e50a3e7cb90af586b6c5faf23e302a970415ac73bd7bd90a515a04b427ef96 \ + --hash=sha256:89b53f3cda69831909888e0494f4fa0bcd3537e3e138dabeb620bd6ad946bae8 \ + --hash=sha256:8a893cc101149f80a653f82062ebc95b34525a2614382e1da5458fe7c6997249 \ + --hash=sha256:8b2bfab86aa71ae13aa41a6a26aab338e0db2b8bc75434b05aea89e011ff35a4 \ + --hash=sha256:8d86d6fc60743dc916eb79e2eb1ec4818e21e427731543af40a3021851174a13 \ + --hash=sha256:915563965d418f986e7e145accc592eae9e1a1be3566ff98a05d7a9ec42a76e1 \ + --hash=sha256:92888bb3187c5ba50500b00b3b310c9f2c651709d28036077680cb5255450a03 \ + --hash=sha256:93223adc95033dd47133a46ccfc316a0139176fd79085762e27202ec56018f03 \ + --hash=sha256:9373ad13ef0d2c0fb761e04e55bfdee5a08b52cef2c882c8fbe9935b1517152e \ + --hash=sha256:9409a8bf35cf78353942504b24a57de3d75b708997a1e4bd8db71ac8633ce364 \ + --hash=sha256:9b7f416ff0978e2f2249330527f0ad6fa02f4932e6199692d3b52da2048c19e4 \ + --hash=sha256:9bde855991b7e362c146535e3136a50bfaffc0487d38b33ca7e5edefc6e23849 \ + --hash=sha256:9cae88599c7219005d879f98e5ed53341e9a122af585e1091200358a3003d2a0 \ + --hash=sha256:9cf9b1a857e25c4baceeb3624e92a56df3668f398c4acba74e174d81fb4d1d3a \ + --hash=sha256:9f56f72050826f63dcee7a7f55b0a77168cb3bfc553fd405e7f8f9ece75a4036 \ + --hash=sha256:a090bb2c68df85450502e3e20d665e3a5af9c65a84d6508ed477badd49166fd3 \ + --hash=sha256:a192e2c40070d92c3ccf777e3a5c4ff515573cd2bb7ed0c537fdadbbec5bbf21 \ + --hash=sha256:a19a731138fc27d5682277d3b9df22855cea1239bce7fcec5f78f42ef2d1f3c3 \ + --hash=sha256:a66c3bc5ab1f0ff2164fc9965ddd611ff0802173f4b9d24554c563f6ab7e1d6e \ + --hash=sha256:a815775b6c38d4e0ff7bcffbeba67feded90202bb6a226b8dd35f1c855217413 \ + --hash=sha256:a89012d6d5476ee112d20d998570ed58df2260a852afb1758809cd6900411d21 \ + --hash=sha256:ae4f5fea5b8b8ccff88238cc8569303e5ee95efae67fa62922a311397a71f346 \ + --hash=sha256:b6856554c4f44d79fc2307d5768854310a8f0096e501c75637542c82292b0429 \ + --hash=sha256:b6b751274acb69d77b3323d6b7dbaa3c7fdfc1eb829b7eb61d262f32e1af9685 \ + --hash=sha256:b736353c0a625bbd5fcec108576e2385db3496f4f771f785ff32e108d3c3bc45 \ + --hash=sha256:b7fd005a73d9e657273b7a10dc71a9e03c8fb9ee6999798d6918ce095b81ac7f \ + --hash=sha256:b91363207bd9dc966a691e959bb47f64b30f7ac4b072be9968b366982f7db77c \ + --hash=sha256:ba0b1d2620edf869789c3879223f52bf2afc5d31b3cb47cc57b3a12c05e2aa9d \ + --hash=sha256:bbbfc8e28816f19d7c0f1816664980c0a9875d01b27cdf8eedddb639d9e108ad \ + --hash=sha256:bd16aabe4a02a297c23417aa17ac6299dbd8c49f673bcd645b4929b11f5a4400 \ + --hash=sha256:c0afc6800ba57ccc350374c5bd6150419915d95ce93cdbab2d783d75eaf30ecb \ + --hash=sha256:c6708715abcf3c73b99508253e961a9967f02fe536532834149574eda6de0d1c \ + --hash=sha256:c7c9ab723cde841fefb34efbad91e87f00a674b1fe1cd0784fde742bf2c154dc \ + --hash=sha256:c8f3d67aeaf55f017982b73683f0e7342ba2f6635a78f69ce89ebb26aa411e5c \ + --hash=sha256:c9790464842f85f437dbbb54417eda1e0e6bfc52dd8d22d6fd1c994b73b2dc74 \ + --hash=sha256:ca403d7e4798f525fdfc78e258820419cbbd0f0ecbab9de7840e3c017cf6b8cf \ + --hash=sha256:d008d90a7f2471519aef0c90dfbe73b3e6e4d5e66ac48e19154c17e89e98b604 \ + --hash=sha256:d19fbd981a488e22cd04883659ca6b08f50b5974f9fd7c95655ef6a043e5893f \ + --hash=sha256:d1befeed746d247c81127bb14de9dc3d30edb6e5976d34f83f86ed262b1d9105 \ + --hash=sha256:d2374b62878abb00cd8309b32af6c0b715cd02dec0ca74ef12e5069bdc64144a \ + --hash=sha256:d376bbd28b3a8999db1a103b3b388aee6f1ddeb3e51bc2172993efdcd86e064d \ + --hash=sha256:d4a7319f304a774bed22115bc891618e45f85065ab44ea6acd07d274e750519a \ + --hash=sha256:d6734d2ef8a50fbf8445c139477da401f50d62a0606bf00e20ec6d87773fefb1 \ + --hash=sha256:d760fe2a4d7c3b226cb9026d6a842868d52a7901bd98420e1baf14e80da85cf5 \ + --hash=sha256:d913de495d90407cd859d263bee2e5d1a4ed3eb6573c04e70d9ec619a7cbed7f \ + --hash=sha256:db19d07e2e0129e974a0e65d0064fc222a446cd5122c2fd4184d2af9fc734a9e \ + --hash=sha256:dca9ab98072a5a54ebacebdc45f53e645336b320c667410b061be1ca588ae709 \ + --hash=sha256:ddc7dacc8ece3a182e7f15cb862d1fd616b46d076cb1ae9dd232b2c38b655874 \ + --hash=sha256:ddf19c062bea7a0cc80f519243d2c01dd091be0cf952a0750d4ad576709559f5 \ + --hash=sha256:def79fa35ef0cef8d2accec024f4fdc7ead3012ff02f5215c783f39f03ef8cfc \ + --hash=sha256:df29a0a7107f7011e77f4eebdddec4c7331e24d787a0b21a46d63bdf7445da95 \ + --hash=sha256:e09a3942ecbdee5cce73ea9d42da82b81b72ac1bf031ce069b93b5adf4eac8cd \ + --hash=sha256:e242bb1c5e76e97dfa9e7f209a71e93a01d7f19ffdd5cfbb2e2d55b4f08f8ab0 \ + --hash=sha256:e243bd13217235fc7290c621941c3f5cc8b66e4872495be821d7436ba2fb838d \ + --hash=sha256:e2af3aad578aa6bd1384bcf4750fc285e5a9de53f40b7d41e5a0bf748edeb2b3 \ + --hash=sha256:e4e81e09c1578b8df602e3db08b0b3ea0a6947ad612f52bf8dc5ea8d47691f0c \ + --hash=sha256:e54da4baf05720032d527874d40b65fa4d7e5c6c6a43d0c3adbeffcaf275a2b3 \ + --hash=sha256:e80e6c2f55656b4824d72065abb4ddd6a525c74bd78a0aab5d9fc2cf4fb5af50 \ + --hash=sha256:ed2a239c0ea213acc1908150a3037257083c7c083128f1a4cec2ec4b97dca491 \ + --hash=sha256:ed905975ab14056a2e5eb1c376cb2e1ebc5396baf84163939c518556fccde9f5 \ + --hash=sha256:ee21e28f0430bd6dc9086c6e525d5e818a44a5ad19720c8a0ef766792f3eb5e5 \ + --hash=sha256:ee43c17b173d46a3212baa6ead3ae258eeabdae48c263a01ccf0218c366dd655 \ + --hash=sha256:ef4fcbf3327382cd4c9f540babd61248208af7b93eec4de397b4d5f58a09e288 \ + --hash=sha256:eff0ac9dbe711a4aee69bf04a83896aa9b85f19641264053a9f6d48573abb7dd \ + --hash=sha256:f0aa869112ef88429ae17820d99c3dd9504c9e9c671d3c246f3d7442cb051084 \ + --hash=sha256:f3c96f633825733f735c5a9cf21d21a257d8e1edf0b1cee0a064b9c424ca0f7d \ + --hash=sha256:f5833ad231be5eb6553de524a70f48d71b2c8563101750531e0b80184e175cd4 \ + --hash=sha256:f5ec61164adcec446f8969a3358ec3f9b26bbda3b9213e5586d219afa8df2915 \ + --hash=sha256:f7d486c83842422badd511868fd8a9a20e9407ace71564b6af47ce7e60a336c1 \ + --hash=sha256:fb9e68df06293761f9fe66ade60a9bc6d0f5e42b8acf2939a9158af86ab0e5bd \ + --hash=sha256:fc14a032f813bf5fe624d991960ea83e9715adc27e4c1830a2361eb1d02ac341 \ + --hash=sha256:fcff63213e8e6e47770541a4607175404f47cbb3ebea7b6058cc82d524a0e424 \ + --hash=sha256:fd1fbe0f116b6e55da77aca2c6ddcddcfac2186cbf78bdebf40fc156efca389d \ + --hash=sha256:fe9753dfee015c570d73df76f899f18444d41388bffcde097deba51c4fadbb9f + # via requests +cryptography==50.0.2 \ + --hash=sha256:0ddc924c04591c2811ca024d62ecad4f7f6f08af8939c211438f48a16bd23602 \ + --hash=sha256:0ec5f09541743261e66e291b4a0cbf0fb2997aeaab6d9e9c740b9dba1b58d1c2 \ + --hash=sha256:0ecbc5652bdb6fc9eaf89a7d196e20941adfe812f43bc4ca05d9150496821047 \ + --hash=sha256:1981f1db4630889b9ef7803fadef12b056f428cb6b85c27ba57b774793b6093c \ + --hash=sha256:1ba34f04897fcdaa73f74145c25f3ec146fbd56593853e88adc2e811303c5f42 \ + --hash=sha256:241449bf940a5d27309bd317e6f9a2af6932113818bb2b8f5c59ddc7ef16da18 \ + --hash=sha256:25784ce8b9621c90c643efb9e1e2162ab3b0224cae446ad5e70e7fcb1ce18b51 \ + --hash=sha256:3dc4fd8058cea1644971207d530e1a03a184a805ffc8ebdddf0599d78a331b81 \ + --hash=sha256:4061c0079120205fb760c58acab6443e217307dcf05e3702cf970e0689972856 \ + --hash=sha256:4a20ce1e5cb4284a86692fdcba7cb8754185c6b2e5c56fcef3751cf451d3cdc2 \ + --hash=sha256:4e81d95e5bafc2d6e34e4bed780e53e4d5b9a2f928573428aa4d35fbec1eb0de \ + --hash=sha256:58a0c478eeca76fe5e07993c5a0703def34a6dc6a0cda4f5564639b33112ffe7 \ + --hash=sha256:58ddb5a8e3179d12f19e4ea34d2d32e9d63a4baa142c875c1eb59f41b7243acd \ + --hash=sha256:630ebfea3bf689d075f82316324ff7433dc447fe6bc1bfc76524b74b4a9567d2 \ + --hash=sha256:6f8700550aa1474a91e5dc07049c46f98b423b5b1ddd0483e0b51362eeeaf5be \ + --hash=sha256:78198641e5be9521beea5aa782bb551a58068d10e6eb04c9c680c1b69f2e7d45 \ + --hash=sha256:79def8d059362e7831389ed3be0ecdf58a89386e1271e35dd9f5af84e81bffd0 \ + --hash=sha256:7a8701d6b584d76e909e3d305b7d126b41439876a5aaf76cddc67fc230eafa2e \ + --hash=sha256:7afa5a6602a9f29af1f3a2965f831bae7c9d5d597b7cbb716d41ab3b7d89879c \ + --hash=sha256:7b46165bb56eb4704e2eaaf86f3c940d19154535d9b0ca7d6d590b04060e00d5 \ + --hash=sha256:7b75de3c8b3be1cdb1052747c929440c3eea46c1bc2cb8a6e3a48388e9b7b452 \ + --hash=sha256:7c6d0330c472d96f6a6afe24d80dfdf15176c33096f0a4397ae4c60f3dd3be48 \ + --hash=sha256:828d49b0ff5a0e3975865571c5d91dbbdd0d38d8289b249a163e9425413a5e05 \ + --hash=sha256:84f964e537f916e2cc85199e5a88742e964939b575ac8598b3f9d6cc416cdaf1 \ + --hash=sha256:85d0d9a31b9098e98534226d5686b47264b95e62ce459dc2e62fdfc809f9fe93 \ + --hash=sha256:87e9ce85beb6b328ba370cc6e6aea483c92617b4c95b1d33a49297eb662bfb04 \ + --hash=sha256:8c71ba2cd31fc93748c38e1b613200ff1c2665cbfd5341fe3a61cfde35a1430e \ + --hash=sha256:92e665960f25fcdc73725b9cec7a3824f279ba97a98653afe9ffac2e43668f67 \ + --hash=sha256:94e5e9f108ee10471288214d3d233fbfbb492840a8457eb85178d643ddeb32c7 \ + --hash=sha256:9c8402a82ea0dc4ceeab793db05f0fafa8ca139ca34fcde5df0f596103c74107 \ + --hash=sha256:9dab55f57c74c3cad24c323bacbbd04be4705ba6eb0d92e920b1fc4837ed5079 \ + --hash=sha256:a582ab2ae1d34f67112cadc86702774c9ea4374df6bca6afe672817203c99134 \ + --hash=sha256:a6557e5f38e065ca9fbdaf7cfc7435ecb1d113aa81a022d1b51921ee7432e227 \ + --hash=sha256:a9f7355e6fab51f6c369b86fb7571cffa05edee2c2121e0380a37fb9ac1cd5c1 \ + --hash=sha256:ab50ee449bf968271e820086f10a33d101dd060370abc10bcd22279be2656539 \ + --hash=sha256:ac9ed99d81760c62fe89d5f0815cdfa1ba9a35141cf30f1c2d044f04b4803d2e \ + --hash=sha256:b13478603dcd0a2479ff8e87e2c19a7d525734686fe3c49542472293a204212d \ + --hash=sha256:c423ab384a46c4dff7217b2ea5ba2e11cffdeab6441acd04cf65a369caf0366c \ + --hash=sha256:c5e67125c7dca78d199ec4e116aa93dbb83494808ecbb8211a2cb09b1bf41dbd \ + --hash=sha256:c71be1cbfa5cd9a41ee452acf1eccd82b2c05950358b106ec8ceb83411d1a020 \ + --hash=sha256:cbc8738fd8526d80f35cb3a40d41f41a2e7030bb3b18b09a6778ef63d291c2fd \ + --hash=sha256:ce47f66801c20ec6c6632453bb5960fe38939e9306970b48b3a5a26de7745d94 \ + --hash=sha256:d370b8d1dfcdf7130178137f6fbee6140774a1acc6cacefc4b42643ec11d0a3a \ + --hash=sha256:d38cdff612d06fa6a32840d5e1b1f7a27cee4a349aa9085d94a67789d6bfd408 \ + --hash=sha256:d8947001be83df1394050758ce0e745dd74fb134eef0a4b5124208dfc3a68c37 \ + --hash=sha256:deb9fde5c60e437ee4821bc9bc39ff31b42135c27e1dc61ef0a629389c1de62e \ + --hash=sha256:dfe9763530994147d9af1def057a5b9658b00e8f8fe8743d144d1e0911c2e454 \ + --hash=sha256:e105ab60406787da31fccc883fc0f733af1efd78f0136a4599692c4083a73d0c \ + --hash=sha256:e275096ea1e60cc595cda2836fd4a6c725d1125108b868be17f53684d164e2cc \ + --hash=sha256:edc3342adf8f697fc5f59c887a304356f147b397809440ed64e2fa6af2f50f37 \ + --hash=sha256:ee247f5c245c9a2fe7c8e2214e295918838e44e00a45a6718451e4004219e767 \ + --hash=sha256:eef4c2f3423810b3070ab391f85436d2f8bbfcb286ac15cbc73190b3563b1f1a \ + --hash=sha256:f21e8a22c8605750c7af886bab299a363721264061b4ac0a30efb73cfd58efc5 \ + --hash=sha256:f265528741e048bce55c3463ed721fb0aa45a5888d8add8cfeccb3035451bbdc \ + --hash=sha256:f2f9bd7f90c64fe89253f0a2c05e3c4856072660429ce8831b4235bf29403a67 \ + --hash=sha256:f785f6161f202ab04d8ca194158968798e480ca058943907972da5f12e2881e8 \ + --hash=sha256:f9f6143a8c75945eb960d9eb98905a441394abfa24afaae239d514ffb2586480 \ + --hash=sha256:fa8f5efb344d6908a1ce62f4a24e2e5780f825d6f53f5f50ec5ffacac72936cb \ + --hash=sha256:fdd28f912fccfec1846a94e2e1e8f9b0012f557f0c46fe4f3eb0d7a87afcf90b + # via secretstorage +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e + # via readme-renderer +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.20 \ + --hash=sha256:a7db850025b95ded1eae8a46181a1a6c56c92c96f0e2b005d9ff8dc0210cab44 \ + --hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c + # via requests +jaraco-classes==3.4.0 \ + --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ + --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 + # via keyring +jaraco-context==6.1.2 \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 + # via keyring +jaraco-functools==4.6.0 \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 + # via keyring +jeepney==0.9.0 \ + --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ + --hash=sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732 + # via + # keyring + # secretstorage +keyring==25.7.0 \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b + # via twine +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a + # via rich +mdurl==0.1.2 \ + --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ + --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba + # via markdown-it-py +more-itertools==11.1.0 \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 + # via + # jaraco-classes + # jaraco-functools +nh3==0.3.7 \ + --hash=sha256:157ec1eb7a62f3d9a7badb8d82d89aa810e3e24e097eedfa481a25d0c8a99877 \ + --hash=sha256:15f5fbf090f5c88d61c820e1fc1fceecb6520cca9fe85649c06b57ef9dc9ff62 \ + --hash=sha256:18f4278ecd157d43cb35acd5aae9f35cfa79f546b4922bd86536adc0f6312102 \ + --hash=sha256:19f288c938ec6eef1f5d2c6cab47838e71fef8097e1c1233802be5a6230ba086 \ + --hash=sha256:4968fe8d2db97c6f047659bf46a449fd8ec377f44ebf3e0a1b96c0d3a333ae32 \ + --hash=sha256:5ffdfcb9a686ffb12765376bcfb6b5b55728516d3c0ee317d29982381ded3df8 \ + --hash=sha256:614dac4a4c36ad084e78447d16fe898dedd762e354a7ab9cda2984e82f67883d \ + --hash=sha256:618e3059caf41ccdf5dcccb3fa9df4cf6e4efe23d1382a8bbfca272a8a4f8bfc \ + --hash=sha256:6698a822132beedab80f131c08d8d0ac5a178ddeb488d02ca4b67716ecfac7af \ + --hash=sha256:6c3aa50eb26e9228238271db9f983cbc3b006dfbfeca2d4dc34c33ddc6ac5ea5 \ + --hash=sha256:6e4280115d44c3b278eef712a86748c1a723105cd79feec46952383117ab4e59 \ + --hash=sha256:70f5ac8626e899a4bab0ef74ca2f5bd602f49c7b739e6e5026b4afc6d63dac42 \ + --hash=sha256:71860d01c16f4d8c72e334e0674beb2b0899dbd0bf760de18932ef4390303848 \ + --hash=sha256:808def0c8c07843e6e50dc84f532457bfa2cfd17417b219a5d9e7c773709331a \ + --hash=sha256:874b7d67a067bd29a59223f6270fc30da4edd8e6d87fd219fc93bcbaa662c946 \ + --hash=sha256:91a4dab4e94d9fc54b9f67b1adfb23e81fab7ab43f33c3b8c97be9aa38f789ba \ + --hash=sha256:94fd6e59553fbb9ffd8ba71bbd5a54e3126ba01799a097ae30d5341d750bc6ac \ + --hash=sha256:9b7279d43323a25225df23576af6594a16693f61431170848b8b2ac21ad4f174 \ + --hash=sha256:bc42bb1193c1e28a1e74c2cabaca178e118a7103e8832699fef8a2b3e2496493 \ + --hash=sha256:be53a4825585f701955cb9baf49f478f56eb81e20294329fe4bc689dd5dd81fa \ + --hash=sha256:d56e76bd3cadb09b6b0cef364850811663734b348a25f5f587a2819c495367bd \ + --hash=sha256:de2b2aab32ea303405debefdcfc58043d3e635fa3f67b9eb140d2b0e0c0d2563 \ + --hash=sha256:e8fd1ab205258b29254f72db377d99e2c96aa7653ef3b015ccab0420b094b506 \ + --hash=sha256:eae64328e46a25785535afcb6885b6f182ecaf5ee8c88f8c075422db8aacc65b \ + --hash=sha256:f04b7d333b27f13ca439da3cf1c75c2fba34f104969f6ce4ac8e7079699c2f4a \ + --hash=sha256:f266d3f1b3647449923a8e406524632220dd5d8b647078dfe45b885d33d10479 \ + --hash=sha256:fd4a70efb45d5372174f718878eb7a35c12677626a63b2f103b23b833457dcac + # via readme-renderer +packaging==26.3 \ + --hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \ + --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c + # via + # build + # twine +pycparser==3.0 \ + --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ + --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 + # via cffi +pygments==2.21.0 \ + --hash=sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9 \ + --hash=sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c + # via + # readme-renderer + # rich +pyproject-hooks==1.3.3 \ + --hash=sha256:5fc53fdac9f7bd63fbcdc868fb5f90b4784d78a53a3d3388cd738b807441a20b \ + --hash=sha256:defda19b854fa0d3bd4f76ea4ddcba8abd7dcfcdd585a6690ade050744fc5f43 + # via build +readme-renderer==46.0 \ + --hash=sha256:af3e964914f6310a33ff67b72a4bdd940bed8d7c3bdecd2d14f40edf284bfe90 \ + --hash=sha256:d0dae1f74bb273b534770cb4cccb6bb78735540afdb03c2146f4e19dcd412560 + # via twine +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed + # via + # requests-toolbelt + # twine +requests-toolbelt==1.0.0 \ + --hash=sha256:7681a0a3d047012b5bdc0ee37d7f8f07ebe76ab08caeccfc3921ce23c88d5bc6 \ + --hash=sha256:cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06 + # via twine +rfc3986==2.0.0 \ + --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ + --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c + # via twine +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 + # via twine +secretstorage==3.5.0 \ + --hash=sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137 \ + --hash=sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be + # via keyring +setuptools==84.0.0 \ + --hash=sha256:51a52592b3b99e102b609654876bd65f19f999935166d1352678931132b0c670 \ + --hash=sha256:f4695c21257f0d9b537ec2692c941d02ee143b7cc1276941349a546573b2ef73 +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 +urllib3==2.8.0 \ + --hash=sha256:0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3 \ + --hash=sha256:63bf2ead4c879426ebf22ef2a781eeb4aa3b4ae798a0435506f8687fd5bb9b63 + # via + # id + # requests + # twine diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 00000000..c23bf676 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,535 @@ +name: release + +# --------------------------------------------------------------------------- +# PUBLISH openDox TO PyPI, BY TRUSTED PUBLISHING — plan 034 T099 (openxFactory +# `specs/034-opendox-standalone-operation/tasks.md`), on RULED openxFactory#656 +# comment 5962754358, item 1 (Brett Heap, 2026-10-02): "Publish to PyPI at the +# cut (Recommended)". Release 1's ruled install line is +# `pip install "opendox[local]"` (#1144 10.3, as T007's batch H addendum +# reads, R1Q15 (b) and R1Q16 (iii), comment 5850003126), and this workflow is +# what makes that line work as written. +# +# DISPATCH ONLY. A publish is an outward-facing act that Brett Heap approves, +# so nothing here runs on a push, a tag or a pull request. `version` must +# equal `pyproject.toml`'s `[project] version`, or the build job refuses +# before it builds anything. +# +# ONLY THE COMMIT THE openDox ROOT PINS (RULED openxFactory#656 comment +# 5963162921, with release 1's version, "0.1.0 (Recommended)"). The root's +# `contracts/code-pin.yaml` names the commit of this leg that the project IS, +# so the build job refuses any other commit. Release 1's order is: the bump of +# `version` to 0.1.0 lands last among phase 3's openDox-code landings, the +# openDox root pins that commit (plan 034 T087), and then this workflow is +# dispatched, on `main` while its head is still the pinned commit, or on a tag +# at that commit once `main` has moved on. +# +# FOUR JOBS, EACH GATED ON THE ONE BEFORE IT: +# build builds the sdist and the wheel in a clean job, runs +# `twine check`, VERIFIES both before any upload, and +# installs the wheel into a fresh venv. It holds no token. +# testpypi the dry run: publishes those exact bytes to TestPyPI, +# in the `testpypi` environment. +# testpypi-install installs `opendox[local]` from TestPyPI into a fresh +# venv, as plan 034's T099 falsifier does, and runs it. +# pypi publishes the same bytes to PyPI, in the `pypi` +# environment. +# +# NO TOKEN AND NO SECRET, ANYWHERE. Both uploads use PyPI's trusted publishing +# (OIDC): the job asks GitHub for a short-lived identity token, and the index +# accepts it only for the publisher configured on its side (owner `opensoft`, +# repository `openDox-code`, workflow `release.yml`, and the environment the +# job runs in). `id-token: write` is granted to the two publish jobs and to no +# other job. +# +# EVERY ACTION IS PINNED BY FULL COMMIT SHA, with its release tag beside it. +# (`validate.yml` pins by tag. A tag can be moved after review, and a commit +# cannot, which matters most in the one workflow whose output is published.) +# --------------------------------------------------------------------------- + +on: + workflow_dispatch: + inputs: + version: + description: >- + The version to publish. It must equal pyproject.toml's [project] + version, in PEP 440's normalized form, or the build job refuses. + required: true + type: string + +# Nothing by default. Each job grants itself exactly what it uses. +permissions: {} + +# One release at a time, and a running one is never cancelled halfway. +concurrency: + group: release + cancel-in-progress: false + +defaults: + run: + # `bash -eo pipefail`: a failed command anywhere in a pipeline fails the step. + shell: bash + +env: + # The input reaches every script through the environment, never through an + # expression pasted into a script, so no input value can become shell or + # Python source. + VERSION: ${{ inputs.version }} + PYTHONUTF8: '1' + +jobs: + build: + name: build and verify + runs-on: ubuntu-latest + timeout-minutes: 20 + permissions: + contents: read + actions: read # the step that reads both environments' protection rules + outputs: + wheel: ${{ steps.digests.outputs.wheel }} + wheel-sha256: ${{ steps.digests.outputs.wheel-sha256 }} + sdist: ${{ steps.digests.outputs.sdist }} + sdist-sha256: ${{ steps.digests.outputs.sdist-sha256 }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: '3.12' + + # THE RELEASE IS THE PINNED COMMIT. This run's commit must equal + # `commit:` in opensoft/openDox `main`'s `contracts/code-pin.yaml`, whose + # `source_repository` must be this repository, and it must be on this + # repository's `main`. A dispatch may name any branch or tag, and the + # commit it names is what would be published, so this is checked first. + # Both repositories are public, and both reads go through the API. + - name: the dispatched commit is the one the openDox root pins + env: + GH_TOKEN: ${{ github.token }} + run: | + if ! pin="$(gh api 'repos/opensoft/openDox/contents/contracts/code-pin.yaml?ref=main' \ + -H 'Accept: application/vnd.github.raw')"; then + echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" + exit 1 + fi + PIN="$pin" python - <<'PY' + import os, re, sys + + pin = os.environ["PIN"] + + def field(name): + found = re.search(rf'^{name}:[ \t]*"?([^"\s#]+)"?[ \t]*(?:#.*)?$', pin, re.M) + return found.group(1) if found else None + + source, pinned = field("source_repository"), field("commit") + here, this = os.environ["GITHUB_REPOSITORY"], os.environ["GITHUB_SHA"] + if source != here: + sys.exit(f"::error::the openDox root's code pin names {source!r}, not {here!r}") + if not pinned or not re.fullmatch(r"[0-9a-f]{40}", pinned): + sys.exit(f"::error::the openDox root's code pin names no full commit: {pinned!r}") + if pinned != this: + sys.exit(f"::error::this run is at {this}, and the openDox root pins {pinned}; " + "a release publishes only the pinned commit") + print(f"{this} is the commit opensoft/openDox main pins for {here}") + PY + status="$(gh api "repos/$GITHUB_REPOSITORY/compare/$GITHUB_SHA...main" --jq .status)" + case "$status" in + identical|ahead) echo "$GITHUB_SHA is on main ($status), dispatched on $GITHUB_REF" ;; + *) + echo "::error::$GITHUB_SHA is not on this repository's main (compare: $status)" + exit 1 + ;; + esac + + # BOTH ENVIRONMENTS MUST ASK A REVIEWER. GitHub creates an environment a + # job names if it does not exist yet, and creates it with no protection + # rule, so a dispatch made before Brett configured `pypi` would publish + # with nobody approving it. This step refuses that case, before anything + # is built. The environments are public metadata of a public repository. + - name: both environments ask a reviewer before they deploy + env: + GH_TOKEN: ${{ github.token }} + run: | + rc=0 + for environment in testpypi pypi; do + if ! reviewers=$(gh api "repos/$GITHUB_REPOSITORY/environments/$environment" \ + --jq '[.protection_rules[] | select(.type == "required_reviewers") | .reviewers[]] | length'); then + echo "::error::the $environment environment cannot be read (it does not exist yet, or this job cannot see it); create it with a required reviewer first" + rc=1 + continue + fi + if [ "$reviewers" -lt 1 ]; then + echo "::error::the $environment environment names no required reviewer, so a deployment to it would not wait for an approval" + rc=1 + else + echo "$environment: $reviewers required reviewer(s)" + fi + done + exit $rc + + # THE RELEASE TOOLS, FROM THEIR HASH LOCK, IN A VENV OF THEIR OWN. The + # build below runs with `--no-isolation` in this venv, so its backend is + # the hash-locked `setuptools` and not whatever an index serves today + # (`.github/release-tools-cpython312-linux.txt` says why). + - name: install the release tools from their hash lock + run: | + python -m venv "$RUNNER_TEMP/tools" + "$RUNNER_TEMP/tools/bin/python" -m pip install --disable-pip-version-check \ + --only-binary :all: --require-hashes \ + -r .github/release-tools-cpython312-linux.txt + + # THE VERSION GATE. The input must be a PEP 440 version in normalized + # form (the form the files are named by), must equal what + # `pyproject.toml` declares, and must not be `0.0.0`, the scaffold's + # placeholder, which is never a release. + - name: the version input is pyproject's version + run: | + "$RUNNER_TEMP/tools/bin/python" - <<'PY' + import os, sys, tomllib + from packaging.version import InvalidVersion, Version + + asked = os.environ["VERSION"] + with open("pyproject.toml", "rb") as handle: + declared = tomllib.load(handle)["project"]["version"] + try: + parsed = Version(asked) + except InvalidVersion: + sys.exit(f"::error::{asked!r} is not a PEP 440 version") + if str(parsed) != asked: + sys.exit(f"::error::{asked!r} is not in PEP 440's normalized form, " + f"which is {str(parsed)!r}; the files would be named by that") + if parsed.local is not None: + sys.exit(f"::error::{asked!r} carries a local label, which no index accepts") + if parsed == Version("0.0.0"): + sys.exit("::error::0.0.0 is the scaffold's placeholder version, never a release") + if asked != declared: + sys.exit(f"::error::the input names {asked!r}, and pyproject.toml " + f"declares {declared!r}; they must be equal") + print(f"version {asked}: pyproject.toml declares it") + PY + + # `python -m build` builds the sdist, then builds the wheel FROM THAT + # SDIST, so a file the sdist lost is missing from the wheel too, and the + # wheel's checks below see it. `SOURCE_DATE_EPOCH` is the commit's time, + # so the wheel's bytes depend on the commit and not on the clock. + - name: build the sdist and the wheel + run: | + SOURCE_DATE_EPOCH="$(git log -1 --format=%ct HEAD)" + export SOURCE_DATE_EPOCH + "$RUNNER_TEMP/tools/bin/python" -m build --no-isolation --outdir dist . + + # `--strict` makes a warning a refusal, such as a long description that + # is missing (pyproject's `readme` supplies it) or does not render. + - name: twine check + run: | + "$RUNNER_TEMP/tools/bin/python" -m twine check --strict dist/* + + # THE ARTIFACT CHECKS, BEFORE ANY UPLOAD. Each prints what it read. + # 1. dist/ holds exactly the sdist and the wheel named for VERSION; + # 2. the wheel's metadata names `opendox` at VERSION; + # 3. the `local` extra is declared, with its requirements (T072); + # 4. the console script `opendox` is declared, as `opendox.cli:main`; + # 5. every file under `src/opendox/web/` that git tracks is in the + # wheel, dotfiles included (T075's `web/**/.*`): 42 of 42 at + # openDox-code#73's head; + # 6. every other file git tracks under `src/` is in the wheel, at its + # import path (the package, `opendox.contracts`' packaged copies, + # and the two top-level modules `route_extension` and + # `subcommand_extension`), less `src/.gitkeep`, which is no + # package's; and the wheel carries nothing else but its + # `.dist-info` and its `.data` directory; + # 7. every `migrations/*.sql` git tracks is in the wheel's data + # directory, `share/opendox/migrations/` (T072). + # The tracked tree is the reference, so a bundle file added later is + # checked with no edit here. + - name: verify the artifacts + run: | + "$RUNNER_TEMP/tools/bin/python" - <<'PY' + import configparser, email.parser, os, pathlib, subprocess, sys, zipfile + + version = os.environ["VERSION"] + wheel = f"opendox-{version}-py3-none-any.whl" + sdist = f"opendox-{version}.tar.gz" + found = sorted(p.name for p in pathlib.Path("dist").iterdir()) + if found != sorted([sdist, wheel]): + sys.exit(f"::error::dist/ holds {found}; a release is exactly {sorted([sdist, wheel])}") + print(f"1. dist/: {found}") + + def tracked(*paths): + out = subprocess.run(["git", "ls-files", "-z", "--", *paths], + capture_output=True, text=True, check=True).stdout + return {p for p in out.split("\0") if p} + + problems = [] + with zipfile.ZipFile(pathlib.Path("dist") / wheel) as archive: + names = {n for n in archive.namelist() if not n.endswith("/")} + info = f"opendox-{version}.dist-info" + data = f"opendox-{version}.data/data" + + meta = email.parser.Parser().parsestr( + archive.read(f"{info}/METADATA").decode("utf-8")) + print(f"2. metadata: Name {meta['Name']}, Version {meta['Version']}") + if meta["Name"] != "opendox" or meta["Version"] != version: + problems.append(f"the wheel's metadata names {meta['Name']} {meta['Version']}, " + f"not opendox {version}") + + extras = meta.get_all("Provides-Extra") or [] + local = [r for r in meta.get_all("Requires-Dist") or [] + if r.replace(" ", "").endswith('extra=="local"')] + print(f"3. extras: {extras}; the local extra requires {local}") + if "local" not in extras or not local: + problems.append("the `local` extra is not declared with its requirements") + + entry_points = configparser.ConfigParser(delimiters=("=",)) + entry_points.optionxform = str + entry_points.read_string(archive.read(f"{info}/entry_points.txt").decode("utf-8")) + scripts = dict(entry_points["console_scripts"]) if entry_points.has_section( + "console_scripts") else {} + print(f"4. console scripts: {scripts}") + if scripts.get("opendox") != "opendox.cli:main": + problems.append("the console script `opendox = opendox.cli:main` is not declared") + + web = {p.removeprefix("src/") for p in tracked("src/opendox/web")} + web_missing = sorted(web - names) + print(f"5. web bundle: {len(web) - len(web_missing)} of {len(web)} tracked files " + f"are in the wheel") + if not web or web_missing: + problems.append(f"the wheel lacks web files: {web_missing}") + + source = {p.removeprefix("src/") for p in tracked("src")} - {".gitkeep"} + source_missing = sorted(source - names) + stray = sorted(n for n in names - source + if not n.startswith((f"{info}/", f"{data}/"))) + print(f"6. source tree: {len(source) - len(source_missing)} of {len(source)} " + f"tracked files under src/ are in the wheel; {len(stray)} other entries") + if source_missing: + problems.append(f"the wheel lacks tracked files: {source_missing}") + if stray: + problems.append(f"the wheel carries files the tree does not track: {stray}") + + migrations = {f"{data}/share/opendox/{p}" for p in tracked("migrations/*.sql")} + migrations_missing = sorted(migrations - names) + print(f"7. migrations: {len(migrations) - len(migrations_missing)} of " + f"{len(migrations)} are in the wheel's data directory") + if not migrations or migrations_missing: + problems.append(f"the wheel lacks migrations: {migrations_missing}") + + for problem in problems: + print(f"::error::{problem}") + if problems: + sys.exit(1) + print("the artifacts carry what an install runs from") + PY + + # THE WHEEL INSTALLS, AND RUNS, OUTSIDE THE CHECKOUT. A fresh venv, + # `opendox[local]` from the built file (10.3's line, from a file), the + # dependencies through this package's own lock, and every command run + # from `$RUNNER_TEMP`, so nothing can import from `src/`. + - name: the built wheel installs into a fresh venv, and opendox --help works + run: | + python -m venv "$RUNNER_TEMP/fresh" + "$RUNNER_TEMP/fresh/bin/python" -m pip install --disable-pip-version-check \ + --only-binary :all: -c constraints-cpython312-linux.txt \ + "$PWD/dist/opendox-$VERSION-py3-none-any.whl[local]" + cd "$RUNNER_TEMP" + "$RUNNER_TEMP/fresh/bin/python" - <<'PY' + import importlib.metadata, os, pathlib, sys + import opendox + where = pathlib.Path(opendox.__file__).resolve() + if pathlib.Path(sys.prefix).resolve() not in where.parents: + sys.exit(f"::error::opendox imported from {where}, outside the fresh venv") + if importlib.metadata.version("opendox") != os.environ["VERSION"]: + sys.exit("::error::the installed opendox is not the version being released") + print(f"opendox {importlib.metadata.version('opendox')} imports from {where.parent}") + PY + "$RUNNER_TEMP/fresh/bin/opendox" --help + "$RUNNER_TEMP/fresh/bin/opendox" generate-and-open --help > help.txt + if ! grep -q -- '--local' help.txt; then + echo "::error::opendox generate-and-open has no --local, so 10.3's command does not parse" + exit 1 + fi + echo "opendox generate-and-open --help names --local" + + - name: record the files and their digests + id: digests + run: | + wheel="opendox-$VERSION-py3-none-any.whl" + sdist="opendox-$VERSION.tar.gz" + { + echo "wheel=$wheel" + echo "wheel-sha256=$(sha256sum "dist/$wheel" | cut -d' ' -f1)" + echo "sdist=$sdist" + echo "sdist-sha256=$(sha256sum "dist/$sdist" | cut -d' ' -f1)" + } | tee -a "$GITHUB_OUTPUT" + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: dist + path: dist/ + if-no-files-found: error + + testpypi: + name: publish to TestPyPI (the dry run) + needs: build + runs-on: ubuntu-latest + timeout-minutes: 15 + environment: + name: testpypi + url: https://test.pypi.org/project/opendox/${{ inputs.version }}/ + permissions: + id-token: write # trusted publishing: the OIDC token TestPyPI accepts + steps: + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: dist + path: dist + + # The files about to be uploaded are the two the build job verified, + # byte for byte, and nothing else. + - name: the files are the ones the build job verified + env: + WHEEL: ${{ needs.build.outputs.wheel }} + WHEEL_SHA256: ${{ needs.build.outputs.wheel-sha256 }} + SDIST: ${{ needs.build.outputs.sdist }} + SDIST_SHA256: ${{ needs.build.outputs.sdist-sha256 }} + run: | + found="$(find dist -mindepth 1 -maxdepth 1 -printf '%f\n' | sort | tr '\n' ' ')" + wanted="$(printf '%s\n%s\n' "$SDIST" "$WHEEL" | sort | tr '\n' ' ')" + if [ "$found" != "$wanted" ]; then + echo "::error::dist/ holds $found, and the build job verified $wanted" + exit 1 + fi + printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ + | sha256sum --strict -c - + + - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 + with: + repository-url: https://test.pypi.org/legacy/ + + testpypi-install: + name: install opendox[local] from TestPyPI + needs: [build, testpypi] + runs-on: ubuntu-latest + timeout-minutes: 20 + permissions: + contents: read + env: + WHEEL: ${{ needs.build.outputs.wheel }} + WHEEL_SHA256: ${{ needs.build.outputs.wheel-sha256 }} + SDIST: ${{ needs.build.outputs.sdist }} + SDIST_SHA256: ${{ needs.build.outputs.sdist-sha256 }} + steps: + # For `constraints-cpython312-linux.txt` alone. + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: '3.12' + + # TestPyPI serves exactly the two files the build job verified. Its JSON + # API can lag the upload by a little, so it is read up to 20 times, 15 + # seconds apart, before this step refuses. + - name: TestPyPI serves the files the build job verified + run: | + python - <<'PY' + import json, os, sys, time, urllib.error, urllib.request + + url = f"https://test.pypi.org/pypi/opendox/{os.environ['VERSION']}/json" + for attempt in range(1, 21): + try: + with urllib.request.urlopen(url, timeout=30) as response: + release = json.load(response) + break + except urllib.error.URLError as error: + code = getattr(error, "code", None) + if code is not None and code != 404 and code < 500: + raise + print(f"attempt {attempt}: {url} did not answer yet ({error})") + time.sleep(15) + else: + sys.exit(f"::error::{url} never answered") + served = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} + wanted = {os.environ["WHEEL"]: os.environ["WHEEL_SHA256"], + os.environ["SDIST"]: os.environ["SDIST_SHA256"]} + print(f"TestPyPI serves {served}") + if served != wanted: + sys.exit(f"::error::TestPyPI serves {served}, and the build job verified {wanted}") + PY + + # T099's DRY-RUN FALSIFIER, as plan 034 writes it, in a fresh venv: + # `pip install --index-url https://test.pypi.org/simple/ + # --extra-index-url https://pypi.org/simple/ "opendox[local]"`, here + # pinned to the version just uploaded, with this package's lock for its + # dependencies, and wheels only. The simple index can lag the JSON API, + # so the install is tried up to ten times, 30 seconds apart. + # + # THE TRADE, stated: with two indexes, pip may take a dependency from + # either. The lock pins every dependency's version, and this job holds no + # token, no OIDC grant and no secret, and publishes nothing. What it can + # spoil is its own verdict, and the files PyPI receives are still the + # build job's, checked by digest in the `pypi` job. + - name: install opendox[local] from TestPyPI into a fresh venv, and run it + run: | + python -m venv "$RUNNER_TEMP/fresh" + installed=0 + for attempt in 1 2 3 4 5 6 7 8 9 10; do + if "$RUNNER_TEMP/fresh/bin/python" -m pip install --disable-pip-version-check \ + --no-cache-dir --only-binary :all: -c constraints-cpython312-linux.txt \ + --index-url https://test.pypi.org/simple/ \ + --extra-index-url https://pypi.org/simple/ \ + "opendox[local]==$VERSION"; then + installed=1 + break + fi + echo "attempt $attempt: the install did not complete; retrying in 30 seconds" + sleep 30 + done + if [ "$installed" != 1 ]; then + echo "::error::opendox[local]==$VERSION did not install from TestPyPI" + exit 1 + fi + cd "$RUNNER_TEMP" + "$RUNNER_TEMP/fresh/bin/python" - <<'PY' + import importlib.metadata, os, sys + if importlib.metadata.version("opendox") != os.environ["VERSION"]: + sys.exit("::error::the installed opendox is not the version just uploaded") + print(f"opendox {importlib.metadata.version('opendox')} installed from TestPyPI") + PY + "$RUNNER_TEMP/fresh/bin/opendox" --help + + pypi: + name: publish to PyPI + needs: [build, testpypi-install] + runs-on: ubuntu-latest + timeout-minutes: 15 + environment: + name: pypi + url: https://pypi.org/project/opendox/${{ inputs.version }}/ + permissions: + id-token: write # trusted publishing: the OIDC token PyPI accepts + steps: + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: dist + path: dist + + - name: the files are the ones the build job verified + env: + WHEEL: ${{ needs.build.outputs.wheel }} + WHEEL_SHA256: ${{ needs.build.outputs.wheel-sha256 }} + SDIST: ${{ needs.build.outputs.sdist }} + SDIST_SHA256: ${{ needs.build.outputs.sdist-sha256 }} + run: | + found="$(find dist -mindepth 1 -maxdepth 1 -printf '%f\n' | sort | tr '\n' ' ')" + wanted="$(printf '%s\n%s\n' "$SDIST" "$WHEEL" | sort | tr '\n' ' ')" + if [ "$found" != "$wanted" ]; then + echo "::error::dist/ holds $found, and the build job verified $wanted" + exit 1 + fi + printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ + | sha256sum --strict -c - + + - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 diff --git a/pyproject.toml b/pyproject.toml index 880ab584..7d066f4e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -18,6 +18,12 @@ name = "opendox" version = "0.0.0" description = "The openDox code leg — the `opendox` package, carved from openxFactory." requires-python = ">=3.12" +# THE PROJECT PAGE ON PyPI (plan 034 T099; RULED openxFactory#656 comment +# 5962754358, item 1, "Publish to PyPI at the cut"). Without it the metadata +# carries no long description, `twine check --strict` refuses both files, and +# the published page is blank. `.github/workflows/release.yml` runs that check +# before any upload. The `.md` suffix makes setuptools declare `text/markdown`. +readme = "README.md" # THE RUNTIME DEPENDENCY SET, round 2 of the levelling pass — RULED Q-L2 # (openxFactory#656, Brett Heap, 2026-09-10: "rule Q-L1 approve, Q-L2 approve, From 647d246e3925df6a5852d24c953195e226f9f047 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 00:15:03 +0000 Subject: [PATCH 02/15] Fix round 1: the full requirement sets, no epoch, the installed closure, the release tag, the install line as written, and tests (Copilot review) Copilot's review of openDox-code#78 at 4173dbc9, and of openxFactory#1220 at 00a5bdae, with the holder's ruling on the split (T101): - The artifact checks compare the wheel's requirements with pyproject.toml's, for the base install and every extra, as normalized requirements, and require the `local` extra to carry opendox[runtime] and pixeltable-pgserver (r4170775660). - The fresh venv walks the whole requirement closure of opendox[local] from the installed metadata, extras and markers included, and runs the bundled server's `initdb --version` and `postgres --version` through opendox.runtime.bundle.server_binaries() (r4170775660). - The version gate refuses an epoch, whose `!` a wheel's file name escapes (r4170775696). - The release-commit step never requires main's head: the commit must equal the openDox root's pin and be on main, and a dispatch on a tag must name v (the holder's ruling). - testpypi-install runs the falsifier's install line as written, unversioned, then requires the installed version to be the one uploaded, each try in a fresh venv (openxFactory#1220, r4170767305). - tests/test_release_workflow.py: 39 hermetic cases that run the steps' own scripts, with a stand-in gh, over a hand-built wheel and sdist (r4170775717). validate.yml's floors are not moved here: they permit a rise, and validate.yml is T095's single-writer file (openDox-code#75). Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 170 +++++++++++--- tests/test_release_workflow.py | 414 +++++++++++++++++++++++++++++++++ 2 files changed, 550 insertions(+), 34 deletions(-) create mode 100644 tests/test_release_workflow.py diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c23bf676..5985e35b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -20,8 +20,10 @@ name: release # so the build job refuses any other commit. Release 1's order is: the bump of # `version` to 0.1.0 lands last among phase 3's openDox-code landings, the # openDox root pins that commit (plan 034 T087), and then this workflow is -# dispatched, on `main` while its head is still the pinned commit, or on a tag -# at that commit once `main` has moved on. +# dispatched at the cut. `main`'s head may have moved past the pinned commit by +# then (T095 lands later), so the head is never required: the run's commit +# must equal the root's pin and be on `main`. The dispatch names `main`, or +# the tag `v` created at the pinned commit, and no other tag. # # FOUR JOBS, EACH GATED ON THE ONE BEFORE IT: # build builds the sdist and the wheel in a clean job, runs @@ -98,16 +100,26 @@ jobs: with: python-version: '3.12' - # THE RELEASE IS THE PINNED COMMIT. This run's commit must equal + # THE RELEASE IS THE PINNED COMMIT. This run's commit must (a) equal # `commit:` in opensoft/openDox `main`'s `contracts/code-pin.yaml`, whose - # `source_repository` must be this repository, and it must be on this - # repository's `main`. A dispatch may name any branch or tag, and the - # commit it names is what would be published, so this is checked first. - # Both repositories are public, and both reads go through the API. + # `source_repository` must be this repository, and (b) be on this + # repository's `main`, as `main` or an ancestor of it. `main`'s head + # need not be that commit. A dispatch on a tag must name `v`. + # A dispatch may name any ref, and the commit it names is what would be + # published, so this is checked first. Both repositories are public, and + # both reads go through the API. - name: the dispatched commit is the one the openDox root pins env: GH_TOKEN: ${{ github.token }} run: | + case "$GITHUB_REF" in + refs/tags/*) + if [ "$GITHUB_REF" != "refs/tags/v$VERSION" ]; then + echo "::error::a release dispatched on a tag names v$VERSION, and this run names $GITHUB_REF" + exit 1 + fi + ;; + esac if ! pin="$(gh api 'repos/opensoft/openDox/contents/contracts/code-pin.yaml?ref=main' \ -H 'Accept: application/vnd.github.raw')"; then echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" @@ -180,9 +192,12 @@ jobs: -r .github/release-tools-cpython312-linux.txt # THE VERSION GATE. The input must be a PEP 440 version in normalized - # form (the form the files are named by), must equal what - # `pyproject.toml` declares, and must not be `0.0.0`, the scaffold's - # placeholder, which is never a release. + # form (the form the files are named by), with no epoch and no local + # label, must equal what `pyproject.toml` declares, and must not be + # `0.0.0`, the scaffold's placeholder, which is never a release. An + # epoch (`1!2.0`) is refused because a wheel's file name escapes its `!`, + # so the files would not carry the version as written, and every check + # below names them by it. - name: the version input is pyproject's version run: | "$RUNNER_TEMP/tools/bin/python" - <<'PY' @@ -201,6 +216,9 @@ jobs: f"which is {str(parsed)!r}; the files would be named by that") if parsed.local is not None: sys.exit(f"::error::{asked!r} carries a local label, which no index accepts") + if parsed.epoch: + sys.exit(f"::error::{asked!r} carries an epoch, which a wheel's file name " + "escapes; this workflow names the files by the version as written") if parsed == Version("0.0.0"): sys.exit("::error::0.0.0 is the scaffold's placeholder version, never a release") if asked != declared: @@ -228,7 +246,10 @@ jobs: # THE ARTIFACT CHECKS, BEFORE ANY UPLOAD. Each prints what it read. # 1. dist/ holds exactly the sdist and the wheel named for VERSION; # 2. the wheel's metadata names `opendox` at VERSION; - # 3. the `local` extra is declared, with its requirements (T072); + # 3. the wheel declares exactly the requirements `pyproject.toml` does, + # for the base install and for every extra, compared as normalized + # requirements; and its `local` extra (T072) carries + # `opendox[runtime]` and the bundled server's package; # 4. the console script `opendox` is declared, as `opendox.cli:main`; # 5. every file under `src/opendox/web/` that git tracks is in the # wheel, dotfiles included (T075's `web/**/.*`): 42 of 42 at @@ -246,7 +267,9 @@ jobs: - name: verify the artifacts run: | "$RUNNER_TEMP/tools/bin/python" - <<'PY' - import configparser, email.parser, os, pathlib, subprocess, sys, zipfile + import configparser, email.parser, os, pathlib, re, subprocess, sys, tomllib, zipfile + from packaging.requirements import Requirement + from packaging.utils import canonicalize_name version = os.environ["VERSION"] wheel = f"opendox-{version}-py3-none-any.whl" @@ -274,12 +297,43 @@ jobs: problems.append(f"the wheel's metadata names {meta['Name']} {meta['Version']}, " f"not opendox {version}") - extras = meta.get_all("Provides-Extra") or [] - local = [r for r in meta.get_all("Requires-Dist") or [] - if r.replace(" ", "").endswith('extra=="local"')] - print(f"3. extras: {extras}; the local extra requires {local}") - if "local" not in extras or not local: - problems.append("the `local` extra is not declared with its requirements") + def named(requirement): + extras = ",".join(sorted(canonicalize_name(e) for e in requirement.extras)) + return canonicalize_name(requirement.name) + (f"[{extras}]" if extras else "") + + def normal(requirement): + return named(requirement) + str(requirement.specifier) + + with open("pyproject.toml", "rb") as handle: + project = tomllib.load(handle)["project"] + declared = {"": {normal(Requirement(r)) for r in project.get("dependencies", [])}} + for extra, requirements in project.get("optional-dependencies", {}).items(): + declared[canonicalize_name(extra)] = {normal(Requirement(r)) for r in requirements} + carried = {""} | {canonicalize_name(e) for e in meta.get_all("Provides-Extra") or []} + carried = {extra: set() for extra in carried} + local_names = set() + for raw in meta.get_all("Requires-Dist") or []: + requirement = Requirement(raw) + # The extra a requirement belongs to is read from its marker's + # text, so a platform marker beside it changes nothing. + owners = sorted({canonicalize_name(e) for e in + re.findall(r'extra\s*==\s*"([^"]+)"', str(requirement.marker or ""))}) + if any(owner not in carried for owner in owners): + problems.append(f"the requirement {raw!r} names an extra the wheel does not provide") + continue + requirement.marker = None + for owner in owners or [""]: + carried[owner].add(normal(requirement)) + if owner == "local": + local_names.add(named(requirement)) + print(f"3. requirements: the local extra carries {sorted(carried.get('local', ()))}") + if carried != declared: + problems.append(f"the wheel's requirements {carried} are not " + f"pyproject.toml's {declared}") + if not {"opendox[runtime]", "pixeltable-pgserver"} <= local_names: + problems.append(f"the `local` extra carries {sorted(local_names)}, not " + "opendox[runtime] and the bundled server's package, " + "pixeltable-pgserver") entry_points = configparser.ConfigParser(delimiters=("=",)) entry_points.optionxform = str @@ -343,6 +397,51 @@ jobs: sys.exit("::error::the installed opendox is not the version being released") print(f"opendox {importlib.metadata.version('opendox')} imports from {where.parent}") PY + # EVERY REQUIREMENT OF `opendox[local]` IS INSTALLED AND SATISFIED, + # walked from the wheel's own metadata, extras and markers included, + # with the tools' `packaging` over the fresh venv's distributions. + "$RUNNER_TEMP/tools/bin/python" - "$("$RUNNER_TEMP/fresh/bin/python" -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')" <<'PY' + import importlib.metadata, sys + from packaging.requirements import Requirement + from packaging.utils import canonicalize_name + + installed = {canonicalize_name(d.metadata["Name"]): d + for d in importlib.metadata.distributions(path=[sys.argv[1]])} + problems, seen = [], set() + + def need(name, extras): + key = (canonicalize_name(name), frozenset(extras)) + if key in seen: + return + seen.add(key) + for raw in installed[key[0]].requires or []: + requirement = Requirement(raw) + if requirement.marker and not any( + requirement.marker.evaluate({"extra": extra}) + for extra in (sorted(extras) or [""])): + continue + dependency = installed.get(canonicalize_name(requirement.name)) + if dependency is None: + problems.append(f"{raw} is not installed") + elif not requirement.specifier.contains(dependency.version, prereleases=True): + problems.append(f"{raw} is not satisfied by {dependency.version}") + else: + need(requirement.name, requirement.extras) + + need("opendox", {"local"}) + for problem in problems: + print(f"::error::{problem}") + if problems: + sys.exit(1) + print(f"opendox[local]: {len({name for name, _ in seen})} distributions, " + "every requirement installed and satisfied") + PY + # THE BUNDLED SERVER ARRIVED WITH THE EXTRA, and its binaries run: + # the product's own locator (`opendox.runtime.bundle`, T072), then + # each binary's `--version`. + for binary in initdb postgres; do + "$("$RUNNER_TEMP/fresh/bin/python" -c 'from opendox.runtime.bundle import server_binaries; print(server_binaries())')/$binary" --version + done "$RUNNER_TEMP/fresh/bin/opendox" --help "$RUNNER_TEMP/fresh/bin/opendox" generate-and-open --help > help.txt if ! grep -q -- '--local' help.txt; then @@ -461,10 +560,13 @@ jobs: # T099's DRY-RUN FALSIFIER, as plan 034 writes it, in a fresh venv: # `pip install --index-url https://test.pypi.org/simple/ - # --extra-index-url https://pypi.org/simple/ "opendox[local]"`, here - # pinned to the version just uploaded, with this package's lock for its - # dependencies, and wheels only. The simple index can lag the JSON API, - # so the install is tried up to ten times, 30 seconds apart. + # --extra-index-url https://pypi.org/simple/ "opendox[local]"`. The + # requirement is the install line as written, unversioned. The step adds + # this package's lock for the dependencies and wheels only, and then + # requires the installed opendox to be the version just uploaded, so an + # index that lags, or an older release, cannot pass for it. The simple + # index can lag the JSON API, so each try is a fresh venv, up to ten + # tries, 30 seconds apart. # # THE TRADE, stated: with two indexes, pip may take a dependency from # either. The lock pins every dependency's version, and this job holds no @@ -473,31 +575,31 @@ jobs: # build job's, checked by digest in the `pypi` job. - name: install opendox[local] from TestPyPI into a fresh venv, and run it run: | - python -m venv "$RUNNER_TEMP/fresh" installed=0 for attempt in 1 2 3 4 5 6 7 8 9 10; do + rm -rf "$RUNNER_TEMP/fresh" + python -m venv "$RUNNER_TEMP/fresh" if "$RUNNER_TEMP/fresh/bin/python" -m pip install --disable-pip-version-check \ --no-cache-dir --only-binary :all: -c constraints-cpython312-linux.txt \ --index-url https://test.pypi.org/simple/ \ --extra-index-url https://pypi.org/simple/ \ - "opendox[local]==$VERSION"; then - installed=1 - break + "opendox[local]"; then + got="$("$RUNNER_TEMP/fresh/bin/python" -c 'import importlib.metadata; print(importlib.metadata.version("opendox"))')" + if [ "$got" = "$VERSION" ]; then + installed=1 + break + fi + echo "attempt $attempt: the install line resolved opendox $got, not $VERSION" fi - echo "attempt $attempt: the install did not complete; retrying in 30 seconds" + echo "attempt $attempt: opendox $VERSION is not installed yet; retrying in 30 seconds" sleep 30 done if [ "$installed" != 1 ]; then - echo "::error::opendox[local]==$VERSION did not install from TestPyPI" + echo "::error::the install line did not install opendox $VERSION from TestPyPI" exit 1 fi cd "$RUNNER_TEMP" - "$RUNNER_TEMP/fresh/bin/python" - <<'PY' - import importlib.metadata, os, sys - if importlib.metadata.version("opendox") != os.environ["VERSION"]: - sys.exit("::error::the installed opendox is not the version just uploaded") - print(f"opendox {importlib.metadata.version('opendox')} installed from TestPyPI") - PY + echo "opendox $VERSION installed from TestPyPI by the install line as written" "$RUNNER_TEMP/fresh/bin/opendox" --help pypi: diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py new file mode 100644 index 00000000..4bfad80b --- /dev/null +++ b/tests/test_release_workflow.py @@ -0,0 +1,414 @@ +"""`.github/workflows/release.yml`, held to the gates it promises (plan 034 T099). + +The release workflow publishes openDox to PyPI by trusted publishing (RULED +openxFactory#656 comment 5962754358, item 1), and it publishes only the commit +the openDox root pins (RULED 5963162921). Nothing executes a workflow until it +is dispatched, and a dispatch is a publish, so every gate it carries would +otherwise go unexercised until the one run that cannot be taken back. Copilot's +review of openDox-code#78 asked for this file. + +SO THE STEPS' OWN SCRIPTS ARE WHAT RUN HERE, extracted from the workflow rather +than restated, as `tests/test_triple_pin.py` runs `validate.yml`'s pin. A copy +of a script in a test is a test of the copy. + +EVERY CASE IS HERMETIC. The two steps that call `gh` run with a stand-in `gh` +first on `PATH`, which answers from the case's own fixture and records nothing, +so no case reaches GitHub (and `tests/hermeticity.py`'s refusal shim stays the +answer for any other `gh`). No case reaches an index. The one step that reads +TestPyPI over HTTP is not run here; it runs at the dry run. + +What is held: + * the shape: dispatch only, one `version` input, no secret, `id-token: write` + on the two publish jobs alone, each in its own environment, every action + pinned by a full commit SHA, and the four jobs chained; + * the release-commit step: the pinned commit on `main` passes, and a commit + the openDox root does not pin, a commit off `main`, another repository's + pin, a tag other than `v` and an unreadable pin each refuse; + * the environment step: both environments with a reviewer pass, and either + one without a reviewer, or unreadable, refuses; + * the version gate, case by case; + * the artifact checks, over a small built-by-hand wheel and sdist in a git + tree of their own: the whole set passes, and each kind of gap refuses; + * the publish jobs' digest check: the verified files pass, and a changed + byte or a third file refuses. +""" + +from __future__ import annotations + +import hashlib +import os +import re +import shutil +import subprocess +import sys +import zipfile +from pathlib import Path + +import pytest +import yaml + +ROOT = Path(__file__).resolve().parents[1] +WORKFLOW = ROOT / ".github" / "workflows" / "release.yml" +REPOSITORY = "opensoft/openDox-code" +PINNED = "047bb4fa394f3e1bf42466062a67ef18e99f8d6a" + +needs_a_shell = pytest.mark.skipif( + shutil.which("bash") is None or shutil.which("git") is None, + reason="the steps are bash scripts, and the artifact checks read git") + + +def _workflow() -> dict: + return yaml.safe_load(WORKFLOW.read_text(encoding="utf-8")) + + +def _step(job: str, name: str) -> dict: + for step in _workflow()["jobs"][job]["steps"]: + if step.get("name") == name: + return step + raise AssertionError(f"the {job} job has no step named {name!r}") + + +def _shim(where: Path) -> Path: + """A directory holding `python`, this interpreter, which has `packaging`.""" + where.mkdir(parents=True, exist_ok=True) + python = where / "python" + if not python.exists(): + python.symlink_to(sys.executable) + return where + + +def _run(script: str, cwd: Path, env: dict[str, str], + path: tuple[Path, ...] = ()) -> subprocess.CompletedProcess[str]: + base = { + "PATH": os.pathsep.join([*(str(p) for p in path), + os.environ.get("PATH", "/usr/bin:/bin")]), + "HOME": str(cwd), + "GIT_CONFIG_GLOBAL": os.devnull, + "GIT_CONFIG_NOSYSTEM": "1", + "LANG": "C.UTF-8", + } + return subprocess.run( + ("bash", "--noprofile", "--norc", "-eo", "pipefail", "-c", script), + cwd=cwd, env={**base, **env}, capture_output=True, text=True) + + +def _tools(runner_temp: Path) -> Path: + """`$RUNNER_TEMP/tools/bin/python`, the release tools' interpreter.""" + return _shim(runner_temp / "tools" / "bin") + + +# --------------------------------------------------------------------------- +# The shape. + + +def test_the_workflow_runs_only_when_dispatched_with_one_version_input() -> None: + workflow = _workflow() + triggers = workflow.get("on", workflow.get(True)) # YAML 1.1 reads `on` as True + assert list(triggers) == ["workflow_dispatch"], triggers + inputs = triggers["workflow_dispatch"]["inputs"] + assert list(inputs) == ["version"] + assert inputs["version"]["required"] is True + + +def test_only_the_two_publish_jobs_can_mint_an_identity_token() -> None: + workflow = _workflow() + assert workflow["permissions"] == {} + minting = {name for name, job in workflow["jobs"].items() + if (job.get("permissions") or {}).get("id-token") == "write"} + assert minting == {"testpypi", "pypi"} + for name in minting: + assert workflow["jobs"][name]["environment"]["name"] == name + + +def test_no_secret_and_no_password_is_named() -> None: + text = WORKFLOW.read_text(encoding="utf-8") + assert "secrets." not in text + for job in _workflow()["jobs"].values(): + for step in job["steps"]: + if step.get("uses", "").startswith("pypa/gh-action-pypi-publish@"): + assert not {"password", "user"} & set(step.get("with") or {}), step + + +def test_every_action_is_pinned_by_a_full_commit_sha() -> None: + lines = [line.strip() for line in WORKFLOW.read_text(encoding="utf-8").splitlines() + if re.match(r"\s*-?\s*uses:", line)] + assert lines + for line in lines: + assert re.fullmatch(r"-?\s*uses: [\w.-]+/[\w.-]+@[0-9a-f]{40} # v\S+", line), line + + +def test_the_four_jobs_are_chained() -> None: + jobs = _workflow()["jobs"] + needs = {name: sorted([job["needs"]] if isinstance(job.get("needs"), str) + else job.get("needs", [])) + for name, job in jobs.items()} + assert needs == {"build": [], "testpypi": ["build"], + "testpypi-install": ["build", "testpypi"], + "pypi": ["build", "testpypi-install"]} + + +# --------------------------------------------------------------------------- +# The release-commit step and the environment step, with a stand-in `gh`. + +FAKE_GH = """#!/usr/bin/env bash +# A stand-in `gh`: it answers from the case's fixture and records nothing. +case "$*" in + *contents/contracts/code-pin.yaml*) + [ -n "${FAKE_PIN_FAILS:-}" ] && { echo "HTTP 404" >&2; exit 1; } + printf '%s\\n' "$FAKE_PIN" ;; + *compare/*) printf '%s\\n' "$FAKE_STATUS" ;; + *environments/*) + args="$*"; environment="${args##*environments/}"; environment="${environment%% *}" + variable="FAKE_REVIEWERS_${environment}" + [ -z "${!variable:-}" ] && { echo "HTTP 404" >&2; exit 1; } + printf '%s\\n' "${!variable}" ;; + *) echo "the stand-in gh has no answer for: $*" >&2; exit 2 ;; +esac +""" + + +def _fake_gh(where: Path) -> Path: + where.mkdir(parents=True, exist_ok=True) + gh = where / "gh" + gh.write_text(FAKE_GH, encoding="utf-8") + gh.chmod(0o755) + return _shim(where) + + +def _pin(commit: str, source: str = REPOSITORY) -> str: + return (f'leg_role: code\nsource_repository: {source}\nsubmodule_path: code\n\n' + f'commit: "{commit}"\nrevision_kind: commit\n') + + +RELEASE_COMMIT_CASES = { + "the pinned commit, at main's head": ( + dict(sha=PINNED, pin=_pin(PINNED), status="identical"), None), + "the pinned commit, with main moved past it": ( + dict(sha=PINNED, pin=_pin(PINNED), status="ahead"), None), + "the pinned commit, dispatched on its v tag": ( + dict(sha=PINNED, pin=_pin(PINNED), status="ahead", ref="refs/tags/v0.1.0"), None), + "a commit the root does not pin": ( + dict(sha="1" * 40, pin=_pin(PINNED), status="identical"), + "a release publishes only the pinned commit"), + "the pinned commit, off main": ( + dict(sha=PINNED, pin=_pin(PINNED), status="behind"), + "is not on this repository's main"), + "another repository's pin": ( + dict(sha=PINNED, pin=_pin(PINNED, "opensoft/openXdox-code"), status="identical"), + "not 'opensoft/openDox-code'"), + "a pin with no full commit": ( + dict(sha=PINNED, pin=_pin("047bb4fa"), status="identical"), + "names no full commit"), + "a tag other than v": ( + dict(sha=PINNED, pin=_pin(PINNED), status="identical", ref="refs/tags/latest"), + "names v0.1.0"), + "an unreadable pin": ( + dict(sha=PINNED, pin="", status="identical", pin_fails=True), + "cannot be read"), +} + + +@needs_a_shell +@pytest.mark.parametrize("case", sorted(RELEASE_COMMIT_CASES)) +def test_the_release_commit_step(case: str, tmp_path: Path) -> None: + given, refusal = RELEASE_COMMIT_CASES[case] + step = _step("build", "the dispatched commit is the one the openDox root pins") + env = {"GITHUB_REPOSITORY": REPOSITORY, "GITHUB_SHA": given["sha"], + "GITHUB_REF": given.get("ref", "refs/heads/main"), "VERSION": "0.1.0", + "FAKE_PIN": given["pin"], "FAKE_STATUS": given["status"], "GH_TOKEN": "unused"} + if given.get("pin_fails"): + env["FAKE_PIN_FAILS"] = "1" + result = _run(step["run"], tmp_path, env, (_fake_gh(tmp_path / "bin"),)) + if refusal is None: + assert result.returncode == 0, result.stdout + result.stderr + assert "is the commit opensoft/openDox main pins" in result.stdout + else: + assert result.returncode != 0, result.stdout + assert refusal in result.stdout + result.stderr, result.stdout + result.stderr + + +ENVIRONMENT_CASES = { + "both name a reviewer": (dict(testpypi="1", pypi="1"), None), + "pypi names none": (dict(testpypi="1", pypi="0"), "the pypi environment names no required reviewer"), + "testpypi does not exist": (dict(pypi="1"), "the testpypi environment cannot be read"), +} + + +@needs_a_shell +@pytest.mark.parametrize("case", sorted(ENVIRONMENT_CASES)) +def test_the_environment_step(case: str, tmp_path: Path) -> None: + reviewers, refusal = ENVIRONMENT_CASES[case] + step = _step("build", "both environments ask a reviewer before they deploy") + env = {"GITHUB_REPOSITORY": REPOSITORY, "GH_TOKEN": "unused", + **{f"FAKE_REVIEWERS_{name}": count for name, count in reviewers.items()}} + result = _run(step["run"], tmp_path, env, (_fake_gh(tmp_path / "bin"),)) + if refusal is None: + assert result.returncode == 0, result.stdout + result.stderr + else: + assert result.returncode != 0, result.stdout + assert refusal in result.stdout, result.stdout + + +# --------------------------------------------------------------------------- +# The version gate. + +VERSION_CASES = { + "the declared version": ("0.1.0", "0.1.0", None), + "another version": ("0.1.0", "0.2.0", "they must be equal"), + "a version not in normalized form": ("0.1.0", "v0.1.0", "normalized form"), + "a local label": ("0.1.0", "0.1.0+local", "local label"), + "an epoch": ("1!2.0", "1!2.0", "carries an epoch"), + "the placeholder": ("0.0.0", "0.0.0", "the scaffold's placeholder"), + "no version at all": ("0.1.0", "banana", "is not a PEP 440 version"), +} + + +@needs_a_shell +@pytest.mark.parametrize("case", sorted(VERSION_CASES)) +def test_the_version_gate(case: str, tmp_path: Path) -> None: + declared, asked, refusal = VERSION_CASES[case] + (tmp_path / "pyproject.toml").write_text( + f'[project]\nname = "opendox"\nversion = "{declared}"\n', encoding="utf-8") + _tools(tmp_path / "rt") + step = _step("build", "the version input is pyproject's version") + result = _run(step["run"], tmp_path, {"VERSION": asked, "RUNNER_TEMP": str(tmp_path / "rt")}) + if refusal is None: + assert result.returncode == 0, result.stdout + result.stderr + else: + assert result.returncode != 0, result.stdout + assert refusal in result.stdout + result.stderr, result.stdout + result.stderr + + +# --------------------------------------------------------------------------- +# The artifact checks, over a wheel and an sdist built by hand. + +PYPROJECT = """[project] +name = "opendox" +version = "0.1.0" +dependencies = ["PyYAML>=6.0"] + +[project.optional-dependencies] +runtime = ["fastapi>=0.115"] +local = ["opendox[runtime]", "pixeltable-pgserver>=0.6.0"] +""" + +REQUIRES = ["PyYAML>=6.0", 'fastapi>=0.115; extra == "runtime"', + 'opendox[runtime]; extra == "local"', 'pixeltable-pgserver>=0.6.0; extra == "local"'] + +TREE = { + "src/.gitkeep": "", + "src/opendox/__init__.py": "", + "src/opendox/cli.py": "def main():\n return 0\n", + "src/opendox/web/index.html": "\n", + "src/opendox/web/vendor/.gitkeep": "", + "src/route_extension.py": "", + "migrations/0001_first.sql": "select 1;\n", +} + + +def _tree(where: Path) -> None: + (where / "pyproject.toml").write_text(PYPROJECT, encoding="utf-8") + for path, text in TREE.items(): + (where / path).parent.mkdir(parents=True, exist_ok=True) + (where / path).write_text(text, encoding="utf-8") + git = ("git", "-c", "user.name=t", "-c", "user.email=t@invalid", "-c", "init.defaultBranch=main") + env = {**os.environ, "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_NOSYSTEM": "1"} + for args in (("init", "-q"), ("add", "-A"), ("commit", "-q", "-m", "tree")): + subprocess.run((*git, *args), cwd=where, check=True, env=env, capture_output=True) + + +def _wheel(where: Path, *, drop: str = "", add: str = "", requires: list[str] | None = None, + scripts: str = "opendox = opendox.cli:main") -> None: + dist = where / "dist" + dist.mkdir(exist_ok=True) + info, data = "opendox-0.1.0.dist-info", "opendox-0.1.0.data/data" + requires = REQUIRES if requires is None else requires + metadata = ("Metadata-Version: 2.4\nName: opendox\nVersion: 0.1.0\n" + + "".join(f"Provides-Extra: {e}\n" for e in ("runtime", "local")) + + "".join(f"Requires-Dist: {r}\n" for r in requires)) + members = { + **{p.removeprefix("src/"): t for p, t in TREE.items() + if p.startswith("src/") and p != "src/.gitkeep"}, + f"{data}/share/opendox/migrations/0001_first.sql": TREE["migrations/0001_first.sql"], + f"{info}/METADATA": metadata, + f"{info}/entry_points.txt": f"[console_scripts]\n{scripts}\n", + f"{info}/WHEEL": "Wheel-Version: 1.0\n", + } + members.pop(drop, None) + if add: + members[add] = "" + with zipfile.ZipFile(dist / "opendox-0.1.0-py3-none-any.whl", "w") as archive: + for name, text in members.items(): + archive.writestr(name, text) + (dist / "opendox-0.1.0.tar.gz").write_bytes(b"an sdist stands here") + + +ARTIFACT_CASES = { + "the whole set": ({}, None), + "a web dotfile missing": (dict(drop="opendox/web/vendor/.gitkeep"), "the wheel lacks web files"), + "a top-level module missing": (dict(drop="route_extension.py"), "the wheel lacks tracked files"), + "a stray file": (dict(add="opendox/stray.txt"), "the tree does not track"), + "a migration missing": (dict(drop="opendox-0.1.0.data/data/share/opendox/migrations/0001_first.sql"), + "the wheel lacks migrations"), + "the server's package dropped from the local extra": ( + dict(requires=[r for r in REQUIRES if "pixeltable" not in r]), "are not pyproject.toml's"), + "a requirement the tree does not declare": ( + dict(requires=[*REQUIRES, 'httpx>=0.27; extra == "local"']), "are not pyproject.toml's"), + "no console script": (dict(scripts="opendox-runtime = opendox.runtime.cli:main"), + "the console script `opendox = opendox.cli:main` is not declared"), +} + + +@needs_a_shell +@pytest.mark.parametrize("case", sorted(ARTIFACT_CASES)) +def test_the_artifact_checks(case: str, tmp_path: Path) -> None: + mutation, refusal = ARTIFACT_CASES[case] + _tree(tmp_path) + _wheel(tmp_path, **mutation) + _tools(tmp_path / "rt") + step = _step("build", "verify the artifacts") + result = _run(step["run"], tmp_path, {"VERSION": "0.1.0", "RUNNER_TEMP": str(tmp_path / "rt")}) + if refusal is None: + assert result.returncode == 0, result.stdout + result.stderr + assert "5. web bundle: 2 of 2 tracked files are in the wheel" in result.stdout + assert "the artifacts carry what an install runs from" in result.stdout + else: + assert result.returncode != 0, result.stdout + assert refusal in result.stdout, result.stdout + result.stderr + + +@needs_a_shell +def test_the_artifact_checks_refuse_a_third_file(tmp_path: Path) -> None: + _tree(tmp_path) + _wheel(tmp_path) + (tmp_path / "dist" / "evil-0.1.0-py3-none-any.whl").write_bytes(b"") + _tools(tmp_path / "rt") + step = _step("build", "verify the artifacts") + result = _run(step["run"], tmp_path, {"VERSION": "0.1.0", "RUNNER_TEMP": str(tmp_path / "rt")}) + assert result.returncode != 0 + assert "a release is exactly" in result.stdout + result.stderr + + +# --------------------------------------------------------------------------- +# The publish jobs' digest check. + + +@needs_a_shell +@pytest.mark.parametrize("job", ["testpypi", "pypi"]) +@pytest.mark.parametrize("change", ["none", "a changed byte", "a third file"]) +def test_the_publish_jobs_check_the_digests(job: str, change: str, tmp_path: Path) -> None: + dist = tmp_path / "dist" + dist.mkdir() + wheel, sdist = "opendox-0.1.0-py3-none-any.whl", "opendox-0.1.0.tar.gz" + (dist / wheel).write_bytes(b"the wheel") + (dist / sdist).write_bytes(b"the sdist") + env = {"WHEEL": wheel, "SDIST": sdist, + "WHEEL_SHA256": hashlib.sha256(b"the wheel").hexdigest(), + "SDIST_SHA256": hashlib.sha256(b"the sdist").hexdigest()} + if change == "a changed byte": + (dist / wheel).write_bytes(b"the wheeL") + elif change == "a third file": + (dist / "evil-0.1.0-py3-none-any.whl").write_bytes(b"") + step = _step(job, "the files are the ones the build job verified") + result = _run(step["run"], tmp_path, env) + assert (result.returncode == 0) == (change == "none"), result.stdout + result.stderr From c324c61d6b84c25f124f9b1b5fe3f9f1d606504d Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 00:55:35 +0000 Subject: [PATCH 03/15] Fix round 2: TestPyPI's JSON read retries until it matches, the token wording is exact, and README's links resolve on PyPI (Copilot review) Copilot's review of openDox-code#78 at 2b0083b9: - r4170898255: the JSON read kept a partial or stale answer as final. It now retries, up to 20 reads 15 s apart, until TestPyPI serves exactly the build job's two files and digests, and only then refuses. - r4170898293: the header said "no token". It now says no PyPI token and no stored secret, and names the one other credential, GitHub's own GITHUB_TOKEN, which the build job passes to gh for its two API reads, with contents: read and actions: read. - r4170898319: README.md's four relative links (SECURITY.md and two docs) are absolute, so they resolve from the PyPI page that `readme` makes of README.md. And openxFactory#1220's r4170908474: a dispatch runs at the head of the ref it names, so the header and the release-commit refusal say to dispatch on the tag v at the pinned commit once main has moved. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 52 +++++++++++++++++++++-------------- README.md | 8 +++--- pyproject.toml | 3 +- 3 files changed, 37 insertions(+), 26 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 5985e35b..f5d9d6d0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -22,13 +22,16 @@ name: release # openDox root pins that commit (plan 034 T087), and then this workflow is # dispatched at the cut. `main`'s head may have moved past the pinned commit by # then (T095 lands later), so the head is never required: the run's commit -# must equal the root's pin and be on `main`. The dispatch names `main`, or -# the tag `v` created at the pinned commit, and no other tag. +# must equal the root's pin and be on `main`. A dispatch runs at the head of +# the ref it names, so it names the tag `v`, created at the pinned +# commit, or `main` only while `main`'s head is still that commit. No other +# tag is accepted. # # FOUR JOBS, EACH GATED ON THE ONE BEFORE IT: # build builds the sdist and the wheel in a clean job, runs # `twine check`, VERIFIES both before any upload, and -# installs the wheel into a fresh venv. It holds no token. +# installs the wheel into a fresh venv. It mints no +# identity token, and its GITHUB_TOKEN reads only. # testpypi the dry run: publishes those exact bytes to TestPyPI, # in the `testpypi` environment. # testpypi-install installs `opendox[local]` from TestPyPI into a fresh @@ -36,12 +39,14 @@ name: release # pypi publishes the same bytes to PyPI, in the `pypi` # environment. # -# NO TOKEN AND NO SECRET, ANYWHERE. Both uploads use PyPI's trusted publishing -# (OIDC): the job asks GitHub for a short-lived identity token, and the index -# accepts it only for the publisher configured on its side (owner `opensoft`, -# repository `openDox-code`, workflow `release.yml`, and the environment the -# job runs in). `id-token: write` is granted to the two publish jobs and to no -# other job. +# NO PyPI TOKEN AND NO STORED SECRET, ANYWHERE. Both uploads use PyPI's +# trusted publishing (OIDC): the job asks GitHub for a short-lived identity +# token, and the index accepts it only for the publisher configured on its side +# (owner `opensoft`, repository `openDox-code`, workflow `release.yml`, and the +# environment the job runs in). `id-token: write` is granted to the two publish +# jobs and to no other job. The one other credential is GitHub's own automatic +# `GITHUB_TOKEN`, which the build job passes to `gh` for its two API reads, +# with `contents: read` and `actions: read` and nothing more. # # EVERY ACTION IS PINNED BY FULL COMMIT SHA, with its release tag beside it. # (`validate.yml` pins by tag. A tag can be moved after review, and a commit @@ -142,7 +147,9 @@ jobs: sys.exit(f"::error::the openDox root's code pin names no full commit: {pinned!r}") if pinned != this: sys.exit(f"::error::this run is at {this}, and the openDox root pins {pinned}; " - "a release publishes only the pinned commit") + "a release publishes only the pinned commit. A dispatch runs at the " + "head of the ref it names, so dispatch on the tag v created " + f"at {pinned}") print(f"{this} is the commit opensoft/openDox main pins for {here}") PY status="$(gh api "repos/$GITHUB_REPOSITORY/compare/$GITHUB_SHA...main" --jq .status)" @@ -529,33 +536,36 @@ jobs: python-version: '3.12' # TestPyPI serves exactly the two files the build job verified. Its JSON - # API can lag the upload by a little, so it is read up to 20 times, 15 - # seconds apart, before this step refuses. + # API can lag the upload, answering 404 or a partial release for a + # while, so it is read up to 20 times, 15 seconds apart, until it serves + # exactly those two files, before this step refuses. - name: TestPyPI serves the files the build job verified run: | python - <<'PY' import json, os, sys, time, urllib.error, urllib.request url = f"https://test.pypi.org/pypi/opendox/{os.environ['VERSION']}/json" + wanted = {os.environ["WHEEL"]: os.environ["WHEEL_SHA256"], + os.environ["SDIST"]: os.environ["SDIST_SHA256"]} + served = None for attempt in range(1, 21): try: with urllib.request.urlopen(url, timeout=30) as response: release = json.load(response) - break + served = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} + if served == wanted: + break + print(f"attempt {attempt}: TestPyPI serves {served} so far") except urllib.error.URLError as error: code = getattr(error, "code", None) if code is not None and code != 404 and code < 500: raise print(f"attempt {attempt}: {url} did not answer yet ({error})") - time.sleep(15) + time.sleep(15) else: - sys.exit(f"::error::{url} never answered") - served = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} - wanted = {os.environ["WHEEL"]: os.environ["WHEEL_SHA256"], - os.environ["SDIST"]: os.environ["SDIST_SHA256"]} - print(f"TestPyPI serves {served}") - if served != wanted: - sys.exit(f"::error::TestPyPI serves {served}, and the build job verified {wanted}") + sys.exit(f"::error::after 20 reads, TestPyPI serves {served}, " + f"and the build job verified {wanted}") + print(f"TestPyPI serves exactly the verified files: {served}") PY # T099's DRY-RUN FALSIFIER, as plan 034 writes it, in a fresh venv: diff --git a/README.md b/README.md index 04886a60..8669de3e 100644 --- a/README.md +++ b/README.md @@ -34,9 +34,9 @@ not here: see and [CODE_OF_CONDUCT.md](https://github.com/opensoft/openDox/blob/main/CODE_OF_CONDUCT.md) in `opensoft/openDox`. Security reports for this repository go through -[SECURITY.md](SECURITY.md). The `validate` check is a required status check +[SECURITY.md](https://github.com/opensoft/openDox-code/blob/main/SECURITY.md). The `validate` check is a required status check on `main`, enforced by a repository ruleset — see -[docs/branch-protection.md](docs/branch-protection.md). It runs the whole +[docs/branch-protection.md](https://github.com/opensoft/openDox-code/blob/main/docs/branch-protection.md). It runs the whole suite, `tests/` and `tests_runtime/`, with the conftest chain in play and a PostgreSQL service, and it declares no exclusion (plan 034 T036; `add-neutral-product-standalone-operability` requirement 9). Until then it ran @@ -56,5 +56,5 @@ adds it — the xFactory family's standing rule, levelled across all six | document | what it is | |---|---| -| [docs/branch-protection.md](docs/branch-protection.md) | the repository ruleset that makes `validate` a required status check on `main`, its `evaluate` → `active` history, and the one policy difference between the two families | -| [docs/profile-registration-runbook.md](docs/profile-registration-runbook.md) | the host profile: the one `opendox.domain_profile.register()` call a descendant makes at process start, what openDox reads off it, the default profile openDox's entry points register where no host has (and until when a host's registration still replaces it), how a process that builds nothing refuses, and how RULED ASK-4 Q5's one registration serves both legs' accessors without an import between them | +| [docs/branch-protection.md](https://github.com/opensoft/openDox-code/blob/main/docs/branch-protection.md) | the repository ruleset that makes `validate` a required status check on `main`, its `evaluate` → `active` history, and the one policy difference between the two families | +| [docs/profile-registration-runbook.md](https://github.com/opensoft/openDox-code/blob/main/docs/profile-registration-runbook.md) | the host profile: the one `opendox.domain_profile.register()` call a descendant makes at process start, what openDox reads off it, the default profile openDox's entry points register where no host has (and until when a host's registration still replaces it), how a process that builds nothing refuses, and how RULED ASK-4 Q5's one registration serves both legs' accessors without an import between them | diff --git a/pyproject.toml b/pyproject.toml index 7d066f4e..0b3ffbf1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -22,7 +22,8 @@ requires-python = ">=3.12" # 5962754358, item 1, "Publish to PyPI at the cut"). Without it the metadata # carries no long description, `twine check --strict` refuses both files, and # the published page is blank. `.github/workflows/release.yml` runs that check -# before any upload. The `.md` suffix makes setuptools declare `text/markdown`. +# before any upload. The `.md` suffix makes setuptools declare `text/markdown`, +# and README.md's links are absolute, so they resolve from the PyPI page too. readme = "README.md" # THE RUNTIME DEPENDENCY SET, round 2 of the levelling pass — RULED Q-L2 From 7cc65f93964d7632ccdf7e7bd02db8f8d40f1dcf Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 01:27:30 +0000 Subject: [PATCH 04/15] Fix round 3: read the root's pin anonymously over git, and refuse pre-releases (Copilot review) Copilot's review at de3e3249 (r4171040914, r4171040935): - The openDox root's contracts/code-pin.yaml is read by an anonymous depth-1 git fetch of opensoft/openDox main (public), with every credential helper cleared and no prompt, in a repository of its own under RUNNER_TEMP. The read of another repository no longer depends on this job's repository-scoped GITHUB_TOKEN. The step's one gh call is the compare with this repository's main. - The version gate refuses a pre-release or a development release: the dry run installs the unversioned "opendox[local]", which pip resolves to a final release once one exists, so it could never install what was uploaded. - tests/test_release_workflow.py: the release-commit cases read a stand-in root through git's url.insteadOf in the environment, with GIT_ALLOW_PROTOCOL=file so no case can reach the network; a root with no pin and an unreachable root refuse; a structural case holds the anonymous read; the version gate refuses 0.2.0rc1 and 0.2.0.dev1 and passes 0.1.0.post1. 44 cases. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 42 ++++++++++++----- tests/test_release_workflow.py | 82 +++++++++++++++++++++++++++++----- 2 files changed, 101 insertions(+), 23 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f5d9d6d0..cdb06ff6 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -45,8 +45,10 @@ name: release # (owner `opensoft`, repository `openDox-code`, workflow `release.yml`, and the # environment the job runs in). `id-token: write` is granted to the two publish # jobs and to no other job. The one other credential is GitHub's own automatic -# `GITHUB_TOKEN`, which the build job passes to `gh` for its two API reads, -# with `contents: read` and `actions: read` and nothing more. +# `GITHUB_TOKEN`, which the build job passes to `gh` for two reads of THIS +# repository's API (the compare with `main`, and the environments), with +# `contents: read` and `actions: read` and nothing more. The openDox root's +# pin, in another repository, is read over git with no credential at all. # # EVERY ACTION IS PINNED BY FULL COMMIT SHA, with its release tag beside it. # (`validate.yml` pins by tag. A tag can be moved after review, and a commit @@ -111,8 +113,14 @@ jobs: # repository's `main`, as `main` or an ancestor of it. `main`'s head # need not be that commit. A dispatch on a tag must name `v`. # A dispatch may name any ref, and the commit it names is what would be - # published, so this is checked first. Both repositories are public, and - # both reads go through the API. + # published, so this is checked first. + # + # THE ROOT'S PIN IS READ ANONYMOUSLY, OVER GIT. opensoft/openDox is + # public, and this job's GITHUB_TOKEN is an installation token scoped + # to this repository, so the read of another repository uses no token: + # a fetch of `main`, depth 1, with every credential helper cleared and + # no prompt, in a repository of its own under `$RUNNER_TEMP`. The + # compare with `main` reads this repository, through the API. - name: the dispatched commit is the one the openDox root pins env: GH_TOKEN: ${{ github.token }} @@ -125,8 +133,12 @@ jobs: fi ;; esac - if ! pin="$(gh api 'repos/opensoft/openDox/contents/contracts/code-pin.yaml?ref=main' \ - -H 'Accept: application/vnd.github.raw')"; then + root="$RUNNER_TEMP/opendox-root" + rm -rf "$root" + git init -q -b main "$root" + if ! GIT_TERMINAL_PROMPT=0 git -C "$root" -c credential.helper= \ + fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git main \ + || ! pin="$(git -C "$root" show FETCH_HEAD:contracts/code-pin.yaml)"; then echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" exit 1 fi @@ -200,11 +212,15 @@ jobs: # THE VERSION GATE. The input must be a PEP 440 version in normalized # form (the form the files are named by), with no epoch and no local - # label, must equal what `pyproject.toml` declares, and must not be - # `0.0.0`, the scaffold's placeholder, which is never a release. An - # epoch (`1!2.0`) is refused because a wheel's file name escapes its `!`, - # so the files would not carry the version as written, and every check - # below names them by it. + # label, must be a final release, must equal what `pyproject.toml` + # declares, and must not be `0.0.0`, the scaffold's placeholder, which + # is never a release. An epoch (`1!2.0`) is refused because a wheel's + # file name escapes its `!`, so the files would not carry the version as + # written, and every check below names them by it. A pre-release or a + # development release (`0.2.0rc1`, `0.2.0.dev1`) is refused because the + # dry run installs the unversioned `opendox[local]`, as 10.3's line + # reads, and pip passes over a pre-release once any final release + # exists, so the dry run could never install what was uploaded. - name: the version input is pyproject's version run: | "$RUNNER_TEMP/tools/bin/python" - <<'PY' @@ -226,6 +242,10 @@ jobs: if parsed.epoch: sys.exit(f"::error::{asked!r} carries an epoch, which a wheel's file name " "escapes; this workflow names the files by the version as written") + if parsed.is_prerelease: + sys.exit(f"::error::{asked!r} is a pre-release or a development release; " + "the dry run installs the unversioned \"opendox[local]\", which " + "pip resolves to a final release once one exists") if parsed == Version("0.0.0"): sys.exit("::error::0.0.0 is the scaffold's placeholder version, never a release") if asked != declared: diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index 4bfad80b..9345ab4f 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -14,8 +14,13 @@ EVERY CASE IS HERMETIC. The two steps that call `gh` run with a stand-in `gh` first on `PATH`, which answers from the case's own fixture and records nothing, so no case reaches GitHub (and `tests/hermeticity.py`'s refusal shim stays the -answer for any other `gh`). No case reaches an index. The one step that reads -TestPyPI over HTTP is not run here; it runs at the dry run. +answer for any other `gh`). The release-commit step reads the openDox root's +pin over git, from `https://github.com/opensoft/openDox.git`. Each case +redirects that URL, through git's own `url..insteadOf` in the +environment's config, to a repository the case builds under its temporary +directory, so the step's script runs unchanged and reaches no network. No case +reaches an index. The one step that reads TestPyPI over HTTP is not run here; +it runs at the dry run. What is held: * the shape: dispatch only, one `version` input, no secret, `id-token: write` @@ -23,10 +28,13 @@ pinned by a full commit SHA, and the four jobs chained; * the release-commit step: the pinned commit on `main` passes, and a commit the openDox root does not pin, a commit off `main`, another repository's - pin, a tag other than `v` and an unreadable pin each refuse; + pin, a tag other than `v`, an unreachable root and a root with no + pin each refuse. The pin is read over git with no credential, and the + step's one `gh` call is the compare with this repository's `main`; * the environment step: both environments with a reviewer pass, and either one without a reviewer, or unreadable, refuses; - * the version gate, case by case; + * the version gate, case by case, a pre-release and a development release + among the refusals; * the artifact checks, over a small built-by-hand wheel and sdist in a git tree of their own: the whole set passes, and each kind of gap refuses; * the publish jobs' digest check: the verified files pass, and a changed @@ -153,9 +161,6 @@ def test_the_four_jobs_are_chained() -> None: FAKE_GH = """#!/usr/bin/env bash # A stand-in `gh`: it answers from the case's fixture and records nothing. case "$*" in - *contents/contracts/code-pin.yaml*) - [ -n "${FAKE_PIN_FAILS:-}" ] && { echo "HTTP 404" >&2; exit 1; } - printf '%s\\n' "$FAKE_PIN" ;; *compare/*) printf '%s\\n' "$FAKE_STATUS" ;; *environments/*) args="$*"; environment="${args##*environments/}"; environment="${environment%% *}" @@ -180,6 +185,40 @@ def _pin(commit: str, source: str = REPOSITORY) -> str: f'commit: "{commit}"\nrevision_kind: commit\n') +ROOT_URL = "https://github.com/opensoft/openDox.git" + + +def _root(where: Path, pin: str | None) -> Path: + """A stand-in openDox root: a repository whose `main` carries `pin` as + `contracts/code-pin.yaml`, or carries no pin at all when `pin` is None.""" + env = {"PATH": os.environ.get("PATH", "/usr/bin:/bin"), "HOME": str(where), + "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_NOSYSTEM": "1", + "GIT_AUTHOR_NAME": "root", "GIT_AUTHOR_EMAIL": "root@example.invalid", + "GIT_COMMITTER_NAME": "root", "GIT_COMMITTER_EMAIL": "root@example.invalid"} + where.mkdir(parents=True) + subprocess.run(("git", "init", "-q", "-b", "main", str(where)), check=True, env=env) + if pin is None: + (where / "README.md").write_text("no pin\n", encoding="utf-8") + else: + (where / "contracts").mkdir() + (where / "contracts" / "code-pin.yaml").write_text(pin, encoding="utf-8") + subprocess.run(("git", "-C", str(where), "add", "-A"), check=True, env=env) + subprocess.run(("git", "-C", str(where), "commit", "-q", "-m", "root"), check=True, env=env) + return where + + +def _redirect(to: Path) -> dict[str, str]: + """Git config, in the environment, that sends the root's URL to `to`. + + `GIT_ALLOW_PROTOCOL=file` makes git refuse every other transport, so a + redirect that failed to apply would refuse the fetch rather than read the + real root over the network: no case can pass by reaching GitHub.""" + return {"GIT_CONFIG_COUNT": "1", + "GIT_CONFIG_KEY_0": f"url.{to.as_uri()}.insteadOf", + "GIT_CONFIG_VALUE_0": ROOT_URL, + "GIT_ALLOW_PROTOCOL": "file"} + + RELEASE_COMMIT_CASES = { "the pinned commit, at main's head": ( dict(sha=PINNED, pin=_pin(PINNED), status="identical"), None), @@ -202,8 +241,11 @@ def _pin(commit: str, source: str = REPOSITORY) -> str: "a tag other than v": ( dict(sha=PINNED, pin=_pin(PINNED), status="identical", ref="refs/tags/latest"), "names v0.1.0"), - "an unreadable pin": ( - dict(sha=PINNED, pin="", status="identical", pin_fails=True), + "an unreachable root": ( + dict(sha=PINNED, pin=_pin(PINNED), status="identical", unreachable=True), + "cannot be read"), + "a root with no pin": ( + dict(sha=PINNED, pin=None, status="identical"), "cannot be read"), } @@ -213,11 +255,14 @@ def _pin(commit: str, source: str = REPOSITORY) -> str: def test_the_release_commit_step(case: str, tmp_path: Path) -> None: given, refusal = RELEASE_COMMIT_CASES[case] step = _step("build", "the dispatched commit is the one the openDox root pins") + root = _root(tmp_path / "root", given["pin"]) + if given.get("unreachable"): + root = tmp_path / "no-such-root" + (tmp_path / "rt").mkdir() env = {"GITHUB_REPOSITORY": REPOSITORY, "GITHUB_SHA": given["sha"], "GITHUB_REF": given.get("ref", "refs/heads/main"), "VERSION": "0.1.0", - "FAKE_PIN": given["pin"], "FAKE_STATUS": given["status"], "GH_TOKEN": "unused"} - if given.get("pin_fails"): - env["FAKE_PIN_FAILS"] = "1" + "RUNNER_TEMP": str(tmp_path / "rt"), "FAKE_STATUS": given["status"], + "GH_TOKEN": "unused", **_redirect(root)} result = _run(step["run"], tmp_path, env, (_fake_gh(tmp_path / "bin"),)) if refusal is None: assert result.returncode == 0, result.stdout + result.stderr @@ -227,6 +272,15 @@ def test_the_release_commit_step(case: str, tmp_path: Path) -> None: assert refusal in result.stdout + result.stderr, result.stdout + result.stderr +def test_the_root_pin_is_read_over_git_with_no_credential() -> None: + script = _step("build", "the dispatched commit is the one the openDox root pins")["run"] + assert "contents/contracts/code-pin.yaml" not in script + assert re.search(r'GIT_TERMINAL_PROMPT=0 git -C "\$root" -c credential\.helper= \\\n' + r'\s+fetch -q --depth 1 --no-tags ' + re.escape(ROOT_URL) + r' main', script), script + calls = [line.strip() for line in script.splitlines() if re.search(r"\bgh\b", line)] + assert len(calls) == 1 and "/compare/" in calls[0], calls + + ENVIRONMENT_CASES = { "both name a reviewer": (dict(testpypi="1", pypi="1"), None), "pypi names none": (dict(testpypi="1", pypi="0"), "the pypi environment names no required reviewer"), @@ -260,6 +314,10 @@ def test_the_environment_step(case: str, tmp_path: Path) -> None: "an epoch": ("1!2.0", "1!2.0", "carries an epoch"), "the placeholder": ("0.0.0", "0.0.0", "the scaffold's placeholder"), "no version at all": ("0.1.0", "banana", "is not a PEP 440 version"), + "a pre-release": ("0.2.0rc1", "0.2.0rc1", "is a pre-release or a development release"), + "a development release": ("0.2.0.dev1", "0.2.0.dev1", + "is a pre-release or a development release"), + "a post-release": ("0.1.0.post1", "0.1.0.post1", None), } From 74b87375f43c75e1a7ee479229af9eb806120ceb Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 01:49:41 +0000 Subject: [PATCH 05/15] Fix round 4: skip-existing on both uploads, PyPI's served digests checked after the upload, and job timeouts that fit the retries (Copilot review) Copilot's review at 7cc65f93 ("Needs a closer look", two items it had missed before): - Both pypa/gh-action-pypi-publish steps set skip-existing: a re-run after a partial upload uploads the rest instead of stopping on the file the index already holds. An index keeps each file name once, so a skipped file could differ from the verified one; each upload is therefore followed by the check that the index serves exactly the build job's two digests. TestPyPI's check already followed its upload, and the pypi job now runs the same script against PyPI after its upload. The one script reads INDEX, JSON_BASE, READS and PAUSE from its step's env, and catches a read timeout too. - testpypi-install's timeout is 75 minutes, its budget in full: the JSON step (20 reads, each up to 30 s, 15 s apart: 15 min), the install (10 tries, each cut off after 300 s, 30 s apart: 55 min), and setup (5 min). The pypi job's is 30 minutes (the upload, the JSON step, the download). - tests/test_release_workflow.py (57 cases): skip-existing on both uploads; the two index checks are one script, and PyPI's follows the upload; the index check against a local server (exact, lagging, another sdist, a third file, a refused read); each job's timeout recomputed from its steps. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 102 +++++++++++++++++++---- tests/test_release_workflow.py | 143 +++++++++++++++++++++++++++++++++ 2 files changed, 230 insertions(+), 15 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index cdb06ff6..3b40a601 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -37,7 +37,8 @@ name: release # testpypi-install installs `opendox[local]` from TestPyPI into a fresh # venv, as plan 034's T099 falsifier does, and runs it. # pypi publishes the same bytes to PyPI, in the `pypi` -# environment. +# environment, then requires PyPI to serve exactly +# those two files, by digest. # # NO PyPI TOKEN AND NO STORED SECRET, ANYWHERE. Both uploads use PyPI's # trusted publishing (OIDC): the job asks GitHub for a short-lived identity @@ -529,15 +530,26 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - + # skip-existing: a re-run after a partial upload skips a file the index + # already holds, instead of stopping on it, and uploads the rest. An + # index keeps each file name once, so a skipped file could differ from + # the verified one; the JSON step that follows each upload requires the + # index to serve exactly the build job's two digests, so it cannot pass. - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 with: repository-url: https://test.pypi.org/legacy/ + skip-existing: true testpypi-install: name: install opendox[local] from TestPyPI needs: [build, testpypi] runs-on: ubuntu-latest - timeout-minutes: 20 + # THE BUDGET, in full, so no retry this job promises is cut short: + # the JSON step: 20 reads, each up to 30 s, 15 s apart 15 min + # the install: 10 tries, each up to 300 s, 30 s apart 55 min + # checkout, setup-python, the venvs and --help 5 min + # tests/test_release_workflow.py recomputes it from the steps. + timeout-minutes: 75 permissions: contents: read env: @@ -557,35 +569,43 @@ jobs: # TestPyPI serves exactly the two files the build job verified. Its JSON # API can lag the upload, answering 404 or a partial release for a - # while, so it is read up to 20 times, 15 seconds apart, until it serves - # exactly those two files, before this step refuses. + # while, so it is read up to READS times, PAUSE seconds apart, until it + # serves exactly those two files, before this step refuses. The `pypi` + # job runs the same script against PyPI after its upload. - name: TestPyPI serves the files the build job verified + env: + INDEX: TestPyPI + JSON_BASE: https://test.pypi.org/pypi/opendox/ + READS: '20' + PAUSE: '15' run: | - python - <<'PY' + python3 - <<'PY' import json, os, sys, time, urllib.error, urllib.request - url = f"https://test.pypi.org/pypi/opendox/{os.environ['VERSION']}/json" + index = os.environ["INDEX"] + url = f"{os.environ['JSON_BASE']}{os.environ['VERSION']}/json" + reads, pause = int(os.environ["READS"]), float(os.environ["PAUSE"]) wanted = {os.environ["WHEEL"]: os.environ["WHEEL_SHA256"], os.environ["SDIST"]: os.environ["SDIST_SHA256"]} served = None - for attempt in range(1, 21): + for attempt in range(1, reads + 1): try: with urllib.request.urlopen(url, timeout=30) as response: release = json.load(response) served = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} if served == wanted: break - print(f"attempt {attempt}: TestPyPI serves {served} so far") - except urllib.error.URLError as error: + print(f"attempt {attempt}: {index} serves {served} so far") + except (urllib.error.URLError, TimeoutError, ConnectionError) as error: code = getattr(error, "code", None) if code is not None and code != 404 and code < 500: raise print(f"attempt {attempt}: {url} did not answer yet ({error})") - time.sleep(15) + time.sleep(pause) else: - sys.exit(f"::error::after 20 reads, TestPyPI serves {served}, " + sys.exit(f"::error::after {reads} reads, {index} serves {served}, " f"and the build job verified {wanted}") - print(f"TestPyPI serves exactly the verified files: {served}") + print(f"{index} serves exactly the verified files: {served}") PY # T099's DRY-RUN FALSIFIER, as plan 034 writes it, in a fresh venv: @@ -596,7 +616,8 @@ jobs: # requires the installed opendox to be the version just uploaded, so an # index that lags, or an older release, cannot pass for it. The simple # index can lag the JSON API, so each try is a fresh venv, up to ten - # tries, 30 seconds apart. + # tries, 30 seconds apart, and each try is cut off after 300 seconds, + # so the job's budget holds. # # THE TRADE, stated: with two indexes, pip may take a dependency from # either. The lock pins every dependency's version, and this job holds no @@ -609,7 +630,7 @@ jobs: for attempt in 1 2 3 4 5 6 7 8 9 10; do rm -rf "$RUNNER_TEMP/fresh" python -m venv "$RUNNER_TEMP/fresh" - if "$RUNNER_TEMP/fresh/bin/python" -m pip install --disable-pip-version-check \ + if timeout 300 "$RUNNER_TEMP/fresh/bin/python" -m pip install --disable-pip-version-check \ --no-cache-dir --only-binary :all: -c constraints-cpython312-linux.txt \ --index-url https://test.pypi.org/simple/ \ --extra-index-url https://pypi.org/simple/ \ @@ -636,7 +657,9 @@ jobs: name: publish to PyPI needs: [build, testpypi-install] runs-on: ubuntu-latest - timeout-minutes: 15 + # THE BUDGET: the upload (10 min), then the JSON step, 20 reads, each up + # to 30 s, 15 s apart (15 min), and the download (5 min). + timeout-minutes: 30 environment: name: pypi url: https://pypi.org/project/opendox/${{ inputs.version }}/ @@ -664,4 +687,53 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - + # skip-existing: a re-run after a partial upload skips a file the index + # already holds, instead of stopping on it, and uploads the rest. An + # index keeps each file name once, so a skipped file could differ from + # the verified one; the JSON step that follows each upload requires the + # index to serve exactly the build job's two digests, so it cannot pass. - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 + with: + skip-existing: true + + # PyPI serves exactly the two files the build job verified: the same + # script as the TestPyPI check, against PyPI. + - name: PyPI serves the files the build job verified + env: + INDEX: PyPI + JSON_BASE: https://pypi.org/pypi/opendox/ + READS: '20' + PAUSE: '15' + WHEEL: ${{ needs.build.outputs.wheel }} + WHEEL_SHA256: ${{ needs.build.outputs.wheel-sha256 }} + SDIST: ${{ needs.build.outputs.sdist }} + SDIST_SHA256: ${{ needs.build.outputs.sdist-sha256 }} + run: | + python3 - <<'PY' + import json, os, sys, time, urllib.error, urllib.request + + index = os.environ["INDEX"] + url = f"{os.environ['JSON_BASE']}{os.environ['VERSION']}/json" + reads, pause = int(os.environ["READS"]), float(os.environ["PAUSE"]) + wanted = {os.environ["WHEEL"]: os.environ["WHEEL_SHA256"], + os.environ["SDIST"]: os.environ["SDIST_SHA256"]} + served = None + for attempt in range(1, reads + 1): + try: + with urllib.request.urlopen(url, timeout=30) as response: + release = json.load(response) + served = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} + if served == wanted: + break + print(f"attempt {attempt}: {index} serves {served} so far") + except (urllib.error.URLError, TimeoutError, ConnectionError) as error: + code = getattr(error, "code", None) + if code is not None and code != 404 and code < 500: + raise + print(f"attempt {attempt}: {url} did not answer yet ({error})") + time.sleep(pause) + else: + sys.exit(f"::error::after {reads} reads, {index} serves {served}, " + f"and the build job verified {wanted}") + print(f"{index} serves exactly the verified files: {served}") + PY diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index 9345ab4f..da6f20c3 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -44,6 +44,7 @@ from __future__ import annotations import hashlib +import json import os import re import shutil @@ -470,3 +471,145 @@ def test_the_publish_jobs_check_the_digests(job: str, change: str, tmp_path: Pat step = _step(job, "the files are the ones the build job verified") result = _run(step["run"], tmp_path, env) assert (result.returncode == 0) == (change == "none"), result.stdout + result.stderr + + +# --------------------------------------------------------------------------- +# Re-running an upload, and what each index serves after it. + + +def test_both_uploads_skip_a_file_the_index_already_holds() -> None: + """A re-run after a partial upload uploads the rest instead of stopping on + the file already there (Copilot's review of openDox-code#78).""" + for job in ("testpypi", "pypi"): + uploads = [step for step in _workflow()["jobs"][job]["steps"] + if step.get("uses", "").startswith("pypa/gh-action-pypi-publish@")] + assert len(uploads) == 1, job + assert uploads[0]["with"]["skip-existing"] is True, job + + +INDEX_STEPS = {"TestPyPI": ("testpypi-install", "TestPyPI serves the files the build job verified"), + "PyPI": ("pypi", "PyPI serves the files the build job verified")} + + +def test_each_upload_is_followed_by_one_index_check_script() -> None: + """skip-existing can only skip a file name the index already holds, so + each upload is followed by the check that the index serves exactly the two + verified digests. Both checks are the same script, differing only in the + index they read.""" + jobs = _workflow()["jobs"] + steps = {index: _step(job, name) for index, (job, name) in INDEX_STEPS.items()} + assert steps["TestPyPI"]["run"] == steps["PyPI"]["run"] + assert steps["TestPyPI"]["env"]["JSON_BASE"] == "https://test.pypi.org/pypi/opendox/" + assert steps["PyPI"]["env"]["JSON_BASE"] == "https://pypi.org/pypi/opendox/" + for index, step in steps.items(): + assert step["env"]["INDEX"] == index + assert (step["env"]["READS"], step["env"]["PAUSE"]) == ("20", "15") + names = [step.get("name") or step.get("uses") for step in jobs["pypi"]["steps"]] + upload = next(i for i, step in enumerate(jobs["pypi"]["steps"]) + if step.get("uses", "").startswith("pypa/gh-action-pypi-publish@")) + assert names[upload + 1] == INDEX_STEPS["PyPI"][1], names + + +class _Index: + """A local JSON API: `answers` is the list of (status, files) it gives, + one per request, the last one repeated.""" + + def __init__(self, answers: list[tuple[int, dict[str, str]]]) -> None: + import http.server + import threading + + self.answers, self.requests = answers, [] + index = self + + class Handler(http.server.BaseHTTPRequestHandler): + def do_GET(self) -> None: # noqa: N802 (the stdlib's name) + index.requests.append(self.path) + status, files = index.answers[min(len(index.requests), len(index.answers)) - 1] + body = json.dumps({"urls": [{"filename": name, "digests": {"sha256": digest}} + for name, digest in files.items()]}).encode() + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def log_message(self, *args: object) -> None: + pass + + self.server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler) + threading.Thread(target=self.server.serve_forever, daemon=True).start() + self.base = f"http://127.0.0.1:{self.server.server_address[1]}/pypi/opendox/" + + def close(self) -> None: + self.server.shutdown() + self.server.server_close() + + +WHEEL_FILE, SDIST_FILE = "opendox-0.1.0-py3-none-any.whl", "opendox-0.1.0.tar.gz" +VERIFIED = {WHEEL_FILE: "a" * 64, SDIST_FILE: "b" * 64} + +INDEX_CASES = { + "it serves exactly the verified files": ([(200, VERIFIED)], None), + "it lags, then serves them": ([(404, {}), (200, {WHEEL_FILE: "a" * 64}), (200, VERIFIED)], None), + "it serves another sdist": ([(200, {**VERIFIED, SDIST_FILE: "c" * 64})], "after 3 reads"), + "it serves a third file": ([(200, {**VERIFIED, "opendox-0.1.0-py2-none-any.whl": "d" * 64})], + "after 3 reads"), + "it refuses the read": ([(403, {})], "HTTP Error 403"), +} + + +@needs_a_shell +@pytest.mark.parametrize("index", sorted(INDEX_STEPS)) +@pytest.mark.parametrize("case", sorted(INDEX_CASES)) +def test_the_index_check(index: str, case: str, tmp_path: Path) -> None: + answers, refusal = INDEX_CASES[case] + step = _step(*INDEX_STEPS[index]) + server = _Index(answers) + try: + env = {"INDEX": index, "JSON_BASE": server.base, "READS": "3", "PAUSE": "0", + "VERSION": "0.1.0", "WHEEL": WHEEL_FILE, "WHEEL_SHA256": "a" * 64, + "SDIST": SDIST_FILE, "SDIST_SHA256": "b" * 64} + result = _run(step["run"], tmp_path, env, (_python3(tmp_path / "bin"),)) + finally: + server.close() + assert server.requests and set(server.requests) == {"/pypi/opendox/0.1.0/json"} + if refusal is None: + assert result.returncode == 0, result.stdout + result.stderr + assert f"{index} serves exactly the verified files" in result.stdout + else: + assert result.returncode != 0, result.stdout + assert refusal in result.stdout + result.stderr, result.stdout + result.stderr + + +def _python3(where: Path) -> Path: + """A directory holding `python3`, this interpreter.""" + where.mkdir(parents=True, exist_ok=True) + python3 = where / "python3" + if not python3.exists(): + python3.symlink_to(sys.executable) + return where + + +def _seconds(text: str, pattern: str) -> int: + found = re.search(pattern, text) + assert found, pattern + return int(found.group(1)) + + +def test_each_job_timeout_covers_the_retries_it_promises() -> None: + """A job cut off by its timeout would break a promised retry (Copilot's + review of openDox-code#78), so each budget is recomputed from the steps.""" + jobs = _workflow()["jobs"] + check = _step(*INDEX_STEPS["TestPyPI"]) + per_read = _seconds(check["run"], r"urlopen\(url, timeout=(\d+)\)") + reads, pause = int(check["env"]["READS"]), int(check["env"]["PAUSE"]) + json_budget = reads * (per_read + pause) + install = _step("testpypi-install", "install opendox[local] from TestPyPI into a fresh venv, and run it")["run"] + tries = len(re.search(r"for attempt in ((?:\d+ ?)+); do", install).group(1).split()) + per_try = _seconds(install, r"timeout (\d+) \"\$RUNNER_TEMP/fresh/bin/python\" -m pip install") + wait = _seconds(install, r"sleep (\d+)") + setup = 5 * 60 + assert jobs["testpypi-install"]["timeout-minutes"] * 60 >= ( + json_budget + tries * (per_try + wait) + setup) + upload = 10 * 60 + assert jobs["pypi"]["timeout-minutes"] * 60 >= upload + json_budget + setup From 994a39d729a1833241e896494a2ae9a0d6d1e99f Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 14:05:03 +0000 Subject: [PATCH 06/15] Fix round 5: the root's pin is checked again right before each upload (Copilot review) Copilot's review at 74b87375 (r4171232529): the pin was checked only in the build job, and the run then waits on approvals (and, before PyPI, on TestPyPI), so a pin the openDox root moved meanwhile would not stop the irreversible upload. - The build job's release-commit step splits in two: the ref and main check (a tag must be v; the compare with this repository's main, through the API), and the pin check (the anonymous git read of opensoft/openDox main's contracts/code-pin.yaml, run with python3). - testpypi and pypi each run that pin step again, the same script, as the step right before their upload; it needs no token. - tests/test_release_workflow.py (62 cases): the release-commit cases run both build steps; the pin step carries no gh call and no env, and the ref step's one gh call is the compare; each publish job's step before its upload is the build job's pin step; a moved pin refuses there and an unchanged one passes, in each job. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 149 ++++++++++++++++++++++++++------- tests/test_release_workflow.py | 53 ++++++++++-- 2 files changed, 165 insertions(+), 37 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 3b40a601..69b3f246 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -25,20 +25,23 @@ name: release # must equal the root's pin and be on `main`. A dispatch runs at the head of # the ref it names, so it names the tag `v`, created at the pinned # commit, or `main` only while `main`'s head is still that commit. No other -# tag is accepted. +# tag is accepted. The pin is checked in the build job and AGAIN right before +# each upload, so a pin that moves while the run waits (on an approval, or on +# TestPyPI) stops the upload. # # FOUR JOBS, EACH GATED ON THE ONE BEFORE IT: # build builds the sdist and the wheel in a clean job, runs # `twine check`, VERIFIES both before any upload, and # installs the wheel into a fresh venv. It mints no # identity token, and its GITHUB_TOKEN reads only. -# testpypi the dry run: publishes those exact bytes to TestPyPI, -# in the `testpypi` environment. +# testpypi the dry run: re-checks the root's pin, then publishes +# those exact bytes to TestPyPI, in the `testpypi` +# environment. # testpypi-install installs `opendox[local]` from TestPyPI into a fresh # venv, as plan 034's T099 falsifier does, and runs it. -# pypi publishes the same bytes to PyPI, in the `pypi` -# environment, then requires PyPI to serve exactly -# those two files, by digest. +# pypi re-checks the root's pin, publishes the same bytes to +# PyPI, in the `pypi` environment, then requires PyPI to +# serve exactly those two files, by digest. # # NO PyPI TOKEN AND NO STORED SECRET, ANYWHERE. Both uploads use PyPI's # trusted publishing (OIDC): the job asks GitHub for a short-lived identity @@ -49,7 +52,8 @@ name: release # `GITHUB_TOKEN`, which the build job passes to `gh` for two reads of THIS # repository's API (the compare with `main`, and the environments), with # `contents: read` and `actions: read` and nothing more. The openDox root's -# pin, in another repository, is read over git with no credential at all. +# pin, in another repository, is read over git with no credential at all, in +# the build job and again right before each upload. # # EVERY ACTION IS PINNED BY FULL COMMIT SHA, with its release tag beside it. # (`validate.yml` pins by tag. A tag can be moved after review, and a commit @@ -108,21 +112,13 @@ jobs: with: python-version: '3.12' - # THE RELEASE IS THE PINNED COMMIT. This run's commit must (a) equal - # `commit:` in opensoft/openDox `main`'s `contracts/code-pin.yaml`, whose - # `source_repository` must be this repository, and (b) be on this - # repository's `main`, as `main` or an ancestor of it. `main`'s head - # need not be that commit. A dispatch on a tag must name `v`. - # A dispatch may name any ref, and the commit it names is what would be - # published, so this is checked first. - # - # THE ROOT'S PIN IS READ ANONYMOUSLY, OVER GIT. opensoft/openDox is - # public, and this job's GITHUB_TOKEN is an installation token scoped - # to this repository, so the read of another repository uses no token: - # a fetch of `main`, depth 1, with every credential helper cleared and - # no prompt, in a repository of its own under `$RUNNER_TEMP`. The - # compare with `main` reads this repository, through the API. - - name: the dispatched commit is the one the openDox root pins + # THE RELEASE IS THE PINNED COMMIT, ON `main`. A dispatch may name any + # ref, and the commit it names is what would be published, so these two + # steps come first. This one: a dispatch on a tag must name + # `v`, and the run's commit must be on this repository's + # `main`, as `main` or an ancestor of it (`main`'s head need not be that + # commit). The compare reads this repository, through the API. + - name: the dispatched commit is on main, dispatched on v or a branch env: GH_TOKEN: ${{ github.token }} run: | @@ -134,6 +130,27 @@ jobs: fi ;; esac + status="$(gh api "repos/$GITHUB_REPOSITORY/compare/$GITHUB_SHA...main" --jq .status)" + case "$status" in + identical|ahead) echo "$GITHUB_SHA is on main ($status), dispatched on $GITHUB_REF" ;; + *) + echo "::error::$GITHUB_SHA is not on this repository's main (compare: $status)" + exit 1 + ;; + esac + + # And this one: the run's commit must equal `commit:` in opensoft/openDox + # `main`'s `contracts/code-pin.yaml`, whose `source_repository` must be + # this repository. The two publish jobs run this same step again, right + # before their upload. + # + # THE ROOT'S PIN IS READ ANONYMOUSLY, OVER GIT. opensoft/openDox is + # public, and a job's GITHUB_TOKEN is an installation token scoped to + # this repository, so the read of another repository uses no token: a + # fetch of `main`, depth 1, with every credential helper cleared and no + # prompt, in a repository of its own under `$RUNNER_TEMP`. + - name: the dispatched commit is the one the openDox root pins + run: | root="$RUNNER_TEMP/opendox-root" rm -rf "$root" git init -q -b main "$root" @@ -143,7 +160,7 @@ jobs: echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" exit 1 fi - PIN="$pin" python - <<'PY' + PIN="$pin" python3 - <<'PY' import os, re, sys pin = os.environ["PIN"] @@ -165,14 +182,6 @@ jobs: f"at {pinned}") print(f"{this} is the commit opensoft/openDox main pins for {here}") PY - status="$(gh api "repos/$GITHUB_REPOSITORY/compare/$GITHUB_SHA...main" --jq .status)" - case "$status" in - identical|ahead) echo "$GITHUB_SHA is on main ($status), dispatched on $GITHUB_REF" ;; - *) - echo "::error::$GITHUB_SHA is not on this repository's main (compare: $status)" - exit 1 - ;; - esac # BOTH ENVIRONMENTS MUST ASK A REVIEWER. GitHub creates an environment a # job names if it does not exist yet, and creates it with no protection @@ -530,6 +539,45 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - + # THE PIN, AGAIN, RIGHT BEFORE THE UPLOAD: the build job's step, the same + # script. Between the build job's check and this upload the run waits on + # an approval (and, before PyPI, on TestPyPI too), and the openDox root + # may pin another commit meanwhile. An upload cannot be taken back, so a + # moved pin stops it here. + - name: the dispatched commit is the one the openDox root pins + run: | + root="$RUNNER_TEMP/opendox-root" + rm -rf "$root" + git init -q -b main "$root" + if ! GIT_TERMINAL_PROMPT=0 git -C "$root" -c credential.helper= \ + fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git main \ + || ! pin="$(git -C "$root" show FETCH_HEAD:contracts/code-pin.yaml)"; then + echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" + exit 1 + fi + PIN="$pin" python3 - <<'PY' + import os, re, sys + + pin = os.environ["PIN"] + + def field(name): + found = re.search(rf'^{name}:[ \t]*"?([^"\s#]+)"?[ \t]*(?:#.*)?$', pin, re.M) + return found.group(1) if found else None + + source, pinned = field("source_repository"), field("commit") + here, this = os.environ["GITHUB_REPOSITORY"], os.environ["GITHUB_SHA"] + if source != here: + sys.exit(f"::error::the openDox root's code pin names {source!r}, not {here!r}") + if not pinned or not re.fullmatch(r"[0-9a-f]{40}", pinned): + sys.exit(f"::error::the openDox root's code pin names no full commit: {pinned!r}") + if pinned != this: + sys.exit(f"::error::this run is at {this}, and the openDox root pins {pinned}; " + "a release publishes only the pinned commit. A dispatch runs at the " + "head of the ref it names, so dispatch on the tag v created " + f"at {pinned}") + print(f"{this} is the commit opensoft/openDox main pins for {here}") + PY + # skip-existing: a re-run after a partial upload skips a file the index # already holds, instead of stopping on it, and uploads the rest. An # index keeps each file name once, so a skipped file could differ from @@ -687,6 +735,45 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - + # THE PIN, AGAIN, RIGHT BEFORE THE UPLOAD: the build job's step, the same + # script. Between the build job's check and this upload the run waits on + # an approval (and, before PyPI, on TestPyPI too), and the openDox root + # may pin another commit meanwhile. An upload cannot be taken back, so a + # moved pin stops it here. + - name: the dispatched commit is the one the openDox root pins + run: | + root="$RUNNER_TEMP/opendox-root" + rm -rf "$root" + git init -q -b main "$root" + if ! GIT_TERMINAL_PROMPT=0 git -C "$root" -c credential.helper= \ + fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git main \ + || ! pin="$(git -C "$root" show FETCH_HEAD:contracts/code-pin.yaml)"; then + echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" + exit 1 + fi + PIN="$pin" python3 - <<'PY' + import os, re, sys + + pin = os.environ["PIN"] + + def field(name): + found = re.search(rf'^{name}:[ \t]*"?([^"\s#]+)"?[ \t]*(?:#.*)?$', pin, re.M) + return found.group(1) if found else None + + source, pinned = field("source_repository"), field("commit") + here, this = os.environ["GITHUB_REPOSITORY"], os.environ["GITHUB_SHA"] + if source != here: + sys.exit(f"::error::the openDox root's code pin names {source!r}, not {here!r}") + if not pinned or not re.fullmatch(r"[0-9a-f]{40}", pinned): + sys.exit(f"::error::the openDox root's code pin names no full commit: {pinned!r}") + if pinned != this: + sys.exit(f"::error::this run is at {this}, and the openDox root pins {pinned}; " + "a release publishes only the pinned commit. A dispatch runs at the " + "head of the ref it names, so dispatch on the tag v created " + f"at {pinned}") + print(f"{this} is the commit opensoft/openDox main pins for {here}") + PY + # skip-existing: a re-run after a partial upload skips a file the index # already holds, instead of stopping on it, and uploads the rest. An # index keeps each file name once, so a skipped file could differ from diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index da6f20c3..21a92010 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -29,8 +29,11 @@ * the release-commit step: the pinned commit on `main` passes, and a commit the openDox root does not pin, a commit off `main`, another repository's pin, a tag other than `v`, an unreachable root and a root with no - pin each refuse. The pin is read over git with no credential, and the - step's one `gh` call is the compare with this repository's `main`; + pin each refuse. The build job runs it as two steps, the ref and `main` + (its one `gh` call is the compare with this repository's `main`), and the + pin, read over git with no credential; + * the pin, again, as the step right before each upload, the build job's + script: a pin that moved meanwhile stops the upload; * the environment step: both environments with a reviewer pass, and either one without a reviewer, or unreadable, refuses; * the version gate, case by case, a pre-release and a development release @@ -187,6 +190,8 @@ def _pin(commit: str, source: str = REPOSITORY) -> str: ROOT_URL = "https://github.com/opensoft/openDox.git" +REF_STEP = "the dispatched commit is on main, dispatched on v or a branch" +PIN_STEP = "the dispatched commit is the one the openDox root pins" def _root(where: Path, pin: str | None) -> Path: @@ -255,7 +260,7 @@ def _redirect(to: Path) -> dict[str, str]: @pytest.mark.parametrize("case", sorted(RELEASE_COMMIT_CASES)) def test_the_release_commit_step(case: str, tmp_path: Path) -> None: given, refusal = RELEASE_COMMIT_CASES[case] - step = _step("build", "the dispatched commit is the one the openDox root pins") + script = "\n".join(_step("build", name)["run"] for name in (REF_STEP, PIN_STEP)) root = _root(tmp_path / "root", given["pin"]) if given.get("unreachable"): root = tmp_path / "no-such-root" @@ -264,7 +269,7 @@ def test_the_release_commit_step(case: str, tmp_path: Path) -> None: "GITHUB_REF": given.get("ref", "refs/heads/main"), "VERSION": "0.1.0", "RUNNER_TEMP": str(tmp_path / "rt"), "FAKE_STATUS": given["status"], "GH_TOKEN": "unused", **_redirect(root)} - result = _run(step["run"], tmp_path, env, (_fake_gh(tmp_path / "bin"),)) + result = _run(script, tmp_path, env, (_fake_gh(tmp_path / "bin"),)) if refusal is None: assert result.returncode == 0, result.stdout + result.stderr assert "is the commit opensoft/openDox main pins" in result.stdout @@ -274,14 +279,50 @@ def test_the_release_commit_step(case: str, tmp_path: Path) -> None: def test_the_root_pin_is_read_over_git_with_no_credential() -> None: - script = _step("build", "the dispatched commit is the one the openDox root pins")["run"] + step = _step("build", PIN_STEP) + script = step["run"] assert "contents/contracts/code-pin.yaml" not in script assert re.search(r'GIT_TERMINAL_PROMPT=0 git -C "\$root" -c credential\.helper= \\\n' r'\s+fetch -q --depth 1 --no-tags ' + re.escape(ROOT_URL) + r' main', script), script - calls = [line.strip() for line in script.splitlines() if re.search(r"\bgh\b", line)] + assert not re.search(r"\bgh\b", script) and "env" not in step, step + ref = _step("build", REF_STEP)["run"] + calls = [line.strip() for line in ref.splitlines() if re.search(r"\bgh\b", line)] assert len(calls) == 1 and "/compare/" in calls[0], calls +def test_each_upload_rechecks_the_pin_right_before_it() -> None: + """The run waits on approvals, and before PyPI on TestPyPI too, so the + root may pin another commit after the build job's check. Each publish + job runs the build job's pin step again, the same script, as the step + right before its upload (Copilot's review of openDox-code#78).""" + jobs = _workflow()["jobs"] + pin = _step("build", PIN_STEP) + for job in ("testpypi", "pypi"): + steps = jobs[job]["steps"] + upload = next(i for i, step in enumerate(steps) + if step.get("uses", "").startswith("pypa/gh-action-pypi-publish@")) + assert steps[upload - 1] == pin, (job, steps[upload - 1]) + + +@needs_a_shell +@pytest.mark.parametrize("job", ["testpypi", "pypi"]) +@pytest.mark.parametrize("moved", [False, True]) +def test_a_pin_that_moved_stops_the_upload(job: str, moved: bool, tmp_path: Path) -> None: + steps = _workflow()["jobs"][job]["steps"] + upload = next(i for i, step in enumerate(steps) + if step.get("uses", "").startswith("pypa/gh-action-pypi-publish@")) + root = _root(tmp_path / "root", _pin("2" * 40 if moved else PINNED)) + (tmp_path / "rt").mkdir() + env = {"GITHUB_REPOSITORY": REPOSITORY, "GITHUB_SHA": PINNED, + "RUNNER_TEMP": str(tmp_path / "rt"), **_redirect(root)} + result = _run(steps[upload - 1]["run"], tmp_path, env) + if moved: + assert result.returncode != 0, result.stdout + assert "a release publishes only the pinned commit" in result.stdout + result.stderr + else: + assert result.returncode == 0, result.stdout + result.stderr + + ENVIRONMENT_CASES = { "both name a reviewer": (dict(testpypi="1", pypi="1"), None), "pypi names none": (dict(testpypi="1", pypi="0"), "the pypi environment names no required reviewer"), From d7e9ff7b9a586a17980a930e55283cd5d523792c Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 14:26:42 +0000 Subject: [PATCH 07/15] Fix round 6: markers compared whole, and a preflight before each upload (Copilot review) Copilot's review at 5cdb2ded (r4173500546, r4173500563): - The requirement comparison keeps each requirement's marker, less only setuptools' extra-ownership clause (`and extra == "X"`, read from the parsed marker), and a direct URL; an extra under a top-level `or`, or nested, is refused. - Before each upload, the index must hold no file of this version but a verified one at its verified digest (a 404 is no file yet), so skip-existing can never make a mixed release. The same script in both publish jobs, before the pin's re-check. - tests/test_release_workflow.py (82 cases): marker cases (an extra's or a base requirement's marker changed or dropped, a direct URL, the extra's clause first, an extra under an or); the preflight's place and the index cases. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 183 +++++++++++++++++++++++++++++---- tests/test_release_workflow.py | 107 ++++++++++++++++++- 2 files changed, 266 insertions(+), 24 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 69b3f246..2f9f96e0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -285,8 +285,11 @@ jobs: # 2. the wheel's metadata names `opendox` at VERSION; # 3. the wheel declares exactly the requirements `pyproject.toml` does, # for the base install and for every extra, compared as normalized - # requirements; and its `local` extra (T072) carries - # `opendox[runtime]` and the bundled server's package; + # requirements: the name, the extras, the version specifier or + # direct URL, and the marker, less only the `extra == "..."` clause + # setuptools adds to say which extra a requirement belongs to; and + # its `local` extra (T072) carries `opendox[runtime]` and the + # bundled server's package; # 4. the console script `opendox` is declared, as `opendox.cli:main`; # 5. every file under `src/opendox/web/` that git tracks is in the # wheel, dotfiles included (T075's `web/**/.*`): 42 of 42 at @@ -338,29 +341,89 @@ jobs: extras = ",".join(sorted(canonicalize_name(e) for e in requirement.extras)) return canonicalize_name(requirement.name) + (f"[{extras}]" if extras else "") - def normal(requirement): - return named(requirement) + str(requirement.specifier) + # A MARKER IS COMPARED WHOLE, less only the clause that says which + # extra a requirement belongs to. setuptools writes a requirement + # of extra X as ` and extra == "X"`, wrapping its + # own marker in parentheses where it needs them. The parsed marker + # (packaging's `Marker._markers`: comparisons as triples, `and` + # and `or` as strings, parentheses as nested lists; the hash lock + # pins packaging) loses that top-level clause, and what is left is + # written out in one form for both sides. An `extra` anywhere else + # (under an `or`, or nested) is refused: it is not setuptools' + # form, and its ownership would not be plain. + def is_extra(node): + return isinstance(node, tuple) and any( + getattr(part, "value", None) == "extra" and type(part).__name__ == "Variable" + for part in (node[0], node[2])) + + def mentions_extra(nodes): + return any(mentions_extra(n) if isinstance(n, list) else is_extra(n) for n in nodes) + + def unwrap(nodes): + while len(nodes) == 1 and isinstance(nodes[0], list): + nodes = nodes[0] + return nodes + + def written(nodes): + return " ".join(f"({written(unwrap(n))})" if isinstance(n, list) + else n if isinstance(n, str) + else " ".join(part.serialize() for part in n) + for n in nodes) + + def split_marker(requirement): + if requirement.marker is None: + return "", [] + nodes = list(requirement.marker._markers) + if any(is_extra(n) and n[1].value != "==" for n in nodes): + raise ValueError("an extra clause that is not `extra == ...`") + owners = [n[2].value if type(n[0]).__name__ == "Variable" else n[0].value + for n in nodes if is_extra(n)] + if owners and "or" in nodes: + raise ValueError("an extra clause under a top-level `or`") + rest = [] + for n in nodes: + if is_extra(n): + if rest and rest[-1] == "and": + rest.pop() + continue + if not rest and n == "and": + continue + rest.append(n) + if mentions_extra(rest): + raise ValueError("an extra clause nested in the marker") + return written(unwrap(rest)), sorted({canonicalize_name(o) for o in owners}) + + def normal(requirement, marker): + pinned = f"@ {requirement.url}" if requirement.url else str(requirement.specifier) + return named(requirement) + pinned + (f"; {marker}" if marker else "") + + def declared_form(text): + requirement = Requirement(text) + marker, owners = split_marker(requirement) + if owners: + raise ValueError(f"pyproject.toml's {text!r} names an extra in its marker") + return normal(requirement, marker) with open("pyproject.toml", "rb") as handle: project = tomllib.load(handle)["project"] - declared = {"": {normal(Requirement(r)) for r in project.get("dependencies", [])}} + declared = {"": {declared_form(r) for r in project.get("dependencies", [])}} for extra, requirements in project.get("optional-dependencies", {}).items(): - declared[canonicalize_name(extra)] = {normal(Requirement(r)) for r in requirements} + declared[canonicalize_name(extra)] = {declared_form(r) for r in requirements} carried = {""} | {canonicalize_name(e) for e in meta.get_all("Provides-Extra") or []} carried = {extra: set() for extra in carried} local_names = set() for raw in meta.get_all("Requires-Dist") or []: requirement = Requirement(raw) - # The extra a requirement belongs to is read from its marker's - # text, so a platform marker beside it changes nothing. - owners = sorted({canonicalize_name(e) for e in - re.findall(r'extra\s*==\s*"([^"]+)"', str(requirement.marker or ""))}) + try: + marker, owners = split_marker(requirement) + except ValueError as error: + problems.append(f"the requirement {raw!r} carries {error}") + continue if any(owner not in carried for owner in owners): problems.append(f"the requirement {raw!r} names an extra the wheel does not provide") continue - requirement.marker = None for owner in owners or [""]: - carried[owner].add(normal(requirement)) + carried[owner].add(normal(requirement, marker)) if owner == "local": local_names.add(named(requirement)) print(f"3. requirements: the local extra carries {sorted(carried.get('local', ()))}") @@ -539,6 +602,46 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - + # BEFORE THE UPLOAD, NOTHING ELSE HELD. skip-existing skips a file name + # the index already holds, so the index must hold no file for this + # version but the verified ones, at the verified digests: otherwise the + # upload would put a verified file beside one nobody verified, and an + # index never replaces a file. A 404 is a version with no file yet. A + # re-run after a partial upload passes, and uploads only what is missing. + - name: TestPyPI holds no file of this version but the verified ones + env: + INDEX: TestPyPI + JSON_BASE: https://test.pypi.org/pypi/opendox/ + WHEEL: ${{ needs.build.outputs.wheel }} + WHEEL_SHA256: ${{ needs.build.outputs.wheel-sha256 }} + SDIST: ${{ needs.build.outputs.sdist }} + SDIST_SHA256: ${{ needs.build.outputs.sdist-sha256 }} + run: | + python3 - <<'PY' + import json, os, sys, urllib.error, urllib.request + + index = os.environ["INDEX"] + url = f"{os.environ['JSON_BASE']}{os.environ['VERSION']}/json" + wanted = {os.environ["WHEEL"]: os.environ["WHEEL_SHA256"], + os.environ["SDIST"]: os.environ["SDIST_SHA256"]} + try: + with urllib.request.urlopen(url, timeout=30) as response: + release = json.load(response) + except urllib.error.HTTPError as error: + if error.code != 404: + raise + print(f"{index} holds no file of opendox {os.environ['VERSION']} yet") + sys.exit(0) + held = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} + other = {name: digest for name, digest in held.items() if wanted.get(name) != digest} + if other: + sys.exit(f"::error::{index} already holds {other} for opendox " + f"{os.environ['VERSION']}, which the build job did not verify; " + "an upload now would make a mixed release, so nothing is uploaded") + print(f"{index} holds {sorted(held)} for opendox {os.environ['VERSION']}, each at " + "the verified digest; skip-existing uploads only what is missing") + PY + # THE PIN, AGAIN, RIGHT BEFORE THE UPLOAD: the build job's step, the same # script. Between the build job's check and this upload the run waits on # an approval (and, before PyPI, on TestPyPI too), and the openDox root @@ -579,10 +682,10 @@ jobs: PY # skip-existing: a re-run after a partial upload skips a file the index - # already holds, instead of stopping on it, and uploads the rest. An - # index keeps each file name once, so a skipped file could differ from - # the verified one; the JSON step that follows each upload requires the - # index to serve exactly the build job's two digests, so it cannot pass. + # already holds, instead of stopping on it, and uploads the rest. The + # step before the pin's re-check refused any file but a verified one at + # its verified digest, and the JSON step after the upload requires the + # index to serve exactly the build job's two digests. - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 with: repository-url: https://test.pypi.org/legacy/ @@ -735,6 +838,46 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - + # BEFORE THE UPLOAD, NOTHING ELSE HELD. skip-existing skips a file name + # the index already holds, so the index must hold no file for this + # version but the verified ones, at the verified digests: otherwise the + # upload would put a verified file beside one nobody verified, and an + # index never replaces a file. A 404 is a version with no file yet. A + # re-run after a partial upload passes, and uploads only what is missing. + - name: PyPI holds no file of this version but the verified ones + env: + INDEX: PyPI + JSON_BASE: https://pypi.org/pypi/opendox/ + WHEEL: ${{ needs.build.outputs.wheel }} + WHEEL_SHA256: ${{ needs.build.outputs.wheel-sha256 }} + SDIST: ${{ needs.build.outputs.sdist }} + SDIST_SHA256: ${{ needs.build.outputs.sdist-sha256 }} + run: | + python3 - <<'PY' + import json, os, sys, urllib.error, urllib.request + + index = os.environ["INDEX"] + url = f"{os.environ['JSON_BASE']}{os.environ['VERSION']}/json" + wanted = {os.environ["WHEEL"]: os.environ["WHEEL_SHA256"], + os.environ["SDIST"]: os.environ["SDIST_SHA256"]} + try: + with urllib.request.urlopen(url, timeout=30) as response: + release = json.load(response) + except urllib.error.HTTPError as error: + if error.code != 404: + raise + print(f"{index} holds no file of opendox {os.environ['VERSION']} yet") + sys.exit(0) + held = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} + other = {name: digest for name, digest in held.items() if wanted.get(name) != digest} + if other: + sys.exit(f"::error::{index} already holds {other} for opendox " + f"{os.environ['VERSION']}, which the build job did not verify; " + "an upload now would make a mixed release, so nothing is uploaded") + print(f"{index} holds {sorted(held)} for opendox {os.environ['VERSION']}, each at " + "the verified digest; skip-existing uploads only what is missing") + PY + # THE PIN, AGAIN, RIGHT BEFORE THE UPLOAD: the build job's step, the same # script. Between the build job's check and this upload the run waits on # an approval (and, before PyPI, on TestPyPI too), and the openDox root @@ -775,10 +918,10 @@ jobs: PY # skip-existing: a re-run after a partial upload skips a file the index - # already holds, instead of stopping on it, and uploads the rest. An - # index keeps each file name once, so a skipped file could differ from - # the verified one; the JSON step that follows each upload requires the - # index to serve exactly the build job's two digests, so it cannot pass. + # already holds, instead of stopping on it, and uploads the rest. The + # step before the pin's re-check refused any file but a verified one at + # its verified digest, and the JSON step after the upload requires the + # index to serve exactly the build job's two digests. - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 with: skip-existing: true diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index 21a92010..3118fd08 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -34,12 +34,16 @@ pin, read over git with no credential; * the pin, again, as the step right before each upload, the build job's script: a pin that moved meanwhile stops the upload; + * before that, the preflight: the index holds no file of this version but a + verified one at its verified digest, so skip-existing can never make a + mixed release; * the environment step: both environments with a reviewer pass, and either one without a reviewer, or unreadable, refuses; * the version gate, case by case, a pre-release and a development release among the refusals; * the artifact checks, over a small built-by-hand wheel and sdist in a git - tree of their own: the whole set passes, and each kind of gap refuses; + tree of their own: the whole set passes, and each kind of gap refuses, + a changed or dropped marker and a direct URL among them; * the publish jobs' digest check: the verified files pass, and a changed byte or a third file refuses. """ @@ -385,16 +389,29 @@ def test_the_version_gate(case: str, tmp_path: Path) -> None: PYPROJECT = """[project] name = "opendox" version = "0.1.0" -dependencies = ["PyYAML>=6.0"] +dependencies = ["PyYAML>=6.0", "colorama>=0.4; sys_platform == 'win32'"] [project.optional-dependencies] -runtime = ["fastapi>=0.115"] +runtime = ["fastapi>=0.115", "uvloop>=0.19; sys_platform != 'win32' or python_version < '3.13'"] local = ["opendox[runtime]", "pixeltable-pgserver>=0.6.0"] """ -REQUIRES = ["PyYAML>=6.0", 'fastapi>=0.115; extra == "runtime"', +# The wheel's requirements as setuptools writes them for PYPROJECT: a +# requirement of an extra gains `and extra == "..."`, and its own marker is +# parenthesized where it needs to be (measured with the locked setuptools). +REQUIRES = ["PyYAML>=6.0", 'colorama>=0.4; sys_platform == "win32"', + 'fastapi>=0.115; extra == "runtime"', + 'uvloop>=0.19; (sys_platform != "win32" or python_version < "3.13") and extra == "runtime"', 'opendox[runtime]; extra == "local"', 'pixeltable-pgserver>=0.6.0; extra == "local"'] + +def _requires(old: str, new: str) -> list[str]: + assert old in REQUIRES, old + return [new if r == old else r for r in REQUIRES] + + +UVLOOP = REQUIRES[3] + TREE = { "src/.gitkeep": "", "src/opendox/__init__.py": "", @@ -456,6 +473,29 @@ def _wheel(where: Path, *, drop: str = "", add: str = "", requires: list[str] | dict(requires=[*REQUIRES, 'httpx>=0.27; extra == "local"']), "are not pyproject.toml's"), "no console script": (dict(scripts="opendox-runtime = opendox.runtime.cli:main"), "the console script `opendox = opendox.cli:main` is not declared"), + # The markers, compared whole but for the extra's own clause (Copilot's + # review of openDox-code#78). + "an extra's marker changed": ( + dict(requires=_requires(UVLOOP, 'uvloop>=0.19; sys_platform == "linux" and extra == "runtime"')), + "are not pyproject.toml's"), + "an extra's marker dropped": ( + dict(requires=_requires(UVLOOP, 'uvloop>=0.19; extra == "runtime"')), "are not pyproject.toml's"), + "a base requirement's marker changed": ( + dict(requires=_requires('colorama>=0.4; sys_platform == "win32"', + 'colorama>=0.4; sys_platform == "linux"')), "are not pyproject.toml's"), + "a base requirement's marker dropped": ( + dict(requires=_requires('colorama>=0.4; sys_platform == "win32"', "colorama>=0.4")), + "are not pyproject.toml's"), + "a direct URL in place of the version": ( + dict(requires=_requires('fastapi>=0.115; extra == "runtime"', + 'fastapi @ https://example.invalid/fastapi.whl ; extra == "runtime"')), + "are not pyproject.toml's"), + "the extra's clause first, unparenthesized": ( + dict(requires=_requires(UVLOOP, 'uvloop>=0.19; extra == "runtime" and (sys_platform != "win32" ' + 'or python_version < "3.13")')), None), + "an extra under an or": ( + dict(requires=_requires(UVLOOP, 'uvloop>=0.19; extra == "runtime" or sys_platform != "win32"')), + "an extra clause under a top-level `or`"), } @@ -654,3 +694,62 @@ def test_each_job_timeout_covers_the_retries_it_promises() -> None: json_budget + tries * (per_try + wait) + setup) upload = 10 * 60 assert jobs["pypi"]["timeout-minutes"] * 60 >= upload + json_budget + setup + + +# --------------------------------------------------------------------------- +# Before each upload: the index holds no file of this version but a verified +# one (Copilot's review of openDox-code#78: skip-existing must never make a +# mixed release). + +PREFLIGHT_STEPS = {"TestPyPI": ("testpypi", "TestPyPI holds no file of this version but the verified ones"), + "PyPI": ("pypi", "PyPI holds no file of this version but the verified ones")} + + +def test_each_upload_is_preceded_by_the_preflight_then_the_pin() -> None: + jobs = _workflow()["jobs"] + scripts = set() + for index, (job, name) in PREFLIGHT_STEPS.items(): + steps = jobs[job]["steps"] + upload = next(i for i, step in enumerate(steps) + if step.get("uses", "").startswith("pypa/gh-action-pypi-publish@")) + assert steps[upload - 1]["name"] == PIN_STEP, job + assert steps[upload - 2]["name"] == name, job + assert steps[upload - 2]["env"]["INDEX"] == index + assert steps[upload]["with"]["skip-existing"] is True + scripts.add(steps[upload - 2]["run"]) + assert len(scripts) == 1, "the two preflights are one script" + assert _step(*PREFLIGHT_STEPS["TestPyPI"])["env"]["JSON_BASE"] == "https://test.pypi.org/pypi/opendox/" + assert _step(*PREFLIGHT_STEPS["PyPI"])["env"]["JSON_BASE"] == "https://pypi.org/pypi/opendox/" + + +PREFLIGHT_CASES = { + "no file of this version yet": ([(404, {})], None), + "both verified files, from an earlier try": ([(200, VERIFIED)], None), + "one verified file, from a partial upload": ([(200, {WHEEL_FILE: "a" * 64})], None), + "the sdist at another digest": ([(200, {SDIST_FILE: "c" * 64})], "would make a mixed release"), + "a file nobody verified": ([(200, {"opendox-0.1.0-py2-none-any.whl": "d" * 64})], + "would make a mixed release"), + "the index refuses the read": ([(503, {})], "HTTP Error 503"), +} + + +@needs_a_shell +@pytest.mark.parametrize("index", sorted(PREFLIGHT_STEPS)) +@pytest.mark.parametrize("case", sorted(PREFLIGHT_CASES)) +def test_the_preflight(index: str, case: str, tmp_path: Path) -> None: + answers, refusal = PREFLIGHT_CASES[case] + step = _step(*PREFLIGHT_STEPS[index]) + server = _Index(answers) + try: + env = {"INDEX": index, "JSON_BASE": server.base, "VERSION": "0.1.0", + "WHEEL": WHEEL_FILE, "WHEEL_SHA256": "a" * 64, + "SDIST": SDIST_FILE, "SDIST_SHA256": "b" * 64} + result = _run(step["run"], tmp_path, env, (_python3(tmp_path / "bin"),)) + finally: + server.close() + assert server.requests == ["/pypi/opendox/0.1.0/json"] + if refusal is None: + assert result.returncode == 0, result.stdout + result.stderr + else: + assert result.returncode != 0, result.stdout + assert refusal in result.stdout + result.stderr, result.stdout + result.stderr From d91fd5b97ff8806747566fd22323ec35a4c1ed26 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 15:37:00 +0000 Subject: [PATCH 08/15] Fix round 6: a changed direct URL is refused too (a case the URL comparison needs) The artifact tests' pyproject gains a direct-URL requirement, and a case whose wheel names another URL for it; with the URL dropped from the comparison, that case passes, so it holds the URL half of r4173500546. 83 cases. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_release_workflow.py | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index 3118fd08..3a31afad 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -392,7 +392,8 @@ def test_the_version_gate(case: str, tmp_path: Path) -> None: dependencies = ["PyYAML>=6.0", "colorama>=0.4; sys_platform == 'win32'"] [project.optional-dependencies] -runtime = ["fastapi>=0.115", "uvloop>=0.19; sys_platform != 'win32' or python_version < '3.13'"] +runtime = ["fastapi>=0.115", "uvloop>=0.19; sys_platform != 'win32' or python_version < '3.13'", + "demo-plugin @ https://example.invalid/demo_plugin-1.0-py3-none-any.whl"] local = ["opendox[runtime]", "pixeltable-pgserver>=0.6.0"] """ @@ -402,6 +403,7 @@ def test_the_version_gate(case: str, tmp_path: Path) -> None: REQUIRES = ["PyYAML>=6.0", 'colorama>=0.4; sys_platform == "win32"', 'fastapi>=0.115; extra == "runtime"', 'uvloop>=0.19; (sys_platform != "win32" or python_version < "3.13") and extra == "runtime"', + 'demo-plugin@ https://example.invalid/demo_plugin-1.0-py3-none-any.whl ; extra == "runtime"', 'opendox[runtime]; extra == "local"', 'pixeltable-pgserver>=0.6.0; extra == "local"'] @@ -490,6 +492,11 @@ def _wheel(where: Path, *, drop: str = "", add: str = "", requires: list[str] | dict(requires=_requires('fastapi>=0.115; extra == "runtime"', 'fastapi @ https://example.invalid/fastapi.whl ; extra == "runtime"')), "are not pyproject.toml's"), + "a direct URL changed": ( + dict(requires=_requires( + 'demo-plugin@ https://example.invalid/demo_plugin-1.0-py3-none-any.whl ; extra == "runtime"', + 'demo-plugin@ https://example.invalid/demo_plugin-6.6-py3-none-any.whl ; extra == "runtime"')), + "are not pyproject.toml's"), "the extra's clause first, unparenthesized": ( dict(requires=_requires(UVLOOP, 'uvloop>=0.19; extra == "runtime" and (sys_platform != "win32" ' 'or python_version < "3.13")')), None), From 960d549b12b8aaad4d7f2dec921a03ad4c09a297 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 16:11:39 +0000 Subject: [PATCH 09/15] Fix round 7: every step after the build declares its timeout, and the test reads each bound (Copilot review) Copilot's review at d91fd5b9 (r4173861533, r4173861563): the PyPI upload had no bound of its own, so it could take the time the JSON check after it is owed, and the test assumed a 10-minute upload instead of reading it. - Every step of testpypi, testpypi-install and pypi declares timeout-minutes (the uploads 10, each JSON check 16, the install 60, the pin step 3, here and in the build job, where it stays the same step), and each job's timeout covers the sum: testpypi 25, testpypi-install 90, pypi 40. A cut-off upload stays recoverable through the preflight and skip-existing. - test_each_job_timeout_covers_the_retries_it_promises reads every bound from the workflow: each step declares one, each job covers the sum, each upload is at most 10 minutes, and each JSON check and the install cover their scripts' retry budgets. Mutants: the PyPI upload unbounded, at 30 minutes, the pypi job at 30, PyPI's JSON step at 10: each fails. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 44 ++++++++++++++++++++++++++-------- tests/test_release_workflow.py | 40 ++++++++++++++++++++----------- 2 files changed, 60 insertions(+), 24 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 2f9f96e0..849d7d5e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -150,6 +150,7 @@ jobs: # fetch of `main`, depth 1, with every credential helper cleared and no # prompt, in a repository of its own under `$RUNNER_TEMP`. - name: the dispatched commit is the one the openDox root pins + timeout-minutes: 3 run: | root="$RUNNER_TEMP/opendox-root" rm -rf "$root" @@ -572,7 +573,10 @@ jobs: name: publish to TestPyPI (the dry run) needs: build runs-on: ubuntu-latest - timeout-minutes: 15 + # THE BUDGET: every step declares its own timeout, and the job's covers + # their sum (5 + 2 + 2 + 3 + 10 = 22 min); the upload's is 10 minutes. + # tests/test_release_workflow.py reads each bound from this file. + timeout-minutes: 25 environment: name: testpypi url: https://test.pypi.org/project/opendox/${{ inputs.version }}/ @@ -580,6 +584,7 @@ jobs: id-token: write # trusted publishing: the OIDC token TestPyPI accepts steps: - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + timeout-minutes: 5 with: name: dist path: dist @@ -587,6 +592,7 @@ jobs: # The files about to be uploaded are the two the build job verified, # byte for byte, and nothing else. - name: the files are the ones the build job verified + timeout-minutes: 2 env: WHEEL: ${{ needs.build.outputs.wheel }} WHEEL_SHA256: ${{ needs.build.outputs.wheel-sha256 }} @@ -609,6 +615,7 @@ jobs: # index never replaces a file. A 404 is a version with no file yet. A # re-run after a partial upload passes, and uploads only what is missing. - name: TestPyPI holds no file of this version but the verified ones + timeout-minutes: 2 env: INDEX: TestPyPI JSON_BASE: https://test.pypi.org/pypi/opendox/ @@ -648,6 +655,7 @@ jobs: # may pin another commit meanwhile. An upload cannot be taken back, so a # moved pin stops it here. - name: the dispatched commit is the one the openDox root pins + timeout-minutes: 3 run: | root="$RUNNER_TEMP/opendox-root" rm -rf "$root" @@ -687,6 +695,7 @@ jobs: # its verified digest, and the JSON step after the upload requires the # index to serve exactly the build job's two digests. - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 + timeout-minutes: 10 with: repository-url: https://test.pypi.org/legacy/ skip-existing: true @@ -695,12 +704,13 @@ jobs: name: install opendox[local] from TestPyPI needs: [build, testpypi] runs-on: ubuntu-latest - # THE BUDGET, in full, so no retry this job promises is cut short: - # the JSON step: 20 reads, each up to 30 s, 15 s apart 15 min - # the install: 10 tries, each up to 300 s, 30 s apart 55 min - # checkout, setup-python, the venvs and --help 5 min - # tests/test_release_workflow.py recomputes it from the steps. - timeout-minutes: 75 + # THE BUDGET, in full, so no retry this job promises is cut short. Every + # step declares its own timeout, each retrying step's covers its retries, + # and the job's covers their sum (5 + 5 + 16 + 60 = 86 min): + # the JSON step: 20 reads, each up to 30 s, 15 s apart 15 min (16) + # the install: 10 tries, each up to 300 s, 30 s apart 55 min (60) + # tests/test_release_workflow.py reads each bound from this file. + timeout-minutes: 90 permissions: contents: read env: @@ -711,10 +721,12 @@ jobs: steps: # For `constraints-cpython312-linux.txt` alone. - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + timeout-minutes: 5 with: persist-credentials: false - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + timeout-minutes: 5 with: python-version: '3.12' @@ -724,6 +736,7 @@ jobs: # serves exactly those two files, before this step refuses. The `pypi` # job runs the same script against PyPI after its upload. - name: TestPyPI serves the files the build job verified + timeout-minutes: 16 env: INDEX: TestPyPI JSON_BASE: https://test.pypi.org/pypi/opendox/ @@ -776,6 +789,7 @@ jobs: # spoil is its own verdict, and the files PyPI receives are still the # build job's, checked by digest in the `pypi` job. - name: install opendox[local] from TestPyPI into a fresh venv, and run it + timeout-minutes: 60 run: | installed=0 for attempt in 1 2 3 4 5 6 7 8 9 10; do @@ -808,9 +822,13 @@ jobs: name: publish to PyPI needs: [build, testpypi-install] runs-on: ubuntu-latest - # THE BUDGET: the upload (10 min), then the JSON step, 20 reads, each up - # to 30 s, 15 s apart (15 min), and the download (5 min). - timeout-minutes: 30 + # THE BUDGET: every step declares its own timeout, and the job's covers + # their sum (5 + 2 + 2 + 3 + 10 + 16 = 38 min). The upload is bounded at + # 10 minutes, so the JSON step after it always has its 20 reads, each up + # to 30 s, 15 s apart (15 min, bounded at 16). A cut-off upload stays + # recoverable: a re-run's preflight and skip-existing upload the rest. + # tests/test_release_workflow.py reads each bound from this file. + timeout-minutes: 40 environment: name: pypi url: https://pypi.org/project/opendox/${{ inputs.version }}/ @@ -818,11 +836,13 @@ jobs: id-token: write # trusted publishing: the OIDC token PyPI accepts steps: - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + timeout-minutes: 5 with: name: dist path: dist - name: the files are the ones the build job verified + timeout-minutes: 2 env: WHEEL: ${{ needs.build.outputs.wheel }} WHEEL_SHA256: ${{ needs.build.outputs.wheel-sha256 }} @@ -845,6 +865,7 @@ jobs: # index never replaces a file. A 404 is a version with no file yet. A # re-run after a partial upload passes, and uploads only what is missing. - name: PyPI holds no file of this version but the verified ones + timeout-minutes: 2 env: INDEX: PyPI JSON_BASE: https://pypi.org/pypi/opendox/ @@ -884,6 +905,7 @@ jobs: # may pin another commit meanwhile. An upload cannot be taken back, so a # moved pin stops it here. - name: the dispatched commit is the one the openDox root pins + timeout-minutes: 3 run: | root="$RUNNER_TEMP/opendox-root" rm -rf "$root" @@ -923,12 +945,14 @@ jobs: # its verified digest, and the JSON step after the upload requires the # index to serve exactly the build job's two digests. - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 + timeout-minutes: 10 with: skip-existing: true # PyPI serves exactly the two files the build job verified: the same # script as the TestPyPI check, against PyPI. - name: PyPI serves the files the build job verified + timeout-minutes: 16 env: INDEX: PyPI JSON_BASE: https://pypi.org/pypi/opendox/ diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index 3a31afad..cee84593 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -686,21 +686,33 @@ def _seconds(text: str, pattern: str) -> int: def test_each_job_timeout_covers_the_retries_it_promises() -> None: """A job cut off by its timeout would break a promised retry (Copilot's - review of openDox-code#78), so each budget is recomputed from the steps.""" + review of openDox-code#78). Every bound is read from the workflow: each + step of the jobs after the build declares its own timeout, each job's + timeout covers the sum of its steps', and each retrying step's timeout + covers the retries its script promises. So an upload can never take the + time the JSON check after it is owed.""" jobs = _workflow()["jobs"] - check = _step(*INDEX_STEPS["TestPyPI"]) - per_read = _seconds(check["run"], r"urlopen\(url, timeout=(\d+)\)") - reads, pause = int(check["env"]["READS"]), int(check["env"]["PAUSE"]) - json_budget = reads * (per_read + pause) - install = _step("testpypi-install", "install opendox[local] from TestPyPI into a fresh venv, and run it")["run"] - tries = len(re.search(r"for attempt in ((?:\d+ ?)+); do", install).group(1).split()) - per_try = _seconds(install, r"timeout (\d+) \"\$RUNNER_TEMP/fresh/bin/python\" -m pip install") - wait = _seconds(install, r"sleep (\d+)") - setup = 5 * 60 - assert jobs["testpypi-install"]["timeout-minutes"] * 60 >= ( - json_budget + tries * (per_try + wait) + setup) - upload = 10 * 60 - assert jobs["pypi"]["timeout-minutes"] * 60 >= upload + json_budget + setup + for job in ("testpypi", "testpypi-install", "pypi"): + steps = jobs[job]["steps"] + bounds = [step.get("timeout-minutes") for step in steps] + assert all(isinstance(bound, int) and bound > 0 for bound in bounds), (job, bounds) + assert jobs[job]["timeout-minutes"] >= sum(bounds), (job, jobs[job]["timeout-minutes"], bounds) + for step in steps: + if step.get("uses", "").startswith("pypa/gh-action-pypi-publish@"): + assert step["timeout-minutes"] <= 10, (job, step) + + for index, (job, name) in INDEX_STEPS.items(): + check = _step(job, name) + per_read = _seconds(check["run"], r"urlopen\(url, timeout=(\d+)\)") + reads, pause = int(check["env"]["READS"]), int(check["env"]["PAUSE"]) + assert check["timeout-minutes"] * 60 >= reads * (per_read + pause) + 30, index + + install = _step("testpypi-install", "install opendox[local] from TestPyPI into a fresh venv, and run it") + tries = len(re.search(r"for attempt in ((?:\d+ ?)+); do", install["run"]).group(1).split()) + per_try = _seconds(install["run"], r"timeout (\d+) \"\$RUNNER_TEMP/fresh/bin/python\" -m pip install") + wait = _seconds(install["run"], r"sleep (\d+)") + venvs = 30 * tries # a venv, the version read and `opendox --help`, per try + assert install["timeout-minutes"] * 60 >= tries * (per_try + wait) + venvs # --------------------------------------------------------------------------- From b75205d43bed4b13eecb539d706185553057f082 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 18:20:34 +0000 Subject: [PATCH 10/15] Fix round 8: the dry run proves it installed the verified TestPyPI wheel before running it (Copilot review) Copilot's review at 960d549b (r4173890172): with PyPI as the extra index, pip merges candidates from both, and opendox is not reserved on PyPI before its first release, so the unversioned install line could resolve an opendox PyPI serves; the step checked only the version string, and ran the venv's interpreter to read it. - The install line stays T099's falsifier as written (unversioned, wheels only) and gains --report. This job's own python3, never the venv's, reads the report before anything from the install runs: the one opendox installed must come from test-files.pythonhosted.org, at VERSION, with the verified wheel's sha256. An older opendox (an index that lags) is tried again; any other opendox (another host, another digest, VERSION or above) refuses at once. opendox --help runs only after that. - tests/test_release_workflow.py (93 cases): the install line's parts and that nothing from the venv runs but pip before the verdict; nine verdict cases over pip reports (the verified wheel; PyPI at, above and below the version, and at the verified digest; TestPyPI older, newer, at another digest; no opendox). Mutants: the host ignored, the digest ignored, the venv's interpreter run before the verdict, no --report: each fails. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 70 +++++++++++++++++++++++++++----- tests/test_release_workflow.py | 73 ++++++++++++++++++++++++++++++++++ 2 files changed, 133 insertions(+), 10 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 849d7d5e..06c5c533 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -776,12 +776,21 @@ jobs: # `pip install --index-url https://test.pypi.org/simple/ # --extra-index-url https://pypi.org/simple/ "opendox[local]"`. The # requirement is the install line as written, unversioned. The step adds - # this package's lock for the dependencies and wheels only, and then - # requires the installed opendox to be the version just uploaded, so an - # index that lags, or an older release, cannot pass for it. The simple - # index can lag the JSON API, so each try is a fresh venv, up to ten - # tries, 30 seconds apart, and each try is cut off after 300 seconds, - # so the job's budget holds. + # this package's lock for the dependencies, wheels only (so no package's + # code runs at install time), and `--report`. The simple index can lag + # the JSON API, so each try is a fresh venv, up to ten tries, 30 seconds + # apart, and each try is cut off after 300 seconds, so the job's budget + # holds. + # + # THE INSTALLED opendox IS THE VERIFIED WHEEL, proven before anything + # from it runs. pip merges candidates from both indexes, and `opendox` + # is not reserved on PyPI before its first release, so the line could + # resolve an `opendox` PyPI serves. This job's own Python, never the + # venv's, reads pip's report: the `opendox` installed must come from + # TestPyPI's file host, at VERSION, with the sha256 of the wheel the + # build job verified. An `opendox` from any other host, or at VERSION + # or above with another digest, refuses at once; an older one (an index + # that lags) is tried again. Only then does `opendox --help` run. # # THE TRADE, stated: with two indexes, pip may take a dependency from # either. The lock pins every dependency's version, and this job holds no @@ -793,19 +802,60 @@ jobs: run: | installed=0 for attempt in 1 2 3 4 5 6 7 8 9 10; do - rm -rf "$RUNNER_TEMP/fresh" + rm -rf "$RUNNER_TEMP/fresh" "$RUNNER_TEMP/report.json" python -m venv "$RUNNER_TEMP/fresh" if timeout 300 "$RUNNER_TEMP/fresh/bin/python" -m pip install --disable-pip-version-check \ --no-cache-dir --only-binary :all: -c constraints-cpython312-linux.txt \ + --report "$RUNNER_TEMP/report.json" \ --index-url https://test.pypi.org/simple/ \ --extra-index-url https://pypi.org/simple/ \ "opendox[local]"; then - got="$("$RUNNER_TEMP/fresh/bin/python" -c 'import importlib.metadata; print(importlib.metadata.version("opendox"))')" - if [ "$got" = "$VERSION" ]; then + verdict=0 + python3 - "$RUNNER_TEMP/report.json" <<'PY' || verdict=$? + # 0: the verified TestPyPI wheel; 1: an older opendox, try again; + # 2: anything else, which refuses at once. + import json, os, sys, urllib.parse + from pip._vendor.packaging.utils import canonicalize_name + from pip._vendor.packaging.version import InvalidVersion, Version + + version, wheel_sha256 = os.environ["VERSION"], os.environ["WHEEL_SHA256"] + with open(sys.argv[1], encoding="utf-8") as handle: + report = json.load(handle) + items = [item for item in report.get("install", []) + if canonicalize_name(item["metadata"]["name"]) == "opendox"] + if len(items) != 1: + print(f"::error::pip's report names {len(items)} opendox distributions, not one") + sys.exit(2) + item = items[0] + got = item["metadata"]["version"] + info = item.get("download_info") or {} + host = urllib.parse.urlsplit(info.get("url", "")).hostname + digest = ((info.get("archive_info") or {}).get("hashes") or {}).get("sha256") + try: + newer = Version(got) >= Version(version) + except InvalidVersion: + newer = True + if host == "test-files.pythonhosted.org" and got == version and digest == wheel_sha256: + print(f"opendox {got} is the verified wheel, from {host}, sha256 {digest}") + sys.exit(0) + if host == "test-files.pythonhosted.org" and not newer: + print(f"TestPyPI resolved opendox {got} so far, not {version}") + sys.exit(1) + if host != "test-files.pythonhosted.org" and not newer: + print(f"opendox {got} came from {host}, older than {version}; TestPyPI lags") + sys.exit(1) + print(f"::error::the install line installed opendox {got} from {host}, sha256 " + f"{digest}, which is not the verified wheel (opendox {version}, sha256 " + f"{wheel_sha256}, from TestPyPI); nothing from it is run") + sys.exit(2) + PY + if [ "$verdict" = 0 ]; then installed=1 break fi - echo "attempt $attempt: the install line resolved opendox $got, not $VERSION" + if [ "$verdict" != 1 ]; then + exit 1 + fi fi echo "attempt $attempt: opendox $VERSION is not installed yet; retrying in 30 seconds" sleep 30 diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index cee84593..b2c2d767 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -772,3 +772,76 @@ def test_the_preflight(index: str, case: str, tmp_path: Path) -> None: else: assert result.returncode != 0, result.stdout assert refusal in result.stdout + result.stderr, result.stdout + result.stderr + + +# --------------------------------------------------------------------------- +# The dry run installs the verified TestPyPI wheel, proven before it runs +# (Copilot's review of openDox-code#78: with PyPI as the extra index, the +# unversioned line could resolve an `opendox` that PyPI serves). + +INSTALL_STEP = "install opendox[local] from TestPyPI into a fresh venv, and run it" + + +def _verdict_script() -> str: + run = _step("testpypi-install", INSTALL_STEP)["run"] + found = re.search(r"python3 - \"\$RUNNER_TEMP/report\.json\" <<'PY' \|\| verdict=\$\?\n(.*?)\nPY\n", run, re.S) + assert found, run + return found.group(1) + + +def test_the_install_line_is_the_falsifiers_and_reports_what_it_installed() -> None: + run = _step("testpypi-install", INSTALL_STEP)["run"] + install = run[run.index("-m pip install"):run.index('"opendox[local]"') + len('"opendox[local]"')] + for part in ("--index-url https://test.pypi.org/simple/", "--extra-index-url https://pypi.org/simple/", + "--only-binary :all:", '--report "$RUNNER_TEMP/report.json"'): + assert part in install, part + assert '"opendox[local]"' in install and "opendox[local]==" not in run + # Nothing from the venv runs but pip, until the report has been read by + # this job's own Python; `opendox --help` comes after the verdict. + venv_calls = re.findall(r'"\$RUNNER_TEMP/fresh/bin/[^"]+"[^\n]*', run) + assert [call.split('"')[1] for call in venv_calls] == [ + "$RUNNER_TEMP/fresh/bin/python", "$RUNNER_TEMP/fresh/bin/opendox"], venv_calls + assert venv_calls[0].startswith('"$RUNNER_TEMP/fresh/bin/python" -m pip install') + assert run.index("<<'PY' || verdict=$?") < run.index('"$RUNNER_TEMP/fresh/bin/opendox" --help') + + +def _report(*items: tuple[str, str, str, str]) -> dict: + return {"version": "1", "install": [ + {"metadata": {"name": name, "version": version}, + "download_info": {"url": f"https://{host}/packages/aa/bb/{name}-{version}-py3-none-any.whl", + "archive_info": {"hash": f"sha256={digest}", "hashes": {"sha256": digest}}}} + for name, version, host, digest in items]} + + +TEST_FILES, PYPI_FILES = "test-files.pythonhosted.org", "files.pythonhosted.org" +VERDICT_CASES = { + "the verified TestPyPI wheel": ([("opendox", "0.1.0", TEST_FILES, "a" * 64), + ("pyyaml", "6.0.2", PYPI_FILES, "e" * 64)], 0, "is the verified wheel"), + "an opendox PyPI serves, at the version": ([("opendox", "0.1.0", PYPI_FILES, "f" * 64)], 2, + "which is not the verified wheel"), + "an opendox PyPI serves, above the version": ([("opendox", "9.9.9", PYPI_FILES, "f" * 64)], 2, + "which is not the verified wheel"), + "the verified digest, served from PyPI": ([("opendox", "0.1.0", PYPI_FILES, "a" * 64)], 2, + "which is not the verified wheel"), + "an older opendox from PyPI": ([("opendox", "0.0.9", PYPI_FILES, "f" * 64)], 1, "TestPyPI lags"), + "an older opendox from TestPyPI": ([("opendox", "0.0.9", TEST_FILES, "f" * 64)], 1, "so far"), + "TestPyPI's wheel at another digest": ([("opendox", "0.1.0", TEST_FILES, "c" * 64)], 2, + "which is not the verified wheel"), + "a newer opendox from TestPyPI": ([("opendox", "0.2.0", TEST_FILES, "c" * 64)], 2, + "which is not the verified wheel"), + "no opendox at all": ([("pyyaml", "6.0.2", PYPI_FILES, "e" * 64)], 2, "names 0 opendox"), +} + + +@pytest.mark.parametrize("case", sorted(VERDICT_CASES)) +def test_the_install_verdict(case: str, tmp_path: Path) -> None: + pytest.importorskip("pip._vendor.packaging.version") + items, code, said = VERDICT_CASES[case] + report = tmp_path / "report.json" + report.write_text(json.dumps(_report(*items)), encoding="utf-8") + result = subprocess.run((sys.executable, "-", str(report)), input=_verdict_script(), + capture_output=True, text=True, + env={"PATH": os.environ.get("PATH", "/usr/bin:/bin"), "LANG": "C.UTF-8", + "VERSION": "0.1.0", "WHEEL_SHA256": "a" * 64}) + assert result.returncode == code, result.stdout + result.stderr + assert said in result.stdout, result.stdout + result.stderr From f59e71052a09bc8b069885b02eb3b3ea2d02dd65 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 18:27:56 +0000 Subject: [PATCH 11/15] The dry run's credential note names the checkout's token (Copilot review) Copilot's review at b75205d4, an item it had missed before: the comment said the testpypi-install job holds no token, but its checkout uses the job's GITHUB_TOKEN (contents: read), without persisting it. The comment now says the install step receives no token, no OIDC grant and no secret, and that the token is used only by the checkout. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 06c5c533..4389bab4 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -793,10 +793,12 @@ jobs: # that lags) is tried again. Only then does `opendox --help` run. # # THE TRADE, stated: with two indexes, pip may take a dependency from - # either. The lock pins every dependency's version, and this job holds no - # token, no OIDC grant and no secret, and publishes nothing. What it can - # spoil is its own verdict, and the files PyPI receives are still the - # build job's, checked by digest in the `pypi` job. + # either. The lock pins every dependency's version. This step receives + # no token, no OIDC grant and no secret: the job's GITHUB_TOKEN + # (`contents: read`) is used only by the checkout above, which does not + # persist it, and the job publishes nothing. What it can spoil is its + # own verdict, and the files PyPI receives are still the build job's, + # checked by digest in the `pypi` job. - name: install opendox[local] from TestPyPI into a fresh venv, and run it timeout-minutes: 60 run: | From b390785891fe527df42524f03f6237f91c2cb21a Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sat, 3 Oct 2026 18:38:27 +0000 Subject: [PATCH 12/15] Fix round 9: each publish job re-checks its approval at use time (Copilot review) Copilot's review at f59e7105 (r4174341950): the build job's check that both environments ask a reviewer goes stale while the run waits; a rule removed meanwhile would let a publish job start with nobody approving it. - Each publish job, right before its preflight, pin check and upload, re-reads its environment: it must still name a required reviewer, and this run's review history (actions/runs//approvals) must hold an approved review of a deployment to it. Either missing refuses, and nothing is uploaded. The publish jobs gain actions: read for those two reads of this repository's API; the header says so. Budgets: testpypi 24 of 25 minutes, pypi 40 of 45. - tests/test_release_workflow.py (108 cases): the step's place, env and permissions in both jobs, one script; seven cases in each job (approved; the rule removed; the environment unreadable; no approval; only the other environment approved; rejected; the history unreadable). Mutants: any environment's approval counts, a rejected review counts, the rule not re-read, no check in pypi: each fails. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 95 +++++++++++++++++++++++++++++++--- tests/test_release_workflow.py | 74 ++++++++++++++++++++++++++ 2 files changed, 163 insertions(+), 6 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 4389bab4..c885c7d1 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -49,9 +49,10 @@ name: release # (owner `opensoft`, repository `openDox-code`, workflow `release.yml`, and the # environment the job runs in). `id-token: write` is granted to the two publish # jobs and to no other job. The one other credential is GitHub's own automatic -# `GITHUB_TOKEN`, which the build job passes to `gh` for two reads of THIS -# repository's API (the compare with `main`, and the environments), with -# `contents: read` and `actions: read` and nothing more. The openDox root's +# `GITHUB_TOKEN`, passed to `gh` for reads of THIS repository's API only: in +# the build job, the compare with `main` and the environments (`contents: +# read`, `actions: read`); in each publish job, its environment and this run's +# review history (`actions: read`). The openDox root's # pin, in another repository, is read over git with no credential at all, in # the build job and again right before each upload. # @@ -574,7 +575,7 @@ jobs: needs: build runs-on: ubuntu-latest # THE BUDGET: every step declares its own timeout, and the job's covers - # their sum (5 + 2 + 2 + 3 + 10 = 22 min); the upload's is 10 minutes. + # their sum (5 + 2 + 2 + 2 + 3 + 10 = 24 min); the upload's is 10 minutes. # tests/test_release_workflow.py reads each bound from this file. timeout-minutes: 25 environment: @@ -582,6 +583,7 @@ jobs: url: https://test.pypi.org/project/opendox/${{ inputs.version }}/ permissions: id-token: write # trusted publishing: the OIDC token TestPyPI accepts + actions: read # this environment's reviewer rule and this run's review history steps: - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 timeout-minutes: 5 @@ -608,6 +610,46 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - + # THE APPROVAL, AT USE TIME. The build job's check that both + # environments ask a reviewer goes stale while the run waits: a rule + # removed meanwhile would let this job start with nobody approving it. + # So, right before the preflight, the pin check and the upload, this + # step requires that this job's environment still names a required + # reviewer AND that this run's review history holds an approval of a + # deployment to it. Either missing refuses, and nothing is uploaded. + - name: this environment still asks a reviewer, and this run's deployment to it was approved + timeout-minutes: 2 + env: + GH_TOKEN: ${{ github.token }} + ENVIRONMENT: testpypi + run: | + if ! reviewers=$(gh api "repos/$GITHUB_REPOSITORY/environments/$ENVIRONMENT" \ + --jq '[.protection_rules[] | select(.type == "required_reviewers") | .reviewers[]] | length'); then + echo "::error::the $ENVIRONMENT environment cannot be read, so its reviewer rule is unknown" + exit 1 + fi + if [ "$reviewers" -lt 1 ]; then + echo "::error::the $ENVIRONMENT environment names no required reviewer now, so this deployment did not wait for an approval" + exit 1 + fi + if ! approvals=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/approvals"); then + echo "::error::this run's review history cannot be read" + exit 1 + fi + APPROVALS="$approvals" python3 - <<'PY' + import json, os, sys + + environment = os.environ["ENVIRONMENT"] + approved = [review["user"]["login"] for review in json.loads(os.environ["APPROVALS"]) + if review.get("state") == "approved" + and any(e.get("name") == environment for e in review.get("environments", []))] + if not approved: + sys.exit(f"::error::this run's review history holds no approval of a deployment " + f"to {environment}, so nothing is uploaded") + print(f"{environment}: a required reviewer is named, and this run's deployment " + f"was approved by {', '.join(approved)}") + PY + # BEFORE THE UPLOAD, NOTHING ELSE HELD. skip-existing skips a file name # the index already holds, so the index must hold no file for this # version but the verified ones, at the verified digests: otherwise the @@ -875,17 +917,18 @@ jobs: needs: [build, testpypi-install] runs-on: ubuntu-latest # THE BUDGET: every step declares its own timeout, and the job's covers - # their sum (5 + 2 + 2 + 3 + 10 + 16 = 38 min). The upload is bounded at + # their sum (5 + 2 + 2 + 2 + 3 + 10 + 16 = 40 min). The upload is bounded at # 10 minutes, so the JSON step after it always has its 20 reads, each up # to 30 s, 15 s apart (15 min, bounded at 16). A cut-off upload stays # recoverable: a re-run's preflight and skip-existing upload the rest. # tests/test_release_workflow.py reads each bound from this file. - timeout-minutes: 40 + timeout-minutes: 45 environment: name: pypi url: https://pypi.org/project/opendox/${{ inputs.version }}/ permissions: id-token: write # trusted publishing: the OIDC token PyPI accepts + actions: read # this environment's reviewer rule and this run's review history steps: - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 timeout-minutes: 5 @@ -910,6 +953,46 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - + # THE APPROVAL, AT USE TIME. The build job's check that both + # environments ask a reviewer goes stale while the run waits: a rule + # removed meanwhile would let this job start with nobody approving it. + # So, right before the preflight, the pin check and the upload, this + # step requires that this job's environment still names a required + # reviewer AND that this run's review history holds an approval of a + # deployment to it. Either missing refuses, and nothing is uploaded. + - name: this environment still asks a reviewer, and this run's deployment to it was approved + timeout-minutes: 2 + env: + GH_TOKEN: ${{ github.token }} + ENVIRONMENT: pypi + run: | + if ! reviewers=$(gh api "repos/$GITHUB_REPOSITORY/environments/$ENVIRONMENT" \ + --jq '[.protection_rules[] | select(.type == "required_reviewers") | .reviewers[]] | length'); then + echo "::error::the $ENVIRONMENT environment cannot be read, so its reviewer rule is unknown" + exit 1 + fi + if [ "$reviewers" -lt 1 ]; then + echo "::error::the $ENVIRONMENT environment names no required reviewer now, so this deployment did not wait for an approval" + exit 1 + fi + if ! approvals=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/approvals"); then + echo "::error::this run's review history cannot be read" + exit 1 + fi + APPROVALS="$approvals" python3 - <<'PY' + import json, os, sys + + environment = os.environ["ENVIRONMENT"] + approved = [review["user"]["login"] for review in json.loads(os.environ["APPROVALS"]) + if review.get("state") == "approved" + and any(e.get("name") == environment for e in review.get("environments", []))] + if not approved: + sys.exit(f"::error::this run's review history holds no approval of a deployment " + f"to {environment}, so nothing is uploaded") + print(f"{environment}: a required reviewer is named, and this run's deployment " + f"was approved by {', '.join(approved)}") + PY + # BEFORE THE UPLOAD, NOTHING ELSE HELD. skip-existing skips a file name # the index already holds, so the index must hold no file for this # version but the verified ones, at the verified digests: otherwise the diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index b2c2d767..941ee4ee 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -170,6 +170,9 @@ def test_the_four_jobs_are_chained() -> None: # A stand-in `gh`: it answers from the case's fixture and records nothing. case "$*" in *compare/*) printf '%s\\n' "$FAKE_STATUS" ;; + *actions/runs/*/approvals*) + [ -n "${FAKE_APPROVALS_FAILS:-}" ] && { echo "HTTP 404" >&2; exit 1; } + printf '%s\\n' "$FAKE_APPROVALS" ;; *environments/*) args="$*"; environment="${args##*environments/}"; environment="${environment%% *}" variable="FAKE_REVIEWERS_${environment}" @@ -845,3 +848,74 @@ def test_the_install_verdict(case: str, tmp_path: Path) -> None: "VERSION": "0.1.0", "WHEEL_SHA256": "a" * 64}) assert result.returncode == code, result.stdout + result.stderr assert said in result.stdout, result.stdout + result.stderr + + +# --------------------------------------------------------------------------- +# The approval, at use time (Copilot's review of openDox-code#78): the build +# job's reviewer check goes stale while the run waits, so each publish job +# re-reads its environment's rule and this run's review history. + +APPROVAL_STEP = "this environment still asks a reviewer, and this run's deployment to it was approved" + + +def test_each_publish_job_rechecks_its_approval_before_the_preflight() -> None: + jobs = _workflow()["jobs"] + scripts = set() + for job in ("testpypi", "pypi"): + assert jobs[job]["permissions"] == {"id-token": "write", "actions": "read"}, job + steps = jobs[job]["steps"] + upload = next(i for i, step in enumerate(steps) + if step.get("uses", "").startswith("pypa/gh-action-pypi-publish@")) + step = steps[upload - 3] + assert step["name"] == APPROVAL_STEP, job + assert step["env"] == {"GH_TOKEN": "${{ github.token }}", + "ENVIRONMENT": jobs[job]["environment"]["name"]}, job + scripts.add(step["run"]) + assert len(scripts) == 1, "the two approval checks are one script" + + +def _approval(state: str, *environments: str) -> dict: + return {"state": state, "comment": "", "user": {"login": "brettheap"}, + "environments": [{"id": 1, "name": name} for name in environments]} + + +APPROVAL_CASES = { + "a reviewer named, and this deployment approved": ( + dict(reviewers="1", approvals=[_approval("approved", "{env}")]), None), + "the reviewer rule removed meanwhile": ( + dict(reviewers="0", approvals=[_approval("approved", "{env}")]), "names no required reviewer now"), + "the environment unreadable": ( + dict(reviewers=None, approvals=[_approval("approved", "{env}")]), "cannot be read"), + "no approval at all": (dict(reviewers="1", approvals=[]), "holds no approval"), + "only the other environment approved": ( + dict(reviewers="1", approvals=[_approval("approved", "{other}")]), "holds no approval"), + "this deployment rejected": ( + dict(reviewers="1", approvals=[_approval("rejected", "{env}")]), "holds no approval"), + "the review history unreadable": ( + dict(reviewers="1", approvals=None), "review history cannot be read"), +} + + +@needs_a_shell +@pytest.mark.parametrize("job", ["testpypi", "pypi"]) +@pytest.mark.parametrize("case", sorted(APPROVAL_CASES)) +def test_the_approval_check(job: str, case: str, tmp_path: Path) -> None: + given, refusal = APPROVAL_CASES[case] + env_name, other = job, ("pypi" if job == "testpypi" else "testpypi") + step = _step(job, APPROVAL_STEP) + approvals = given["approvals"] + env = {"GITHUB_REPOSITORY": REPOSITORY, "GITHUB_RUN_ID": "1", "GH_TOKEN": "unused", + "ENVIRONMENT": step["env"]["ENVIRONMENT"]} + if given["reviewers"] is not None: + env[f"FAKE_REVIEWERS_{env_name}"] = given["reviewers"] + if approvals is None: + env["FAKE_APPROVALS_FAILS"] = "1" + else: + env["FAKE_APPROVALS"] = json.dumps(approvals).replace("{env}", env_name).replace("{other}", other) + result = _run(step["run"], tmp_path, env, (_fake_gh(tmp_path / "bin"),)) + if refusal is None: + assert result.returncode == 0, result.stdout + result.stderr + assert f"{env_name}: a required reviewer is named" in result.stdout + else: + assert result.returncode != 0, result.stdout + assert refusal in result.stdout + result.stderr, result.stdout + result.stderr From 7fc0c46e570b388b56e75fa3b1799b7043c5f6a2 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 5 Oct 2026 10:07:38 +0000 Subject: [PATCH 13/15] Round 10: record the digests before the smoke test, and refuse a yanked file Copilot's review of openDox-code#78 at e5f7f554, as the holder ruled: - r4182777667: the build job recorded the files' digests after the smoke test. That test installs and runs third-party wheels, which the lock pins by version but not by hash, so a dependency's code could rewrite dist/ between the artifact checks and the recording, and the digests would bless the new bytes. The digests are now recorded right after "verify the artifacts", before any third-party wheel is installed or run. A new step after the smoke test requires dist/ to still hold exactly those bytes, and names the cause before the artifact is uploaded. The publish jobs' digest checks are unchanged. - r4182777770: a yanked file. The preflight refuses a held file that is yanked, naming the remedy (un-yank it on the index, or release a new version). The check after each upload refuses a yanked file at once. Each script stays one script for both indexes. The release tests go from 108 to 119: the build job's step order, the re-check over four cases, and three yanked cases on both indexes. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 110 ++++++++++++++++++++++++++------- tests/test_release_workflow.py | 86 ++++++++++++++++++++++++-- 2 files changed, 168 insertions(+), 28 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c885c7d1..4b5f82e0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -31,9 +31,10 @@ name: release # # FOUR JOBS, EACH GATED ON THE ONE BEFORE IT: # build builds the sdist and the wheel in a clean job, runs -# `twine check`, VERIFIES both before any upload, and -# installs the wheel into a fresh venv. It mints no -# identity token, and its GITHUB_TOKEN reads only. +# `twine check`, VERIFIES both before any upload, records +# their digests, and installs the wheel into a fresh venv, +# after which `dist/` must still match those digests. It +# mints no identity token, and its GITHUB_TOKEN reads only. # testpypi the dry run: re-checks the root's pin, then publishes # those exact bytes to TestPyPI, in the `testpypi` # environment. @@ -41,7 +42,7 @@ name: release # venv, as plan 034's T099 falsifier does, and runs it. # pypi re-checks the root's pin, publishes the same bytes to # PyPI, in the `pypi` environment, then requires PyPI to -# serve exactly those two files, by digest. +# serve exactly those two files, by digest, neither yanked. # # NO PyPI TOKEN AND NO STORED SECRET, ANYWHERE. Both uploads use PyPI's # trusted publishing (OIDC): the job asks GitHub for a short-lived identity @@ -478,6 +479,27 @@ jobs: print("the artifacts carry what an install runs from") PY + # THE DIGESTS ARE RECORDED HERE, BEFORE ANY THIRD-PARTY CODE RUNS. Up to + # this step only the hash-locked release tools and this package's own + # build have run. The smoke test after it installs and runs third-party + # wheels, which the lock pins by version but not by hash, and any of + # them could rewrite `dist/` (a `.pth` file runs at every interpreter + # start). So the files are recorded as the artifact checks read them, + # the step after the smoke test requires `dist/` to still hold exactly + # these bytes, and each publish job checks the artifact against these + # digests again before it uploads (Copilot's review of openDox-code#78). + - name: record the files and their digests + id: digests + run: | + wheel="opendox-$VERSION-py3-none-any.whl" + sdist="opendox-$VERSION.tar.gz" + { + echo "wheel=$wheel" + echo "wheel-sha256=$(sha256sum "dist/$wheel" | cut -d' ' -f1)" + echo "sdist=$sdist" + echo "sdist-sha256=$(sha256sum "dist/$sdist" | cut -d' ' -f1)" + } | tee -a "$GITHUB_OUTPUT" + # THE WHEEL INSTALLS, AND RUNS, OUTSIDE THE CHECKOUT. A fresh venv, # `opendox[local]` from the built file (10.3's line, from a file), the # dependencies through this package's own lock, and every command run @@ -552,17 +574,29 @@ jobs: fi echo "opendox generate-and-open --help names --local" - - name: record the files and their digests - id: digests + # THE SMOKE TEST LEFT `dist/` AS THE ARTIFACT CHECKS READ IT: the same + # two files, at the digests recorded before it ran. A change refuses + # here, before the artifact is uploaded. The publish jobs would refuse + # it too, at their own digest check, so nothing changed here can be + # published either way; this step names the cause, and names it early. + - name: the smoke test left dist/ as the digests recorded it + env: + WHEEL: ${{ steps.digests.outputs.wheel }} + WHEEL_SHA256: ${{ steps.digests.outputs.wheel-sha256 }} + SDIST: ${{ steps.digests.outputs.sdist }} + SDIST_SHA256: ${{ steps.digests.outputs.sdist-sha256 }} run: | - wheel="opendox-$VERSION-py3-none-any.whl" - sdist="opendox-$VERSION.tar.gz" - { - echo "wheel=$wheel" - echo "wheel-sha256=$(sha256sum "dist/$wheel" | cut -d' ' -f1)" - echo "sdist=$sdist" - echo "sdist-sha256=$(sha256sum "dist/$sdist" | cut -d' ' -f1)" - } | tee -a "$GITHUB_OUTPUT" + found="$(find dist -mindepth 1 -maxdepth 1 -printf '%f\n' | sort | tr '\n' ' ')" + wanted="$(printf '%s\n%s\n' "$SDIST" "$WHEEL" | sort | tr '\n' ' ')" + if [ "$found" != "$wanted" ]; then + echo "::error::after the smoke test dist/ holds $found, and the digests were recorded for $wanted, so the artifact is not uploaded" + exit 1 + fi + if ! printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ + | sha256sum --strict -c -; then + echo "::error::the smoke test changed dist/ after its digests were recorded, so the artifact is not uploaded" + exit 1 + fi - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: @@ -687,6 +721,15 @@ jobs: sys.exit(f"::error::{index} already holds {other} for opendox " f"{os.environ['VERSION']}, which the build job did not verify; " "an upload now would make a mixed release, so nothing is uploaded") + # A YANKED FILE STAYS YANKED. skip-existing leaves it in place, and an + # unpinned install (10.3's line) skips a yanked file, so a re-run would + # go green on a release that line cannot install. + yanked = sorted(f["filename"] for f in release["urls"] if f.get("yanked")) + if yanked: + sys.exit(f"::error::{index} holds {yanked} for opendox {os.environ['VERSION']} " + "at the verified digest, but yanked, and an unpinned install skips a " + "yanked file; un-yank it on the index, or release a new version. " + "Nothing is uploaded") print(f"{index} holds {sorted(held)} for opendox {os.environ['VERSION']}, each at " "the verified digest; skip-existing uploads only what is missing") PY @@ -735,7 +778,7 @@ jobs: # already holds, instead of stopping on it, and uploads the rest. The # step before the pin's re-check refused any file but a verified one at # its verified digest, and the JSON step after the upload requires the - # index to serve exactly the build job's two digests. + # index to serve exactly the build job's two digests, neither yanked. - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 timeout-minutes: 10 with: @@ -772,11 +815,13 @@ jobs: with: python-version: '3.12' - # TestPyPI serves exactly the two files the build job verified. Its JSON - # API can lag the upload, answering 404 or a partial release for a - # while, so it is read up to READS times, PAUSE seconds apart, until it - # serves exactly those two files, before this step refuses. The `pypi` - # job runs the same script against PyPI after its upload. + # TestPyPI serves exactly the two files the build job verified, neither + # yanked. Its JSON API can lag the upload, answering 404 or a partial + # release for a while, so it is read up to READS times, PAUSE seconds + # apart, until it serves exactly those two files, before this step + # refuses. A yanked file refuses at once: waiting does not un-yank it, + # and an unpinned install skips it. The `pypi` job runs the same script + # against PyPI after its upload. - name: TestPyPI serves the files the build job verified timeout-minutes: 16 env: @@ -799,6 +844,11 @@ jobs: with urllib.request.urlopen(url, timeout=30) as response: release = json.load(response) served = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} + yanked = sorted(f["filename"] for f in release["urls"] if f.get("yanked")) + if served == wanted and yanked: + sys.exit(f"::error::{index} serves the verified files, but {yanked} " + "yanked, and an unpinned install skips a yanked file; un-yank " + "it on the index, or release a new version") if served == wanted: break print(f"attempt {attempt}: {index} serves {served} so far") @@ -811,7 +861,7 @@ jobs: else: sys.exit(f"::error::after {reads} reads, {index} serves {served}, " f"and the build job verified {wanted}") - print(f"{index} serves exactly the verified files: {served}") + print(f"{index} serves exactly the verified files, neither yanked: {served}") PY # T099's DRY-RUN FALSIFIER, as plan 034 writes it, in a fresh venv: @@ -1030,6 +1080,15 @@ jobs: sys.exit(f"::error::{index} already holds {other} for opendox " f"{os.environ['VERSION']}, which the build job did not verify; " "an upload now would make a mixed release, so nothing is uploaded") + # A YANKED FILE STAYS YANKED. skip-existing leaves it in place, and an + # unpinned install (10.3's line) skips a yanked file, so a re-run would + # go green on a release that line cannot install. + yanked = sorted(f["filename"] for f in release["urls"] if f.get("yanked")) + if yanked: + sys.exit(f"::error::{index} holds {yanked} for opendox {os.environ['VERSION']} " + "at the verified digest, but yanked, and an unpinned install skips a " + "yanked file; un-yank it on the index, or release a new version. " + "Nothing is uploaded") print(f"{index} holds {sorted(held)} for opendox {os.environ['VERSION']}, each at " "the verified digest; skip-existing uploads only what is missing") PY @@ -1078,7 +1137,7 @@ jobs: # already holds, instead of stopping on it, and uploads the rest. The # step before the pin's re-check refused any file but a verified one at # its verified digest, and the JSON step after the upload requires the - # index to serve exactly the build job's two digests. + # index to serve exactly the build job's two digests, neither yanked. - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 timeout-minutes: 10 with: @@ -1112,6 +1171,11 @@ jobs: with urllib.request.urlopen(url, timeout=30) as response: release = json.load(response) served = {f["filename"]: f["digests"]["sha256"] for f in release["urls"]} + yanked = sorted(f["filename"] for f in release["urls"] if f.get("yanked")) + if served == wanted and yanked: + sys.exit(f"::error::{index} serves the verified files, but {yanked} " + "yanked, and an unpinned install skips a yanked file; un-yank " + "it on the index, or release a new version") if served == wanted: break print(f"attempt {attempt}: {index} serves {served} so far") @@ -1124,5 +1188,5 @@ jobs: else: sys.exit(f"::error::after {reads} reads, {index} serves {served}, " f"and the build job verified {wanted}") - print(f"{index} serves exactly the verified files: {served}") + print(f"{index} serves exactly the verified files, neither yanked: {served}") PY diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index 941ee4ee..f3d8f900 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -45,7 +45,12 @@ tree of their own: the whole set passes, and each kind of gap refuses, a changed or dropped marker and a direct URL among them; * the publish jobs' digest check: the verified files pass, and a changed - byte or a third file refuses. + byte or a third file refuses; + * the build job records the digests right after the artifact checks, before + the smoke test installs and runs any third-party wheel, and re-checks + `dist/` after it: a changed byte, a third file or a missing file refuses; + * a yanked file: the preflight refuses one the index already holds, and the + check after each upload refuses one the index serves. """ from __future__ import annotations @@ -564,6 +569,66 @@ def test_the_publish_jobs_check_the_digests(job: str, change: str, tmp_path: Pat assert (result.returncode == 0) == (change == "none"), result.stdout + result.stderr +# --------------------------------------------------------------------------- +# The build job records the digests before any third-party code runs +# (Copilot's review of openDox-code#78: the smoke test installs and runs +# wheels the lock pins by version but not by hash, and any of them could +# rewrite `dist/` before digests recorded after it blessed the new bytes). + +DIGESTS_STEP = "record the files and their digests" +SMOKE_STEP = "the built wheel installs into a fresh venv, and opendox --help works" +RECHECK_STEP = "the smoke test left dist/ as the digests recorded it" + + +def test_the_build_records_the_digests_before_the_smoke_test_and_rechecks_after() -> None: + build = _workflow()["jobs"]["build"] + steps = build["steps"] + names = [step.get("name") or step["uses"].split("@")[0] for step in steps] + verify = names.index("verify the artifacts") + assert names[verify:] == ["verify the artifacts", DIGESTS_STEP, SMOKE_STEP, RECHECK_STEP, + "actions/upload-artifact"], names + assert steps[verify + 1]["id"] == "digests" + assert steps[verify + 3]["env"] == { + "WHEEL": "${{ steps.digests.outputs.wheel }}", + "WHEEL_SHA256": "${{ steps.digests.outputs.wheel-sha256 }}", + "SDIST": "${{ steps.digests.outputs.sdist }}", + "SDIST_SHA256": "${{ steps.digests.outputs.sdist-sha256 }}", + } + assert build["outputs"] == { + name: f"${{{{ steps.digests.outputs.{name} }}}}" + for name in ("wheel", "wheel-sha256", "sdist", "sdist-sha256")} + + +@needs_a_shell +@pytest.mark.parametrize("change", ["none", "a changed byte", "a third file", "a missing file"]) +def test_the_recheck_refuses_a_dist_the_smoke_test_changed(change: str, tmp_path: Path) -> None: + dist = tmp_path / "dist" + dist.mkdir() + (dist / WHEEL_FILE).write_bytes(b"the wheel") + (dist / SDIST_FILE).write_bytes(b"the sdist") + output = tmp_path / "github-output" + output.write_text("") + record = _run(_step("build", DIGESTS_STEP)["run"], tmp_path, + {"VERSION": "0.1.0", "GITHUB_OUTPUT": str(output)}) + assert record.returncode == 0, record.stdout + record.stderr + outputs = dict(line.split("=", 1) for line in output.read_text().splitlines()) + recorded = {"WHEEL": outputs["wheel"], "WHEEL_SHA256": outputs["wheel-sha256"], + "SDIST": outputs["sdist"], "SDIST_SHA256": outputs["sdist-sha256"]} + assert recorded == {"WHEEL": WHEEL_FILE, "SDIST": SDIST_FILE, + "WHEEL_SHA256": hashlib.sha256(b"the wheel").hexdigest(), + "SDIST_SHA256": hashlib.sha256(b"the sdist").hexdigest()} + if change == "a changed byte": + (dist / WHEEL_FILE).write_bytes(b"the wheeL") + elif change == "a third file": + (dist / "evil-0.1.0-py3-none-any.whl").write_bytes(b"") + elif change == "a missing file": + (dist / SDIST_FILE).unlink() + result = _run(_step("build", RECHECK_STEP)["run"], tmp_path, recorded) + assert (result.returncode == 0) == (change == "none"), result.stdout + result.stderr + if change != "none": + assert "so the artifact is not uploaded" in result.stdout, result.stdout + result.stderr + + # --------------------------------------------------------------------------- # Re-running an upload, and what each index serves after it. @@ -602,8 +667,10 @@ def test_each_upload_is_followed_by_one_index_check_script() -> None: class _Index: - """A local JSON API: `answers` is the list of (status, files) it gives, - one per request, the last one repeated.""" + """A local JSON API: `answers` is the list of (status, files) or (status, + files, yanked) it gives, one per request, the last one repeated. Each file + carries `yanked`, as Warehouse's JSON API does: true for a name in + `yanked`, false otherwise.""" def __init__(self, answers: list[tuple[int, dict[str, str]]]) -> None: import http.server @@ -615,8 +682,11 @@ def __init__(self, answers: list[tuple[int, dict[str, str]]]) -> None: class Handler(http.server.BaseHTTPRequestHandler): def do_GET(self) -> None: # noqa: N802 (the stdlib's name) index.requests.append(self.path) - status, files = index.answers[min(len(index.requests), len(index.answers)) - 1] - body = json.dumps({"urls": [{"filename": name, "digests": {"sha256": digest}} + answer = index.answers[min(len(index.requests), len(index.answers)) - 1] + status, files = answer[0], answer[1] + yanked = answer[2] if len(answer) > 2 else set() + body = json.dumps({"urls": [{"filename": name, "digests": {"sha256": digest}, + "yanked": name in yanked} for name, digest in files.items()]}).encode() self.send_response(status) self.send_header("Content-Type", "application/json") @@ -646,6 +716,8 @@ def close(self) -> None: "it serves a third file": ([(200, {**VERIFIED, "opendox-0.1.0-py2-none-any.whl": "d" * 64})], "after 3 reads"), "it refuses the read": ([(403, {})], "HTTP Error 403"), + "it serves the verified files, the wheel yanked": ([(200, VERIFIED, {WHEEL_FILE})], + "un-yank it on the index, or release a new version"), } @@ -752,6 +824,10 @@ def test_each_upload_is_preceded_by_the_preflight_then_the_pin() -> None: "a file nobody verified": ([(200, {"opendox-0.1.0-py2-none-any.whl": "d" * 64})], "would make a mixed release"), "the index refuses the read": ([(503, {})], "HTTP Error 503"), + "the verified wheel, from an earlier try, yanked": ([(200, {WHEEL_FILE: "a" * 64}, {WHEEL_FILE})], + "un-yank it on the index, or release a new version"), + "both verified files, the release yanked": ([(200, VERIFIED, {WHEEL_FILE, SDIST_FILE})], + "un-yank it on the index, or release a new version"), } From 93d13217c76e29f053cbe634c69a7dc1992e94d4 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 5 Oct 2026 10:52:55 +0000 Subject: [PATCH 14/15] Round 11: refs/heads/main, the environments' deployment limit, and the stated limits Lane 3's review D8 of openDox-code#78 at 7fc0c46e, as the holder ruled it in openxFactory#656 comment 5992918154: - F1, fixed: the openDox root's pin is fetched as refs/heads/main in all three jobs, and this repository's compare reads refs/heads/main. git resolves a bare `main` on the remote as a tag before the branch, so a tag named `main` at an unreviewed commit could have named the release commit. A new case builds a stand-in root whose tag `main`, at a commit off the branch, pins another commit: in each job the branch's pin passes and the tag's pin refuses. - F2, fixed: a dispatch runs the dispatched ref's own release.yml, and the trusted publisher never matches the ref, so each environment's deployment limit is the one server-side control over the ref. The build job's environment step, and each publish job's use-time step, now refuse unless the environment's deployment_branch_policy is set with custom_branch_policies. Brett's environments already carry it: both name the tag pattern v* alone. The header states that a dispatch on main cannot deploy, and the remaining limit: while sessions act as brettheap, the approval click is the last line of defense. - F3, an accepted limit, stated in the header: the publish action is pinned by SHA, but it runs its image from ghcr.io by registry tag. The follow-on is a hash-locked twine upload with the manual OIDC exchange. - F4, an accepted limit, stated in the header as a runbook line: after the TestPyPI upload, only "Re-run failed jobs", never "Re-run all jobs", because the sdist is not byte-reproducible. The follow-on is a reproducible sdist. The release tests go from 119 to 129. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 199 +++++++++++++++++++++++---------- tests/test_release_workflow.py | 119 ++++++++++++++++---- 2 files changed, 236 insertions(+), 82 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 4b5f82e0..5af9de0f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -23,11 +23,26 @@ name: release # dispatched at the cut. `main`'s head may have moved past the pinned commit by # then (T095 lands later), so the head is never required: the run's commit # must equal the root's pin and be on `main`. A dispatch runs at the head of -# the ref it names, so it names the tag `v`, created at the pinned -# commit, or `main` only while `main`'s head is still that commit. No other -# tag is accepted. The pin is checked in the build job and AGAIN right before -# each upload, so a pin that moves while the run waits (on an approval, or on -# TestPyPI) stops the upload. +# the ref it names, so the release is dispatched on the tag `v`, +# created at the commit T087 pinned. No other tag is accepted. The pin is +# checked in the build job and AGAIN right before each upload, so a pin that +# moves while the run waits (on an approval, or on TestPyPI) stops the +# upload. `main` here is always `refs/heads/main`, in the root's fetch and in +# this repository's compare: a bare `main` would resolve to a TAG named +# `main` before the branch, and a tag needs no review. +# +# ONLY A `v*` TAG CAN DEPLOY. A dispatch runs the dispatched ref's OWN copy +# of this file, and PyPI's trusted publisher matches the repository, the +# workflow's file name and the environment, never the ref. So every gate in +# this file binds only where the dispatched ref carries the reviewed file. +# The one server-side control over the ref is each environment's deployment +# limit, and both environments MUST carry one: Brett set each to the tag +# pattern `v*` alone, so a dispatch on `main` or any branch builds and +# verifies but cannot deploy. The build job refuses when either environment +# has no such limit, and each publish job checks it again at use time. The +# limit that remains, accepted: while sessions act as brettheap, the approval +# click is the last line of defense, and a ruleset over `v*` tags is +# recommended, not required. # # FOUR JOBS, EACH GATED ON THE ONE BEFORE IT: # build builds the sdist and the wheel in a clean job, runs @@ -44,6 +59,18 @@ name: release # PyPI, in the `pypi` environment, then requires PyPI to # serve exactly those two files, by digest, neither yanked. # +# AFTER THE TestPyPI UPLOAD, RE-RUN ONLY THE FAILED JOBS. Use "Re-run failed +# jobs", never "Re-run all jobs", and never a second dispatch of the same +# version. The wheel is byte-reproducible, but the sdist is not (its archive +# carries file times and a gzip header the build does not normalize), so a +# rebuild records new digests, the index keeps the first files under the same +# names for good, and every later preflight refuses, correctly: the version +# could then never complete through this workflow without a version bump and a +# new T087 pin. A failed job re-run from the same run reuses the build's +# artifact and its recorded digests. An accepted limit (RULED openxFactory#656 +# comment 5992918154, on lane 3's review D8, F4); the follow-on is a +# reproducible sdist, with normalized member times and gzip header. +# # NO PyPI TOKEN AND NO STORED SECRET, ANYWHERE. Both uploads use PyPI's # trusted publishing (OIDC): the job asks GitHub for a short-lived identity # token, and the index accepts it only for the publisher configured on its side @@ -60,6 +87,16 @@ name: release # EVERY ACTION IS PINNED BY FULL COMMIT SHA, with its release tag beside it. # (`validate.yml` pins by tag. A tag can be moved after review, and a commit # cannot, which matters most in the one workflow whose output is published.) +# ONE LIMIT, ACCEPTED (RULED openxFactory#656 comment 5992918154, on lane 3's +# review D8, F3): the SHA pins the publish action's own files, not all the +# code it runs. pypa/gh-action-pypi-publish is a container action, and called +# from any repository but its own it runs the image +# `ghcr.io/pypa/gh-action-pypi-publish:`, pulled by registry TAG +# (its `create-docker-action.py`), and a registry tag can be pushed again. So +# the step that holds the OIDC token runs an image that a tag names, and the +# JSON check after an upload could catch other bytes only once they are +# published. The follow-on: a hash-locked `twine upload` (twine is already in +# the release tools' lock) with the documented manual OIDC token exchange. # --------------------------------------------------------------------------- on: @@ -119,7 +156,9 @@ jobs: # steps come first. This one: a dispatch on a tag must name # `v`, and the run's commit must be on this repository's # `main`, as `main` or an ancestor of it (`main`'s head need not be that - # commit). The compare reads this repository, through the API. + # commit). The compare reads this repository, through the API, against + # `refs/heads/main`: the bare name would resolve to a tag named `main` + # first. - name: the dispatched commit is on main, dispatched on v or a branch env: GH_TOKEN: ${{ github.token }} @@ -132,7 +171,7 @@ jobs: fi ;; esac - status="$(gh api "repos/$GITHUB_REPOSITORY/compare/$GITHUB_SHA...main" --jq .status)" + status="$(gh api "repos/$GITHUB_REPOSITORY/compare/$GITHUB_SHA...refs/heads/main" --jq .status)" case "$status" in identical|ahead) echo "$GITHUB_SHA is on main ($status), dispatched on $GITHUB_REF" ;; *) @@ -150,7 +189,10 @@ jobs: # public, and a job's GITHUB_TOKEN is an installation token scoped to # this repository, so the read of another repository uses no token: a # fetch of `main`, depth 1, with every credential helper cleared and no - # prompt, in a repository of its own under `$RUNNER_TEMP`. + # prompt, in a repository of its own under `$RUNNER_TEMP`. The fetch + # names `refs/heads/main`: git resolves a bare `main` on the remote as a + # tag before a branch, and a tag `main` at an unreviewed commit would + # then name the release commit. - name: the dispatched commit is the one the openDox root pins timeout-minutes: 3 run: | @@ -158,7 +200,7 @@ jobs: rm -rf "$root" git init -q -b main "$root" if ! GIT_TERMINAL_PROMPT=0 git -C "$root" -c credential.helper= \ - fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git main \ + fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git refs/heads/main \ || ! pin="$(git -C "$root" show FETCH_HEAD:contracts/code-pin.yaml)"; then echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" exit 1 @@ -186,29 +228,50 @@ jobs: print(f"{this} is the commit opensoft/openDox main pins for {here}") PY - # BOTH ENVIRONMENTS MUST ASK A REVIEWER. GitHub creates an environment a - # job names if it does not exist yet, and creates it with no protection - # rule, so a dispatch made before Brett configured `pypi` would publish - # with nobody approving it. This step refuses that case, before anything - # is built. The environments are public metadata of a public repository. - - name: both environments ask a reviewer before they deploy + # BOTH ENVIRONMENTS MUST ASK A REVIEWER, AND LIMIT THE REFS THAT DEPLOY. + # GitHub creates an environment a job names if it does not exist yet, + # and creates it with no protection rule, so a dispatch made before + # Brett configured `pypi` would publish with nobody approving it. And a + # dispatch runs the dispatched ref's own copy of this file, so an + # environment that any ref may deploy to would publish whatever a branch + # carries. Each environment must name a required reviewer, and its + # `deployment_branch_policy` must be set, with `custom_branch_policies` + # (the tag pattern `v*`, as Brett configured them). This step refuses + # either gap, before anything is built. The environments are public + # metadata of a public repository. + - name: both environments ask a reviewer and limit the refs that deploy env: GH_TOKEN: ${{ github.token }} run: | rc=0 for environment in testpypi pypi; do - if ! reviewers=$(gh api "repos/$GITHUB_REPOSITORY/environments/$environment" \ - --jq '[.protection_rules[] | select(.type == "required_reviewers") | .reviewers[]] | length'); then - echo "::error::the $environment environment cannot be read (it does not exist yet, or this job cannot see it); create it with a required reviewer first" + if ! settings=$(gh api "repos/$GITHUB_REPOSITORY/environments/$environment"); then + echo "::error::the $environment environment cannot be read (it does not exist yet, or this job cannot see it); create it with a required reviewer and a deployment limit to the tags v* first" rc=1 continue fi - if [ "$reviewers" -lt 1 ]; then - echo "::error::the $environment environment names no required reviewer, so a deployment to it would not wait for an approval" - rc=1 - else - echo "$environment: $reviewers required reviewer(s)" - fi + ENVIRONMENT="$environment" SETTINGS="$settings" python3 - <<'PY' || rc=1 + import json, os, sys + + environment, settings = os.environ["ENVIRONMENT"], json.loads(os.environ["SETTINGS"]) + reviewers = sum(len(rule.get("reviewers") or []) for rule in settings.get("protection_rules") or [] + if rule.get("type") == "required_reviewers") + policy = settings.get("deployment_branch_policy") + problems = [] + if reviewers < 1: + problems.append(f"the {environment} environment names no required reviewer, " + "so a deployment to it would not wait for an approval") + if not policy or policy.get("custom_branch_policies") is not True: + problems.append(f"the {environment} environment does not limit the refs that may " + f"deploy to it (deployment_branch_policy {policy!r}): a dispatch on any " + "branch runs that branch's own release.yml, so limit it to the tags v*") + for problem in problems: + print(f"::error::{problem}") + if problems: + sys.exit(1) + print(f"{environment}: {reviewers} required reviewer(s), and only the refs its " + "deployment limit names can deploy") + PY done exit $rc @@ -644,44 +707,50 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - - # THE APPROVAL, AT USE TIME. The build job's check that both - # environments ask a reviewer goes stale while the run waits: a rule - # removed meanwhile would let this job start with nobody approving it. - # So, right before the preflight, the pin check and the upload, this - # step requires that this job's environment still names a required - # reviewer AND that this run's review history holds an approval of a - # deployment to it. Either missing refuses, and nothing is uploaded. - - name: this environment still asks a reviewer, and this run's deployment to it was approved + # THE APPROVAL, AT USE TIME. The build job's check of both environments + # goes stale while the run waits: a rule removed meanwhile would let + # this job start with nobody approving it, and a deployment limit + # removed meanwhile would let any ref deploy. So, right before the + # preflight, the pin check and the upload, this step requires that this + # job's environment still names a required reviewer AND still limits + # the refs that deploy to it (`custom_branch_policies`), AND that this + # run's review history holds an approval of a deployment to it. Any one + # missing refuses, and nothing is uploaded. + - name: this environment still asks a reviewer and limits its refs, and this run's deployment to it was approved timeout-minutes: 2 env: GH_TOKEN: ${{ github.token }} ENVIRONMENT: testpypi run: | - if ! reviewers=$(gh api "repos/$GITHUB_REPOSITORY/environments/$ENVIRONMENT" \ - --jq '[.protection_rules[] | select(.type == "required_reviewers") | .reviewers[]] | length'); then + if ! settings=$(gh api "repos/$GITHUB_REPOSITORY/environments/$ENVIRONMENT"); then echo "::error::the $ENVIRONMENT environment cannot be read, so its reviewer rule is unknown" exit 1 fi - if [ "$reviewers" -lt 1 ]; then - echo "::error::the $ENVIRONMENT environment names no required reviewer now, so this deployment did not wait for an approval" - exit 1 - fi if ! approvals=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/approvals"); then echo "::error::this run's review history cannot be read" exit 1 fi - APPROVALS="$approvals" python3 - <<'PY' + SETTINGS="$settings" APPROVALS="$approvals" python3 - <<'PY' import json, os, sys - environment = os.environ["ENVIRONMENT"] + environment, settings = os.environ["ENVIRONMENT"], json.loads(os.environ["SETTINGS"]) + reviewers = sum(len(rule.get("reviewers") or []) for rule in settings.get("protection_rules") or [] + if rule.get("type") == "required_reviewers") + if reviewers < 1: + sys.exit(f"::error::the {environment} environment names no required reviewer now, " + "so this deployment did not wait for an approval") + policy = settings.get("deployment_branch_policy") + if not policy or policy.get("custom_branch_policies") is not True: + sys.exit(f"::error::the {environment} environment no longer limits the refs that may " + f"deploy to it (deployment_branch_policy {policy!r}), so nothing is uploaded") approved = [review["user"]["login"] for review in json.loads(os.environ["APPROVALS"]) if review.get("state") == "approved" and any(e.get("name") == environment for e in review.get("environments", []))] if not approved: sys.exit(f"::error::this run's review history holds no approval of a deployment " f"to {environment}, so nothing is uploaded") - print(f"{environment}: a required reviewer is named, and this run's deployment " - f"was approved by {', '.join(approved)}") + print(f"{environment}: a required reviewer is named, the refs that deploy are limited, " + f"and this run's deployment was approved by {', '.join(approved)}") PY # BEFORE THE UPLOAD, NOTHING ELSE HELD. skip-existing skips a file name @@ -746,7 +815,7 @@ jobs: rm -rf "$root" git init -q -b main "$root" if ! GIT_TERMINAL_PROMPT=0 git -C "$root" -c credential.helper= \ - fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git main \ + fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git refs/heads/main \ || ! pin="$(git -C "$root" show FETCH_HEAD:contracts/code-pin.yaml)"; then echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" exit 1 @@ -1003,44 +1072,50 @@ jobs: printf '%s %s\n%s %s\n' "$WHEEL_SHA256" "dist/$WHEEL" "$SDIST_SHA256" "dist/$SDIST" \ | sha256sum --strict -c - - # THE APPROVAL, AT USE TIME. The build job's check that both - # environments ask a reviewer goes stale while the run waits: a rule - # removed meanwhile would let this job start with nobody approving it. - # So, right before the preflight, the pin check and the upload, this - # step requires that this job's environment still names a required - # reviewer AND that this run's review history holds an approval of a - # deployment to it. Either missing refuses, and nothing is uploaded. - - name: this environment still asks a reviewer, and this run's deployment to it was approved + # THE APPROVAL, AT USE TIME. The build job's check of both environments + # goes stale while the run waits: a rule removed meanwhile would let + # this job start with nobody approving it, and a deployment limit + # removed meanwhile would let any ref deploy. So, right before the + # preflight, the pin check and the upload, this step requires that this + # job's environment still names a required reviewer AND still limits + # the refs that deploy to it (`custom_branch_policies`), AND that this + # run's review history holds an approval of a deployment to it. Any one + # missing refuses, and nothing is uploaded. + - name: this environment still asks a reviewer and limits its refs, and this run's deployment to it was approved timeout-minutes: 2 env: GH_TOKEN: ${{ github.token }} ENVIRONMENT: pypi run: | - if ! reviewers=$(gh api "repos/$GITHUB_REPOSITORY/environments/$ENVIRONMENT" \ - --jq '[.protection_rules[] | select(.type == "required_reviewers") | .reviewers[]] | length'); then + if ! settings=$(gh api "repos/$GITHUB_REPOSITORY/environments/$ENVIRONMENT"); then echo "::error::the $ENVIRONMENT environment cannot be read, so its reviewer rule is unknown" exit 1 fi - if [ "$reviewers" -lt 1 ]; then - echo "::error::the $ENVIRONMENT environment names no required reviewer now, so this deployment did not wait for an approval" - exit 1 - fi if ! approvals=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID/approvals"); then echo "::error::this run's review history cannot be read" exit 1 fi - APPROVALS="$approvals" python3 - <<'PY' + SETTINGS="$settings" APPROVALS="$approvals" python3 - <<'PY' import json, os, sys - environment = os.environ["ENVIRONMENT"] + environment, settings = os.environ["ENVIRONMENT"], json.loads(os.environ["SETTINGS"]) + reviewers = sum(len(rule.get("reviewers") or []) for rule in settings.get("protection_rules") or [] + if rule.get("type") == "required_reviewers") + if reviewers < 1: + sys.exit(f"::error::the {environment} environment names no required reviewer now, " + "so this deployment did not wait for an approval") + policy = settings.get("deployment_branch_policy") + if not policy or policy.get("custom_branch_policies") is not True: + sys.exit(f"::error::the {environment} environment no longer limits the refs that may " + f"deploy to it (deployment_branch_policy {policy!r}), so nothing is uploaded") approved = [review["user"]["login"] for review in json.loads(os.environ["APPROVALS"]) if review.get("state") == "approved" and any(e.get("name") == environment for e in review.get("environments", []))] if not approved: sys.exit(f"::error::this run's review history holds no approval of a deployment " f"to {environment}, so nothing is uploaded") - print(f"{environment}: a required reviewer is named, and this run's deployment " - f"was approved by {', '.join(approved)}") + print(f"{environment}: a required reviewer is named, the refs that deploy are limited, " + f"and this run's deployment was approved by {', '.join(approved)}") PY # BEFORE THE UPLOAD, NOTHING ELSE HELD. skip-existing skips a file name @@ -1105,7 +1180,7 @@ jobs: rm -rf "$root" git init -q -b main "$root" if ! GIT_TERMINAL_PROMPT=0 git -C "$root" -c credential.helper= \ - fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git main \ + fetch -q --depth 1 --no-tags https://github.com/opensoft/openDox.git refs/heads/main \ || ! pin="$(git -C "$root" show FETCH_HEAD:contracts/code-pin.yaml)"; then echo "::error::opensoft/openDox main's contracts/code-pin.yaml cannot be read, so the release commit is unknown" exit 1 diff --git a/tests/test_release_workflow.py b/tests/test_release_workflow.py index f3d8f900..173a7db6 100644 --- a/tests/test_release_workflow.py +++ b/tests/test_release_workflow.py @@ -37,8 +37,12 @@ * before that, the preflight: the index holds no file of this version but a verified one at its verified digest, so skip-existing can never make a mixed release; - * the environment step: both environments with a reviewer pass, and either - one without a reviewer, or unreadable, refuses; + * the environment step: both environments with a reviewer and a deployment + limit pass, and either one without a reviewer, without a limit (or with + protected branches in place of one), or unreadable, refuses; + * `main` is `refs/heads/main`, in the root's fetch and in the compare: a + stand-in root whose tag `main`, at a commit off the branch, pins another + commit cannot name the release commit, in any of the three jobs; * the version gate, case by case, a pre-release and a development release among the refusals; * the artifact checks, over a small built-by-hand wheel and sdist in a git @@ -180,7 +184,7 @@ def test_the_four_jobs_are_chained() -> None: printf '%s\\n' "$FAKE_APPROVALS" ;; *environments/*) args="$*"; environment="${args##*environments/}"; environment="${environment%% *}" - variable="FAKE_REVIEWERS_${environment}" + variable="FAKE_ENVIRONMENT_${environment}" [ -z "${!variable:-}" ] && { echo "HTTP 404" >&2; exit 1; } printf '%s\\n' "${!variable}" ;; *) echo "the stand-in gh has no answer for: $*" >&2; exit 2 ;; @@ -196,6 +200,23 @@ def _fake_gh(where: Path) -> Path: return _shim(where) +LIMITED = {"custom_branch_policies": True, "protected_branches": False} + + +def _environment(reviewers: int, policy: dict | None = LIMITED) -> str: + """An environment as `GET /repos/{owner}/{repo}/environments/{name}` + answers it (the shape read from opensoft/openDox-code's own environments + on 2026-10-05): `reviewers` required reviewers, and `policy` as its + `deployment_branch_policy`, which is null when any ref may deploy.""" + rules = [{"id": 1, "type": "required_reviewers", "prevent_self_review": False, + "reviewers": [{"type": "User", "reviewer": {"login": f"reviewer{n}"}} + for n in range(reviewers)]}] + if policy is not None: + rules.append({"id": 2, "type": "branch_policy"}) + return json.dumps({"id": 1, "name": "an-environment", "protection_rules": rules, + "deployment_branch_policy": policy}) + + def _pin(commit: str, source: str = REPOSITORY) -> str: return (f'leg_role: code\nsource_repository: {source}\nsubmodule_path: code\n\n' f'commit: "{commit}"\nrevision_kind: commit\n') @@ -295,11 +316,52 @@ def test_the_root_pin_is_read_over_git_with_no_credential() -> None: script = step["run"] assert "contents/contracts/code-pin.yaml" not in script assert re.search(r'GIT_TERMINAL_PROMPT=0 git -C "\$root" -c credential\.helper= \\\n' - r'\s+fetch -q --depth 1 --no-tags ' + re.escape(ROOT_URL) + r' main', script), script + r'\s+fetch -q --depth 1 --no-tags ' + re.escape(ROOT_URL) + r' refs/heads/main \\\n', + script), script assert not re.search(r"\bgh\b", script) and "env" not in step, step ref = _step("build", REF_STEP)["run"] calls = [line.strip() for line in ref.splitlines() if re.search(r"\bgh\b", line)] assert len(calls) == 1 and "/compare/" in calls[0], calls + # `main` is the branch, never a bare name that a tag `main` would win. + assert '/compare/$GITHUB_SHA...refs/heads/main"' in calls[0], calls + + +def _root_with_a_tag_named_main(where: Path, tagged: str) -> Path: + """A stand-in root whose branch `main` pins PINNED, and whose TAG `main` + is a commit off that branch pinning `tagged` (lane 3's review D8, F1: git + resolves a bare `main` on the remote as the tag, before the branch).""" + root = _root(where, _pin(PINNED)) + env = {"PATH": os.environ.get("PATH", "/usr/bin:/bin"), "HOME": str(where), + "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_NOSYSTEM": "1", + "GIT_AUTHOR_NAME": "root", "GIT_AUTHOR_EMAIL": "root@example.invalid", + "GIT_COMMITTER_NAME": "root", "GIT_COMMITTER_EMAIL": "root@example.invalid"} + for args in (("checkout", "-q", "-b", "side"),): + subprocess.run(("git", "-C", str(root), *args), check=True, env=env) + (root / "contracts" / "code-pin.yaml").write_text(_pin(tagged), encoding="utf-8") + for args in (("commit", "-q", "-am", "an unreviewed pin, reachable only from the tag"), + ("tag", "main"), ("checkout", "-q", "main"), ("branch", "-q", "-D", "side")): + subprocess.run(("git", "-C", str(root), *args), check=True, env=env) + return root + + +@needs_a_shell +@pytest.mark.parametrize("job", ["build", "testpypi", "pypi"]) +@pytest.mark.parametrize("sha", ["the branch's pin", "the tag's pin"]) +def test_a_tag_named_main_in_the_root_cannot_name_the_release_commit( + job: str, sha: str, tmp_path: Path) -> None: + other = "2" * 40 + root = _root_with_a_tag_named_main(tmp_path / "root", other) + (tmp_path / "rt").mkdir() + env = {"GITHUB_REPOSITORY": REPOSITORY, "RUNNER_TEMP": str(tmp_path / "rt"), + "GITHUB_SHA": PINNED if sha == "the branch's pin" else other, **_redirect(root)} + result = _run(_step(job, PIN_STEP)["run"], tmp_path, env) + if sha == "the branch's pin": + assert result.returncode == 0, result.stdout + result.stderr + assert "is the commit opensoft/openDox main pins" in result.stdout + else: + assert result.returncode != 0, result.stdout + assert f"the openDox root pins {PINNED}" in result.stdout + result.stderr, \ + result.stdout + result.stderr def test_each_upload_rechecks_the_pin_right_before_it() -> None: @@ -335,10 +397,20 @@ def test_a_pin_that_moved_stops_the_upload(job: str, moved: bool, tmp_path: Path assert result.returncode == 0, result.stdout + result.stderr +ENVIRONMENT_STEP = "both environments ask a reviewer and limit the refs that deploy" + ENVIRONMENT_CASES = { - "both name a reviewer": (dict(testpypi="1", pypi="1"), None), - "pypi names none": (dict(testpypi="1", pypi="0"), "the pypi environment names no required reviewer"), - "testpypi does not exist": (dict(pypi="1"), "the testpypi environment cannot be read"), + "both name a reviewer and limit their refs": ( + dict(testpypi=_environment(1), pypi=_environment(1)), None), + "pypi names none": (dict(testpypi=_environment(1), pypi=_environment(0)), + "the pypi environment names no required reviewer"), + "testpypi does not exist": (dict(pypi=_environment(1)), "the testpypi environment cannot be read"), + "pypi limits no refs": (dict(testpypi=_environment(1), pypi=_environment(1, None)), + "the pypi environment does not limit the refs"), + "testpypi allows its protected branches, not named refs": ( + dict(testpypi=_environment(1, {"custom_branch_policies": False, "protected_branches": True}), + pypi=_environment(1)), + "the testpypi environment does not limit the refs"), } @@ -346,9 +418,9 @@ def test_a_pin_that_moved_stops_the_upload(job: str, moved: bool, tmp_path: Path @pytest.mark.parametrize("case", sorted(ENVIRONMENT_CASES)) def test_the_environment_step(case: str, tmp_path: Path) -> None: reviewers, refusal = ENVIRONMENT_CASES[case] - step = _step("build", "both environments ask a reviewer before they deploy") + step = _step("build", ENVIRONMENT_STEP) env = {"GITHUB_REPOSITORY": REPOSITORY, "GH_TOKEN": "unused", - **{f"FAKE_REVIEWERS_{name}": count for name, count in reviewers.items()}} + **{f"FAKE_ENVIRONMENT_{name}": document for name, document in reviewers.items()}} result = _run(step["run"], tmp_path, env, (_fake_gh(tmp_path / "bin"),)) if refusal is None: assert result.returncode == 0, result.stdout + result.stderr @@ -931,7 +1003,8 @@ def test_the_install_verdict(case: str, tmp_path: Path) -> None: # job's reviewer check goes stale while the run waits, so each publish job # re-reads its environment's rule and this run's review history. -APPROVAL_STEP = "this environment still asks a reviewer, and this run's deployment to it was approved" +APPROVAL_STEP = ("this environment still asks a reviewer and limits its refs, " + "and this run's deployment to it was approved") def test_each_publish_job_rechecks_its_approval_before_the_preflight() -> None: @@ -956,19 +1029,25 @@ def _approval(state: str, *environments: str) -> dict: APPROVAL_CASES = { - "a reviewer named, and this deployment approved": ( - dict(reviewers="1", approvals=[_approval("approved", "{env}")]), None), + "a reviewer named, the refs limited, and this deployment approved": ( + dict(environment=_environment(1), approvals=[_approval("approved", "{env}")]), None), "the reviewer rule removed meanwhile": ( - dict(reviewers="0", approvals=[_approval("approved", "{env}")]), "names no required reviewer now"), + dict(environment=_environment(0), approvals=[_approval("approved", "{env}")]), + "names no required reviewer now"), + "the deployment limit removed meanwhile": ( + dict(environment=_environment(1, None), approvals=[_approval("approved", "{env}")]), + "no longer limits the refs"), "the environment unreadable": ( - dict(reviewers=None, approvals=[_approval("approved", "{env}")]), "cannot be read"), - "no approval at all": (dict(reviewers="1", approvals=[]), "holds no approval"), + dict(environment=None, approvals=[_approval("approved", "{env}")]), "cannot be read"), + "no approval at all": (dict(environment=_environment(1), approvals=[]), "holds no approval"), "only the other environment approved": ( - dict(reviewers="1", approvals=[_approval("approved", "{other}")]), "holds no approval"), + dict(environment=_environment(1), approvals=[_approval("approved", "{other}")]), + "holds no approval"), "this deployment rejected": ( - dict(reviewers="1", approvals=[_approval("rejected", "{env}")]), "holds no approval"), + dict(environment=_environment(1), approvals=[_approval("rejected", "{env}")]), + "holds no approval"), "the review history unreadable": ( - dict(reviewers="1", approvals=None), "review history cannot be read"), + dict(environment=_environment(1), approvals=None), "review history cannot be read"), } @@ -982,8 +1061,8 @@ def test_the_approval_check(job: str, case: str, tmp_path: Path) -> None: approvals = given["approvals"] env = {"GITHUB_REPOSITORY": REPOSITORY, "GITHUB_RUN_ID": "1", "GH_TOKEN": "unused", "ENVIRONMENT": step["env"]["ENVIRONMENT"]} - if given["reviewers"] is not None: - env[f"FAKE_REVIEWERS_{env_name}"] = given["reviewers"] + if given["environment"] is not None: + env[f"FAKE_ENVIRONMENT_{env_name}"] = given["environment"] if approvals is None: env["FAKE_APPROVALS_FAILS"] = "1" else: From 07cc8f363bcbfebef5d31823e0ba7d2800c36c13 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 5 Oct 2026 11:26:31 +0000 Subject: [PATCH 15/15] Round 12, text only: the workflow checks a custom deployment limit, not its pattern Copilot's r4183379111 at 93d13217, RULED an accepted limit in openxFactory#656 comment 5993462741: the build job refuses when either environment has no custom deployment limit, and each publish job checks that again at use time. The limit's pattern, the tag v* alone, is configured (by the lane, 2026-10-05) and verified through the API, but the workflow does not re-read it. Changing it is an admin act, as is changing the reviewer rule (the reasoning that declined r4182777733). Comments only. The parsed workflow is identical to 93d13217's. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/release.yml | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 5af9de0f..c31a26df 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -36,13 +36,17 @@ name: release # workflow's file name and the environment, never the ref. So every gate in # this file binds only where the dispatched ref carries the reviewed file. # The one server-side control over the ref is each environment's deployment -# limit, and both environments MUST carry one: Brett set each to the tag -# pattern `v*` alone, so a dispatch on `main` or any branch builds and -# verifies but cannot deploy. The build job refuses when either environment -# has no such limit, and each publish job checks it again at use time. The -# limit that remains, accepted: while sessions act as brettheap, the approval -# click is the last line of defense, and a ruleset over `v*` tags is -# recommended, not required. +# limit, and both environments MUST carry one. The build job refuses when +# either environment has NO CUSTOM DEPLOYMENT LIMIT, and each publish job +# checks that again at use time. The limit's pattern is the tag `v*` alone, +# as the lane configured both environments on 2026-10-05 and verified through +# the API, so a dispatch on `main` or any branch builds and verifies but +# cannot deploy. The workflow does not re-read the pattern: changing it is an +# admin act, as is changing the reviewer rule, and an admin can edit this +# file too (accepted, RULED openxFactory#656 comment 5993462741). The limit +# that remains, accepted: while sessions act as brettheap, the approval click +# is the last line of defense, and a ruleset over `v*` tags is recommended, +# not required. # # FOUR JOBS, EACH GATED ON THE ONE BEFORE IT: # build builds the sdist and the wheel in a clean job, runs @@ -235,10 +239,12 @@ jobs: # dispatch runs the dispatched ref's own copy of this file, so an # environment that any ref may deploy to would publish whatever a branch # carries. Each environment must name a required reviewer, and its - # `deployment_branch_policy` must be set, with `custom_branch_policies` - # (the tag pattern `v*`, as Brett configured them). This step refuses - # either gap, before anything is built. The environments are public - # metadata of a public repository. + # `deployment_branch_policy` must be set, with `custom_branch_policies`: + # a custom deployment limit. Its pattern, the tag `v*`, is configured + # and was verified through the API, but this step does not re-read it + # (changing it is an admin act). This step refuses either gap, before + # anything is built. The environments are public metadata of a public + # repository. - name: both environments ask a reviewer and limit the refs that deploy env: GH_TOKEN: ${{ github.token }}