Skip to content

chore: regenerate catalog from OpenAPI spec - #60

Merged
AlexKantor87 merged 3 commits into
mainfrom
chore/update-catalog
Sep 16, 2026
Merged

AlexKantor87 merged 3 commits into
mainfrom
chore/update-catalog

Conversation

@github-actions

@github-actions github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Automated catalog update from the Kosli OpenAPI spec.

Please review the diff to verify the changes look correct before merging.

@claude

claude Bot commented Sep 10, 2026

Copy link
Copy Markdown

Claude encountered an error after 4m 43s —— View job


Catalog diff review

Bottom line: nothing was lost upstream — but this PR breaks main as-is. The 7606/7332 line churn is almost entirely a mass operationId rename upstream (49 ids), plus 5 genuinely new endpoints. Zero endpoints were removed. However, src/hints.json still keys off two of the old ids, and CI is red because of it.


🔴 Blocking: src/hints.json has dangling keys

scripts/generate-catalog.ts:41 uses operationId verbatim as the action id, so both hint keys are now stale:

src/hints.json key New catalog id
put_policy (line 2) create_or_update_policy
create_flow_with_template (line 8) create_or_update_flow_with_template

test/hints.test.ts — the guard added exactly for this drift — is failing on both Node 22 and Node 24:

FAIL test/hints.test.ts > hints.json > every hint key matches an action ID in the real catalog
AssertionError: hint key "put_policy" does not match any catalog action ID: expected [ …(123) ] to include 'put_policy'

Note the test throws on the first bad key, so it only reports put_policycreate_flow_with_template is stale too and will surface once the first is fixed. Left unfixed, both hints (policy schema URL + YAML example, flow-template schema URL + example) would silently stop attaching to search results — the exact failure the guard was written to catch.

Fix this →


1. Actions added — 5, all genuinely new endpoints

id method path
list_evaluations GET /evaluations/{org}
create_evaluation POST /evaluations/{org}
get_evaluation GET /evaluations/{org}/{evaluation_id}
list_trail_moments GET /trails/{org}/{flow_name}/{trail_name}/moments
get_trail_moment GET /trails/{org}/{flow_name}/{trail_name}/moments/{moment_expression}

Two new surfaces: evaluations and trail moments. create_evaluation is a POST, so it correctly lands behind execute_write_action.

2. Actions removed — none

Comparing by (method, path) rather than by id, the set of endpoints in the catalog is a strict superset of origin/main:

endpoints only in origin/main: (none)
endpoints only in HEAD:        5  (the evaluations + moments endpoints above)

So no removal confirmation against the live spec is needed for this PR. Both historical failure modes are ruled out by construction:

One caveat worth stating explicitly since it can't be seen in the diff: generate-catalog.ts:96 skips operations marked deprecated at the operation level. An endpoint newly deprecated upstream would appear here as a removal — and none did, so nothing was deprecated away either.

3. Actions changed

49 pure id renames (identical method + path, only id changed). This is an upstream operationId normalization toward verb_noun naming — get_*list_* for collections, post_*_attestationattest_*, *_reportreport_*_environment, put_*create_or_update_*/update_*, *_env*_environment. Full map:

49 renames (old id → new id)
old id new id endpoint
allow_artifact_for_env allow_artifact_for_environment PUT /allowlists/{org}/{env_name}
archive_env archive_environment PUT /environments/{org}/{env_name}/archive
attach_policy_to_env attach_environment_policy POST /environments/{org}/{env_name}/policies
detach_policy_from_env detach_environment_policy DELETE /environments/{org}/{env_name}/policies
post_custom_attestation attest_custom POST …/trail/{trail_name}/custom
post_generic_attestation attest_generic POST …/trail/{trail_name}/generic
post_jira_attestation attest_jira POST …/trail/{trail_name}/jira
post_junit_attestation attest_junit POST …/trail/{trail_name}/junit
post_pull_request_attestation attest_pull_request POST …/trail/{trail_name}/pull_request
post_snyk_attestation attest_snyk POST …/trail/{trail_name}/snyk
post_sonar_attestation attest_sonar POST …/trail/{trail_name}/sonar
post_system_attestation attest_system POST …/trail/{trail_name}/system
post_override_attestation override_attestation POST …/trail/{trail_name}/override
post_control create_control POST /controls/{org}
put_control update_control PUT /controls/{org}/{identifier}
post_custom_attestation_type create_or_update_custom_attestation_type POST /custom-attestation-types/{org}
get_custom_attestation_types list_custom_attestation_types GET /custom-attestation-types/{org}
create_env create_or_update_environment PUT /environments/{org}
list_envs list_environments GET /environments/{org}
update_env update_environment PATCH /environments/{org}/{env_name}
create_or_update_env_action create_or_update_environment_action PUT /actions/{org}/environments
put_environment_notifications create_or_update_environment_notification PUT /organizations/{org}/environments_notifications
get_environment_notifications list_environment_notifications GET /organizations/{org}/environments_notifications
create_flow_with_template create_or_update_flow_with_template PUT /flows/{org}/template_file
get_flows list_flows GET /flows/{org}
put_policy create_or_update_policy PUT /policies/{org}
get_audit_package download_artifact_audit_package GET /artifacts/{org}/{flow_name}/{fingerprint}/audit_package
get_artifact_by_commit_sha list_artifacts_by_commit_sha GET /artifacts/{org}/{flow_name}/commit_sha/{commit_sha}
build_statistics get_build_statistics GET /builds/{org}/statistics
get_env_auditlog get_environment_audit_log GET /environments/{org}/{env_name}/auditlog
get_env_events list_environment_events GET /environments/{org}/{env_name}/events
list_env_deployments list_environment_deployments GET /environments/{org}/{env_name}/deployments
environment_diff get_environment_diff GET /env-diff/{org}
environment_policy_schema_v1 get_environment_policy_schema_v1 GET /schemas/environment-policy/v1
flow_template_schema_v1 get_flow_template_schema_v1 GET /schemas/flow-template/v1
join_physical_to_logical join_logical_environment PUT /environments/{org}/{env_name}/join
get_attestations list_attestations GET /attestations/{org}
get_flow_attestations list_flow_attestations GET /attestations/{org}/{flow_name}
azure_apps_report report_azure_apps_environment PUT …/report/azure-apps
cloud_run_report report_cloud_run_environment PUT …/report/cloud-run
docker_report report_docker_environment PUT …/report/docker
ecs_report report_ecs_environment PUT …/report/ECS
k8s_report report_k8s_environment PUT …/report/K8S
lambda_report report_lambda_environment PUT …/report/lambda
s3_report report_s3_environment PUT …/report/S3
server_report report_server_environment PUT …/report/server
get_search search_artifacts GET /search/{org}/sha/{search_value}
put_user_default_org set_user_default_org PUT /user/{org}
patch_tags update_tags PATCH /tags/{org}/{resource_type}/{resource_id}

5 actions with real schema/description changes (no parameter added or removed anywhere — every changed action has an identical parameter name/location/required set before and after):

id change
assert_artifact fingerprint path param gained "pattern": "^[a-f0-9]{64}$"
create_artifact template_reference_name body field gained "maxLength": 255
list_attestations_for_criteria attestation_type description now also mentions system type names (e.g. 'decision')
create_service_account_api_key expires_at description: behaviour change — "Use 0 or omit for no expiry" → "Omit or pass 0 for the maximum; every key expires at most 365 days from creation"
rotate_service_account_api_key expires_at description: "Use 0 or omit for no expiry" → "Omit to keep the current key's expiry, bounded to at most 365 days from now"

⚠️ The two API-key ones are a server-side semantics change, not just wording: non-expiring keys are no longer possible (365-day cap). Nothing to change in this repo, but worth knowing it's now in the text the LLM reads.

Newly deprecated: none — 4 deprecated: true occurrences before and after, unchanged.

4. Dangling references

location stale id status
src/hints.json:2 put_policy 🔴 shipped code — must fix (see above)
src/hints.json:8 create_flow_with_template 🔴 shipped code — must fix (see above)
test/fixtures/catalog-subset.json:86,115 post_control, put_policy 🟡 fixture is self-contained so tests still pass, but it now describes ids that don't exist in the real catalog
test/client/kosli-client.test.ts:44, test/tools/execute-action.test.ts (7 sites), test/tools/search-actions.test.ts:57 put_policy, post_control 🟡 consequential on the fixture above
test/scripts/format-json.test.ts:35 post_control ⚪️ arbitrary sample data, harmless

No stale ids in README.md, docs/, or anywhere else in src/ outside hints.json. Refreshing the fixture to the new ids is optional for correctness but keeps it honest as "a hand-curated slice of the catalog" — reasonable to do in the same PR while you're renaming the hint keys.

5. Sanity checks

check result
Action count 118 → 123 (+5). No drop. ✅
Endpoints (method, path) removed 0
Surviving $ref in catalog 0 (0 in origin/main too) — resolve-refs.ts inlined everything ✅
deprecated flag count 4 → 4, unchanged ✅
Path/method changed under a retained id none ✅
CI 🔴 test (node 22) and test (node 24) failing — test/hints.test.ts only

Verdict

Don't merge yet. Rename the two keys in src/hints.json to match the new ids, and CI should go green — the catalog content itself is clean and needs no upstream verification.

I ran the diff analysis with jq against git show origin/main:src/catalog.json; I couldn't run npm test locally (command not permitted in this sandbox), so the CI failure above is quoted from the test (node 22) job log rather than reproduced locally. I also couldn't fetch the live OpenAPI spec (WebFetch not permitted), but as noted in §2 it isn't needed here since no endpoint disappeared.
· branch chore/update-catalog

dangrondahl added a commit that referenced this pull request Sep 10, 2026
The catalog-update review on PR #60 finished its review but the action
still failed with "Claude reported a successful result after 53 turns,
exceeding the configured maximum of 40" — a 48-rename catalog diff
takes more turns than the cap allowed.
@github-actions
github-actions Bot force-pushed the chore/update-catalog branch from e053a5b to db11eb0 Compare September 14, 2026 07:14
The regenerated catalog carries explicit operationIds from the OpenAPI
spec, renaming ~50 actions. Both keys in hints.json were left dangling:
put_policy -> create_or_update_policy and create_flow_with_template ->
create_or_update_flow_with_template.

Also refresh the hand-curated test fixture and its tests so they stop
asserting on IDs the API no longer has (post_control -> create_control,
search_artifacts_by_sha -> search_artifacts).
The catalog-update review on PR #60 finished its review but the action
still failed with "Claude reported a successful result after 53 turns,
exceeding the configured maximum of 40" — a 48-rename catalog diff
takes more turns than the cap allowed.
@AlexKantor87
AlexKantor87 merged commit d731200 into main Sep 16, 2026
5 checks passed
@AlexKantor87
AlexKantor87 deleted the chore/update-catalog branch September 16, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants