[ericdog]

What does this PR do?

Replace all Docker 20.10/20.10-py3 CI image references with already-published, digest-pinned images containing internal Buildx v0.37.2-dd.3:

• Docker 27.3.1 for layer builds, layer-size checks, and layer bundles.
• Docker 29.4.0-noble-py3 for the existing Python-based integration/signing/publishing jobs and their utility jobs.

Both image manifests come from merged images/master commit af53db03938b864dfea79025c679312a0479726d (Docker build, Python Docker build). No new image publication is required.

Motivation

The existing Docker images report upstream Buildx v0.8.0 and miss our internal reliability patches. Reuse maintained images instead of creating additional Docker 20.10 variants in images#12452. Related: https://datadoghq.atlassian.net/browse/CIEXE-2681.

Noble's system Python is externally managed, and the published Python image lacks AWS CLI. Install awscli/python3-venv/python-is-python3 before the affected jobs, and create the Python virtual environment before installing dependencies. This preserves the required AWS commands without writing Python packages into the managed system interpreter.

Testing Guidelines

• Rendered the before/after pipelines with gomplate and parsed them with Ruby/Psych. Verified all 88 affected generated jobs change only image selection and required tool setup, and AWS/Python consumers receive that setup.
• Bash syntax and ShellCheck error checks pass. A local script harness creates a real virtual environment and confirms dependency installs use it (package downloads stubbed).
• Registry manifests and published provenance verified; both images package internal Buildx v0.37.2-dd.3.
• git diff --check passes.

Full container/CI integration validation remains. The local Docker filesystem is nearly full. The commit no longer contains a CI-skip directive. Normal pipelines include automatic sandbox publishing and integration deployments; a deployment-free validation run must set the existing SKIP_E2E_TESTS=true pipeline variable.

Additional Notes

This also upgrades Docker and the Python job OS from the legacy image; reviewers should validate representative layer builds and AWS/Python workflows before merge. Library code, Lambda runtime images, release conditions, and deployment targets are unchanged.

Types of Changes

• Misc (CI dependency upgrade)

Check all that apply

• This PR's description is comprehensive
• This PR passes the integration tests (requires an explicitly approved run)

[ericdog]

/ddci trigger

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-11-04 16:27:34 UTC ℹ️ Start processing command /ddci trigger

---

2026-11-04 16:27:37 UTC

Tasks sourcing running asynchronously in workflow devflow:4924896d-a00c-476c-a539-b3c721df0f9f_47:01a1a763-25c5-7b53-9ea7-7b5e2b14c881 under run id 01a1a763-2e2d-7a17-b0cb-f742a5242fe9

[datadog-datadog-prod-us1-2]

With SKIP_PIP_INSTALL unset, the Noble-based layer-publishing jobs still invoke system pip and abort under PEP 668 before publishing any layers.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 0970e69}

[datadog-datadog-prod-us1-2]

Skip the redundant system-pip install in layer-publishing jobs

With SKIP_PIP_INSTALL unset, publish-layer-* jobs still execute pip install awscli in ci/publish_layers.sh despite installing AWS CLI through apt here. Noble's externally managed Python rejects that installation, and set -e aborts publishing before any layers are published. Export the existing skip flag for these jobs.

Suggested change

	- *install-python-tools
	- *install-python-tools
	- export SKIP_PIP_INSTALL=true

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means

• 👍 Useful: correct, and you're fixing it or will.
• 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
• 😕 Unclear: you can't tell what it means or why it matters.
• 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

_{🤖 Bits Code Review · Open Bits AI session}