Skip to content

[CIEXE-2681] Pin internal Buildx in the Lambda extension CI image - #1484

Merged
lucaspimentel merged 3 commits into
DataDog:mainfrom
ericdog:ericlim/internal-buildx-adoption-20261101
Nov 7, 2026
Merged

lucaspimentel merged 3 commits into
DataDog:mainfrom
ericdog:ericlim/internal-buildx-adoption-20261101

Conversation

@ericdog

@ericdog ericdog commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Overview

CI still selects an upstream Buildx client, so it misses our internal reliability fixes and diagnostics. Observed github.com/docker/buildx v0.11.2 9872040 in integration-suite: [payload-size-e2e] during the October 30–November 1 adoption audit.

Install Compute-published v0.38.1-dd.1 from the release mirror, verify its architecture-specific SHA-256 before extraction, and assert the exact client selected by docker buildx version. The custom CI image retains its existing Docker version. Separately, replace all legacy Docker 20.10 job-image references with the published Docker 27.3.1 multiarch manifest containing internal Buildx v0.37.2-dd.3. This covers the CI-image builder and 27 generated jobs, including layer/image builds and the E2E-status poller (which only needs shell/curl/jq, not Python).

Related: https://datadoghq.atlassian.net/browse/CIEXE-2681

Testing

Validated both published archive checksums and real Docker CLI selection with both binaries: native arm64 and amd64 through the host’s existing emulation. Shell syntax, ShellCheck, and BuildKit Dockerfile parsing pass. A local harness rejects unsupported architectures, download/checksum/extraction/install errors, and an upstream plugin shadowing the internal client. Full repository image builds have not been run; this does not validate every parent image or runtime user environment.

Remaining rollout

After review, build/sign the changed CI image through its normal workflow, verify docker buildx version in the resulting image, and update consumer pins to that published artifact. The direct Docker job pins take effect when merged; jobs using the custom CI image still require that image rebuild and pin update. CI is skipped on this review commit to avoid automatic publishing or deployments. The bootstrap path now uses the verified Docker 27.3.1 manifest built from merged images/master commit af53db03938b864dfea79025c679312a0479726d. No new shared-image release is required.

The consumer template renders with gomplate and parses with Ruby/Psych; all 27 generated changes are image-only. Root CI YAML parsing and git diff --check pass. No pipeline or deployment was triggered for the consumer-pin update.

@ericdog
ericdog force-pushed the ericlim/internal-buildx-adoption-20261101 branch from a7c1852 to a02424f Compare November 3, 2026 17:07
@ericdog
ericdog marked this pull request as ready for review November 5, 2026 11:39
@ericdog
ericdog requested a review from a team as a code owner November 5, 2026 11:39
@ericdog
ericdog requested review from clifordshelton and a balanced review from Copilot November 5, 2026 11:39
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-07T02:08:42.982793Z c941b5f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Full multiarchitecture image builds and representative build, signing, and E2E-poller CI jobs were not validated against the pinned image, leaving runtime compatibility unresolved.

No open issues 🎉

What changed in this PR

Updates Lambda extension CI to use internal Buildx releases with reliability fixes and diagnostics, while retaining the custom CI image’s existing Docker version.

Changes:

  • Installs Buildx v0.38.1-dd.1 in the custom CI image, verifies architecture-specific checksums, and checks the selected client version.
  • Replaces legacy Docker job images with a digest-pinned Docker 27.3.1 multiarch image containing internal Buildx v0.37.2-dd.3.

The custom CI image still needs to be rebuilt, signed, and adopted through updated consumer pins.

File Description
.gitlab/​templates/​pipeline.yaml.tpl Updates build, packaging, and E2E-status job image pins.
.gitlab/​Dockerfile Adds verified internal Buildx installation and client-selection validation.
.gitlab-ci.yml Updates the CI-image builder’s Docker image pin.

Review effort: Balanced

@lucaspimentel
lucaspimentel merged commit 9e08df1 into DataDog:main Nov 7, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants