Repository navigation
[SVLS-9963] chore(bottlecap): upgrade libddwaf, zstd, and serverless-components - #1467
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
c8b4070 to
bb0f175
Compare
|
🔗 Commit SHA: 53775b8 | Docs | View more details | Give us feedback! |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the dependency upgrades and AppSec API migration.
No open issues 🎉
What changed in this PR
Updates Bottlecap’s compression and WAF dependencies, adapting its AppSec integration to libddwaf 2.1.
Changes:
- Upgrades zstd to 0.14.3 and libddwaf/libddwaf-sys to 2.1.0, enabling AWS-LC for WAF binary downloads.
- Migrates AppSec initialization, execution, and object handling to the updated WAF API.
- Updates integration-test expectations for WAF version lookup and numeric array indices in trigger output.
| File | Description |
|---|---|
bottlecap/tests/appsec_processor_test.rs |
Updates version lookup and expected trigger JSON. |
bottlecap/src/appsec/processor/mod.rs |
Adapts WAF initialization and diagnostics types. |
bottlecap/src/appsec/processor/context.rs |
Migrates WAF execution, serialization, and collection construction. |
bottlecap/Cargo.toml |
Updates dependency versions and enables the downloader’s AWS-LC provider. |
bottlecap/Cargo.lock |
Records updated dependencies and transitive resolution changes. |
Review effort: Balanced
cba1d45 to
98a7959
Compare
litianningdatadog
changed the title
4856b50 to
5f3d734
Compare
73ea2d3 to
6090bf3
Compare
|
/code blockers |
|
View all feedbacks in Devflow UI.
No merge blockers detected. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6090bf3e2f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| libdd-trace-protobuf = "5.1.1" | ||
| libdd-trace-utils = { version = "16.0.0", default-features = false, features = ["mini_agent"] } |
There was a problem hiding this comment.
Align libdd-trace-normalization with the newly upgraded trace packages. The resulting lockfile now contains normalization 3.0.1, which pulls libdd-trace-protobuf 4.0.1, alongside normalization 4.3.1 and protobuf 5.1.1 pulled by libdd-trace-utils 16.0.0. Every build therefore compiles two normalization/protobuf generations, undermining the dependency alignment and increasing build/artifact overhead; pinning the direct normalization dependency to the 4.3.1 release line removes the obsolete copies.
Useful? React with 👍 / 👎.
| datadog-opentelemetry = { version = "0.6.0", default-features = false, features = ["_unstable_propagation"] } | ||
| dogstatsd = { git = "https://github.com/DataDog/serverless-components", rev = "e462105e918904e6b1ad2ebc153ae561cebe5a34", default-features = false } |
There was a problem hiding this comment.
Update the PR overview before merging: it says this change pins serverless-components to fd24c365ade64078b697e00f57f823aff32a65f7 and datadog-opentelemetry to 0.5.4, while the final manifest actually pins e462105e918904e6b1ad2ebc153ae561cebe5a34 and 0.6.0. As written, reviewers and release audits are being asked to validate different dependency revisions from the ones that will ship.
AGENTS.md reference: AGENTS.md:L22-L23
Useful? React with 👍 / 👎.
| tonic = { version = "0.14", features = ["transport", "codegen", "server", "channel", "router"], default-features = false } | ||
| tonic-types = { version = "0.14", default-features = false } | ||
| zstd = { version = "0.13.3", default-features = false } | ||
| zstd = { version = "0.14.3", default-features = false } |
There was a problem hiding this comment.
Remove the zstd-related entries for RUSTSEC-2026-0391, RUSTSEC-2026-0398, and RUSTSEC-2026-0403 from bottlecap/deny.toml as part of this upgrade. The comments at deny.toml:49-52 explicitly say the new zstd 0.14/zstd-safe 8.2 dependency line fixes these advisories and that the ignores should be dropped with the bump; leaving them active means a future dependency resolution that reintroduces an affected zstd version will still be silently accepted by the cargo-deny audit.
Useful? React with 👍 / 👎.
6090bf3 to
3b9f18c
Compare
3b9f18c to
53775b8
Compare
Keep the #1467 dependency pins without ddsketch-agent, regenerate Cargo.lock and LICENSE-3rdparty.csv, and list libddwaf-sys in the cargo-machete ignore list: it is a dependency only to enable its aws-lc-rs feature.
**TL;DR:** Removes 6 dependencies that no code uses: indexmap, serde-aux, lazy_static, log, ddsketch-agent and proptest. **Why:** Less to compile, audit and keep up to date. The next PR (#1471) adds a CI check so that unused dependencies do not come back. ## Overview Jira: [SVLS-9975](https://datadoghq.atlassian.net/browse/SVLS-9975) Based on `main` (#1469 is merged). - Remove 6 dependencies that no code uses: `indexmap`, `serde-aux`, `lazy_static`, `log`, `ddsketch-agent` and the dev-dependency `proptest`. The lock drops the 4 packages that only `proptest` pulled in: `bit-set` 0.8, `bit-vec` 0.8, `rusty-fork` and `wait-timeout`. Older `bit-set` (0.5) and `bit-vec` (0.6) stay in the lock, through the `httpmock` dev dependency. `LICENSE-3rdparty.csv` drops the same 4 rows, so the required "Valid LICENSE-3rdparty.csv" check passes. - `[package.metadata.cargo-machete] ignored` lists the 5 crates that stay on purpose, with the reason in a comment: version bounds on transitive crates, `aws-lc-rs` reached through `rustls::crypto`, `serde_bytes` used in a `#[serde(with)]` string, and `libddwaf-sys`, which is listed only to enable its `aws-lc-rs` feature (#1467). The next PR (#1471) adds the CI job that reads it. Size: about 16 core lines; the rest is `Cargo.lock` and `LICENSE-3rdparty.csv`. ## Testing - applies on the previous PR of the stack: pass - `cargo metadata --locked`: pass - `cargo deny --locked check`: pass - `cargo machete`: pass - `dd-rust-license-tool check`: pass - `cargo clippy --workspace --all-targets --features default -- -D warnings`: pass - `cargo clippy --workspace --all-targets --features default,test-mode -- -D warnings`: pass - Before this stack, the FIPS flavor compiled in Docker with the removal (`compile_bottlecap.sh`, arm64, FIPS=1). [SVLS-9975]: https://datadoghq.atlassian.net/browse/SVLS-9975?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
…h-suite/feature Adapt the benches to what the merge brings in from main and the fixtures: - init: the fixtures build the enrichment tags with enrich_tags(), not ENRICH_TAGS. - traces: the libdatadog bump (#1467) renamed obfuscate_span to obfuscate_pb_span, HttpConfig.remove_paths_with_digits to remove_path_digits, and new_without_connection_pooling to new_periodic. tag_replace_rules is now built by startup::build_tag_replace_rules, made doc(hidden) pub so the bench builds it as production does.
Overview
Upgrades zstd to 0.14.3 and libddwaf/libddwaf-sys to 2.1.0, updates Bottlecap AppSec for the libddwaf 2.1 API, enables its AWS-LC downloader provider, and refreshes the affected third-party license notices. The zstd upgrade also removes the obsolete cargo-deny suppressions for advisories fixed by the new zstd/zstd-safe dependency line.
Updates Bottlecap to serverless-components commit
e462105e918904e6b1ad2ebc153ae561cebe5a34, which moves datadog-opentelemetry to 0.6.0 and aligns the direct libdatadog dependencies on that release line. In particular, the direct normalization pin now uses 4.3.1, eliminating the obsolete normalization 3.0.1 and protobuf 4.0.1 generation from the resolved graph.The libdatadog obfuscation API now expects concrete tag-replacement rules built with its regex engine, rather than Bottlecap passing its optional configuration through unchanged.
startup.rstherefore converts an absent configuration to an empty rule list, rebuilds configured rules with the required regex type, preserves literal-versus-capture-group replacement behavior, and carries forward the renamed HTTP path-digit option. This keepsDD_APM_REPLACE_TAGSbehavior intact across the dependency upgrade.Compatibility
No change to Bottlecap’s public extension endpoints or supported configuration is intended. The WAF 2.1 upgrade does change the AppSec
_dd.appsec.jsontag: array indexes inrule_matches[].parameters[].key_pathare now JSON numbers (for example,0) rather than numeric strings (for example,"0"). Downstream consumers that parse this tag must accept numeric indexes.Jira: https://datadoghq.atlassian.net/browse/SVLS-9963
Testing
cargo fmt --checkcargo check --workspace --lockedcargo test --locked --test appsec_processor_test test_processorcargo test --locked traces::propagation --libcargo test --locked traces::stats_concentrator_service --libcargo test --locked startup::tests::build_tag_replace_rules_preserves_literal_and_capture_replacements --libdd-rust-license-tool checkBSD-3-Clause.