[AMD] [AgentX] Add MI355X Qwen3.5 FP4 SGLang MTP

[cquil11]

Summary

Add MI355X Qwen3.5-397B-A17B MXFP4 AgentX with SGLang native EAGLE MTP.

Performance scope

• Retain the measured resident frontier: TP2/EP2 c1-c20 and TP4 c1-c40.
• Use the committed golden synthetic acceptance length for throughput and real target verification for evaluation.
• Export SGLang Prometheus/cache metrics to AIPerf.
• Disable AITER all-reduce fusion to prevent TP>1 EAGLE rank divergence.

Validation

• Complete discovery (41/41 terminal): https://github.com/SemiAnalysisAI/InferenceX/actions/runs/31560100623
• 26 resident arms passed with zero profiled request errors. TP2 reaches its capacity transition at c20; TP4 peaks at c40 before throughput declines.
• HiCache isolation: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/31579113764
• The supported page-first/kernel/page64 path reaches hybrid-pool attachment, but both TP2 and TP4 then hit upstream SGLang HybridLinearKVPool/EAGLE incompatibility. Offload is therefore excluded rather than patched around in this recipe.

Next: full-duration official sweep on the retained matrix.

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

An inline bug was found that will break the throughput sweep for this recipe, so this needs a human look and a fix before merge.

Extended reasoning...

This PR is a new benchmark recipe (script + config + changelog entry) for an existing model/hardware pattern, but the inline finding is a real, run-breaking bug: appending --use-chat-template to REPLAY_CMD after build_replay_cmd will make aiperf profile reject the flag and fail every throughput concurrency point in this recipe.

[claude]

🔴 Line 177 appends REPLAY_CMD+=" --use-chat-template" after build_replay_cmd, but build_replay_cmd (benchmark_lib.sh:1927) builds an aiperf profile invocation for the agentic /v1/chat/completions replay path, not the run_benchmark_serving (benchmark_serving.py) path where this flag is actually recognized. This will make aiperf profile reject the unrecognized argument and exit non-zero, failing every non-eval (throughput) concurrency point in this new recipe. Fix: delete line 177 entirely.

Extended reasoning...

build_replay_cmd in benchmark_lib.sh:1887-2053 constructs REPLAY_CMD as an $AIPERF_CLI profile --scenario inferencex-agentx-mvp ... invocation, targeting --endpoint /v1/chat/completions with --endpoint-type chat. Nowhere in this function (or anywhere in the aiperf profile argument set) is --use-chat-template handled — that flag is defined and consumed exclusively inside run_benchmark_serving (benchmark_lib.sh:520 for parsing, :636-638 for application), which drives utils/bench_serving/benchmark_serving.py for the fixed-seq-len scripts, not the agentic replay path.

In qwen3.5_fp4_mi355x_sglang_mtp.sh:176-179, the non-EVAL_ONLY branch calls build_replay_cmd "$RESULT_DIR" and then appends REPLAY_CMD+=" --use-chat-template" before calling run_agentic_replay_and_write_outputs "$RESULT_DIR". That function executes $REPLAY_CMD literally at benchmark_lib.sh:2112 ($REPLAY_CMD 2>&1 | tee "$result_dir/benchmark.log") and gates success on the exit code (replay_rc). Since aiperf profile has no such flag, it will reject the unrecognized CLI argument and exit non-zero, causing run_agentic_replay_and_write_outputs to treat the run as failed.

This is not a stylistic mismatch — it's a straightforward "passing an unsupported flag to a CLI tool" bug. The only reason it isn't obviously wrong from a shallow read is that --use-chat-template is a real, valid flag elsewhere in the same file, which makes it plausible at a glance that it belongs here too. But tracing which command REPLAY_CMD actually becomes shows it doesn't apply to this call site.

Corroborating evidence from the rest of the codebase: (1) the direct sibling B200 recipe qwen3.5_fp4_b200_sglang_mtp.sh appends --server-metrics after build_replay_cmd, never --use-chat-template; (2) no other agentic *_mtp.sh script (dsv4 variants, glm5.2, etc.) adds this flag to REPLAY_CMD; (3) kimik3_fp4_b300_vllm_mtp.sh:35-38 has an explicit comment stating exactly why agentic recipes must not add it: "AGENTS.md requires MTP scripts to pass --use-chat-template to run_benchmark_serving. Agentic recipes never call it -- the replay drives AIPerf against /v1/chat/completions, so prompts are already chat-formatted ... Nothing to add here."

Step-by-step proof of the failure: (1) Script reaches the else branch (EVAL_ONLY defaults to false, so this is the default/majority path — the sweep's agentic-coding scenario in configs/amd-master.yaml runs many conc-list throughput points, e.g. conc-list: [1, 4, 8, 12, 16, ...]). (2) build_replay_cmd "$RESULT_DIR" sets REPLAY_CMD="$AIPERF_CLI profile --scenario inferencex-agentx-mvp --url http://localhost:$PORT --endpoint /v1/chat/completions --endpoint-type chat ... --output-artifact-dir $RESULT_DIR/aiperf_artifacts ...". (3) Line 177 appends --use-chat-template to this string. (4) run_agentic_replay_and_write_outputs runs $REPLAY_CMD, i.e. invokes aiperf profile ... --use-chat-template. (5) aiperf's CLI parser (argparse or similar) has no such option registered for the profile subcommand and will error out ("unrecognized arguments: --use-chat-template") with a non-zero exit code. (6) replay_rc becomes non-zero, and the run is recorded as failed — for every throughput concurrency point across all four search-space rows in the new qwen3.5-fp4-mi355x-sglang-agentic-mtp config entry.

The fix is simply to delete line 177 (REPLAY_CMD+=" --use-chat-template"), leaving build_replay_cmd's output untouched, matching every other agentic *_mtp.sh script's pattern.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31544080186
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=31544080186