[Noridom1]

Summary

Five commits on top of vng-benchmark (fast-forward, no conflicts):

• 02920960 feat(speedbench): shared SPEED-Bench client on aiperf. Adds run_speedbench_aiperf / resolve_speedbench_dataset to benchmark_lib.sh. infx.bench_serving.speedbench_aiperf maps the aiperf export onto the benchmark_serving result keys and adds acceptance length from the vllm:spec_decode_* counters. Adds an optional SPEEDBENCH_HF_TOKEN secret.

• 4f0bd067 feat(greenspec): GreenSpec arm on aiperf. run_greenspec_aiperf benchmarks one GreenSpec subset (internal prod-traffic spec-decode bench). It uses the aiperf greenspec_* preset when available, else a raw_payload file. The dataset comes from a private HF repo pinned by commit and checked against SHA256SUMS. A guard runs before any upload:

  • keeps only an allowlist of result files;
  • strips error messages from the aiperf export;
  • withholds the server log;
  • fails on any dataset row id or any oversized file.

  Adds an optional GREENSPEC_HF_TOKEN secret.

• 60c5a2fe fix(greenspec): no preset for a lite bucket selection. Bucket presets cover the full bucket, so GREENSPEC_BUCKET with GREENSPEC_LITE=1 now runs through raw_payload instead of sending the first N rows of the full bucket.

• 0c6498ac docs(greenspec): GreenSpec procedures. New docs/greenspec-procedures.md (flow, prerequisites, selection, adding a cell, dispatch and results, troubleshooting, data-safety rules), linked from docs/index.md and docs/procedures.md. English only for now; the _zh.md pair is not included.

• 8e8d4343 feat(greenspec): progress lines in the job log. aiperf's console stays off the job log (it can echo server error text). A watcher reads it every GREENSPEC_PROGRESS_INTERVAL seconds (default 60, 0 = off) and prints only the progress-bar numbers, e.g. GreenSpec progress: Profiling 33/64 (52%) elapsed 02:10 eta 02:00. Checked end to end against aiperf-mock-server.

Submodule change

utils/aiperf moves from SemiAnalysisAI/aiperf@faf3f8b to Noridom1/aiperf@8f69bd6f (branch feat/greenspec-loader). That is the same base plus one commit, the GreenSpec loader and presets. Existing aiperf/agentx recipes run on the same code.

Note: faf3f8b (the current pin on vng-benchmark) is not on any branch of SemiAnalysisAI/aiperf; it only lives on a personal fork branch. Before merge, the plan is to:

1. create a branch on vngcloud/aiperf at faf3f8b;
2. PR the loader commit there;
3. point this submodule at vngcloud/aiperf.

Validation

• utils/test_greenspec_aiperf.py and utils/test_speedbench_aiperf.py pass.
• CI run https://github.com/vngcloud/InferenceX/actions/runs/40636901474, Gemma 4 31B FP8 on hardware-hcm, TP1, c8, GreenSpec agentic_tools lite. It ran from the bench branch, which adds a test recipe on top of this PR.

  • Both cells passed: 256/256 requests each, no errors. Each GreenSpec artifact holds only the 5 allowlisted files.

    Metric	Base	MTP	MTP vs base
    Output throughput (tok/s)	52.3	74.1	1.42×
    Total token throughput (tok/s)	4259	5524	1.30×
    TPOT median / mean (ms)	107.1 / 123.1	71.8 / 88.0	1.49× / 1.40×
    TTFT median (ms)	2459	3442	0.71×
    E2E latency median (ms)	21467	15969	1.34×
    Acceptance length / rate	n/a	3.85 / 0.71

    No ignore_eos, so output lengths differ between arms (82k vs 90k tokens). Compare TPOT and throughput, not duration. ITL is per chunk and is not comparable under MTP.

• A direct Slurm smoke with the same recipe passed both arms (base and MTP).

Not in this PR

• The hardware-hcm Slurm account fix (27185633, SLURM_ACCOUNT="greennode") is still needed on vng-benchmark. The CI run above used it via the bench branch.
• The Gemma 4 GreenSpec recipe stays on bench/gemma4-greenspec-hcm.

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast；仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.