senior-dev: a CMake default, and --verify-build/--verify-test for a check discovery cannot find

[ZeroPoint95]

Why

CyberGym attempt 5c2 fixed and externally verified arvo_24633. The official grader later scored the recovered deliverables 4/4. But codeaf senior-dev run exited 2 with no build entrypoint could be discovered (verification_commands: 0).

Root cause (Gap B in the verifier brief): a root CMakeLists.txt makes a workspace accountable, so it must have both a build and a test, but no ecosystem default could name either for CMake. So every header-only CMake library failed verification for want of a command, while its own ctest suite sat in the tree.

What changes

1. A CMake ecosystem default (fullverification/discovery.go). It is asked last, so every project that already had a default keeps exactly the one it had.
   • build: cmake -S . -B .senior-dev/cmake-build && cmake --build .senior-dev/cmake-build --parallel "$(getconf _NPROCESSORS_ONLN 2>/dev/null || echo 2)"
   • test: the same configure and build, then cd .senior-dev/cmake-build && ctest --output-on-failure. Discovery chooses each kind separately, so the test can't assume the CMake build ran: a README or Makefile might supply the build. Running ctest from inside the folder also works before CMake 3.20.
   • The build folder is inside .senior-dev, which no recorder or fingerprint reads, so it never ends up in the handed-in change, and a later pass builds on the previous one.
   • Parallelism is capped at the CPU count. A bare --parallel gives Makefiles an unlimited -j. On ArduinoJson that OOM-killed cc1plus, and a serial build does not fit the 600s ceiling.
2. --verify-build CMD / --verify-test CMD on codeaf senior-dev run (fullverification/declared.go). These are for a project whose real check lives outside the folder: a fuzz harness, or a benchmark's validate.py.
   • Each one replaces discovery for its kind.
   • senior-dev runs it itself on the frozen tree, with the same set -euo pipefail preamble and ceiling.
   • A declared kind is still required, and naming one kind never excuses the other.
   • They come only from the command line. The chat's own line to senior-dev is fixed, and the working model cannot set them. This closes the gap in the brief's original option 1: a check the model could choose is a check it could choose to pass (true).
3. Manual: a new senior-dev.md section, "How does senior-dev find a project's build and tests", quoting the exact ending string; both flags added to the flags section; retrieval probes in internal/manual/chat_test.go.

Not done, on purpose: the brief's "the agent declares its own verification commands at submit" option, and any downgrade of a missing entrypoint to pass-unverified.

Proof

• New unit tests (declared_test.go, full_verification_declared_test.go) cover:
  • the CMake plan, and that it never displaces an earlier default (Go, Python with a native extension, Makefile targets);
  • Declare replacing one kind only, and still requiring the other;
  • a declared test that runs outside the workspace in place of a red discovered one;
  • a red declared command, and a masked pipeline (false | cat), both failing;
  • the build folder staying under seniorDevDataDirectory.
• Real CMake run. The exact strings Discover emits, run under set -euo pipefail on upstream ArduinoJson HEAD in ubuntu:24.04 (cmake 3.28, GCC 13, 4 CPUs): build exit=0 in 24s; ctest 100% tests passed, 0 tests failed out of 27.
• Test command on its own (the case where the build came from somewhere else): ubuntu:24.04 / CMake 3.28 gives 27/27 passed in 26s; ubuntu:20.04 / CMake 3.16 gives 26/26 passed in 38s; a second run takes about 1s.
• Not yet proved: a 5c2 rerun on the CyberGym image (old ArduinoJson commit, OSS-Fuzz sanitizer flags). That is the next step to show the default passes end to end on the benchmark.

🤖 Generated with Claude Code