Skip to content

fix: robust extract_code_block (full-file block, nested fences) - #2

Open
datj9 wants to merge 1 commit into
mainfrom
fix/extract-code-block
Open

datj9 wants to merge 1 commit into
mainfrom
fix/extract-code-block

Conversation

@datj9

@datj9 datj9 commented Sep 14, 2026

Copy link
Copy Markdown
Owner

What

ownbench/runner.py::extract_code_block had two defects that could silently zero a correct answer:

  1. Last block wins. It returned blocks[-1]. When a model emits the complete fixed file first and then appends prose containing a small ```py snippet, the tiny (wrong) trailing block was returned instead of the full file.
  2. Non-greedy capture + unanchored fence. The regex \``[^\n]\n(.?)```` used DOTALL with a closing fence that was not anchored to the start of a line. A ``` appearing inside file content (e.g. an indented fence in a docstring) ended the capture early, returning a truncated slice.

Fix

Replaced the single regex with a line-based scan:

  • Opening fence: a line matching ^\s*```[^]*$` (leading whitespace allowed, optional info string), only when not already inside a block.
  • Closing fence: a line matching ^```\s*$, i.e. three backticks at column 0 (optional trailing whitespace).
  • Collects every top-level block (inner lines joined by \n, preserving the newline before the closing fence to match prior semantics).
  • Returns the block with the greatest character length (max(blocks, key=len)), or None when there are none.

The full file outweighs a trailing snippet, and inner docstring fences are indented so they never match a column-0 closing fence. Signature/return type unchanged.

Tests added (tests/test_extract.py)

  • test_trailing_snippet_ignored — full-file block followed by prose + a tiny ```py snippet; the full block must win.
  • test_nested_fence_in_docstring — a block containing an indented fence inside a docstring must be returned whole, not truncated.
  • test_regression — a single clean block is extracted exactly; text with no fence returns None.

Verification

  • RED (before fix): the two defect tests fail.
  • GREEN (after fix): full suite green — 18 passed / 0 failed.

Co-authored-by: CommandCodeBot noreply@commandcode.ai

…block

Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant