Fix verse range marker placement bug by benjaminking · Pull Request #371 · sillsdev/machine.py

benjaminking · 2026-09-15T19:36:40Z

This PR fixes #370, which is caused by verse ranges containing empty rows with empty alignment matrices, resulting in no markers being placed for the verse range. This changes the marker placement algorithm to ignore any empty rows with an empty alignment matrix.

This change is

A verse range in the USFM being updated can be matched by more than one row. The updater concatenated all of their texts but kept only the last row's metadata, so the place markers handler received alignment info that did not describe the text it was given. Extracting a verse range puts all of its text on the first verse and leaves the rest empty, so the common shape is one row with text followed by empty rows. The empty row's metadata won, its alignment matrix was empty, and the handler silently placed no markers at all for that range. The block now carries every matched row's metadata alongside the existing single dict, which is left as it was so that existing handlers reading block.metadata are unaffected. The place markers handler ignores rows that have no alignment info or an empty alignment matrix, since they contributed no text to align. If more than one row is left it warns, as only one row's alignment can be used and it describes only part of the block's text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

claude · 2026-09-15T19:41:03Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Enkidu93

@Enkidu93 reviewed 4 files and all commit messages, and made 1 comment.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on ddaspit).

ddaspit

@ddaspit reviewed 4 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on benjaminking).

machine/corpora/place_markers_usfm_update_block_handler.py line 45 at r1 (raw file):

                len(infos),
            )
        return infos[-1] if len(infos) > 0 else None

Unless I'm missing something, this seems to contradict the comments above.

machine/corpora/usfm_update_block.py line 32 at r1 (raw file):

    @property
    def metadata(self) -> dict[str, object]:

Nit: maybe we should remove this property altogether and just use row_metadata. It is unclear what the difference is between the two properties.

Enkidu93 · 2026-09-30T02:18:32Z

@benjaminking No rush, but if you could wrap this up when you get a chance, that would be great!

…ces within a single update block.

codecov-commenter · 2026-10-01T15:47:00Z

Codecov Report

❌ Patch coverage is 99.60474% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.22%. Comparing base (e2af2c8) to head (414db9d).
⚠️ Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
machine/corpora/usfm_update_block.py	91.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #371      +/-   ##
==========================================
+ Coverage   92.13%   92.22%   +0.08%     
==========================================
  Files         390      390              
  Lines       24644    24838     +194     
==========================================
+ Hits        22705    22906     +201     
+ Misses       1939     1932       -7

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

claude · 2026-10-01T15:54:12Z

-
-    def pop(self) -> UsfmUpdateBlockElement:
-        return self._elements.pop()
+    def rows(self) -> Sequence[UsfmUpdateBlockRow]:


Minor: F2. UsfmUpdateBlock is exported in __all__, and this removes its metadata property, its metadata= keyword, and add_text/add_token/add_embed/extend_last_element/update_refs/get_last_paragraph/pop, with no deprecation. A downstream handler that reads block.metadata gets AttributeError after a wheel upgrade, and C# UsfmUpdateBlock still has all of these. Is the break intended? If so, release it as one, or keep a metadata shim.

F2 still applies at c8973a5: making rows keyword-only fixes F1, but metadata and the builder methods are still removed from the exported UsfmUpdateBlock, with no shim and no deprecation note. A positional third argument is now a TypeError too. Is this going out as a breaking release?

The change in interface was intentional. There are no clients who use this particular property, so it's ok to make a breaking change.

claude · 2026-10-01T15:54:15Z

-            or not any(
-                e.is_placeable(alignment_info["paragraph_behavior"], alignment_info["style_behavior"]) for e in elements
-            )
+        if len(elements) == 0 or not any(


FYI: F4. Removing the row_count == 0 or column_count == 0 early return changes single-row output and diverges from C# PlaceMarkersUsfmUpdateBlockHandler, which still has that check. On the test_single_row_with_empty_alignment_matrix_places_markers_at_end input, base gives ... parte. \f + \ft note\f* then \p \w \w*, and this PR moves the embed after \p \w \w*. Issue #370 is about verse ranges, so if the single-row change is deliberate, note it for the C# port.

This was an intentional change. Previously, this could result in no markers being placed for some rows. The C# version should be updated to match.

Each row's alignment info describes only that row's text, so it is now always merged, even when another row has the same info. Deduplicating by identity broke rows that held equal copies, and deduplicating by equality would drop a repeated verse whose text and alignment are identical. Aligned word pairs outside a row's tokens are now rejected for every row, not only when rows are merged. A single row with such a pair used to place markers silently or fail with an IndexError that bypassed the error handler. Blocks with no markers to place return before their alignment is read, so they no longer log warnings about missing alignment info. The rows argument of UsfmUpdateBlock is keyword-only, so a metadata dict passed where it used to go raises a TypeError instead of being taken as rows. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

A block whose alignment info strips every marker it has is left alone, so its alignment is never used. Checking the aligned word pairs while reading the alignment info, before the info's behaviors were known, let such a block fail the whole update. The pairs are now checked only once there are markers to place, and still before rows are merged. Also trim the comments added on this branch to the non-obvious reasons behind the code. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

benjaminking

@benjaminking partially reviewed 5 files, made 5 comments, and resolved 4 discussions.
Reviewable status: 0 of 5 files reviewed, 1 unresolved discussion (waiting on ddaspit and Enkidu93).

machine/corpora/place_markers_usfm_update_block_handler.py line 45 at r1 (raw file):

Previously, ddaspit (Damien Daspit) wrote…

Unless I'm missing something, this seems to contradict the comments above.

Yes, it was a confusing way of writing it. Logically that line could only be reached if there was exactly one item in infos, so this was a way of selecting the only item. It should be more clear now.

machine/corpora/usfm_update_block.py line 32 at r1 (raw file):

Previously, ddaspit (Damien Daspit) wrote…

Nit: maybe we should remove this property altogether and just use row_metadata. It is unclear what the difference is between the two properties.

Done.

benjaminking · 2026-10-13T17:32:21Z

-            or not any(
-                e.is_placeable(alignment_info["paragraph_behavior"], alignment_info["style_behavior"]) for e in elements
-            )
+        if len(elements) == 0 or not any(


This was an intentional change. Previously, this could result in no markers being placed for some rows. The C# version should be updated to match.

benjaminking · 2026-10-13T17:32:22Z

-
-    def pop(self) -> UsfmUpdateBlockElement:
-        return self._elements.pop()
+    def rows(self) -> Sequence[UsfmUpdateBlockRow]:


The change in interface was intentional. There are no clients who use this particular property, so it's ok to make a breaking change.

claude · 2026-10-13T17:35:04Z

Review

Verdict: approve with fixes.
Most important: F2, machine/corpora/usfm_update_block.py:18. The exported UsfmUpdateBlock still has no metadata or builder methods, and rows is keyword-only. This breaks the published wheel, and the author still needs to say whether that is intended.
Counts: Critical 0, Important 1 (F2), Low 1 (F4).
Ran poetry run pytest tests/corpora -q at the merge of 414db9d into main: 316 passed, 3 skipped. That run includes test_alignment_is_not_checked_when_alignment_info_behaviors_leave_nothing_to_place, which is the F7 input. Ran /code-review high, which returned 10 candidates, and none is posted:
- 3 repeat F2, F3 and F6.
- 4 are on code this round did not change (update_usfm_parser_handler.py, the missing-info warning, the end-style stack). The elements[-1] access in _end_update_block is in main too.
- 1 is a cleanup the diff gave no reason for (pairs computed twice).
- The behaviors of the first row were already used at c8973a5.
- The worry that a range-wide info attached to every row now gets merged twice does not match the one consumer I found: silnlp/common/postprocessor.py builds one info per row from the tokens of that row.
Not verified: ./local_check.sh --agent-strict, which needed an approval this run did not have. Also not verified: whether anyone outside the sillsdev org uses the removed UsfmUpdateBlock members, or attaches one info to every row of a range.

Public API and parity: unchanged since the last round and still open as F2. UsfmUpdateBlock (exported from machine.corpora) loses metadata and its builders, and its third parameter is now keyword-only rows. C# UsfmUpdateBlock keeps Metadata. No optional dependencies changed.

Findings: F1 addressed, F2 open, F3 addressed, F4 open, F5 addressed, F6 addressed, F7 addressed.

Reviewed at 414db9d

Enkidu93

@Enkidu93 reviewed 5 files and all commit messages, and dismissed @claude[bot] from 2 discussions.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on ddaspit).

Ben King and others added 2 commits September 15, 2026 13:55

Simplify UsfmUpdateBlock constructor

fb1d80e

benjaminking requested a review from Enkidu93 September 15, 2026 19:36

Enkidu93 requested a review from ddaspit September 17, 2026 16:19

Enkidu93 approved these changes Sep 21, 2026

View reviewed changes

ddaspit requested changes Sep 22, 2026

View reviewed changes

ddaspit mentioned this pull request Sep 22, 2026

Improve the Claude code review workflow #377

Merged

Implement support for multiple text rows and multiple alignment matri…

aef9ece

…ces within a single update block.

claude Bot reviewed Oct 1, 2026

View reviewed changes

Comment thread machine/corpora/usfm_update_block.py

claude Bot reviewed Oct 1, 2026

View reviewed changes

Comment thread machine/corpora/place_markers_usfm_update_block_handler.py Outdated

claude Bot reviewed Oct 1, 2026

View reviewed changes

Comment thread machine/corpora/place_markers_usfm_update_block_handler.py Outdated

claude Bot reviewed Oct 1, 2026

View reviewed changes

Comment thread machine/corpora/place_markers_usfm_update_block_handler.py Outdated

This comment has been minimized.

Sign in to view

claude Bot reviewed Oct 9, 2026

View reviewed changes

Comment thread machine/corpora/place_markers_usfm_update_block_handler.py Outdated

This comment has been minimized.

Sign in to view

benjaminking commented Oct 13, 2026

View reviewed changes

Enkidu93 approved these changes Oct 16, 2026

View reviewed changes

Conversation

benjaminking commented Sep 15, 2026 • edited by ddaspit Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot commented Sep 15, 2026

Code review

Uh oh!

Enkidu93 left a comment

Choose a reason for hiding this comment

Uh oh!

ddaspit left a comment

Choose a reason for hiding this comment

Uh oh!

Enkidu93 commented Sep 30, 2026

Uh oh!

codecov-commenter commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

claude Bot Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Oct 9, 2026

Choose a reason for hiding this comment

Uh oh!

benjaminking Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

claude Bot Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

benjaminking Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

This comment has been minimized.

benjaminking left a comment

Choose a reason for hiding this comment

Uh oh!

benjaminking Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

benjaminking Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot commented Oct 13, 2026

Review

Uh oh!

Enkidu93 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

benjaminking commented Sep 15, 2026 •

edited by ddaspit

Loading

codecov-commenter commented Oct 1, 2026 •

edited

Loading