Limit zip file entries to 100MB by default

[claude]

Zip entries read through the Paratext and DBL corpora are now refused when they are larger than 100MB or compressed more than 100:1. Ports machine PR #481.

Changes

• open_bounded_stream in machine/utils/zip_entry_utils.py is the counterpart of OpenBoundedStream. It raises BadZipFile when the declared size or compression ratio exceeds the limits.
• BoundedStream in machine/utils/bounded_stream.py raises OSError if more than the limit is actually read, which covers entries whose headers understate their size.
• ZipEntryStreamContainer, ZipParatextProjectFileHandler and DblBundleTextCorpus now open entries through it. Before, they used archive.read or archive.open with no limit.
• The C# Write, SetLength and seek handling is not ported. The Python stream is read-only, since only reads are used.

Tests

tests/utils/test_zip_entry_utils.py ports the four C# tests and adds an empty-entry case.

./local_check.sh --agent-strict: 868 passed, 3 skipped.

Closes #358

🤖 Generated with Claude Code

---

This change is 

[Enkidu93]

@claude, the CI is failing with:

poetry run pyright
/Users/runner/work/machine.py/machine.py/machine/corpora/dbl_bundle_text_corpus.py
  /Users/runner/work/machine.py/machine.py/machine/corpora/dbl_bundle_text_corpus.py:34:39 - error: Argument of type "BoundedStream" cannot be assigned to parameter "buffer" of type "_BufferT_co@TextIOWrapper" in function "__init__"
    Type "BoundedStream" is not assignable to type "_WrappedBuffer"
      "BoundedStream" is incompatible with protocol "_WrappedBuffer"
        "name" is not present
          "_WrappedBuffer" is not assignable to "BoundedStream" (reportArgumentType)
/Users/runner/work/machine.py/machine.py/machine/utils/zip_entry_utils.py
  /Users/runner/work/machine.py/machine.py/machine/utils/zip_entry_utils.py:30:26 - error: Argument of type "IO[bytes]" cannot be assigned to parameter "inner_stream" of type "BinaryIO" in function "__init__"
    "IO[bytes]" is not assignable to "BinaryIO" (reportArgumentType)
2 errors, 0 warnings, 0 informations

I'm not sure how local_check could have passed for you. Please address.

[claude]

Claude finished @Enkidu93's task in 4m 36s —— View job

---

Pyright errors fixed

I pushed 085c99d to this branch, which fixes both pyright errors. After the change, ./local_check.sh --agent-strict reports pyright with 0 errors and 868 passed, 3 skipped.

• machine/utils/zip_entry_utils.py:30: archive.open returns IO[bytes], not BinaryIO. BoundedStream.__init__ in machine/utils/bounded_stream.py now types inner_stream as IO[bytes].
• machine/corpora/dbl_bundle_text_corpus.py:34: TextIOWrapper needs a buffer that has name, which BoundedStream lacks. I wrapped it as TextIOWrapper(BufferedReader(stream), ...). Closing the wrapper still closes the bounded stream and the zip entry.

black also reformatted that call in dbl_bundle_text_corpus.py.

How the first run passed: I can't say for certain, and I didn't investigate. The pyright version here (v1.1.411) or the platform may differ from CI, which ran on macOS. The 868 passed figure in the PR body is unchanged.

The review findings on corpus-level tests (F1) are not addressed here, since you didn't ask for them.
· branch claude/issue-358-20261007-1926

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.18%. Comparing base (4a1ff2c) to head (ba37fcb).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #406      +/-   ##
==========================================
+ Coverage   92.17%   92.18%   +0.01%     
==========================================
  Files         396      398       +2     
  Lines       25225    25267      +42     
==========================================
+ Hits        23251    23293      +42     
  Misses       1974     1974              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Enkidu93]

@claude Please address the comments on this PR.

@Enkidu93 reviewed 6 files and all commit messages, made 4 comments, and resolved 1 discussion.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on claude[bot]).

[Enkidu93]

@claude Please address the comments on this PR.

[claude]

Claude finished @Enkidu93's task in 4m 2s —— View job

---

Review comments addressed

• zip_paratext_project_file_handler.py:31 (corpus-level tests): left as is, per your comment.
• zip_entry_utils.py:27 (100:1 ratio with no minimum size): left as is, per your comment.
• bounded_stream.py:24 (does the runtime limit ever fire?): checked.

What I found. I built a zip with a deflated 5000-byte entry and patched the central-directory file_size to 100. Then I called open_bounded_stream(..., max_uncompressed_size=100) and read the stream. It raised BadZipFile: Bad CRC-32, not the OSError from BoundedStream. CPython's ZipExtFile never returns more than the declared size. An understated header therefore ends in a CRC failure, and the OSError is unreachable for zip entries. The "covers entries whose headers understate their size" line in the PR description is wrong for Python. In C# the check is needed because DeflateStream has no such trim.

What I changed. I kept BoundedStream, since it is a faithful port and still guards any other inner stream, and its direct test remains. I corrected the open_bounded_stream docstring in 89685bd so it no longer says that reading more than the limit raises OSError. I can't edit the PR description, so please drop or reword that sentence. If you would rather remove BoundedStream and rely on the header check plus the CRC, say so and I'll do it.

./local_check.sh --agent-strict: 868 passed, 3 skipped.
· branch claude/issue-358-20261007-1926

[Enkidu93]

Alright, @claude, it looks like we don't need the bounded stream implementation then. Just check the size from the zip file's header.

[claude]

FYI: F4. The new docstring promises more than CPython delivers. ZipExtFile checks the CRC against the central-directory CRC over the bytes it has yielded, so if a crafted entry sets file_size and CRC to match a truncated prefix, the entry is silently cut short and no BadZipFile is raised. The size bound still holds. Saying "normally fails the CRC check" would be accurate. Unverified: I could not run Python here.

[claude]

F4 still applies at bdce2ab: the docstring now says outright that an understated header "fails the CRC check with BadZipFile", and that claim is the reason BoundedStream was dropped. An entry whose CRC matches the truncated prefix is still cut short without any error. F5 covers the memory side of the same gap.

[claude]

Claude finished @Enkidu93's task in 3m 40s —— View job

---

BoundedStream removed

• Delete machine/utils/bounded_stream.py and its test
• Make open_bounded_stream check only the header size and ratio, then return archive.open(...)
• Revert the BufferedReader wrapper in dbl_bundle_text_corpus.py, which was only needed for BoundedStream
• Run ./local_check.sh --agent-strict

I pushed bdce2ab with these changes. open_bounded_stream keeps its name and signature but now returns the plain IO[bytes] from ZipFile.open. It still raises BadZipFile when the declared size exceeds 100MB or the ratio exceeds 100:1. An entry whose header understates its size fails with a CRC BadZipFile.

./local_check.sh --agent-strict: 867 passed, 3 skipped. The count dropped by one because I removed the BoundedStream test.

Please update the PR description. It still says BoundedStream raises OSError and covers understated headers, and it still lists BoundedStream in the changes.
· branch claude/issue-358-20261007-1926

[claude]

Minor: F5. With BoundedStream gone, a single read of a crafted entry no longer has a memory bound. ZipEntryStreamContainer.open_stream and ZipParatextProjectFileHandler.open call stream.read(), and ZipExtFile.read(-1) calls _read1(MAX_N). That decompresses up to 2 GiB in one call before cutting the output to file_size, so file_size=1000, compress_size=2MB passes both checks. Before bdce2ab, RawIOBase.readall read in 8 KB chunks. Reading in chunks at those two sites (e.g. shutil.copyfileobj) restores the bound. Unverified: I reasoned this from CPython zipfile and could not run it.

[claude]

FYI: F6. A bzip2 or LZMA entry is not bounded by these checks, even if the reads are chunked. For those types, ZipExtFile._read1 calls decompress(data) with no max_length, so one compressed chunk can expand without limit before the output is cut to file_size. This means F5's chunked-read fix covers only deflate. C# ZipArchive cannot read these types, so refusing anything except ZIP_STORED/ZIP_DEFLATED here would match C#. Unverified: I reasoned this from CPython zipfile and could not run Python here.

[claude]

Review summary (round 5)

1. Verdict: approve with fixes.
2. Most important: F5 (Important) is still open. The stream.read() at zip_entry_stream_container.py:25 and zip_paratext_project_file_handler.py:32 can decompress up to 2 GiB in one call from an entry that passes both header checks (machine/utils/zip_entry_utils.py:28). F6 is new: chunked reads do not fix this for bzip2/LZMA entries.
3. Counts: Critical 0, Important 1 open (F5), Low 2 open (F4, F6). F1 accepted, F2 withdrawn, F3 addressed.
4. What I ran: compare bdce2ab...ba37fcb is diverged, because the branch was rebased onto 4a1ff2c, so I reviewed the whole PR again with git diff 4a1ff2c ba37fcb. Its content matches what round 4 reviewed: the header-only open_bounded_stream, the three call sites, and four helper tests. /code-review high 406 returned 9 candidates:
   • One became F6.
   • Five duplicate existing findings: F5 (one-shot decompression), F4 (docstring), F2 (ratio check on tiny entries, and no way to pass limits), F1 (no call-site tests) and F3 (stale PR description).
   • I did not post the other two, since neither is Critical. One is that the limit errors do not name the entry and reuse BadZipFile, which matches C#. The other is helper-test gaps: no boundary tests, no ZipInfo argument, no STORED or compress_size == 0 entry.
5. Not verified: ./local_check.sh --agent-strict and every Python command needed approval here, so I have not confirmed the "867 passed" reported for the pre-rebase head, or anything on ba37fcb. F4, F5 and F6 are reasoned from CPython zipfile and not reproduced.

Surface changed: ZipParatextProjectFileHandler, ZipEntryStreamContainer and DblBundleTextCorpus raise BadZipFile for entries over 100MB or compressed more than 100:1, and callers cannot pass a looser limit. The new module machine.utils.zip_entry_utils has no __all__ change, and no optional dependency changed. Parity with sillsdev/machine is not exact: C# keeps BoundedStream as a runtime limit, and this port relies on ZipExtFile cutting output to file_size instead. F5 and F6 are where that difference shows. The PR description still describes BoundedStream and four ported C# tests, so please update it (F3).

Findings:

• F1 (Important): accepted.
• F2 (Low): withdrawn.
• F3 (Low): addressed in 89685bd. The PR description is still out of date.
• F4 (Low): open, unverified.
• F5 (Important): open, unverified.
• F6 (Low): new, open, unverified.

Reviewed at ba37fcb

[Enkidu93]

@Enkidu93 reviewed 2 files and all commit messages, and made 1 comment.
Reviewable status: 4 of 6 files reviewed, 3 unresolved discussions.