[datasets] fix zero-dimension synthetic image for blank glyphs (#2016) - #2123
Open
ousamabenyounes wants to merge 1 commit into
Open
[datasets] fix zero-dimension synthetic image for blank glyphs (#2016)#2123ousamabenyounes wants to merge 1 commit into
ousamabenyounes wants to merge 1 commit into
Conversation
…e#2016) font.getbbox returns a zero-height (or zero-width) box for whitespace and glyphs a font renders blank. synthesize_text_img then built a 0-dimension image, which crashes the recognition training resize with 'Input and output sizes should be greater than 0'. Clamp each dimension to a minimum of 1 pixel.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2123 +/- ##
==========================================
- Coverage 97.00% 96.99% -0.01%
==========================================
Files 169 169
Lines 9611 9613 +2
==========================================
+ Hits 9323 9324 +1
- Misses 288 289 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
felixdittrich92
requested changes
Aug 17, 2026
felixdittrich92
left a comment
Collaborator
There was a problem hiding this comment.
Hi @ousamabenyounes 👋,
Thanks for the PR! However, this is a false positive.
- Whitespace characters should not be included in the vocabulary
- Instead of silently overriding the character when there is no font that can render it, we should explicitly raise an error. In this case, it’s a user error that should be fixed by providing font(s) capable of rendering all characters
Maybe a pre-check would be useful to ensure that all characters in the vocabulary can be rendered correctly by the provided font(s)?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2016
Recognition training with the synthetic
WordGeneratorintermittently crashes in aDataLoader worker with:
Root cause:
synthesize_text_imgsizes the canvas fromfont.getbbox(text). Forwhitespace — and for any glyph a given font renders blank (a common case when training
with many custom
--fontfiles over a large vocab) —getbboxreturns a zero-height(or zero-width) box.
h = round(1.3 * text_h)then becomes0, so the function returnsa 0-dimension image. When that image reaches the recognition
Resize,F.resize/interpolaterejects the 0-sized input and the whole run dies.Fix: clamp each synthesized dimension to a minimum of 1 pixel (
MIN_SYNTH_IMG_DIM).A blank glyph has no ink, so any positive size yields an equivalent (background-only)
image while keeping the tensor valid. Normal text is unaffected —
1.3*text_hand1.1*text_ware already>= 1, so the clamp is a no-op there.Reproduces with (unmodified
main)Test verification (RED -> GREEN)
New regression test
tests/common/test_datasets.py::test_synthesize_text_img_never_degenerateasserts the synthesized image is always at least 1x1.
RED — on unmodified
main(fix reverted, test present):GREEN — with the fix:
Full local validation suite (affected areas)
pytest tests/common/test_datasets.py— 4 passed (2 pre-existing + 2 new)pytest tests/pytorch/test_datasets_pt.py::test_charactergenerator ::test_wordgenerator— 2 passedruff format --check,ruff check,mypy doctr/datasets/generator/base.py— cleanmainbefore the change: 2 passed in the same suite)Files changed
doctr/datasets/generator/base.pyMIN_SYNTH_IMG_DIM(1 px)tests/common/test_datasets.py