Repository navigation
Cap malloc arenas so extraction jobs fit their memory limit - #288
Merged
Merged
Conversation
Each extraction child runs under RLIMIT_AS, which counts address space rather than memory in use. glibc reserves a malloc arena of up to 64 MB per thread, up to eight per CPU core, so the same job's address space grows with the host's core count. On an eight-core workstation a job that uses about 250 MB reached about 820 MB of address space, and every job failed with MemoryError under the 768 MB default; on Fly's one-CPU worker the ceiling is much lower. Set MALLOC_ARENA_MAX=2 for the spawned children (an operator's own value is kept). The same job then peaks at about 440 MB of address space and completes under the default limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused change correctly addresses spawned-worker address-space overhead and includes appropriate regression coverage.
0 open findings
What changed in this PR
Caps glibc malloc arenas for spawned extraction workers to keep address-space usage within existing limits.
Changes:
- Defaults
MALLOC_ARENA_MAXto2while preserving operator overrides. - Tests default and preconfigured values.
| File | Description |
|---|---|
efile_app/efile/management/commands/process_document_extractions.py |
Configures malloc arenas before spawning extraction jobs. |
efile_app/efile/tests/test_document_extractions.py |
Verifies default and overridden settings. |
🧠 Review effort: Balanced
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every document extraction job failed with
MemoryErrorwhen the worker ran on a multi-core workstation, so uploads never got analyzed locally.Each job runs in a child process under
RLIMIT_AS(DOCUMENT_EXTRACTION_MEMORY_MB, 768 MB by default), which limits address space, not memory in use. glibc reserves a malloc arena of up to 64 MB per thread, up to eight per CPU core, so the same job's address space grows with the host's core count. Measured on an eight-core machine with the sample PDF:MemoryErrorwhile loading the form crosswalkMALLOC_ARENA_MAX=2The worker now sets
MALLOC_ARENA_MAX=2before spawning children, keeping an operator's own value if one is set. The limit itself is unchanged; Fly's worker has one CPU, so its arena ceiling was already low.Validation: worker tests (including that a spawned child sees the setting, and that an existing value is kept), full Python suite in the pre-push hook, and the real
process_document_extractions --oncecompleting a queued job under the default limit.🤖 Generated with Claude Code