Skip to content

fix: pin plugin dependencies broken by upstream releases - #427

Merged
KartikP merged 4 commits into
mainfrom
kp/fix-plugin-dependency-drift
Oct 15, 2026
Merged

KartikP merged 4 commits into
mainfrom
kp/fix-plugin-dependency-drift

Conversation

@KartikP

@KartikP KartikP commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Three plugins break against newly released third-party packages. All three failures showed up in the CI for #420. That PR edits model_helpers, which makes CI test every language plugin, and none of the failures involve its diff.

Failures and fixes

  • wikitext data: the plugin requires datasets==2.*, which does not cap pyarrow.
    • pyarrow 26.0.0 (released 2026-10-09) raises ImportError: pyarrow requires NumPy 2.0 or newer against our numpy<2 pin.
    • This fails tests/test_model_helpers/test_huggingface.py::TestNextWord::test_0_dimensional_tensor (Non-Plugins) on any PR that runs it.
    • Fix: pin pyarrow<26.
  • earley_parser: NLTK 3.10 refuses to read corpus directories that are not listed in nltk.data.path.
    • Loading the bundled treebank fails with PermissionError: Security Violation [CorpusReader.__init__]: Unauthorized path .../earley_parser/treebank.
    • Fix: add the treebank directory to nltk.data.path before reading it.
  • lm1b: an unpinned tensorflow install upgrades NumPy to 2.x.
    • The plugin's xarray import then fails at test collection with AttributeError: np.unicode_ was removed in the NumPy 2.0 release.
    • Fix: add numpy<2.

Verification

  • pyarrow==26.0.0 with numpy==1.26.4 reproduces the ImportError. pyarrow==25.0.1 with the same NumPy imports cleanly.
  • nltk==3.10.3 reproduces the PermissionError on the bundled treebank. With the path added, all 11 trees in sample_treebank parse.
  • pip install --dry-run for linux x86_64 / Python 3.11:
    • tensorflow alone resolves to numpy 2.4.6, which reproduces the lm1b failure.
    • tensorflow with numpy<2 resolves to tensorflow 2.20.0 and numpy 1.26.4.
    • datasets==2.* with pyarrow<26 and numpy<2 resolves to datasets 2.21.0, pyarrow 25.0.1 and numpy 1.26.4.
  • The model tests themselves have not been run locally. This PR touches all three plugins, so CI runs their tests.

KartikP and others added 4 commits October 15, 2026 09:20
pyarrow 26.0.0 (2026-10-09) requires NumPy 2 at import time, but core and
language pin numpy<2, so loading wikitext-2 fails with an ImportError.
NLTK 3.10 refuses to read corpora outside nltk.data.path, so
BracketParseCorpusReader raises PermissionError on the bundled treebank.
An unconstrained tensorflow install upgrades numpy to 2.x, which breaks
the plugin's xarray import (np.unicode_ was removed in NumPy 2).
@mike-ferguson mike-ferguson added the submission_prepared Attached to a PR is metadata and layer mapping is successful. label Oct 15, 2026
@KartikP
KartikP merged commit 3b4fde5 into main Oct 15, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

submission_prepared Attached to a PR is metadata and layer mapping is successful.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants