Python suite: 50 failures + no CI (collection fixed in #10)

[chitcommit]

Found while investigating a review comment on PR #6. Both breaks are on main and predate that branch.

The Python test suite has never been runnable on a case-sensitive filesystem (i.e. Linux, i.e. any CI).

Break 1 — pypdf2 vs PyPDF2

document_processor.py:9:

import pypdf2

requirements.txt:6 pins pypdf2>=3.0.0. pip normalizes the distribution name to lowercase, but the installed import name is PyPDF2:

$ python -c "import PyPDF2; print('ok')"   → ok
$ python -c "import pypdf2"                → ModuleNotFoundError: No module named 'pypdf2'

This works on macOS (case-insensitive filesystem) and fails everywhere else, which is the classic way this survives unnoticed. document_processor.py:77 also calls pypdf2.PdfReader(...), so the fix is both the import and the call site.

Note PyPDF2 3.0.1 itself warns: "PyPDF2 is deprecated. Please move to the pypdf library instead." Worth migrating to pypdf rather than just correcting the casing.

Break 2 — langchain.text_splitter moved

With break 1 patched locally, collection then fails:

ModuleNotFoundError: No module named 'langchain.text_splitter'

That module moved to the standalone langchain_text_splitters package in langchain 1.x, and requirements.txt currently resolves langchain 1.4.0.

Impact

document_processor → claude_openai_extension → the test module. Any import failure in that chain errors the whole suite at collection:

ERROR tests/integration/test_api_endpoints.py

An uncollectable suite reports as an error, but in a pipeline that only greps for failing assertions it reads as zero failures — a check you appear to have and do not.

Reproduce

python3 -m venv /tmp/v && /tmp/v/bin/pip install -r requirements.txt
/tmp/v/bin/python -m pytest tests/integration/test_api_endpoints.py::TestHealthEndpoint

Also worth deciding

Once collection is fixed, the suite's actual state is unknown — it may have accumulated further rot behind these two errors. Nothing has ever exercised it.

Separately: there is no CI running these tests at all. .github/workflows/ contains only chittyconnect-sync.yml, and main's branch protection has an empty required_status_checks.contexts list, so no check gates merge. Fixing the imports without adding a workflow that runs them would just re-hide this.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PSJp8eaY4ABrZKGkcD1xgM

[chitcommit]

Collection is fixed in PR #10 — 0 tests running → 122 collected, 69 passing. Both breaks were as described above.

One addition not in the original report: running the suite writes evidence/signing_key.pem — a generated private key — plus evidence/public_key.pem and expert_validation/qualified_experts.json into the working tree, and none were gitignored. Covered in #10.

Remaining work, now visible

With collection fixed, the suite's real state is 50 failed / 69 passed / 3 errors. It clusters into five root causes rather than scattered rot:

1. Python 3.12 incompatibility (~21 failures) — the largest. asyncio.coroutine was removed in Python 3.11:

AttributeError: module 'asyncio' has no attribute 'coroutine'
TypeError: 'coroutine' object does not support the asynchronous context manager protocol
TypeError: object AsyncMock can't be used in 'await' expression

The suite was written for Python ≤3.10. Worth deciding deliberately: modernize the async code, or pin the tested interpreter. The former is the real fix.

2. 403 Forbidden (~26 failures) — FastAPI endpoint tests expect 200/503 and get 403. Auth config absent under test; likely API_TOKENS unset. Probably one fixture change.

3. Enum serialization (5) — TypeError: Object of type ExpertQualificationLevel is not JSON serializable.

4. Assertion drift (3) — expects similarity_search('test query'), actual is similarity_search('test query', k=10). The code gained a parameter; the assertion did not.

5. Test isolation — a live bug, not a test bug. config.py:5:

BASE_DIR = Path(__file__).parent.parent

config.py sits at the repo root, so parent.parent resolves to CHITTYAPPS/ — the repo's parent directory. scan_documents(base_path=BASE_DIR) therefore recurses across sibling repositories. The failure output contains real paths from mychitty/services/chittyclaw-extract/node_modules/ and fincabrinca/public/. In production this means the document scanner's default root is outside the project it belongs to.

I would fix (5) first — it is the only one that misbehaves outside the test suite.

CI

Still none. .github/workflows/ has only chittyconnect-sync.yml, and main's branch protection has an empty required_status_checks.contexts, so nothing gates merge. I deliberately did not add a workflow in #10: with 50 failures it would be red on arrival, and a permanently-red required check gets ignored, which is worse than no check. Adding CI should follow the clusters above, in the same change.

🤖 Generated with Claude Code