Skip to content

perf: reduce walker setup and add bounded graph read primitives - #61

Merged
eldonm merged 1 commit into
mainfrom
fix/walker-construction-performance
Nov 2, 2026
Merged

eldonm merged 1 commit into
mainfrom
fix/walker-construction-performance

Conversation

@eldonm

@eldonm eldonm commented Oct 31, 2026

Copy link
Copy Markdown
Member

Problem and change

Walker construction repeatedly introspected Pydantic private factories and instance descriptors, while large graph-backed reads lacked an adapter-level way to retrieve selected values or verify bounded one-hop binding facts without hydrating every entity.

  • Discover decorated Walker methods from declared class functions, preserving overrides and per-instance state without evaluating arbitrary descriptors. Avoid repeated builtin-factory signature parsing.
  • Add public selected-value reads with a portable default, native PostgreSQL projections, missing/null distinction and full filter/order semantics.
  • Add immutable LiteralBindingFact and GraphBindingSummary contracts with optional bounded PostgreSQL aggregation. Unsupported adapters return None; capped results remain explicitly incomplete. These report storage facts, never authorization grants.
  • Preserve transaction connection boundaries, cache bypass for partial records and upstream observability.
  • Correct compound identity indexes to use root id/entity paths and document the additive public APIs.

Validation

  • Full framework suite with coverage: 2,636 passed, 221 optional-service skips.
  • Final framework Core/database regression suite: 1,306 passed, 100 optional-adapter skips, including actual PostgreSQL projection, malformed-binding and transaction tests.
  • Pinned pre-commit formatting, lint, typing and secret checks pass. The final checks corrected capability-property typing and made transaction projection explicitly use its own find method.
  • Downstream integration against a reviewed local wheel passed exact-record and authority checks, and all six 50,000-entry PostgreSQL regression budgets (79–424 ms p95). This is local protected-owner evidence, not hosted or mixed-role capacity qualification.

Review and release boundary

Companion Core PR: V75inc/integral-core#197

No package version change or publication is included. Core retains its current published dependency and compatible fallback; adopting these native optimizations in a release requires publishing this framework change and separately updating Core's pin. Keep both PRs open for review; do not merge during this handoff.

@github-actions

Copy link
Copy Markdown

Benchmark comparison

Threshold: ±25% (informational, does not block merge)

benchmark baseline (s) current (s) delta status
tests/benchmarks/test_deferred_save_benchmarks.py::test_bench_deferred_save_batched_100 0.027652 0.010997 -60.2% IMPROVED (-60.2%)
tests/benchmarks/test_deferred_save_benchmarks.py::test_bench_immediate_save_100 0.027265 0.010770 -60.5% IMPROVED (-60.5%)
tests/benchmarks/test_graph_projection_benchmarks.py::test_warm_projection_capture[hierarchy] 0.004792 0.002051 -57.2% IMPROVED (-57.2%)
tests/benchmarks/test_graph_projection_benchmarks.py::test_warm_projection_capture[ontology] 0.004515 0.002040 -54.8% IMPROVED (-54.8%)
tests/benchmarks/test_graph_projection_benchmarks.py::test_warm_projection_capture[relationships] 0.004537 0.001998 -56.0% IMPROVED (-56.0%)
tests/benchmarks/test_graph_projection_benchmarks.py::test_warm_rich_projection_capture[hierarchy] 0.003863 0.001744 -54.9% IMPROVED (-54.9%)
tests/benchmarks/test_graph_projection_benchmarks.py::test_warm_rich_projection_capture[ontology] 0.003808 0.001708 -55.1% IMPROVED (-55.1%)
tests/benchmarks/test_graph_projection_benchmarks.py::test_warm_rich_projection_capture[relationships] 0.004045 0.001724 -57.4% IMPROVED (-57.4%)
tests/benchmarks/test_jsondb_benchmarks.py::test_bench_jsondb_batched_saves_500 0.503749 0.375118 -25.5% IMPROVED (-25.5%)
tests/benchmarks/test_jsondb_benchmarks.py::test_bench_jsondb_count_empty_query 0.002080 0.001109 -46.7% IMPROVED (-46.7%)
tests/benchmarks/test_jsondb_benchmarks.py::test_bench_jsondb_count_filtered 0.099028 0.051101 -48.4% IMPROVED (-48.4%)
tests/benchmarks/test_jsondb_benchmarks.py::test_bench_jsondb_find_filtered 0.102074 0.050745 -50.3% IMPROVED (-50.3%)
tests/benchmarks/test_jsondb_benchmarks.py::test_bench_jsondb_save_throughput 0.001701 0.001162 -31.7% IMPROVED (-31.7%)
tests/benchmarks/test_postgres_benchmarks.py::test_bench_postgres_find_many_bulk 0.001119 0.000491 -56.1% IMPROVED (-56.1%)
tests/benchmarks/test_postgres_benchmarks.py::test_bench_postgres_traverse_depth 0.000658 0.000276 -58.0% IMPROVED (-58.0%)
tests/benchmarks/test_sqlite_benchmarks.py::test_bench_sqlite_count_empty 0.000260 0.000113 -56.7% IMPROVED (-56.7%)
tests/benchmarks/test_sqlite_benchmarks.py::test_bench_sqlite_count_fallback_via_regex 0.030648 0.015038 -50.9% IMPROVED (-50.9%)
tests/benchmarks/test_sqlite_benchmarks.py::test_bench_sqlite_count_pushdown 0.001514 0.000701 -53.7% IMPROVED (-53.7%)
tests/benchmarks/test_sqlite_benchmarks.py::test_bench_sqlite_find_fallback_via_regex 0.030852 0.015282 -50.5% IMPROVED (-50.5%)
tests/benchmarks/test_sqlite_benchmarks.py::test_bench_sqlite_find_pushdown 0.000430 0.000201 -53.3% IMPROVED (-53.3%)
tests/benchmarks/test_sqlite_benchmarks.py::test_bench_sqlite_sort_limit_pushdown 0.002620 0.001302 -50.3% IMPROVED (-50.3%)

@eldonm eldonm self-assigned this Nov 2, 2026
@eldonm
eldonm merged commit 5dba7f0 into main Nov 2, 2026
6 checks passed
eldonm added a commit that referenced this pull request Nov 2, 2026
Main now includes the projection and walker changes from #61. Projection SQL stays shared so pooled and transactional reads use the same statement.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant