Conversation
…oving string dispatch
…ps with direct methods on Id accepting T: Into<GID>
…ing NodeNameFilterBuilder/NodeTypeFilterBuilder
…ing NodeNameFilterBuilder/NodeTypeFilterBuilder
…rs (windowed + layered)
…_expr # Conflicts: # raphtory/src/db/graph/views/filter/model/node_expr.rs
…e full filter-building pipeline on each type
…ilder → TemporalProp/Quantified/Aggregated
…ilterOps from prelude
… to restore dot-syntax property filter ops
…e .property().temporal() instead
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.
| Benchmark suite | Current: 7b2239a | Previous: 9823ef7 | Ratio |
|---|---|---|---|
lotr_graph/num_edges |
5 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
lotr_graph/num_nodes |
5 ns/iter (± 0) |
1 ns/iter (± 0) |
5 |
lotr_graph/graph_latest |
3 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
lotr_graph_materialise/materialize |
7376762 ns/iter (± 259838) |
1564816 ns/iter (± 35303) |
4.71 |
lotr_graph_window_100/num_nodes |
15 ns/iter (± 1) |
5 ns/iter (± 0) |
3 |
lotr_graph_window_100_materialise/materialize |
7951537 ns/iter (± 339616) |
1669150 ns/iter (± 10700) |
4.76 |
lotr_graph_window_10_materialise/materialize |
3078255 ns/iter (± 168682) |
971980 ns/iter (± 4278) |
3.17 |
lotr_graph_subgraph_10pc_materialise/materialize |
2055639 ns/iter (± 99729) |
334634 ns/iter (± 1287) |
6.14 |
lotr_graph_subgraph_10pc_windowed_materialise/materialize |
1150672 ns/iter (± 4029) |
230399 ns/iter (± 2617) |
4.99 |
lotr_graph_window_50_layered/has_node_existing |
335 ns/iter (± 20) |
129 ns/iter (± 12) |
2.60 |
lotr_graph_window_50_layered_materialise/materialize |
29673365 ns/iter (± 1919660) |
3488825 ns/iter (± 24948) |
8.51 |
lotr_graph_persistent_window_50_layered/num_edges_temporal |
527496 ns/iter (± 9622) |
192686 ns/iter (± 1569) |
2.74 |
lotr_graph_persistent_window_50_layered/has_node_existing |
414 ns/iter (± 396) |
174 ns/iter (± 83) |
2.38 |
lotr_graph_persistent_window_50_layered/iterate_exploded_edges |
3478723 ns/iter (± 292828) |
1659940 ns/iter (± 19402) |
2.10 |
lotr_graph_persistent_window_50_layered_materialise/materialize |
52767497 ns/iter (± 3951732) |
5298035 ns/iter (± 147912) |
9.96 |
This comment was automatically generated by workflow using github-action-benchmark.
…tifier ops with list-aware aggregates
- edge expressions build their ops from the pre-transformed view, matching the node side and the CreateFilter contract; the orphaned boolean/set ops are removed - constant comparison operands are validated by value castability, matching the runtime coercion; type-level compatibility remains for expression-vs-expression comparisons - two tests pinning pre-db_v4 semantics (windowed is_self_loop, agg overflow) yield to their filter_tests twins - the node_expr unit tests compile against the current API - unused imports left behind by the bridge are removed
Coercing a concrete wrapper (Latest<T>, SnapshotAt<T>, ...) into Arc<dyn DynEdgeFilterFactory> materialises a vtable whose own wrap methods coerce deeper wrappers, so monomorphisation never terminates; the compiler gave up 40 windows deep (E0275) once anything compiled these bindings, which is exactly what cargo test --all-features does. The four wrap methods now erase self before wrapping (the same trick dyn_window already used), closing the set over wrapper-of-erased types. The erased factory picks up the traits those wrappers need through the existing Arc blankets: DynEntityExpr and DynCreateView become supertraits, EdgeFilterFactory is implemented directly, and the blanket's unused EdgeViewFilterOps bound is dropped.
Section markers drop their provenance notes; the FilterValue alias comment explains the two types' roles and that both leave with the composite path.
First execution of the python bindings surfaced defects invisible to
compilation:
- the module registered classes while the entry points are instance
methods; the module attributes are now ready-made root instances
- the erased wrap methods dispatched back into themselves through the
vtable (an unconditional runtime loop); the window family constructs
the wrapper over the erased factory directly with the same bounds
clamping as ViewWrapOps, and the manual Arc impls dispatch through
as_ref() so the blanket on Arc cannot self-select
- comparison and string operators accept plain python values as well as
expressions (extracted as Prop constants)
- temporal() is exposed on PropertyExpr through the existing DynTemporal
- predicates return FilterExpr: comparisons wrap their CreateFilter
impls, factory predicates route through Dyn{Node,Edge}ViewFilterOps
instead of Scoped ops, and NodeWindow carries the NodeViewFilterOps
bound
- the erased expression type forwards prop_type and nullable, so set
coercion and build-time validation see real types instead of Empty
The filter test files use one construction idiom (attribute style,
matching the rust docs).
any() and all() on an expression now return the qualifier expressions
themselves instead of terminating in an implicit eq(true), so the
comparison written after them applies per element and the qualifiers
collapse the results (innermost list level first):
NodeFilter.property("p").temporal().any().eq(7)
lowers to the same op chain as the trailing form. The pieces:
- CreateOp::create_qualified_{node,edge}_op separates leading
qualifiers from the value expression (forwarded through the dyn
layer); AnyExpr/AllExpr strip themselves, aggregates pass qualifiers
through and apply per element
- the comparison, string and set filters lower a qualified lhs to the
list-aware elementwise ops wrapped in the qualifier collapse chain,
validating against the element type
- a bare qualifier used as a filter keeps its old meaning (elementwise
eq(true) then collapse) through one marker-dispatching CreateFilter
Also: the node_type expression yields the storage's default type key
for untyped nodes, matching the composite type mask (negations now
include untyped nodes); node metadata lookups raise the metadata error;
endpoint and exploded-edge properties expose temporal(). Three tests
that had pinned the untyped-node divergence follow their filter_tests
twins.
The expression ops now report the types they actually produce, which turns a family of silent no-matches into build-time errors and makes the coercion rules uniform: - id, name, node_type and degree lowerings carry their static types (ids take the graph's id dtype); temporal ops report List of the property dtype; aggregates report their output structurally (the innermost list level collapses, outer levels survive for pending qualifiers); Arc<dyn EdgeOp> and the endpoint bridge forward prop_type; edge metadata stays untyped since its runtime shape depends on the edge's layers - multi-qualifier chains collapse in the written order: leading qualifiers read outermost-first, trailing ones innermost-first - qualified is_some/is_none apply elementwise through the list-aware unary ops; aggregates validate against the qualified element type; qualifiers over a known scalar are errors - ordering operators are rejected for map and list properties as well as booleans; string operators validate their constant operand; the edge set path coerces values like the node path; map constants compare structurally (partial schemas) but non-map constants against map properties are errors - u64/i64 sums that overflow promote to Decimal and still compare - python comparisons raise TypeError at the call site when the expression's type is statically known and the value cannot coerce; string operands are checked eagerly The python filter tests, which had never been executable, are adjudicated against these semantics: provably wrong hand-written expectations follow the rust parity suite, error-message assertions use the expression-path messages, and mistyped-operand cases build their filters inside the raise assertions.
The wire schema is unchanged; the DynFilter conversion now builds expression filters instead of the composite enums. Field conditions reuse the existing where-translations and dispatch onto typed field expressions, property condition trees walk onto the dyn expression chain (wrappers extend the expression in written order, combinators branch, leaves become predicates), views wrap the lowered filter, and edge endpoints evaluate a full lowered node filter per edge through the new EdgeEndpointNodeFilter adapter. The composite conversions remain for the python remote client, which still serialises filters through the tree representation.
Python filter objects now carry both the compiled expression and the GraphQL wire tree, recorded at construction. The remote client reads the carried tree; expressions with no wire form (an expression on the rhs of a comparison) are rejected at the remote boundary with a clear error. The client error envelope now surfaces each GraphQL error's message text instead of the raw JSON object, so quotes inside diagnostics are no longer escaped. Test expectations updated for expression-engine wording and semantics: bool constants coerce to ints in comparisons, mistyped is_in values are rejected eagerly, and nodes.filter() defers uniformly for every filter kind — narrowing membership is spelled nodes[...] / select (reverses the #2690 special case).
Composite filters no longer execute: the CreateFilter impls on the composite enums, the GraphQL-to-composite conversions, and the TryAsCompositeFilter export trait are gone. The enums survive as plain data — the transportable wire descriptor inside FilterTree — and the composite-to-GraphQL direction stays for client transport. Server resolvers pass wire filters straight to the expression lowering. Recording view chains at construction had inverted non-commuting chains on the wire (window then latest arrived as latest then window). Both wire builders now nest views in application order, with parity tests covering both orders of window+latest at graph, node and edge scope.
The builder factories, ops traits, field builders, validation module and leaf executors are gone, along with their prelude exports. The rust test suite runs on the expression API — the builder half of the A/B test pair is deleted, its two unique tests moved into the expression suite. Aggregations over latest list-valued properties now work on the typed rust surface, matching what python already exposed. The exploded-edge expression filter now passes deletion events through, as the removed executor did; persistent-graph histories pin the behaviour.
The upstream rewrite of the deleted builder ops stays deleted; the OrOp and AndOp const-value fixes and their tests are kept, and the NodeOp trait's const_value_in_domain now receives storage. Upstream's renamed membership test matches the uniform filter semantics on this branch, and its new collection-filter suites pass against the expression engine unchanged.
Node-id equality and set membership resolve their evaluation domain through the storage index instead of visiting every node, carrying the narrowing #2754 added to the removed builder op onto expressions. A constant whose type does not match the graph's id type keeps the full domain rather than guessing, and every other predicate is unchanged. A sum widens at runtime but declared its element type, so a constant beyond that element's range was rejected before the sum ran — no comparison above 255 was expressible on a list of u8. Sum now declares what the evaluator produces; reductions returning an element keep the element type. The generated stubs and the docstrings pointing at them still named builder classes that no longer exist.
A constant was accepted whenever it could be cast into the property's type, so a numeric string or a bool compared against a number and a string property compared against a decimal. Constants now follow the rule the property filters already stated: numerics compare with one another, everything else only with its own type. Ordering returns to booleans, which are ordered. Set membership drops members of a type the property can never equal rather than refusing the filter — being absent from a set is an answer, where a comparison has no answer to give. Degree keeps casting its constant, so a count still accepts the string form a wire query sends. That cast was previously skipped, leaving the comparison to fall through to a variant ordering rather than comparing counts at all.
The upstream rewrite of the deleted builder ops stays deleted; its indexed node-type op is kept and now reaches the storage trait it needs. The prelude carries both the expression factories and the new storage arguments, and the node-type subgraph tests follow the assertion helper to its new home.
filter.Node, Edge, ExplodedEdge and Graph are classes of static methods again, with their view methods returning the view-scoped NodeFilter, EdgeFilter, ExplodedEdgeFilter and GraphFilter; Expr and PropertyExpr are registered so annotations and isinstance work. The stub generator no longer lets a submodule's star import shadow the classes the submodule defines, every filter method documents its arguments and return type, and the remote client's refusal of expression-rhs filters is pinned by a test. Also: wire field names are shared constants, the set-cast helper is fallible, one GID conversion serves both the wire and domain narrowing, the EntityExprBuilder marker is renamed PredicateLhs, and the explicit-false not()/is_true()/is_false() aliases are gone in favour of eq(); not() is the set complement everywhere, as in python.
FilterExpr prints the FilterTree it carries in the notation the composite filters already use, or says it is local-only when there is none; layer selections print as names rather than debug output.
Edge.window(0, 5).src().property("p") read the source node's property
outside the window: the endpoint was built from an unscoped node filter,
so the view chain that reached src()/dst() was dropped locally while the
server applied it. The endpoint now carries the same views onto the node
it reads from, and a parity test pins both sides agreeing.
A compiled comparison cast its constant to the property's type and, when the cast did not fit, fell back to an ordering that only knows same-type pairs, so a u64 above i64::MAX against an i64 property matched nothing. Prop::compare and Prop::equals are what the property filters used: exact through i128 and Decimal, f64 when a float is involved, structural for everything else.
The expression tree is the single form of a filter: Rust compiles it, Python holds it, GraphQL accepts it as FilterExpr, the remote client sends it and permission grants store it. The per-entity GraphQL grammar and its lowering are removed; view collections take one filter entry; a view leg of an and applies first and composes only with and; numeric comparisons compare by value; endpoint reads keep the edge's views; the filtering user guide is rewritten.
Takes the hash-consistent property comparison (#2777) and the iterator locking refactor (#2773). Every conflict was in code this branch removed (the composite filters, the degree and property filter ops, the reverse wire conversion) or in generated stubs; the branch's side is kept, the filter set helpers use the new HashableProp, and the stubs are regenerated.
|
|
||
| from typing import * | ||
| from raphtory import * | ||
| from raphtory import ( |
Collaborator
There was a problem hiding this comment.
do we need to spell this out, it is going to be hard to maintain
| with pytest.raises( | ||
| Exception, | ||
| match='Invalid filter: Filter value type does not match node ID type. Expected Str but got "U64"', | ||
| match=r"Invalid filter: value I64\(3\) of type I64 cannot be coerced to Str", |
Collaborator
There was a problem hiding this comment.
We shouldn't have escape characters in the error message!
| filter_expr = filter.Node.property("p20").temporal().all().starts_with("Gold") | ||
| result_ids = sorted(graph.filter(filter_expr).nodes.id) | ||
| expected_ids = ["3", "4"] | ||
| expected_ids = ["1", "3", "4"] |
Collaborator
There was a problem hiding this comment.
why has this changed?
| //! ```rust,ignore | ||
| //! NodeFilter.id() // Id — e.g. .eq(GID::Str("v1".into())) | ||
| //! NodeFilter.name() // Name — e.g. .eq("Alice") | ||
| //! NodeFilter.node_type() // Type — e.g. .is_some::<Prop>() |
Collaborator
There was a problem hiding this comment.
what's up with the turbo fish?
| impl<E: EntityExpr + Clone + Send + Sync + 'static> PredicateLhs for TemporalPropExpr<E> {} | ||
|
|
||
| impl<E: EntityExpr + Clone + Send + Sync + 'static> EntityAggOps for TemporalPropExpr<E> { | ||
| fn sum(self) -> SumExpr<Self> { |
Collaborator
There was a problem hiding this comment.
These seem like they can be default-implemented?
| if let Some(Some(value)) = right.const_value() { | ||
| if let Some(gid) = gid_for_id_lookup(id_type, &value) { | ||
| return Ok(Arc::new(IdDomainNodeOp { | ||
| gids: Arc::from([gid]), |
Collaborator
There was a problem hiding this comment.
node GID-> VID mapping can happen here, no need to defer that to the call to domain
ljeub-pometry
requested changes
Sep 22, 2026
| type Output = Option<Prop>; | ||
|
|
||
| fn apply(&self, _storage: &GraphStorage, node: VID) -> Option<Prop> { | ||
| self.graph.node(node)?.properties().get_by_id(self.prop_id) |
Collaborator
There was a problem hiding this comment.
graph.node() triggers an unnecessary clone of the graph (unfortunate consequence of the current api signature), (&&self.graph).node() probably does the trick (make sure the returned type is NodeView<&G>).
We should refactor the apis to return reference by default but that is out-of-scope of this PR.
| type Output = Option<Prop>; | ||
|
|
||
| fn apply(&self, _storage: &GraphStorage, node: VID) -> Option<Prop> { | ||
| self.graph.node(node)?.metadata().get_by_id(self.prop_id) |
Collaborator
There was a problem hiding this comment.
make sure this doesn't clone the graph arc.
| // ───────────────────────────────────────────────────────────────────────────── | ||
|
|
||
| #[derive(Clone)] | ||
| pub(crate) struct WithPropType<T> { |
Collaborator
There was a problem hiding this comment.
You can trivially implement the type correctly for all the operations that are currently wrapped by this. This isn't needed.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type-safe filter expression API
What this PR does
A filter is one thing everywhere: an expression tree. Rust compiles it, Python
holds it, GraphQL accepts it as
FilterExpr, the remote client sends it, andpermission grants store it. The old composite and builder paths are deleted,
and so is the per-entity GraphQL grammar (
node: { property: { name, where } },window: { …, expr }).The layers
filter/model/tree.rsdefinesExpr(aconst, areadof afield/degree/property/metadata, or
temporal/sum/any/… over one) andFilterExpr(comparisons with anExpron both sides, string ops,presence, membership, structural tests, views,
and/or/not). Itcompiles onto the erased factories in
dyn_factory.rs; the typedexpression API (
EntityExprFilterOps,EntityAggOps, view wrappers) iswhat it compiles to. The builder DSL is gone and the Rust suite runs on
expressions.
extends the tree, running compiles it,
repr()prints it. Comparisons,membership and presence tests are checked where they are written when the
left side's type is known statically.
FilterExpr,Expr,Read,Scope,ViewOp,Target,Cmp,Membership). Everyfilter(expr:)andselect(expr:)takes it; the view collections'nodeFilter/edgeFilterentries collapse into one
filterentry. Policy placeholders ({"var"},{"claim"}) sit in theconstandvaluesslots.FilterExpr!variable. The onefilter that cannot travel,
by_state_column, raisesValueErrorat theclient.
Breaking changes
be rewritten as trees; nothing converts on load. The matching pometry-storage
change is Pometry/pometry-storage#390: the store keeps trees, rejects the old
grammar with a message that says what to rewrite, and its ABAC scanner reads
placeholders out of the tree.
raphtory-graphql/resources/index.html) still issuesthe old grammar, so the UI test jobs are red until the UI is rebuilt against
schema.graphql. The change on the UI side is a handful of call sites andthe playground example strings.
nodes.filter()defers uniformly for every filter kind (as Node ID and OR filter bug #2754 didupstream); membership narrowing is
nodes[...],select, orgraph.filter.Semantics decided
andapplies first and the other legs run inside it,the way
graph.window(..).filter(expr)does; nestedands flatten. A viewunder
orornotis refused when written (Python) and when compiled(server). Before this, a view inside any composition was silently dropped:
window & predreturnedpredalone.db_v4's family rule; numeric comparisons compare byvalue (a constant wider than the property's type no longer matches nothing).
Bool constants are ordered.
is_indrops members outside the family.Edge.window(0, 5).src().property("x")reads inside the window.
and/or/vieware refused;notis a complement.statically, at apply time otherwise, with the same error locally and remotely.
Follow-ups folded in
scanning (the Node ID and OR filter bug #2754 narrowing, carried onto expressions).
sumdeclares the type it actually produces, so comparisons above theelement's range are expressible.
db_v4's type surface (filter.Node,Edge,ExplodedEdge,Graphas classes of static methods); the regenerated stubs are fully typed,and the stub generator no longer lets a star-import shadow a submodule's
classes.
that run.
Known engine bug found on the way (not fixed here)
graph.filter(node_expr).filter(<edge filter containing a not>)empties thegraph, and
exclude_nodes(..).filter(~node_expr)loses every edge: the chainedNotFilteredGraphnegates the node filtering it inherits. The tree'sandgives the right answer. The RBAC equivalence harness works around it with a
comment; the issue write-up has the repro and a suggested fix.
Verification
raphtory+raphtory-graphql+raphtory-teststest_filters630 scenariostest_base_install(incl. GraphQL e2e and local/remote parity)raphtory-auth138 tests incl. the restriction-equivalence property tests; RBAC Python 345 passed · 1 xfailed