Repository navigation
Conversation
There was a problem hiding this comment.
@jimador most of this store can move to Drivine's StatelessGraphObjectManager. Much of the PR rebuilds features Drivine already has.
Why use the object manager
- Neo4j, FalkorDB, Memgraph and Oracle all come for free.
- Drivine owns the engine quirks (async constraints, temporal coercion, no rollback on FalkorDB), so this module doesn't have to.
- Far less code to review and maintain.
Already in Drivine
| This PR | Drivine 0.1.6 |
|---|---|
FalkorDbStorageSchema: create the index, poll for OPERATIONAL, handle FAILED, recover after data repair |
@Unique (composite at class level) plus SchemaCatalog.fromFragments. ConstraintManager already waits for FalkorDB's UNDER CONSTRUCTION → OPERATIONAL/FAILED, drops a failed constraint and reports the rows that violate it |
A version UUID compared on write, with up to 64 retries (readVersion / compareAndSet / mutate) |
@NodeStamp plus update(id, attempts) { … }: reloads and reapplies on StaleObjectException, and returns null if the node was deleted |
pin / unpin / addProvenance / setProvenance |
update { it.withPinned(true) } and so on |
touchAccessed: a read and a write per id |
patchAll: named fields of many nodes in one statement |
delete / clear* using FOREACH/CASE |
delete(id) and deleteAll(class, where) |
findById / findByEntity / findByStatus / findByMinLevel / count |
load plus the typed DSL (eq, isIn, contains, gte, anyOf, not, orderBy, limit) |
The proposition's JSON aggregate, with query fields copied out beside it (createdS/createdN, legacyText, contextOrder), is what forces custom Cypher. As a @NodeFragment (@Unique id, @NodeStamp, @JsonPacked provenance), the table above applies directly.
The three places that needed Cypher are now Drivine issues, 78 to 80 inclusive, and will be there:
- liberation-data/drivine4j#78: a create-only save, so two first saves of one id both keep their evidence (what
MERGE … ON CREATEplus the constraint retry does here) - liberation-data/drivine4j#79: order by an expression, for ranking by effective confidence
- liberation-data/drivine4j#80: a
saveAllover more than one statement is not atomic on FalkorDB, whichrecordProcessedrelies on (hash plus bookmark)
The chunk nodes are keyed on (context, hash) and (context, source), so they need a single @NodeId: a synthetic id, with the composite @Unique still enforcing the pair.
Smaller
- Statements are built with
" " +concatenation; Embabel uses multiline strings.
|
@jimador Drivine 0.1.7 has the three features from liberation-data/drivine4j#78–#80, so the remaining hand-written Cypher can go. It reaches Maven Central within the hour. Create without losing a concurrent first write (#78) val stored = sgom.create(proposition) // null when the id is already there
?: sgom.update<Proposition>(proposition.id) { it.merge(proposition) }Rank by effective confidence in the database (#79) sgom.loadAll<Proposition> {
where { proposition.context isIn scope.contextIdValues }
orderBy {
case(
(proposition.validFrom above asOf) or (proposition.validTo atMost asOf) then 0.0,
otherwise = proposition.confidence *
exp(-proposition.decay * k * floor((epochSeconds(asOf) - proposition.anchorSeconds) / 86400)),
).desc()
proposition.id.asc()
}
limit(limit)
}On FalkorDB, keep the anchor as a numeric seconds property, as the PR already does. Hash and bookmark written together (#80) sgom.saveAll(listOf(ChunkHash(record), Bookmark(record))) // one statement on FalkorDB: all or nothingA batch of fragments only is one |
Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
Signed-off-by: James Dunnam <7660553+jimador@users.noreply.github.com>
028629c to
ef4e7a0
Compare
dabff50 to
c44fcba
Compare
Add durable proposition and chunk-history stores backed by FalkorDB through Drivine. Complete propositions and native query fields update atomically, with guarded revisions that preserve concurrent evidence.
Native structured and context-set reads apply filters, grants, ranking and limits in the database. Fixed identity constraints become operational before writes; initialization can recover a failed constraint after duplicate data is repaired. Incompatible caller transactions are rejected.