WAL snapshot format fix, prepare() completion gate

README.md

@@ -24,7 +24,7 @@ your workload is synchronous and fits `node:sqlite`, use `node:sqlite`**
 — nothing needs installing and nothing needs compiling. This package
 exists for the parts it does not cover.
 
-Verified against `@appthreat/sqlite3` 9.0.1 on **Node v24.18.0 and
+Verified against `@appthreat/sqlite3` 9.0.2 on **Node v24.18.0 and
 v26.7.0**, which expose an identical `node:sqlite` surface and behave
 identically on every point below — so this table holds across the whole
 range this package supports. `node:sqlite` did grow quickly during 24.x
@@ -225,8 +225,13 @@ await using stmt = db2.prepare("SELECT 1"); // finalized the same way
 ```
 
 `each()` stays callback-only — the async iterator is its promise-based
-replacement. `db.prepare()` and `db.backup()` keep their synchronous return
-in every form.
+replacement. `db.backup()` keeps its synchronous return in every form.
+`db.prepare()` returns the statement synchronously in its callback form; the
+no-callback form returns the statement wrapped so that `await db.prepare(sql)`
+resolves only once the prepare has completed and the introspection accessors
+(`columns`, `parameterCount`, `parameterNames`, `readonly`) are populated —
+and yields the statement itself. The wrapper still forwards every statement
+method, so `db.prepare(sql).run(...)` chaining is unchanged.
 
 ## Performance options
 
@@ -421,7 +426,13 @@ A JS function called per row is the wrong tool for bulk filtering —
 fetch and filter in JS (or write the predicate in SQL). A JS collation is
 even sharper: sorting 100k rows costs O(N log N) round trips (~17 s).
 Where they shine is pushing _logic_ into a query — a regexp, a domain
-checksum, a custom aggregate over a bounded group.
+checksum, a custom aggregate over a bounded group. SQLite invokes a
+scalar function per row and needs each result before the next row is
+read, so there is no batched/array form: the per-call cost is
+structural. [docs/performance.md](docs/performance.md#javascript-functions-the-crossover-to-fetch-and-filter)
+works out the crossover — a few thousand candidate rows is where
+fetch-and-filter overtakes a UDF predicate, measured at ~60× on a real
+16k-row scan.
 
 Two deliberate restrictions follow from the threading model:
 
@@ -433,6 +444,17 @@ Two deliberate restrictions follow from the threading model:
   to run entirely (remove it with `removeCollation()` or use the async
   API): a comparison would need the blocked JS thread, and unlike
   functions, a collation callback has no way to report an error.
+  `db.withCollation(name, cmp, fn)` scopes a registration to the awaited
+  body, registering before it runs and removing after — even when the
+  body throws — so the sync methods are only gated for the block:
+
+  ```js
+  const rows = await db.withCollation(
+    "locale",
+    (a, b) => a.localeCompare(b, "de"),
+    () => db.all("SELECT name FROM t ORDER BY name COLLATE locale"),
+  ); // here the collation is removed and getSync() works again
+  ```
 
 Errors: a throwing callback surfaces as a `SQLITE_ERROR` whose message
 names the function, with the original JS error attached as `err.cause`;
@@ -528,10 +550,12 @@ service it). The token form has no such restriction.
 ### Statement and connection introspection
 
 ```js
-const stmt = await db.prepare("SELECT name AS who FROM users WHERE id = ?");
+const stmt = await db.prepare("SELECT name AS who FROM users WHERE id = $id", { $id: 1 });
 stmt.readonly; // true — sqlite3_stmt_readonly
 stmt.parameterCount; // 1
-stmt.parameterNames; // ['?1'] (null entries for positional `?`)
+stmt.parameterNames; // ['$id'] — a fully positional statement (`?`) has
+// no names at all and reports undefined; mixed statements keep null at
+// every positional index so indices stay aligned
 stmt.columns; // [{ name: 'who', declaredType: 'TEXT',
 //    database: 'main', table: 'users', origin: 'name' }]
 stmt.status(sqlite3.STMTSTATUS_FULLSCAN_STEP); // >0: the query scanned
@@ -545,7 +569,9 @@ await db.dbConfig(sqlite3.DBCONFIG_DEFENSIVE, true); // safe db_config switches
 
 The statement accessors serve a snapshot taken when the statement was
 prepared, so reading them never touches the sqlite handle and cannot
-race a running query; fields SQLite reports as absent (an expression
+race a running query; `await db.prepare(sql)` resolves only after that
+snapshot is published, so the accessors are populated at the first read.
+Fields SQLite reports as absent (an expression
 column has no origin, a typeless column no declared type) are omitted
 rather than nulled. Integer modes apply to `changes`/`totalChanges` as
 everywhere else. `tableInfo` runs a `PRAGMA table_info`, so a
@@ -616,12 +642,18 @@ const copy = await sqlite3.deserializeFromBytes(bytes, {
 ```
 
 `serializeToBytes` returns the exact bytes a file copy would contain
-(the FIFO-ordering `db.serialize()` keeps its old meaning). The bytes
-are named deliberately: overloading `serialize()` would be the worst
-API decision available. `deserializeFromBytes` **copies** into
-SQLite-owned memory — handing a JS buffer to SQLite directly is a
-use-after-free waiting to happen — and rejects corrupt input with
-`SQLITE_NOTADB` rather than crashing later.
+(the FIFO-ordering `db.serialize()` keeps its old meaning). The
+snapshot includes every committed transaction — serialization reads
+through the pager, and the pager reads through the WAL — and for a WAL
+database the returned bytes are rewritten to rollback-journal format,
+so the output always round-trips through `deserializeFromBytes` (a
+deserialized copy has no `-wal` file and could not open a WAL-format
+image demanding recovery). The live database's journal mode is
+untouched. The bytes are named deliberately: overloading `serialize()`
+would be the worst API decision available. `deserializeFromBytes`
+**copies** into SQLite-owned memory — handing a JS buffer to SQLite
+directly is a use-after-free waiting to happen — and rejects corrupt
+input with `SQLITE_NOTADB` rather than crashing later.
 
 ## Incremental blob I/O (v9)
 

docs/install.md

@@ -42,7 +42,7 @@ dependent allowlists it. This package declares `"install": "node-gyp-build"`,
 so pnpm prints a notice like:
 
 ```
-[ERR_PNPM_IGNORED_BUILDS] Ignored build scripts: @appthreat/sqlite3@9.0.1
+[ERR_PNPM_IGNORED_BUILDS] Ignored build scripts: @appthreat/sqlite3@9.0.2
 ```
 
 **You can ignore that notice.** Verified empirically (pnpm 11.23.0, macOS,

docs/performance.md

@@ -324,7 +324,76 @@ README quotes both.
   autocommit writes. Removing a listener returns exactly to baseline.
 - **JS functions/aggregates/collations**: ~19 µs per invocation, O(n log
   n) comparisons for a collation sort. Use them for glue, never per row
-  over large scans.
+  over large scans — the next section is the full picture.
+
+## JavaScript functions: the crossover to fetch-and-filter
+
+A scalar UDF registered with `db.function()` pays one blocking
+cross-thread round trip per invocation: the query steps on a libuv
+worker, and each call marshals its arguments to the JS thread, waits for
+your function, and carries the result back — **~19 µs per call** on
+Apple Silicon (19.6 µs measured; ~28 µs in the Linux container), against
+~0.5 µs for the same work as a SQL expression.
+
+That cost is invisible when the function filters a bounded candidate
+set and decisive when it filters a large scan. The crossover is easy to
+estimate: a UDF predicate wins while
+
+```
+rows × 19 µs  <  (rows × per-row read cost) + (your filtering in JS)
+```
+
+— with a ~0.5 µs/row read, **fetch-and-filter overtakes the UDF at a
+few thousand candidate rows**. A real integration (a version-comparison
+predicate over a 16,046-row candidate set) measured the two shapes
+end to end:
+
+| strategy                                            | time    | result             |
+| --------------------------------------------------- | ------- | ------------------ |
+| the predicate as a JS UDF inside SQL                 | 331 ms  | identical hit set  |
+| narrow projection over the candidates, filter in JS  | 5.5 ms  | identical hit set  |
+
+— ~60× apart for identical results. The UDF shape is *correct*, just
+slow, which is what makes it dangerous: nothing fails, the query simply
+costs one thread-park per row.
+
+Rules of thumb:
+
+- **Push logic, not filtering.** A UDF shines when SQL needs your
+  domain logic on a *bounded* set — a regexp over candidate rows, a
+  checksum on a handful of values, a custom aggregate over a grouped
+  result. Marking the function `deterministic` lets SQLite use indexes;
+  `directOnly` (the default) keeps it out of attacker-reachable schema
+  SQL.
+- **Bulk filtering belongs in SQL or after `all()`.** Write the
+  predicate in SQL, or fetch a narrow projection and filter in JS. Both
+  are shown in the table above; filtering in JS after `all()` costs
+  ~1.25 µs/row in the README's 100k-row example, against ~19 µs/row for
+  the per-row UDF.
+- **There is no batched UDF.** SQLite invokes a scalar function per row
+  at step time and needs each result before the next row is read, so
+  calls cannot be coalesced into one array-in/array-out round trip
+  without changing results. The per-call cost is structural, which is
+  why the crossover above is the tool to reason with.
+
+The same arithmetic applies to aggregates (`step` is one round trip per
+row) and collations (O(n log n) round trips for a sort — sorting in JS
+after `all()` is faster for anything but small or one-off sorts).
+
+### Future direction: UDFs on the synchronous fast path
+
+The refusal of JS functions on the sync methods is policy, not the
+structural limit above. On `getSync`/`runSync`/`allSync` the JS thread
+is already the one executing SQL, so a callback could be invoked
+re-entrantly — a direct call with no cross-thread round trip, the way
+`node:sqlite` runs UDFs inline — and unlike collations, a function has
+an error channel (`sqlite3_result_error`), so failures are reportable
+mid-query. The current build refuses instead with an explicit error
+(`src/function.cc`, `SyncRefusalMessage`). Landing inline sync-path
+invocation would make the natural `... AND vers_compare(?, vers)`
+shape genuinely fast, closing the gap the README's comparison table
+attributes to `node:sqlite`. Deliberate follow-up work, out of scope
+for this release.
 
 ## Where this package loses
 

lib/augment.d.ts

@@ -277,19 +277,38 @@ declare module './native.js' {
             callback: (this: Database, err: SqliteError | null) => void,
         ): this;
 
-        /** Prepares a statement; returns it synchronously in every form. */
+        /**
+         * Prepares a statement. The no-callback form returns the statement
+         * wrapped in a thenable: awaiting it settles only once the native
+         * prepare (and any bind) has completed and the introspection
+         * accessors — `columns`, `parameterCount`, `parameterNames`,
+         * `readonly` — are populated, and yields the statement itself.
+         * Before the await, the wrapper forwards every statement member,
+         * so `db.prepare(sql).run(...)` keeps its synchronous surface. A
+         * prepare failure rejects the await.
+         * @since 9.0.2 the await gates on prepare completion.
+         */
+        prepare(sql: string): Statement & Promise<Statement>;
+        /** Prepares a statement; the callback is an error-only errback. */
         prepare(
             sql: string,
-            callback?: (this: Statement, err: SqliteError | null) => void,
+            callback: (this: Statement, err: SqliteError | null) => void,
         ): Statement;
         /** Prepares a statement with one array/named bind object. */
         prepare(
             sql: string,
             params: BindParams,
             callback?: (this: Statement, err: SqliteError | null) => void,
         ): Statement;
-        /** Prepares a statement with variadic bind values. */
-        prepare(sql: string, ...params: [...BindValue[]]): Statement;
+        /**
+         * Prepares a statement with variadic bind values; awaiting the
+         * result gates on the prepare and the bind both completing.
+         * @since 9.0.2 the await gates on prepare + bind completion.
+         */
+        prepare(
+            sql: string,
+            ...params: [...BindValue[]]
+        ): Statement & Promise<Statement>;
         /** Prepares a statement with variadic bind values and a callback. */
         prepare(
             sql: string,
@@ -474,6 +493,27 @@ declare module './native.js' {
          */
         removeCollation(name: string): this;
 
+        /**
+         * Runs `fn` with a JavaScript collation registered, removing it
+         * again afterwards: the blast radius of the registration is the
+         * awaited block, not the connection's lifetime. Inside the block
+         * the synchronous methods refuse to run, as with `collation()`;
+         * an error thrown by `fn` still removes the collation before the
+         * rejection propagates. Interleaved or nested scopes for the same
+         * name are last-wins — use distinct names for concurrent scopes.
+         *
+         * @since 9.0.2
+         * @example
+         * const rows = await db.withCollation('locale',
+         *     (a, b) => a.localeCompare(b, 'de'),
+         *     () => db.all('SELECT name FROM t ORDER BY name COLLATE locale'));
+         */
+        withCollation(
+            name: string,
+            cmp: (a: string, b: string) => number,
+            fn: (db: Database) => unknown,
+        ): Promise<unknown>;
+
         // ---- Hooks, authorizer, progress, WAL, introspection (v9).
 
         /**

lib/native.d.ts

@@ -1284,8 +1284,10 @@ export declare class Statement extends EventEmitter {
 
     /**
      * The bind parameter names in 1-based order: `':a'`, `'@b'`, `'$c'`,
-     * `'?1'`. Positional `?` parameters have no name; their entries are
-     * null so indices stay aligned.
+     * `'?1'`. `undefined` for a fully positional statement (every
+     * parameter a bare `?`): there are no names to report. A mixed
+     * statement keeps one array entry per parameter, with null at every
+     * positional index so indices stay aligned.
      *
      * @since 9.0.0
      */

lib/promises.js

@@ -5,8 +5,10 @@
 // Every wrapped method is dual-mode: when the last argument is a function
 // the call behaves exactly like the callback API (and returns `this`, so
 // chaining keeps working); otherwise it returns a promise. Database#prepare
-// and Database#backup deliberately keep their synchronous return in both
-// forms — see the handoff notes for 03.
+// is not dual-mode: its callback form returns the statement synchronously,
+// and its no-callback form returns the statement wrapped in a thenable
+// that settles once the prepare has landed — see lib/sqlite3.js.
+// Database#backup keeps its synchronous return in both forms.
 
 import { AsyncLocalStorage } from 'node:async_hooks';
 import { Readable, Writable } from 'node:stream';