[hary73]

Summary

Chat History is moving to listing only sessions stamped with metadata["tfg.source"] = "ui" (#1118). Sessions created before that stamp would vanish from history, so this adds a one-time data migration that stamps every existing interactive session.

Related: AGE-2528

Changes

• New migration 20261028_000001_session_tfg_source_backfill (Postgres + SQLite mirror) that sets tfg.source = ui on sessions where:
  • source IS NULL (not a schedule run), and
  • external_id IS NULL (schedule runs are the only sessions with an external id; this also excludes schedule runs created before the source column existed), and
  • the key is not already present (never overwrites an existing value).
• Existing metadata (e.g. is_create_agent) is preserved; updated_at is not touched, so list ordering is unchanged.
• New migration 20261028_000002_session_tfg_source_list_idx (Postgres + SQLite mirror): session (tenant_id, (created_by_subject->>'subject_id'), (metadata->>'tfg.source'), updated_at, session_id) — equality columns, then the keyset sort columns. Serves the Chat History list (created_by_me + metadata[tfg.source]=ui) in order without a sort or filtered rows. Tuned for the UI query; an index for metadata-free listing can be added later if needed.
• The SQLite list query is unchanged, so the SQLite index is for schema consistency only for now.
• Postgres listSessions now adds metadata->>key = value per metadata filter pair next to the existing metadata @> …. Values are strings, so results are unchanged; it lets the planner use per-key expression indexes, which @> alone cannot. Other keys still use session_metadata_gin. The key is inlined as a literal (sql.lit; keys are validated to [A-Za-z0-9._:-]) because expression indexes only match a literal key, not a bound parameter under generic plans.
• down (backfill) is irreversible, matching the other data backfills: backfilled rows are indistinguishable from UI-stamped ones.

Rollout note

This must ship in the same release as (or after) #1118, which switches the stamp/filter key to tfg.source. Sessions created between this migration running and the UI starting to stamp would otherwise be hidden from Chat History.

How was this tested?

Manually, end to end on Postgres 17:

1. Booted the server from main (no backfill) and created: an inline New Chat session, a builder session (is_create_agent: true), a named Try-agent session, an SDK-style session with custom metadata (foo: bar), a session already carrying tfg.source: other, a session with external_id but no source (pre-source schedule run), and two schedule sessions via Run now.
2. Restarted the server on this branch; only 20261028_000001_session_tfg_source_backfill ran.
3. Result:
   • The four interactive sessions gained tfg.source: ui, keeping their existing keys.
   • tfg.source: other was left as is.
   • Both Run-now sessions and the external_id-only session were left untouched.
   • updated_at unchanged on every row.
   • GET /sessions?created_by_me=true&metadata[tfg.source]=ui (the Chat History query) returns exactly the four interactive sessions; the unfiltered list still returns all sessions.

Index check (Postgres 17, 200k seeded sessions, 50 users, ~1/3 tagged ui):

• Before the query change, the Chat History query used no metadata index (walked the tenant newest-first, ~3k rows filtered per page).
• With it, PostgresSessionStore.listSessions (captured via auto_explain) uses Index Scan Backward using session_tfg_source_list_idx, no sort, ~0.2 ms per page; pages are disjoint and all ui.
• Filters on other keys still use session_metadata_gin.
• Postgres session-store contract suite passes, including metadata containment filtering.

---

Note

Medium Risk
One-time, irreversible session metadata backfill must deploy with the Chat History filter change; mis-timing could hide sessions, though scope is limited to list metadata and indexing.

Overview
Adds a one-time data migration (Postgres + SQLite) that sets metadata["tfg.source"] = "ui" on existing interactive sessions (source and external_id both null, key not already set), merging into existing metadata and without bumping updated_at. Schedule-run and already-tagged sessions are skipped; the backfill migration is irreversible.

Introduces session_tfg_source_list_idx on (tenant_id, subject, tfg.source, updated_at, session_id) so Chat History-style lists (created_by_me + metadata[tfg.source]=ui) can keyset-scan in sort order. Postgres listSessions now adds per-key metadata->>'key' = value predicates alongside the existing @> filter (same rows) so the planner can use the new index; other metadata filters still rely on the GIN index.

Ship with the release that filters Chat History on tfg.source, or older UI sessions can disappear until this runs.

^{Reviewed by Cursor Bugbot for commit 85483e3. Bugbot is set up for automated code reviews on this repo. Configure here.}

[changeset-bot]

🦋 Changeset detected

Latest commit: 85483e3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@truefoundry/trueforge	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 805ab35. Configure here.}