Skip to content

[4.41] Tracking: sync upstream chat@4.31.0 → chat@4.41.1 (0.4.41 wave) #184

Closed

Description

@patrick-chinchill

Status (2026-10-01)

Wave complete. The work is merged and main is release-prepped as 0.4.41 (UPSTREAM_PARITY = "4.41.1", strict fidelity pin chat@4.41.1 922/922 green). Not yet tagged or published.

Before cutting the release, maintainers need to run these live checks: #262 (Teams SDK 2.1 streaming/edit/delete), #206 (Slack streamed mentions), #208 (stream rotation over 5 min), #215 (agent_view Stop button).

Remaining by choice: #186 (stale branch deletion), #188 (optional contracts), #78 (100 unmapped core tests), #308, #189/#241 (demand-gated).

Why

Upstream vercel/chat is at chat@4.41.1 (2026-09-28, f690af3c). We are at chat@4.31.0 parity (tagged 2026-06-16), shipped as 0.4.31.3 with UPSTREAM_PARITY = "4.31.0". Since then there have been 12 releases (4.32.0 → 4.41.1, including 4.38.1 and 4.41.1) and 254 commits (183 of them touch packages/ source; the rest are docs site, examples and CI). Mapped fidelity files gained about 130 it() cases, and upstream added about 10 new core test files.

The gap is structural. Upstream reworked concurrency (lock heartbeat, per-thread drain isolation), mention semantics (tri-state is_mention), conversation-scoped AI tools, a unified History API, lifecycle events (message updated/deleted, installed/uninstalled, app context, agent sessions) and streaming (placeholder sentinel, Slack native-stream fallback and rotation). It also landed about 30 hardening commits: webhook identity binding, guarded downloads, callback-token replay, tenant-scoped caches and log hygiene. Several of these fix correctness bugs that Slack and Teams streaming users hit today:

  • drop-strategy locks expire during long streamed replies;
  • Slack native streams die after about 5 minutes;
  • incoming Slack code blocks lose their first line;
  • Teams rewrites @ inside email addresses as <at> mentions.

This issue tracks the whole wave. Every child issue is written so an agent can pick it up with no other context.

Strategy

One layered jump to 0.4.41, not staged 0.4.35 / 0.4.39 releases:

  • Dependencies point the wrong way. The keystone contracts land late upstream: tri-state is_mention (4.41), conversation context (4.36), History API (4.39) and turn cancellation (4.39). Adapter changes from 4.32 to 4.35 already depend on them, so an honest "0.4.35" would port contracts that 4.41 then changes.
  • Intermediate states get superseded. Teams mentions moved to the shared scanner in 4.33, and 4.39 then removed the conversion. Telegram MarkdownV2 trimming was reversed. 636da193 prunes and renames tests. Staging means porting code and tests we would then delete.
  • The fidelity pin moves once. The pin can only move to a tag where every mapped test passes. One bump to 4.41.1 gives one authoritative missing-test list, not three rounds.

No separate security patch release. chat-sdk-python has effectively one downstream consumer (chinchill). It pins by git ref, runs single-tenant deployments (so the cross-tenant classes don't apply), and uses Slack and Teams rather than the adapters with the most severe webhook-auth gaps. A 0.4.31.4 would duplicate work that the wave then rewrites. Security fixes ship inside the wave issues and are prioritized first, because most of them have no core blockers (see Start here).

Parity and version flip only at the end. UPSTREAM_PARITY stays "4.31.0" and the version stays 0.4.31.x until #203, which is honest: parity plus extras. Each PR reports its non-strict fidelity count against 4.41.1 in its description (tooling from #185).

Downstream checkpoints (validated by git ref, no release):

  1. Core checkpoint: after [4.41/C1a] Core concurrency: lock heartbeat, max_lock_lifetime_ms, per-thread drain isolation, debounce drain #190 through [4.41/C10] Cards & modals core: Chart, Table options, Button tooltip, Card width, DateInput/NumberInput, select change events #202 merge. Run the downstream boundary suite and the Slack live loop against the main SHA. The lock heartbeat ([4.41/C1a] Core concurrency: lock heartbeat, max_lock_lifetime_ms, per-thread drain isolation, debounce drain #190) can be checked earlier, on its own.
  2. Slack + Teams checkpoint: after [4.41/SL1] Slack outbound mentions: code/URL-aware resolution, native-stream path #206–[4.41/SL4] Slack inbound normalization: self-mention decode, content is_mention, mrkdwn fixes, bot author ids, socket retries #209 and [4.41/T1] Teams routing & outbound text: stop <at> rewriting, conversationType in thread ids, per-service-URL clients #216–[4.41/T6] Teams auth: custom token factory precedence, webhook_verifier, sovereign endpoint allowlists #221 merge. Run the Slack live loop (streaming, tool use, markdown, rotation past 5 minutes) and a Teams live loop (mentions, placeholder streaming, propagate_handler_errors).

Escape hatch: if the wave slips and downstream needs the lock heartbeat or stream rotation sooner, cut 0.4.31.5 from main as a backport release. That follows the existing Python-only patch convention.

Start here

These have no blockers and can run in parallel now, in recommended priority order:

  1. [4.41/P0] Fidelity tooling for the 4.41 wave: single pin constant, SHA pin, it.each expansion, map new core test files #185: fidelity tooling. It unblocks all of core ([4.41/C1a] Core concurrency: lock heartbeat, max_lock_lifetime_ms, per-thread drain isolation, debounce drain #190, [4.41/C2b] Shared text utilities: bare-mention scanner, code-fence helpers, to_plain_text structural whitespace #193, [4.41/CB] Callback tokens: consume once, bind to conversation, shorter TTL, preserve button fields #194, [4.41/C10] Cards & modals core: Chart, Table options, Button tooltip, Card width, DateInput/NumberInput, select change events #202) and [4.41/SH1] Shared guarded downloader (redirect policy, byte cap, timeout, credential host binding) #204.
  2. Security items with no blockers:
  3. Correctness and auth with no blockers: [4.41/T6] Teams auth: custom token factory precedence, webhook_verifier, sovereign endpoint allowlists #221 (Teams auth; the Teams streaming path), [4.41/ST1] Postgres state: reclaim expired set_if_not_exists rows, migration-managed schemas #240 (Postgres set_if_not_exists expiry, which [4.41/C1b] Core lifecycle: init retry, dedupe TTL 10min, propagate_handler_errors, webhook dedupe option #191's dedupe relies on), [4.41/W1] WhatsApp: business-scoped user ids (BSUID) and inbound context variants #236 (WhatsApp BSUID KeyError), [4.41/GH1] GitHub: configurable bot user id #233 (GitHub bot-id self-reply loop).
  4. Hygiene, any time: [4.41/HK1] Housekeeping: prune stale pre-4.31 remote branches #186, [4.41/X1] Shared adapter test contracts (thread-id + self-message) — optional #188 (optional). [4.41/DEF1] Deferred upstream 4.32–4.41 features (demand-gated backlog) #189 is a backlog record, not work.

Once #185 merges, start the Slack/Teams streaming prerequisites right away:

Critical path

The path to the Slack streaming fixes that matter most downstream:

P0 → C1a (lock heartbeat, max_lock_lifetime_ms) → C1b → C2a → C5 (placeholder sentinel, lightweight threads)
   → SL2 (stream() → None without context, native_streaming + post/edit fallback) → SL3 (rotate native streams before expiry)
P0 → C2b → SL1 (code/URL-aware mentions, resolved stream buffer) ──────────────────────────────────────────→ SL3

Dependency graph

Arrows point from blocker to blocked. Only hard "Blocked by" edges are shown; soft ordering notes are in each issue.

flowchart LR
  subgraph Tooling
    P0["P0 fidelity tooling"]
    HK1["HK1 prune branches"]
    X1["X1 test contracts"]
    DEF1["DEF1 deferred backlog"]
    C11["C11 pin bump + 0.4.41"]
  end
  subgraph Core
    C1a["C1a lock heartbeat / drains"]
    C1b["C1b init retry / dedupe / propagate"]
    C2a["C2a tri-state is_mention / Message"]
    C2b["C2b mention scanner / code fences"]
    CB["CB callback tokens"]
    C3["C3 conversation scope / AI tools"]
    C4["C4 lifecycle events"]
    C5["C5 streaming / thread API"]
    C6["C6 reply / mark_as_read"]
    C7["C7 turn cancellation"]
    C8["C8 History API"]
    C9["C9 AI messages"]
    C10["C10 cards & modals core"]
    LOG1["LOG1 no raw body logs"]
  end
  subgraph Shared
    SH1["SH1 guarded downloader"]
  end
  subgraph Slack
    SL0["SL0 install-scoped caches"]
    SL1["SL1 outbound mentions"]
    SL2["SL2 stream fallback"]
    SL3["SL3 stream rotation"]
    SL4["SL4 inbound normalization"]
    SL5["SL5 tables / alert attachments"]
    SL6["SL6 edit / delete events"]
    SL7["SL7 cards & modals"]
    SL8["SL8 downloads / Grid / proxy"]
    SL9["SL9 agent_view"]
    SL10["SL10 agent sessions"]
  end
  subgraph Teams
    T1["T1 routing / outbound text"]
    T2["T2 install lifecycle"]
    T3["T3 email / secure downloads"]
    T4["T4 reactions / ephemeral / streaming"]
    T5["T5 cards & dialogs"]
    T6["T6 auth / verifier"]
  end
  subgraph Other
    G1["G1 GChat JWT binding"]
    G2["G2 GChat identity / media"]
    TG0["TG0 TG webhook verification"]
    TG1["TG1 TG inbound media"]
    TG2["TG2 TG streaming"]
    TG3["TG3 TG polling / media groups"]
    TG4["TG4 TG replies"]
    D1["D1 Discord security"]
    D2["D2 Discord features"]
    L0["L0 Linear ownership"]
    L1["L1 Linear session ids"]
    GH1["GH1 GitHub bot id"]
    MS1["MS1 Messenger downloads"]
    TW1["TW1 Twilio isolation"]
    W1["W1 WA BSUID"]
    W2["W2 WA templates / errors"]
    W3["W3 WA outbound media"]
    W4["W4 WA+MS reply / read / downloads"]
    ST1["ST1 Postgres expiry"]
    GMAIL["GMAIL Gmail adapter"]
  end
  P0 --> C1a & C2b & CB & C10 & SH1
  C1a --> C1b & C3
  CB --> C3
  C1b --> C2a & TG3
  C2a --> C5 & SL4 & T1 & T3 & TG4 & L1
  C2b --> SL1 & SL4 & D1 & W4
  C3 --> C4 & C8 & D1
  C4 --> C7 & SL6 & SL9 & T2
  C8 --> C9
  C5 --> C6 & C7 & SL2 & T4 & TG2
  C6 --> T4 & TG4 & W4 & GMAIL
  C7 --> SL10
  C10 --> SL7 & T5
  C1a & C1b & C2a & C2b & CB & C3 & C4 & C5 & C6 & C7 & C8 & C9 & C10 --> C11
  SH1 --> SL8 & T3 & TG1 & W4 & D1
  SL0 --> SL8
  SL1 --> SL3
  SL2 --> SL3
  SL4 --> SL5
  SL9 --> SL10
  T1 --> T2
  G1 --> G2
  D1 --> D2
  L0 --> L1
  W1 --> W2 --> W3
Loading

Checklist by workstream

Tooling & meta

Core

Shared

Slack

Teams

Google Chat

Telegram

Discord

Linear & GitHub

WhatsApp, Messenger, Twilio

State

Deferred (demand-gated)

Release gate for 0.4.41

Must be merged before 0.4.41 is tagged and published:

May trail 0.4.41 as documented non-parity rows in docs/UPSTREAM_SYNC.md (the Linear-deferred precedent from 0.4.30). Each needs a row naming its refs and "tracked in #N":

Conventions for agents

  • One issue = one PR on branch sync/4.41-<id-lowercase> (e.g. sync/4.41-c1a). Only split when the issue's Metadata says so (e.g. [4.41/SL8] Slack: guarded downloads, Enterprise Grid org-wide installs, egress proxy #213, [4.41/SL10] Slack Agent Sessions lifecycle + native stop #215, [4.41/GMAIL] Gmail adapter (demand-gated): primitives + Chat adapter #241); use -a/-b suffixes. Keep each PR under about 1.5k LOC.
  • Put Closes #N in the PR body, with a link to this tracking issue.
  • Respect "Blocked by". Don't start an issue until its blockers are merged to main. Many core issues touch chat.py, thread.py and types.py, and the blocked-by order is what keeps them from conflicting. Soft-ordering notes ("should land after…") are in each issue's Dependencies section. When you land second, you wire the other issue's hooks, as each issue describes.
  • Upstream checkout: git clone https://github.com/vercel/chat.git /tmp/vercel-chat && git -C /tmp/vercel-chat fetch --tags. Read the upstream code at a tag with git -C /tmp/vercel-chat show chat@4.41.1:<path>, and each commit with git show <hash>. Don't check out tags in a shared clone.
  • Before opening the PR:
    • Run the full CLAUDE.md validation: uv run ruff check src/ tests/ scripts/ && uv run ruff format --check src/ tests/ scripts/ && uv run python scripts/audit_test_quality.py && uv run python scripts/verify_test_fidelity.py && uv run pytest tests/ --tb=short -q.
    • Run strict fidelity at the current pin: clone the pinned tag as CLAUDE.md shows, then run TS_ROOT=<that clone> uv run python scripts/verify_test_fidelity.py --strict.
    • After [4.41/P0] Fidelity tooling for the 4.41 wave: single pin constant, SHA pin, it.each expansion, map new core test files #185 lands, report the non-strict missing count against 4.41.1 in the PR description.
  • Apply the docs/SELF_REVIEW.md checks: input sweeps, emit/parse symmetry, unforgeable sentinels, divergence budget and the adversarial-reviewer question.
  • Port rules (docs/UPSTREAM_SYNC.md):
    • is not None instead of truthiness for ??;
    • snake_case inside, camelCase at the wire;
    • hmac.compare_digest for any secret comparison;
    • UTC-aware datetimes;
    • lazy optional imports;
    • AsyncMock for async, and fake clocks instead of sleeps in tests;
    • no duplicate tests.
  • Changelog: add an entry under an "Unreleased (4.41 wave)" heading in CHANGELOG.md, and call out consumer-visible behavior changes explicitly. Record every divergence or skip in docs/UPSTREAM_SYNC.md in the same PR.
  • Security wording: this repo is public. Describe weaknesses at class level and cite the upstream fix and its tests. No exploit steps or payloads in PRs, commits or tests.
  • Don't bump UPSTREAM_PARITY, the fidelity pin or the package version anywhere except [4.41/C11] Bump fidelity pin + UPSTREAM_PARITY to chat@4.41.1, record non-parity rows, cut 0.4.41 #203.

Housekeeping

Activity

  1. patrick-chinchill commented on Sep 30, 2026

    @patrick-chinchill
    CollaboratorAuthor

    #213 (SL8) is shipping as two PRs, as the issue planned. PR A (guarded downloads + egress proxy) is on sync/4.41-sl8. The Enterprise Grid half moved to #268 (SL8b, sync/4.41-sl8-grid).

  2. patrick-chinchill commented on Oct 1, 2026

    @patrick-chinchill
    CollaboratorAuthor

    #203 is open as draft PR #309. It moves the fidelity pin to chat@4.41.1 (922/922 strict), sets UPSTREAM_PARITY = "4.41.1", bumps the version to 0.4.41 and records the skip/deferred non-parity rows. It does not tag or publish.

    Every gate issue above is merged on main. Before tagging and publishing 0.4.41, run these checks against the merge commit:

    Note: #206 and #215 are merged (9d59bb8, a78919d) but their issues are still open.

  3. patrick-chinchill commented on Oct 1, 2026

    @patrick-chinchill
    CollaboratorAuthor

    Released: v0.4.41 (2026-10-01), on PyPI as chat-sdk==0.4.41. Synced to upstream chat@4.41.1; strict fidelity 922/922; live-validated on chinchill customer-demo (see the CHANGELOG "Live validation" section). Teams 2.1 and the #215 Agent Sessions Stop were not live-verified (no Teams tenant; opt-in feature unused by consumers).

    Open follow-ups, tracked separately:

    Chinchill adoption deltas are recorded on chinchill-api draft PRs #1532 (main) and #1534 (oss/agent-framework). Pin chat-sdk==0.4.41 from PyPI rather than the git SHA; the agent-framework Docker builder has no git.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions