Repository navigation
[Tracking]: Model support status #83
Description
Purpose
Keep one rolling overview of which System1-Omni models are available, which integrations are in progress, and which models remain candidates. Record implementation status separately from validation on real hardware, and link each row to its model issue, implementation PRs and recipes.
Last checked: 2026-10-13. Baseline: main at 3e9cc33. Maintenance qualification is recorded separately from merged availability. The existing checklist states below are preserved.
Related: #9 (model roadmap), Supported models and hardware, and #61 (shared worker API contract). Keep this issue open as a rolling tracker.
Status definitions
- Available: a runnable worker or recipe is merged into
main, within the scope stated below. - In progress: implementation PRs are open; the proposed integration is not yet available from
main. - Proposed: a dedicated model issue/RFC exists, with no complete implementation on
main. - Candidate: listed in the roadmap, with no dedicated implementation issue found.
Hardware validation applies only to the linked worker, checkpoint and tested workload. CPU checkpoint loading and tests with a stub encoder do not establish GPU model support.
Model support
| Model / checkpoint | Input and question scope | Status on main |
Pending integration / tracking |
|---|---|---|---|
| LAYA, English checkpoint | Text; choice, score, noul |
Available: external CPU, Python MPS/CPU, and native Rust/CUDA worker. | Native CUDA requires Hopper sm_90a; see native validation. Pending bounded workspace cache #74; fresh CPU checks pass, prior H800 evidence remains historical. |
| Cua-S1 4B 0.2, text adapter | Text; choice |
Available: reference and native Rust/CUDA workers; opt-in text Graph replay. | #141 numerical follow-up remains open. Oct13 RTX4090 diagnosis: 139 exact token rows; three native/BF16 winner flips have reference ties; original FP32-relative drift and margin gates pass in the recorded environment. Exact original five-case report/environment is not reproduced. |
| Cua-S1 4B 0.2, multimodal adapter | One PNG/JPEG screenshot; choice |
Available: reference and native screenshot execution, including opt-in vision and multimodal language Graph. | Pending shared vision #117, bounded cache #127 and GPU-resident features #135. Oct13 exact maintained #127/#135 functional GPU checks pass on RTX4090; no new screenshot performance claim. #127/#135 target #117. New native screenshot recipe in #135. |
| Open-Jev-27B-v1.1 | Text candidates; choice, score, noul |
Available: native Rust/CUDA worker (#55; model issue #54 completed). | Native recipe and validation scope. Broader multi-candidate/workload coverage remains to be recorded. |
| CLM-v0.1-8B | Text; choice, score, noul |
Available: external stub-encoder recipe and native engine using a separate Qwen3-8B embeddings server. | #29 merged; native validation distinguishes engine-only same-vector agreement, real-encoder execution and decision quality. |
| Kev | Text; target choice, score, noul; 4B first, wider 0.5B–9B family proposed |
Proposed: #27; no complete worker on main. |
Implement adapter merge, prompt construction and trained pointer head. Draft scoring kernel #69 is shared work for CLM/Kev, not a complete Kev integration. |
| LiquidAI/LFM2.5-350M | Text; choice |
Available: Transformers candidate-batching worker. | L40S FP16 recorded execution checks; full CPU checkpoint and native execution remain unverified. |
| Mapika/decider-2b, v11 | Text; choice, score, noul |
Available: native Rust/CUDA worker; #94/#110 merged. | Pending request batching #111, CUDA Graph #112, shared prefix/fixed GEMM #113. Oct13 maintained heads pass frozen five-case/20-row reference gates on RTX4090 plus retirement/fallback checks; maximum probability drift 0.012, Score drift 0.01. No fresh speedup claim. |
| JEMM 27B | Text and ordered 1–4-image decisions | In progress: #117 → #118 → #122 → #123, all open. | Historical A800 v2 16-request/25-question parity remains scoped to its recorded binaries. Oct13 cuDNN/FLA eleven reference-operator checks pass on RTX4090; full 27B checkpoint and combined v2 multimodal language Graph remain unverified on this 24GB server. #123 now documents inherited Graph controls and validation limits. |
Hardware and validation evidence
| Worker / path | CPU | NVIDIA CUDA | Apple GPU | Evidence and limits |
|---|---|---|---|---|
| LAYA external upstream worker | Validated | Unverified | Use the dedicated MPS worker below | #2; CPU recipe; CUDA benchmarking tracked in #39. |
| LAYA Python MPS/CPU worker | Contract checks documented | No validation recorded for this path | PyTorch MPS validated on M1 Pro; M4/M5 checks also documented | #30, #67; Apple Silicon recipe. This is a Python MPS path; a native Metal executor remains planned. |
| Cua-S1 text reference worker | Unverified | Validated | Unverified | #13; supported hardware matrix. |
| Cua-S1 text native worker | Not supported | Validated on sm_89; documented requirement sm_80+ | Not supported | #19, #52; build compatibility does not establish validation on every supported GPU. |
| Cua-S1 multimodal reference worker | Not supported | Validated | Not supported | #12, #17, #18; single-image choice scope. |
| Open-Jev native worker | Not supported | Validated on H200 / sm_90 | Not supported | #55; recorded comparison covers 74 single-candidate noul requests. |
CLM external recipe on main |
Stub-encoder contract checks only | Real encoder not validated by the merged recipe | Unverified | #23; recipe. Open #29 reports real-encoder smoke checks separately. |
| LFM2.5-350M merged worker | Tiny-model regression checks; full checkpoint unverified | L40S FP16 recorded execution checks | Unverified | Recipe and evidence; candidate batching, cache isolation and frontend parity, within the recorded scope. |
| Cua-S1 native multimodal worker and pending optimizations | Contract/preprocessing/cache checks | RTX4090 functional checks, source-scoped | Not supported | Native screenshot/Graph execution is merged; #117/#127/#135 remain proposed increments. Oct13 shared-head cache/device/capture-error checks pass, without a new latency campaign. |
The current supported-model page records LAYA MPS, native LAYA, and both CLM paths and links this tracker. Decider and native Cua/JEMM pending optimization coverage above is tracked explicitly; open-PR device evidence is not promoted to merged support.
Maintenance integration — 2026-10-13
All twelve maintained PRs (#74/#93/#111/#112/#113/#116/#117/#118/#122/#123/#127/#135) contain the baseline main snapshot via normal merges and are conflict-free at that baseline. Exact-head CPU checks and hosted CI/Docs pass; JEMM stacked heads use linked upstream workflow-dispatch maintenance statuses. No PR was merged during this maintenance. Standalone ABI contracts are 6 for main/#117/#118/#111/#112, 7 for #113, and 8 for #135. The separately published cumulative ABI8 integration passes 262 CPU/163 Python checks, 17 device invocations, eleven exact JEMM operators and all five Decider fidelity controls, plus CI/Docs. It includes the cross-feature transient resident allocation lifetime fix; individual PR results do not substitute for this separate combined-source evidence. Full27B/Hopper and performance limits remain unchanged.
#93 now explicitly selects Score wire precision (default4 / Decider2 decimals) and retains analytical bounds. #141 also edits the benchmark validator and requires a deliberate overlap resolution when landing. #116 now classifies validation/close finishing after deadline as timeout; blocking stages do not guarantee a hard return deadline.
Additional candidates from #9
| Candidate | Published checkpoint / scope | Tracking |
|---|---|---|
| AFM-DE | ariacompute/afm-de, LAYA fine-tune | Candidate; compatibility with the LAYA worker still needs validation. |
| OpenJev / Verdict | heman10x/rlcd-modernbert-151m, ModernBERT 151M with an abstain slot | Candidate; separate checkpoint and integration from Open-Jev-27B-v1.1. |
| Julia-1 | SupersonicLabs/Julia-1, mmBERT-small 144M | Candidate; no dedicated integration issue found. |
| LFM2.5-2.6B-RLCD | monotykamary/LFM2.5-2.6B-RLCD | Candidate; the 350M issue/PR does not establish 2.6B support. |
Updating this tracker
When a model integration or validation lands, update its row and the last-checked date. Record the checkpoint/revision, worker path, modalities/question types, exact tested device, and links to merged code and reproducible validation. Keep results from open PRs identified as proposed evidence until integration and validation are complete.
- Searched existing issues and reviewed the current model/hardware documentation, recipes and PR states.
- Record implemented paths, pending integrations and additional candidates with evidence links.
- Reconcile
docs/supported-models.mdwith the merged LAYA MPS work and CLM contract recipe. - Add a link to this tracker from the Supported Models documentation.
Activity