Skip to content

perf(cuda): add calibrated mixed precision and fuse diffusion linears - #204

Merged
elprofesoriqo merged 1 commit into
mainfrom
perf/cuda-mixed-precision
Oct 15, 2026
Merged

elprofesoriqo merged 1 commit into
mainfrom
perf/cuda-mixed-precision

Conversation

@elprofesoriqo

@elprofesoriqo elprofesoriqo commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • add an explicit approximate FP16-deep diffusion mode while keeping exact FP32 as the default
  • calibrate the mixed-precision layer set against matched PushT seeds and reject the quality-regressing seven-layer candidate
  • fuse 1xK linear bias and optional Mish on CUDA, reducing the ten-step launch census from 1,000 to 850
  • expose precision selection through the C ABI, Python extension, LeRobot adapter/config, rollout CLI, and evaluators
  • check in complete alternating timing/parity samples and PushT calibration results

Evidence

GTX 1650 4 GB, WSL2, Release CUDA, pinned diffusion_pusht checkpoint:

  • approximate mode gate: 104.216 ms to 82.620 ms median native p50 before fusion, 20.72% lower, all 5 paired trials won
  • matched PushT seeds 7-16: exact 8/10 success, approximate 8/10; mean max reward 0.998138 vs 0.998099
  • rejected calibration: converting the seventh deep convolution reduced success to 7/10
  • exact fusion gate: 105.815 ms to 104.257 ms median p50, 1.47% lower, all 5 paired trials won, bitwise-identical actions
  • post-fusion exact vs approximate: 98.736 ms vs 83.574 ms median native p50

This is PushT simulation evidence. SO-101 task success is not yet measured. The 10 ms target is not met.

Verification

  • scripts/verify_local.ps1 passed on the final tree
  • Windows native CTest: 85/85
  • focused native kernels: 19/19
  • LeRobot unittest suite: 110 passed, 1 expected artifact-evidence skip in the canonical run
  • CUDA focused C ABI and Diffusion tests: 16/16
  • CUDA fused linear reference test passed
  • Release CUDA and WSL CPU builds passed
  • diffusion parity remained within the existing exact-mode gates

Dependencies

#202 and #203 are merged. This PR is rebased onto current main and contains one new commit.

@elprofesoriqo
elprofesoriqo force-pushed the perf/cuda-mixed-precision branch from 39168a4 to 341c340 Compare October 15, 2026 14:20
@elprofesoriqo
elprofesoriqo force-pushed the perf/cuda-mixed-precision branch from 341c340 to 8bd7beb Compare October 15, 2026 14:26
@elprofesoriqo
elprofesoriqo merged commit ff28e2a into main Oct 15, 2026
13 checks passed
@elprofesoriqo
elprofesoriqo deleted the perf/cuda-mixed-precision branch October 15, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant