Skip to content

Render images and Mermaid diagrams in file previews - #145

Merged
webdevcody merged 3 commits into
mainfrom
cursor/inline-graphics-rendering-57fa
Oct 9, 2026
Merged

webdevcody merged 3 commits into
mainfrom
cursor/inline-graphics-rendering-57fa

Conversation

@webdevcody

@webdevcody webdevcody commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

This adds inline visual previews to nebula open, file tabs, and the tree browser so agents can show local images, mockups, .mmd files, and Markdown Mermaid blocks without pasting file contents into chat.

Contents

What users get

  • Images open as images — PNG, JPEG, GIF first frames, WebP, and SVG via local rsvg-convert/magick fallback render in file tabs and the tree browser.
  • Mermaid diagrams become previews — .mmd files and ```mermaid fenced blocks render through local mmdc when it is on PATH, with a source-plus-hint fallback when it is not.
  • Agents get concise guidance — Claude’s appended prompt now tells it to save images/mockups/diagrams locally and run nebula open <path> only after an explicit user ask.
  • A setting controls visuals — inline_graphics accepts auto, protocol names, halfblocks, or off; enabled modes currently use the portable half-block renderer.

Screenshot

Nebula tree browser rendering a PNG mockup through inline half-block graphics

How it works

flowchart LR
  Agent[Agent session] -->|nebula open path| CLI[CLI validation]
  CLI -->|text/image/mmd only| Daemon[DAEMON FilesOpened]
  Daemon --> Tabs[File tabs / tree preview]
  Tabs --> Dispatch[Preview dispatch]
  Dispatch --> Images[Local image decode]
  Dispatch --> Mermaid[mmdc when available]
  Images --> Halfblocks[Half-block raster renderer]
  Mermaid --> Halfblocks
  Mermaid --> Fallback[Source + install hint]
Loading

Risk read

🟡 Medium overall: this adds local file decoding and an optional local Mermaid subprocess, but keeps network fetching out of scope.

Area Risk Notes
🔒 Security / merge risk Medium Reads only user-selected local files; Mermaid rendering shells out to mmdc on PATH and falls back cleanly when absent.
⚡ Performance Medium Image/Mermaid work rides the existing background preview jobs; resized half-block lines are cached per view size.
🧩 Codebase fit Low Reuses file tabs, tree browser, Markdown rendering and nebula open IPC plumbing.

Rollback: reverting this PR removes visual previews, the inline_graphics setting, the new image dependency, screenshot scene, and the updated agent guidance.

Technical overview

  • nebula-tui/src/graphics.rs owns image detection/decoding, Mermaid extraction/rendering, SVG local-tool fallback, content caching, and half-block painting.
  • tree_browser::Preview now carries optional visual/Mermaid state shared by the tree browser and file tabs.
  • markdown.rs can receive pre-rendered Mermaid diagrams and inject them where fenced blocks used to draw as code.
  • ipc.rs and CLI/help docs now accept text, images and Mermaid diagrams while refusing unrelated binary files.
  • scripts/shot/scenes/graphics-preview.* captures a deterministic tree-browser PNG preview scene.

Dependency and size impact

  • New dependency: image =0.25.6, default features off, with only png, jpeg, gif, and webp enabled.
  • Release binary size measured locally after cargo build --release --locked:
    • origin/main: 16,608,536 bytes
    • this branch: 17,572,096 bytes
    • delta: +963,560 bytes (+5.8%)

Verification

  • GitHub CI ✅ 3/3 checks successful (ubuntu cargo, macOS cargo, Claude review)
  • cargo +stable check -p nebula-tui ✅
  • cargo +stable test -p nebula-tui ✅
  • RUSTUP_TOOLCHAIN=stable cargo fmt --all -- --check ✅
  • RUSTUP_TOOLCHAIN=stable cargo check --workspace --all-targets --locked ✅
  • RUSTUP_TOOLCHAIN=stable cargo clippy --workspace --all-targets ✅ (completed with existing non-fatal warnings)
  • RUSTUP_TOOLCHAIN=stable cargo test --workspace -- --skip tui_drag_past_the_pane_top_autoscrolls_and_copies_the_run ✅
  • RUSTUP_TOOLCHAIN=stable make shot SCENE=graphics-preview ✅ for .ansi/.txt; PNG rendered manually with the harness renderer and Linux DejaVu Sans Mono after installing python3.12-venv because the default font is macOS Menlo.

Note: an unskipped RUSTUP_TOOLCHAIN=stable cargo test --workspace passed through all daemon and nebula open e2e coverage, then hung locally on the unrelated tui_drag_past_the_pane_top_autoscrolls_and_copies_the_run e2e test; the skipped rerun completed green.

Follow-ups

  • Native Kitty/iTerm/Sixel protocol rendering can replace the current accepted-protocol-name-to-halfblocks fallback later.
  • SVG rendering remains external-tool based (rsvg-convert or magick) to avoid bundling a browser/vector stack.
  • macOS behavior is covered by the green GitHub CI cargo check; local manual artifact capture was Linux.
Open in Web Open in Cursor 

@cursor
cursor Bot force-pushed the cursor/inline-graphics-rendering-57fa branch from 800837e to b3875bf Compare October 9, 2026 13:08
cursoragent and others added 3 commits October 9, 2026 18:09
Co-authored-by: Web Dev Cody <webdevcody@gmail.com>
Co-authored-by: Web Dev Cody <webdevcody@gmail.com>
Co-authored-by: Web Dev Cody <webdevcody@gmail.com>
@cursor
cursor Bot force-pushed the cursor/inline-graphics-rendering-57fa branch from b3875bf to fdfab07 Compare October 9, 2026 18:10
@webdevcody
webdevcody marked this pull request as ready for review October 9, 2026 18:10
@webdevcody
webdevcody merged commit bd708a6 into main Oct 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants