Skip to content

Preserve rich formatting with AI rewrites - #63

Merged
janglad merged 3 commits into
mainfrom
fix/web-4064-pen-cmd-j-strips-formatting
Sep 25, 2026
Merged

janglad merged 3 commits into
mainfrom
fix/web-4064-pen-cmd-j-strips-formatting

Conversation

@janglad

@janglad janglad commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Preserves rich formatting, lists, images, and whitespace during AI selection rewrites. Keeps partial selections scoped and retains their existing marks.


Note

Medium Risk
Changes AI rewrite routing, model context, and markdown ingest/normalization on a core editing path; behavior is heavily tested but regressions could affect selection scope or formatting round-trips.

Overview
Expands inline AI rewrites so whole-block selections (headings, lists, images, tables, etc.—not just paragraphs) route through markdown scoped-range replacement, with host chrome blocks trimmed from the scope. Partial selections stay partial: the model and egress inventory only see the highlighted text, and resolveCommonSelectionMarks carries shared bold/italic/underline through text-splice and suggestion streaming.

Flow markdown prompts now allow the rich GFM subset Pen can round-trip and tell the model to preserve structure; normalizeFlowMarkdownOutput and stripBlockAnnotations stop collapsing internal blank lines and preserve NBSP empty paragraphs.

Ingest reconstructs empty paragraphs from Pen’s extra newline separators, parses <u> underline marks, and suggest mode skips redundant format suggestions on newly inserted blocks. Spec and fidelity docs updated accordingly.

Reviewed by Cursor Bugbot for commit 82e288c. Bugbot is set up for automated code reviews on this repo. Configure here.

This update introduces improvements to the AI selection handling, ensuring that rich formatting and block structures are preserved during selection rewrites. Key changes include the addition of a new context for whole-block selections, which allows for the retention of markdown formatting, and the implementation of tests to verify the correct behavior of these features.

- Added support for preserving rich formatting in AI selection rewrites.
- Introduced a new context for whole-block selections in the AI controller.
- Updated tests to validate the preservation of formatting and block structures.
Comment thread packages/extensions/tools/src/utils/documentContext.ts Fixed
This commit enhances the handling of whitespace in markdown content by introducing a more efficient trimming function. The `trimMarkdownEnvelope` function has been implemented to remove leading and trailing whitespace while preserving the integrity of the content. Additionally, a new test has been added to validate the performance of the `stripBlockAnnotations` function, ensuring it can handle long internal whitespace runs effectively.

Key changes:
- Refactored whitespace trimming logic in markdown processing.
- Added `isMarkdownEnvelopeWhitespace` utility function.
- Introduced a performance test for `stripBlockAnnotations` to handle large whitespace scenarios.
@janglad

janglad commented Sep 24, 2026

Copy link
Copy Markdown
Member Author

bugbot run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/extensions/tools/src/utils/documentContext.ts
This commit introduces a new test case to ensure that consecutive blank lines are preserved when stripping echoed annotations from markdown content. The test validates the behavior of the `normalizeFlowMarkdownOutput` function, enhancing the overall robustness of markdown processing.

Key changes:
- Added a test for maintaining consecutive blank lines in `flowMarkdown.test.ts`.
- Adjusted the `stripBlockAnnotations` function to improve whitespace handling by removing unnecessary replacements.
@janglad

janglad commented Sep 24, 2026

Copy link
Copy Markdown
Member Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 82e288c. Configure here.

@janglad
janglad merged commit cbbb24c into main Sep 25, 2026
35 checks passed
@janglad
janglad deleted the fix/web-4064-pen-cmd-j-strips-formatting branch September 25, 2026 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants