Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/web-4052-preserve-pasted-formatting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"@input/pen-interop": patch
"@input/pen-schema": patch
"@input/pen-dom": patch
"@input/pen-react": patch
"@input/pen-vue": patch
---

Preserve HTML block structure, blank-line spacing, inline marks, lists, and text alignment when pasting formatted HTML.
6 changes: 3 additions & 3 deletions .size-limit.baseline.json
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,9 @@
{
"name": "@input/pen-interop",
"path": "packages/extensions/interop/dist/*.mjs",
"baselineBytes": 104761,
"limitBytes": 104761,
"note": "First baseline 2026-08-24 (SF2). Budgeted as the whole shipped ESM closure, not dist/index.mjs: the four format subpaths code-split into content-hashed chunks, so the root entry is 1722 B of re-exports while the chunks carry 100335 B \u2014 an index.mjs budget would guard 1.6% of the package. Replaces seven budgets totalling 100361 B (30302 import-html + 16369 export-xml + 15551 export-json + 14228 import-json + 10107 export-html + 9197 import-markdown + 4607 export-markdown). 100361 \u2192 104761, +4.4% for per-format barrel wiring. Byte total reproduced across three rebuilds."
"baselineBytes": 118975,
"limitBytes": 118975,
"note": "First baseline 2026-08-24 (SF2). Budgeted as the whole shipped ESM closure, not dist/index.mjs: the four format subpaths code-split into content-hashed chunks, so the root entry is 1722 B of re-exports while the chunks carry 100335 B \u2014 an index.mjs budget would guard 1.6% of the package. Replaces seven budgets totalling 100361 B (30302 import-html + 16369 export-xml + 15551 export-json + 14228 import-json + 10107 export-html + 9197 import-markdown + 4607 export-markdown). 100361 \u2192 104761, +4.4% for per-format barrel wiring. Re-recorded 2026-09-24 for WEB-4052: 104761 \u2192 118831, +14070 B (+13.4%) for preserving safe stylesheet class declarations, inline marks, lists, and text alignment on HTML paste; 118831 \u2192 118975, +144 B to preserve the first safe rule after braceless at-rules. Byte total reproduced from a clean @input/pen-interop build."
},
{
"name": "@input/pen-snapshots",
Expand Down
29 changes: 14 additions & 15 deletions packages/docs/src/generated/pasteCorpus.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,34 +88,33 @@ export const PASTE_CORPUS_ROWS: readonly PasteCorpusRow[] = [
code: "none",
links: "kept",
images: "none",
marks: "wrapper <b> bolds the entire paste; span font-weight / font-style / text-decoration lost",
marks: "span bold and italic kept; font-weight normal wrapper ignored",
colors: "span color kept as textColor",
intentionalLosses: [
"The docs-internal-guid <b> wrapper is treated as inline bold, so headings, lists, and tables collapse into one paragraph",
"Google Docs bold/italic/strike expressed as span font-weight / font-style / text-decoration do not become marks",
"The docs-internal-guid <b> wrapper still collapses headings, lists, and tables into one paragraph",
"Adjacent block text is concatenated with no separator",
],
},
{
id: "apple-notes",
source: "Apple Notes",
provenance: "synthetic-until-capture",
captured: false,
application: "",
version: "",
capturedAt: "",
host: "",
approximates: "Apple Notes clipboard fragment: StartFragment, heading plus div-wrapped paragraphs, semantic lists. Notes typically uses <div> rather than <p>.",
headings: "h1",
lists: "flat bullets",
provenance: "captured: Apple Notes 4.13 (2026-09-23)",
captured: true,
application: "Apple Notes",
version: "4.13",
capturedAt: "2026-09-23",
host: "Arc (Chromium paste event)",
approximates: "n/a — hand-captured browser paste payload",
headings: "none",
lists: "flat numbered and bullets",
tables: "none",
code: "none",
links: "kept",
links: "none in capture",
images: "none",
marks: "bold kept after the mixed-inline split",
marks: "bold, italic, and class-based underline kept",
colors: "none",
intentionalLosses: [
"A Notes <div> with mixed text and <b> fragments into adjacent paragraphs; leading text is trimmed",
"font family, size, margins, minimum heights, and list marker CSS are discarded",
],
},
{
Expand Down
17 changes: 8 additions & 9 deletions packages/extensions/interop/PASTE-CORPUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Clipboard `text/html` + `text/plain` pairs measured through the generic HTML import path (`parseHtmlToBlocks`). Pen does not sniff `mso` classes or `docs-internal-guid`. A documented flattening is the paste contract; an undocumented one is a regression.

These fixtures are **synthetic-until-capture**: documented approximations of what each application emits, not hand-captured clipboard dumps. A real Word clipboard payload is hundreds of kilobytes of `<style>` and `mso-` attributes; these fixtures are a few hundred bytes. Do not invent markup and label it captured. The replacement procedure is `src/html/import/__tests__/pasteCorpus/CAPTURE.md`.
1 of 9 sources are hand-captured. Still synthetic-until-capture: `word-desktop`, `word-web`, `google-docs`, `notion`, `vscode`, `article`, `excel-sheets`, `pen`. The replacement procedure is `src/html/import/__tests__/pasteCorpus/CAPTURE.md`.

Generated from `src/html/import/__tests__/pasteCorpus/` by `src/html/import/__tests__/pasteCorpus.test.ts`. Do not edit by hand.

Expand All @@ -12,8 +12,8 @@ Generated from `src/html/import/__tests__/pasteCorpus/` by `src/html/import/__te
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Microsoft Word (desktop) | synthetic-until-capture | h1 | flattened to paragraphs | 2×2, no header row | none | kept | dropped (img inside MsoNormal paragraph is not lifted) | bold, italic, link | none | Word numbering metadata (mso-list on paragraphs) becomes plain paragraphs with the visible bullet glyph; the generic path does not sniff MsoListParagraph; Office <style> blocks and html/head/meta wrappers are stripped by the sanitizer; An <img> nested in <p class=MsoNormal> does not become an image block; an empty paragraph remains |
| Microsoft Word (web) | synthetic-until-capture | h1 | nested bullets (indent 0/1) | 2×2, no header row | none | kept | none | bold, italic, strike, link | none | Office xmlns / Mso* class names are ignored; conversion uses tags only; No Word numbering metadata is preserved beyond the emitted <ul>/<li> tree |
| Google Docs | synthetic-until-capture | flattened into one paragraph | flattened into one paragraph | flattened (cell text concatenated) | none | kept | none | wrapper <b> bolds the entire paste; span font-weight / font-style / text-decoration lost | span color kept as textColor | The docs-internal-guid <b> wrapper is treated as inline bold, so headings, lists, and tables collapse into one paragraph; Google Docs bold/italic/strike expressed as span font-weight / font-style / text-decoration do not become marks; Adjacent block text is concatenated with no separator |
| Apple Notes | synthetic-until-capture | h1 | flat bullets | none | none | kept | none | bold kept after the mixed-inline split | none | A Notes <div> with mixed text and <b> fragments into adjacent paragraphs; leading text is trimmed |
| Google Docs | synthetic-until-capture | flattened into one paragraph | flattened into one paragraph | flattened (cell text concatenated) | none | kept | none | span bold and italic kept; font-weight normal wrapper ignored | span color kept as textColor | The docs-internal-guid <b> wrapper still collapses headings, lists, and tables into one paragraph; Adjacent block text is concatenated with no separator |
| Apple Notes | captured: Apple Notes 4.13 (2026-09-23) | none | flat numbered and bullets | none | none | none in capture | none | bold, italic, and class-based underline kept | none | font family, size, margins, minimum heights, and list marker CSS are discarded |
| Notion | synthetic-until-capture | h1 | nested bullets (indent 0/1) plus checklist | none | codeBlock language ts | kept | 1 remote src kept | bold, italic, strike, link | none | Notion-specific block identity (if a real capture adds data-block-id) is ignored; conversion uses tags only |
| VS Code | synthetic-until-capture | none | none | none | flattened to one paragraph per token span | none | none | none | token colors kept as textColor | Styled VS Code copy has no pre/code wrapper, so it does not become a codeBlock; Each colored span becomes its own paragraph |
| Browser article | synthetic-until-capture | h1 | none | none | codeBlock | kept | 1 remote src kept; title becomes caption; figcaption becomes a paragraph | italic, bold, link | none | figure/figcaption wrappers unwrap; caption text is a sibling paragraph, not image.caption (title attribute is) |
Expand Down Expand Up @@ -50,18 +50,17 @@ Generated from `src/html/import/__tests__/pasteCorpus/` by `src/html/import/__te
- **Approximates:** Google Docs clipboard HTML: <b id=docs-internal-guid-…> wrapper, span-styled marks, semantic headings/lists/tables inside the wrapper. Typical Docs copies also append Apple-interchange-newline.
- **Markers:** `docs-internal-guid`, `font-weight:700`, `Apple-interchange-newline`

- The docs-internal-guid <b> wrapper is treated as inline bold, so headings, lists, and tables collapse into one paragraph
- Google Docs bold/italic/strike expressed as span font-weight / font-style / text-decoration do not become marks
- The docs-internal-guid <b> wrapper still collapses headings, lists, and tables into one paragraph
- Adjacent block text is concatenated with no separator

### Apple Notes

- **id:** `apple-notes`
- **Provenance:** `synthetic-until-capture`
- **Approximates:** Apple Notes clipboard fragment: StartFragment, heading plus div-wrapped paragraphs, semantic lists. Notes typically uses <div> rather than <p>.
- **Markers:** `StartFragment`, `div-wrapped body`
- **Provenance:** `captured: Apple Notes 4.13 (2026-09-23)`
- **Approximates:** n/a — hand-captured browser paste payload
- **Markers:** `Cocoa HTML Writer`, `span.s1`, `ol.ol1`, `ul.ul1`

- A Notes <div> with mixed text and <b> fragments into adjacent paragraphs; leading text is trimmed
- font family, size, margins, minimum heights, and list marker CSS are discarded

### Notion

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,26 @@ describe("@input/pen-interop/html dom-to-blocks: element mapping", () => {
});
});

it("IOP2 imports a placeholder break as one empty paragraph", () => {
const blocks = convert(
"<p>hello there</p><p><br></p><p>this is a test</p>",
);

expect(blocks).toMatchObject([
{ type: "paragraph", content: "hello there" },
{ type: "paragraph", content: "" },
{ type: "paragraph", content: "this is a test" },
]);
});

it("IOP2 preserves a break between inline text", () => {
const blocks = convert("<p>hello<br>there</p>");

expect(blocks).toMatchObject([
{ type: "paragraph", content: "hello\nthere" },
]);
});

it("script tag is stripped (AC 29)", () => {
const blocks = convert('<script>alert("xss")</script><p>safe</p>');

Expand All @@ -50,9 +70,7 @@ describe("@input/pen-interop/html dom-to-blocks: element mapping", () => {
const blocks = convert('<div onclick="alert(1)">text</div>');

expect(blocks.length).toBeGreaterThanOrEqual(1);
const hasText = blocks.some(
(b) => b.content?.includes("text"),
);
const hasText = blocks.some((b) => b.content?.includes("text"));
expect(hasText).toBe(true);
});

Expand All @@ -72,6 +90,56 @@ describe("@input/pen-interop/html dom-to-blocks: element mapping", () => {
expect(blocks[0].marks?.some((m) => m.type === "italic")).toBe(true);
});

it("IOP2 preserves marks expressed as inline styles", () => {
const blocks = convert(
'<p><span style="font-weight: 700">Bold</span> <span style="font-style: italic">italic</span> <span style="text-decoration: underline line-through">both</span></p>',
);

expect(blocks).toHaveLength(1);
expect(blocks[0]).toMatchObject({
type: "paragraph",
content: "Bold italic both",
});
expect(blocks[0].marks).toEqual(
expect.arrayContaining([
{ type: "bold", start: 0, end: 4 },
{ type: "italic", start: 5, end: 11 },
{ type: "underline", start: 12, end: 16 },
{ type: "strikethrough", start: 12, end: 16 },
]),
);
});

it("IOP2 preserves marks expressed by pasted stylesheet classes", () => {
const blocks = convert(
'<style>span.s1 {text-decoration: underline}</style><p>normal, <span class="s1">underline</span></p>',
);

expect(blocks).toHaveLength(1);
expect(blocks[0]).toMatchObject({
type: "paragraph",
content: "normal, underline",
});
expect(blocks[0].marks).toContainEqual({
type: "underline",
start: 8,
end: 17,
});
});

it("IOP2 preserves text alignment on default text blocks", () => {
const blocks = convert(
'<p style="text-align: center">Centered</p><h2 align="right">Right</h2><blockquote style="text-align: justify">Quoted</blockquote><ul style="text-align: end"><li>Listed</li></ul>',
);

expect(blocks).toMatchObject([
{ type: "paragraph", props: { textAlignment: "center" } },
{ type: "heading", props: { level: 2, textAlignment: "right" } },
{ type: "blockquote", props: { textAlignment: "justify" } },
{ type: "bulletListItem", props: { textAlignment: "end" } },
]);
});

it("link mark with href (AC 34)", () => {
const blocks = convert('<p><a href="https://example.com">text</a></p>');

Expand Down Expand Up @@ -111,9 +179,7 @@ describe("@input/pen-interop/html dom-to-blocks: element mapping", () => {
});

it("nested list with indent (AC 37)", () => {
const blocks = convert(
"<ul><li>a<ul><li>b</li></ul></li></ul>",
);
const blocks = convert("<ul><li>a<ul><li>b</li></ul></li></ul>");

expect(blocks).toHaveLength(2);
expect(blocks[0]).toMatchObject({
Expand Down Expand Up @@ -207,7 +273,12 @@ describe("@input/pen-interop/html dom-to-blocks: element mapping", () => {
const dom = parseHTML("<strong>bold at root</strong>");
const blocks = domToBlocks(dom, stubRegistry);

expect(blocks.some((b) => b.type === "paragraph" && b.content?.includes("bold at root"))).toBe(true);
expect(
blocks.some(
(b) =>
b.type === "paragraph" &&
b.content?.includes("bold at root"),
),
).toBe(true);
});

});
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,52 @@ describe("sanitizeHTML", () => {
expect(result).toContain("text");
});

it("SEC3: resolves safe simple class rules before stripping stylesheets", () => {
const result = sanitizeHTML(
'<style>.s1 {text-decoration: none} span.s1 {text-decoration: underline}</style><p><span class="s1">underlined</span></p>',
);
expect(result).not.toContain("style>");
expect(result).toContain('style="text-decoration: underline"');
});

it("SEC3: keeps simple rules separated by CSS comments", () => {
const result = sanitizeHTML(
'<style>.bold {font-weight: bold} /* source note */ .underlined {text-decoration: underline}</style><p><span class="bold">bold</span><span class="underlined">underlined</span></p>',
);
expect(result).toContain('class="bold" style="font-weight: bold"');
expect(result).toContain(
'class="underlined" style="text-decoration: underline"',
);
});

it("SEC3: keeps the first rule after braceless at-rules", () => {
const result = sanitizeHTML(
'<style>@charset "UTF-8"; @import "theme.css"; .underlined {text-decoration: underline}</style><p><span class="underlined">underlined</span></p>',
);
expect(result).toContain(
'class="underlined" style="text-decoration: underline"',
);
});

it("SEC3: drops commented stylesheet values", () => {
const result = sanitizeHTML(
'<style>.bold {font-weight: b/**/old}</style><p><span class="bold">plain</span></p>',
);
expect(result).not.toContain("font-weight");
});

it("SEC3: ignores complex stylesheet selectors and unsafe declarations", () => {
const result = sanitizeHTML(
'<style>span.s1 em {font-weight: bold} @media all {span.s1 {font-style: italic}} span.s1 {position: fixed; background-image: url(https://evil.example); text-decoration: underline}</style><p><span class="s1" style="text-decoration: none">plain</span></p>',
);
expect(result).not.toContain("font-weight");
expect(result).not.toContain("font-style");
expect(result).not.toContain("position");
expect(result).not.toContain("background-image");
expect(result).not.toContain("evil.example");
expect(result).toContain('style="text-decoration: none"');
});

it("strips <iframe> tags (AC 42)", () => {
const result = sanitizeHTML('<iframe src="evil.com"></iframe><p>ok</p>');
expect(result).not.toContain("iframe");
Expand Down Expand Up @@ -56,6 +102,25 @@ describe("sanitizeHTML", () => {
expect(result).not.toContain("z-index:");
});

it("SEC3: keeps enumerated mark-bearing inline styles", () => {
const result = sanitizeHTML(
'<span style="font-weight: 700; font-style: italic; text-decoration: underline line-through">styled</span>',
);
expect(result).toContain(
"font-weight: 700; font-style: italic; text-decoration: underline line-through",
);
});

it("SEC3: drops unsupported mark-bearing inline style values", () => {
const result = sanitizeHTML(
'<span style="font-weight: calc(700); font-style: inherit; text-decoration: underline overline">plain</span>',
);
expect(result).toContain("plain");
expect(result).not.toContain("font-weight");
expect(result).not.toContain("font-style");
expect(result).not.toContain("text-decoration");
});

it("SEC3: admits each enumerated text-align keyword and drops inherit", () => {
for (const value of ["left", "right", "center", "justify", "start", "end"]) {
const result = sanitizeHTML(
Expand Down Expand Up @@ -201,4 +266,72 @@ describe("parseInlineContent", () => {
expect(result.marks.some((m) => m.type === "bold")).toBe(true);
expect(result.marks.some((m) => m.type === "italic")).toBe(true);
});

it("uses the last declaration for the same property", () => {
const node: DOMNode = {
type: "element",
tagName: "span",
attributes: {
style: "font-weight: normal; font-weight: 700",
},
children: [{ type: "text", textContent: "bold" }],
};
expect(parseInlineContent(node).marks).toContainEqual({
type: "bold",
start: 0,
end: 4,
});
});

it("resolves text-decoration shorthand and longhand in source order", () => {
const node = (style: string): DOMNode => ({
type: "element",
tagName: "span",
attributes: { style },
children: [{ type: "text", textContent: "text" }],
});
expect(
parseInlineContent(
node(
"text-decoration-line: underline; text-decoration: none",
),
).marks,
).toEqual([]);
expect(
parseInlineContent(
node(
"text-decoration: none; text-decoration-line: underline",
),
).marks,
).toEqual([{ type: "underline", start: 0, end: 4 }]);
});

it("lets explicit styles reset semantic and inherited marks", () => {
const node: DOMNode = {
type: "element",
tagName: "p",
attributes: { style: "font-weight: bold" },
children: [
{
type: "element",
tagName: "span",
attributes: { style: "font-weight: normal" },
children: [{ type: "text", textContent: "normal" }],
},
{ type: "text", textContent: " bold" },
{
type: "element",
tagName: "u",
attributes: { style: "text-decoration: none" },
children: [{ type: "text", textContent: " plain" }],
},
],
};
const result = parseInlineContent(node);
expect(result.text).toBe("normal bold plain");
expect(result.marks).toEqual([{ type: "bold", start: 6, end: 17 }]);
expect(result.marks).not.toContainEqual(
expect.objectContaining({ type: "underline" }),
);
});
});
Loading
Loading