Skip to content

Unescape regular-language matches only once - #2135

Open
FanWu-ai wants to merge 1 commit into
prompt-toolkit:mainfrom
FanWu-ai:fix-grammar-double-unescape-20261011
Open

FanWu-ai wants to merge 1 commit into
prompt-toolkit:mainfrom
FanWu-ai:fix-grammar-double-unescape-20261011

Conversation

@FanWu-ai

Copy link
Copy Markdown

Match.end_nodes() and Match.variables() already decode matched variables, but GrammarCompleter and GrammarValidator unescape those values again. Non-idempotent codecs therefore lose literal backslashes or decode percent sequences twice. This can remove valid completions and give validators different text from the matched argument.

Use each MatchVariable's already-decoded value directly. Keep raw replacement spans and final completion escaping unchanged. Add completion and validation regressions for literal/repeated backslashes, escaped spaces, plain text, and text following the cursor.

Validation on Linux/Python 3.12.14: pristine suite 156 passed; nine regression cases fail on original source; final complete suite 171 passed. Ruff lint/format, typos, strict Mypy for all three configured platforms, all 12 pinned hooks, builds, strict package metadata checks, and diff checks pass. An additional 3,063 shell/URL/JSON codec checks pass. Standard docs build retains two existing mocked-Windows-module warnings.

AI assistance was used to investigate the bug and prepare the implementation and tests. Other native interpreter/OS combinations and hosted checks have not been verified here.

AI assistance: OpenAI Codex was used for investigation, implementation, regression tests, and automated validation.
@FanWu-ai
FanWu-ai marked this pull request as ready for review October 11, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant