Skip to content

fix: remap chunk results after skipped messages - #283

Open
Kevin Turcios (KRRT7) wants to merge 5 commits into
microsoft:mainfrom
KRRT7:review-changes-pr277
Open

Kevin Turcios (KRRT7) wants to merge 5 commits into
microsoft:mainfrom
KRRT7:review-changes-pr277

Conversation

@KRRT7

@KRRT7 Kevin Turcios (KRRT7) commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Remap source-stream chunk locations to consecutive storage message ordinals when failed messages are skipped.
  • Preserve correct chunk embedding and semantic-reference locations for messages committed after a skipped message.
  • Add end-to-end coverage for a failed message followed by a successful one.
  • Retain regression coverage for graceful shutdown with a full chunk queue.

Validation

  • make format check test
  • GitHub CI and CodeQL

…Result docstring

The success property was removed in a previous commit but the docstring
still referenced it, causing confusion.
Propagate shutdown_event to dispatcher for coordinated pipeline shutdown.
When shutdown is requested, skip remaining chunks instead of processing them
after the producer has stopped.
@KRRT7
Kevin Turcios (KRRT7) force-pushed the review-changes-pr277 branch 2 times, most recently from 30d6b02 to 004ed74 Compare September 19, 2026 02:37
@KRRT7 Kevin Turcios (KRRT7) changed the title fix: add graceful shutdown support and prevent producer deadlock in add_messages pipeline test: cover graceful shutdown with a full chunk queue Sep 19, 2026
@KRRT7 Kevin Turcios (KRRT7) changed the title test: cover graceful shutdown with a full chunk queue fix: remap chunk results after skipped messages Sep 19, 2026
@KRRT7

Copy link
Copy Markdown
Contributor Author

Bernhard Merkle (@bmerkle) robgruen if you wouldn't mind reviewing, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant