Aeron integration - #29
Open
hrstoyanov wants to merge 11 commits into
Open
hrstoyanov wants to merge 11 commits into
hrstoyanov wants to merge 11 commits into
Conversation
Adds scoped view transition support (#1157) * Add `viewTransitionSelector` * Clean up * Remove unused args * Add bundles * Add to ADR fix: only delete old records with automatic backups (eclipse-datagrid#26) Prevent triggering the record deletion when a manual backup request comes from the user. fixing #405 and #414 (#277) fixing #405 and #414 by sync on object registry only fix: correct DynamoDB query and scan pagination to prevent data trunc… (#680) * fix: correct DynamoDB query and scan pagination to prevent data truncation Use `queryPaginator` and `scanPaginator` to iterate all result pages instead of single-page responses, preventing silent data loss when multi-blob files or directory listings exceed 1 MB page limits. Also fix batch reading logic by removing BufferedInputStream wrapper that caused up to 8 KiB data loss at blob boundaries, and ensure offset advancement during multi-read operations. * fix: eagerly drain DynamoDB scan paginator to prevent ResourceNotFoundException leaks Change from lazy stream to eager list collection within try block to ensure ResourceNotFoundException is caught consistently across all pages, regardless of SDK paginator fetch behavior. Replaces ScanIterable with ScanResponse and removes nested stream operations. Fix test compilation errors Bump patch version Fix repo links (#2376) * udpate link * update links
…ints, lifecycle safety, strict topology validation, failure propagation, and focused tests/docs. Harden Aeron crash testing and Kafka replication durability, validation, and failure handling.
…ience, crash coverage, and performance benchmarks.
Contributor
|
Hello and thank you for your pull request. I will be able to take a look at this in the upcoming weeks. |
Author
|
Thnaks @m-stoer I also have forked a much slimmer version in the "main" branch of https://github.com/peruncs/datagrid that is Aeron-only and with some layers removed - it is closer to being just an embeddable library, far less external assumptions (quartz, http backups, java web servers). I have also added comprehensive cluster tests, although the Kafka version and cache in general may need more. I am actively improving the slimmed down version as I need it soon |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@fh-ms @m-stoer @hg-ms @zdenek-jonas
This is Aeron implementation alternative to the exiting Kafka . The motivation for this is here:
#16
The changes are significant, but the core and Kafka are minimally touched. (if at all). I also added comprehensive integration tests .