Feature / save long-running sim outputs to cache. Manager preempts sending identical work in favor of pulling from cache - #1585
Merged
Merged
Conversation
jlnav
changed the title
jlnav
marked this pull request as ready for review
September 19, 2025 20:57
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #1585 +/- ##
===========================================
+ Coverage 88.43% 88.77% +0.33%
===========================================
Files 73 73
Lines 8353 8658 +305
Branches 1330 1402 +72
===========================================
+ Hits 7387 7686 +299
+ Misses 731 725 -6
- Partials 235 247 +12 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
jlnav
force-pushed
the
feature/shelve_sims
branch
from
September 8, 2026 15:48
8762847 to
e88228e
Compare
jlnav
force-pushed
the
feature/shelve_sims
branch
from
September 24, 2026 20:35
68dbe87 to
f8e727d
Compare
…or f) out of a given update-index to file.
… and then seeing if values exist in that cache before sending sims...
…he cache being created. iterate over the cache_keys corresponding to those in gen_specs out, then check if they're close to a cache entry. grab those indexes so we can slot in the corresponding data into the manager's H later on
…ve determined points_to_evaluate
…istory_f on/with those indexes and the associated cache values, pretending that the array of cache-values are a worker message
…rrides .recv, and in _send_work_order that forms a buffer from corresponding cache entries
- Combined HEAD's early return with incoming worker-specific cache retrieval - Moved cache handling to top of _receive_from_workers - Added workerID, calc_status, and calc_type to cache D_recv
Use _update_state_from_cache helper method from incoming commit Add self.cache_index initialization in __init__
…asily accessible args
- Added self.use_cache from libE_specs cache_long_sims - Moved cache init into use_cache conditional - Removed duplicate cache_index initialization
- Added calc_status_strings import alongside existing imports
…nce we're comparing to returned calc statuses
…opying the cache values into the manager's local record
…e local record. most clarifying comments
…nstead of saving/loading each sim. move cache location to HOME/.cache/libensemble. Fix init issue. Fix redundant uniqueness checking
- Removed unused imports (launcher, Timer, os) since TCP was removed from base - Kept cache hashing logic (compute_config_hash, spec_hash) - Kept GC-only block from HEAD - Kept cache_name=None, cache_dir defaults from f268eda
…alues, callable-instance state, VOCS, and structured H0 data. The automatically generated filename uses the full SHA-256 hash. Fixed cache-hit bookkeeping and dispatch so multiple hits are retained, only matching rows are filtered, and remaining rows are sent normally. Added cache_tolerances, a field-name-to-absolute-tolerance dictionary. Unspecified numeric fields match exactly; relative tolerance is zero. Expanded test_cache_sims.py to check cached values, runtimes, and invalidation after changing the simulator. Updated the cache settings documentation. Fixed the cache-scan crash when the cache has no prior hits, and fixed cached results being filled from the outbound input fields instead of the cached result fields.
jlnav
force-pushed
the
feature/shelve_sims
branch
from
October 1, 2026 18:00
9716c03 to
673d62c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cache generated based on the hashed specs dicts + source codes of included callables