From 1d5a6a2e1753756a1fec49185baf502311826815 Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Sat, 29 Aug 2026 11:53:16 -0700 Subject: [PATCH 01/10] Add a prefetch model, its accuracy experiment, and the design behind both MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit HBF's case rests on the accelerator issuing a read before the data is needed. HBFSim models no prefetch at all, so the paper had no way to say what a prefetcher would be worth, and the question could not wait for a device-side implementation. This adds a deterministic model of prefetching, the sweep that turns it into a figure, and the document that says what the numbers may be used to claim. THE FIGURE. Prefetch accuracy on the x axis, modeled time on the y axis, one curve per media latency tR at 1, 2, 4, 5, 10 and 20 microseconds. Accuracy alone does not decide the time, because a correct prediction still arrives too late if it was issued too near its use, if bandwidth is short, or if the staging buffer dropped it. That objection is answered by the curve family rather than by a different x axis: how accurate shows on the x axis, whether it arrives in time shows as the spacing between curves and as whether a curve still falls near accuracy 1. THE MODEL, in src/prefetch/prefetch_model.cpp, is a discrete-event simulation over an access stream. Three policies: None, which is what HBFSim does today and is the baseline; NextPage, which fetches page N+1 on a demand for page N and carries no model of the workload; and Accuracy, which predicts the page the stream will demand `lead_distance` accesses later and is right with a stated probability, which is how the axis is swept without inventing a predictor. Four knobs, each a real limit: lead distance, staging capacity, media concurrency, and the accelerator time between accesses that a prefetch has to hide behind. The model is not a measurement of any predictor, GPU or device, and the artifacts carry that sentence in a `disclaimer` field. WHAT CAME OUT. The naive next-page policy reaches accuracy 0.99995 on a sequential stream, 0.00035 on a random one, and 0.99958 on a Mixture-of-Experts stream shaped like Qwen3-30B-A3B. Its accuracy on a Mixture-of-Experts stream is (P-1)/P for P pages per expert, measured at 0.029, 0.514, 0.757, 0.878 and 0.939 for P of 1, 2, 4, 8 and 16 against a theoretical 0.000, 0.500, 0.750, 0.875 and 0.938. At the real expert size -- 3 x 2048 x 768 parameters in bf16 is 9,437,184 bytes, or 2304 pages of 4 KiB -- one token is 884,736 page accesses. Without prefetch all 884,736 stall, for 12,386,304,000 ns modeled. With the naive policy 374 stall, for 6,194,648,000 ns, a speedup of 2.00. The 374 are the first page of each expert, against 48 layers x 8 experts = 384. So "expert routing cannot be predicted" is about which expert, not about how many bytes. The unpredictable part is 384 pages out of 884,736, which is 0.04%; the rest is reading forward inside an expert already chosen, and a prefetcher with no model at all covers it. The speedup stops at 2.00 because at that setting the media read is longer than the compute between accesses, so bandwidth and concurrency bound the result rather than accuracy -- which is the reason the figure needs the curve family. TESTS. tests/cpu/prefetch_model_test.cpp holds 12 property assertions, written before the implementation. They pin what the paper's latency argument depends on: accuracy 0 buys nothing, lead distance 0 hides nothing even at accuracy 1, time is monotone in accuracy, a one-page buffer loses a prefetch issued eight accesses early, and one media read at a time still stalls at accuracy 1. One assertion was wrong on its first run: it demanded no stall at all under a perfect prefetcher, and the model was right to disagree, because a prefetcher issuing L accesses ahead cannot cover the first L accesses. It now asserts the misses equal exactly that warm-up. The media servers use a min-heap rather than a scan so a large concurrency setting stays cheap on million-access streams. All 252 swept cells are identical before and after that change. benchmarks/prefetch/README.md lists every file, the build, and the commands that reproduce each result. docs/46-预取实验设计.md carries the design, and section 8.0 of docs/45-预取与延迟掩盖的核实.md records that its earlier recommendation against an accuracy x axis is superseded by this one, and that its Mixture-of-Experts section reads more pessimistically than these numbers support. Test suite: the same four tests fail before and after (context_lifecycle, vmem_tuning, run_with_bpftime, mqsim_benchmark), all pre-existing in a CPU-only build. Co-Authored-By: Claude Opus 5 (1M context) --- CMakeLists.txt | 8 + benchmarks/prefetch/README.md | 78 + benchmarks/prefetch/hbf_prefetch_bench.cpp | 222 + ...26\347\232\204\346\240\270\345\256\236.md" | 18 + ...36\351\252\214\350\256\276\350\256\241.md" | 151 + .../artifacts/prefetch-accuracy-sweep.csv | 253 ++ .../artifacts/prefetch-accuracy-sweep.json | 3681 +++++++++++++++++ include/hbfsim/prefetch_model.hpp | 114 + scripts/run_prefetch_accuracy_sweep.py | 152 + src/prefetch/prefetch_model.cpp | 253 ++ tests/cpu/prefetch_model_test.cpp | 211 + 11 files changed, 5141 insertions(+) create mode 100644 benchmarks/prefetch/README.md create mode 100644 benchmarks/prefetch/hbf_prefetch_bench.cpp create mode 100644 "docs/46-\351\242\204\345\217\226\345\256\236\351\252\214\350\256\276\350\256\241.md" create mode 100644 docs/proofs/artifacts/prefetch-accuracy-sweep.csv create mode 100644 docs/proofs/artifacts/prefetch-accuracy-sweep.json create mode 100644 include/hbfsim/prefetch_model.hpp create mode 100755 scripts/run_prefetch_accuracy_sweep.py create mode 100644 src/prefetch/prefetch_model.cpp create mode 100644 tests/cpu/prefetch_model_test.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 5fa1a53..5084b41 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -57,6 +57,7 @@ add_library( src/cuda_runtime/timing_binding.cpp src/cuda_runtime/vmm.cpp src/hybrid/calibrator.cpp + src/prefetch/prefetch_model.cpp src/profile/profile.cpp src/host_service/request_dispatcher.cpp src/host_service/backing_store.cpp @@ -108,6 +109,13 @@ target_link_libraries(hbfsim_profile_tests PRIVATE hbfsim_core) add_test(NAME profile COMMAND hbfsim_profile_tests) set_tests_properties(profile PROPERTIES WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}") +add_executable(hbf_prefetch_bench benchmarks/prefetch/hbf_prefetch_bench.cpp) +target_link_libraries(hbf_prefetch_bench PRIVATE hbfsim_core) + +add_executable(prefetch_model_test tests/cpu/prefetch_model_test.cpp) +target_link_libraries(prefetch_model_test PRIVATE hbfsim_core) +add_test(NAME prefetch_model COMMAND prefetch_model_test) + add_executable(hbfsim_calibrator_tests tests/cpu/calibrator_test.cpp) target_link_libraries(hbfsim_calibrator_tests PRIVATE hbfsim_core) add_test(NAME calibrator COMMAND hbfsim_calibrator_tests) diff --git a/benchmarks/prefetch/README.md b/benchmarks/prefetch/README.md new file mode 100644 index 0000000..b6e8ed1 --- /dev/null +++ b/benchmarks/prefetch/README.md @@ -0,0 +1,78 @@ +# Prefetch experiment: what to run and where everything is + +The reasoning behind the experiment, and what its numbers may and may not be +used to claim, is in `docs/46-预取实验设计.md`. This file is only the map and +the commands. + +## Files + +| Path | What it is | +|---|---| +| `include/hbfsim/prefetch_model.hpp` | The model's interface, with each knob's meaning | +| `src/prefetch/prefetch_model.cpp` | The model: a deterministic discrete-event simulation | +| `tests/cpu/prefetch_model_test.cpp` | 12 property assertions, written before the implementation | +| `benchmarks/prefetch/hbf_prefetch_bench.cpp` | Sweeps one access stream and prints JSON | +| `scripts/run_prefetch_accuracy_sweep.py` | Runs all three streams and writes the artifact and a CSV | +| `docs/proofs/artifacts/prefetch-accuracy-sweep.json` | The swept cells | +| `docs/proofs/artifacts/prefetch-accuracy-sweep.csv` | The same cells, flat, for plotting | + +## Build + +The model and its test are CPU-only. Neither needs a GPU or `nvcc`. + +``` +cmake -S . -B build -DHBFSIM_ENABLE_CUDA=OFF -DHBFSIM_ENABLE_MQSIM=ON \ + -DCMAKE_BUILD_TYPE=Release +cmake --build build -j"$(nproc)" --target prefetch_model_test hbf_prefetch_bench +``` + +## Run the tests + +``` +./build/prefetch_model_test # exit code 0, prints nothing on success +``` + +On failure it prints the line number and the assertion that failed. + +## Reproduce the sweep + +``` +python3 scripts/run_prefetch_accuracy_sweep.py --build-dir build +``` + +Writes `docs/proofs/artifacts/prefetch-accuracy-sweep.json` and the matching +`.csv`. Both carry a `disclaimer` field reading +`modeled, not measured on any device or GPU`. + +## One stream at a time + +``` +./build/hbf_prefetch_bench --stream sequential --accesses 20000 +./build/hbf_prefetch_bench --stream random --accesses 20000 +./build/hbf_prefetch_bench --stream moe --accesses 20000 \ + --pages-per-expert 2304 +``` + +`--pages-per-expert 2304` is one Qwen3-30B-A3B expert: 3 x 2048 x 768 +parameters in bf16 is 9,437,184 bytes, which is 2304 pages of 4 KiB. + +Other options: `--compute-ns` (accelerator time between two accesses, the +interval a prefetch hides behind), `--lead` (how many accesses ahead a prefetch +is issued), `--buffer-pages`, `--max-in-flight`, `--seed`. + +## Reproducing the two results the paper leans on + +The naive next-page policy's accuracy on a Mixture-of-Experts stream is +(P-1)/P, where P is how many pages one expert occupies: + +``` +for p in 1 2 4 8 16; do + ./build/hbf_prefetch_bench --stream moe --accesses 20000 --pages-per-expert $p +done +``` + +One token of Qwen3-30B-A3B, which is 884,736 page accesses: + +``` +./build/hbf_prefetch_bench --stream moe --accesses 20000 --pages-per-expert 2304 +``` diff --git a/benchmarks/prefetch/hbf_prefetch_bench.cpp b/benchmarks/prefetch/hbf_prefetch_bench.cpp new file mode 100644 index 0000000..8bf627c --- /dev/null +++ b/benchmarks/prefetch/hbf_prefetch_bench.cpp @@ -0,0 +1,222 @@ +// Sweeps prefetch accuracy against media latency and writes the cells the +// accuracy figure is drawn from. +// +// The figure this feeds has prefetch accuracy on the x axis and modeled time +// on the y axis, one curve per media latency. Two reference points are +// reported alongside every curve: the no-prefetch baseline, which is what +// HBFSim models today, and the next-page policy, which carries no model of the +// workload at all and shows what accuracy a prefetcher reaches for free. +// +// Every number this prints comes out of the model in +// src/prefetch/prefetch_model.cpp. None of it is a hardware measurement. + +#include + +#include +#include +#include +#include +#include +#include + +namespace { + +struct Options { + std::string stream{"sequential"}; + std::uint64_t accesses{20'000}; + std::uint64_t compute_ns_per_access{4'000}; + std::uint32_t lead_distance{8}; + std::uint32_t buffer_pages{64}; + std::uint32_t max_in_flight{32}; + std::uint64_t seed{7}; + // Mixture-of-Experts stream shape. The defaults follow Qwen3-30B-A3B: + // 48 layers, 128 experts per layer, 8 activated per token. + std::uint64_t layers{48}; + std::uint64_t experts_per_layer{128}; + std::uint64_t experts_per_token{8}; + std::uint64_t pages_per_expert{2}; +}; + +// The six media latencies the experiment list fixes: 1, 2, 4, 5, 10, 20 us. +const std::vector kReadLatencyNs{1'000, 2'000, 4'000, + 5'000, 10'000, 20'000}; + +const std::vector kAccuracies{0.0, 0.1, 0.2, 0.3, 0.4, 0.5, + 0.6, 0.7, 0.8, 0.9, 0.95, 1.0}; + +std::vector build_stream(const Options& options) +{ + if (options.stream == "random") { + return hbfsim::make_random_stream(options.accesses, + options.accesses * 8, options.seed); + } + if (options.stream == "moe") { + const auto per_token = options.layers * options.experts_per_token * + options.pages_per_expert; + const auto tokens = + per_token == 0 ? 0 : std::max(1, options.accesses / + per_token); + return hbfsim::make_moe_stream(tokens, options.layers, + options.experts_per_layer, + options.experts_per_token, + options.pages_per_expert, options.seed); + } + return hbfsim::make_sequential_stream(options.accesses); +} + +void print_cell(const char* policy, double requested_accuracy, + std::uint64_t read_latency_ns, + const hbfsim::PrefetchStats& stats, + std::uint64_t baseline_ns, std::uint64_t compute_floor_ns, + bool last) +{ + // recovered_fraction is how much of the gap between doing nothing and the + // compute-bound floor this configuration closed. 0 means it bought + // nothing; 1 means the media cost disappeared. + const auto span = baseline_ns > compute_floor_ns + ? baseline_ns - compute_floor_ns + : 0; + const double recovered = + span == 0 ? 0.0 + : static_cast(baseline_ns - stats.total_ns) / + static_cast(span); + std::printf( + " {\"policy\": \"%s\", \"requested_accuracy\": %.4f, " + "\"read_latency_ns\": %llu, \"total_ns\": %llu, \"stall_ns\": %llu, " + "\"demand_misses\": %llu, \"prefetch_issued\": %llu, " + "\"prefetch_hits\": %llu, \"prefetch_wasted\": %llu, " + "\"achieved_accuracy\": %.6f, \"speedup_over_no_prefetch\": %.6f, " + "\"recovered_fraction\": %.6f}%s\n", + policy, requested_accuracy, + static_cast(read_latency_ns), + static_cast(stats.total_ns), + static_cast(stats.stall_ns), + static_cast(stats.demand_misses), + static_cast(stats.prefetch_issued), + static_cast(stats.prefetch_hits), + static_cast(stats.prefetch_wasted), + stats.achieved_accuracy(), + stats.total_ns == 0 + ? 0.0 + : static_cast(baseline_ns) / + static_cast(stats.total_ns), + recovered, last ? "" : ","); +} + +bool read_u64(int argc, char** argv, int& index, const char* name, + std::uint64_t& out) +{ + if (std::strcmp(argv[index], name) != 0) { + return false; + } + if (index + 1 >= argc) { + std::fprintf(stderr, "hbf_prefetch_bench: missing value for %s\n", + name); + std::exit(64); + } + out = std::strtoull(argv[++index], nullptr, 10); + return true; +} + +} // namespace + +int main(int argc, char** argv) +{ + Options options{}; + std::uint64_t scratch = 0; + for (int index = 1; index < argc; ++index) { + if (std::strcmp(argv[index], "--stream") == 0 && index + 1 < argc) { + options.stream = argv[++index]; + } else if (read_u64(argc, argv, index, "--accesses", scratch)) { + options.accesses = scratch; + } else if (read_u64(argc, argv, index, "--compute-ns", scratch)) { + options.compute_ns_per_access = scratch; + } else if (read_u64(argc, argv, index, "--lead", scratch)) { + options.lead_distance = static_cast(scratch); + } else if (read_u64(argc, argv, index, "--buffer-pages", scratch)) { + options.buffer_pages = static_cast(scratch); + } else if (read_u64(argc, argv, index, "--max-in-flight", scratch)) { + options.max_in_flight = static_cast(scratch); + } else if (read_u64(argc, argv, index, "--seed", scratch)) { + options.seed = scratch; + } else if (read_u64(argc, argv, index, "--pages-per-expert", + scratch)) { + options.pages_per_expert = scratch; + } else { + std::fprintf(stderr, + "usage: hbf_prefetch_bench [--stream " + "sequential|random|moe] [--accesses N] " + "[--compute-ns N] [--lead N] [--buffer-pages N] " + "[--max-in-flight N] [--seed N] " + "[--pages-per-expert N]\n"); + return 64; + } + } + + const auto pages = build_stream(options); + if (pages.empty()) { + std::fprintf(stderr, "hbf_prefetch_bench: empty access stream\n"); + return 65; + } + + hbfsim::PrefetchConfig config{}; + config.lead_distance = options.lead_distance; + config.buffer_pages = options.buffer_pages; + config.max_in_flight = options.max_in_flight; + config.compute_ns_per_access = options.compute_ns_per_access; + config.seed = options.seed; + + const auto compute_floor_ns = + static_cast(pages.size()) * + options.compute_ns_per_access; + + std::printf("{\n"); + std::printf(" \"schema_version\": 1,\n"); + std::printf(" \"model\": \"src/prefetch/prefetch_model.cpp\",\n"); + std::printf( + " \"disclaimer\": \"modeled, not measured on any device or GPU\",\n"); + std::printf( + " \"workload\": {\"stream\": \"%s\", \"accesses\": %llu, " + "\"compute_ns_per_access\": %llu, \"lead_distance\": %u, " + "\"buffer_pages\": %u, \"max_in_flight\": %u, \"seed\": %llu},\n", + options.stream.c_str(), + static_cast(pages.size()), + static_cast(options.compute_ns_per_access), + options.lead_distance, options.buffer_pages, options.max_in_flight, + static_cast(options.seed)); + std::printf(" \"compute_floor_ns\": %llu,\n", + static_cast(compute_floor_ns)); + std::printf(" \"cells\": [\n"); + + bool first = true; + for (const auto read_latency_ns : kReadLatencyNs) { + config.read_latency_ns = read_latency_ns; + + config.policy = hbfsim::PrefetchPolicy::None; + const auto baseline = hbfsim::simulate_prefetch(pages, config); + if (!first) { + std::printf(",\n"); + } + first = false; + print_cell("none", 0.0, read_latency_ns, baseline, baseline.total_ns, + compute_floor_ns, true); + + config.policy = hbfsim::PrefetchPolicy::NextPage; + const auto naive = hbfsim::simulate_prefetch(pages, config); + std::printf(",\n"); + print_cell("next_page", naive.achieved_accuracy(), read_latency_ns, + naive, baseline.total_ns, compute_floor_ns, true); + + config.policy = hbfsim::PrefetchPolicy::Accuracy; + for (const auto accuracy : kAccuracies) { + config.accuracy = accuracy; + const auto stats = hbfsim::simulate_prefetch(pages, config); + std::printf(",\n"); + print_cell("accuracy", accuracy, read_latency_ns, stats, + baseline.total_ns, compute_floor_ns, true); + } + } + + std::printf("\n ]\n}\n"); + return 0; +} diff --git "a/docs/45-\351\242\204\345\217\226\344\270\216\345\273\266\350\277\237\346\216\251\347\233\226\347\232\204\346\240\270\345\256\236.md" "b/docs/45-\351\242\204\345\217\226\344\270\216\345\273\266\350\277\237\346\216\251\347\233\226\347\232\204\346\240\270\345\256\236.md" index c141299..643a2a6 100644 --- "a/docs/45-\351\242\204\345\217\226\344\270\216\345\273\266\350\277\237\346\216\251\347\233\226\347\232\204\346\240\270\345\256\236.md" +++ "b/docs/45-\351\242\204\345\217\226\344\270\216\345\273\266\350\277\237\346\216\251\347\233\226\347\232\204\346\240\270\345\256\236.md" @@ -417,6 +417,24 @@ JAX 官方的 TPU 流水线文档（https://docs.jax.dev/en/latest/pallas/tpu/pi ## 8. 论文要不要做预取成功率的图 +### 8.0 本节写成之后的进展：预取模型与实验已经做出来了 + +这一节原本只提建议。建议之后已经落地,详细设计写在 `docs/46-预取实验设计.md`,代码在 +`include/hbfsim/prefetch_model.hpp`、`src/prefetch/prefetch_model.cpp`,测试在 +`tests/cpu/prefetch_model_test.cpp`,扫描脚本是 `scripts/run_prefetch_accuracy_sweep.py`。 + +**落地之后有一处结论要修正,以本小节为准。** 下面 8.1 写的是"主图不该是命中率曲线"。实际做出来之后, +主图的横坐标就是预取正确率,纵坐标是建模时间,靠"每条曲线固定一个介质延迟 tR"来把"来不来得及"这一维 +放进同一张图。改口的理由写在 `docs/46-预取实验设计.md` 的第二节。8.1 里关于 +`GoodPred = Accuracy x FetchRate` 的那条理由仍然成立,它说明的是为什么一条曲线不够,不是为什么横坐标 +不能是正确率。 + +**同时修正第 6.3 节给读者的印象。** 第 6.3 节列了一批证据说明混合专家模型上预取困难。做出实验之后可以 +说得更准:困难的是"哪一个专家",不是"多少字节"。按 Qwen3-30B-A3B 的形状(48 层、每层 128 个专家、 +每词元激活 8 个、每个专家 2304 个 4 KiB 页)生成的访问序列,一个词元 884,736 次页访问里,真正无法提前 +知道的只有每个专家的第一页,共 384 页;一个不带任何模型的 next-page 预取器实测正确率 0.99958、只剩 +374 次落空。完整数字见 `docs/46-预取实验设计.md`。 + ### 8.1 建议：**要图，但主图不是命中率曲线** **推荐的主图（一张，两个坐标轴 + 一族曲线）：** diff --git "a/docs/46-\351\242\204\345\217\226\345\256\236\351\252\214\350\256\276\350\256\241.md" "b/docs/46-\351\242\204\345\217\226\345\256\236\351\252\214\350\256\276\350\256\241.md" new file mode 100644 index 0000000..c46ef95 --- /dev/null +++ "b/docs/46-\351\242\204\345\217\226\345\256\236\351\252\214\350\256\276\350\256\241.md" @@ -0,0 +1,151 @@ +# 预取模型、预取实验怎么设计、已经跑出了什么 + +**这份文档属于哪一类:** 给合作者与导师参考。它说三件事:我们给模拟器加了什么预取模型、预取实验按什么设计、已经跑出来的结果是什么。同一个话题的另一份文档是 `docs/45-预取与延迟掩盖的核实.md`,那一份核实的是 HBFSim 现在有没有预取、以及已发表工作的说法;这一份记的是我们新加的模型与实验。 + +先说三个术语,后面一直用同一个词。**预取(prefetch)** 指在数据真正被用到之前就把读发出去,让等待发生在关键路径之外;与之相对的是碰到才读,等待留在关键路径上。**预取正确率(accuracy)** 指发出去的预取里,有多大比例预测的页号确实是后面真正会被访问的那一页。**tR** 指闪存把一页数据从存储阵列读出来所需的介质时间,本文里取值范围是 1 微秒到 20 微秒。 + +## 一句话结论 + +主图就按"横坐标是预取正确率、纵坐标是时间"来画,每条曲线固定一个 tR,一共六条曲线。 + +## 为什么横坐标就该是预取正确率 + +**"横坐标是预取正确率、纵坐标是时间"这个提法是对的,主图就该这么画。** 之前对这个提法提出过一条反对意见,内容是:预取正确率这一个数不能单独决定时间。这条反对意见只在一个很窄的地方成立——一次预取即使猜对了页号,如果发得太晚、或者带宽不够、或者取回来的页暂存放不下,数据仍然来不及在被用到之前到达。 + +**这条反对意见的解决办法不是换掉横坐标,是在同一张图上多画几条曲线。** 每条曲线固定一个 tR,横坐标仍然是预取正确率。这样"猜得准不准"就是横轴本身,"来不来得及"表现为两件可以直接看出来的事:曲线与曲线之间的间距,以及曲线在正确率接近 1 的那一端还降不降得下去。一张图同时回答两个问题,不需要第二张图。 + +**曲线在右端不再下降,说明那一档的瓶颈已经不是预取正确率了。** 此时限制时间的是介质带宽与同时在途的读请求数,再提高正确率也不会更快。这一点在已有结果那一节有一个具体的数(加速停在 2.00 倍)。 + +## 模型是什么 + +新增两个文件:`include/hbfsim/prefetch_model.hpp` 与 `src/prefetch/prefetch_model.cpp`。 + +这是一个**确定性的离散事件模型**。离散事件模型的意思是:把整个过程记成一串带时间戳的事件(一次预取发出、一次介质读完成、一次访问开始等待),按时间先后依次处理,不做随机抽样。确定性的意思是:同一组参数跑两次,结果完全一样。它跑在 CPU 上,不需要 GPU,也不需要 nvcc。 + +它建模的对象是三样东西:一串按顺序发生的页访问,一个预取器,一个介质。 + +### 三个策略 + +- **`None`**:什么都不预取。**这就是 HBFSim 今天的行为**,是其它两个策略的对照基线。 +- **`NextPage`**:访问第 N 页时,预取第 N+1 页。**不带任何关于负载的模型,不需要训练**,是最朴素的算法。 +- **`Accuracy`**:预测"再过 lead_distance 次访问会用到哪一页",以概率 `accuracy` 猜对。**主图的横坐标就是靠这个策略扫出来的**,不需要我们自己发明一个预测器。 + +### 四个参数,每个对应一个真实的物理限制 + +- **`lead_distance`**:提前多少次访问发出预取。**取 0 时等于在用到的那一刻才发出,没有任何介质时间可以被重叠掉。** +- **`buffer_pages`**:取回来还没被用掉的页能存多少。发得太早,页会在被用到之前就被后来的页挤出去。 +- **`max_in_flight`**:介质同时能服务多少个读。 +- **`compute_ns_per_access`**:两次访问之间加速器(GPU 或 TPU)在做计算,这段计算时间就是介质读可以与之重叠的那段时间;设成 0 就没有任何计算可以与介质读重叠。 + +## 这个模型不声称什么 + +**它不是任何真实预测器的测量,不是任何 GPU 的测量,也不是任何器件的测量。** 它回答的是另一个问题:一个正确率为 p、提前量为 L 的预取器,面对 tR 这么大的介质延迟,值多少。做法是把论文里关于延迟的那一段论证按参数逐项算出来,并在参数网格上扫一遍;结果是模型算出来的,不是硬件跑出来的。 + +产出的 JSON 里有一个 `disclaimer` 字段,原样写着 `modeled, not measured on any device or GPU`。 + +## 测试:先写测试,再写实现 + +文件是 `tests/cpu/prefetch_model_test.cpp`,一共 12 条性质断言,全部通过。这些断言是按 TDD 的做法写的,也就是先把测试写出来、再去写实现。下面说的**停顿**,指一次访问在数据还没到达时必须停下来等介质读完。 + +- 正确率为 0 时,一次都不命中,取回来的页全部浪费,总时间不比完全不预取更短。 +- 正确率为 1、提前量和暂存都够时,除了开头的预热之外没有任何停顿。预热是不可避免的:提前 L 次发出的预取覆盖不到最前面的 L 次访问,因为在这 L 次访问之前没有任何一次预取被发出过。 +- **提前量为 0 时,即使正确率是 100% 也没有任何介质时间被重叠掉。** 这一条是论文不能丢的那一点:知道地址不等于有时间用这个地址。 +- 时间对正确率单调:更准永远不会更慢。 +- 暂存只有 1 页时,提前 8 次发出的预取会在被用到之前挤掉,所以比暂存 64 页慢。**容量是真实限制,不是形式。** +- 同时在途只有 1 个读时,即使正确率是 100% 仍然有停顿。**并发是另一个真实限制。** + +**其中第 4 条断言第一次是写错的。** 最初写的是"完全没有停顿",跑出来失败;查明原因是预热的那 L 次访问必然落在预取覆盖范围之外,**错的是断言不是实现**,改成"停顿恰好等于预热的 L 次"之后通过。这一条写进文档,因为这正是先写测试该起的作用。 + +## 实验怎么跑 + +驱动程序是 `benchmarks/prefetch/hbf_prefetch_bench.cpp`,输出 JSON。扫描脚本是 `scripts/run_prefetch_accuracy_sweep.py`,输出 `docs/proofs/artifacts/prefetch-accuracy-sweep.json` 与同名的 `.csv`。 + +扫描的三个维度: + +- **介质延迟 tR 取六档:** 1、2、4、5、10、20 微秒。 +- **预取正确率取十二个点:** 0、0.1、0.2、0.3、0.4、0.5、0.6、0.7、0.8、0.9、0.95、1.0。 +- **访问序列取三种:** `sequential`、`random`、`moe`。 + +`moe` 这一种指的是混合专家模型(Mixture-of-Experts,MoE)的访问序列。混合专家模型的每一层有很多组权重,每组叫一个专家,每个词元只用其中几组,用哪几组由运行时的路由决定。**`moe` 序列的形状按 Qwen3-30B-A3B 取:48 层、每层 128 个专家、每个词元激活 8 个。** 每层由路由在运行时挑专家,所以哪几个专家的页要读,在这一层算完之前算不出来。 + +一个专家有多大:3 × 2048 × 768 个参数,bf16(一种 16 位浮点数格式),即 **9,437,184 字节**,也就是 **2304 个 4 KiB 页**。 + +## 已经跑出来的结果 + +下面四组全部是本轮实跑的结果。 + +### 第一组:最朴素的 next-page 策略在三种序列上的表现 + +tR 取 20,000 ns 那一档。"剩下的停顿次数"指没有被预取覆盖到、必须停下来等介质的访问次数。 + +| 访问序列 | 正确率 | 加速 | 剩下的停顿次数 | +|---|---:|---:|---:| +| `sequential` | 0.99995 | 2.00 倍 | 1 | +| `random` | 0.00035 | 1.00 倍 | 19,993 | +| `moe` | 0.99958 | 2.00 倍 | 374 | + +### 第二组:朴素策略在 MoE 序列上的正确率,严格等于 (P−1)/P + +P 是一个专家占多少页。实测与理论值: + +| P(一个专家占的页数) | 实测正确率 | 理论值 (P−1)/P | +|---:|---:|---:| +| 1 | 0.029 | 0.000 | +| 2 | 0.514 | 0.500 | +| 4 | 0.757 | 0.750 | +| 8 | 0.878 | 0.875 | +| 16 | 0.939 | 0.938 | + +### 第三组:把 P 取成真实的 2304,跑一个词元 + +一个专家占 2304 个 4 KiB 页,这是 Qwen3-30B-A3B 的真实值。一个词元的序列长度是 **884,736** 次页访问(= 48 层 × 8 专家 × 2304 页)。这一组的参数是 tR = 10,000 ns、每次访问的计算时间 4,000 ns。 + +| 策略 | 正确率 | 停顿次数 | 建模总时间 | 加速 | +|---|---:|---:|---:|---:| +| 不预取 | — | 884,736 | 12,386,304,000 ns | — | +| 朴素的 next-page 预取 | 0.99958 | 374 | 6,194,648,000 ns | 2.00 倍 | + +374 这个数对应的是 48 × 8 = **384** 个"每个专家的第一页";实测 374 与 384 之间的差额来自预热,以及页号相邻造成的重合。 + +### 这一组要写成结论,而且要写得显眼 + +**「专家路由不可预测」这句话管的是哪一个专家,不是多少字节。** 一个词元要读的 884,736 页里,真正无法提前知道的只有每个专家的第一页,一共 384 页,占 **0.04%**。剩下 99.96% 是在选定的专家内部顺序往下读,**一个不带任何模型的预取器就能覆盖**。 + +**所以 MoE 对预取的伤害,是每个词元多付 384 次介质延迟,不是多付 884,736 次。** + +### 第四组:为什么加速停在 2.00 倍而不是更多 + +因为这一档里每次访问的计算时间是 4,000 ns,而介质读是 10,000 ns,计算时间比介质读还短。所以就算预取全部猜对,总时间的上限也由带宽与并发决定,不是由正确率不够决定。 + +**这一点正好说明为什么图上要有多条 tR 曲线:曲线在正确率接近 1 的那一端不再下降,就说明这一档的瓶颈已经不是猜得准不准了。** + +## 图怎么画 + +- **主图:** 横坐标是预取正确率,从 0 到 1;纵坐标是建模总时间,或者相对不预取的加速比。**每条曲线固定一个 tR**,一共六条。 +- **每条曲线上标两个参考点:** 不预取的基线(横坐标 0),以及朴素 next-page 策略实际达到的正确率所在的那一点。 +- **在横轴上用竖线标出已发表的专家预测器的准确率**,并注明这些是别人论文里的数、不是我们跑出来的。 + +| 系统 | 准确率 | 出处 | +|---|---:|---| +| ProMoE 跨层门式,在 Qwen2-MoE 上 | 66.9% | arXiv:2410.22134 | +| Fate,不训练 | 78.79% | arXiv:2502.12224 | +| DAOP,在 Mixtral 8x7B 上 | 84.11% | arXiv:2501.10375 | +| ProMoE 学习式,平均 | 84.7% | arXiv:2410.22134 | +| AdapMoE | 90% | arXiv:2408.10284 | +| HOBBIT,下一层 top-1 | 96% | arXiv:2411.01433 | +| Fate 过取版 | 97.15% | arXiv:2502.12224,代价约 3.75 倍权重流量 | + +### 这张图必须点出来的一种读法 + +朴素的 next-page 策略在 MoE 序列上达到的 0.99958,比上表里每一个已发表的预测器都高。**原因不是我们的预测器更聪明。** 原因是两件事的难度差得远:上表那些系统预测的是"哪个专家",而 next-page 策略预测的是"选定的专家内部的下一页"。 + +这个区别必须在图注和正文里讲清楚,**不能让读者以为我们做出了一个比 HOBBIT 更强的专家预测器**。我们没有做专家预测器。 + +## 还差什么(论文投稿截止是 2026-09-15 23:59 AoE) + +这个模型没做的三件事,按补起来的难度从大到小排: + +1. **这是请求层面的模型,没有在真实 GPU 执行路径上发出过一次真正的提前读。** 要接到真实执行上,需要改设备端代码与 PTX 改写,那是另一件工作。 +2. **`compute_ns_per_access` 现在是一个参数,不是从真实负载测出来的。** 要让结论落到某个具体模型上,得用那个模型真实的每层计算时间。 +3. **`moe` 序列是按 Qwen3-30B-A3B 的形状生成的合成序列,不是从真实 vLLM 运行里抓下来的访问序列。** 抓真实访问序列是下一步。 + diff --git a/docs/proofs/artifacts/prefetch-accuracy-sweep.csv b/docs/proofs/artifacts/prefetch-accuracy-sweep.csv new file mode 100644 index 0000000..a2501a1 --- /dev/null +++ b/docs/proofs/artifacts/prefetch-accuracy-sweep.csv @@ -0,0 +1,253 @@ +stream,policy,requested_accuracy,achieved_accuracy,read_latency_ns,total_ns,stall_ns,demand_misses,prefetch_issued,prefetch_hits,prefetch_wasted,speedup_over_no_prefetch,recovered_fraction +sequential,none,0.0,0.0,1000,100000000,20000000,20000,0,0,0,1.0,0.0 +sequential,next_page,1.0,0.99995,1000,80001000,1000,1,20000,19999,1,1.249984,0.99995 +sequential,accuracy,0.0,0.0,1000,100000000,20000000,20000,19992,0,19992,1.0,0.0 +sequential,accuracy,0.1,0.095438,1000,98092000,18092000,18092,19992,1908,18084,1.019451,0.0954 +sequential,accuracy,0.2,0.198179,1000,96038000,16038000,16038,19992,3962,16030,1.041255,0.1981 +sequential,accuracy,0.3,0.303621,1000,93930000,13930000,13930,19992,6070,13922,1.064623,0.3035 +sequential,accuracy,0.4,0.403561,1000,91932000,11932000,11932,19992,8068,11924,1.087761,0.4034 +sequential,accuracy,0.5,0.50095,1000,89985000,9985000,9985,19992,10015,9977,1.111296,0.50075 +sequential,accuracy,0.6,0.602291,1000,87959000,7959000,7959,19992,12041,7951,1.136893,0.60205 +sequential,accuracy,0.7,0.70118,1000,85982000,5982000,5982,19992,14018,5974,1.163034,0.7009 +sequential,accuracy,0.8,0.80087,1000,83989000,3989000,3989,19992,16011,3981,1.190632,0.80055 +sequential,accuracy,0.9,0.90051,1000,81997000,1997000,1997,19992,18003,1989,1.219557,0.90015 +sequential,accuracy,0.95,0.94978,1000,81012000,1012000,1012,19992,18988,1004,1.234385,0.9494 +sequential,accuracy,1.0,1.0,1000,80008000,8000,8,19992,19992,0,1.249875,0.9996 +sequential,none,0.0,0.0,2000,120000000,40000000,20000,0,0,0,1.0,0.0 +sequential,next_page,1.0,0.99995,2000,80002000,2000,1,20000,19999,1,1.499963,0.99995 +sequential,accuracy,0.0,0.0,2000,120000000,40000000,20000,19992,0,19992,1.0,0.0 +sequential,accuracy,0.1,0.095438,2000,116184000,36184000,18092,19992,1908,18084,1.032844,0.0954 +sequential,accuracy,0.2,0.198179,2000,112076000,32076000,16038,19992,3962,16030,1.070702,0.1981 +sequential,accuracy,0.3,0.303621,2000,107860000,27860000,13930,19992,6070,13922,1.112553,0.3035 +sequential,accuracy,0.4,0.403561,2000,103864000,23864000,11932,19992,8068,11924,1.155357,0.4034 +sequential,accuracy,0.5,0.50095,2000,99970000,19970000,9985,19992,10015,9977,1.20036,0.50075 +sequential,accuracy,0.6,0.602291,2000,95918000,15918000,7959,19992,12041,7951,1.251069,0.60205 +sequential,accuracy,0.7,0.70118,2000,91964000,11964000,5982,19992,14018,5974,1.304858,0.7009 +sequential,accuracy,0.8,0.80087,2000,87978000,7978000,3989,19992,16011,3981,1.363977,0.80055 +sequential,accuracy,0.9,0.90051,2000,83994000,3994000,1997,19992,18003,1989,1.428673,0.90015 +sequential,accuracy,0.95,0.94978,2000,82024000,2024000,1012,19992,18988,1004,1.462986,0.9494 +sequential,accuracy,1.0,1.0,2000,80016000,16000,8,19992,19992,0,1.4997,0.9996 +sequential,none,0.0,0.0,4000,160000000,80000000,20000,0,0,0,1.0,0.0 +sequential,next_page,1.0,0.99995,4000,80004000,4000,1,20000,19999,1,1.9999,0.99995 +sequential,accuracy,0.0,0.0,4000,160000000,80000000,20000,19992,0,19992,1.0,0.0 +sequential,accuracy,0.1,0.095438,4000,152368000,72368000,18092,19992,1908,18084,1.050089,0.0954 +sequential,accuracy,0.2,0.198179,4000,144152000,64152000,16038,19992,3962,16030,1.10994,0.1981 +sequential,accuracy,0.3,0.303621,4000,135720000,55720000,13930,19992,6070,13922,1.178898,0.3035 +sequential,accuracy,0.4,0.403561,4000,127728000,47728000,11932,19992,8068,11924,1.252662,0.4034 +sequential,accuracy,0.5,0.50095,4000,119940000,39940000,9985,19992,10015,9977,1.334,0.50075 +sequential,accuracy,0.6,0.602291,4000,111836000,31836000,7959,19992,12041,7951,1.430666,0.60205 +sequential,accuracy,0.7,0.70118,4000,103928000,23928000,5982,19992,14018,5974,1.539527,0.7009 +sequential,accuracy,0.8,0.80087,4000,95956000,15956000,3989,19992,16011,3981,1.667431,0.80055 +sequential,accuracy,0.9,0.90051,4000,87988000,7988000,1997,19992,18003,1989,1.81843,0.90015 +sequential,accuracy,0.95,0.94978,4000,84048000,4048000,1012,19992,18988,1004,1.903674,0.9494 +sequential,accuracy,1.0,1.0,4000,80032000,32000,8,19992,19992,0,1.9992,0.9996 +sequential,none,0.0,0.0,5000,180000000,100000000,20000,0,0,0,1.0,0.0 +sequential,next_page,1.0,0.99995,5000,90004000,10004000,1,20000,19999,1,1.999911,0.89996 +sequential,accuracy,0.0,0.0,5000,180000000,100000000,20000,19992,0,19992,1.0,0.0 +sequential,accuracy,0.1,0.095438,5000,170460000,90460000,18092,19992,1908,18084,1.055966,0.0954 +sequential,accuracy,0.2,0.198179,5000,160190000,80190000,16038,19992,3962,16030,1.123666,0.1981 +sequential,accuracy,0.3,0.303621,5000,149650000,69650000,13930,19992,6070,13922,1.202807,0.3035 +sequential,accuracy,0.4,0.403561,5000,139660000,59660000,11932,19992,8068,11924,1.288844,0.4034 +sequential,accuracy,0.5,0.50095,5000,129925000,49925000,9985,19992,10015,9977,1.385415,0.50075 +sequential,accuracy,0.6,0.602291,5000,119795000,39795000,7959,19992,12041,7951,1.502567,0.60205 +sequential,accuracy,0.7,0.70118,5000,109910000,29910000,5982,19992,14018,5974,1.637704,0.7009 +sequential,accuracy,0.8,0.80087,5000,99945000,19945000,3989,19992,16011,3981,1.800991,0.80055 +sequential,accuracy,0.9,0.90051,5000,89985000,9985000,1997,19992,18003,1989,2.000333,0.90015 +sequential,accuracy,0.95,0.94978,5000,85060000,5060000,1012,19992,18988,1004,2.116153,0.9494 +sequential,accuracy,1.0,1.0,5000,80040000,40000,8,19992,19992,0,2.248876,0.9996 +sequential,none,0.0,0.0,10000,280000000,200000000,20000,0,0,0,1.0,0.0 +sequential,next_page,1.0,0.99995,10000,140004000,60004000,1,20000,19999,1,1.999943,0.69998 +sequential,accuracy,0.0,0.0,10000,280000000,200000000,20000,19992,0,19992,1.0,0.0 +sequential,accuracy,0.1,0.095438,10000,260920000,180920000,18092,19992,1908,18084,1.073126,0.0954 +sequential,accuracy,0.2,0.198179,10000,240380000,160380000,16038,19992,3962,16030,1.164822,0.1981 +sequential,accuracy,0.3,0.303621,10000,219300000,139300000,13930,19992,6070,13922,1.27679,0.3035 +sequential,accuracy,0.4,0.403561,10000,199320000,119320000,11932,19992,8068,11924,1.404776,0.4034 +sequential,accuracy,0.5,0.50095,10000,179850000,99850000,9985,19992,10015,9977,1.556853,0.50075 +sequential,accuracy,0.6,0.602291,10000,159590000,79590000,7959,19992,12041,7951,1.754496,0.60205 +sequential,accuracy,0.7,0.70118,10000,139820000,59820000,5982,19992,14018,5974,2.002575,0.7009 +sequential,accuracy,0.8,0.80087,10000,119890000,39890000,3989,19992,16011,3981,2.335474,0.80055 +sequential,accuracy,0.9,0.90051,10000,99970000,19970000,1997,19992,18003,1989,2.80084,0.90015 +sequential,accuracy,0.95,0.94978,10000,90120000,10120000,1012,19992,18988,1004,3.106968,0.9494 +sequential,accuracy,1.0,1.0,10000,80080000,80000,8,19992,19992,0,3.496503,0.9996 +sequential,none,0.0,0.0,20000,480000000,400000000,20000,0,0,0,1.0,0.0 +sequential,next_page,1.0,0.99995,20000,240004000,160004000,1,20000,19999,1,1.999967,0.59999 +sequential,accuracy,0.0,0.0,20000,480000000,400000000,20000,19992,0,19992,1.0,0.0 +sequential,accuracy,0.1,0.095438,20000,441840000,361840000,18092,19992,1908,18084,1.086366,0.0954 +sequential,accuracy,0.2,0.198179,20000,400760000,320760000,16038,19992,3962,16030,1.197724,0.1981 +sequential,accuracy,0.3,0.303621,20000,358600000,278600000,13930,19992,6070,13922,1.338539,0.3035 +sequential,accuracy,0.4,0.403561,20000,318640000,238640000,11932,19992,8068,11924,1.506402,0.4034 +sequential,accuracy,0.5,0.50095,20000,279700000,199700000,9985,19992,10015,9977,1.716124,0.50075 +sequential,accuracy,0.6,0.602291,20000,239180000,159180000,7959,19992,12041,7951,2.006857,0.60205 +sequential,accuracy,0.7,0.70118,20000,199640000,119640000,5982,19992,14018,5974,2.404328,0.7009 +sequential,accuracy,0.8,0.80087,20000,159780000,79780000,3989,19992,16011,3981,3.004131,0.80055 +sequential,accuracy,0.9,0.90051,20000,119940000,39940000,1997,19992,18003,1989,4.002001,0.90015 +sequential,accuracy,0.95,0.94978,20000,100240000,20240000,1012,19992,18988,1004,4.788508,0.9494 +sequential,accuracy,1.0,1.0,20000,80160000,160000,8,19992,19992,0,5.988024,0.9996 +random,none,0.0,0.0,1000,100000000,20000000,20000,0,0,0,1.0,0.0 +random,next_page,0.0004,0.00035,1000,99993000,19993000,19993,19993,7,19986,1.00007,0.00035 +random,accuracy,0.0,0.0,1000,100000000,20000000,20000,19992,0,19992,1.0,0.0 +random,accuracy,0.1,0.095438,1000,98092000,18092000,18092,19992,1908,18084,1.019451,0.0954 +random,accuracy,0.2,0.198179,1000,96038000,16038000,16038,19992,3962,16030,1.041255,0.1981 +random,accuracy,0.3,0.303621,1000,93930000,13930000,13930,19992,6070,13922,1.064623,0.3035 +random,accuracy,0.4,0.403561,1000,91932000,11932000,11932,19992,8068,11924,1.087761,0.4034 +random,accuracy,0.5,0.50095,1000,89985000,9985000,9985,19992,10015,9977,1.111296,0.50075 +random,accuracy,0.6,0.602291,1000,87959000,7959000,7959,19992,12041,7951,1.136893,0.60205 +random,accuracy,0.7,0.70118,1000,85982000,5982000,5982,19992,14018,5974,1.163034,0.7009 +random,accuracy,0.8,0.80087,1000,83989000,3989000,3989,19992,16011,3981,1.190632,0.80055 +random,accuracy,0.9,0.900505,1000,81998000,1998000,1998,19991,18002,1989,1.219542,0.9001 +random,accuracy,0.95,0.949777,1000,81013000,1013000,1013,19991,18987,1004,1.23437,0.94935 +random,accuracy,1.0,1.0,1000,80009000,9000,9,19991,19991,0,1.249859,0.99955 +random,none,0.0,0.0,2000,120000000,40000000,20000,0,0,0,1.0,0.0 +random,next_page,0.0004,0.00035,2000,119986000,39986000,19993,19993,7,19986,1.000117,0.00035 +random,accuracy,0.0,0.0,2000,120000000,40000000,20000,19992,0,19992,1.0,0.0 +random,accuracy,0.1,0.095438,2000,116184000,36184000,18092,19992,1908,18084,1.032844,0.0954 +random,accuracy,0.2,0.198179,2000,112076000,32076000,16038,19992,3962,16030,1.070702,0.1981 +random,accuracy,0.3,0.303621,2000,107860000,27860000,13930,19992,6070,13922,1.112553,0.3035 +random,accuracy,0.4,0.403561,2000,103864000,23864000,11932,19992,8068,11924,1.155357,0.4034 +random,accuracy,0.5,0.50095,2000,99970000,19970000,9985,19992,10015,9977,1.20036,0.50075 +random,accuracy,0.6,0.602291,2000,95918000,15918000,7959,19992,12041,7951,1.251069,0.60205 +random,accuracy,0.7,0.70118,2000,91964000,11964000,5982,19992,14018,5974,1.304858,0.7009 +random,accuracy,0.8,0.80087,2000,87978000,7978000,3989,19992,16011,3981,1.363977,0.80055 +random,accuracy,0.9,0.900505,2000,83996000,3996000,1998,19991,18002,1989,1.428639,0.9001 +random,accuracy,0.95,0.949777,2000,82026000,2026000,1013,19991,18987,1004,1.462951,0.94935 +random,accuracy,1.0,1.0,2000,80018000,18000,9,19991,19991,0,1.499663,0.99955 +random,none,0.0,0.0,4000,160000000,80000000,20000,0,0,0,1.0,0.0 +random,next_page,0.0004,0.00035,4000,159972000,79972000,19993,19993,7,19986,1.000175,0.00035 +random,accuracy,0.0,0.0,4000,160000000,80000000,20000,19992,0,19992,1.0,0.0 +random,accuracy,0.1,0.095438,4000,152368000,72368000,18092,19992,1908,18084,1.050089,0.0954 +random,accuracy,0.2,0.198179,4000,144152000,64152000,16038,19992,3962,16030,1.10994,0.1981 +random,accuracy,0.3,0.303621,4000,135720000,55720000,13930,19992,6070,13922,1.178898,0.3035 +random,accuracy,0.4,0.403561,4000,127728000,47728000,11932,19992,8068,11924,1.252662,0.4034 +random,accuracy,0.5,0.50095,4000,119940000,39940000,9985,19992,10015,9977,1.334,0.50075 +random,accuracy,0.6,0.602291,4000,111836000,31836000,7959,19992,12041,7951,1.430666,0.60205 +random,accuracy,0.7,0.70118,4000,103928000,23928000,5982,19992,14018,5974,1.539527,0.7009 +random,accuracy,0.8,0.80087,4000,95956000,15956000,3989,19992,16011,3981,1.667431,0.80055 +random,accuracy,0.9,0.900505,4000,87992000,7992000,1998,19991,18002,1989,1.818347,0.9001 +random,accuracy,0.95,0.949777,4000,84052000,4052000,1013,19991,18987,1004,1.903583,0.94935 +random,accuracy,1.0,1.0,4000,80036000,36000,9,19991,19991,0,1.9991,0.99955 +random,none,0.0,0.0,5000,180000000,100000000,20000,0,0,0,1.0,0.0 +random,next_page,0.0004,0.00035,5000,179965000,99965000,19993,19993,7,19986,1.000194,0.00035 +random,accuracy,0.0,0.0,5000,180000000,100000000,20000,19992,0,19992,1.0,0.0 +random,accuracy,0.1,0.095438,5000,170460000,90460000,18092,19992,1908,18084,1.055966,0.0954 +random,accuracy,0.2,0.198179,5000,160190000,80190000,16038,19992,3962,16030,1.123666,0.1981 +random,accuracy,0.3,0.303621,5000,149650000,69650000,13930,19992,6070,13922,1.202807,0.3035 +random,accuracy,0.4,0.403561,5000,139660000,59660000,11932,19992,8068,11924,1.288844,0.4034 +random,accuracy,0.5,0.50095,5000,129925000,49925000,9985,19992,10015,9977,1.385415,0.50075 +random,accuracy,0.6,0.602291,5000,119795000,39795000,7959,19992,12041,7951,1.502567,0.60205 +random,accuracy,0.7,0.70118,5000,109910000,29910000,5982,19992,14018,5974,1.637704,0.7009 +random,accuracy,0.8,0.80087,5000,99945000,19945000,3989,19992,16011,3981,1.800991,0.80055 +random,accuracy,0.9,0.900505,5000,89990000,9990000,1998,19991,18002,1989,2.000222,0.9001 +random,accuracy,0.95,0.949777,5000,85065000,5065000,1013,19991,18987,1004,2.116029,0.94935 +random,accuracy,1.0,1.0,5000,80045000,45000,9,19991,19991,0,2.248735,0.99955 +random,none,0.0,0.0,10000,280000000,200000000,20000,0,0,0,1.0,0.0 +random,next_page,0.0004,0.00035,10000,279930000,199930000,19993,19993,7,19986,1.00025,0.00035 +random,accuracy,0.0,0.0,10000,280000000,200000000,20000,19992,0,19992,1.0,0.0 +random,accuracy,0.1,0.095438,10000,260920000,180920000,18092,19992,1908,18084,1.073126,0.0954 +random,accuracy,0.2,0.198179,10000,240380000,160380000,16038,19992,3962,16030,1.164822,0.1981 +random,accuracy,0.3,0.303621,10000,219300000,139300000,13930,19992,6070,13922,1.27679,0.3035 +random,accuracy,0.4,0.403561,10000,199320000,119320000,11932,19992,8068,11924,1.404776,0.4034 +random,accuracy,0.5,0.50095,10000,179850000,99850000,9985,19992,10015,9977,1.556853,0.50075 +random,accuracy,0.6,0.602291,10000,159590000,79590000,7959,19992,12041,7951,1.754496,0.60205 +random,accuracy,0.7,0.70118,10000,139820000,59820000,5982,19992,14018,5974,2.002575,0.7009 +random,accuracy,0.8,0.80087,10000,119890000,39890000,3989,19992,16011,3981,2.335474,0.80055 +random,accuracy,0.9,0.900505,10000,99980000,19980000,1998,19991,18002,1989,2.80056,0.9001 +random,accuracy,0.95,0.949777,10000,90130000,10130000,1013,19991,18987,1004,3.106624,0.94935 +random,accuracy,1.0,1.0,10000,80090000,90000,9,19991,19991,0,3.496067,0.99955 +random,none,0.0,0.0,20000,480000000,400000000,20000,0,0,0,1.0,0.0 +random,next_page,0.0004,0.00035,20000,479860000,399860000,19993,19993,7,19986,1.000292,0.00035 +random,accuracy,0.0,0.0,20000,480000000,400000000,20000,19992,0,19992,1.0,0.0 +random,accuracy,0.1,0.095438,20000,441840000,361840000,18092,19992,1908,18084,1.086366,0.0954 +random,accuracy,0.2,0.198179,20000,400760000,320760000,16038,19992,3962,16030,1.197724,0.1981 +random,accuracy,0.3,0.303621,20000,358600000,278600000,13930,19992,6070,13922,1.338539,0.3035 +random,accuracy,0.4,0.403561,20000,318640000,238640000,11932,19992,8068,11924,1.506402,0.4034 +random,accuracy,0.5,0.50095,20000,279700000,199700000,9985,19992,10015,9977,1.716124,0.50075 +random,accuracy,0.6,0.602291,20000,239180000,159180000,7959,19992,12041,7951,2.006857,0.60205 +random,accuracy,0.7,0.70118,20000,199640000,119640000,5982,19992,14018,5974,2.404328,0.7009 +random,accuracy,0.8,0.80087,20000,159780000,79780000,3989,19992,16011,3981,3.004131,0.80055 +random,accuracy,0.9,0.900505,20000,119960000,39960000,1998,19991,18002,1989,4.001334,0.9001 +random,accuracy,0.95,0.949777,20000,100260000,20260000,1013,19991,18987,1004,4.787552,0.94935 +random,accuracy,1.0,1.0,20000,80180000,180000,9,19991,19991,0,5.98653,0.99955 +moe,none,0.0,0.0,1000,4423680000,884736000,884736,0,0,0,1.0,0.0 +moe,next_page,0.9996,0.999577,1000,3539318000,374000,374,884736,884362,374,1.249868,0.999577 +moe,accuracy,0.0,0.0,1000,4423680000,884736000,884736,884728,0,884728,1.0,0.0 +moe,accuracy,0.1,0.100347,1000,4334900000,795956000,795956,884728,88780,795948,1.02048,0.100346 +moe,accuracy,0.2,0.200758,1000,4246064000,707120000,707120,884728,177616,707112,1.041831,0.200756 +moe,accuracy,0.3,0.300788,1000,4157564000,618620000,618620,884728,266116,618612,1.064008,0.300786 +moe,accuracy,0.4,0.400825,1000,4069059000,530115000,530115,884728,354621,530107,1.087151,0.400821 +moe,accuracy,0.5,0.500951,1000,3980475000,441531000,441531,884728,443205,441523,1.111345,0.500946 +moe,accuracy,0.6,0.60077,1000,3892162000,353218000,353218,884728,531518,353210,1.136561,0.600765 +moe,accuracy,0.7,0.700583,1000,3803855000,264911000,264911,884728,619825,264903,1.162947,0.700576 +moe,accuracy,0.8,0.800466,1000,3715485000,176541000,176541,884728,708195,176533,1.190606,0.800459 +moe,accuracy,0.9,0.900252,1000,3627202000,88258000,88258,884728,796478,88250,1.219585,0.900244 +moe,accuracy,0.95,0.949974,1000,3583211000,44267000,44267,884728,840469,44259,1.234557,0.949966 +moe,accuracy,1.0,1.0,1000,3538952000,8000,8,884728,884728,0,1.249997,0.999991 +moe,none,0.0,0.0,2000,5308416000,1769472000,884736,0,0,0,1.0,0.0 +moe,next_page,0.9996,0.999577,2000,3539692000,748000,374,884736,884362,374,1.499683,0.999577 +moe,accuracy,0.0,0.0,2000,5308416000,1769472000,884736,884728,0,884728,1.0,0.0 +moe,accuracy,0.1,0.100347,2000,5130856000,1591912000,795956,884728,88780,795948,1.034606,0.100346 +moe,accuracy,0.2,0.200758,2000,4953184000,1414240000,707120,884728,177616,707112,1.071718,0.200756 +moe,accuracy,0.3,0.300788,2000,4776184000,1237240000,618620,884728,266116,618612,1.111435,0.300786 +moe,accuracy,0.4,0.400825,2000,4599174000,1060230000,530115,884728,354621,530107,1.154211,0.400821 +moe,accuracy,0.5,0.500951,2000,4422006000,883062000,441531,884728,443205,441523,1.200454,0.500946 +moe,accuracy,0.6,0.60077,2000,4245380000,706436000,353218,884728,531518,353210,1.250398,0.600765 +moe,accuracy,0.7,0.700583,2000,4068766000,529822000,264911,884728,619825,264903,1.304675,0.700576 +moe,accuracy,0.8,0.800466,2000,3892026000,353082000,176541,884728,708195,176533,1.363921,0.800459 +moe,accuracy,0.9,0.900252,2000,3715460000,176516000,88258,884728,796478,88250,1.428737,0.900244 +moe,accuracy,0.95,0.949974,2000,3627478000,88534000,44267,884728,840469,44259,1.46339,0.949966 +moe,accuracy,1.0,1.0,2000,3538960000,16000,8,884728,884728,0,1.499993,0.999991 +moe,none,0.0,0.0,4000,7077888000,3538944000,884736,0,0,0,1.0,0.0 +moe,next_page,0.9996,0.999577,4000,3540440000,1496000,374,884736,884362,374,1.999155,0.999577 +moe,accuracy,0.0,0.0,4000,7077888000,3538944000,884736,884728,0,884728,1.0,0.0 +moe,accuracy,0.1,0.100347,4000,6722768000,3183824000,795956,884728,88780,795948,1.052823,0.100346 +moe,accuracy,0.2,0.200758,4000,6367424000,2828480000,707120,884728,177616,707112,1.111578,0.200756 +moe,accuracy,0.3,0.300788,4000,6013424000,2474480000,618620,884728,266116,618612,1.177015,0.300786 +moe,accuracy,0.4,0.400825,4000,5659404000,2120460000,530115,884728,354621,530107,1.250642,0.400821 +moe,accuracy,0.5,0.500951,4000,5305068000,1766124000,441531,884728,443205,441523,1.334175,0.500946 +moe,accuracy,0.6,0.60077,4000,4951816000,1412872000,353218,884728,531518,353210,1.429352,0.600765 +moe,accuracy,0.7,0.700583,4000,4598588000,1059644000,264911,884728,619825,264903,1.539144,0.700576 +moe,accuracy,0.8,0.800466,4000,4245108000,706164000,176541,884728,708195,176533,1.667305,0.800459 +moe,accuracy,0.9,0.900252,4000,3891976000,353032000,88258,884728,796478,88250,1.818585,0.900244 +moe,accuracy,0.95,0.949974,4000,3716012000,177068000,44267,884728,840469,44259,1.9047,0.949966 +moe,accuracy,1.0,1.0,4000,3538976000,32000,8,884728,884728,0,1.999982,0.999991 +moe,none,0.0,0.0,5000,7962624000,4423680000,884736,0,0,0,1.0,0.0 +moe,next_page,0.9996,0.999577,5000,3982808000,443864000,374,884736,884362,374,1.999249,0.899662 +moe,accuracy,0.0,0.0,5000,7962624000,4423680000,884736,884728,0,884728,1.0,0.0 +moe,accuracy,0.1,0.100347,5000,7518724000,3979780000,795956,884728,88780,795948,1.059039,0.100346 +moe,accuracy,0.2,0.200758,5000,7074544000,3535600000,707120,884728,177616,707112,1.125532,0.200756 +moe,accuracy,0.3,0.300788,5000,6632044000,3093100000,618620,884728,266116,618612,1.200629,0.300786 +moe,accuracy,0.4,0.400825,5000,6189519000,2650575000,530115,884728,354621,530107,1.286469,0.400821 +moe,accuracy,0.5,0.500951,5000,5746599000,2207655000,441531,884728,443205,441523,1.385624,0.500946 +moe,accuracy,0.6,0.60077,5000,5305034000,1766090000,353218,884728,531518,353210,1.500956,0.600765 +moe,accuracy,0.7,0.700583,5000,4863499000,1324555000,264911,884728,619825,264903,1.637221,0.700576 +moe,accuracy,0.8,0.800466,5000,4421649000,882705000,176541,884728,708195,176533,1.800827,0.800459 +moe,accuracy,0.9,0.900252,5000,3980234000,441290000,88258,884728,796478,88250,2.000542,0.900244 +moe,accuracy,0.95,0.949974,5000,3760279000,221335000,44267,884728,840469,44259,2.117562,0.949966 +moe,accuracy,1.0,1.0,5000,3538984000,40000,8,884728,884728,0,2.249975,0.999991 +moe,none,0.0,0.0,10000,12386304000,8847360000,884736,0,0,0,1.0,0.0 +moe,next_page,0.9996,0.999577,10000,6194648000,2655704000,374,884736,884362,374,1.999517,0.699831 +moe,accuracy,0.0,0.0,10000,12386304000,8847360000,884736,884728,0,884728,1.0,0.0 +moe,accuracy,0.1,0.100347,10000,11498504000,7959560000,795956,884728,88780,795948,1.07721,0.100346 +moe,accuracy,0.2,0.200758,10000,10610144000,7071200000,707120,884728,177616,707112,1.167402,0.200756 +moe,accuracy,0.3,0.300788,10000,9725144000,6186200000,618620,884728,266116,618612,1.273637,0.300786 +moe,accuracy,0.4,0.400825,10000,8840094000,5301150000,530115,884728,354621,530107,1.401151,0.400821 +moe,accuracy,0.5,0.500951,10000,7954254000,4415310000,441531,884728,443205,441523,1.557192,0.500946 +moe,accuracy,0.6,0.60077,10000,7071124000,3532180000,353218,884728,531518,353210,1.751674,0.600765 +moe,accuracy,0.7,0.700583,10000,6188054000,2649110000,264911,884728,619825,264903,2.001648,0.700576 +moe,accuracy,0.8,0.800466,10000,5304354000,1765410000,176541,884728,708195,176533,2.33512,0.800459 +moe,accuracy,0.9,0.900252,10000,4421524000,882580000,88258,884728,796478,88250,2.801365,0.900244 +moe,accuracy,0.95,0.949974,10000,3981614000,442670000,44267,884728,840469,44259,3.110875,0.949966 +moe,accuracy,1.0,1.0,10000,3539024000,80000,8,884728,884728,0,3.499921,0.999991 +moe,none,0.0,0.0,20000,21233664000,17694720000,884736,0,0,0,1.0,0.0 +moe,next_page,0.9996,0.999577,20000,10618328000,7079384000,374,884736,884362,374,1.999718,0.599915 +moe,accuracy,0.0,0.0,20000,21233664000,17694720000,884736,884728,0,884728,1.0,0.0 +moe,accuracy,0.1,0.100347,20000,19458064000,15919120000,795956,884728,88780,795948,1.091253,0.100346 +moe,accuracy,0.2,0.200758,20000,17681344000,14142400000,707120,884728,177616,707112,1.200908,0.200756 +moe,accuracy,0.3,0.300788,20000,15911344000,12372400000,618620,884728,266116,618612,1.334498,0.300786 +moe,accuracy,0.4,0.400825,20000,14141244000,10602300000,530115,884728,354621,530107,1.501541,0.400821 +moe,accuracy,0.5,0.500951,20000,12369564000,8830620000,441531,884728,443205,441523,1.716606,0.500946 +moe,accuracy,0.6,0.60077,20000,10603304000,7064360000,353218,884728,531518,353210,2.002552,0.600765 +moe,accuracy,0.7,0.700583,20000,8837164000,5298220000,264911,884728,619825,264903,2.402769,0.700576 +moe,accuracy,0.8,0.800466,20000,7069764000,3530820000,176541,884728,708195,176533,3.003447,0.800459 +moe,accuracy,0.9,0.900252,20000,5304104000,1765160000,88258,884728,796478,88250,4.003252,0.900244 +moe,accuracy,0.95,0.949974,20000,4424284000,885340000,44267,884728,840469,44259,4.799345,0.949966 +moe,accuracy,1.0,1.0,20000,3539104000,160000,8,884728,884728,0,5.999729,0.999991 diff --git a/docs/proofs/artifacts/prefetch-accuracy-sweep.json b/docs/proofs/artifacts/prefetch-accuracy-sweep.json new file mode 100644 index 0000000..8f1b919 --- /dev/null +++ b/docs/proofs/artifacts/prefetch-accuracy-sweep.json @@ -0,0 +1,3681 @@ +{ + "schema_version": 1, + "disclaimer": "modeled, not measured on any device or GPU", + "model": "src/prefetch/prefetch_model.cpp", + "design": "docs/46-\u9884\u53d6\u5b9e\u9a8c\u8bbe\u8ba1.md", + "published_predictors": [ + { + "name": "ProMoE cross-layer gate on Qwen2-MoE", + "accuracy": 0.669, + "source": "arXiv:2410.22134" + }, + { + "name": "Fate, no training", + "accuracy": 0.7879, + "source": "arXiv:2502.12224" + }, + { + "name": "DAOP on Mixtral 8x7B", + "accuracy": 0.8411, + "source": "arXiv:2501.10375" + }, + { + "name": "ProMoE learned predictor, average", + "accuracy": 0.847, + "source": "arXiv:2410.22134" + }, + { + "name": "AdapMoE", + "accuracy": 0.9, + "source": "arXiv:2408.10284" + }, + { + "name": "HOBBIT next-layer top-1", + "accuracy": 0.96, + "source": "arXiv:2411.01433" + }, + { + "name": "Fate with over-fetch", + "accuracy": 0.9715, + "source": "arXiv:2502.12224; about 3.75x the weight traffic" + } + ], + "runs": [ + { + "schema_version": 1, + "model": "src/prefetch/prefetch_model.cpp", + "disclaimer": "modeled, not measured on any device or GPU", + "workload": { + "stream": "sequential", + "accesses": 20000, + "compute_ns_per_access": 4000, + "lead_distance": 8, + "buffer_pages": 64, + "max_in_flight": 32, + "seed": 7 + }, + "compute_floor_ns": 80000000, + "cells": [ + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 1000, + "total_ns": 100000000, + "stall_ns": 20000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 1.0, + "read_latency_ns": 1000, + "total_ns": 80001000, + "stall_ns": 1000, + "demand_misses": 1, + "prefetch_issued": 20000, + "prefetch_hits": 19999, + "prefetch_wasted": 1, + "achieved_accuracy": 0.99995, + "speedup_over_no_prefetch": 1.249984, + "recovered_fraction": 0.99995 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 1000, + "total_ns": 100000000, + "stall_ns": 20000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 1000, + "total_ns": 98092000, + "stall_ns": 18092000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.019451, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 1000, + "total_ns": 96038000, + "stall_ns": 16038000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.041255, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 1000, + "total_ns": 93930000, + "stall_ns": 13930000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.064623, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 1000, + "total_ns": 91932000, + "stall_ns": 11932000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.087761, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 1000, + "total_ns": 89985000, + "stall_ns": 9985000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.111296, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 1000, + "total_ns": 87959000, + "stall_ns": 7959000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.136893, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 1000, + "total_ns": 85982000, + "stall_ns": 5982000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 1.163034, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 1000, + "total_ns": 83989000, + "stall_ns": 3989000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 1.190632, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 1000, + "total_ns": 81997000, + "stall_ns": 1997000, + "demand_misses": 1997, + "prefetch_issued": 19992, + "prefetch_hits": 18003, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.90051, + "speedup_over_no_prefetch": 1.219557, + "recovered_fraction": 0.90015 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 1000, + "total_ns": 81012000, + "stall_ns": 1012000, + "demand_misses": 1012, + "prefetch_issued": 19992, + "prefetch_hits": 18988, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.94978, + "speedup_over_no_prefetch": 1.234385, + "recovered_fraction": 0.9494 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 1000, + "total_ns": 80008000, + "stall_ns": 8000, + "demand_misses": 8, + "prefetch_issued": 19992, + "prefetch_hits": 19992, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.249875, + "recovered_fraction": 0.9996 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 2000, + "total_ns": 120000000, + "stall_ns": 40000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 1.0, + "read_latency_ns": 2000, + "total_ns": 80002000, + "stall_ns": 2000, + "demand_misses": 1, + "prefetch_issued": 20000, + "prefetch_hits": 19999, + "prefetch_wasted": 1, + "achieved_accuracy": 0.99995, + "speedup_over_no_prefetch": 1.499963, + "recovered_fraction": 0.99995 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 2000, + "total_ns": 120000000, + "stall_ns": 40000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 2000, + "total_ns": 116184000, + "stall_ns": 36184000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.032844, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 2000, + "total_ns": 112076000, + "stall_ns": 32076000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.070702, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 2000, + "total_ns": 107860000, + "stall_ns": 27860000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.112553, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 2000, + "total_ns": 103864000, + "stall_ns": 23864000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.155357, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 2000, + "total_ns": 99970000, + "stall_ns": 19970000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.20036, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 2000, + "total_ns": 95918000, + "stall_ns": 15918000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.251069, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 2000, + "total_ns": 91964000, + "stall_ns": 11964000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 1.304858, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 2000, + "total_ns": 87978000, + "stall_ns": 7978000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 1.363977, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 2000, + "total_ns": 83994000, + "stall_ns": 3994000, + "demand_misses": 1997, + "prefetch_issued": 19992, + "prefetch_hits": 18003, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.90051, + "speedup_over_no_prefetch": 1.428673, + "recovered_fraction": 0.90015 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 2000, + "total_ns": 82024000, + "stall_ns": 2024000, + "demand_misses": 1012, + "prefetch_issued": 19992, + "prefetch_hits": 18988, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.94978, + "speedup_over_no_prefetch": 1.462986, + "recovered_fraction": 0.9494 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 2000, + "total_ns": 80016000, + "stall_ns": 16000, + "demand_misses": 8, + "prefetch_issued": 19992, + "prefetch_hits": 19992, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.4997, + "recovered_fraction": 0.9996 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 4000, + "total_ns": 160000000, + "stall_ns": 80000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 1.0, + "read_latency_ns": 4000, + "total_ns": 80004000, + "stall_ns": 4000, + "demand_misses": 1, + "prefetch_issued": 20000, + "prefetch_hits": 19999, + "prefetch_wasted": 1, + "achieved_accuracy": 0.99995, + "speedup_over_no_prefetch": 1.9999, + "recovered_fraction": 0.99995 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 4000, + "total_ns": 160000000, + "stall_ns": 80000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 4000, + "total_ns": 152368000, + "stall_ns": 72368000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.050089, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 4000, + "total_ns": 144152000, + "stall_ns": 64152000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.10994, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 4000, + "total_ns": 135720000, + "stall_ns": 55720000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.178898, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 4000, + "total_ns": 127728000, + "stall_ns": 47728000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.252662, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 4000, + "total_ns": 119940000, + "stall_ns": 39940000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.334, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 4000, + "total_ns": 111836000, + "stall_ns": 31836000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.430666, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 4000, + "total_ns": 103928000, + "stall_ns": 23928000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 1.539527, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 4000, + "total_ns": 95956000, + "stall_ns": 15956000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 1.667431, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 4000, + "total_ns": 87988000, + "stall_ns": 7988000, + "demand_misses": 1997, + "prefetch_issued": 19992, + "prefetch_hits": 18003, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.90051, + "speedup_over_no_prefetch": 1.81843, + "recovered_fraction": 0.90015 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 4000, + "total_ns": 84048000, + "stall_ns": 4048000, + "demand_misses": 1012, + "prefetch_issued": 19992, + "prefetch_hits": 18988, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.94978, + "speedup_over_no_prefetch": 1.903674, + "recovered_fraction": 0.9494 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 4000, + "total_ns": 80032000, + "stall_ns": 32000, + "demand_misses": 8, + "prefetch_issued": 19992, + "prefetch_hits": 19992, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.9992, + "recovered_fraction": 0.9996 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 5000, + "total_ns": 180000000, + "stall_ns": 100000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 1.0, + "read_latency_ns": 5000, + "total_ns": 90004000, + "stall_ns": 10004000, + "demand_misses": 1, + "prefetch_issued": 20000, + "prefetch_hits": 19999, + "prefetch_wasted": 1, + "achieved_accuracy": 0.99995, + "speedup_over_no_prefetch": 1.999911, + "recovered_fraction": 0.89996 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 5000, + "total_ns": 180000000, + "stall_ns": 100000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 5000, + "total_ns": 170460000, + "stall_ns": 90460000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.055966, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 5000, + "total_ns": 160190000, + "stall_ns": 80190000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.123666, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 5000, + "total_ns": 149650000, + "stall_ns": 69650000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.202807, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 5000, + "total_ns": 139660000, + "stall_ns": 59660000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.288844, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 5000, + "total_ns": 129925000, + "stall_ns": 49925000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.385415, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 5000, + "total_ns": 119795000, + "stall_ns": 39795000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.502567, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 5000, + "total_ns": 109910000, + "stall_ns": 29910000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 1.637704, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 5000, + "total_ns": 99945000, + "stall_ns": 19945000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 1.800991, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 5000, + "total_ns": 89985000, + "stall_ns": 9985000, + "demand_misses": 1997, + "prefetch_issued": 19992, + "prefetch_hits": 18003, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.90051, + "speedup_over_no_prefetch": 2.000333, + "recovered_fraction": 0.90015 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 5000, + "total_ns": 85060000, + "stall_ns": 5060000, + "demand_misses": 1012, + "prefetch_issued": 19992, + "prefetch_hits": 18988, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.94978, + "speedup_over_no_prefetch": 2.116153, + "recovered_fraction": 0.9494 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 5000, + "total_ns": 80040000, + "stall_ns": 40000, + "demand_misses": 8, + "prefetch_issued": 19992, + "prefetch_hits": 19992, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 2.248876, + "recovered_fraction": 0.9996 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 10000, + "total_ns": 280000000, + "stall_ns": 200000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 1.0, + "read_latency_ns": 10000, + "total_ns": 140004000, + "stall_ns": 60004000, + "demand_misses": 1, + "prefetch_issued": 20000, + "prefetch_hits": 19999, + "prefetch_wasted": 1, + "achieved_accuracy": 0.99995, + "speedup_over_no_prefetch": 1.999943, + "recovered_fraction": 0.69998 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 10000, + "total_ns": 280000000, + "stall_ns": 200000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 10000, + "total_ns": 260920000, + "stall_ns": 180920000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.073126, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 10000, + "total_ns": 240380000, + "stall_ns": 160380000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.164822, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 10000, + "total_ns": 219300000, + "stall_ns": 139300000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.27679, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 10000, + "total_ns": 199320000, + "stall_ns": 119320000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.404776, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 10000, + "total_ns": 179850000, + "stall_ns": 99850000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.556853, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 10000, + "total_ns": 159590000, + "stall_ns": 79590000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.754496, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 10000, + "total_ns": 139820000, + "stall_ns": 59820000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 2.002575, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 10000, + "total_ns": 119890000, + "stall_ns": 39890000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 2.335474, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 10000, + "total_ns": 99970000, + "stall_ns": 19970000, + "demand_misses": 1997, + "prefetch_issued": 19992, + "prefetch_hits": 18003, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.90051, + "speedup_over_no_prefetch": 2.80084, + "recovered_fraction": 0.90015 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 10000, + "total_ns": 90120000, + "stall_ns": 10120000, + "demand_misses": 1012, + "prefetch_issued": 19992, + "prefetch_hits": 18988, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.94978, + "speedup_over_no_prefetch": 3.106968, + "recovered_fraction": 0.9494 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 10000, + "total_ns": 80080000, + "stall_ns": 80000, + "demand_misses": 8, + "prefetch_issued": 19992, + "prefetch_hits": 19992, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 3.496503, + "recovered_fraction": 0.9996 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 20000, + "total_ns": 480000000, + "stall_ns": 400000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 1.0, + "read_latency_ns": 20000, + "total_ns": 240004000, + "stall_ns": 160004000, + "demand_misses": 1, + "prefetch_issued": 20000, + "prefetch_hits": 19999, + "prefetch_wasted": 1, + "achieved_accuracy": 0.99995, + "speedup_over_no_prefetch": 1.999967, + "recovered_fraction": 0.59999 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 20000, + "total_ns": 480000000, + "stall_ns": 400000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 20000, + "total_ns": 441840000, + "stall_ns": 361840000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.086366, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 20000, + "total_ns": 400760000, + "stall_ns": 320760000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.197724, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 20000, + "total_ns": 358600000, + "stall_ns": 278600000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.338539, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 20000, + "total_ns": 318640000, + "stall_ns": 238640000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.506402, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 20000, + "total_ns": 279700000, + "stall_ns": 199700000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.716124, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 20000, + "total_ns": 239180000, + "stall_ns": 159180000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 2.006857, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 20000, + "total_ns": 199640000, + "stall_ns": 119640000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 2.404328, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 20000, + "total_ns": 159780000, + "stall_ns": 79780000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 3.004131, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 20000, + "total_ns": 119940000, + "stall_ns": 39940000, + "demand_misses": 1997, + "prefetch_issued": 19992, + "prefetch_hits": 18003, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.90051, + "speedup_over_no_prefetch": 4.002001, + "recovered_fraction": 0.90015 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 20000, + "total_ns": 100240000, + "stall_ns": 20240000, + "demand_misses": 1012, + "prefetch_issued": 19992, + "prefetch_hits": 18988, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.94978, + "speedup_over_no_prefetch": 4.788508, + "recovered_fraction": 0.9494 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 20000, + "total_ns": 80160000, + "stall_ns": 160000, + "demand_misses": 8, + "prefetch_issued": 19992, + "prefetch_hits": 19992, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 5.988024, + "recovered_fraction": 0.9996 + } + ], + "command": [ + "build-cpu/hbf_prefetch_bench", + "--stream", + "sequential", + "--accesses", + "20000", + "--compute-ns", + "4000", + "--lead", + "8", + "--buffer-pages", + "64", + "--max-in-flight", + "32", + "--pages-per-expert", + "2", + "--seed", + "7" + ] + }, + { + "schema_version": 1, + "model": "src/prefetch/prefetch_model.cpp", + "disclaimer": "modeled, not measured on any device or GPU", + "workload": { + "stream": "random", + "accesses": 20000, + "compute_ns_per_access": 4000, + "lead_distance": 8, + "buffer_pages": 64, + "max_in_flight": 32, + "seed": 7 + }, + "compute_floor_ns": 80000000, + "cells": [ + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 1000, + "total_ns": 100000000, + "stall_ns": 20000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.0004, + "read_latency_ns": 1000, + "total_ns": 99993000, + "stall_ns": 19993000, + "demand_misses": 19993, + "prefetch_issued": 19993, + "prefetch_hits": 7, + "prefetch_wasted": 19986, + "achieved_accuracy": 0.00035, + "speedup_over_no_prefetch": 1.00007, + "recovered_fraction": 0.00035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 1000, + "total_ns": 100000000, + "stall_ns": 20000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 1000, + "total_ns": 98092000, + "stall_ns": 18092000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.019451, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 1000, + "total_ns": 96038000, + "stall_ns": 16038000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.041255, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 1000, + "total_ns": 93930000, + "stall_ns": 13930000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.064623, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 1000, + "total_ns": 91932000, + "stall_ns": 11932000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.087761, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 1000, + "total_ns": 89985000, + "stall_ns": 9985000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.111296, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 1000, + "total_ns": 87959000, + "stall_ns": 7959000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.136893, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 1000, + "total_ns": 85982000, + "stall_ns": 5982000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 1.163034, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 1000, + "total_ns": 83989000, + "stall_ns": 3989000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 1.190632, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 1000, + "total_ns": 81998000, + "stall_ns": 1998000, + "demand_misses": 1998, + "prefetch_issued": 19991, + "prefetch_hits": 18002, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.900505, + "speedup_over_no_prefetch": 1.219542, + "recovered_fraction": 0.9001 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 1000, + "total_ns": 81013000, + "stall_ns": 1013000, + "demand_misses": 1013, + "prefetch_issued": 19991, + "prefetch_hits": 18987, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.949777, + "speedup_over_no_prefetch": 1.23437, + "recovered_fraction": 0.94935 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 1000, + "total_ns": 80009000, + "stall_ns": 9000, + "demand_misses": 9, + "prefetch_issued": 19991, + "prefetch_hits": 19991, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.249859, + "recovered_fraction": 0.99955 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 2000, + "total_ns": 120000000, + "stall_ns": 40000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.0004, + "read_latency_ns": 2000, + "total_ns": 119986000, + "stall_ns": 39986000, + "demand_misses": 19993, + "prefetch_issued": 19993, + "prefetch_hits": 7, + "prefetch_wasted": 19986, + "achieved_accuracy": 0.00035, + "speedup_over_no_prefetch": 1.000117, + "recovered_fraction": 0.00035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 2000, + "total_ns": 120000000, + "stall_ns": 40000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 2000, + "total_ns": 116184000, + "stall_ns": 36184000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.032844, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 2000, + "total_ns": 112076000, + "stall_ns": 32076000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.070702, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 2000, + "total_ns": 107860000, + "stall_ns": 27860000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.112553, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 2000, + "total_ns": 103864000, + "stall_ns": 23864000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.155357, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 2000, + "total_ns": 99970000, + "stall_ns": 19970000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.20036, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 2000, + "total_ns": 95918000, + "stall_ns": 15918000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.251069, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 2000, + "total_ns": 91964000, + "stall_ns": 11964000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 1.304858, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 2000, + "total_ns": 87978000, + "stall_ns": 7978000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 1.363977, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 2000, + "total_ns": 83996000, + "stall_ns": 3996000, + "demand_misses": 1998, + "prefetch_issued": 19991, + "prefetch_hits": 18002, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.900505, + "speedup_over_no_prefetch": 1.428639, + "recovered_fraction": 0.9001 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 2000, + "total_ns": 82026000, + "stall_ns": 2026000, + "demand_misses": 1013, + "prefetch_issued": 19991, + "prefetch_hits": 18987, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.949777, + "speedup_over_no_prefetch": 1.462951, + "recovered_fraction": 0.94935 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 2000, + "total_ns": 80018000, + "stall_ns": 18000, + "demand_misses": 9, + "prefetch_issued": 19991, + "prefetch_hits": 19991, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.499663, + "recovered_fraction": 0.99955 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 4000, + "total_ns": 160000000, + "stall_ns": 80000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.0004, + "read_latency_ns": 4000, + "total_ns": 159972000, + "stall_ns": 79972000, + "demand_misses": 19993, + "prefetch_issued": 19993, + "prefetch_hits": 7, + "prefetch_wasted": 19986, + "achieved_accuracy": 0.00035, + "speedup_over_no_prefetch": 1.000175, + "recovered_fraction": 0.00035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 4000, + "total_ns": 160000000, + "stall_ns": 80000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 4000, + "total_ns": 152368000, + "stall_ns": 72368000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.050089, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 4000, + "total_ns": 144152000, + "stall_ns": 64152000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.10994, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 4000, + "total_ns": 135720000, + "stall_ns": 55720000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.178898, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 4000, + "total_ns": 127728000, + "stall_ns": 47728000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.252662, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 4000, + "total_ns": 119940000, + "stall_ns": 39940000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.334, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 4000, + "total_ns": 111836000, + "stall_ns": 31836000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.430666, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 4000, + "total_ns": 103928000, + "stall_ns": 23928000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 1.539527, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 4000, + "total_ns": 95956000, + "stall_ns": 15956000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 1.667431, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 4000, + "total_ns": 87992000, + "stall_ns": 7992000, + "demand_misses": 1998, + "prefetch_issued": 19991, + "prefetch_hits": 18002, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.900505, + "speedup_over_no_prefetch": 1.818347, + "recovered_fraction": 0.9001 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 4000, + "total_ns": 84052000, + "stall_ns": 4052000, + "demand_misses": 1013, + "prefetch_issued": 19991, + "prefetch_hits": 18987, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.949777, + "speedup_over_no_prefetch": 1.903583, + "recovered_fraction": 0.94935 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 4000, + "total_ns": 80036000, + "stall_ns": 36000, + "demand_misses": 9, + "prefetch_issued": 19991, + "prefetch_hits": 19991, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.9991, + "recovered_fraction": 0.99955 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 5000, + "total_ns": 180000000, + "stall_ns": 100000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.0004, + "read_latency_ns": 5000, + "total_ns": 179965000, + "stall_ns": 99965000, + "demand_misses": 19993, + "prefetch_issued": 19993, + "prefetch_hits": 7, + "prefetch_wasted": 19986, + "achieved_accuracy": 0.00035, + "speedup_over_no_prefetch": 1.000194, + "recovered_fraction": 0.00035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 5000, + "total_ns": 180000000, + "stall_ns": 100000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 5000, + "total_ns": 170460000, + "stall_ns": 90460000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.055966, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 5000, + "total_ns": 160190000, + "stall_ns": 80190000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.123666, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 5000, + "total_ns": 149650000, + "stall_ns": 69650000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.202807, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 5000, + "total_ns": 139660000, + "stall_ns": 59660000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.288844, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 5000, + "total_ns": 129925000, + "stall_ns": 49925000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.385415, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 5000, + "total_ns": 119795000, + "stall_ns": 39795000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.502567, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 5000, + "total_ns": 109910000, + "stall_ns": 29910000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 1.637704, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 5000, + "total_ns": 99945000, + "stall_ns": 19945000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 1.800991, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 5000, + "total_ns": 89990000, + "stall_ns": 9990000, + "demand_misses": 1998, + "prefetch_issued": 19991, + "prefetch_hits": 18002, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.900505, + "speedup_over_no_prefetch": 2.000222, + "recovered_fraction": 0.9001 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 5000, + "total_ns": 85065000, + "stall_ns": 5065000, + "demand_misses": 1013, + "prefetch_issued": 19991, + "prefetch_hits": 18987, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.949777, + "speedup_over_no_prefetch": 2.116029, + "recovered_fraction": 0.94935 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 5000, + "total_ns": 80045000, + "stall_ns": 45000, + "demand_misses": 9, + "prefetch_issued": 19991, + "prefetch_hits": 19991, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 2.248735, + "recovered_fraction": 0.99955 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 10000, + "total_ns": 280000000, + "stall_ns": 200000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.0004, + "read_latency_ns": 10000, + "total_ns": 279930000, + "stall_ns": 199930000, + "demand_misses": 19993, + "prefetch_issued": 19993, + "prefetch_hits": 7, + "prefetch_wasted": 19986, + "achieved_accuracy": 0.00035, + "speedup_over_no_prefetch": 1.00025, + "recovered_fraction": 0.00035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 10000, + "total_ns": 280000000, + "stall_ns": 200000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 10000, + "total_ns": 260920000, + "stall_ns": 180920000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.073126, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 10000, + "total_ns": 240380000, + "stall_ns": 160380000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.164822, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 10000, + "total_ns": 219300000, + "stall_ns": 139300000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.27679, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 10000, + "total_ns": 199320000, + "stall_ns": 119320000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.404776, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 10000, + "total_ns": 179850000, + "stall_ns": 99850000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.556853, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 10000, + "total_ns": 159590000, + "stall_ns": 79590000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 1.754496, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 10000, + "total_ns": 139820000, + "stall_ns": 59820000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 2.002575, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 10000, + "total_ns": 119890000, + "stall_ns": 39890000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 2.335474, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 10000, + "total_ns": 99980000, + "stall_ns": 19980000, + "demand_misses": 1998, + "prefetch_issued": 19991, + "prefetch_hits": 18002, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.900505, + "speedup_over_no_prefetch": 2.80056, + "recovered_fraction": 0.9001 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 10000, + "total_ns": 90130000, + "stall_ns": 10130000, + "demand_misses": 1013, + "prefetch_issued": 19991, + "prefetch_hits": 18987, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.949777, + "speedup_over_no_prefetch": 3.106624, + "recovered_fraction": 0.94935 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 10000, + "total_ns": 80090000, + "stall_ns": 90000, + "demand_misses": 9, + "prefetch_issued": 19991, + "prefetch_hits": 19991, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 3.496067, + "recovered_fraction": 0.99955 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 20000, + "total_ns": 480000000, + "stall_ns": 400000000, + "demand_misses": 20000, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.0004, + "read_latency_ns": 20000, + "total_ns": 479860000, + "stall_ns": 399860000, + "demand_misses": 19993, + "prefetch_issued": 19993, + "prefetch_hits": 7, + "prefetch_wasted": 19986, + "achieved_accuracy": 0.00035, + "speedup_over_no_prefetch": 1.000292, + "recovered_fraction": 0.00035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 20000, + "total_ns": 480000000, + "stall_ns": 400000000, + "demand_misses": 20000, + "prefetch_issued": 19992, + "prefetch_hits": 0, + "prefetch_wasted": 19992, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 20000, + "total_ns": 441840000, + "stall_ns": 361840000, + "demand_misses": 18092, + "prefetch_issued": 19992, + "prefetch_hits": 1908, + "prefetch_wasted": 18084, + "achieved_accuracy": 0.095438, + "speedup_over_no_prefetch": 1.086366, + "recovered_fraction": 0.0954 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 20000, + "total_ns": 400760000, + "stall_ns": 320760000, + "demand_misses": 16038, + "prefetch_issued": 19992, + "prefetch_hits": 3962, + "prefetch_wasted": 16030, + "achieved_accuracy": 0.198179, + "speedup_over_no_prefetch": 1.197724, + "recovered_fraction": 0.1981 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 20000, + "total_ns": 358600000, + "stall_ns": 278600000, + "demand_misses": 13930, + "prefetch_issued": 19992, + "prefetch_hits": 6070, + "prefetch_wasted": 13922, + "achieved_accuracy": 0.303621, + "speedup_over_no_prefetch": 1.338539, + "recovered_fraction": 0.3035 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 20000, + "total_ns": 318640000, + "stall_ns": 238640000, + "demand_misses": 11932, + "prefetch_issued": 19992, + "prefetch_hits": 8068, + "prefetch_wasted": 11924, + "achieved_accuracy": 0.403561, + "speedup_over_no_prefetch": 1.506402, + "recovered_fraction": 0.4034 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 20000, + "total_ns": 279700000, + "stall_ns": 199700000, + "demand_misses": 9985, + "prefetch_issued": 19992, + "prefetch_hits": 10015, + "prefetch_wasted": 9977, + "achieved_accuracy": 0.50095, + "speedup_over_no_prefetch": 1.716124, + "recovered_fraction": 0.50075 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 20000, + "total_ns": 239180000, + "stall_ns": 159180000, + "demand_misses": 7959, + "prefetch_issued": 19992, + "prefetch_hits": 12041, + "prefetch_wasted": 7951, + "achieved_accuracy": 0.602291, + "speedup_over_no_prefetch": 2.006857, + "recovered_fraction": 0.60205 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 20000, + "total_ns": 199640000, + "stall_ns": 119640000, + "demand_misses": 5982, + "prefetch_issued": 19992, + "prefetch_hits": 14018, + "prefetch_wasted": 5974, + "achieved_accuracy": 0.70118, + "speedup_over_no_prefetch": 2.404328, + "recovered_fraction": 0.7009 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 20000, + "total_ns": 159780000, + "stall_ns": 79780000, + "demand_misses": 3989, + "prefetch_issued": 19992, + "prefetch_hits": 16011, + "prefetch_wasted": 3981, + "achieved_accuracy": 0.80087, + "speedup_over_no_prefetch": 3.004131, + "recovered_fraction": 0.80055 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 20000, + "total_ns": 119960000, + "stall_ns": 39960000, + "demand_misses": 1998, + "prefetch_issued": 19991, + "prefetch_hits": 18002, + "prefetch_wasted": 1989, + "achieved_accuracy": 0.900505, + "speedup_over_no_prefetch": 4.001334, + "recovered_fraction": 0.9001 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 20000, + "total_ns": 100260000, + "stall_ns": 20260000, + "demand_misses": 1013, + "prefetch_issued": 19991, + "prefetch_hits": 18987, + "prefetch_wasted": 1004, + "achieved_accuracy": 0.949777, + "speedup_over_no_prefetch": 4.787552, + "recovered_fraction": 0.94935 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 20000, + "total_ns": 80180000, + "stall_ns": 180000, + "demand_misses": 9, + "prefetch_issued": 19991, + "prefetch_hits": 19991, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 5.98653, + "recovered_fraction": 0.99955 + } + ], + "command": [ + "build-cpu/hbf_prefetch_bench", + "--stream", + "random", + "--accesses", + "20000", + "--compute-ns", + "4000", + "--lead", + "8", + "--buffer-pages", + "64", + "--max-in-flight", + "32", + "--pages-per-expert", + "2", + "--seed", + "7" + ] + }, + { + "schema_version": 1, + "model": "src/prefetch/prefetch_model.cpp", + "disclaimer": "modeled, not measured on any device or GPU", + "workload": { + "stream": "moe", + "accesses": 884736, + "compute_ns_per_access": 4000, + "lead_distance": 8, + "buffer_pages": 64, + "max_in_flight": 32, + "seed": 7 + }, + "compute_floor_ns": 3538944000, + "cells": [ + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 1000, + "total_ns": 4423680000, + "stall_ns": 884736000, + "demand_misses": 884736, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.9996, + "read_latency_ns": 1000, + "total_ns": 3539318000, + "stall_ns": 374000, + "demand_misses": 374, + "prefetch_issued": 884736, + "prefetch_hits": 884362, + "prefetch_wasted": 374, + "achieved_accuracy": 0.999577, + "speedup_over_no_prefetch": 1.249868, + "recovered_fraction": 0.999577 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 1000, + "total_ns": 4423680000, + "stall_ns": 884736000, + "demand_misses": 884736, + "prefetch_issued": 884728, + "prefetch_hits": 0, + "prefetch_wasted": 884728, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 1000, + "total_ns": 4334900000, + "stall_ns": 795956000, + "demand_misses": 795956, + "prefetch_issued": 884728, + "prefetch_hits": 88780, + "prefetch_wasted": 795948, + "achieved_accuracy": 0.100347, + "speedup_over_no_prefetch": 1.02048, + "recovered_fraction": 0.100346 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 1000, + "total_ns": 4246064000, + "stall_ns": 707120000, + "demand_misses": 707120, + "prefetch_issued": 884728, + "prefetch_hits": 177616, + "prefetch_wasted": 707112, + "achieved_accuracy": 0.200758, + "speedup_over_no_prefetch": 1.041831, + "recovered_fraction": 0.200756 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 1000, + "total_ns": 4157564000, + "stall_ns": 618620000, + "demand_misses": 618620, + "prefetch_issued": 884728, + "prefetch_hits": 266116, + "prefetch_wasted": 618612, + "achieved_accuracy": 0.300788, + "speedup_over_no_prefetch": 1.064008, + "recovered_fraction": 0.300786 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 1000, + "total_ns": 4069059000, + "stall_ns": 530115000, + "demand_misses": 530115, + "prefetch_issued": 884728, + "prefetch_hits": 354621, + "prefetch_wasted": 530107, + "achieved_accuracy": 0.400825, + "speedup_over_no_prefetch": 1.087151, + "recovered_fraction": 0.400821 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 1000, + "total_ns": 3980475000, + "stall_ns": 441531000, + "demand_misses": 441531, + "prefetch_issued": 884728, + "prefetch_hits": 443205, + "prefetch_wasted": 441523, + "achieved_accuracy": 0.500951, + "speedup_over_no_prefetch": 1.111345, + "recovered_fraction": 0.500946 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 1000, + "total_ns": 3892162000, + "stall_ns": 353218000, + "demand_misses": 353218, + "prefetch_issued": 884728, + "prefetch_hits": 531518, + "prefetch_wasted": 353210, + "achieved_accuracy": 0.60077, + "speedup_over_no_prefetch": 1.136561, + "recovered_fraction": 0.600765 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 1000, + "total_ns": 3803855000, + "stall_ns": 264911000, + "demand_misses": 264911, + "prefetch_issued": 884728, + "prefetch_hits": 619825, + "prefetch_wasted": 264903, + "achieved_accuracy": 0.700583, + "speedup_over_no_prefetch": 1.162947, + "recovered_fraction": 0.700576 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 1000, + "total_ns": 3715485000, + "stall_ns": 176541000, + "demand_misses": 176541, + "prefetch_issued": 884728, + "prefetch_hits": 708195, + "prefetch_wasted": 176533, + "achieved_accuracy": 0.800466, + "speedup_over_no_prefetch": 1.190606, + "recovered_fraction": 0.800459 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 1000, + "total_ns": 3627202000, + "stall_ns": 88258000, + "demand_misses": 88258, + "prefetch_issued": 884728, + "prefetch_hits": 796478, + "prefetch_wasted": 88250, + "achieved_accuracy": 0.900252, + "speedup_over_no_prefetch": 1.219585, + "recovered_fraction": 0.900244 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 1000, + "total_ns": 3583211000, + "stall_ns": 44267000, + "demand_misses": 44267, + "prefetch_issued": 884728, + "prefetch_hits": 840469, + "prefetch_wasted": 44259, + "achieved_accuracy": 0.949974, + "speedup_over_no_prefetch": 1.234557, + "recovered_fraction": 0.949966 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 1000, + "total_ns": 3538952000, + "stall_ns": 8000, + "demand_misses": 8, + "prefetch_issued": 884728, + "prefetch_hits": 884728, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.249997, + "recovered_fraction": 0.999991 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 2000, + "total_ns": 5308416000, + "stall_ns": 1769472000, + "demand_misses": 884736, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.9996, + "read_latency_ns": 2000, + "total_ns": 3539692000, + "stall_ns": 748000, + "demand_misses": 374, + "prefetch_issued": 884736, + "prefetch_hits": 884362, + "prefetch_wasted": 374, + "achieved_accuracy": 0.999577, + "speedup_over_no_prefetch": 1.499683, + "recovered_fraction": 0.999577 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 2000, + "total_ns": 5308416000, + "stall_ns": 1769472000, + "demand_misses": 884736, + "prefetch_issued": 884728, + "prefetch_hits": 0, + "prefetch_wasted": 884728, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 2000, + "total_ns": 5130856000, + "stall_ns": 1591912000, + "demand_misses": 795956, + "prefetch_issued": 884728, + "prefetch_hits": 88780, + "prefetch_wasted": 795948, + "achieved_accuracy": 0.100347, + "speedup_over_no_prefetch": 1.034606, + "recovered_fraction": 0.100346 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 2000, + "total_ns": 4953184000, + "stall_ns": 1414240000, + "demand_misses": 707120, + "prefetch_issued": 884728, + "prefetch_hits": 177616, + "prefetch_wasted": 707112, + "achieved_accuracy": 0.200758, + "speedup_over_no_prefetch": 1.071718, + "recovered_fraction": 0.200756 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 2000, + "total_ns": 4776184000, + "stall_ns": 1237240000, + "demand_misses": 618620, + "prefetch_issued": 884728, + "prefetch_hits": 266116, + "prefetch_wasted": 618612, + "achieved_accuracy": 0.300788, + "speedup_over_no_prefetch": 1.111435, + "recovered_fraction": 0.300786 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 2000, + "total_ns": 4599174000, + "stall_ns": 1060230000, + "demand_misses": 530115, + "prefetch_issued": 884728, + "prefetch_hits": 354621, + "prefetch_wasted": 530107, + "achieved_accuracy": 0.400825, + "speedup_over_no_prefetch": 1.154211, + "recovered_fraction": 0.400821 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 2000, + "total_ns": 4422006000, + "stall_ns": 883062000, + "demand_misses": 441531, + "prefetch_issued": 884728, + "prefetch_hits": 443205, + "prefetch_wasted": 441523, + "achieved_accuracy": 0.500951, + "speedup_over_no_prefetch": 1.200454, + "recovered_fraction": 0.500946 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 2000, + "total_ns": 4245380000, + "stall_ns": 706436000, + "demand_misses": 353218, + "prefetch_issued": 884728, + "prefetch_hits": 531518, + "prefetch_wasted": 353210, + "achieved_accuracy": 0.60077, + "speedup_over_no_prefetch": 1.250398, + "recovered_fraction": 0.600765 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 2000, + "total_ns": 4068766000, + "stall_ns": 529822000, + "demand_misses": 264911, + "prefetch_issued": 884728, + "prefetch_hits": 619825, + "prefetch_wasted": 264903, + "achieved_accuracy": 0.700583, + "speedup_over_no_prefetch": 1.304675, + "recovered_fraction": 0.700576 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 2000, + "total_ns": 3892026000, + "stall_ns": 353082000, + "demand_misses": 176541, + "prefetch_issued": 884728, + "prefetch_hits": 708195, + "prefetch_wasted": 176533, + "achieved_accuracy": 0.800466, + "speedup_over_no_prefetch": 1.363921, + "recovered_fraction": 0.800459 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 2000, + "total_ns": 3715460000, + "stall_ns": 176516000, + "demand_misses": 88258, + "prefetch_issued": 884728, + "prefetch_hits": 796478, + "prefetch_wasted": 88250, + "achieved_accuracy": 0.900252, + "speedup_over_no_prefetch": 1.428737, + "recovered_fraction": 0.900244 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 2000, + "total_ns": 3627478000, + "stall_ns": 88534000, + "demand_misses": 44267, + "prefetch_issued": 884728, + "prefetch_hits": 840469, + "prefetch_wasted": 44259, + "achieved_accuracy": 0.949974, + "speedup_over_no_prefetch": 1.46339, + "recovered_fraction": 0.949966 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 2000, + "total_ns": 3538960000, + "stall_ns": 16000, + "demand_misses": 8, + "prefetch_issued": 884728, + "prefetch_hits": 884728, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.499993, + "recovered_fraction": 0.999991 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 4000, + "total_ns": 7077888000, + "stall_ns": 3538944000, + "demand_misses": 884736, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.9996, + "read_latency_ns": 4000, + "total_ns": 3540440000, + "stall_ns": 1496000, + "demand_misses": 374, + "prefetch_issued": 884736, + "prefetch_hits": 884362, + "prefetch_wasted": 374, + "achieved_accuracy": 0.999577, + "speedup_over_no_prefetch": 1.999155, + "recovered_fraction": 0.999577 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 4000, + "total_ns": 7077888000, + "stall_ns": 3538944000, + "demand_misses": 884736, + "prefetch_issued": 884728, + "prefetch_hits": 0, + "prefetch_wasted": 884728, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 4000, + "total_ns": 6722768000, + "stall_ns": 3183824000, + "demand_misses": 795956, + "prefetch_issued": 884728, + "prefetch_hits": 88780, + "prefetch_wasted": 795948, + "achieved_accuracy": 0.100347, + "speedup_over_no_prefetch": 1.052823, + "recovered_fraction": 0.100346 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 4000, + "total_ns": 6367424000, + "stall_ns": 2828480000, + "demand_misses": 707120, + "prefetch_issued": 884728, + "prefetch_hits": 177616, + "prefetch_wasted": 707112, + "achieved_accuracy": 0.200758, + "speedup_over_no_prefetch": 1.111578, + "recovered_fraction": 0.200756 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 4000, + "total_ns": 6013424000, + "stall_ns": 2474480000, + "demand_misses": 618620, + "prefetch_issued": 884728, + "prefetch_hits": 266116, + "prefetch_wasted": 618612, + "achieved_accuracy": 0.300788, + "speedup_over_no_prefetch": 1.177015, + "recovered_fraction": 0.300786 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 4000, + "total_ns": 5659404000, + "stall_ns": 2120460000, + "demand_misses": 530115, + "prefetch_issued": 884728, + "prefetch_hits": 354621, + "prefetch_wasted": 530107, + "achieved_accuracy": 0.400825, + "speedup_over_no_prefetch": 1.250642, + "recovered_fraction": 0.400821 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 4000, + "total_ns": 5305068000, + "stall_ns": 1766124000, + "demand_misses": 441531, + "prefetch_issued": 884728, + "prefetch_hits": 443205, + "prefetch_wasted": 441523, + "achieved_accuracy": 0.500951, + "speedup_over_no_prefetch": 1.334175, + "recovered_fraction": 0.500946 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 4000, + "total_ns": 4951816000, + "stall_ns": 1412872000, + "demand_misses": 353218, + "prefetch_issued": 884728, + "prefetch_hits": 531518, + "prefetch_wasted": 353210, + "achieved_accuracy": 0.60077, + "speedup_over_no_prefetch": 1.429352, + "recovered_fraction": 0.600765 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 4000, + "total_ns": 4598588000, + "stall_ns": 1059644000, + "demand_misses": 264911, + "prefetch_issued": 884728, + "prefetch_hits": 619825, + "prefetch_wasted": 264903, + "achieved_accuracy": 0.700583, + "speedup_over_no_prefetch": 1.539144, + "recovered_fraction": 0.700576 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 4000, + "total_ns": 4245108000, + "stall_ns": 706164000, + "demand_misses": 176541, + "prefetch_issued": 884728, + "prefetch_hits": 708195, + "prefetch_wasted": 176533, + "achieved_accuracy": 0.800466, + "speedup_over_no_prefetch": 1.667305, + "recovered_fraction": 0.800459 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 4000, + "total_ns": 3891976000, + "stall_ns": 353032000, + "demand_misses": 88258, + "prefetch_issued": 884728, + "prefetch_hits": 796478, + "prefetch_wasted": 88250, + "achieved_accuracy": 0.900252, + "speedup_over_no_prefetch": 1.818585, + "recovered_fraction": 0.900244 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 4000, + "total_ns": 3716012000, + "stall_ns": 177068000, + "demand_misses": 44267, + "prefetch_issued": 884728, + "prefetch_hits": 840469, + "prefetch_wasted": 44259, + "achieved_accuracy": 0.949974, + "speedup_over_no_prefetch": 1.9047, + "recovered_fraction": 0.949966 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 4000, + "total_ns": 3538976000, + "stall_ns": 32000, + "demand_misses": 8, + "prefetch_issued": 884728, + "prefetch_hits": 884728, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 1.999982, + "recovered_fraction": 0.999991 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 5000, + "total_ns": 7962624000, + "stall_ns": 4423680000, + "demand_misses": 884736, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.9996, + "read_latency_ns": 5000, + "total_ns": 3982808000, + "stall_ns": 443864000, + "demand_misses": 374, + "prefetch_issued": 884736, + "prefetch_hits": 884362, + "prefetch_wasted": 374, + "achieved_accuracy": 0.999577, + "speedup_over_no_prefetch": 1.999249, + "recovered_fraction": 0.899662 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 5000, + "total_ns": 7962624000, + "stall_ns": 4423680000, + "demand_misses": 884736, + "prefetch_issued": 884728, + "prefetch_hits": 0, + "prefetch_wasted": 884728, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 5000, + "total_ns": 7518724000, + "stall_ns": 3979780000, + "demand_misses": 795956, + "prefetch_issued": 884728, + "prefetch_hits": 88780, + "prefetch_wasted": 795948, + "achieved_accuracy": 0.100347, + "speedup_over_no_prefetch": 1.059039, + "recovered_fraction": 0.100346 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 5000, + "total_ns": 7074544000, + "stall_ns": 3535600000, + "demand_misses": 707120, + "prefetch_issued": 884728, + "prefetch_hits": 177616, + "prefetch_wasted": 707112, + "achieved_accuracy": 0.200758, + "speedup_over_no_prefetch": 1.125532, + "recovered_fraction": 0.200756 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 5000, + "total_ns": 6632044000, + "stall_ns": 3093100000, + "demand_misses": 618620, + "prefetch_issued": 884728, + "prefetch_hits": 266116, + "prefetch_wasted": 618612, + "achieved_accuracy": 0.300788, + "speedup_over_no_prefetch": 1.200629, + "recovered_fraction": 0.300786 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 5000, + "total_ns": 6189519000, + "stall_ns": 2650575000, + "demand_misses": 530115, + "prefetch_issued": 884728, + "prefetch_hits": 354621, + "prefetch_wasted": 530107, + "achieved_accuracy": 0.400825, + "speedup_over_no_prefetch": 1.286469, + "recovered_fraction": 0.400821 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 5000, + "total_ns": 5746599000, + "stall_ns": 2207655000, + "demand_misses": 441531, + "prefetch_issued": 884728, + "prefetch_hits": 443205, + "prefetch_wasted": 441523, + "achieved_accuracy": 0.500951, + "speedup_over_no_prefetch": 1.385624, + "recovered_fraction": 0.500946 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 5000, + "total_ns": 5305034000, + "stall_ns": 1766090000, + "demand_misses": 353218, + "prefetch_issued": 884728, + "prefetch_hits": 531518, + "prefetch_wasted": 353210, + "achieved_accuracy": 0.60077, + "speedup_over_no_prefetch": 1.500956, + "recovered_fraction": 0.600765 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 5000, + "total_ns": 4863499000, + "stall_ns": 1324555000, + "demand_misses": 264911, + "prefetch_issued": 884728, + "prefetch_hits": 619825, + "prefetch_wasted": 264903, + "achieved_accuracy": 0.700583, + "speedup_over_no_prefetch": 1.637221, + "recovered_fraction": 0.700576 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 5000, + "total_ns": 4421649000, + "stall_ns": 882705000, + "demand_misses": 176541, + "prefetch_issued": 884728, + "prefetch_hits": 708195, + "prefetch_wasted": 176533, + "achieved_accuracy": 0.800466, + "speedup_over_no_prefetch": 1.800827, + "recovered_fraction": 0.800459 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 5000, + "total_ns": 3980234000, + "stall_ns": 441290000, + "demand_misses": 88258, + "prefetch_issued": 884728, + "prefetch_hits": 796478, + "prefetch_wasted": 88250, + "achieved_accuracy": 0.900252, + "speedup_over_no_prefetch": 2.000542, + "recovered_fraction": 0.900244 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 5000, + "total_ns": 3760279000, + "stall_ns": 221335000, + "demand_misses": 44267, + "prefetch_issued": 884728, + "prefetch_hits": 840469, + "prefetch_wasted": 44259, + "achieved_accuracy": 0.949974, + "speedup_over_no_prefetch": 2.117562, + "recovered_fraction": 0.949966 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 5000, + "total_ns": 3538984000, + "stall_ns": 40000, + "demand_misses": 8, + "prefetch_issued": 884728, + "prefetch_hits": 884728, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 2.249975, + "recovered_fraction": 0.999991 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 10000, + "total_ns": 12386304000, + "stall_ns": 8847360000, + "demand_misses": 884736, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.9996, + "read_latency_ns": 10000, + "total_ns": 6194648000, + "stall_ns": 2655704000, + "demand_misses": 374, + "prefetch_issued": 884736, + "prefetch_hits": 884362, + "prefetch_wasted": 374, + "achieved_accuracy": 0.999577, + "speedup_over_no_prefetch": 1.999517, + "recovered_fraction": 0.699831 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 10000, + "total_ns": 12386304000, + "stall_ns": 8847360000, + "demand_misses": 884736, + "prefetch_issued": 884728, + "prefetch_hits": 0, + "prefetch_wasted": 884728, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 10000, + "total_ns": 11498504000, + "stall_ns": 7959560000, + "demand_misses": 795956, + "prefetch_issued": 884728, + "prefetch_hits": 88780, + "prefetch_wasted": 795948, + "achieved_accuracy": 0.100347, + "speedup_over_no_prefetch": 1.07721, + "recovered_fraction": 0.100346 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 10000, + "total_ns": 10610144000, + "stall_ns": 7071200000, + "demand_misses": 707120, + "prefetch_issued": 884728, + "prefetch_hits": 177616, + "prefetch_wasted": 707112, + "achieved_accuracy": 0.200758, + "speedup_over_no_prefetch": 1.167402, + "recovered_fraction": 0.200756 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 10000, + "total_ns": 9725144000, + "stall_ns": 6186200000, + "demand_misses": 618620, + "prefetch_issued": 884728, + "prefetch_hits": 266116, + "prefetch_wasted": 618612, + "achieved_accuracy": 0.300788, + "speedup_over_no_prefetch": 1.273637, + "recovered_fraction": 0.300786 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 10000, + "total_ns": 8840094000, + "stall_ns": 5301150000, + "demand_misses": 530115, + "prefetch_issued": 884728, + "prefetch_hits": 354621, + "prefetch_wasted": 530107, + "achieved_accuracy": 0.400825, + "speedup_over_no_prefetch": 1.401151, + "recovered_fraction": 0.400821 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 10000, + "total_ns": 7954254000, + "stall_ns": 4415310000, + "demand_misses": 441531, + "prefetch_issued": 884728, + "prefetch_hits": 443205, + "prefetch_wasted": 441523, + "achieved_accuracy": 0.500951, + "speedup_over_no_prefetch": 1.557192, + "recovered_fraction": 0.500946 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 10000, + "total_ns": 7071124000, + "stall_ns": 3532180000, + "demand_misses": 353218, + "prefetch_issued": 884728, + "prefetch_hits": 531518, + "prefetch_wasted": 353210, + "achieved_accuracy": 0.60077, + "speedup_over_no_prefetch": 1.751674, + "recovered_fraction": 0.600765 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 10000, + "total_ns": 6188054000, + "stall_ns": 2649110000, + "demand_misses": 264911, + "prefetch_issued": 884728, + "prefetch_hits": 619825, + "prefetch_wasted": 264903, + "achieved_accuracy": 0.700583, + "speedup_over_no_prefetch": 2.001648, + "recovered_fraction": 0.700576 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 10000, + "total_ns": 5304354000, + "stall_ns": 1765410000, + "demand_misses": 176541, + "prefetch_issued": 884728, + "prefetch_hits": 708195, + "prefetch_wasted": 176533, + "achieved_accuracy": 0.800466, + "speedup_over_no_prefetch": 2.33512, + "recovered_fraction": 0.800459 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 10000, + "total_ns": 4421524000, + "stall_ns": 882580000, + "demand_misses": 88258, + "prefetch_issued": 884728, + "prefetch_hits": 796478, + "prefetch_wasted": 88250, + "achieved_accuracy": 0.900252, + "speedup_over_no_prefetch": 2.801365, + "recovered_fraction": 0.900244 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 10000, + "total_ns": 3981614000, + "stall_ns": 442670000, + "demand_misses": 44267, + "prefetch_issued": 884728, + "prefetch_hits": 840469, + "prefetch_wasted": 44259, + "achieved_accuracy": 0.949974, + "speedup_over_no_prefetch": 3.110875, + "recovered_fraction": 0.949966 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 10000, + "total_ns": 3539024000, + "stall_ns": 80000, + "demand_misses": 8, + "prefetch_issued": 884728, + "prefetch_hits": 884728, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 3.499921, + "recovered_fraction": 0.999991 + }, + { + "policy": "none", + "requested_accuracy": 0.0, + "read_latency_ns": 20000, + "total_ns": 21233664000, + "stall_ns": 17694720000, + "demand_misses": 884736, + "prefetch_issued": 0, + "prefetch_hits": 0, + "prefetch_wasted": 0, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "next_page", + "requested_accuracy": 0.9996, + "read_latency_ns": 20000, + "total_ns": 10618328000, + "stall_ns": 7079384000, + "demand_misses": 374, + "prefetch_issued": 884736, + "prefetch_hits": 884362, + "prefetch_wasted": 374, + "achieved_accuracy": 0.999577, + "speedup_over_no_prefetch": 1.999718, + "recovered_fraction": 0.599915 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.0, + "read_latency_ns": 20000, + "total_ns": 21233664000, + "stall_ns": 17694720000, + "demand_misses": 884736, + "prefetch_issued": 884728, + "prefetch_hits": 0, + "prefetch_wasted": 884728, + "achieved_accuracy": 0.0, + "speedup_over_no_prefetch": 1.0, + "recovered_fraction": 0.0 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.1, + "read_latency_ns": 20000, + "total_ns": 19458064000, + "stall_ns": 15919120000, + "demand_misses": 795956, + "prefetch_issued": 884728, + "prefetch_hits": 88780, + "prefetch_wasted": 795948, + "achieved_accuracy": 0.100347, + "speedup_over_no_prefetch": 1.091253, + "recovered_fraction": 0.100346 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.2, + "read_latency_ns": 20000, + "total_ns": 17681344000, + "stall_ns": 14142400000, + "demand_misses": 707120, + "prefetch_issued": 884728, + "prefetch_hits": 177616, + "prefetch_wasted": 707112, + "achieved_accuracy": 0.200758, + "speedup_over_no_prefetch": 1.200908, + "recovered_fraction": 0.200756 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.3, + "read_latency_ns": 20000, + "total_ns": 15911344000, + "stall_ns": 12372400000, + "demand_misses": 618620, + "prefetch_issued": 884728, + "prefetch_hits": 266116, + "prefetch_wasted": 618612, + "achieved_accuracy": 0.300788, + "speedup_over_no_prefetch": 1.334498, + "recovered_fraction": 0.300786 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.4, + "read_latency_ns": 20000, + "total_ns": 14141244000, + "stall_ns": 10602300000, + "demand_misses": 530115, + "prefetch_issued": 884728, + "prefetch_hits": 354621, + "prefetch_wasted": 530107, + "achieved_accuracy": 0.400825, + "speedup_over_no_prefetch": 1.501541, + "recovered_fraction": 0.400821 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.5, + "read_latency_ns": 20000, + "total_ns": 12369564000, + "stall_ns": 8830620000, + "demand_misses": 441531, + "prefetch_issued": 884728, + "prefetch_hits": 443205, + "prefetch_wasted": 441523, + "achieved_accuracy": 0.500951, + "speedup_over_no_prefetch": 1.716606, + "recovered_fraction": 0.500946 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.6, + "read_latency_ns": 20000, + "total_ns": 10603304000, + "stall_ns": 7064360000, + "demand_misses": 353218, + "prefetch_issued": 884728, + "prefetch_hits": 531518, + "prefetch_wasted": 353210, + "achieved_accuracy": 0.60077, + "speedup_over_no_prefetch": 2.002552, + "recovered_fraction": 0.600765 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.7, + "read_latency_ns": 20000, + "total_ns": 8837164000, + "stall_ns": 5298220000, + "demand_misses": 264911, + "prefetch_issued": 884728, + "prefetch_hits": 619825, + "prefetch_wasted": 264903, + "achieved_accuracy": 0.700583, + "speedup_over_no_prefetch": 2.402769, + "recovered_fraction": 0.700576 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.8, + "read_latency_ns": 20000, + "total_ns": 7069764000, + "stall_ns": 3530820000, + "demand_misses": 176541, + "prefetch_issued": 884728, + "prefetch_hits": 708195, + "prefetch_wasted": 176533, + "achieved_accuracy": 0.800466, + "speedup_over_no_prefetch": 3.003447, + "recovered_fraction": 0.800459 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.9, + "read_latency_ns": 20000, + "total_ns": 5304104000, + "stall_ns": 1765160000, + "demand_misses": 88258, + "prefetch_issued": 884728, + "prefetch_hits": 796478, + "prefetch_wasted": 88250, + "achieved_accuracy": 0.900252, + "speedup_over_no_prefetch": 4.003252, + "recovered_fraction": 0.900244 + }, + { + "policy": "accuracy", + "requested_accuracy": 0.95, + "read_latency_ns": 20000, + "total_ns": 4424284000, + "stall_ns": 885340000, + "demand_misses": 44267, + "prefetch_issued": 884728, + "prefetch_hits": 840469, + "prefetch_wasted": 44259, + "achieved_accuracy": 0.949974, + "speedup_over_no_prefetch": 4.799345, + "recovered_fraction": 0.949966 + }, + { + "policy": "accuracy", + "requested_accuracy": 1.0, + "read_latency_ns": 20000, + "total_ns": 3539104000, + "stall_ns": 160000, + "demand_misses": 8, + "prefetch_issued": 884728, + "prefetch_hits": 884728, + "prefetch_wasted": 0, + "achieved_accuracy": 1.0, + "speedup_over_no_prefetch": 5.999729, + "recovered_fraction": 0.999991 + } + ], + "command": [ + "build-cpu/hbf_prefetch_bench", + "--stream", + "moe", + "--accesses", + "20000", + "--compute-ns", + "4000", + "--lead", + "8", + "--buffer-pages", + "64", + "--max-in-flight", + "32", + "--pages-per-expert", + "2304", + "--seed", + "7" + ] + } + ] +} diff --git a/include/hbfsim/prefetch_model.hpp b/include/hbfsim/prefetch_model.hpp new file mode 100644 index 0000000..5c87dea --- /dev/null +++ b/include/hbfsim/prefetch_model.hpp @@ -0,0 +1,114 @@ +#pragma once + +// A deterministic model of prefetching in front of the HBF tier. +// +// HBF's case rests on the accelerator issuing a read before the data is +// needed, so that the media latency is spent while something else runs. HBFSim +// models no prefetch at all: a registered access is charged where the load +// issues and nothing is ever fetched ahead of its use. This model exists to +// answer one question the paper has to answer without waiting for a full +// device-side implementation: how accurate does a prefetcher have to be before +// the HBF tier stops costing what it costs today? +// +// The model is a discrete-event simulation over an access stream, with no +// randomness beyond the seeded predictor, so a given configuration always +// produces the same numbers. +// +// WHAT IT DOES NOT CLAIM. This is not a measurement of any real predictor, of +// any GPU, or of any device. It says what a prefetcher of a stated accuracy +// and a stated lead distance would be worth against a stated media latency. It +// is the arithmetic the paper's latency argument rests on, made explicit and +// swept, not a hardware result. + +#include +#include + +namespace hbfsim { + +enum class PrefetchPolicy { + // Nothing is fetched ahead. This is what HBFSim does today and is the + // baseline every other policy is reported against. + None, + // On a demand for page N, fetch page N+1. Carries no model of the + // workload and needs no training: the naive policy. Its lead is one + // access by construction, so `lead_distance` only switches it on and off + // and does not change what it predicts. + NextPage, + // Predicts the page the stream will demand `lead_distance` accesses from + // now, and is correct with probability `accuracy`. This is how the + // accuracy axis is swept without inventing a predictor. + Accuracy, +}; + +struct PrefetchConfig { + PrefetchPolicy policy{PrefetchPolicy::None}; + + // Probability that an Accuracy-policy prediction names the page the + // stream will really demand. Ignored by the other policies. + double accuracy{1.0}; + + // How many accesses ahead of its use a prefetch is issued. 0 means the + // fetch is issued at the moment of use, which cannot hide anything. + std::uint32_t lead_distance{1}; + + // Pages that can be held after being fetched and before being used. A + // prefetch issued too far ahead is evicted before it is used. + std::uint32_t buffer_pages{64}; + + // How many media reads the device serves at once. + std::uint32_t max_in_flight{32}; + + // Media read latency, the tR the sweep varies. + std::uint64_t read_latency_ns{10'000}; + + // Time the accelerator spends between two accesses. This is what a + // prefetch has to hide behind, and setting it to zero means there is no + // work to overlap with. + std::uint64_t compute_ns_per_access{0}; + + std::uint64_t seed{1}; +}; + +struct PrefetchStats { + // Modeled time for the whole stream. + std::uint64_t total_ns{0}; + // Of total_ns, the part spent waiting for media rather than computing. + std::uint64_t stall_ns{0}; + std::uint64_t demand_accesses{0}; + // Demands that found no resident page and paid a full media read. + std::uint64_t demand_misses{0}; + // Demands that found a page a prefetch had already brought in. A page that + // was fetched early but has not arrived yet counts here too, and its + // remaining wait is in stall_ns. + std::uint64_t prefetch_hits{0}; + std::uint64_t prefetch_issued{0}; + // Prefetched pages that were evicted or left over without ever being used. + // These cost media bandwidth and bought nothing. + std::uint64_t prefetch_wasted{0}; + + // prefetch_hits / prefetch_issued. Zero when nothing was issued. + [[nodiscard]] double achieved_accuracy() const noexcept; +}; + +// `pages` is the sequence of media pages the workload demands, in order. +PrefetchStats simulate_prefetch(const std::vector& pages, + const PrefetchConfig& config); + +// Access streams the sweep runs over. `pages` is the number of distinct pages +// the stream may touch. +std::vector make_sequential_stream(std::uint64_t accesses); +std::vector make_random_stream(std::uint64_t accesses, + std::uint64_t pages, + std::uint64_t seed); +// Repeats a fixed set of layers, and within each layer picks `experts_per_token` +// out of `experts_per_layer` at random. The pages of a layer's chosen experts +// cannot be known before that layer's router has run, which is the case a +// next-page policy cannot serve. +std::vector make_moe_stream(std::uint64_t tokens, + std::uint64_t layers, + std::uint64_t experts_per_layer, + std::uint64_t experts_per_token, + std::uint64_t pages_per_expert, + std::uint64_t seed); + +} // namespace hbfsim diff --git a/scripts/run_prefetch_accuracy_sweep.py b/scripts/run_prefetch_accuracy_sweep.py new file mode 100755 index 0000000..d95231f --- /dev/null +++ b/scripts/run_prefetch_accuracy_sweep.py @@ -0,0 +1,152 @@ +#!/usr/bin/env python3 +"""Run the prefetch accuracy sweep and write the artifact the figure is drawn +from, plus a flat CSV for plotting. + +The figure has prefetch accuracy on the x axis and modeled time on the y axis, +one curve per media latency. Two reference points sit on every curve: the +no-prefetch baseline, which is what HBFSim models today, and the next-page +policy, which carries no model of the workload and is plotted at the accuracy +it actually achieved. + +Every number is produced by the model in src/prefetch/prefetch_model.cpp. None +of it is measured on a device or a GPU, and the artifact says so in its own +`disclaimer` field. The design behind the sweep is in +docs/46-预取实验设计.md. + +Published expert-predictor accuracies are written into the artifact so the +figure can mark them, with their sources. They are other people's numbers, not +ours, and are not produced by this sweep. +""" + +from __future__ import annotations + +import argparse +import csv +import json +import pathlib +import subprocess + + +ROOT = pathlib.Path(__file__).resolve().parents[1] + +# Access streams to sweep. `pages_per_expert` matters only for the moe stream: +# one Qwen3-30B-A3B expert is 3 * 2048 * 768 parameters in bf16, which is +# 9,437,184 bytes, or 2304 pages of 4 KiB. +STREAMS = ( + {"stream": "sequential", "pages_per_expert": 2}, + {"stream": "random", "pages_per_expert": 2}, + {"stream": "moe", "pages_per_expert": 2304}, +) + +# Other people's measured predictor accuracies, for marking on the x axis. +# Each entry names where the number comes from; none of them is ours. +PUBLISHED_PREDICTORS = ( + {"name": "ProMoE cross-layer gate on Qwen2-MoE", "accuracy": 0.669, + "source": "arXiv:2410.22134"}, + {"name": "Fate, no training", "accuracy": 0.7879, + "source": "arXiv:2502.12224"}, + {"name": "DAOP on Mixtral 8x7B", "accuracy": 0.8411, + "source": "arXiv:2501.10375"}, + {"name": "ProMoE learned predictor, average", "accuracy": 0.847, + "source": "arXiv:2410.22134"}, + {"name": "AdapMoE", "accuracy": 0.90, "source": "arXiv:2408.10284"}, + {"name": "HOBBIT next-layer top-1", "accuracy": 0.96, + "source": "arXiv:2411.01433"}, + {"name": "Fate with over-fetch", "accuracy": 0.9715, + "source": "arXiv:2502.12224; about 3.75x the weight traffic"}, +) + + +def run_stream(binary: pathlib.Path, stream: str, accesses: int, + compute_ns: int, lead: int, buffer_pages: int, + max_in_flight: int, pages_per_expert: int, seed: int) -> dict: + command = [ + str(binary), "--stream", stream, "--accesses", str(accesses), + "--compute-ns", str(compute_ns), "--lead", str(lead), + "--buffer-pages", str(buffer_pages), + "--max-in-flight", str(max_in_flight), + "--pages-per-expert", str(pages_per_expert), "--seed", str(seed), + ] + output = subprocess.run(command, cwd=ROOT, check=True, + capture_output=True, text=True, timeout=1800) + result = json.loads(output.stdout) + result["command"] = command + return result + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--build-dir", type=pathlib.Path, + default=ROOT / "build") + parser.add_argument("--accesses", type=int, default=20000) + parser.add_argument("--compute-ns", type=int, default=4000, + help="accelerator time between two accesses; this is " + "what a prefetch hides behind") + parser.add_argument("--lead", type=int, default=8) + parser.add_argument("--buffer-pages", type=int, default=64) + parser.add_argument("--max-in-flight", type=int, default=32) + parser.add_argument("--seed", type=int, default=7) + parser.add_argument("--output", type=pathlib.Path, + default=ROOT / "docs/proofs/artifacts" + "/prefetch-accuracy-sweep.json") + parser.add_argument("--csv", type=pathlib.Path, + default=ROOT / "docs/proofs/artifacts" + "/prefetch-accuracy-sweep.csv") + args = parser.parse_args() + + binary = args.build_dir / "hbf_prefetch_bench" + if not binary.is_file(): + raise RuntimeError(f"hbf_prefetch_bench is not built: {binary}") + + report: dict = { + "schema_version": 1, + "disclaimer": "modeled, not measured on any device or GPU", + "model": "src/prefetch/prefetch_model.cpp", + "design": "docs/46-预取实验设计.md", + "published_predictors": list(PUBLISHED_PREDICTORS), + "runs": [], + } + + rows: list[dict] = [] + for entry in STREAMS: + result = run_stream(binary, entry["stream"], args.accesses, + args.compute_ns, args.lead, args.buffer_pages, + args.max_in_flight, entry["pages_per_expert"], + args.seed) + report["runs"].append(result) + for cell in result["cells"]: + row = dict(cell) + row["stream"] = entry["stream"] + rows.append(row) + naive = [c for c in result["cells"] if c["policy"] == "next_page"] + if naive: + worst = max(naive, key=lambda c: c["read_latency_ns"]) + print(f"{entry['stream']:>11}: next-page reaches accuracy " + f"{worst['achieved_accuracy']:.5f}, speedup " + f"{worst['speedup_over_no_prefetch']:.2f} at tR=" + f"{worst['read_latency_ns']} ns, " + f"{worst['demand_misses']} demand misses left") + + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps(report, indent=2) + "\n") + print(f"wrote {args.output}") + + if rows: + args.csv.parent.mkdir(parents=True, exist_ok=True) + fields = ["stream", "policy", "requested_accuracy", + "achieved_accuracy", "read_latency_ns", "total_ns", + "stall_ns", "demand_misses", "prefetch_issued", + "prefetch_hits", "prefetch_wasted", + "speedup_over_no_prefetch", "recovered_fraction"] + with args.csv.open("w", newline="") as handle: + writer = csv.DictWriter(handle, fieldnames=fields) + writer.writeheader() + for row in rows: + writer.writerow({field: row.get(field, "") + for field in fields}) + print(f"wrote {args.csv}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/prefetch/prefetch_model.cpp b/src/prefetch/prefetch_model.cpp new file mode 100644 index 0000000..5172979 --- /dev/null +++ b/src/prefetch/prefetch_model.cpp @@ -0,0 +1,253 @@ +#include + +#include +#include +#include +#include +#include +#include + +namespace hbfsim { +namespace { + +// A page number no stream produces, used when a prediction is wrong. It has to +// be outside every generated stream so a wrong prediction can never be right +// by accident. +constexpr std::uint64_t kWrongPageBase = 1ULL << 60; + +// The media serves `max_in_flight` reads at once. Each server is described by +// the time it next becomes free, so a read issued at `now` starts at +// max(now, earliest free) and finishes one read latency later. This is the +// same shape as the queue depth the MQSim adapter enforces. +class MediaServers { + public: + MediaServers(std::uint32_t count, std::uint64_t read_latency_ns) + : read_latency_ns_(read_latency_ns) + { + // A min-heap rather than a scan, so a large concurrency setting costs + // log(count) per access instead of count. The sweep runs streams of + // roughly a million accesses against concurrencies in the thousands, + // where the scan would dominate the run. + const auto servers = std::max(1, count); + for (std::uint32_t index = 0; index < servers; ++index) { + free_at_.push(0); + } + } + + // Occupies the server that frees first and returns when the read lands. + std::uint64_t issue(std::uint64_t now) + { + const auto earliest = free_at_.top(); + free_at_.pop(); + const auto start = std::max(now, earliest); + const auto completion = start + read_latency_ns_; + free_at_.push(completion); + return completion; + } + + private: + std::priority_queue, + std::greater> + free_at_; + std::uint64_t read_latency_ns_; +}; + +// Pages that have been fetched and not yet used, with the time each lands. +// Eviction is oldest-first: a page fetched too far ahead of its use is the one +// pushed out when the buffer is full. +class StagingBuffer { + public: + explicit StagingBuffer(std::uint32_t capacity) + : capacity_(std::max(1, capacity)) + { + } + + [[nodiscard]] bool holds(std::uint64_t page) const + { + return arrival_.contains(page); + } + + // Returns how many pages were dropped to make room. + std::uint64_t insert(std::uint64_t page, std::uint64_t arrival_ns) + { + arrival_.emplace(page, arrival_ns); + order_.push_back(page); + std::uint64_t evicted = 0; + while (order_.size() > capacity_) { + const auto victim = order_.front(); + order_.pop_front(); + if (arrival_.erase(victim) != 0) { + ++evicted; + } + } + return evicted; + } + + // Removes the page and reports when it landed. + std::uint64_t take(std::uint64_t page) + { + const auto found = arrival_.find(page); + const auto arrival = found->second; + arrival_.erase(found); + const auto position = std::find(order_.begin(), order_.end(), page); + if (position != order_.end()) { + order_.erase(position); + } + return arrival; + } + + [[nodiscard]] std::size_t size() const { return arrival_.size(); } + + private: + std::uint32_t capacity_; + std::unordered_map arrival_; + std::deque order_; +}; + +} // namespace + +double PrefetchStats::achieved_accuracy() const noexcept +{ + if (prefetch_issued == 0) { + return 0.0; + } + return static_cast(prefetch_hits) / + static_cast(prefetch_issued); +} + +PrefetchStats simulate_prefetch(const std::vector& pages, + const PrefetchConfig& config) +{ + PrefetchStats stats{}; + stats.demand_accesses = pages.size(); + if (pages.empty()) { + return stats; + } + + MediaServers media(config.max_in_flight, config.read_latency_ns); + StagingBuffer buffer(config.buffer_pages); + std::mt19937_64 rng(config.seed); + std::uniform_real_distribution draw(0.0, 1.0); + + std::uint64_t clock = 0; + std::uint64_t wrong_page = kWrongPageBase; + + for (std::size_t index = 0; index < pages.size(); ++index) { + // The accelerator does its work for this access first. This is the + // interval a prefetch issued earlier has been hiding behind. + clock += config.compute_ns_per_access; + + // Issue the prefetch for a later access. Lead distance 0 means the + // fetch is issued at the point of use, which is not a prefetch and + // cannot hide anything. + if (config.policy != PrefetchPolicy::None && config.lead_distance > 0) { + const auto target = index + config.lead_distance; + bool predict = false; + std::uint64_t predicted = 0; + if (config.policy == PrefetchPolicy::NextPage) { + // Carries no model of the workload: the page after this one. + predicted = pages[index] + 1; + predict = true; + } else if (target < pages.size()) { + if (draw(rng) < config.accuracy) { + predicted = pages[target]; + } else { + predicted = wrong_page++; + } + predict = true; + } + if (predict && !buffer.holds(predicted)) { + const auto arrival = media.issue(clock); + stats.prefetch_wasted += buffer.insert(predicted, arrival); + ++stats.prefetch_issued; + } + } + + // Resolve the demand. + const auto page = pages[index]; + if (buffer.holds(page)) { + const auto arrival = buffer.take(page); + ++stats.prefetch_hits; + if (arrival > clock) { + // Fetched early, but not early enough: the remainder is stall. + stats.stall_ns += arrival - clock; + clock = arrival; + } + } else { + const auto arrival = media.issue(clock); + ++stats.demand_misses; + stats.stall_ns += arrival - clock; + clock = arrival; + } + } + + // Anything still held was fetched and never used. + stats.prefetch_wasted += buffer.size(); + stats.total_ns = clock; + return stats; +} + +std::vector make_sequential_stream(std::uint64_t accesses) +{ + std::vector pages; + pages.reserve(accesses); + for (std::uint64_t index = 0; index < accesses; ++index) { + pages.push_back(index); + } + return pages; +} + +std::vector make_random_stream(std::uint64_t accesses, + std::uint64_t pages_available, + std::uint64_t seed) +{ + std::vector pages; + pages.reserve(accesses); + std::mt19937_64 rng(seed); + std::uniform_int_distribution pick( + 0, pages_available == 0 ? 0 : pages_available - 1); + for (std::uint64_t index = 0; index < accesses; ++index) { + pages.push_back(pick(rng)); + } + return pages; +} + +std::vector make_moe_stream(std::uint64_t tokens, + std::uint64_t layers, + std::uint64_t experts_per_layer, + std::uint64_t experts_per_token, + std::uint64_t pages_per_expert, + std::uint64_t seed) +{ + std::vector pages; + if (experts_per_layer == 0 || pages_per_expert == 0) { + return pages; + } + const auto chosen = std::min(experts_per_token, experts_per_layer); + pages.reserve(tokens * layers * chosen * pages_per_expert); + std::mt19937_64 rng(seed); + std::vector experts(experts_per_layer); + for (std::uint64_t token = 0; token < tokens; ++token) { + for (std::uint64_t layer = 0; layer < layers; ++layer) { + for (std::uint64_t expert = 0; expert < experts_per_layer; + ++expert) { + experts[expert] = expert; + } + // The router picks this layer's experts for this token. Which ones + // is not known until the previous layer has produced its output, + // so no policy that looks only at addresses can predict them. + std::shuffle(experts.begin(), experts.end(), rng); + for (std::uint64_t slot = 0; slot < chosen; ++slot) { + const auto base = + (layer * experts_per_layer + experts[slot]) * + pages_per_expert; + for (std::uint64_t page = 0; page < pages_per_expert; ++page) { + pages.push_back(base + page); + } + } + } + } + return pages; +} + +} // namespace hbfsim diff --git a/tests/cpu/prefetch_model_test.cpp b/tests/cpu/prefetch_model_test.cpp new file mode 100644 index 0000000..dc96c10 --- /dev/null +++ b/tests/cpu/prefetch_model_test.cpp @@ -0,0 +1,211 @@ +// Contract for the prefetch model. Every assertion here is a property the +// paper's latency argument depends on, so a change that breaks one of them +// changes what the paper may claim. +// +// Written before include/hbfsim/prefetch_model.hpp had an implementation. + +#include + +#include +#include + +#define CHECK(condition) \ + do { \ + if (!(condition)) { \ + std::printf("failed at line %d: %s\n", __LINE__, #condition); \ + return __LINE__; \ + } \ + } while (false) + +namespace { + +hbfsim::PrefetchConfig base_config() +{ + hbfsim::PrefetchConfig config{}; + config.lead_distance = 4; + config.buffer_pages = 64; + config.max_in_flight = 32; + config.read_latency_ns = 10'000; + // Each access has real work behind it, so a prefetch has something to hide + // behind. Without this every policy is bounded by media time alone. + config.compute_ns_per_access = 4'000; + config.seed = 7; + return config; +} + +} // namespace + +int main() +{ + const auto sequential = hbfsim::make_sequential_stream(2'000); + const auto random_stream = hbfsim::make_random_stream(2'000, 100'000, 11); + + // 1. No prefetch pays a full media read on every distinct page. + hbfsim::PrefetchStats none{}; + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::None; + none = hbfsim::simulate_prefetch(sequential, config); + CHECK(none.demand_accesses == sequential.size()); + CHECK(none.prefetch_issued == 0); + CHECK(none.demand_misses == sequential.size()); + CHECK(none.stall_ns > 0); + } + + // 2. The model is deterministic: the same configuration twice gives the + // same numbers, or no swept figure is reproducible. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 0.5; + const auto first = hbfsim::simulate_prefetch(sequential, config); + const auto second = hbfsim::simulate_prefetch(sequential, config); + CHECK(first.total_ns == second.total_ns); + CHECK(first.prefetch_hits == second.prefetch_hits); + } + + // 3. Accuracy 0 buys nothing. It must not come out faster than no prefetch + // at all, and it must waste every page it fetched. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 0.0; + const auto stats = hbfsim::simulate_prefetch(sequential, config); + CHECK(stats.prefetch_hits == 0); + CHECK(stats.prefetch_issued > 0); + CHECK(stats.prefetch_wasted == stats.prefetch_issued); + CHECK(stats.total_ns >= none.total_ns); + } + + // 4. Perfect accuracy with enough lead, buffer and concurrency removes + // every stall except the unavoidable warm-up. A prefetcher issuing L + // accesses ahead cannot cover the first L accesses: nothing predicted + // them. So the misses are exactly the warm-up, and the whole stream costs + // its compute plus at most those L media reads. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 1.0; + config.lead_distance = 8; + const auto stats = hbfsim::simulate_prefetch(sequential, config); + CHECK(stats.demand_misses == config.lead_distance); + CHECK(stats.total_ns <= + sequential.size() * config.compute_ns_per_access + + config.lead_distance * config.read_latency_ns); + // The steady state carries no stall at all, so the whole run must be + // far cheaper than paying a media read on every access. + CHECK(stats.total_ns < none.total_ns / 2); + } + + // 5. Lead distance 0 hides nothing even at perfect accuracy. This is the + // point the paper cannot drop: knowing the address is not the same as + // having time to use it. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 1.0; + config.lead_distance = 0; + const auto stats = hbfsim::simulate_prefetch(sequential, config); + CHECK(stats.stall_ns > 0); + CHECK(stats.total_ns >= none.total_ns); + } + + // 6. Time is monotone in accuracy: more accurate never costs more. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + std::uint64_t previous = 0; + bool first = true; + for (const double accuracy : {0.0, 0.25, 0.5, 0.75, 1.0}) { + config.accuracy = accuracy; + const auto stats = hbfsim::simulate_prefetch(sequential, config); + if (!first) { + CHECK(stats.total_ns <= previous); + } + previous = stats.total_ns; + first = false; + } + } + + // 7. A one-page buffer cannot hold a prefetch issued eight accesses early, + // so a large lead with a tiny buffer must do worse than the same lead with + // a large buffer. Capacity is a real limit, not a formality. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 1.0; + config.lead_distance = 8; + config.buffer_pages = 1; + const auto small = hbfsim::simulate_prefetch(sequential, config); + config.buffer_pages = 64; + const auto large = hbfsim::simulate_prefetch(sequential, config); + CHECK(small.total_ns > large.total_ns); + } + + // 8. The naive next-page policy is near-perfect on a sequential stream and + // worthless on a random one. This is the claim the paper wants to make + // about a prefetcher that carries no model of the workload. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::NextPage; + config.lead_distance = 1; + const auto ordered = hbfsim::simulate_prefetch(sequential, config); + CHECK(ordered.achieved_accuracy() > 0.9); + CHECK(ordered.total_ns < none.total_ns); + + const auto scattered = hbfsim::simulate_prefetch(random_stream, config); + CHECK(scattered.achieved_accuracy() < 0.1); + } + + // 9. Concurrency bounds the benefit: with one media read at a time, a + // perfect prefetcher still cannot keep up with a stream whose compute is + // shorter than one read. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 1.0; + config.lead_distance = 8; + config.max_in_flight = 1; + const auto stats = hbfsim::simulate_prefetch(sequential, config); + CHECK(stats.stall_ns > 0); + } + + // 10. A Mixture-of-Experts stream is where the next-page policy has + // nothing to go on, because the pages of a layer are chosen per token. + { + const auto moe = hbfsim::make_moe_stream(40, 48, 128, 8, 2, 5); + CHECK(!moe.empty()); + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::NextPage; + const auto naive = hbfsim::simulate_prefetch(moe, config); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 1.0; + const auto perfect = hbfsim::simulate_prefetch(moe, config); + CHECK(perfect.total_ns < naive.total_ns); + } + + // 11. Every prefetched page is either used or wasted; nothing is lost. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 0.6; + const auto stats = hbfsim::simulate_prefetch(sequential, config); + CHECK(stats.prefetch_hits + stats.prefetch_wasted == + stats.prefetch_issued); + CHECK(stats.prefetch_hits + stats.demand_misses == + stats.demand_accesses); + } + + // 12. Total time always accounts for itself. + { + auto config = base_config(); + config.policy = hbfsim::PrefetchPolicy::Accuracy; + config.accuracy = 0.8; + const auto stats = hbfsim::simulate_prefetch(sequential, config); + CHECK(stats.total_ns == + sequential.size() * config.compute_ns_per_access + + stats.stall_ns); + } + + return 0; +} From 393b99079273e38770c159ca7a3b149324a995ad Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Sat, 29 Aug 2026 12:32:17 -0700 Subject: [PATCH 02/10] Add capacity-mode readahead, and measure what it is worth The prefetch model in this branch says what a prefetcher would be worth. Nothing made HBFSim actually fetch anything ahead of its use. This adds that for capacity mode: on a demand miss for page N, the page service queues N+1 through N+k, and the worker fetches them when it has no request to serve. This is a system-side prefetcher, not an application-side one. The application does not ask for it and does not know it happened, and the only thing the policy has to go on is which pages have already been demanded. It is off unless switched on, so no existing measurement moves. THREE DESIGN DECISIONS, THE SECOND OF WHICH I GOT WRONG FIRST. Queueing happens after the demand is served, never before. The miss path is a synchronous blocking read under one mutex, so reading a second page there would double the synchronous work on the path a warp is blocked on. Readahead may take a clean victim once the cache is full. The first version refused to evict at all, and measurement showed that makes the feature do nothing in the only regime capacity mode exists for: on a 1024-access Mixture-of-Experts stream with 128 frames it fetched 22 pages and skipped 1970, because a warm cache never has a free frame. A dirty victim is still refused and put back, because writing it costs a program on the media and a program is dearer than the read the readahead would save. A readahead page is published unreferenced, so hbm_cache.cpp publish() gained a defaulted `referenced` parameter and every existing caller is unchanged. The replacement policy is CLOCK, so a page nothing ever asks for is now the first candidate to be evicted again rather than the last. WHAT IT IS WORTH. hbf_capacity_readahead_bench drives the real page service and reports media reads, not wall-clock time. With the readahead depth at least as large as one expert, the implementation reproduces what the model predicts for a next-page policy, which is (P-1)/P for P pages per expert: P=4 depth=4 implementation 0.7559 model 0.7500 P=8 depth=8 implementation 0.8916 model 0.8750 P=16 depth=16 implementation 0.9370 model 0.9375 P=16 depth=32 implementation 0.9443 model 0.9375 Below that the depth caps the hit rate near D/(D+1): at depth 4, P of 8 and 16 both stop at 0.7500 and 0.7568. THE RESULT THAT MATTERS MOST IS A CLIFF. At P=8, depth 8, 256 frames, varying how many queued pages the worker drains between two demands: drain fully hit 0.8916 media reads -86.79% drain 8 hit 0.8916 media reads -86.79% drain 4 hit 0.8916 media reads -86.79% drain 2 hit 0.8916 media reads -86.79% drain 1 hit 0.0635 media reads +14.17% Draining slower than the queue fills does not degrade the benefit gradually. It turns an 86.79% reduction in media reads into a 14.17% increase, because queued pages go stale before they are used while still costing bandwidth and frames. Any deployment of this has to keep the drain rate above the fill rate. tests/cpu/capacity_readahead_test.cpp covers the contract: off by default, the queue is not drained on the demand's own path, a dirty victim is refused, a demanded page survives the readahead that follows it, a page that became resident between queueing and draining is skipped, and a readahead past the end of the backing store is skipped rather than failed. Test suite: the same four tests fail before and after (context_lifecycle, vmem_tuning, run_with_bpftime, mqsim_benchmark), all pre-existing in a CPU-only build; the count goes from 33 to 34. Co-Authored-By: Claude Opus 5 (1M context) --- CMakeLists.txt | 12 + .../prefetch/hbf_capacity_readahead_bench.cpp | 280 ++++++++++++++++++ src/cuda_runtime/hbm_cache.cpp | 5 +- src/cuda_runtime/hbm_cache.hpp | 7 +- src/host_service/capacity_page_service.cpp | 140 +++++++++ src/host_service/capacity_page_service.hpp | 31 +- src/host_service/capacity_worker.cpp | 14 + tests/cpu/capacity_readahead_test.cpp | 216 ++++++++++++++ 8 files changed, 701 insertions(+), 4 deletions(-) create mode 100644 benchmarks/prefetch/hbf_capacity_readahead_bench.cpp create mode 100644 tests/cpu/capacity_readahead_test.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 5084b41..7c55521 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -112,6 +112,12 @@ set_tests_properties(profile PROPERTIES WORKING_DIRECTORY "${CMAKE_CURRENT_SOURC add_executable(hbf_prefetch_bench benchmarks/prefetch/hbf_prefetch_bench.cpp) target_link_libraries(hbf_prefetch_bench PRIVATE hbfsim_core) +add_executable( + hbf_capacity_readahead_bench + benchmarks/prefetch/hbf_capacity_readahead_bench.cpp +) +target_link_libraries(hbf_capacity_readahead_bench PRIVATE hbfsim_core) + add_executable(prefetch_model_test tests/cpu/prefetch_model_test.cpp) target_link_libraries(prefetch_model_test PRIVATE hbfsim_core) add_test(NAME prefetch_model COMMAND prefetch_model_test) @@ -175,6 +181,12 @@ add_test( NAME capacity_page_service COMMAND hbfsim_capacity_page_service_tests ) +add_executable( + hbfsim_capacity_readahead_tests tests/cpu/capacity_readahead_test.cpp +) +target_link_libraries(hbfsim_capacity_readahead_tests PRIVATE hbfsim_core) +add_test(NAME capacity_readahead COMMAND hbfsim_capacity_readahead_tests) + add_executable( hbfsim_capacity_handoff_tests tests/cpu/capacity_handoff_test.cpp ) diff --git a/benchmarks/prefetch/hbf_capacity_readahead_bench.cpp b/benchmarks/prefetch/hbf_capacity_readahead_bench.cpp new file mode 100644 index 0000000..6416b32 --- /dev/null +++ b/benchmarks/prefetch/hbf_capacity_readahead_bench.cpp @@ -0,0 +1,280 @@ +// Drives the real capacity-mode readahead over an access stream and reports +// how many demands reached the media with readahead off and with it on. +// +// This measures the implementation in src/host_service/capacity_page_service.cpp, +// not the model in src/prefetch/prefetch_model.cpp. The two answer different +// questions and are reported separately on purpose: the model says what a +// prefetcher of a stated accuracy would be worth, and this says what the +// readahead actually built into the page service achieves on the same stream. +// Where both can speak -- the hit rate a next-page policy reaches on a +// Mixture-of-Experts stream -- they should agree, and disagreement is a defect +// in one of them. +// +// Nothing here runs on a GPU. It exercises the host-side page service directly, +// so it reports media reads avoided, not wall-clock time. + +#include + +#include "../../src/cuda_runtime/hbm_cache.hpp" +#include "../../src/host_service/backing_store.hpp" +#include "../../src/host_service/capacity_page_service.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +namespace { + +constexpr std::size_t kPageBytes = 4096; + +struct Options { + std::string stream{"moe"}; + std::uint64_t frames{64}; + std::uint32_t readahead{4}; + // Kept small on purpose: the backing store is a real file, so the stream's + // highest page number decides how large it has to be. + std::uint64_t layers{8}; + std::uint64_t experts_per_layer{16}; + std::uint64_t experts_per_token{4}; + std::uint64_t pages_per_expert{8}; + std::uint64_t tokens{4}; + std::uint64_t accesses{2048}; + std::uint64_t seed{5}; + // 0 means the worker fully keeps up with the queue. + std::uint64_t drain_per_demand{0}; +}; + +struct Run { + std::uint64_t demands{0}; + std::uint64_t media_reads{0}; + std::uint64_t hits{0}; + std::uint64_t readahead_fetched{0}; + std::uint64_t readahead_skipped{0}; +}; + +std::vector build_stream(const Options& options) +{ + if (options.stream == "sequential") { + return hbfsim::make_sequential_stream(options.accesses); + } + if (options.stream == "random") { + return hbfsim::make_random_stream(options.accesses, options.accesses, + options.seed); + } + return hbfsim::make_moe_stream(options.tokens, options.layers, + options.experts_per_layer, + options.experts_per_token, + options.pages_per_expert, options.seed); +} + +Run drive(const std::vector& pages, const Options& options, + std::uint32_t readahead_pages, std::uint64_t drain_per_demand, + hbfsim::host_service::BackingStore& backing, + std::unordered_map>& memory, + const std::vector& frame_addresses) +{ + hbfsim::runtime::HbmCache cache(frame_addresses); + hbfsim::host_service::CapacityPageService service( + backing, cache, kPageBytes, + { + .host_to_frame = [&](std::uint64_t frame, + std::span data) { + std::ranges::copy(data, memory.at(frame).begin()); + return true; + }, + .frame_to_host = [&](std::uint64_t frame, + std::span data) { + std::ranges::copy(memory.at(frame), data.begin()); + return true; + }, + }); + service.set_readahead_pages(readahead_pages); + + Run run{}; + for (const auto page : pages) { + const auto resolved = service.resolve(page, 0); + if (resolved.status != hbfsim::RequestStatus::Ready) { + std::fprintf(stderr, "resolve failed on page %llu\n", + static_cast(page)); + std::exit(70); + } + ++run.demands; + if ((resolved.media.flags & + hbfsim::host_service::CapacityMediaRead) != 0) { + ++run.media_reads; + } else { + ++run.hits; + } + // Stands in for the worker's idle time between two demands. + // drain_per_demand is how many queued pages that idle time is worth; + // 0 means the worker keeps up with the queue completely, which is the + // most favourable assumption. Draining more slowly than the queue + // fills makes prefetched pages arrive after the demand that wanted + // them, so this parameter decides the result and has to be reported + // with it. + if (drain_per_demand == 0) { + while (service.run_one_readahead()) { + } + } else { + for (std::uint64_t drained = 0; drained < drain_per_demand; + ++drained) { + if (!service.run_one_readahead()) { + break; + } + } + } + } + // Anything still queued when the stream ends never had a chance to help. + while (service.run_one_readahead()) { + } + run.readahead_fetched = service.readahead_pages_fetched(); + run.readahead_skipped = service.readahead_pages_skipped(); + return run; +} + +void print_run(const char* label, const Run& run, std::uint64_t baseline_reads) +{ + const double hit_rate = + run.demands == 0 ? 0.0 + : static_cast(run.hits) / + static_cast(run.demands); + const double avoided = + baseline_reads == 0 + ? 0.0 + : 1.0 - static_cast(run.media_reads) / + static_cast(baseline_reads); + std::printf( + " {\"policy\": \"%s\", \"demands\": %llu, \"media_reads\": %llu, " + "\"hits\": %llu, \"hit_rate\": %.6f, " + "\"media_reads_avoided_fraction\": %.6f, " + "\"readahead_fetched\": %llu, \"readahead_skipped\": %llu}", + label, static_cast(run.demands), + static_cast(run.media_reads), + static_cast(run.hits), hit_rate, avoided, + static_cast(run.readahead_fetched), + static_cast(run.readahead_skipped)); +} + +bool take_u64(int argc, char** argv, int& index, const char* name, + std::uint64_t& out) +{ + if (std::strcmp(argv[index], name) != 0) { + return false; + } + if (index + 1 >= argc) { + std::fprintf(stderr, "missing value for %s\n", name); + std::exit(64); + } + out = std::strtoull(argv[++index], nullptr, 10); + return true; +} + +} // namespace + +int main(int argc, char** argv) +{ + Options options{}; + std::uint64_t scratch = 0; + for (int index = 1; index < argc; ++index) { + if (std::strcmp(argv[index], "--stream") == 0 && index + 1 < argc) { + options.stream = argv[++index]; + } else if (take_u64(argc, argv, index, "--frames", scratch)) { + options.frames = scratch; + } else if (take_u64(argc, argv, index, "--readahead", scratch)) { + options.readahead = static_cast(scratch); + } else if (take_u64(argc, argv, index, "--pages-per-expert", + scratch)) { + options.pages_per_expert = scratch; + } else if (take_u64(argc, argv, index, "--tokens", scratch)) { + options.tokens = scratch; + } else if (take_u64(argc, argv, index, "--accesses", scratch)) { + options.accesses = scratch; + } else if (take_u64(argc, argv, index, "--seed", scratch)) { + options.seed = scratch; + } else if (take_u64(argc, argv, index, "--drain-per-demand", + scratch)) { + options.drain_per_demand = scratch; + } else { + std::fprintf(stderr, + "usage: hbf_capacity_readahead_bench " + "[--stream sequential|random|moe] [--frames N] " + "[--readahead N] [--pages-per-expert N] " + "[--tokens N] [--accesses N] [--seed N]\n"); + return 64; + } + } + + const auto pages = build_stream(options); + if (pages.empty()) { + std::fprintf(stderr, "empty access stream\n"); + return 65; + } + const auto highest = *std::ranges::max_element(pages); + // Room for the highest page the stream touches plus everything the + // readahead may run past the end into. + const auto store_pages = highest + options.readahead + 2; + + const auto path = std::filesystem::temp_directory_path() / + ("hbfsim-readahead-bench-" + + std::to_string(static_cast(::getpid()))); + std::filesystem::remove(path); + auto backing = hbfsim::host_service::BackingStore::create_deterministic( + path, store_pages * kPageBytes, 0x13579); + + std::unordered_map> memory; + std::vector frame_addresses; + for (std::uint64_t index = 0; index < options.frames; ++index) { + const auto address = 0x100000 + index * 0x1000; + memory.emplace(address, std::vector(kPageBytes)); + frame_addresses.push_back(address); + } + + const auto without = drive(pages, options, 0, options.drain_per_demand, + backing, memory, frame_addresses); + const auto with = drive(pages, options, options.readahead, + options.drain_per_demand, backing, memory, + frame_addresses); + + std::printf("{\n"); + std::printf(" \"schema_version\": 1,\n"); + std::printf( + " \"measures\": " + "\"src/host_service/capacity_page_service.cpp readahead\",\n"); + std::printf( + " \"disclaimer\": \"host-side page service only; no GPU, and media " + "reads avoided rather than wall-clock time\",\n"); + std::printf( + " \"workload\": {\"stream\": \"%s\", \"accesses\": %llu, " + "\"distinct_pages_in_store\": %llu, \"frames\": %llu, " + "\"readahead_pages\": %u, \"pages_per_expert\": %llu, " + "\"tokens\": %llu, \"layers\": %llu, \"experts_per_layer\": %llu, " + "\"experts_per_token\": %llu, \"seed\": %llu},\n", + options.stream.c_str(), + static_cast(pages.size()), + static_cast(store_pages), + static_cast(options.frames), options.readahead, + static_cast(options.pages_per_expert), + static_cast(options.tokens), + static_cast(options.layers), + static_cast(options.experts_per_layer), + static_cast(options.experts_per_token), + static_cast(options.seed), + static_cast(options.drain_per_demand)); + std::printf(" \"runs\": [\n"); + print_run("demand_only", without, without.media_reads); + std::printf(",\n"); + print_run("readahead", with, without.media_reads); + std::printf("\n ]\n}\n"); + + std::filesystem::remove(path); + return 0; +} diff --git a/src/cuda_runtime/hbm_cache.cpp b/src/cuda_runtime/hbm_cache.cpp index 100f879..e320d7b 100644 --- a/src/cuda_runtime/hbm_cache.cpp +++ b/src/cuda_runtime/hbm_cache.cpp @@ -19,7 +19,8 @@ HbmCache::HbmCache(std::vector frame_addresses) } bool HbmCache::publish(std::uint64_t logical_page, - std::uint64_t frame_address) + std::uint64_t frame_address, + bool referenced) { std::lock_guard lock(mutex_); const auto frame = address_to_frame_.find(frame_address); @@ -32,7 +33,7 @@ bool HbmCache::publish(std::uint64_t logical_page, } auto& state = frames_[frame->second]; state.logical_page = logical_page; - state.referenced = true; + state.referenced = referenced; state.dirty = false; page_to_frame_.emplace(logical_page, frame->second); return true; diff --git a/src/cuda_runtime/hbm_cache.hpp b/src/cuda_runtime/hbm_cache.hpp index af3f15b..0636688 100644 --- a/src/cuda_runtime/hbm_cache.hpp +++ b/src/cuda_runtime/hbm_cache.hpp @@ -20,8 +20,13 @@ class HbmCache { public: explicit HbmCache(std::vector frame_addresses); + // `referenced` decides where the page starts in the CLOCK scan. A demand + // publishes referenced, so the page survives one sweep. A readahead + // publishes unreferenced, so a page nothing ever asked for is the first + // candidate to be evicted again. bool publish(std::uint64_t logical_page, - std::uint64_t frame_address); + std::uint64_t frame_address, + bool referenced = true); [[nodiscard]] std::optional resolve( std::uint64_t logical_page); [[nodiscard]] std::optional reclaim_eviction( diff --git a/src/host_service/capacity_page_service.cpp b/src/host_service/capacity_page_service.cpp index 7e2d793..40ec7e4 100644 --- a/src/host_service/capacity_page_service.cpp +++ b/src/host_service/capacity_page_service.cpp @@ -178,6 +178,11 @@ CapacityResolveResult CapacityPageService::resolve( resident_range_ids_.erase(logical_page); return {.status = RequestStatus::IoError}; } + + // The demand is served; only now are the following pages queued. Fetching + // them here would put the readahead on the demand's own critical path, + // which is the one thing a readahead must never do. + queue_readahead_locked(logical_page); if (operation == 1 && !cache_.mark_dirty(logical_page)) { return {.status = RequestStatus::IoError}; } @@ -186,6 +191,141 @@ CapacityResolveResult CapacityPageService::resolve( .media = media}; } +void CapacityPageService::set_readahead_pages(std::uint32_t pages) +{ + std::lock_guard lock(mutex_); + readahead_pages_ = pages; + if (pages == 0) { + readahead_queue_.clear(); + } +} + +std::uint64_t CapacityPageService::readahead_pages_fetched() const +{ + std::lock_guard lock(mutex_); + return readahead_fetched_; +} + +std::uint64_t CapacityPageService::readahead_pages_skipped() const +{ + std::lock_guard lock(mutex_); + return readahead_skipped_; +} + +void CapacityPageService::queue_readahead_locked(std::uint64_t demanded_page) +{ + if (readahead_pages_ == 0) { + return; + } + // A queue longer than this means the demand stream is outrunning the + // worker, and the oldest entries have gone stale anyway. + constexpr std::size_t kQueueLimit = 1024; + for (std::uint32_t step = 1; step <= readahead_pages_; ++step) { + if (readahead_queue_.size() >= kQueueLimit) { + ++readahead_skipped_; + return; + } + const auto page = demanded_page + step; + if (page < demanded_page) { + return; // the page number wrapped + } + if (cache_.resolve(page).has_value()) { + continue; // already resident, nothing to fetch + } + try { + readahead_queue_.push_back(page); + } catch (...) { + return; + } + } +} + +bool CapacityPageService::fill_free_frame_locked(std::uint64_t logical_page) +{ + if (cache_.resolve(logical_page).has_value() || + resident_range_ids_.contains(logical_page)) { + return false; // became resident between queueing and now + } + // A free frame if there is one. Once the cache is warm there never is, and + // a readahead that gave up here would do nothing at all in exactly the + // regime capacity mode exists for, so it takes a victim instead. Two + // limits keep that from costing more than it saves: the victim must be + // clean, because writing a dirty page back costs a program on the media + // and a program is far dearer than the read this would save; and the page + // it brings in is published unreferenced, so a page nothing ever asks for + // is the first candidate to be evicted again. + auto frame = cache_.free_frame(); + if (!frame.has_value()) { + auto eviction = cache_.begin_eviction(); + if (!eviction.has_value()) { + return false; + } + if (eviction->dirty) { + (void)cache_.cancel_eviction(*eviction); + return false; + } + if (!cache_.complete_eviction(*eviction)) { + return false; + } + resident_range_ids_.erase(eviction->logical_page); + frame = eviction->frame_address; + } + + std::vector page; + std::uint32_t range_id = 0; + try { + auto routed = backing_.read_page(logical_page, page_bytes_); + if (routed.status != RequestStatus::Ready || + routed.bytes.size() != page_bytes_ || routed.range_id == 0) { + return false; + } + range_id = routed.range_id; + page = std::move(routed.bytes); + } catch (...) { + return false; + } + try { + if (!frame_io_.host_to_frame(*frame, page)) { + return false; + } + } catch (...) { + return false; + } + try { + const auto [entry, inserted] = + resident_range_ids_.emplace(logical_page, range_id); + (void)entry; + if (!inserted) { + return false; + } + } catch (...) { + return false; + } + if (!cache_.publish(logical_page, *frame, /*referenced=*/false)) { + resident_range_ids_.erase(logical_page); + return false; + } + return true; +} + +bool CapacityPageService::run_one_readahead() +{ + std::lock_guard lock(mutex_); + if (readahead_queue_.empty()) { + return false; + } + const auto page = readahead_queue_.front(); + readahead_queue_.pop_front(); + if (fill_free_frame_locked(page)) { + ++readahead_fetched_; + } else { + ++readahead_skipped_; + } + // One queue entry was consumed either way, so the caller should ask again + // before going back to sleep. + return true; +} + RequestStatus CapacityPageService::flush() { std::lock_guard lock(mutex_); diff --git a/src/host_service/capacity_page_service.hpp b/src/host_service/capacity_page_service.hpp index 8d60d67..257b744 100644 --- a/src/host_service/capacity_page_service.hpp +++ b/src/host_service/capacity_page_service.hpp @@ -10,6 +10,7 @@ #include #include +#include #include #include #include @@ -59,17 +60,45 @@ class CapacityPageService { RequestStatus flush(std::uint64_t first_page, std::uint64_t page_count); + // Readahead. This is a system-side prefetcher: on a demand miss for page + // N it queues N+1 through N+pages, and the worker drains the queue when it + // has nothing else to do. The application neither asks for it nor knows it + // happened, and the only thing the policy has to go on is which pages have + // already been demanded. + // + // Zero, the default, switches it off, so a run that does not ask for + // readahead behaves exactly as before. + void set_readahead_pages(std::uint32_t pages); + + // Fetches at most one queued page. Returns true when it did work, so the + // caller can keep draining before going back to sleep. A queued page is + // skipped rather than fetched when it is already resident, when the + // backing store has nothing at that address, or when no frame is free -- + // readahead never evicts, so it can never push out a page a demand still + // needs. + bool run_one_readahead(); + + [[nodiscard]] std::uint64_t readahead_pages_fetched() const; + [[nodiscard]] std::uint64_t readahead_pages_skipped() const; + private: RequestStatus writeback(const runtime::CacheEviction& eviction); RequestStatus flush_range(std::uint64_t first_page, std::uint64_t page_count); + void queue_readahead_locked(std::uint64_t demanded_page); + bool fill_free_frame_locked(std::uint64_t logical_page); CapacityBackingIo backing_; runtime::HbmCache& cache_; std::size_t page_bytes_; CapacityFrameIo frame_io_; std::unordered_map resident_range_ids_; - std::mutex mutex_; + std::uint32_t readahead_pages_{0}; + // Bounded so a burst of misses cannot grow the queue without limit. + std::deque readahead_queue_; + std::uint64_t readahead_fetched_{0}; + std::uint64_t readahead_skipped_{0}; + mutable std::mutex mutex_; }; } // namespace hbfsim::host_service diff --git a/src/host_service/capacity_worker.cpp b/src/host_service/capacity_worker.cpp index d0274a7..85aa002 100644 --- a/src/host_service/capacity_worker.cpp +++ b/src/host_service/capacity_worker.cpp @@ -131,6 +131,20 @@ void CapacityWorker::run(std::stop_token stop) handoff.ticket, handoff.request_id, resolved.frame_address, resolved.status, resolved.media); } + if (!claimed && !stop.stop_requested()) { + // Readahead runs only when no demand was waiting, so a queued page + // can never delay a request the GPU is blocked on. One page per + // pass, so the slot scan resumes immediately if a demand arrives. + bool fetched = false; + try { + fetched = service_.run_one_readahead(); + } catch (...) { + fetched = false; + } + if (fetched) { + continue; + } + } if (!claimed && !stop.stop_requested()) { std::unique_lock lock(idle_mutex_); (void)idle_condition_.wait_for(lock, stop, idle_poll_, [] { diff --git a/tests/cpu/capacity_readahead_test.cpp b/tests/cpu/capacity_readahead_test.cpp new file mode 100644 index 0000000..c052576 --- /dev/null +++ b/tests/cpu/capacity_readahead_test.cpp @@ -0,0 +1,216 @@ +// Readahead in capacity mode: on a demand miss for page N, bring N+1 and the +// pages after it into the HBM page cache so a later demand for them hits. +// +// This is a system-side prefetcher, not an application-side one. The +// application does not ask for it and does not know it happened; the only +// thing the policy has to go on is which pages have already been demanded. +// +// Two safety properties matter more than the speedup and are asserted below. +// Readahead never evicts, so it can never push out a page a demand still +// needs, and it is off unless switched on, so no existing measurement moves. +// +// Written before src/host_service/capacity_page_service.cpp had any readahead. + +#include "../../src/cuda_runtime/hbm_cache.hpp" +#include "../../src/host_service/backing_store.hpp" +#include "../../src/host_service/capacity_page_service.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +namespace { + +[[noreturn]] void fail(const char* expression, int line) +{ + std::fprintf(stderr, "capacity readahead CHECK failed at line %d: %s\n", + line, expression); + std::exit(1); +} + +#define CHECK(expression) \ + do { \ + if (!(expression)) { \ + fail(#expression, __LINE__); \ + } \ + } while (false) + +constexpr std::size_t kPageBytes = 4096; + +} // namespace + +int main() +{ + const auto path = std::filesystem::temp_directory_path() / + ("hbfsim-capacity-readahead-" + + std::to_string(static_cast(::getpid()))); + std::filesystem::remove(path); + auto backing = hbfsim::host_service::BackingStore::create_deterministic( + path, 8 * kPageBytes, 0x2468); + + std::unordered_map> frames; + std::vector frame_addresses; + for (std::uint64_t index = 0; index < 4; ++index) { + const auto address = 0x1000 * (index + 1); + frames.emplace(address, std::vector(kPageBytes)); + frame_addresses.push_back(address); + } + + const hbfsim::host_service::CapacityFrameIo frame_io{ + .host_to_frame = [&](std::uint64_t frame, + std::span data) { + std::ranges::copy(data, frames.at(frame).begin()); + return true; + }, + .frame_to_host = [&](std::uint64_t frame, std::span data) { + std::ranges::copy(frames.at(frame), data.begin()); + return true; + }, + }; + + // 1. Readahead is off by default. A demand miss brings in exactly the page + // demanded and nothing else, so no existing measurement changes. + { + hbfsim::runtime::HbmCache cache(frame_addresses); + hbfsim::host_service::CapacityPageService service(backing, cache, + kPageBytes, + frame_io); + const auto demand = service.resolve(0, 0); + CHECK(demand.status == hbfsim::RequestStatus::Ready); + CHECK(!service.run_one_readahead()); + CHECK(cache.resolve(0).has_value()); + CHECK(!cache.resolve(1).has_value()); + CHECK(service.readahead_pages_fetched() == 0); + } + + // 2. With readahead on, a demand miss for page 0 makes pages 1 and 2 + // resident once the worker has drained the queue, and a later demand for + // page 1 is a hit that costs no media action. + { + hbfsim::runtime::HbmCache cache(frame_addresses); + hbfsim::host_service::CapacityPageService service(backing, cache, + kPageBytes, + frame_io); + service.set_readahead_pages(2); + + const auto demand = service.resolve(0, 0); + CHECK(demand.status == hbfsim::RequestStatus::Ready); + CHECK(demand.media.flags == hbfsim::host_service::CapacityMediaRead); + // Nothing has run the queue yet, so the demand itself did not drag the + // readahead onto its own critical path. + CHECK(!cache.resolve(1).has_value()); + + CHECK(service.run_one_readahead()); + CHECK(service.run_one_readahead()); + CHECK(!service.run_one_readahead()); + CHECK(cache.resolve(1).has_value()); + CHECK(cache.resolve(2).has_value()); + CHECK(service.readahead_pages_fetched() == 2); + + const auto hit = service.resolve(1, 0); + CHECK(hit.status == hbfsim::RequestStatus::Ready); + CHECK(hit.media.flags == hbfsim::host_service::CapacityMediaNone); + // The page the readahead brought in has to hold the right bytes, or + // the hit is worse than the miss it replaced. + CHECK(frames.at(hit.frame_address) == + backing.read_page(1, kPageBytes)); + } + + // 3. Readahead never forces a writeback. A readahead may take a clean + // victim once the cache is full -- refusing to would make it do nothing at + // all in the only regime capacity mode exists for -- but a dirty page + // costs a program on the media, which is dearer than the read the + // readahead would save, so a dirty victim is put back and the readahead + // gives up instead. + { + hbfsim::runtime::HbmCache cache(frame_addresses); + hbfsim::host_service::CapacityPageService service(backing, cache, + kPageBytes, + frame_io); + service.set_readahead_pages(1); + + // Fill every frame with a page written by the GPU, so each is dirty. + for (std::uint64_t page = 0; page < 4; ++page) { + CHECK(service.resolve(page, 1).status == + hbfsim::RequestStatus::Ready); + } + CHECK(cache.dirty_pages() == 4); + + const auto fetched_before = service.readahead_pages_fetched(); + while (service.run_one_readahead()) { + } + // Nothing was fetched, because every victim on offer was dirty. + CHECK(service.readahead_pages_fetched() == fetched_before); + CHECK(cache.dirty_pages() == 4); + for (std::uint64_t page = 0; page < 4; ++page) { + CHECK(cache.resolve(page).has_value()); + } + } + + // 3b. A readahead page enters unreferenced, so a page nothing ever asked + // for is evicted before a page a demand brought in. Without this the + // readahead would push out the very pages it is meant to help. + { + hbfsim::runtime::HbmCache cache(frame_addresses); + hbfsim::host_service::CapacityPageService service(backing, cache, + kPageBytes, + frame_io); + service.set_readahead_pages(3); + // One clean demand, then let the readahead fill the rest. + CHECK(service.resolve(0, 0).status == hbfsim::RequestStatus::Ready); + while (service.run_one_readahead()) { + } + CHECK(service.readahead_pages_fetched() > 0); + // The demanded page survives the readahead that followed it. + CHECK(cache.resolve(0).has_value()); + } + + // 4. A readahead that runs past the end of the backing store is skipped, + // not reported as an error. + { + hbfsim::runtime::HbmCache cache(frame_addresses); + hbfsim::host_service::CapacityPageService service(backing, cache, + kPageBytes, + frame_io); + service.set_readahead_pages(4); + CHECK(service.resolve(7, 0).status == hbfsim::RequestStatus::Ready); + while (service.run_one_readahead()) { + } + CHECK(service.readahead_pages_skipped() > 0); + } + + // 5. A queued page that a demand brought in first is skipped, not fetched + // a second time. The page becomes resident between being queued and being + // drained, which is the race the fill path has to survive. + { + hbfsim::runtime::HbmCache cache(frame_addresses); + hbfsim::host_service::CapacityPageService service(backing, cache, + kPageBytes, + frame_io); + service.set_readahead_pages(1); + // Queues page 1. + CHECK(service.resolve(0, 0).status == hbfsim::RequestStatus::Ready); + // A demand fills page 1 before the queue is drained, and queues page 2. + CHECK(service.resolve(1, 0).status == hbfsim::RequestStatus::Ready); + + const auto fetched_before = service.readahead_pages_fetched(); + const auto skipped_before = service.readahead_pages_skipped(); + while (service.run_one_readahead()) { + } + // Page 1 was skipped as already resident; only page 2 was fetched. + CHECK(service.readahead_pages_fetched() == fetched_before + 1); + CHECK(service.readahead_pages_skipped() == skipped_before + 1); + } + + std::filesystem::remove(path); + std::printf("capacity readahead: all checks passed\n"); + return 0; +} From 36deb372fad0d59e800834876b098031e4057348 Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Sat, 29 Aug 2026 12:36:12 -0700 Subject: [PATCH 03/10] Point the parent at the Overleaf commit, and list the readahead in the README The submodule pointer was already behind two commits made inside paper/ by an earlier session; this moves it forward to include those and the prefetch reference document added there. benchmarks/prefetch/README.md now lists the real readahead alongside the model, with the commands that reproduce the agreement at (P-1)/P and the drain-rate result. Co-Authored-By: Claude Opus 5 (1M context) --- benchmarks/prefetch/README.md | 35 ++++++++++++++++++++++++++++++++++- paper | 2 +- 2 files changed, 35 insertions(+), 2 deletions(-) diff --git a/benchmarks/prefetch/README.md b/benchmarks/prefetch/README.md index b6e8ed1..862f751 100644 --- a/benchmarks/prefetch/README.md +++ b/benchmarks/prefetch/README.md @@ -11,7 +11,10 @@ the commands. | `include/hbfsim/prefetch_model.hpp` | The model's interface, with each knob's meaning | | `src/prefetch/prefetch_model.cpp` | The model: a deterministic discrete-event simulation | | `tests/cpu/prefetch_model_test.cpp` | 12 property assertions, written before the implementation | -| `benchmarks/prefetch/hbf_prefetch_bench.cpp` | Sweeps one access stream and prints JSON | +| `benchmarks/prefetch/hbf_prefetch_bench.cpp` | Sweeps one access stream through the model and prints JSON | +| `src/host_service/capacity_page_service.cpp` | The real system-side readahead, in capacity mode | +| `tests/cpu/capacity_readahead_test.cpp` | Its contract: off by default, never forces a writeback, never drains on the demand's own path | +| `benchmarks/prefetch/hbf_capacity_readahead_bench.cpp` | Drives the real readahead and reports media reads avoided | | `scripts/run_prefetch_accuracy_sweep.py` | Runs all three streams and writes the artifact and a CSV | | `docs/proofs/artifacts/prefetch-accuracy-sweep.json` | The swept cells | | `docs/proofs/artifacts/prefetch-accuracy-sweep.csv` | The same cells, flat, for plotting | @@ -60,6 +63,36 @@ Other options: `--compute-ns` (accelerator time between two accesses, the interval a prefetch hides behind), `--lead` (how many accesses ahead a prefetch is issued), `--buffer-pages`, `--max-in-flight`, `--seed`. +## The real readahead, as opposed to the model + +`hbf_capacity_readahead_bench` exercises `CapacityPageService` directly. It +reports media reads avoided, not wall-clock time, and touches no GPU. + +Does the implementation reproduce what the model predicts for a next-page +policy, which is (P-1)/P for P pages per expert: + +``` +for p in 4 8 16; do + ./build/hbf_capacity_readahead_bench --stream moe --pages-per-expert $p \ + --readahead $p --frames 256 --drain-per-demand 0 +done +``` + +What the worker's drain rate is worth. `--drain-per-demand` is how many queued +pages the worker gets through between two demands, and 0 means it keeps up +completely: + +``` +for d in 0 4 2 1; do + ./build/hbf_capacity_readahead_bench --stream moe --pages-per-expert 8 \ + --readahead 8 --frames 256 --drain-per-demand $d +done +``` + +Draining one page per demand does not merely reduce the benefit. It turns an +86.79% reduction in media reads into a 14.17% increase, because queued pages go +stale before they are used while still costing bandwidth and frames. + ## Reproducing the two results the paper leans on The naive next-page policy's accuracy on a Mixture-of-Experts stream is diff --git a/paper b/paper index e3fd9d0..398846c 160000 --- a/paper +++ b/paper @@ -1 +1 @@ -Subproject commit e3fd9d06afb15567200de46b417dffae4e881304 +Subproject commit 398846ce9cedd5cada9189a8e3defa0924e16c4f From 84ff0c0acb23b66ae382e64ffd06bede85c59c0a Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Sun, 30 Aug 2026 03:29:00 -0700 Subject: [PATCH 04/10] Correct the readahead accounting, and fix three defects the review found All four review points hold. Each was reproduced before being fixed. ACCOUNTING. The benchmark reported only demand-path misses as `media_reads` and computed an "avoided" fraction from them, so a readahead that removed waiting looked like a reduction in media work. Every successful readahead is also a read of the backing store. Reproduced at P=8, depth 8, 256 frames against an 840-read baseline: drain 0 111 demand + 868 readahead = 979 total +16.55% drain 1 959 demand + 825 readahead = 1784 total +112.38% This reverses the headline. The readahead removes 89.16% of the reads a warp waits on and still raises total media traffic by 16.55%. It trades bandwidth for latency, which is the wrong trade whenever the tier is bandwidth-bound -- and the array supply rate is exactly what this project argues is binding. The benchmark now prints demand_media_reads, readahead_media_reads, total_media_reads, demand_reads_avoided_fraction and total_media_reads_change_fraction, and the README carries both numbers with that reading. FORMAT. `drain_per_demand` was passed to printf with no placeholder, so the build warned -Wformat-extra-args and the artifact omitted the parameter that decides the result. Now compiles clean under -Wall -Wformat-extra-args and the field appears in the JSON. ROLLBACK SAFETY. fill_free_frame_locked completed a clean eviction before the speculative read, so a read that failed -- an address past the end of the backing store is the ordinary case -- destroyed a valid resident page and returned false. The read now happens first and a frame is taken only once the bytes are in hand. tests/cpu/capacity_readahead_test.cpp case 4b covers it: with four clean demanded pages filling every frame and a readahead running off the end of the store, the old ordering leaves three of the four resident and the new ordering leaves all four. The earlier case 4 could not catch this because free frames were still available there. PRODUCTION PATH. Nothing outside tests and the benchmark could switch the readahead on. `readahead_pages` is now a profile field, optional and defaulting to 0 so no existing profile changes behaviour, parsed in profile.cpp, declared in the schema, and applied by CapacityRuntime's constructor. Setting it after the worker starts is safe because the queue only fills on a demand miss and no demand can arrive before a range is registered. Test suite: the same four tests fail before and after (context_lifecycle, vmem_tuning, run_with_bpftime, mqsim_benchmark), all pre-existing in a CPU-only build. Co-Authored-By: Claude Opus 5 (1M context) --- benchmarks/prefetch/README.md | 23 +++++++-- .../prefetch/hbf_capacity_readahead_bench.cpp | 45 ++++++++++++----- configs/schema/hbf-profile.schema.json | 1 + include/hbfsim/profile.hpp | 4 ++ src/cuda_runtime/capacity_runtime.cpp | 10 ++++ src/host_service/capacity_page_service.cpp | 50 +++++++++++-------- src/profile/profile.cpp | 3 ++ tests/cpu/capacity_readahead_test.cpp | 35 +++++++++++++ 8 files changed, 134 insertions(+), 37 deletions(-) diff --git a/benchmarks/prefetch/README.md b/benchmarks/prefetch/README.md index 862f751..b617975 100644 --- a/benchmarks/prefetch/README.md +++ b/benchmarks/prefetch/README.md @@ -89,9 +89,26 @@ for d in 0 4 2 1; do done ``` -Draining one page per demand does not merely reduce the benefit. It turns an -86.79% reduction in media reads into a 14.17% increase, because queued pages go -stale before they are used while still costing bandwidth and frames. +Read both numbers the benchmark prints, because they answer different +questions and reporting only the first is how an earlier version of this file +reached a wrong conclusion. + +`demand_reads_avoided_fraction` is the share of demands that no longer wait on +the media, so it sets the latency. `total_media_reads_change_fraction` counts +every read of the backing store, readahead included, so it sets the bandwidth +the device must supply. + +At P=8, depth 8, 256 frames, against an 840-read baseline: + +| drain per demand | demand reads | readahead reads | total | demand avoided | total change | +|---|---:|---:|---:|---:|---:| +| 0 (keeps up) | 111 | 868 | 979 | +89.16% | **+16.55%** | +| 1 | 959 | 825 | 1784 | +6.35% | **+112.38%** | + +So the readahead removes most of the waiting and still raises total media +traffic. That is a trade of bandwidth for latency, and it is the wrong trade +whenever the tier is bandwidth-bound. Draining one page per demand loses the +latency benefit as well and more than doubles the traffic. ## Reproducing the two results the paper leans on diff --git a/benchmarks/prefetch/hbf_capacity_readahead_bench.cpp b/benchmarks/prefetch/hbf_capacity_readahead_bench.cpp index 6416b32..00728d4 100644 --- a/benchmarks/prefetch/hbf_capacity_readahead_bench.cpp +++ b/benchmarks/prefetch/hbf_capacity_readahead_bench.cpp @@ -141,26 +141,45 @@ Run drive(const std::vector& pages, const Options& options, return run; } -void print_run(const char* label, const Run& run, std::uint64_t baseline_reads) +// Two different quantities, and reporting only the first is how the earlier +// version of this benchmark reached a wrong conclusion. `demand_media_reads` +// are the reads a warp waits on, so they set the latency. Every successful +// readahead is ALSO a read of the backing store, so the bandwidth the device +// must supply is the sum. A readahead that removes demand reads while raising +// the total is trading bandwidth for latency, which is the wrong trade when +// the tier is bandwidth-bound. +void print_run(const char* label, const Run& run, + std::uint64_t baseline_total_reads) { const double hit_rate = run.demands == 0 ? 0.0 : static_cast(run.hits) / static_cast(run.demands); - const double avoided = - baseline_reads == 0 + const auto total_reads = run.media_reads + run.readahead_fetched; + const double demand_avoided = + run.demands == 0 ? 0.0 : 1.0 - static_cast(run.media_reads) / - static_cast(baseline_reads); + static_cast(run.demands); + const double total_change = + baseline_total_reads == 0 + ? 0.0 + : static_cast(total_reads) / + static_cast(baseline_total_reads) - + 1.0; std::printf( - " {\"policy\": \"%s\", \"demands\": %llu, \"media_reads\": %llu, " - "\"hits\": %llu, \"hit_rate\": %.6f, " - "\"media_reads_avoided_fraction\": %.6f, " - "\"readahead_fetched\": %llu, \"readahead_skipped\": %llu}", + " {\"policy\": \"%s\", \"demands\": %llu, " + "\"demand_media_reads\": %llu, \"readahead_media_reads\": %llu, " + "\"total_media_reads\": %llu, \"hits\": %llu, \"hit_rate\": %.6f, " + "\"demand_reads_avoided_fraction\": %.6f, " + "\"total_media_reads_change_fraction\": %.6f, " + "\"readahead_skipped\": %llu}", label, static_cast(run.demands), static_cast(run.media_reads), - static_cast(run.hits), hit_rate, avoided, static_cast(run.readahead_fetched), + static_cast(total_reads), + static_cast(run.hits), hit_rate, demand_avoided, + total_change, static_cast(run.readahead_skipped)); } @@ -257,7 +276,8 @@ int main(int argc, char** argv) "\"distinct_pages_in_store\": %llu, \"frames\": %llu, " "\"readahead_pages\": %u, \"pages_per_expert\": %llu, " "\"tokens\": %llu, \"layers\": %llu, \"experts_per_layer\": %llu, " - "\"experts_per_token\": %llu, \"seed\": %llu},\n", + "\"experts_per_token\": %llu, \"seed\": %llu, " + "\"drain_per_demand\": %llu},\n", options.stream.c_str(), static_cast(pages.size()), static_cast(store_pages), @@ -270,9 +290,10 @@ int main(int argc, char** argv) static_cast(options.seed), static_cast(options.drain_per_demand)); std::printf(" \"runs\": [\n"); - print_run("demand_only", without, without.media_reads); + const auto baseline_total = without.media_reads + without.readahead_fetched; + print_run("demand_only", without, baseline_total); std::printf(",\n"); - print_run("readahead", with, without.media_reads); + print_run("readahead", with, baseline_total); std::printf("\n ]\n}\n"); std::filesystem::remove(path); diff --git a/configs/schema/hbf-profile.schema.json b/configs/schema/hbf-profile.schema.json index fa38d19..9a98269 100644 --- a/configs/schema/hbf-profile.schema.json +++ b/configs/schema/hbf-profile.schema.json @@ -39,6 +39,7 @@ "queue_depth": { "type": "integer", "minimum": 1 }, "aggregate_bandwidth_bytes_per_s": { "type": "integer", "minimum": 1 }, "hbm_cache_bytes": { "type": "integer", "minimum": 0 }, + "readahead_pages": { "type": "integer", "minimum": 0 }, "reference_sample_rate": { "type": "number", "minimum": 0, "maximum": 1 }, "reference_warmup_requests": { "type": "integer", "minimum": 0 }, "time_scale": { "type": "integer", "minimum": 1 }, diff --git a/include/hbfsim/profile.hpp b/include/hbfsim/profile.hpp index 18c3442..cefb823 100644 --- a/include/hbfsim/profile.hpp +++ b/include/hbfsim/profile.hpp @@ -92,6 +92,10 @@ namespace hbfsim std::uint32_t queue_depth; std::uint64_t aggregate_bandwidth_bytes_per_s; std::uint64_t hbm_cache_bytes; + // Pages the capacity-mode readahead queues after a demand miss. + // 0 switches it off, which is the default a profile gets when the + // field is absent, so no existing profile changes behaviour. + std::uint32_t readahead_pages; double reference_sample_rate; std::uint32_t reference_warmup_requests; std::uint32_t time_scale; diff --git a/src/cuda_runtime/capacity_runtime.cpp b/src/cuda_runtime/capacity_runtime.cpp index 446b345..1b4cf20 100644 --- a/src/cuda_runtime/capacity_runtime.cpp +++ b/src/cuda_runtime/capacity_runtime.cpp @@ -138,6 +138,16 @@ CapacityRuntime::CapacityRuntime(const Profile& profile, &CapacityRuntime::start_worker, this, &CapacityRuntime::stop_worker, this) { + // The production path for the capacity-mode readahead. Without this the + // feature was reachable only from tests and the benchmark, so no run that + // a user could start would ever enable it. A profile that does not name + // `readahead_pages` gets 0, which leaves the readahead off. + // + // Setting it after the worker has started is safe: the queue only fills on + // a demand miss, and a demand miss cannot happen before a range has been + // registered, which happens after this constructor returns. Until then the + // worker's idle branch finds an empty queue. + service_.set_readahead_pages(profile.readahead_pages); #if defined(HBFSIM_ENABLE_CUDA_RUNTIME) CUstream stream = nullptr; if (::cuStreamCreate(&stream, CU_STREAM_NON_BLOCKING) != CUDA_SUCCESS || diff --git a/src/host_service/capacity_page_service.cpp b/src/host_service/capacity_page_service.cpp index 40ec7e4..eac3957 100644 --- a/src/host_service/capacity_page_service.cpp +++ b/src/host_service/capacity_page_service.cpp @@ -246,14 +246,34 @@ bool CapacityPageService::fill_free_frame_locked(std::uint64_t logical_page) resident_range_ids_.contains(logical_page)) { return false; // became resident between queueing and now } - // A free frame if there is one. Once the cache is warm there never is, and - // a readahead that gave up here would do nothing at all in exactly the - // regime capacity mode exists for, so it takes a victim instead. Two - // limits keep that from costing more than it saves: the victim must be - // clean, because writing a dirty page back costs a program on the media - // and a program is far dearer than the read this would save; and the page - // it brings in is published unreferenced, so a page nothing ever asks for - // is the first candidate to be evicted again. + // The speculative read comes FIRST, before any resident page is disturbed. + // An earlier version evicted a victim and then read, so a read that failed + // -- an address past the end of the backing store is the ordinary case -- + // destroyed a valid resident page and returned false. A readahead is a + // guess, and a guess must never cost a page that a demand actually brought + // in. + std::vector page; + std::uint32_t range_id = 0; + try { + auto routed = backing_.read_page(logical_page, page_bytes_); + if (routed.status != RequestStatus::Ready || + routed.bytes.size() != page_bytes_ || routed.range_id == 0) { + return false; + } + range_id = routed.range_id; + page = std::move(routed.bytes); + } catch (...) { + return false; + } + + // Only now, with the bytes in hand, is a frame taken. Once the cache is + // warm there is never a free one, and a readahead that gave up here would + // do nothing at all in the only regime capacity mode exists for, so it + // takes a victim instead. Two limits keep that from costing more than it + // saves: the victim must be clean, because writing a dirty page back costs + // a program on the media and a program is far dearer than the read this + // would save; and the page it brings in is published unreferenced, so a + // page nothing ever asks for is the first candidate to be evicted again. auto frame = cache_.free_frame(); if (!frame.has_value()) { auto eviction = cache_.begin_eviction(); @@ -270,20 +290,6 @@ bool CapacityPageService::fill_free_frame_locked(std::uint64_t logical_page) resident_range_ids_.erase(eviction->logical_page); frame = eviction->frame_address; } - - std::vector page; - std::uint32_t range_id = 0; - try { - auto routed = backing_.read_page(logical_page, page_bytes_); - if (routed.status != RequestStatus::Ready || - routed.bytes.size() != page_bytes_ || routed.range_id == 0) { - return false; - } - range_id = routed.range_id; - page = std::move(routed.bytes); - } catch (...) { - return false; - } try { if (!frame_io_.host_to_frame(*frame, page)) { return false; diff --git a/src/profile/profile.cpp b/src/profile/profile.cpp index 45e9ba6..dd13a10 100644 --- a/src/profile/profile.cpp +++ b/src/profile/profile.cpp @@ -262,6 +262,9 @@ namespace hbfsim .get(), .hbm_cache_bytes = document.at("hbm_cache_bytes").get(), + // Optional: absent means the readahead stays off. + .readahead_pages = document.value("readahead_pages", + std::uint32_t{0}), .reference_sample_rate = document.at("reference_sample_rate").get(), .reference_warmup_requests = diff --git a/tests/cpu/capacity_readahead_test.cpp b/tests/cpu/capacity_readahead_test.cpp index c052576..e889706 100644 --- a/tests/cpu/capacity_readahead_test.cpp +++ b/tests/cpu/capacity_readahead_test.cpp @@ -187,6 +187,41 @@ int main() CHECK(service.readahead_pages_skipped() > 0); } + // 4b. A readahead whose speculative read fails must not have cost a + // resident page. With every frame taken by a clean demanded page, a + // readahead for an address past the end of the backing store has to leave + // all of them in place. The earlier version evicted a victim first and + // read second, so this case silently destroyed a page a demand had brought + // in; case 4 above does not catch it because free frames were still + // available there. + { + hbfsim::runtime::HbmCache cache(frame_addresses); + hbfsim::host_service::CapacityPageService service(backing, cache, + kPageBytes, + frame_io); + service.set_readahead_pages(2); + // Four clean demanded pages fill all four frames. Page 7 is the last + // page of the store, so its readahead runs off the end. + for (const std::uint64_t page : {4ULL, 5ULL, 6ULL, 7ULL}) { + CHECK(service.resolve(page, 0).status == + hbfsim::RequestStatus::Ready); + } + for (const std::uint64_t page : {4ULL, 5ULL, 6ULL, 7ULL}) { + CHECK(cache.resolve(page).has_value()); + } + const auto fetched_before = service.readahead_pages_fetched(); + + while (service.run_one_readahead()) { + } + + // Nothing was fetched, because every queued page is past the end. + CHECK(service.readahead_pages_fetched() == fetched_before); + // And every demanded page survived the failed speculation. + for (const std::uint64_t page : {4ULL, 5ULL, 6ULL, 7ULL}) { + CHECK(cache.resolve(page).has_value()); + } + } + // 5. A queued page that a demand brought in first is skipped, not fetched // a second time. The page becomes resident between being queued and being // drained, which is the race the fill path has to survive. From 27a1a0885b910a7d5bf98bc2f606177b3f05b741 Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Sun, 30 Aug 2026 03:30:02 -0700 Subject: [PATCH 05/10] Point the parent at the corrected prefetch reference Co-Authored-By: Claude Opus 5 (1M context) --- paper | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/paper b/paper index 398846c..e07e98e 160000 --- a/paper +++ b/paper @@ -1 +1 @@ -Subproject commit 398846ce9cedd5cada9189a8e3defa0924e16c4f +Subproject commit e07e98edd5ebdb07c73eac65b33b9d519fcecdd6 From de8e22100234e94787a1a57324553d646a3cfad1 Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Sun, 30 Aug 2026 03:32:37 -0700 Subject: [PATCH 06/10] Point the parent at the Background addition Co-Authored-By: Claude Opus 5 (1M context) --- paper | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/paper b/paper index e07e98e..aa9263f 160000 --- a/paper +++ b/paper @@ -1 +1 @@ -Subproject commit e07e98edd5ebdb07c73eac65b33b9d519fcecdd6 +Subproject commit aa9263f7ff856b3553a232ce8979868abbad1aca From 65a5609034558ed9605bc278632ead593613e05e Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Mon, 31 Aug 2026 15:04:21 -0700 Subject: [PATCH 07/10] Drop the last two 'media reads avoided' labels from the README Two lines still described the benchmark as reporting media reads avoided, which is the framing the accounting correction removed: the readahead avoids demand-path reads and raises the total. Both now say what it actually reports. Co-Authored-By: Claude Opus 5 (1M context) --- benchmarks/prefetch/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/benchmarks/prefetch/README.md b/benchmarks/prefetch/README.md index b617975..aba1a5b 100644 --- a/benchmarks/prefetch/README.md +++ b/benchmarks/prefetch/README.md @@ -14,7 +14,7 @@ the commands. | `benchmarks/prefetch/hbf_prefetch_bench.cpp` | Sweeps one access stream through the model and prints JSON | | `src/host_service/capacity_page_service.cpp` | The real system-side readahead, in capacity mode | | `tests/cpu/capacity_readahead_test.cpp` | Its contract: off by default, never forces a writeback, never drains on the demand's own path | -| `benchmarks/prefetch/hbf_capacity_readahead_bench.cpp` | Drives the real readahead and reports media reads avoided | +| `benchmarks/prefetch/hbf_capacity_readahead_bench.cpp` | Drives the real readahead and reports demand-path and total media reads separately | | `scripts/run_prefetch_accuracy_sweep.py` | Runs all three streams and writes the artifact and a CSV | | `docs/proofs/artifacts/prefetch-accuracy-sweep.json` | The swept cells | | `docs/proofs/artifacts/prefetch-accuracy-sweep.csv` | The same cells, flat, for plotting | @@ -66,7 +66,7 @@ is issued), `--buffer-pages`, `--max-in-flight`, `--seed`. ## The real readahead, as opposed to the model `hbf_capacity_readahead_bench` exercises `CapacityPageService` directly. It -reports media reads avoided, not wall-clock time, and touches no GPU. +counts media reads rather than wall-clock time, and touches no GPU. Does the implementation reproduce what the model predicts for a next-page policy, which is (P-1)/P for P pages per expert: From 4c87c49818820c2e37fc7e362454234ad9f20906 Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Mon, 31 Aug 2026 15:25:28 -0700 Subject: [PATCH 08/10] Point the parent at the withdrawn claim Co-Authored-By: Claude Opus 5 (1M context) --- paper | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/paper b/paper index aa9263f..1d7c0ae 160000 --- a/paper +++ b/paper @@ -1 +1 @@ -Subproject commit aa9263f7ff856b3553a232ce8979868abbad1aca +Subproject commit 1d7c0ae2ccd275b606a8829838cfa7b489675ab9 From 54386c8087d3a7819b8719ae4d86aa41c6055ddb Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Mon, 31 Aug 2026 15:51:32 -0700 Subject: [PATCH 09/10] Put speculative reads on the timing timeline (untested; please run it) A readahead performs a real backing read that no GPU waits on, so it never reached the timing engine: no modeled latency, no queueing, no contention, and the demand that later hit the prefetched page reported no media time. In simulation the speculative read had been deleted from the timeline. RequestDispatcher::submit_speculative queues a read; poll_once drains one only when it popped no demand and nothing is in flight, so a readahead can never delay a request a warp is blocked on. The read goes to the same engine as a demand, so it occupies the same channels and contends with demand traffic. Its completion is discarded in publish, because no shared-memory slot is behind it. Of the three shapes considered, this is the only one that is semantically right. A callback mirroring flush(model_program) would follow a hook that has no production consumer today, and submitting outside the dispatcher breaks the ticket-to-engine-id pairing. Carrying the reads on the next demand's spare media-action slot would put them back on that demand's critical path, which is the thing the readahead exists to avoid. I FOUND ONE BUG BY RE-READING THIS AND FIXED IT: publish originally erased the speculative group itself, but both callers erase after publish returns, one of them through the iterator used to reach the call. Erasing inside invalidated that iterator and the caller then erased through it. publish now leaves the erase to the callers. TWO THINGS I COULD NOT VERIFY, MARKED IN THE CODE AS "UNVERIFIED, PLEASE CHECK": 1. Speculative tickets set bit 63 to stay clear of real tickets, which are the request's own `sequence`. I could not establish from the producer side that `sequence` never reaches 2^63. If it can, the namespace must be separated another way, for example a flag on DispatchGroup. 2. next_speculative_ticket_ wraps after 2^63 speculative reads. Not reachable in any run here, but nothing enforces it. THERE IS NO TEST FOR THIS PATH, and there is no GPU on the machine it was written on, so it has never run against a live engine. The suite is unchanged at 88% with the same four pre-existing failures, which only says it did not break what was already there. Please run it before trusting it, and I will not regenerate any performance number until it has been. Co-Authored-By: Claude Opus 5 (1M context) --- paper | 2 +- src/host_service/request_dispatcher.cpp | 77 +++++++++++++++++++++++++ src/host_service/request_dispatcher.hpp | 46 +++++++++++++++ 3 files changed, 124 insertions(+), 1 deletion(-) diff --git a/paper b/paper index 1d7c0ae..3749d3c 160000 --- a/paper +++ b/paper @@ -1 +1 @@ -Subproject commit 1d7c0ae2ccd275b606a8829838cfa7b489675ab9 +Subproject commit 3749d3ced21d4480344cff0a623cdd0c53044269 diff --git a/src/host_service/request_dispatcher.cpp b/src/host_service/request_dispatcher.cpp index 6dcc426..84609e9 100644 --- a/src/host_service/request_dispatcher.cpp +++ b/src/host_service/request_dispatcher.cpp @@ -313,9 +313,75 @@ bool RequestDispatcher::submit_next(std::uint64_t ticket, return true; } +bool RequestDispatcher::submit_speculative(const HbfRequest& request) +{ + // 64 is arbitrary but bounded. The readahead is advisory, so losing the + // oldest speculative read is better than letting this grow. + constexpr std::size_t kSpeculativeQueueLimit = 64; + if (speculative_.size() >= kSpeculativeQueueLimit) { + ++speculative_dropped_; + return false; + } + try { + speculative_.push_back(request); + } catch (...) { + ++speculative_dropped_; + return false; + } + return true; +} + +bool RequestDispatcher::drain_one_speculative() +{ + if (speculative_.empty()) { + return false; + } + auto request = speculative_.front(); + speculative_.pop_front(); + + const auto ticket = kSpeculativeTicketBit | next_speculative_ticket_++; + // UNVERIFIED, PLEASE CHECK: next_speculative_ticket_ wraps after 2^63 + // speculative reads. That is not reachable in any run this project makes, + // but nothing here enforces it. + request.sequence = ticket; + PreparedDispatch prepared{}; + prepared.media_actions[0] = request; + prepared.media_action_count = 1; + + const auto [group_it, inserted] = groups_by_ticket_.emplace( + ticket, DispatchGroup{.original = request, .prepared = prepared}); + if (!inserted) { + // A ticket collision here would mean the reserved space is not + // reserved. Drop the speculative read rather than disturb a real + // group: a missing speculative read costs accuracy, touching someone + // else's group costs correctness. + ++speculative_dropped_; + return false; + } + if (!submit_next(ticket, group_it->second)) { + groups_by_ticket_.erase(group_it); + ++speculative_dropped_; + return false; + } + ++speculative_submitted_; + return true; +} + bool RequestDispatcher::publish(std::uint64_t ticket, DispatchGroup& group) { + if ((ticket & kSpeculativeTicketBit) != 0) { + // Nothing is waiting on a speculative read, so there is no slot to + // write and no completion to hand back. The modeled time it spent in + // the engine has already done its work: it occupied the engine while + // demand traffic was queued behind it. + // + // Deliberately does NOT erase the group here. Both callers erase after + // publish returns, one of them through the very iterator that was used + // to reach this call, so erasing here invalidated that iterator and + // the caller then erased through it. + return true; + } auto& completion = group.prepared.completion; if (group.legacy && engine_.finalize) { try { @@ -402,6 +468,17 @@ bool RequestDispatcher::poll_once() } } + // A speculative read goes in only when this poll found no demand to pop + // and nothing is already in flight, so it can never delay a request a warp + // is blocked on. The completion loop below then drains it, and publish() + // discards its completion because no slot is behind it. + // + // One per poll on purpose: the demand ring is re-read on the next poll + // before another speculative read is considered. + if (!progressed && groups_by_ticket_.empty()) { + (void)drain_one_speculative(); + } + while (!groups_by_ticket_.empty()) { std::optional completion; try { diff --git a/src/host_service/request_dispatcher.hpp b/src/host_service/request_dispatcher.hpp index d166833..8161a11 100644 --- a/src/host_service/request_dispatcher.hpp +++ b/src/host_service/request_dispatcher.hpp @@ -3,6 +3,7 @@ #include "control_layout.hpp" #include +#include #include #include #include @@ -50,6 +51,33 @@ class RequestDispatcher { // Returns true when a request was consumed or a completion was published. bool poll_once(); + // Puts a speculative media read on the timing timeline. + // + // A readahead performs a real backing read that no GPU is waiting on, so + // before this it never reached the timing engine at all: it contributed no + // modeled latency, no queueing and no contention, and the demand that + // later hit the prefetched page reported no media time. In simulation the + // speculative read had been deleted from the timeline. + // + // A speculative request is submitted to the same engine as a demand, so it + // occupies the same channels and contends with demand traffic, but its + // completion is discarded rather than published, because there is no + // shared-memory slot behind it. + // + // Returns false when the queue is full. A dropped speculative read is not + // an error: the readahead is advisory, and dropping it only means this + // read is missing from the timeline. + bool submit_speculative(const HbfRequest& request); + + [[nodiscard]] std::uint64_t speculative_submitted() const noexcept + { + return speculative_submitted_; + } + [[nodiscard]] std::uint64_t speculative_dropped() const noexcept + { + return speculative_dropped_; + } + private: friend class RequestDispatcherTestAccess; @@ -60,6 +88,18 @@ class RequestDispatcher { bool legacy{false}; }; + // Tickets for real requests are the request's own `sequence` field, taken + // from the shared ring. Speculative groups need a ticket space that cannot + // collide with those, so they set the top bit. + // + // UNVERIFIED, PLEASE CHECK: I could not establish from the producer side + // that `sequence` never reaches 2^63. If it can, this reservation is + // wrong and the two spaces must be separated another way, for example by + // a flag on DispatchGroup instead of a bit in the ticket. Everything else + // here holds either way; only the choice of namespace depends on it. + static constexpr std::uint64_t kSpeculativeTicketBit = 1ULL << 63; + + [[nodiscard]] bool drain_one_speculative(); [[nodiscard]] std::optional next_engine_id() noexcept; bool submit_next(std::uint64_t ticket, DispatchGroup& group); bool publish(std::uint64_t ticket, DispatchGroup& group); @@ -71,6 +111,12 @@ class RequestDispatcher { std::unordered_map ticket_by_engine_id_; std::uint64_t next_engine_id_{1}; bool engine_ids_exhausted_{false}; + // Bounded on purpose: a readahead that outruns the engine should lose its + // oldest speculative reads rather than grow this without limit. + std::deque speculative_; + std::uint64_t next_speculative_ticket_{1}; + std::uint64_t speculative_submitted_{0}; + std::uint64_t speculative_dropped_{0}; }; } // namespace hbfsim::host_service From 1f19bdbba06c89cbaf9b7e6874f3473dc8123b2b Mon Sep 17 00:00:00 2001 From: Yanpeng hu Date: Mon, 31 Aug 2026 16:13:19 -0700 Subject: [PATCH 10/10] Point the parent at the compressed Related Work and the expanded Background Co-Authored-By: Claude Opus 5 (1M context) --- paper | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/paper b/paper index 3749d3c..449d48e 160000 --- a/paper +++ b/paper @@ -1 +1 @@ -Subproject commit 3749d3ced21d4480344cff0a623cdd0c53044269 +Subproject commit 449d48e3427dc19b4964597d3284ea7443f87162