feat(dashboard): per-day CPR, five years of chart history, and a Daily timeframe - #174
Merged
Merged
Conversation
A five-year pull is ~21 requests over ~10 minutes. Every chunk was held in memory and written once at the end, so a failure in the last request threw away every earlier one -- a bad trade for a download you repeat each time you refresh the data. Chunks are now appended as they arrive and a manifest beside the CSV records how far the run got, so re-running the same command picks up from the last completed chunk. The pattern is ported from the expired-options engine rather than invented: same manifest shape, same run signature, same append-and-checkpoint order. Two of its hazards are load-bearing and carried over deliberately. Progress is trusted only when the run signature matches -- start date, interval, chunk size, security id, segment. Progress from a DIFFERENT run is the dangerous case: a narrow earlier run followed by the five-year backfill would skip every chunk ending before the stored point, and the command would report success over a file missing most of its history. The manifest is written AFTER the rows, so it can lag the file but can never claim rows that are not there. `truncate_to` repairs the lag by rolling the CSV back to the last vouched-for byte length; nothing could repair the other direction. A CSV with no manifest is restarted rather than appended to, since there is no last timestamp to de-duplicate against. Appending relies on `validate_ohlc_frame` REJECTING unordered timestamps rather than sorting them, which is what lets the file stay ascending without re-sorting five years of rows on every chunk. `--no-resume` keeps the original all-at-once behaviour as an escape hatch, so `atomic_write_csv` and its tests stay live rather than becoming dead code. One bug found by its own test and fixed here: an all-empty range (a run of holidays) records a resume point but never creates a CSV, and the first version of the "file is gone" guard read that as damage and re-requested those windows forever. It now only distrusts a manifest that claims rows. Also corrects two stale lines in the Readme: --from-date/--to-date do not exist (they are --start-date/--end-date), and DHAN_ACCESS_TOKEN is the key the code prefers over the older DHAN_TOKEN_ID. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Probing Dhan's index history for the five-year pull turned up two kinds of
row that are not session data, and they differ by era:
- windows before roughly mid-2022 come back with ~650 bars a day running
out to 17:59, against the 375 a real session has;
- the CURRENT day carries a synthetic "now" bar stamped at the wall clock
with flat OHLC and zero volume (measured: 2026-09-16 18:44:00).
Both are minute-ALIGNED, which makes them more dangerous than the malformed
kind rather than less: `validate_ohlc_frame` accepts them, so they would land
in the CSV and drag a day's high, low and close -- and therefore that day's
CPR -- along with them.
Clipping happens BEFORE validation, because for an older window those rows
are the bulk of the response: validating first would either pass them through
or fail the whole chunk on rows nobody wants. An all-outside chunk now
returns empty, which the caller already treats as a holiday rather than an
error.
The window is `market_data_health`'s own MARKET_SESSION_START/END rather than
a second pair of literals, so the extractor and the runner cannot drift on
what a session is.
Measured effect: a 2022-05 window goes from ~652 bars/day to exactly 375,
with no duplicates, monotonic timestamps and no candle-geometry violations.
Not fixed here, and worth knowing: 2021-09-17 carries ~23 in-session bars
stamped one second off the minute, which `validate_ohlc_frame` rightly
refuses. Every window probed from 2021-09-27 onward is clean, so the backfill
starts there rather than weakening the alignment guard for one bad day.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e backfill The five-year pull died on chunk 2 with "impossible candle geometry". The cause was ONE row in 23,380 across 2021-12-26..2022-03-25: 2022-03-25 09:15, open 17289.00 against a high of 17287.10 -- an opening print whose high sits below its own open. That row is provably wrong; no reading of a candle makes its high lower than the open it contains. Failing five years of history on it is the wrong trade, so isolated ones are now dropped and NAMED in the output, never silently. The tolerance is bounded on purpose. More than 0.1% of a chunk being self-contradicting does not mean noisy prints, it means the response is not the series we asked for, and that keeps failing loudly. The real rate that motivated this was 0.004%. Scope matters here: this is the historical backfill path only. `validate_ohlc_frame` is deliberately NOT relaxed -- it still refuses every one of these rows, on the live feed and everywhere else. They are removed before it runs rather than by weakening the guard that protects trading. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The backfill died on chunk 3 with "Dhan chunk contains invalid volume": some 2022 index bars come back with a null volume. An ABSENT volume column was already treated as zero a few lines above, so an absent cell is the same statement about the same thing and now gets the same answer. Index instruments carry no meaningful volume in any case -- Dhan returns zeros for the whole of 2021, and every backtest loader in this repo forces Volume to 0 regardless. A negative or infinite volume is a different claim, and is still refused. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two more walls in the five-year backfill, both in the pre-mid-2022 era. WEEKENDS. Those windows carry rows dated Saturday and Sunday, sitting inside session HOURS so the time-of-day clip let them through. Their prices are visibly not the index: on Saturday 2022-04-09 the close goes 18396 -> 18584 -> 18371 -> 18842 inside one hour. The clip now also requires a weekday, and the effect is decisive -- across 2022-03-26..2022-06-23 every one of the 61 remaining sessions holds exactly 375 bars, none more, none fewer. VOLUME. 221 bars in that same window carry small negative counters (-1, -2, -3). An index has no traded volume: Dhan returns zeros for the whole of 2021, and every backtest loader in this repo forces Volume to 0 regardless. Since the PRICES on those bars are sound -- they are part of the exactly-375-bar sessions above -- dropping them to protect a field nobody reads would lose real information for nothing. The field is zeroed and the count reported. That clamp is scoped to INDEX instruments, which is why `instrument_type` now reaches `normalize_response_data`. The engine also serves NSE_EQ, where volume is a real quantity and a negative one is corruption; there it still refuses the chunk. One existing test moved dates rather than changed meaning: `test_fetch_chunk_rejects_timestamp_outside_requested_window` used 2026-01-03 as its out-of-window stamp, which is a Saturday -- the weekday clip now removes it before the window check runs, so it proved nothing. It uses the following Monday. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five years of one-minute candles is 459,152 bars -- about 45 MB of JSON. The
dashboard's transport has no compression and writes one body in one call, so
that cannot be a single response. History is therefore cut into pages of 2,000
bars (~160 KB each) and the browser asks for the next older one as the
operator scrolls back.
`Dependencies/dashboard_history.py` is a module of its own rather than part of
`dashboard_indicators.py` because that module is PURE --
`test_the_module_reads_no_configuration` AST-asserts it never imports `os` --
and nothing that opens a file can live there.
Nothing here touches the market-data store, a worker, the broker or the
session state. It reads one CSV. That is what lets the chart gain five years
of scroll-back without adding a single call to the trading path, and why the
safety-contract tests pass unchanged.
MEASURED ON THE REAL FILE, and it changed the design twice:
- 29.5 seconds to read 28.8 MB and render 277 pages. On the builder thread
that would stop the live view rebuilding for half a minute, and the page
declares the runner stale after five quiet intervals. History loads on a
thread of ITS OWN, and a test proves the live view keeps advancing while
it does -- verified to fail when the load is put back on the builder.
- ~63 MB of RSS. Real, but the dashboard is opt-in and off by default.
Page 0 is the NEWEST slice, because that is the order a browser wants them:
scroll left, ask for 0, then 1, then 2. Numbering from the oldest end would
make the first request depend on how much history happens to exist. A missing
key IS the "nothing older" answer, so the endpoint needs no bounds arithmetic
of its own and cannot disagree with the store.
`/api/history` is the first endpoint that reads anything from the request, so
it validates rather than infers: the timeframe must be in a frozen set, the
page a plain non-negative integer, blank values included (`keep_blank_values`
so `?page=` is refused rather than silently read as absent). Everything else
is a 4xx with a one-line body. No ETag, deliberately -- every response here is
`no-store`, so the browser has nothing to revalidate against and a 304 would
leave it with no body to draw.
The resampler is injected, not imported, so history uses the STRATEGIES' own
`resample_ohlc_from_1m` -- the same object the live chart uses. Two resamplers
would eventually disagree about a bucket boundary and the chart would show a
seam exactly where history meets the live session.
The loader repeats the extractor's session and weekday checks rather than
trusting the file: a resumed download can leave one CSV cleaned by two rule
versions, which is exactly what happened here -- 509 weekend rows survived in
the chunks fetched before the weekday rule existed.
A missing or unreadable CSV is not an error. The chart shows the live session
alone, exactly as it did before history existed, and the log says how to
populate it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… scroll
The chart now draws years of candles: history pages are held separately from
`chartPayload` and spliced in at draw time, so the once-a-minute refresh that
replaces the live block wholesale no longer throws away everything the
operator scrolled back to.
The seam matters. The live block and the newest history page overlap, and
lightweight-charts drops bars SILENTLY when times are not strictly ascending
-- the same failure that produced a half-empty 5m chart before. History is cut
at the first live timestamp and the live copy wins, because it is the one that
keeps updating.
Indicator columns stay on the live window alone; history pages carry candles
and nothing else. That needs no padding because these are {time, value}
points: the library places them by time, so they stop where the live window
starts instead of sliding out of step with the bars underneath.
Prepending never refits. `fitContent` is the one fit call site and it exists
for timeframe switches; firing it when a page arrived would yank the viewport
back to the whole series, which is the opposite of what scrolling back is for.
Prepending N bars shifts every logical index by N, so the range is moved by
the same N.
Three bugs found by driving the real five years in a browser, not by reading:
- The first page never loaded at all when the container had no width yet.
The trigger was a visible-range event, which only arrives once a FIT has
landed, and `applyPendingFit` refuses a zero-width container -- a state the
chart can sit in indefinitely, because the ResizeObserver meant to catch
the layout has been measured NOT firing in some panes. The first page is
now fetched outright rather than inferred from a range event.
- A 404 while the server was still building history disabled scroll-back for
the whole session. The client could not tell "not ready yet" from "no more
pages"; building measured 29.5 seconds, so this was easy to hit. A 404 now
only means exhausted once a page has actually loaded.
- A page falling entirely INSIDE the live window was dropped by the merge, so
the chart gained nothing, the viewport did not move, and no range event
arrived to ask for the next one -- scroll-back simply stopped. Not
reachable at the shipped sizes (2,000 against 375) but DASHBOARD_CHART_BARS
goes to 2,200, which would have broken it silently. A page that adds
nothing now pulls the next one.
Verified against the real file: 230 pages, loads strictly sequentially 0..N,
stops exactly when a page reaches back past the live window, and the chart
spans 2026-09-08 to 2026-09-16 with the weekend correctly absent.
Also guards the `/api/chart` fetch, which is fired without `await` from the
poll loop and could otherwise put two requests in the air for one version.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The chart drew ONE CPR across the whole of history. TradingView draws a
different one for each day, from that day's predecessor, spanning only that
day -- and that is what the levels actually mean.
`createPriceLine` cannot express this: a price line is full chart width by
definition, which is the API rather than a style. Each level is now a
LineSeries whose data is two points per band, plus a WHITESPACE point -- a
time carrying no value -- to break the line before the next day. Without that
break the series would draw a diagonal from yesterday's pivot to today's,
straight across the overnight gap.
Verified by sampling the rendered canvas rather than trusting the look: inside
the plot area the pivot's cyan has a maximum column height of ONE pixel and no
column holds four or more. Every band is a pure horizontal segment with
nothing joining it to the next.
The bands come from `pivot_levels`, the repository's own algebra, via the same
group-aggregate-shift shape `_add_daily_cpr` already uses -- not a second copy
of the formulas. A test compares the two directly. The prior window stays
09:15-15:15, the deliberate CHART-ONLY divergence, and it is the operator's
own rule: the closing auction can distort the day's close, so the level that
matters is where the market actually was at 15:15. A test proves a 15:20 spike
cannot reach the levels.
The monthly ladder is the same function over `to_period("M")`, ready for the
Daily timeframe where a one-bar-wide daily band would say nothing. The first
day and the first month have NO band rather than a band of zeros, because they
have no predecessor.
Two things the design had to get right:
- The memo signature keyed on a single pivot. With 1,225 bands that is
wrong: it now keys on the set's extent and its newest pivot, so a new
session, a newly loaded page and a timeframe switch each change it.
Exercised in the browser -- ticking a group off and back on redraws both
ways.
- Bands are CLIPPED to the bars actually loaded. Every series shares one
time scale built from the union of their times, so handing over five years
of bands while a week of candles is loaded would stretch the scale across
five years of empty chart.
Today's band always comes from the LIVE payload, never the stored ladder: the
history CSV can be days old, and today is the day being traded.
The ladders ride `/api/history` because they are the same kind of thing --
static chart data, rendered once, handed out as bytes -- which is why the
frozen set it validates against is now HISTORY_SERIES rather than
HISTORY_TIMEFRAMES.
Measured on the real file: 1,225 daily bands and 60 monthly ones, and the
newest pivot matches the live payload's to the paisa.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five years of candles are only useful at a timeframe that can show them. The
chart gains D beside 1m and 5m: 1,226 daily bars spanning 2021-09 to today,
for the next-session macro read.
Its CPR is MONTHLY, not daily. A daily band on a daily candle is one bar wide
and says nothing; a monthly one -- each month's levels taken from the month
before it -- is the statement that matters at this range. The ladder is the
same function as the daily one, over `to_period("M")`.
The Daily series is history plus today, and today is folded out of the LIVE
minute bars rather than waited for: the runner's store holds minutes and the
history CSV is only as fresh as the last download, so without that the
timeframe would stop at yesterday. Today's bar is stamped at 09:15 of the
session exactly as the server stamps its own, so it lands on the same grid.
The Daily series carries its OWN stochastic, computed on daily candles. The
live payload's is computed on one-minute bars and says nothing about a daily
one. A broken indicator costs its column and not the chart -- tested.
VWAP is hidden on Daily and its control dimmed, rather than drawn as a line
that looks like it means something: a session VWAP has no meaning on a daily
candle.
Both captions were made to tell the truth here, because both would otherwise
lie: the title says "CPR from the prior MONTH", and the provenance line
describes the month band -- "Prior month 2026-08 - H 24774.3 L 23993.6 -
pivot 24282.77 (chart) - sessions truncated at 15:15" -- instead of a session
nobody is looking at. Read against the served payload in a browser, they
match to the paisa.
Deliberately NOT done: the minute timeframes carry no indicator columns in
history, so scrolled-back candles show the CPR bands alone. VWAP and a minute
stochastic are statements about the session being traded, and shipping them
for five years would grow every page for a number nobody reads there.
Two gate findings fixed on the way:
- `test_every_element_the_page_script_looks_up_exists_in_the_markup` treated
ANY two-string array as a wiring pair, so an ordinary
`for (const name of ["stoch_k", "stoch_d"])` read as a missing element id.
It now reads pairs from inside the wiring loop only -- and still catches a
genuinely missing id in both shapes, verified by removing two.
- `dashboard_history` was importing its sibling in a way that made mypy see
one file under two module names. It uses the package name like every other
import here; the module is type-checked through
`mypy nifty_multi_strategy_master.py`, which follows and checks imports --
the same coverage `execution_ledger` and `startup_exposure` have. That run
is how the `.loc` typing error in this file was caught.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Records what the four preceding commits changed, and the numbers they were measured at rather than estimated from: 459,152 bars, 28.8 MB, 1,226 sessions, 277 pages, 29.5 seconds to build, ~63 MB of RSS. ADR-0017 gains an amendment rather than a rewrite: the truncated window, the chart-only divergence and the test that proves the algebra has not forked are all unchanged. What changed is that the levels are a statement about ONE session and are now drawn that way -- and that the Daily timeframe takes a monthly ladder, because a daily band on a daily candle is one bar wide. Six traps added, each one something that cost real time here: a scroll-back trigger that silently depends on a fit having landed, the two meanings of a 404, a history page that can add nothing and stall the cascade, bands that must be clipped to the bars loaded or they stretch the time scale across five empty years, a memo that cannot key on one pivot, and assets being read into memory at startup so on-disk edits never reach a running server. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two bugs from this branch's own self-review, both in the monthly ladder, both invisible to the suite. THE MONTH WAS NOT TRUNCATED AT 15:15. `cpr_segments` read the month's high, low and close off the DAILY bars. That looks equivalent to reading them off minutes and is not: every daily bar is stamped 09:15, so the 09:15-15:15 filter matched all of them and truncated nothing, and the month's close silently became the closing-auction print that ADR-0017 exists to keep out. Measured on the real file: August 2026 shipped a pivot of 24282.77 against the truncated 24249.25, with the high 70.4 points out. The caption said "sessions truncated at 15:15" the whole time, which made it a page telling the operator something untrue -- the one thing this dashboard is built not to do. `_segments` now takes its levels and its span from DIFFERENT frames, because they are different questions. Levels always come from MINUTE bars, the only frame where truncating means anything. The span comes from whatever timeframe the band is drawn on: minutes for the daily ladder, daily bars for the monthly one. THE CURRENT MONTH WAS DROPPED ON THE DAILY TIMEFRAME. `band.to < todayStart` exists only to clear room for the live band, and there is no live band on the monthly ladder. Every daily bar is its own day, so the walk-back that finds "where today starts" never moves and `todayStart` is just the last bar -- exactly where the newest month band ends. The filter therefore excluded the current month every time, and the chart drew last month's levels while the caption named this month's. The monthly test could not have caught either: its fixture emits one bar a minute from 09:15, so it never produces a bar after 15:15 and the assertions hold whichever frame the levels come from. The daily ladder has had `test_the_prior_session_is_truncated_at_1515` since it was written; the monthly one now has its counterpart, with a 15:25 print that must not reach the levels, plus a test pinning the span to the daily bars. Verified to fail with the old derivation restored. Confirmed against the five-year file: the newest month now reports H 24703.9 L 23993.6 pivot 24249.25, the caption matches it, all 60 bands survive the Daily filter, and the pivot line reaches the right edge of the plot. The daily ladder is untouched at 1,225 bands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
None of these change what the feature does; each closes a way it was quietly wrong or wasteful. DATA GUARDS THAT HAD BEEN WIDENED TOO FAR `fillna(0.0)` ran BEFORE the volume validity test, so a NaN could never reach the INDEX-scoped branch and every instrument was exempted from a guard meant to be relaxed for indices alone. This engine also serves NSE_EQ and NSE_FNO, where an absent volume is corruption rather than a non-answer. NaN is now part of `invalid` and only an index forgives it. An unparseable timestamp was being DROPPED rather than refused. `coerce` makes it NaT, and NaT compares False against a `datetime.time`, so the session clip discarded the row: the chunk came back quietly short, was appended, and moved the resume point past a gap nobody was told about. It now fails the chunk, before anything can filter it away. The manifest recorded how MUCH of a file was its own but never WHICH file. A CSV replaced, restored or hand-edited between runs still "fits" the stored byte count, so `truncate_to` would cut at a boundary that means nothing and the resume would append past whatever was lost -- leaving a hole that still looks ascending and de-duplicated. The manifest now stores the first data row and refuses progress that does not match it. CORRECTNESS ON THE CHART Today's daily bar was built from whatever the live window happened to hold. A mid-session start -- which this system explicitly supports -- gave it a partial open, high and low while stamping it as the whole session, and because that stamp collides with history's own bar for today the partial one REPLACED the complete one instead of losing to it. History's bar now wins when there is one; when there is not, a partial bar still beats today missing entirely. A 404 before any page had loaded was never treated as final, which was right for the 29.5s build but meant a runner with no CSV re-asked every minute for the life of the session. Now bounded by TIME rather than a retry count: a count would be spent by the several `applyTimeframe` calls of the first second, where five minutes comfortably covers the build and then stops. WASTE `bar_records` was called over the whole frame and then sliced, building every bar as a dict before any of it was needed -- on the order of 150-250 MB for the 459,152-bar minute series, live at once beside the rendered pages. Pages are now serialised from a frame slice, so the peak is one page. A CPR checkbox went through `applyTimeframe`, re-uploading every candle, VWAP point and stochastic point for a toggle that touches none of them. It redraws the CPR series alone. DEAD CODE `HistorySeries.page()` and `DashboardHistory.summary()` were reached only by their own tests. `page()` is gone -- the transport serves from `page_map`, so what that map contains IS the contract, and the test now says so. `summary()` became `describe()`, which the master logs: a page count alone is not checkable, but a date range is, and a backfill that quietly starts two years late looks identical by page count. Verified in a browser on 8799: the R3/R4 toggle adds and removes its levels and the price scale follows. Worth recording that two pixel-sampling checks I ran against it were WRONG, not the code -- #ef5f5f is also the candle down-colour, so the first counted 1,226 candles, and the second's tolerance was too tight for a 1px antialiased line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DoRmAmMu1997
added a commit
that referenced
this pull request
Sep 21, 2026
Per-day CPR (#174) drew a shallow DIAGONAL from one day's levels to the next on 1m and 5m instead of separate horizontal bands. The cause is in the library, not the data. lightweight-charts 5.2.1 runs `rows.filter(hasValue)` in its data layer, so the whitespace point the bands carried -- a time with no value, meant to break the stroke -- never reached the series. The line was one continuous path and the gap never existed. The whitespace contributed exactly one empty column. Two things stretched that artifact from a step into a long diagonal: a band's `to` of 15:29 is not a 5-minute bucket boundary, and bands were admitted on interval overlap alone, so a CSV older than the live window produced a run of bands with no candles under them. Each level now carries ONE point per band, placed on the first loaded candle at or after the band's start, drawn with `LineType.WithSteps` -- the renderer's step branch is `lineTo(x, previousY)` then `lineTo(x, y)`, so a slope is unreachable. A band with no loaded candle is skipped, which leaves the previous level running rather than drawing into empty space, and every point sits on a real bar time so the CPR series adds no columns to the shared scale. Measured on a scratch server against the five-year CSV, in the shape that reproduces it -- history ending days before the live window. Summed vertical travel of each level line across the view, before -> after: pivot 43.4px -> 0.0, BC/TC 47.9 -> 0.0, PDH/PDL 50.8 -> 0.2, R1/R2 22.5 -> 0.0, S1/S2 66.3 -> 0.0. 1m, 5m and Daily all step cleanly and the group toggles still follow immediately. ADR-0017 asserted the whitespace mechanism, so it carries a correction rather than a quiet edit. The LLD gains both traps, including why the #174 pixel check passed: it measured column HEIGHT, which a shallow diagonal satisfies by construction. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three things you asked for — per-day CPR, five years of history, and a Daily timeframe with a monthly CPR — plus the data work that turned out to be needed to get there. Ten staged commits, each left green.
What the chart does now
A CPR for every day. It drew one across the whole of history, because
createPriceLineis full chart width by definition. Each level is now a LineSeries carrying two points per band plus a whitespace point — a time with no value — to break the line before the next day. Without that break the series draws a diagonal from yesterday's pivot to today's, straight across the overnight gap.Verified by sampling the rendered canvas rather than trusting the look: inside the plot area the pivot's cyan has a maximum column height of one pixel, and no column holds four or more. Every band is a pure horizontal segment with nothing joining it to the next.
Five years, paged. 459,152 bars is ~45 MB of JSON, so history is cut into pages of 2,000 bars (~160 KB) and the browser asks for the next older one as you scroll back. Page 0 is the newest, so scrolling left just asks 0, 1, 2; a missing key is the "nothing older" answer, so the endpoint needs no bounds arithmetic that could disagree with the store.
A Daily timeframe with a monthly CPR. 1,226 daily bars, each month's levels from the month before. A daily band on a daily candle is one bar wide and says nothing. Today's daily bar is folded out of the live minute bars, so the timeframe does not stop at yesterday. VWAP is hidden there — a session VWAP means nothing on a daily candle — and the Daily series carries its own stochastic, computed on daily candles rather than borrowed from the live minute payload.
Measurement changed the design twice
The data was worse than the plan assumed
None of this was visible until the download ran. Each fix is bounded, loud, and scoped to the historical backfill —
validate_ohlc_frameis deliberately not relaxed, because it guards the live feed too.2022-03-25 09:15)2026-09-16 18:44, flat OHLC)The fetcher also resumes now — it was all-or-nothing over ~21 requests. It paid for itself twice during this work: when a chunk failed, 23,293 already-downloaded rows stayed on disk.
I re-downloaded from scratch at the end rather than accept the first result, because resume had left the file cleaned by two different rule versions — 509 weekend rows survived in chunks fetched before the weekday rule existed. Final dataset: 459,152 rows, 1,226 sessions, zero weekend rows, zero out-of-session rows, zero negative volume, 1,128 days of exactly 375 bars. The two short days (43 and 60 bars) are Diwali Muhurat sessions, correctly kept.
Bugs the browser found that reading would not have
applyPendingFitrefuses a zero-width container, a state the chart can sit in indefinitely because theResizeObservermeant to catch layout has been measured not firing in some panes.DASHBOARD_CHART_BARSgoes to 2,200.Safety
Nothing here touches the market-data store, a worker, the broker or the session state. It reads one file. The safety-contract tests pass unchanged. A missing or unreadable CSV is not an error — the chart shows the live session exactly as before, and the log says how to populate it.
Verification
606 master unittests · 28 market-data-health · 1,558 pytest · ruff · mypy ×2 · compileall · bandit · pre-commit ·
node --check·algo.py check-envclean.Browser-driven on port 8799, never 8787: 230 pages loading strictly sequentially and stopping exactly when one reaches past the live window; per-day bands stepping with no connector; monthly bands across 2022–2026; captions read against the served payload and matching to the paisa.
One limitation, stated plainly: I could not test a genuine mouse-drag scroll-back — synthetic drags do not reach lightweight-charts' own hit-testing. I proved the cascade by shrinking the page size to 60 bars so the real code pages repeatedly on its own (13 pages, 0→12). The trigger, merge, viewport-shift and paging are all verified; the specific "operator drags left" gesture is verified by construction. Worth a minute of your own scrolling.
🤖 Generated with Claude Code