diff --git a/CANONICAL.md b/CANONICAL.md index e03a544..60e4eba 100644 --- a/CANONICAL.md +++ b/CANONICAL.md @@ -36,7 +36,8 @@ Exactly what production `explorer.qmd` loads, all under `https://data.isamples.o The ~9-file facet family looks baroque but is load-bearing: it is the price of fast multi-filter counts with no server. See `EXPLORER_QUERIES.md` for how each -is queried and `DATA_PROVENANCE.md` for how each is built. +is queried and `DATA_PROVENANCE.md` for how the derived files are built (that +doc covers the pre-#290/#304 substrate; see its coverage caveat). ## 2. Superseded versions (still served; do not use) diff --git a/DATA_PROVENANCE.md b/DATA_PROVENANCE.md index 063ba43..73ecf4c 100644 --- a/DATA_PROVENANCE.md +++ b/DATA_PROVENANCE.md @@ -1,8 +1,23 @@ # iSamples Explorer — Data Provenance -How every parquet file the explorer uses is generated, from root to publish. +How the explorer's derived parquet files are generated, from root to publish. +**Not exhaustive as of 2026-08-05 — see the coverage caveat below.** *Reviewed 2026-06-02 (CC, via codebase audit). Complements `SERIALIZATIONS.md` (format/schema reference); this file is the end-to-end build chain + the automation gaps.* +> ⚠️ **Coverage caveat (2026-08-05).** The DAG below documents the **seven-file +> derived substrate as of the 2026-06-02 review**. It does *not* cover the whole +> live `202608` family. Known omissions: +> +> - `sample_facet_masks`, `facet_node_bits`, `sample_facet_index`, +> `sample_facet_index_meta` (the bitmask count path, #299/#304/#305/#313) +> - `sample_facet_membership` +> - `facet_tree_summaries`, `facet_tree_cross_filter` (the tree facet path, #290) +> - the sharded search index `isamples_202608_search_index_v1/` (#171) +> +> The *build chain and the automation gaps* it describes are still accurate for +> the files it does cover; treat it as incomplete rather than wrong. Authoritative +> current inventory: `isamples_202608_release_manifest.json` / `CANONICAL.md`. + > **Load-bearing constraint:** the **root export cannot be regenerated.** It was produced from the iSamples Central Solr API (`central.isample.xyz`), **offline since Aug 2025**. The Zenodo-archived export is a **frozen root**. Any *new* data (e.g. concept URIs, thumbnails) therefore must come from a **per-source supplementary file merged into the base by `pid`** — the "sidecar" pattern (see Stage 3) — not from re-exporting. ## Pipeline DAG @@ -70,6 +85,15 @@ Eric Kansa maintains OpenContext PQG **independently** on GCS (`storage.googleap ## Documentation / automation gaps (remaining) +> ⚠️ **Snapshot note (2026-08-05).** The version-skew bullets below were written +> when the deployed derived files were `202601` and the wide was `202604`. That is +> no longer the live state: the Explorer now serves the **`202608`** family +> (`sample_facets_v4`, `samples_map_lite_v3`, `wide` at 300,303,095 B). The +> *reproducibility* gap the bullets describe is still real and still unresolved — +> only the version numbers in them are historical. Authoritative current inventory: +> [`isamples_202608_release_manifest.json`](https://data.isamples.org/isamples_202608_release_manifest.json), +> human twin `CANONICAL.md`. + - ⚠️ **The deployed `202601` derived files are NOT reproducible** from any available wide. A rebuild yields **528,983** root-material rows (pre-#271); the deployed `sample_facets_v2` has **346,768** — so the live files came from a different/unrecorded Stage-4 process, *and* the data has since rolled (wide is now `202604`). Treat a fresh `build_frontend_derived.py` run as the new source of truth, not as a bit-for-bit reproduction of the deployed files. - **Version skew:** the deployed derived files are `202601` while the wide they should derive from is `202604` (the popup reads `202604`). Rebuilding from `202604` resolves it (tracked in the pipeline epic). - **No R2 upload automation** — file upload to bucket `isamples-ry` + `current/manifest.json` update are manual `wrangler`/dashboard steps. diff --git a/EXPLORER_QUERIES.md b/EXPLORER_QUERIES.md index dacd044..93e7c2a 100644 --- a/EXPLORER_QUERIES.md +++ b/EXPLORER_QUERIES.md @@ -23,17 +23,17 @@ server. You can open any of these URLs directly, or point DuckDB at them one place, `explorer.qmd` around **line 800-864**, e.g.: ```js -lite_url = `${R2_BASE}/isamples_202608_samples_map_lite_v2.parquet` // map points + table +lite_url = `${R2_BASE}/isamples_202608_samples_map_lite_v3.parquet` // map points + table wide_url = `${R2_BASE}/isamples_202608_wide.parquet` // full sample detail -facets_url = `${R2_BASE}/isamples_202608_sample_facets_v3.parquet` // material/context/object_type + search text +facets_url = `${R2_BASE}/isamples_202608_sample_facets_v4.parquet` // material/context/object_type + search text h3_res4_url = `${R2_BASE}/isamples_202608_h3_summary_res4.parquet` // pre-counted globe dots (world zoom) ``` | File | Plain-English role | Roughly how big | |---|---|---| -| `..._wide.parquet` | Full detail for every sample (one row each) — everything else is derived from this | ~280 MB | -| `..._samples_map_lite_v2.parquet` | Slim version with just what the map/table need: coords, label, place, date | ~50-60 MB | -| `..._sample_facets_v3.parquet` | One row per sample: material/context(sampled feature)/object_type as plain URIs, plus a search-text blob | ~60 MB | +| `..._wide.parquet` | Full detail for every sample (one row each) — everything else is derived from this | ~300 MB | +| `..._samples_map_lite_v3.parquet` | Slim version with just what the map/table need: coords, label, place, date | ~63 MB | +| `..._sample_facets_v4.parquet` | One row per sample: material/context(sampled feature)/object_type as plain URIs, plus a search-text blob | ~69 MB | | `..._h3_summary_res{4,6,8}.parquet` | Pre-counted dots for the globe at 3 zoom tiers (continent / region / neighborhood), so zooming out never counts 6M rows live | tiny–few MB | | `..._facet_summaries.parquet`, `..._facet_cross_filter.parquet`, `..._facet_tree_*.parquet` | Pre-computed facet-checkbox counts at various levels of "how many filters are active" — the whole point of these is to avoid a live COUNT over millions of rows | KB–tens of MB | | `..._sample_facet_masks.parquet`, `..._sample_facet_index.parquet` | Bitmask tricks so 2+ facet filters at once are still fast (see `SERIALIZATIONS.md` §4.12 if you want the gory detail) | ~10 MB each | @@ -62,7 +62,7 @@ pre-counted hexagon summaries instead. As you zoom past a threshold, the Explorer swaps to res6, then res8 H3 tiles (same idea, finer hexagons), and eventually to individual points from -`samples_map_lite_v2.parquet` once there are few enough in view to draw +`samples_map_lite_v3.parquet` once there are few enough in view to draw directly. ### ...click a facet checkbox (Material / Sampled Feature / Object Type / Source) @@ -98,15 +98,15 @@ not tens of MB. *(`buildSearchFilterSubstrate()` + `assets/js/search_substrate.j index contract in `SEARCH_INDEX_V1.md`.)* **Fallback path (`?fts=off`, and automatically for identifier queries):** the -original `ILIKE`-style scan against `sample_facets_v3.parquet`'s description +original `ILIKE`-style scan against `sample_facets_v4.parquet`'s description column: ```sql -SELECT pid, label, source, place_name FROM read_parquet('sample_facets_v3.parquet') +SELECT pid, label, source, place_name FROM read_parquet('sample_facets_v4.parquet') WHERE description ILIKE '%pottery%' ``` *(`buildSearchFilter()` in `explorer.qmd`.)* This path downloads much more -data on first search (the scan touches most of the ~60 MB file) but handles +data on first search (the scan touches most of the ~69 MB file) but handles one thing the index cannot: **pasted identifiers** (ARK / IGSN / DOI — e.g. `ark:/28722/k2000hz7r`), which get exact-matched against the `pid` column. Identifier-looking queries are routed here automatically; you never need the @@ -119,20 +119,20 @@ mode. ### ...view the Samples table -The table pages through `samples_map_lite_v2.parquet` (coords/label/place/ -date) and, as of #311, joins in `sample_facets_v3.parquet` for +The table pages through `samples_map_lite_v3.parquet` (coords/label/place/ +date) and, as of #311, joins in `sample_facets_v4.parquet` for material/object type/sampled feature — one query per page (default page size), not the whole result set: ```sql WITH page AS ( SELECT pid, label, source, latitude, longitude, place_name, result_time - FROM read_parquet('samples_map_lite_v2.parquet') + FROM read_parquet('samples_map_lite_v3.parquet') WHERE ORDER BY pid LIMIT 50 OFFSET 0 ) SELECT page.*, f.material, f.context, f.object_type -FROM page LEFT JOIN read_parquet('sample_facets_v3.parquet') AS f ON f.pid = page.pid +FROM page LEFT JOIN read_parquet('sample_facets_v4.parquet') AS f ON f.pid = page.pid ``` *(`loadPage()`, `explorer.qmd` ~line 2755-2775.)* "Download CSV" (#312) runs the same shape without the `LIMIT`/`OFFSET` (capped at 50,000 rows so an @@ -151,7 +151,7 @@ LEFT JOIN read_parquet('vocab_labels.parquet') mat_lbl ON mat_lbl.uri = mat.pid WHERE s.pid = '' ``` This is the one query that reads from `wide.parquet` on click (everything -above deliberately avoids touching the 280 MB wide file until you actually +above deliberately avoids touching the 300 MB wide file until you actually need full detail on one sample). ## Try it yourself @@ -170,7 +170,7 @@ GROUP BY n ORDER BY 2 DESC; -- the default path since 2026-07-17 probes the sharded search index instead -- (JS, not a single SQL statement — see SEARCH_INDEX_V1.md) SELECT pid, label, source -FROM read_parquet('https://data.isamples.org/isamples_202608_sample_facets_v3.parquet') +FROM read_parquet('https://data.isamples.org/isamples_202608_sample_facets_v4.parquet') WHERE description ILIKE '%pottery%' LIMIT 20; ``` diff --git a/EXPLORER_STATE.md b/EXPLORER_STATE.md index 3a537d8..c0088f9 100644 --- a/EXPLORER_STATE.md +++ b/EXPLORER_STATE.md @@ -328,6 +328,14 @@ compatible with all of them. ### Light-path addendum: two-button scope selection ([#178](https://github.com/isamplesorg/isamplesorg.github.io/issues/178), 2026-05-08) +> ⚠️ **Historical (2026-05-08).** This addendum records the decision as taken and +> the SQL as it stood then — including the `sample_facets_v2` / `samples_map_lite` +> filenames, which are two generations stale (now `_v4` / `_v3`), and the in-browser +> ILIKE scan, which is no longer the default search (the sharded BM25 substrate +> shipped as default 2026-07-17; ILIKE survives as `?fts=off`). The two-button scope +> *decision* still holds; treat the filenames and search mechanics here as a record +> of the time, not as current reference. + Hana's mockup ([Figma 213:394](https://www.figma.com/design/Nqkuqh3Z4aqVh0nmwUAgKg/iSamples-Wireframe-1.0?node-id=213-394)) proposed a two-button search UI: "Search Selected Areas" (viewport-scoped) and "Search Entire World" (full-corpus). Implemented as a Light extension diff --git a/README.md b/README.md index ee611b5..8b472dc 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,15 @@ title: isamples.github.io subtitle: README for the isamples.github.io source --- +> ⚠️ **Snapshot note (2026-08-05).** The **`202601`** file examples below pin a +> stable, versioned snapshot. Those objects still exist and their byte counts are +> correct, but they are **not what the Interactive Explorer serves today** — the +> live app reads the **`202608`** family, including `sample_facets_v4`, +> `samples_map_lite_v3`, `wide`, and the sharded search index. (Other generations +> may also appear here, such as the `current/` alias or the `202512` narrow file.) +> Use `202601` for a stable citable snapshot; for what the Explorer actually +> loads, see `isamples_202608_release_manifest.json` (human twin: `CANONICAL.md`). + # isamplesorg.github.io This repository provides the source for [SMR fork isamplesorg.github.io](https://smrgeoinfo.github.io/isamplesorg.github.io/). diff --git a/SERIALIZATIONS.md b/SERIALIZATIONS.md index a5e0a74..f2afdc6 100644 --- a/SERIALIZATIONS.md +++ b/SERIALIZATIONS.md @@ -7,6 +7,15 @@ toc: true categories: [data, architecture, parquet] --- +> ⚠️ **Snapshot note (2026-08-05).** The **`202601`** file examples below pin a +> stable, versioned snapshot. Those objects still exist and their byte counts are +> correct, but they are **not what the Interactive Explorer serves today** — the +> live app reads the **`202608`** family, including `sample_facets_v4`, +> `samples_map_lite_v3`, `wide`, and the sharded search index. (Other generations +> may also appear here, such as the `current/` alias or the `202512` narrow file.) +> Use `202601` for a stable citable snapshot; for what the Explorer actually +> loads, see `isamples_202608_release_manifest.json` (human twin: `CANONICAL.md`). + ## 1. Purpose and scope iSamples has roughly a dozen parquet files in circulation at any given diff --git a/data.qmd b/data.qmd index be0c075..d238026 100644 --- a/data.qmd +++ b/data.qmd @@ -5,6 +5,22 @@ toc: true categories: [data, parquet, download] --- +::: {.callout-important} +## Snapshot note (2026-08-05) + +The **`202601`** file examples on this page pin a stable, versioned snapshot. +Those objects still exist and their byte counts are correct, but they are **not +what the Interactive Explorer serves today** — the live app reads the **`202608`** +family, including `sample_facets_v4`, `samples_map_lite_v3`, `wide`, and the +sharded search index. (This page may also reference other generations, such as +the `current/` alias or the `202512` narrow file.) + +Use `202601` when you want a stable, citable reference. For "what the Explorer is +actually loading right now", the authoritative inventory is +[`isamples_202608_release_manifest.json`](https://data.isamples.org/isamples_202608_release_manifest.json) +(human twin: `CANONICAL.md`), which the Explorer cross-checks at boot. +::: + ::: {.callout-tip} **Quick start**: every file on this page is queryable directly from a URL — no bulk download needed. DuckDB's `httpfs` extension fetches only the diff --git a/explorer.qmd b/explorer.qmd index e121185..08d3e39 100644 --- a/explorer.qmd +++ b/explorer.qmd @@ -649,9 +649,9 @@ format: ::: {.callout-note collapse="true"} ## How It Works -1. **Instant** (<1s): Pre-aggregated H3 res4 summary (580 KB) → 38K colored circles +1. **Instant** (<1s): Pre-aggregated H3 res4 summary (0.5 MB) → 38K colored circles 2. **Zoom in**: Automatically switches to res6 (112K) then res8 (176K) clusters -3. **Zoom deeper** (<120 km): Individual sample points from 60 MB lite parquet +3. **Zoom deeper** (<120 km): Individual sample points from 63 MB lite parquet 4. **Click**: Cluster info or individual sample card with full metadata 5. **Search**: Find samples by name — results fly to the location on the globe @@ -2679,7 +2679,7 @@ phase1 = { // === Load facet summaries + SKOS prefLabels, populate filter checkboxes === // // Checkbox value = full URI (matches the URI strings stored in -// sample_facets_v2.parquet's material / context / object_type columns). +// sample_facets_v4.parquet's material / context / object_type columns). // Display label = SKOS prefLabel (en) when available, URI tail otherwise. // Default state: UNCHECKED — empty = no filter. facetFilters = { @@ -6042,7 +6042,7 @@ zoomWatcher = { // Compute the side-panel relevance score in the SAME scan that builds // the pid-set, and materialize the columns doSearch's results list // needs (label / source / place_name + score). This lets doSearch read - // the small search_pids table instead of RE-scanning the 63 MB facets + // the small search_pids table instead of RE-scanning the 69 MB facets // parquet — the old double-scan that pushed broad terms like "pottery" // past CI's 90s smoke budget. search_pids stays pid-keyed (one row per // unique pid), so every other surface's `pid IN (SELECT pid FROM @@ -6621,7 +6621,7 @@ zoomWatcher = { // The search-term match + relevance score are already materialized // in `search_pids` by buildSearchFilter (aliased `s` below), so the // results SELECT and the follow-up COUNT both read that small table - // instead of re-scanning the 63 MB facets parquet — the old + // instead of re-scanning the 69 MB facets parquet — the old // double-scan (one in buildSearchFilter, one here) that pushed // broad terms like `pottery` past CI's 90s smoke budget. The single // remaining facets scan lives in buildSearchFilter. @@ -7633,13 +7633,74 @@ Pre-aggregated H3 hexagonal indices achieve near-instant globe rendering, with s | Phase | Data | Size | Points | |-------|------|------|--------| -| **Instant** | H3 res4 | 580 KB | 38K clusters (continental) | -| **Zoom in** | H3 res6 | 1.6 MB | 112K clusters (city) | -| **Zoom more** | H3 res8 | 2.5 MB | 176K clusters (neighborhood) | -| **Zoom deep** | Map lite | 60 MB (range req.) | Up to 5K individual samples | -| **Click sample** | Full dataset | ~280 MB (range req.) | Full metadata for 1 sample | +| **Instant** | H3 res4 | 0.5 MB | 38K clusters (continental) | +| **Zoom in** | H3 res6 | 1.3 MB | 112K clusters (city) | +| **Zoom more** | H3 res8 | 2.0 MB | 176K clusters (neighborhood) | +| **Zoom deep** | Map lite | 63 MB (range req.) | Up to 5K individual samples | +| **Click sample** | Full dataset | 300 MB (range req.) | Full metadata for 1 sample | + +*Sizes are decimal MB (10⁶ bytes), matching the manifest's byte counts.* + +**Static files, zero backend.** All queries run in your browser via DuckDB-WASM +with HTTP range requests — only the bytes you need are transferred. The manifest +directly inventories 21 objects — including five search-index sidecars — and +summarizes the index's 847 shard files (256 base plus 591 hot); the index +directory holds 852 objects in all. Any one view touches only a handful of them. +The authoritative inventory is +[`isamples_202608_release_manifest.json`](https://data.isamples.org/isamples_202608_release_manifest.json) +(human twin: `CANONICAL.md`), which the Explorer cross-checks at boot. + + + +::: {.callout-tip collapse="true"} +## Check these numbers against the source + +Every **file-size** figure above comes from the published release manifest, which +the Explorer also cross-checks at boot. (The cluster counts come from the parquet +row counts — see below — and the timings, the 120 km threshold and the 5K point +budget are properties of the app, not the data.) To re-derive the sizes without +taking this page's word for it: + +```bash +curl -s https://data.isamples.org/isamples_202608_release_manifest.json \ + | jq '.files | to_entries[] | select(.key|test("h3_summary|_wide\\.|map_lite")) + | {file: .key, bytes: .value.size_bytes}' +``` + +Cluster counts come from the parquet row counts, not the manifest: -**4 parquet files, zero backend.** All queries run in your browser via DuckDB-WASM with HTTP range requests — only the bytes you need are transferred. +```sql +SELECT count(*) FROM read_parquet( + 'https://data.isamples.org/isamples_202608_h3_summary_res4.parquet'); +``` +::: ## See Also diff --git a/how-to-use.qmd b/how-to-use.qmd index 1a0676c..84859d6 100644 --- a/how-to-use.qmd +++ b/how-to-use.qmd @@ -4,6 +4,22 @@ subtitle: "Get started exploring 6.7 million scientific samples" number-sections: false --- +::: {.callout-important} +## Snapshot note (2026-08-05) + +The **`202601`** file examples on this page pin a stable, versioned snapshot. +Those objects still exist and their byte counts are correct, but they are **not +what the Interactive Explorer serves today** — the live app reads the **`202608`** +family, including `sample_facets_v4`, `samples_map_lite_v3`, `wide`, and the +sharded search index. (This page may also reference other generations, such as +the `current/` alias or the `202512` narrow file.) + +Use `202601` when you want a stable, citable reference. For "what the Explorer is +actually loading right now", the authoritative inventory is +[`isamples_202608_release_manifest.json`](https://data.isamples.org/isamples_202608_release_manifest.json) +(human twin: `CANONICAL.md`), which the Explorer cross-checks at boot. +::: + ## Quick Start {.unnumbered} 1. **Open the [Interactive Explorer](/explorer.html)** — a 3D globe loads with clustered sample data diff --git a/index.qmd b/index.qmd index 82b9680..a4a0e26 100644 --- a/index.qmd +++ b/index.qmd @@ -51,7 +51,7 @@ iSamples integrates data from four major scientific repositories: The project uses **geoparquet files + DuckDB-WASM** for efficient, browser-based data access and analysis — no server required. -- **iSamples Full Dataset**: ~280 MB wide format, 6.7M samples +- **iSamples Full Dataset**: ~300 MB wide format, 6.7M samples - **Available via**: Cloudflare R2 with HTTP range requests - **Interactive tools**: [Interactive Explorer](/explorer.html) — search, filter, and explore 6.7M samples on a 3D globe or in a paginated table diff --git a/query-spec.qmd b/query-spec.qmd index 8b61290..a224879 100644 --- a/query-spec.qmd +++ b/query-spec.qmd @@ -8,6 +8,35 @@ sidebar: false categories: [spec, architecture, query] --- +::: {.callout-important} +## Known drift as of 2026-08-05 — read before relying on the substrate bindings + +This spec has not kept pace with the shipped Explorer. The drift is **not +limited to the examples below** — the Explorer-specific substrate bindings, the +availability notes and the open-questions section all contain additional +historical material. Treat the **substrate-neutral query model** as current +intent, and treat anything describing *what the Explorer actually does today* as +a record of the time until this is revised. + +Known instances (illustrative, not exhaustive): + +- **Filenames are a generation stale.** The bindings below name + `sample_facets_v2.parquet`; the Explorer now reads + `isamples_202608_sample_facets_v4.parquet` (and `samples_map_lite_v3`). The + authoritative inventory is + [`isamples_202608_release_manifest.json`](https://data.isamples.org/isamples_202608_release_manifest.json), + human twin `CANONICAL.md`. +- **Text search is described as the old full-scan path.** The `text MATCHES` + binding presents the `ILIKE` scan as what happens "currently" and the + substrate-backed index ([#169](https://github.com/isamplesorg/isamplesorg.github.io/issues/169)) + as future work. In fact the sharded BM25 substrate **shipped as the default on + 2026-07-17**; the `ILIKE` scan survives only as the `?fts=off` escape hatch. +- **Availability notes are stale too.** The spec says `objectType` is "not + currently exposed" — the Explorer ships it as the **Specimen Type** facet. It + also describes `lite` as carrying only H3 res8; `samples_map_lite_v3` carries + res4 and res6 as well. +::: + ::: {.callout-warning} ## Draft — v0.2 diff --git a/tutorials/explorer_guided_tour.qmd b/tutorials/explorer_guided_tour.qmd index c12482d..72a1adf 100644 --- a/tutorials/explorer_guided_tour.qmd +++ b/tutorials/explorer_guided_tour.qmd @@ -12,6 +12,13 @@ format: toc: true toc-depth: 2 theme: cosmo + # #343: the site sets number-sections: true globally, which prefixed these + # headings with an auto-number on top of the "Step N" already in the title — + # so they rendered as "2 Step 2 — ...". The steps are numbered by hand + # because the numbering is part of the tour's voice ("Step 4" is referred to + # as such), so switch OFF the automatic numbering for this page rather than + # stripping the manual numbers. + number-sections: false --- The [Interactive Explorer](https://isamples.org/explorer.html) covers ~6.7 @@ -19,11 +26,11 @@ million physical samples — rock cores, potsherds, tissue vouchers, museum specimens — about 6 million of them on one globe, running entirely in your browser with no server behind it. This tour walks through seven views of it, each answering a question about the data. -Most stops are **live links** — click one and the Explorer opens at the position, -search, and zoom being described (the URL carries that state; a couple of stops +Most steps are **live links** — click one and the Explorer opens at the position, +search, and zoom being described (the URL carries that state; a couple of steps ask you to click one control yourself, where the URL doesn't reach). -A few stops also include a small **"the number behind the view"** code cell that +A few steps also include a small **"the number behind the view"** code cell that computes what you're seeing, live, from the same public data files the Explorer itself reads. Unfold the code to see how — and to take your first step from *viewing* the data to *querying* it. @@ -73,7 +80,7 @@ VOCAB_LABELS = 'https://data.isamples.org/vocab_labels_202608.parquet' H3_RES4 = 'https://data.isamples.org/isamples_202608_h3_summary_res4.parquet' ``` -## Stop 1 — The world at a glance +## Step 1 — The world at a glance **[Open the Explorer →](https://isamples.org/explorer.html)** @@ -116,7 +123,7 @@ with no usable coordinates: they never appear on the globe or in the viewport table, but a world-scope search still finds them. Counting only what's visible would overstate the map and understate the archive. -## Stop 2 — Search that knows concepts +## Step 2 — Search that knows concepts **[Search "pottery Cyprus" →](https://isamples.org/explorer.html?search=pottery+Cyprus)** @@ -135,7 +142,7 @@ Search runs on a pre-built sharded index: your query fetches a few small files, not the whole dataset. The classic full-scan search still exists — add `&fts=off` to the URL to feel the difference. -## Stop 3 — Two sites, two sciences +## Step 3 — Two sites, two sciences **[Fly to central Anatolia →](https://isamples.org/explorer.html#v=1&lat=37.6619&lng=32.8334&alt=150000&mode=point)** @@ -170,7 +177,7 @@ html`

The five densest cells on the globe (click to fly there):

` ``` -## Stop 4 — Slicing by material +## Step 4 — Slicing by material **[Open the Explorer and expand the *Material* facet →](https://isamples.org/explorer.html)** @@ -204,7 +211,7 @@ Rock and mineral dominate (SESAR's size shows through), but biogenic and anthropogenic materials are each hundreds of thousands strong — this is genuinely a cross-domain collection, not a geology database with guests. -## Stop 5 — Density vs identity +## Step 5 — Density vs identity **[Toggle the heatmap →](https://isamples.org/explorer.html)** *(checkbox in the right panel)*, and try the **2D map** (globe button in the map toolbar → @@ -216,7 +223,7 @@ mode answers "which samples, exactly." The honest rule of thumb: heatmaps for patterns, points for identity, and never trust a color's *intensity* as a count — click and read the number instead. -## Stop 6 — From view to data-in-hand +## Step 6 — From view to data-in-hand Any view's sample table (below the globe) shows PID, place, date, and a **Source URL** linking each sample back to its home collection's record. @@ -225,7 +232,7 @@ carries the whole view state, **Copy Link to Current View** gives you a citation-grade pointer to *this exact slice* — paste it in a paper, a class assignment, or an issue report. -## Stop 7 — Under the hood (and doing this yourself) +## Step 7 — Under the hood (and doing this yourself) There is no server. The Explorer is a static page querying public [Parquet](https://parquet.apache.org/) files over HTTP range requests with @@ -235,7 +242,7 @@ further: - **[EXPLORER_QUERIES.md](https://github.com/isamplesorg/isamplesorg.github.io/blob/main/EXPLORER_QUERIES.md)** — plain-English walkthrough of every query the Explorer runs, with copy-paste DuckDB snippets - **[CANONICAL.md](https://github.com/isamplesorg/isamplesorg.github.io/blob/main/CANONICAL.md)** — which data files are canonical (this page uses only those) -- **[DATA_PROVENANCE.md](https://github.com/isamplesorg/isamplesorg.github.io/blob/main/DATA_PROVENANCE.md)** — how every derived file is built from the source export +- **[DATA_PROVENANCE.md](https://github.com/isamplesorg/isamplesorg.github.io/blob/main/DATA_PROVENANCE.md)** — how the derived files are built from the source export (covers the pre-#290/#304 substrate; see its coverage caveat) - The **[iSamples Zenodo community](https://zenodo.org/communities/isamples)** — the archived, citable snapshot of the data substrate diff --git a/tutorials/index.qmd b/tutorials/index.qmd index d3db435..fc02fb3 100644 --- a/tutorials/index.qmd +++ b/tutorials/index.qmd @@ -4,6 +4,22 @@ subtitle: "Learn to explore 6.7 million physical samples from scientific collect number-sections: false --- +::: {.callout-important} +## Snapshot note (2026-08-05) + +The **`202601`** file examples on this page pin a stable, versioned snapshot. +Those objects still exist and their byte counts are correct, but they are **not +what the Interactive Explorer serves today** — the live app reads the **`202608`** +family, including `sample_facets_v4`, `samples_map_lite_v3`, `wide`, and the +sharded search index. (This page may also reference other generations, such as +the `current/` alias or the `202512` narrow file.) + +Use `202601` when you want a stable, citable reference. For "what the Explorer is +actually loading right now", the authoritative inventory is +[`isamples_202608_release_manifest.json`](https://data.isamples.org/isamples_202608_release_manifest.json) +(human twin: `CANONICAL.md`), which the Explorer cross-checks at boot. +::: + ## Start Here {.unnumbered} | Tutorial | What You'll Learn | diff --git a/tutorials/why_h3.qmd b/tutorials/why_h3.qmd index 70e2327..cb05ab7 100644 --- a/tutorials/why_h3.qmd +++ b/tutorials/why_h3.qmd @@ -9,6 +9,22 @@ format: toc-depth: 3 --- +::: {.callout-important} +## Snapshot note (2026-08-05) + +The **`202601`** file examples on this page pin a stable, versioned snapshot. +Those objects still exist and their byte counts are correct, but they are **not +what the Interactive Explorer serves today** — the live app reads the **`202608`** +family, including `sample_facets_v4`, `samples_map_lite_v3`, `wide`, and the +sharded search index. (This page may also reference other generations, such as +the `current/` alias or the `202512` narrow file.) + +Use `202601` when you want a stable, citable reference. For "what the Explorer is +actually loading right now", the authoritative inventory is +[`isamples_202608_release_manifest.json`](https://data.isamples.org/isamples_202608_release_manifest.json) +(human twin: `CANONICAL.md`), which the Explorer cross-checks at boot. +::: + The progressive globe and the Interactive Explorer both render millions of samples by aggregating points into pre-computed [H3](https://h3geo.org/) cells at three resolutions. This page documents *why* H3, why *those* resolutions, and what we considered before adopting it. ::: {.callout-tip}