diff --git a/.claude/knowledge/dr512-epistemic-pm-invariant.md b/.claude/knowledge/dr512-epistemic-pm-invariant.md new file mode 100644 index 00000000..217a63fd --- /dev/null +++ b/.claude/knowledge/dr512-epistemic-pm-invariant.md @@ -0,0 +1,132 @@ +# DR512 / Codec9 universal epistemic P × magnitude law + +**Operator ruling 2026-10-28.** This is the one DR512 law, not a numerical mode +selected by a consumer. It supersedes the signed-negative-bit reading in the +historical `palette512-signed-magnitude9-experiment-20261027.md` and the +two-plane probe in ndarray #381. This is a proposed lab CI gate, not +permission for BGZ wire migration, Quack source authorization, DR64³ layout, +or DT64 production dispatch. + +| Presence P | Magnitude M | Meaning | +|---|---|---| +| 0 | 0 | **-0 / NaN / epistemically NEVER TOUCHED** | +| 0 | 1..255 | negative numeric value -1..-255 | +| 1 | 0 | valid **+0** | +| 1 | 1..255 | positive numeric value +1..+255 | + +One u8 magnitude plus **one** presence bit per numeric row: +64 magnitudes + one u64 presence word = 72 bytes per 64 rows; +512 distinct codewords, exactly 511 signed numeric values and one +never-touched code. Numeric polarity P is NOT access control or alpha +write authority. Negative numeric evidence also is not a source-attested +Boolean negation unless such a mapping is established separately. + +## The ONLY never-touched WHERE + +```text +WHERE (P == 0) AND (M == 0) +``` + +No additional 8-KiB sentinel mask is required. A consumer can derive a +bitmap ON DEMAND if its vector engine needs one, but absence cannot be +inferred from P alone: P=0,M>0 carries a valid negative value. + +## Touched write: normalize zero to +0 + +A write/calculation already admitted by its source owner obtains a **touched** +numeric value. Encode with `Codec9::encode_touched`. If the quantized +magnitude is zero, emit P=1,M=0 unconditionally. Pre-existing untouched +P=0,M=0 rows are not touched, not claimed and must remain untouched. +This holds especially for -0.49999, -0.4 and -0.0: + +- TiesEven: -0.5 rounds to **valid +0**; -0.50001 → -1. +- TiesAway: -0.5 → -1; -0.4 → **valid +0**. +- Explicitly touched NaN/infinity is rejected, not converted into the + never-touched sentinel. Finite out-of-range values saturate at ±255. +- LUT9 preserves an *existing* -0/NaN; when a nonzero negative LUT input + becomes magnitude zero, it emits +0 and never fabricates absence. +- No numerical threshold or range filter `(-0.5,0)` is required. + The determinant is the touched operation and final quantized zero. + +## Existing compile-time owner patterns + +`Codec9` intentionally reuses ndarray +`codec8::{Rounding, TiesEven, TiesAway, Calibration}` and fixes the +other semantic parameters by construction. Asym, Floor and +`NoNaN` must **not** select a second DR512 bit interpretation. Codebook +enums with 512 arbitrary tokens are different from DR512 signed numeric +codes: they must retain their own typed IDs and source mapping. Never +reinterpret `bgz-tensor::WeightPalette` u8 archetypes as signed magnitudes. + +## Cross-repo ownership and CI tripwires + +1. **ndarray:** `src/hpc/codec9.rs`, `src/simd_stride.rs`, + `examples/palette512_probe.rs`, `examples/sm9_presence_probe.rs`, + `examples/lut9_bench.rs`, `crates/simd-masking-parity/src/lib.rs`. + Exhaustive 512-codeword oracle, negative-zero ceiling test, + backend/shape parity, WHERE 64K geometry. +2. **lance-graph DT64/Quack/DR64:** keep owner-issued source versions, + source authorization, K3 predicates, sparse write/snapshot and alpha + claims distinct. A numerical P=0 is **not** whitelist, permission or + proof that an ontology relationship is absent. +3. **Odoo, OGIT/OGAR and crai:** use pinned source codebook IDs; + guard incomplete registry, unresolved IDs and source-attested WHEN + / ON / DO. Test 255/256/257 codebooks explicitly. +4. **MedCare:** no physical clinical-route promotion from a numeric + sentinel or unsigned-magnitude sign bit without its independent + compilation, ACL and source evidence. + +This document does not grant a merge or production ABI change. Every +subsequent consumer PR must either prove exact P×M parity using the +independent oracle or refuse the integration. Operator-rejected +lance-graph #1812 stays quarantined and is NOT implementation evidence. + + +## [-a,+b] LUT distance efficiency, independent cold gate + +For **valid numeric** DR512 pairs `x`, `y` with identical positive +magnitude step, distance need not use a 512×512 LUT: + +```text +same P: abs(m_x - m_y) +opposite P: m_x + m_y +``` + +The result is `u16` (maximum **510**), **not** `u8`. The +512-codeword space has exactly one forbidden -0 sentinel that must +refuse distance instead of being treated as 0. Unit tests independently +decode all **511² = 261,121** valid numeric pairs and compare distances. + +For a separately proposed calibrated asymmetric range [-a,+b], with +negative step a/255 and positive step b/255: + +```text +both negative: abs(m_x-m_y) * a/255 +both positive: abs(m_x-m_y) * b/255 +opposite: m_negative * a/255 + m_positive * b/255 +``` + +The `tests/dr512_epistemic_contract.rs` independent oracle checks the +[-3,+7] case over all 511² valid pairs. **This is a distance-law +analysis, not an approved alternative P×M codec, nor a measured +speedup.** A full pairwise distance table has 512² entries, or **512 KiB** +at u16; even separate same/opposite 256² u16 tables total **256 KiB**. +The arithmetic is a subtract/abs or an add and should be the baseline +before claiming LUT efficiency. Any real SIMD/L1/L2-cache comparison +requires a pinned host/ISA benchmark with end-to-end memory costs. + +## CausalEdge64 extension: DEFERRED, do not implement in this PR + +Current CausalEdge64 bits 0..23 are THREE 8-bit SPO palette codes; +24..39 are NARS truth, 40..58 Pearl rungs, 59..63 epistemic/reasoning. +If each SPO code were extended from 256 to 512 choices, THREE extra +bits are required per edge (one per S/P/O), not an available 3-bit +space in the existing 64-bit contract. A 3-bit sidecar might be +theoretically sufficient but needs all THREE original palette identities +to be associated with the same pinned source/codebook generation and +the sidecar to be versioned and snapshot-attested with them. +Cross-page splits, stale codebooks, incomplete witnesses and partial +writes must refuse reconstruction. Do NOT steal CausalEdge64 truth, +Pearl or Epi5 bits, alter current codecs, or promote a physical layout. +Evaluate independently, under lance-graph's CausalEdge64/DT64/DR64 +owner, only after evidence of correctness and benefit. **DEFERRED**. diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index bd9b51e9..c2778596 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -472,8 +472,30 @@ jobs: # mutually exclusive (`compile_error!`). - run: cargo +nightly doc --no-deps --features approx,serde,rayon + # Canonical DR512 P×M never-touched contract. This must fail loudly on + # an inverted presence bit, a fabricated -0/NaN, a lost negative number, + # or a stale independent SIMD/SM9 reference. No optional/passive gate. + dr512-epistemic-tripwire: + runs-on: ubuntu-latest + name: dr512/epistemic-presence + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@1.99.0 + - uses: Swatinem/rust-cache@v2 + - name: Exhaustive codeword, distance, sparse-WHERE and negative-ceiling oracles + run: cargo test --test dr512_epistemic_contract + - name: Codec9 independent compile-time and scalar oracles + run: cargo test --lib hpc::codec9::tests + - name: LUT9 and its independent scalar lane oracle + run: cargo test --lib simd_stride::tests::lut9 + - name: Legacy Palette512 arithmetic probe must agree on polarity + run: cargo test --example palette512_probe + - name: SM9 presence probe must have only one polarity plane + run: cargo test --example sm9_presence_probe + conclusion: needs: + - dr512-epistemic-tripwire - clippy - format - nostd diff --git a/crates/simd-masking-parity/src/lib.rs b/crates/simd-masking-parity/src/lib.rs index fb19d172..8bf6c4ce 100644 --- a/crates/simd-masking-parity/src/lib.rs +++ b/crates/simd-masking-parity/src/lib.rs @@ -2651,6 +2651,14 @@ fn oracle_fma_f64(a: f64, b: f64, c: f64) -> f64 { // the output sign plane bit by bit; it never calls the primitive. fn check_lut9() -> Result<(), u32> { + // Canonical P×M four-way truth table, independent of random parity. + let identity: [u8; 256] = core::array::from_fn(|i| i as u8); + let mut actual = [0u8; 4]; + let mut actual_presence = [u64::MAX]; + lut9(&[0, 0, 255, 255], &[0b1010], &identity, &mut actual, &mut actual_presence); + if actual != [0, 0, 255, 255] || actual_presence != [0b1010] { + return Err(0x1430); + } let mut rng = SplitMix64(0x1400_0000_0001); // A random table with some zero entries away from index 0, so a sign bit // must be cleared at nonzero magnitudes too, and every quarter differs. @@ -2670,14 +2678,14 @@ fn check_lut9() -> Result<(), u32> { let mags: Vec = (0..n) .map(|i| if pass == 0 { stride(i) } else { rng.next() as u8 }) .collect(); - let mut signs: Vec = (0..words).map(|_| rng.next()).collect(); + let mut presences: Vec = (0..words).map(|_| rng.next()).collect(); if words > 0 && !n.is_multiple_of(64) { // Junk sign bits past the last row must not reach the output. - signs[words - 1] |= u64::MAX << (n % 64); + presences[words - 1] |= u64::MAX << (n % 64); } let mut out = vec![0xAAu8; n + 3]; - let mut out_signs = vec![u64::MAX; words + 1]; - lut9(&mags, &signs, &table, &mut out, &mut out_signs); + let mut out_presences = vec![u64::MAX; words + 1]; + lut9(&mags, &presences, &table, &mut out, &mut out_presences); for i in 0..n { if out[i] != table[mags[i] as usize] { return Err(0x1400 | (pass << 7) | case); @@ -2687,9 +2695,9 @@ fn check_lut9() -> Result<(), u32> { return Err(0x1410 | (pass << 7) | case); } let want = reference_mask(n, words + 1, |i| { - (signs[i / 64] >> (i % 64)) & 1 == 1 && (table[mags[i] as usize] != 0 || mags[i] == 0) + ((presences[i / 64] >> (i % 64)) & 1 == 1) || (table[mags[i] as usize] == 0 && mags[i] != 0) }); - if out_signs != want { + if out_presences != want { return Err(0x1420 | (pass << 7) | case); } } diff --git a/examples/lut9_bench.rs b/examples/lut9_bench.rs index 3f040fc0..0d5df1ca 100644 --- a/examples/lut9_bench.rs +++ b/examples/lut9_bench.rs @@ -5,7 +5,7 @@ //! * `i8 table`: the i8 `Sym` form, one 255-entry table indexed by the //! signed code (`table[c + 127]`), no sign plane; //! * `u8 table + sign`: the scalar form of what `lut9` computes, a -//! 256-entry lookup plus the sign plane cleared where a nonzero input +//! 256-entry lookup plus the epistemic presence bit set where a touched negative input //! gives 0; //! * `lut9`: the SIMD primitive. //! @@ -40,7 +40,7 @@ fn main() { s }; let mags: Vec = (0..N).map(|_| next() as u8).collect(); - let signs: Vec = (0..N / 64).map(|_| next()).collect(); + let presences: Vec = (0..N / 64).map(|_| next()).collect(); let codes: Vec = (0..N) .map(|_| ((next() % 255) as i16 - 127) as i8) .collect(); @@ -57,25 +57,25 @@ fn main() { // Correctness first: the SIMD output must equal the scalar form. let mut out_simd = vec![0u8; N]; - let mut signs_simd = vec![0u64; N / 64]; - lut9(&mags, &signs, &table, &mut out_simd, &mut signs_simd); + let mut presences_simd = vec![0u64; N / 64]; + lut9(&mags, &presences, &table, &mut out_simd, &mut presences_simd); let mut out_scalar = vec![0u8; N]; - let mut signs_scalar = vec![0u64; N / 64]; - let scalar = |out: &mut [u8], out_signs: &mut [u64]| { + let mut presences_scalar = vec![0u64; N / 64]; + let scalar = |out: &mut [u8], out_presences: &mut [u64]| { let rows = mags.as_chunks::<64>().0.iter(); for (w, (chunk, o)) in rows.zip(out.as_chunks_mut::<64>().0).enumerate() { - let mut nonzero = 0u64; + let mut became_positive_zero = 0u64; for (k, (&m, d)) in chunk.iter().zip(o.iter_mut()).enumerate() { *d = table[m as usize]; - // Keep the sign where the result is nonzero or the input is -0. - nonzero |= u64::from(*d != 0 || m == 0) << k; + // P=0,M>0 rounded to zero becomes valid +0. Pre-existing -0 persists. + became_positive_zero |= u64::from(*d == 0 && m != 0) << k; } - out_signs[w] = signs[w] & nonzero; + out_presences[w] = presences[w] | became_positive_zero; } }; - scalar(&mut out_scalar, &mut signs_scalar); + scalar(&mut out_scalar, &mut presences_scalar); assert_eq!(out_simd, out_scalar, "magnitudes differ"); - assert_eq!(signs_simd, signs_scalar, "sign planes differ"); + assert_eq!(presences_simd, presences_scalar, "sign planes differ"); let mut out_i8 = vec![0i8; N]; let t_i8 = best(|| { @@ -85,12 +85,12 @@ fn main() { black_box(&out_i8); }); let t_scalar = best(|| { - scalar(&mut out_scalar, &mut signs_scalar); - black_box((&out_scalar, &signs_scalar)); + scalar(&mut out_scalar, &mut presences_scalar); + black_box((&out_scalar, &presences_scalar)); }); let t_simd = best(|| { - lut9(black_box(&mags), black_box(&signs), &table, &mut out_simd, &mut signs_simd); - black_box((&out_simd, &signs_simd)); + lut9(black_box(&mags), black_box(&presences), &table, &mut out_simd, &mut presences_simd); + black_box((&out_simd, &presences_simd)); }); println!( diff --git a/examples/palette512_probe.rs b/examples/palette512_probe.rs index 6359505f..48de6397 100644 --- a/examples/palette512_probe.rs +++ b/examples/palette512_probe.rs @@ -1,8 +1,8 @@ //! Palette512 / SignedMagnitude9 isolated arithmetic probe. //! //! This is NOT a 512-centroid codebook. It uses one u8 magnitude per lane -//! and a 64-bit sign plane for 64 lanes: 72 bytes total. Of 512 codewords, -//! sign=1,magnitude=0 is the sole NaN sentinel; the other 511 are numbers. +//! and a 64-bit epistemic presence plane for 64 lanes: 72 bytes total. Of 512 codewords, +//! presence=0,magnitude=0 is the sole -0/NaN/never-touched codeword; the other 511 are numbers. //! No production codec, public ABI, calibration, or wire format is changed. #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -18,17 +18,17 @@ struct SignedMagnitude9 { } impl SignedMagnitude9 { - /// Bit 8 is the sign; bits 0..7 are the magnitude. + /// Bit 8 is epistemic presence: 0 means negative, 1 means positive. fn from_code(code: u16) -> Self { assert!(code < 512, "not a 9-bit codeword"); Self { magnitude: code as u8, - negative: (code & 0x100) != 0, + negative: (code & 0x100) == 0, } } fn code(self) -> u16 { - u16::from(self.magnitude) | ((self.negative as u16) << 8) + u16::from(self.magnitude) | (u16::from(!self.negative) << 8) } fn value(self) -> Result { @@ -53,33 +53,33 @@ impl SignedMagnitude9 { } } -/// Bit i of signs belongs to lane i of magnitudes. +/// Bit i of presences belongs to lane i of magnitudes. /// repr(C) pins the 72-byte in-memory footprint, NOT a persisted wire format. #[repr(C)] #[derive(Clone, Debug)] struct MagnitudeSignBlock64 { magnitudes: [u8; 64], - signs: u64, + presences: u64, } impl MagnitudeSignBlock64 { fn from_codes(codes: &[u16; 64]) -> Self { let mut magnitudes = [0u8; 64]; - let mut signs = 0u64; + let mut presences = 0u64; for (i, &code) in codes.iter().enumerate() { let lane = SignedMagnitude9::from_code(code); magnitudes[i] = lane.magnitude; - if lane.negative { - signs |= 1u64 << i; + if !lane.negative { + presences |= 1u64 << i; } } - Self { magnitudes, signs } + Self { magnitudes, presences } } fn lane(&self, i: usize) -> SignedMagnitude9 { SignedMagnitude9 { magnitude: self.magnitudes[i], - negative: ((self.signs >> i) & 1) != 0, + negative: ((self.presences >> i) & 1) == 0, } } @@ -94,8 +94,8 @@ impl MagnitudeSignBlock64 { } fn main() { - let plus = SignedMagnitude9::from_code(0x0ff); - let minus = SignedMagnitude9::from_code(0x1ff); + let plus = SignedMagnitude9::from_code(0x1ff); + let minus = SignedMagnitude9::from_code(0x0ff); let signed_product = plus.mul_add(minus, 0).expect("valid product"); let lanes = [plus.code(); 64]; let block = MagnitudeSignBlock64::from_codes(&lanes); @@ -126,23 +126,23 @@ mod tests { Ok(v) => { assert!((-255..=255).contains(&v)); let expected = if code < 256 { - code as i16 - } else { -((code & 255) as i16) + } else { + (code & 255) as i16 }; assert_eq!(v, expected); valid += 1; } Err(ProbeError::Nan) => { - assert_eq!(code, 0x100); + assert_eq!(code, 0); nan += 1; } Err(ProbeError::Overflow) => panic!("decoding cannot overflow"), } } assert_eq!((valid, nan), (511, 1)); - assert_eq!(SignedMagnitude9::from_code(0).value(), Ok(0)); - assert_eq!(SignedMagnitude9::from_code(0x100).value(), Err(ProbeError::Nan)); + assert_eq!(SignedMagnitude9::from_code(0x100).value(), Ok(0)); + assert_eq!(SignedMagnitude9::from_code(0).value(), Err(ProbeError::Nan)); } #[test] @@ -164,15 +164,15 @@ mod tests { fn sixty_four_lane_sign_plane_matches_wide_reference() { assert_eq!(std::mem::size_of::(), 72); let lhs = std::array::from_fn(|i| match i % 4 { - 0 => 0x0FF, // +255 - 1 => 0x1FF, // -255 - 2 => 0x000, // +0 - _ => 0x100 | i as u16, // negative magnitudes 3..63 + 0 => 0x1FF, // +255 + 1 => 0x0FF, // -255 + 2 => 0x100, // +0 + _ => i as u16, // negative magnitudes 3..63 }); let rhs = std::array::from_fn(|i| match i % 3 { - 0 => 0x1FF, - 1 => 0x0FF, - _ => 0x000, + 0 => 0x0FF, + 1 => 0x1FF, + _ => 0x100, }); let a = MagnitudeSignBlock64::from_codes(&lhs); let b = MagnitudeSignBlock64::from_codes(&rhs); @@ -185,9 +185,9 @@ mod tests { #[test] fn nan_propagates_even_when_other_operand_is_zero() { - let zeros = [0u16; 64]; + let zeros = [0x100u16; 64]; let mut negzero = zeros; - negzero[19] = 0x100; + negzero[19] = 0; let z = MagnitudeSignBlock64::from_codes(&zeros); let n = MagnitudeSignBlock64::from_codes(&negzero); assert_eq!(n.fused_dot(&z), Err(ProbeError::Nan)); @@ -196,8 +196,8 @@ mod tests { #[test] fn overflow_is_not_misreported_as_nan() { - let plus = SignedMagnitude9::from_code(255); - let minus = SignedMagnitude9::from_code(511); + let plus = SignedMagnitude9::from_code(511); + let minus = SignedMagnitude9::from_code(255); assert_eq!(plus.mul_add(plus, i32::MAX), Err(ProbeError::Overflow)); assert_eq!(minus.mul_add(plus, i32::MIN), Err(ProbeError::Overflow)); } diff --git a/examples/sm9_presence_probe.rs b/examples/sm9_presence_probe.rs index 895fb475..45310955 100644 --- a/examples/sm9_presence_probe.rs +++ b/examples/sm9_presence_probe.rs @@ -1,102 +1,77 @@ -//! Sign-magnitude 9-bit lanes with NULL carried by the presence plane. +//! DR512 P × magnitude laboratory. ONE bit plane: P=0,M=0 is -0/NaN/never touched. +//! P=0,M>0 negative; P=1,M=0 +0; P=1,M>0 positive. +//! No independent sign or NULL bit and no second bitmap for WHERE(P=0 && M=0). +//! Odoo/DT64 source authority and alpha-claim admission are separate. +//! Retains the original comparative i8 vs magnitude-LUT numerical experiments. //! -//! A follow-up to the Palette512 / SignedMagnitude9 probe (#380). There, the -//! codeword "sign set, magnitude 0" was reserved as the NaN sentinel. Here NaN -//! is not a codeword: a row is NaN (NULL) because its presence bit is clear, -//! whatever its payload holds. That is the NaN/presence rule `ndarray::simd` -//! ships for float lanes (`not_nan_*_to_mask`, `nan_where_clear_*`, -//! `presence_nan_mismatches_*`), applied to an integer lane, where the presence -//! plane is the only carrier. -//! -//! Per 64-lane block: 64 `u8` magnitudes, one `u64` sign plane, one `u64` -//! presence plane. Every magnitude/sign pair is a value, -255..=+255; the -//! writer stores +0 for zero and for missing rows, so each row has one encoding. -//! -//! What it measures: -//! -//! 1. The masked dot product: lanes split by `sign_a ^ sign_b` into a positive -//! and a negative unsigned accumulator, NULL rows excluded by the presence -//! AND, `n` the popcount of the same mask. Checked exhaustively against a -//! decoded oracle. Scalar reference only: the SIMD cost of the u8 x u8 -//! accumulation is not measured here. -//! 2. tanh exactness against i8 symmetric (-127..=127), both as a 256-entry -//! lookup table and as output quantization alone. tanh is odd, so the -//! sign-magnitude table is indexed by magnitude and the sign plane passes -//! through untouched. -//! -//! Run `cargo run --release --example sm9_presence_probe` for the table and -//! `cargo test --release --example sm9_presence_probe` for the checks. - -use ndarray::simd::nan_where_clear_f64; +//! Run cargo test --example sm9_presence_probe for exhaustive lab evidence. -/// One 64-lane block. `presence` bit i clear means lane i is NULL (NaN). +/// One 64-lane DR512 block: u8 magnitude + ONE epistemic presence bit. +/// P=0,M=0 is never touched (-0/NaN), P=0,M>0 negative. +/// P=1,M=0 valid +0, P=1,M>0 positive. +/// Never conflate this numeric code with source-write authorization. #[derive(Clone, Debug)] struct Sm9Block { magnitudes: [u8; 64], - signs: u64, presence: u64, } impl Sm9Block { - /// Store `values[i]` (`None` = NULL). A NULL or zero row is written as +0. fn from_values(values: &[Option; 64]) -> Self { - let mut b = Self { + let mut out = Self { magnitudes: [0; 64], - signs: 0, presence: 0, }; for (i, v) in values.iter().enumerate() { if let Some(v) = *v { - assert!((-255..=255).contains(&v), "value {v} outside -255..=255"); - b.presence |= 1 << i; - b.magnitudes[i] = v.unsigned_abs() as u8; - if v < 0 { - b.signs |= 1 << i; + assert!((-255..=255).contains(&v), "out of DR512 range: {v}"); + out.magnitudes[i] = v.unsigned_abs() as u8; + // Touched valid zero is ALWAYS +0. Negative nonzero clears P. + if v >= 0 { + out.presence |= 1u64 << i; } } } - b + out } - /// The value of lane `i`, `None` when the presence bit is clear. fn value(&self, i: usize) -> Option { - if (self.presence >> i) & 1 == 0 { - return None; + let magnitude = i16::from(self.magnitudes[i]); + let positive = (self.presence >> i) & 1 != 0; + if !positive && magnitude == 0 { + None + } else { + Some(if positive { magnitude } else { -magnitude }) } - let m = i16::from(self.magnitudes[i]); - Some(if (self.signs >> i) & 1 == 1 { -m } else { m }) } - /// Masked dot product over the rows both blocks hold (`filter` narrows it - /// further). Returns `(dot, n)`. The sign is a mask, not arithmetic: lanes - /// whose signs differ go to the negative accumulator. + fn never_touched(&self, i: usize) -> bool { + (self.presence >> i) & 1 == 0 && self.magnitudes[i] == 0 + } + + /// Filter/WHERE is a selector, not proof of source authority. fn masked_dot(&self, rhs: &Self, filter: u64) -> (i64, u32) { - let sel = self.presence & rhs.presence & filter; - let negative = self.signs ^ rhs.signs; - let (mut pos, mut neg) = (0u64, 0u64); - let mut bits = sel; - while bits != 0 { - let i = bits.trailing_zeros() as usize; - let p = u64::from(self.magnitudes[i]) * u64::from(rhs.magnitudes[i]); - if (negative >> i) & 1 == 1 { - neg += p; - } else { - pos += p; + let mut acc = 0i64; + let mut count = 0u32; + let mut selected = filter; + while selected != 0 { + let i = selected.trailing_zeros() as usize; + if let (Some(a), Some(b)) = (self.value(i), rhs.value(i)) { + acc += i64::from(a) * i64::from(b); + count += 1; } - bits &= bits - 1; + selected &= selected - 1; } - (pos as i64 - neg as i64, sel.count_ones()) + (acc, count) } - /// Decode to an f64 lane at `scale`, NaN where the presence bit is clear. + // Used only by the tests below; `main` counts NULL rows with `never_touched`. + #[cfg(test)] fn decode(&self, scale: f64) -> [f64; 64] { - let mut out = [0.0f64; 64]; - for (i, o) in out.iter_mut().enumerate() { - let m = f64::from(self.magnitudes[i]) * scale; - *o = if (self.signs >> i) & 1 == 1 { -m } else { m }; - } - nan_where_clear_f64(&[self.presence], &mut out); - out + std::array::from_fn(|i| match self.value(i) { + Some(value) => f64::from(value) * scale, + None => f64::NAN, + }) } } @@ -236,7 +211,7 @@ fn main() { let b: [Option; 64] = std::array::from_fn(|_| Some(-255)); let (ba, bb) = (Sm9Block::from_values(&a), Sm9Block::from_values(&b)); let (dot, n) = ba.masked_dot(&bb, u64::MAX); - let nulls = ba.decode(1.0 / 255.0).iter().filter(|x| x.is_nan()).count(); + let nulls = (0..64).filter(|&i| ba.never_touched(i)).count(); let first = ba.value(1).map_or(0, i16::from); println!(); println!("masked dot: dot {dot}, n {n}, NULL rows {nulls} (decode NaN), lane 1 = {first}"); @@ -245,7 +220,6 @@ fn main() { #[cfg(test)] mod tests { use super::*; - use ndarray::simd::presence_nan_mismatches_f64; /// Every (value or NULL) x (value or NULL) pair: the masked product /// equals the decoded product, and NULL contributes nothing to dot or n. @@ -274,23 +248,24 @@ mod tests { assert_eq!(checked, 512 * 512); } - /// Zero and NULL rows are stored as +0, so no row has two encodings, and - /// the payload under a clear presence bit never reaches the result. #[test] - fn null_and_zero_store_canonical_plus_zero_and_payload_is_ignored() { - let mut v = [Some(7i16); 64]; - v[3] = None; - v[4] = Some(0); - let mut b = Sm9Block::from_values(&v); - assert_eq!((b.magnitudes[3], (b.signs >> 3) & 1), (0, 0)); - assert_eq!((b.magnitudes[4], (b.signs >> 4) & 1), (0, 0)); + fn never_touched_negative_zero_positive_zero_and_negative_values() { + let mut values = [Some(7i16); 64]; + values[3] = None; + values[4] = Some(0); + values[5] = Some(-255); + let block = Sm9Block::from_values(&values); + assert_eq!((block.presence >> 3) & 1, 0); + assert_eq!((block.presence >> 4) & 1, 1); + assert_eq!((block.presence >> 5) & 1, 0); + assert!(block.never_touched(3)); + assert!(!block.never_touched(4)); + assert!(!block.never_touched(5)); + assert_eq!(block.value(3), None); + assert_eq!(block.value(4), Some(0)); + assert_eq!(block.value(5), Some(-255)); let ones = Sm9Block::from_values(&[Some(1i16); 64]); - let before = b.masked_dot(&ones, u64::MAX); - // Garbage under the clear bit changes nothing. - b.magnitudes[3] = 255; - b.signs |= 1 << 3; - assert_eq!(b.masked_dot(&ones, u64::MAX), before); - assert_eq!(before, (7 * 62, 63)); + assert_eq!(block.masked_dot(&ones, u64::MAX), (7 * 61 - 255, 63)); } /// Random blocks with NULLs and a filter, against a wide decoded oracle. @@ -331,7 +306,7 @@ mod tests { assert!(nulls_skipped > 10_000, "only {nulls_skipped} NULL rows skipped"); } - /// Decoding gives a float lane on which the NaN/presence bijection holds. + /// Decoding returns NaN precisely for P=0,M=0, not every P=0 row. #[test] fn decode_satisfies_the_float_bijection() { let mut v = [Some(-200i16); 64]; @@ -340,7 +315,9 @@ mod tests { } let b = Sm9Block::from_values(&v); let lane = b.decode(1.0 / 255.0); - assert_eq!(presence_nan_mismatches_f64(&lane, &[b.presence]), 0); + for (i, x) in lane.iter().enumerate() { + assert_eq!(x.is_nan(), b.never_touched(i), "row {i}"); + } assert_eq!(lane.iter().filter(|x| x.is_nan()).count(), 22); } diff --git a/src/hpc/codec9.rs b/src/hpc/codec9.rs new file mode 100644 index 00000000..1015cfc4 --- /dev/null +++ b/src/hpc/codec9.rs @@ -0,0 +1,247 @@ +//! Canonical DR512 numerical Codec9, reusing Codec8's compile-time calibration +//! and nearest-rounding types without borrowing Codec8's storage or NaN policy. +//! +//! One u8 magnitude plus ONE epistemic presence/polarity bit per lane: +//! +//! | P | Magnitude | Meaning | +//! |---|-----------|---------| +//! | 0 | 0 | -0 = NaN = epistemically NEVER TOUCHED | +//! | 0 | 1..=255 | negative -1..=-255 | +//! | 1 | 0 | valid +0 | +//! | 1 | 1..=255 | positive +1..=+255 | +//! +//! 512 codewords, 511 numeric values, 1 never-touched sentinel. A value +//! quantized from -0.4 to zero **must** emit +0, never the -0 sentinel. +//! A WHERE(P=0 AND magnitude=0) is sufficient; do not allocate a second mask. +//! Epistemic presence is NOT a source-write or alpha-claim authorization. +//! +//! Storage, polarity, Sym and NaN policy are intentionally NOT configurable. +//! An Asym mode, a separate sign/validity plane, or an optional NaN policy +//! would violate this contract. The only compile-time knobs are the existing +//! nearest rounding mode and scale calibration. +//! +//! ``` +//! use ndarray::hpc::codec8::{Calibration, TiesEven}; +//! use ndarray::hpc::codec9::{Codec9, EpiLane9}; +//! struct Unit; +//! impl Calibration for Unit { +//! const SCALE: f32 = 1.0; +//! const ZERO_POINT: i32 = 0; +//! } +//! type Q = Codec9; +//! assert_eq!(Q::encode_touched(-0.4).unwrap().codeword(), 0x100); +//! assert_eq!(EpiLane9::NEVER_TOUCHED.codeword(), 0); +//! ``` + +use core::marker::PhantomData; + +use super::codec8::{Calibration, Rounding, TiesAway, TiesEven}; + +/// Restricts DR512 quantization to nearest rounding, never Floor. +/// Floor would turn -0.4 into -1 rather than the required +0. +pub trait NearestRounding9: Rounding {} + +impl NearestRounding9 for TiesEven {} +impl NearestRounding9 for TiesAway {} + +/// One of the canonical 512 DR512 codewords. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct EpiLane9 { + magnitude: u8, + positive: bool, +} + +impl EpiLane9 { + /// The unique -0 codeword, meaning epistemically never touched. + pub const NEVER_TOUCHED: Self = Self { + magnitude: 0, + positive: false, + }; + + /// Returns the canonical codeword, with presence in bit 8. + #[must_use] + pub const fn codeword(self) -> u16 { + (self.magnitude as u16) | ((self.positive as u16) << 8) + } + + /// Reads one 9-bit codeword, refusing values beyond 511. + #[must_use] + pub const fn from_codeword(code: u16) -> Option { + if code > 511 { + None + } else { + Some(Self { + magnitude: code as u8, + positive: code & 0x100 != 0, + }) + } + } + + /// Evaluates WHERE(P=0 AND M=0), without creating another bitmap. + #[must_use] + pub const fn never_touched(self) -> bool { + !self.positive && self.magnitude == 0 + } + + /// Returns a signed numeric value, or None only for untouched -0. + #[must_use] + pub const fn value(self) -> Option { + if self.never_touched() { + None + } else if self.positive { + Some(self.magnitude as i16) + } else { + Some(-(self.magnitude as i16)) + } + } + + /// Returns the stored unsigned magnitude. + #[must_use] + pub const fn magnitude(self) -> u8 { + self.magnitude + } + + /// Returns the canonical epistemic presence/polarity bit. + #[must_use] + pub const fn presence(self) -> bool { + self.positive + } +} + +/// A finite-value-only encoder never invents "never touched". +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Codec9Error { + /// An explicitly touched NaN or infinity is not a numeric value. + NonFinite, +} + +/// Compile-time-calibrated canonical DR512 numeric codec. +/// +/// Reuses Codec8's `Rounding` and `Calibration` traits. Neither the +/// bit meaning nor storage/NaN policy can be selected or adapted. +/// The calibration must have positive finite SCALE and ZERO_POINT=0. +/// +/// ```compile_fail +/// use ndarray::hpc::codec8::{Calibration, Floor}; +/// use ndarray::hpc::codec9::Codec9; +/// struct Unit; +/// impl Calibration for Unit { +/// const SCALE: f32 = 1.0; +/// const ZERO_POINT: i32 = 0; +/// } +/// let _ = Codec9::::encode_touched(-0.4); +/// ``` +pub struct Codec9(PhantomData<(R, C)>); + +impl Codec9 { + #[inline(always)] + fn assert_calibration() { + const { + assert!(C::SCALE > 0.0 && C::SCALE < f32::INFINITY, "Codec9 scale must be finite and positive"); + assert!(C::ZERO_POINT == 0, "Codec9 requires Sym with ZERO_POINT=0"); + } + } + + /// Encodes a TOUCHED finite value. Any rounded zero is canonical +0, + /// regardless of input sign; -0/NaN is never manufactured by rounding. + /// Finite out-of-range values saturate at magnitude 255. + pub fn encode_touched(value: f32) -> Result { + Self::assert_calibration(); + if !value.is_finite() { + return Err(Codec9Error::NonFinite); + } + let quantized = R::round(value / C::SCALE); + let magnitude = quantized.abs().clamp(0.0, 255.0) as u8; + Ok(EpiLane9 { + magnitude, + positive: quantized >= 0.0 || magnitude == 0, + }) + } + + /// Decodes -0 into a floating NaN and all other codes using calibration. + /// This is an explicit numeric view, not a permission or NULL decision. + #[must_use] + pub fn decode(lane: EpiLane9) -> f32 { + Self::assert_calibration(); + match lane.value() { + None => f32::NAN, + Some(v) => f32::from(v) * C::SCALE, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + struct Unit; + impl Calibration for Unit { + const SCALE: f32 = 1.0; + const ZERO_POINT: i32 = 0; + } + type Even = Codec9; + type Away = Codec9; + + #[test] + fn all_512_codewords_are_unique_and_absence_is_exactly_zero_and_zero() { + let mut absent = 0usize; + for code in 0..512u16 { + let lane = EpiLane9::from_codeword(code).unwrap(); + assert_eq!(lane.codeword(), code); + if lane.never_touched() { + absent += 1; + assert_eq!(code, 0); + assert!(Even::decode(lane).is_nan()); + } else { + let expected = if code < 256 { + -(code as i16) + } else { + (code - 256) as i16 + }; + assert_eq!(lane.value(), Some(expected)); + assert_eq!(Even::decode(lane), expected as f32); + } + } + assert_eq!(absent, 1); + assert_eq!(EpiLane9::from_codeword(512), None); + } + + #[test] + fn negative_ceiling_zero_never_fabricates_epistemic_absence() { + for v in [-0.0, -0.00001, -0.4, -0.49999, -0.5, 0.0, 0.4] { + let code = Even::encode_touched(v).unwrap(); + assert_eq!(code.codeword(), 0x100, "touched {v} must be valid +0"); + assert!(!code.never_touched()); + } + assert_eq!(Away::encode_touched(-0.4).unwrap().codeword(), 0x100); + assert_eq!(Away::encode_touched(-0.5).unwrap().codeword(), 0x001); + assert_eq!(Even::encode_touched(-0.50001).unwrap().codeword(), 0x001); + assert_eq!(Even::encode_touched(-1.0).unwrap().codeword(), 0x001); + assert_eq!(Even::encode_touched(0.0).unwrap().codeword(), 0x100); + assert_eq!(Even::encode_touched(255.0).unwrap().codeword(), 0x1ff); + assert_eq!(Even::encode_touched(-255.0).unwrap().codeword(), 0x0ff); + assert_eq!(Even::encode_touched(1000.0).unwrap().codeword(), 0x1ff); + assert_eq!(Even::encode_touched(-1000.0).unwrap().codeword(), 0x0ff); + } + + #[test] + fn only_explicitly_unvisited_lanes_get_the_sentinel() { + assert_eq!(EpiLane9::NEVER_TOUCHED.codeword(), 0); + for value in -255..=255 { + let encoded = Even::encode_touched(value as f32).unwrap(); + assert!(!encoded.never_touched()); + assert_eq!(encoded.value(), Some(value)); + } + for invalid in [f32::NAN, f32::INFINITY, f32::NEG_INFINITY] { + assert_eq!(Even::encode_touched(invalid), Err(Codec9Error::NonFinite)); + } + } + + #[test] + fn decode_and_reencode_touched_values_preserve_the_entire_numeric_domain() { + for code in 1..512u16 { + let lane = EpiLane9::from_codeword(code).unwrap(); + assert_eq!(Even::encode_touched(Even::decode(lane)).unwrap(), lane); + } + } +} diff --git a/src/hpc/mod.rs b/src/hpc/mod.rs index 1fe2ee9c..ebe3fc38 100644 --- a/src/hpc/mod.rs +++ b/src/hpc/mod.rs @@ -47,6 +47,8 @@ pub mod cogrecord; pub mod graph; pub mod quantized; pub mod codec8; +/// Canonical DR512 numerical codec, with compile-time rounding and calibration. +pub mod codec9; pub mod lapack; pub mod fft; pub mod vml; diff --git a/src/simd_stride.rs b/src/simd_stride.rs index d54c1027..ecf7b15a 100644 --- a/src/simd_stride.rs +++ b/src/simd_stride.rs @@ -55,85 +55,48 @@ pub fn lut_u8(values: &[u8], table: &[u8; 256], out: &mut [u8]) { } // ──────────────────────────────────────────────────────────────────────── -// lut9: an odd-function table over a sign-magnitude 9-bit (SM9) lane. -// -// A sign-magnitude 9-bit lane stores each value as a `u8` magnitude plus one -// bit in a `u64` sign plane per 64 rows; whether the row holds a value at all -// is a separate presence plane, and the writer stores `+0` (magnitude 0, sign -// clear) for zero and for missing rows. For an odd function -// (`f(-x) = -f(x)`, e.g. tanh, sin, x³, sign-preserving clamps) one table -// over the 256 magnitudes covers both signs: the magnitude is looked up and -// the sign plane passes through. The presence plane is not an input — a -// missing row is magnitude 0, `f(0) = 0`, so it stays `+0`. +// Canonical DR512: P=0,M=0 is -0/NaN/NEVER TOUCHED. +// P=0,M>0 is negative; P=1,M=0 is valid +0; P=1,M>0 is positive. +// ONE presence/polarity plane. No second sign or missing-value plane. +// Negative nonzero that rounds to zero becomes valid +0, never NaN. // ──────────────────────────────────────────────────────────────────────── -/// Applies an odd function, given as a 256-entry magnitude table, to a -/// sign-magnitude lane. +/// Odd-function LUT for canonical DR512 epistemic presence × magnitude. /// -/// For every row `i < magnitudes.len()`: -/// -/// * `out_magnitudes[i] = table[magnitudes[i]]`; -/// * the output sign bit is the input sign bit, cleared where a nonzero -/// input gives a zero result. A negative value whose result rounds to -/// zero is stored as `+0`, never `-0`. -/// * an input `-0` (magnitude 0, sign set) comes out as `-0`. A writer -/// that only stores `+0` never produces it, so under the plain numeric -/// reading this changes nothing. A class whose reading gives `-0` a -/// meaning of its own, such as "observed, value NaN" beside a clear -/// presence bit for "not observed", keeps that meaning through the -/// function, the way IEEE arithmetic carries NaN through. Clearing it -/// would turn that reading's NaN into the value 0. -/// -/// Rows past `magnitudes.len()` are not written in `out_magnitudes`. In -/// `out_signs` the bits past `magnitudes.len()` and any surplus words are -/// written as `0`. The presence plane is not read or written: because -/// `table[0]` must be `0`, a missing row (stored as `+0`) stays `+0`. -/// -/// Facet stride: every 16 magnitudes are one 128-bit value whose positions -/// `{0}..{15}` are read where they sit, and each position addresses the table -/// directly. No byte is shuffled. The sign plane is 64 rows per -/// [`crate::simd::U8x64::cmpeq_mask`]. The final partial chunk is padded with -/// magnitude 0 and goes through the same path, with the padding lanes masked -/// off the sign plane. +/// On entry, P=0,M=0 denotes never touched/NaN and stays unchanged. +/// For a touched numeric input, a rounded output of zero is always valid +0 +/// (P=1,M=0), including input -0.4 quantized to zero. Never infer +/// authorization or alpha-claim from these bits. /// /// # Panics -/// -/// * if `table[0] != 0`, since an odd function maps 0 to 0; -/// * if `out_magnitudes.len() < magnitudes.len()`; -/// * if `signs.len()` or `out_signs.len()` is less than -/// `magnitudes.len().div_ceil(64)`. +/// If table[0] != 0 or input/output slices are too short. /// /// # Examples -/// /// ``` /// use ndarray::simd::lut9; -/// -/// // f(m) = m / 2, an odd function on the signed value. -/// let mut table = [0u8; 256]; -/// for (m, t) in table.iter_mut().enumerate() { -/// *t = (m / 2) as u8; -/// } -/// // Values: +10, -10, -1, +0 -/// let magnitudes = [10u8, 10, 1, 0]; -/// let signs = [0b0110u64]; -/// let mut out = [0u8; 4]; -/// let mut out_signs = [0u64; 1]; -/// lut9(&magnitudes, &signs, &table, &mut out, &mut out_signs); -/// assert_eq!(out, [5, 5, 0, 0]); -/// // -1 / 2 rounds to zero and is stored as +0: only row 1 keeps its sign. -/// assert_eq!(out_signs[0], 0b0010); +/// let table: [u8; 256] = core::array::from_fn(|m| (m / 2) as u8); +/// let magnitudes = [10u8, 10, 1, 0]; // +10,-10,-1,+0 +/// let presences = [0b1001u64]; // bit 1 means positive +/// let mut output = [0u8; 4]; +/// let mut out_presences = [0u64; 1]; +/// lut9(&magnitudes, &presences, &table, &mut output, &mut out_presences); +/// assert_eq!(output, [5, 5, 0, 0]); +/// assert_eq!(out_presences[0], 0b1101); // -1 rounded to valid +0 /// ``` -pub fn lut9(magnitudes: &[u8], signs: &[u64], table: &[u8; 256], out_magnitudes: &mut [u8], out_signs: &mut [u64]) { +pub fn lut9( + magnitudes: &[u8], presences: &[u64], table: &[u8; 256], out_magnitudes: &mut [u8], out_presences: &mut [u64], +) { use crate::simd::U8x64; let n = magnitudes.len(); let words = n.div_ceil(64); assert_eq!(table[0], 0, "lut9: an odd function maps 0 to 0, so table[0] must be 0"); assert!(out_magnitudes.len() >= n, "lut9: out_magnitudes holds {} rows, need {n}", out_magnitudes.len()); - assert!(signs.len() >= words, "lut9: signs holds {} words, need {words}", signs.len()); - assert!(out_signs.len() >= words, "lut9: out_signs holds {} words, need {words}", out_signs.len()); - - // FACET STRIDE: the magnitude lookup reads positions where they sit - // ([`lut_u8`]); no byte is shuffled. + assert!(presences.len() >= words, "lut9: presences holds {} words, need {words}", presences.len()); + assert!( + out_presences.len() >= words, + "lut9: out_presences holds {} words, need {words}", + out_presences.len() + ); let fill = |chunk: &[u8; 64], out: &mut [u8; 64]| lut_u8(chunk, table, out); let zero = U8x64::splat(0); let (in_chunks, tail) = magnitudes.as_chunks::<64>(); @@ -141,8 +104,9 @@ pub fn lut9(magnitudes: &[u8], signs: &[u64], table: &[u8; 256], out_magnitudes: for (w, (chunk, out)) in in_chunks.iter().zip(out_chunks).enumerate() { fill(chunk, out); let (m, r) = (U8x64::from_array(*chunk), U8x64::from_array(*out)); - // Keep the sign where the result is nonzero or the input is `-0`. - out_signs[w] = signs[w] & (!r.cmpeq_mask(zero) | m.cmpeq_mask(zero)); + // Only a nonzero input that rounded to zero is canonicalized. + // Pre-existing P=0,M=0 (never touched) must remain P=0. + out_presences[w] = presences[w] | (r.cmpeq_mask(zero) & !m.cmpeq_mask(zero)); } let mut w = in_chunks.len(); if !tail.is_empty() { @@ -152,14 +116,12 @@ pub fn lut9(magnitudes: &[u8], signs: &[u64], table: &[u8; 256], out_magnitudes: fill(&padded, &mut looked_up); out_tail.copy_from_slice(&looked_up[..tail.len()]); let (m, r) = (U8x64::from_array(padded), U8x64::from_array(looked_up)); - // Padding lanes look like input `-0` (magnitude 0), so the keep-`-0` - // rule would pass their junk sign bits; mask them off. - out_signs[w] = - signs[w] & (!r.cmpeq_mask(zero) | m.cmpeq_mask(zero)) & crate::simd_masking_ops::live64(tail.len()); + out_presences[w] = + (presences[w] | (r.cmpeq_mask(zero) & !m.cmpeq_mask(zero))) & crate::simd_masking_ops::live64(tail.len()); w += 1; } - for s in &mut out_signs[w..] { - *s = 0; + for p in &mut out_presences[w..] { + *p = 0; } } @@ -169,16 +131,16 @@ mod tests { // ── lut9 (SM9 odd table) ────────────────────────────────────────────────────────── - fn lut9_reference(mags: &[u8], signs: &[u64], table: &[u8; 256]) -> (Vec, Vec) { + fn lut9_reference(mags: &[u8], presences: &[u64], table: &[u8; 256]) -> (Vec, Vec) { let out: Vec = mags.iter().map(|&m| table[m as usize]).collect(); - let mut out_signs = vec![0u64; mags.len().div_ceil(64)]; + let mut out_presences = vec![0u64; mags.len().div_ceil(64)]; for (i, &o) in out.iter().enumerate() { - let neg = (signs[i / 64] >> (i % 64)) & 1 == 1; - if neg && (o != 0 || mags[i] == 0) { - out_signs[i / 64] |= 1 << (i % 64); + let positive = (presences[i / 64] >> (i % 64)) & 1 == 1; + if positive || (o == 0 && mags[i] != 0) { + out_presences[i / 64] |= 1 << (i % 64); } } - (out, out_signs) + (out, out_presences) } fn lut9_xorshift(s: &mut u64) -> u64 { @@ -206,40 +168,35 @@ mod tests { // Every magnitude appears at every lane position across shifts. let mags: Vec = (0..n).map(|i| ((i * 37 + shift) % 256) as u8).collect(); let words = n.div_ceil(64); - let mut signs: Vec = (0..words).map(|_| lut9_xorshift(&mut s)).collect(); + let mut presences: Vec = (0..words).map(|_| lut9_xorshift(&mut s)).collect(); // Junk sign bits past the last row must not leak into the output. - if let Some(last) = signs.last_mut() { + if let Some(last) = presences.last_mut() { *last |= !crate::simd_masking_ops::live64(n - (words - 1) * 64); } - let (want, want_signs) = lut9_reference(&mags, &signs, &table); + let (want, want_presences) = lut9_reference(&mags, &presences, &table); let mut out = vec![0xAAu8; n + 5]; - let mut out_signs = vec![u64::MAX; words + 2]; - lut9(&mags, &signs, &table, &mut out, &mut out_signs); + let mut out_presences = vec![u64::MAX; words + 2]; + lut9(&mags, &presences, &table, &mut out, &mut out_presences); assert_eq!(&out[..n], &want[..], "n={n} shift={shift}"); assert!(out[n..].iter().all(|&b| b == 0xAA), "rows past the end written, n={n}"); - assert_eq!(&out_signs[..words], &want_signs[..], "n={n} shift={shift}"); - assert!(out_signs[words..].iter().all(|&w| w == 0), "surplus words, n={n}"); + assert_eq!(&out_presences[..words], &want_presences[..], "n={n} shift={shift}"); + assert!(out_presences[words..].iter().all(|&w| w == 0), "surplus words, n={n}"); } } } - /// FAILS IF: a negative value whose result rounds to zero keeps its sign - /// (a stored `-0`), or an input `-0` loses its sign. Silence half: a - /// nonzero negative result keeps its sign, and `+0` stays `+0`. + /// Negative rounding to zero must not create a fictitious -0/NaN. #[test] - fn lut9_rounds_to_plus_zero_and_keeps_an_input_minus_zero() { - let mut table = [0u8; 256]; - for (m, t) in table.iter_mut().enumerate() { - *t = (m / 4) as u8; - } - // -0, -3 (rounds to 0), -4 (-> -1), +3, +200, +0 + fn lut9_never_fabricates_epistemic_absence() { + let table: [u8; 256] = core::array::from_fn(|m| (m / 4) as u8); + // -0 untouched, -3 touched, -4 touched, +3, +200, +0. let mags = [0u8, 3, 4, 3, 200, 0]; - let signs = [0b000111u64]; + let presences = [0b111000u64]; let mut out = [0u8; 6]; - let mut out_signs = [u64::MAX; 1]; - lut9(&mags, &signs, &table, &mut out, &mut out_signs); + let mut out_presences = [u64::MAX; 1]; + lut9(&mags, &presences, &table, &mut out, &mut out_presences); assert_eq!(out, [0, 0, 1, 0, 50, 0]); - assert_eq!(out_signs[0], 0b000101); + assert_eq!(out_presences[0], 0b111010); } #[test] @@ -251,8 +208,8 @@ mod tests { } #[test] - #[should_panic(expected = "signs holds 1 words, need 2")] - fn lut9_refuses_a_short_sign_plane() { + #[should_panic(expected = "presences holds 1 words, need 2")] + fn lut9_refuses_a_short_presence_plane() { let table = [0u8; 256]; lut9(&[0; 65], &[0], &table, &mut [0; 65], &mut [0; 2]); } diff --git a/tests/dr512_epistemic_contract.rs b/tests/dr512_epistemic_contract.rs new file mode 100644 index 00000000..8044d2d6 --- /dev/null +++ b/tests/dr512_epistemic_contract.rs @@ -0,0 +1,115 @@ +#![cfg(feature = "std")] +//! Independent DR512 P×M truth-table and distance-oracle CI tripwires. +use ndarray::hpc::codec8::{Calibration, TiesAway, TiesEven}; +use ndarray::hpc::codec9::{Codec9, EpiLane9}; +use ndarray::simd::lut9; + +struct Unit; +impl Calibration for Unit { + const SCALE: f32 = 1.0; + const ZERO_POINT: i32 = 0; +} + +#[test] +fn all_512_codes_agree_in_codec9_and_lut9() { + let identity: [u8; 256] = core::array::from_fn(|i| i as u8); + let mags: Vec = (0..512).map(|i| i as u8).collect(); + let presence: Vec = (0..8).map(|w| if w < 4 { 0 } else { u64::MAX }).collect(); + let mut out = vec![0u8; 512]; + let mut out_p = vec![u64::MAX; 8]; + lut9(&mags, &presence, &identity, &mut out, &mut out_p); + assert_eq!(out, mags); + assert_eq!(out_p, presence); + + let mut absent = 0; + for code in 0..512u16 { + let lane = EpiLane9::from_codeword(code).expect("9-bit code"); + let i = usize::from(code); + assert_eq!(lane.codeword(), code); + assert_eq!(lane.magnitude(), out[i]); + assert_eq!(lane.presence(), (out_p[i / 64] >> (i % 64)) & 1 == 1); + let expected = match code { + 0 => None, + 1..=255 => Some(-(code as i16)), + _ => Some((code - 256) as i16), + }; + assert_eq!(lane.value(), expected); + absent += usize::from(lane.never_touched()); + } + assert_eq!(absent, 1); +} + +#[test] +fn where_zero_and_zero_needs_no_materialized_8k_mask() { + let n = 65_536usize; + let magnitudes: Vec = (0..n).map(|i| if i % 4 < 2 { 0 } else { 9 }).collect(); + let presence = vec![0xAAAA_AAAA_AAAA_AAAAu64; n / 64]; + let absent = (0..n) + .filter(|&i| (presence[i / 64] >> (i % 64)) & 1 == 0 && magnitudes[i] == 0) + .count(); + assert_eq!(absent, 16_384); + // A negative nonzero lane must survive the same WHERE. + assert_eq!(magnitudes[2], 9); + assert_eq!((presence[0] >> 2) & 1, 0); +} + +#[test] +fn touched_ceiling_to_valid_plus_zero_not_never_touched() { + type Even = Codec9; + type Away = Codec9; + for x in [-0.0, -0.00001, -0.4, -0.49999, -0.5, 0.0, 0.4] { + assert_eq!(Even::encode_touched(x).unwrap().codeword(), 0x100, "{x}"); + } + assert_eq!(Away::encode_touched(-0.4).unwrap().codeword(), 0x100); + assert_eq!(Away::encode_touched(-0.5).unwrap().codeword(), 1); + assert_eq!(Even::encode_touched(-0.50001).unwrap().codeword(), 1); + assert_eq!(EpiLane9::NEVER_TOUCHED.codeword(), 0); +} + +/// The distance for equal numeric scale is abs(m1-m2) for equal polarity, +/// m1+m2 otherwise. Its maximum is 510: never narrow the result to u8. +#[test] +fn all_numeric_dr512_pairs_match_independent_i16_distance() { + let mut checked = 0; + let mut max_distance = 0; + for ca in 1..512u16 { + for cb in 1..512u16 { + let a = EpiLane9::from_codeword(ca).unwrap(); + let b = EpiLane9::from_codeword(cb).unwrap(); + let derived = if a.presence() == b.presence() { + u16::from(a.magnitude().abs_diff(b.magnitude())) + } else { + u16::from(a.magnitude()) + u16::from(b.magnitude()) + }; + let independent = (i32::from(a.value().unwrap()) - i32::from(b.value().unwrap())).unsigned_abs() as u16; + assert_eq!(derived, independent, "codes {ca} {cb}"); + max_distance = max_distance.max(derived); + checked += 1; + } + } + assert_eq!(checked, 511 * 511); + assert_eq!(max_distance, 510); +} + +/// A hypothetical [-a,+b] calibrated range changes per-side scales, +/// but never changes DR512's P×M code interpretation. Mathematical oracle +/// only: NOT an admitted new Codec9 mode or a performance measurement. +#[test] +fn asymmetric_calibrated_distance_is_not_a_uniform_u8_lookup() { + let (a_scale, b_scale) = (3.0f64 / 255.0, 7.0f64 / 255.0); + for ca in 1..512u16 { + let x = EpiLane9::from_codeword(ca).unwrap(); + let signed = |v: EpiLane9| -> f64 { f64::from(v.magnitude()) * if v.presence() { b_scale } else { -a_scale } }; + for cb in 1..512u16 { + let y = EpiLane9::from_codeword(cb).unwrap(); + let folded = if x.presence() == y.presence() { + f64::from(x.magnitude().abs_diff(y.magnitude())) * if x.presence() { b_scale } else { a_scale } + } else { + let p = if x.presence() { x } else { y }; + let n = if x.presence() { y } else { x }; + f64::from(p.magnitude()) * b_scale + f64::from(n.magnitude()) * a_scale + }; + assert!((folded - (signed(x) - signed(y)).abs()).abs() < 1e-12); + } + } +}