Entropy-ladder foundation: reliability suite + edge-codec flavors + entropy_ladder#218
Merged
Merged
Conversation
…ate/invalidate) Two reusable hpc modules + a comparison harness so edge encodings can be measured and selected on evidence rather than guessed: - hpc::reliability — Pearson r, Spearman ρ, Cronbach α, ICC(2,1) absolute agreement, plus a bundled FidelityReport. General (&[f64]→f64) primitives; the measurement layer for validate/invalidate work (consumable by jc/pillar). - hpc::edge_codec — three commensurable quantizers sharing the canonical 16-byte edge block by INTERPRETATION, not layout: CoarseOnly (1 B palette index), CoarseResidue (1 + D/2, value-slab signed-4-bit residue), Pq32x4 (16 B = 32 subquantizers × 4-bit, the turbovec PQ model). Compact seeded k-means; nibble packing lo=code[2t]/hi=code[2t+1]. - examples/edge_codec_compare — encodes every flavor across blob/continuous regimes, reports the full reliability suite on pairwise-distance preservation, and checks the AMX matmul_i8_to_i32 assignment matches scalar. Measured: CoarseResidue dominates agreement (ICC 0.97-0.99, ρ 0.98, α 0.99); Pq32x4 preserves rank (ρ 0.60-0.67) but not absolute distance (ICC 0.11-0.29); CoarseOnly collapses on continuous data (ICC 0.003). AMX assign 100% vs scalar. 16 unit + 9 doctests; clippy -D warnings clean. Also silences a pre-existing excessive_precision lint in golden_helix_probe. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
… + Pearl-2³ SPO
The unifying foundation for the SPO rung ladder: a fact's position on the
cognitive ladder IS an entropy level. Reads a CausalEdge64's EXISTING fields
(3×palette-256 S/P/O indices + Pearl 2³ mask + packed NARS f/c) — NO
re-quantization (the operator's "the 3×palette256 SPO inside CausalEdge64 is
exact enough").
- nars_entropy(f,c) = 1 − c·|2f−1| (Staunen high ↔ Wisdom low crystalline).
- EntropyRung {Syntax, Semantics, Pragmatics} banded high→low entropy.
- Quadrant {Staunen, Confusion, Boredom, Wisdom} from entropy×energy (board canon;
energy supplied by the caller, e.g. MailboxSoA.energy).
- PEARL_SUBSETS (2³) + decompose_spo → SpoLadderPoint{entropy, rung, active,
basin_key (HHTL-routable, inactive components zeroed), class}.
- entropy_class(h) → 2-bit reliability class for CausalEdge64 v2 spare bits [63:61].
Validated as a reliability proxy: ρ(entropy, empirical prediction accuracy)
= −0.78 over a synthetic NARS population (test gates ρ < −0.5), grounded via
hpc::reliability. examples/entropy_ladder_probe prints rung/quadrant partition
+ Pearl-2³ SPO decomposition.
7 unit + 5 doctests; clippy -D warnings clean.
https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
Contributor
|
Warning Review limit reached
More reviews will be available in 14 minutes and 14 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThree new public HPC submodules are added: ChangesHPC Submodule Expansion and Validation
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Poem
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83be7c35f6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/hpc/edge_codec.rs`:
- Around line 265-270: The reconstruct method (and the similar decode function
around line 338-343) trust unvalidated lengths from caller-provided encoded
buffers and can panic when unpack_nibbles_signed attempts to access data beyond
the buffer bounds. Add explicit length validation before calling
unpack_nibbles_signed to verify that code.residue has at least dim/2 bytes
(since nibbles are packed as 2 per byte). Either add these checks inline before
the function calls, or refactor the methods to return Result types that fail
gracefully with validation errors instead of panicking. Apply the same
validation fix at both affected locations: the reconstruct method at line
265-270 and the second decode method at lines 338-343.
- Around line 131-152: The public methods Codebook::assign, Codebook::centroid,
CoarseResidueCodec::encode, CoarseResidueCodec::reconstruct,
ProductQuantizer::encode, and ProductQuantizer::reconstruct are missing doc
comment examples. Add `///` doc comments with runnable examples above each of
these methods to comply with the coding guideline that all public APIs must have
doc comments with examples. Each example should demonstrate basic usage of the
method with appropriate input values and show the expected output or behavior.
In `@src/hpc/entropy_ladder.rs`:
- Around line 1-26: The `entropy_ladder.rs` module in src/hpc violates the
architectural boundary by embedding thinking layer semantics (NARS concepts,
cognition logic like Staunen/Wisdom/Semantics) and protocol interpretation
(CausalEdge64, decompose_spo) in what should be a hardware-primitive layer. Move
the module's semantics, documentation, and implementation logic (including lines
137-190) to the appropriate architectural layers: lance-graph for thinking-layer
concerns (entropy classification, Quadrant logic, NARS reliability) and
causal-edge for protocol concerns (SPO palette index interpretation,
CausalEdge64 field handling). Retain only low-level hardware-primitive
operations in src/hpc if any remain.
In `@src/hpc/reliability.rs`:
- Around line 246-249: The FidelityReport::compute method uses `.min()` to
silently truncate mismatched input arrays to the shorter length, but the API
documentation specifies equal-length samples are required. This silent
truncation can hide upstream bugs and skew fidelity measurements by dropping
tail errors. Either enforce that truth and estimate have equal lengths by adding
a validation check (panic or return an error) in FidelityReport::compute, or
update the API documentation to explicitly state that truncation occurs and
rename the method to reflect this behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a12805ac-26e5-4c53-b137-ed730d84458d
📒 Files selected for processing (8)
Cargo.tomlexamples/edge_codec_compare.rsexamples/entropy_ladder_probe.rsexamples/golden_helix_probe.rssrc/hpc/edge_codec.rssrc/hpc/entropy_ladder.rssrc/hpc/mod.rssrc/hpc/reliability.rs
Comment on lines
+1
to
+26
| //! Entropy ladder — the Staunen↔Wisdom coordinate that unifies NARS truth, the | ||
| //! 3×palette-256 SPO already inside a `CausalEdge64`, and the | ||
| //! syntax→semantics→pragmatics rungs. | ||
| //! | ||
| //! The organizing idea (operator framing, board canon): a fact's position on the | ||
| //! cognitive ladder IS an entropy level. | ||
| //! | ||
| //! * **Staunen** — high entropy — raw *stimulus*, not yet crystallized. Maps to | ||
| //! **syntax** (surface form: many ways to say it). | ||
| //! * **Wisdom** — low entropy — *crystalline* knowledge / settled fact. Maps to | ||
| //! **pragmatics** (one settled intent in context). | ||
| //! * **Semantics** sits between. | ||
| //! | ||
| //! Crucially this reads a `CausalEdge64`'s EXISTING fields — the three | ||
| //! palette-256 indices (`s_idx`/`p_idx`/`o_idx`), the Pearl 2³ causal mask, and | ||
| //! the packed NARS `(f, c)` — so there is NO re-quantization (the operator's | ||
| //! "the 3×palette256 SPO values inside CausalEdge64 are exact enough"). | ||
| //! | ||
| //! The two axes (board's entropy×energy quadrant): **entropy** here, **energy** | ||
| //! supplied by the caller (e.g. `MailboxSoA.energy`). Together they classify the | ||
| //! [`Quadrant`] (Staunen / Confusion / Boredom / Wisdom). | ||
| //! | ||
| //! Validation: [`nars_entropy`] is a *reliability proxy* — it anti-correlates | ||
| //! with an edge's empirical prediction accuracy (see the module tests and the | ||
| //! `entropy_ladder_probe` example), grounded via [`crate::hpc::reliability`]. | ||
|
|
Contributor
There was a problem hiding this comment.
Layering violation: thinking/protocol semantics are embedded in src/hpc.
nars_entropy ladder semantics plus decompose_spo/CausalEdge64 protocol interpretation are not hardware-primitive concerns. This couples ndarray’s HPC layer to cognition/protocol logic and breaks the declared module boundary.
As per coding guidelines, “src/hpc/**/*.rs: Follow the architecture rule: ndarray provides hardware primitives … lance-graph provides thinking layer … causal-edge provides protocol …”.
Also applies to: 137-190
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/hpc/entropy_ladder.rs` around lines 1 - 26, The `entropy_ladder.rs`
module in src/hpc violates the architectural boundary by embedding thinking
layer semantics (NARS concepts, cognition logic like Staunen/Wisdom/Semantics)
and protocol interpretation (CausalEdge64, decompose_spo) in what should be a
hardware-primitive layer. Move the module's semantics, documentation, and
implementation logic (including lines 137-190) to the appropriate architectural
layers: lance-graph for thinking-layer concerns (entropy classification,
Quadrant logic, NARS reliability) and causal-edge for protocol concerns (SPO
palette index interpretation, CausalEdge64 field handling). Retain only
low-level hardware-primitive operations in src/hpc if any remain.
Source: Coding guidelines
Owner
Author
There was a problem hiding this comment.
Acknowledged — operator decision: validate in ndarray::hpc now, migrate to the lance-graph thinking layer later.
Rationale for keeping it here during the validation phase: the module has no dependency on the thinking/protocol crates — it is pure functions on (u8, f64) + plain enums, and decompose_spo takes raw palette indices, never a CausalEdge64. It sits beside the pre-existing hpc::nars (NarsTruth), hpc::causal_diff (CausalEdge64), and hpc::styles (cognitive primitives) that already live in this layer.
The decision + planned migration are recorded in the module header (commit 3c6cd2b). Leaving this thread open as the migration tracker rather than resolving it.
Generated by Claude Code
Contributor
There was a problem hiding this comment.
@AdaWorldAPI — understood, and the rationale holds up: zero cross-crate coupling, pure (u8, f64) → scalar/enum functions, and decompose_spo operating on raw indices rather than a typed CausalEdge64 — that keeps the implementation genuinely self-contained within hpc. The migration note in the module header is a good anchor for the future move. Keeping this thread open as a tracker makes sense.
I'll record the operator decision for future reviews.
✏️ Learnings added
Learnt from: AdaWorldAPI
Repo: AdaWorldAPI/ndarray PR: 218
File: src/hpc/entropy_ladder.rs:1-35
Timestamp: 2026-06-14T11:06:51.445Z
Learning: In `AdaWorldAPI/ndarray`, the `src/hpc/entropy_ladder.rs` module intentionally lives in the HPC layer (alongside `hpc::nars`, `hpc::causal_diff`, `hpc::styles`) during a validation phase, by operator decision. The rationale: it has zero dependency on thinking/protocol crates — all functions are pure `(u8, f64)` → scalar/enum, and `decompose_spo` accepts raw palette indices rather than a typed `CausalEdge64`. A planned migration to the lance-graph thinking layer (or causal-edge for protocol concerns) is recorded in the module header. Do not re-flag this as a layering violation until the migration window is explicitly opened.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
… basin=phase Treats the syntax/semantics/episodic → angle/residue/phase reframing as a multitrait-multimethod hypothesis and tests it with the reliability suite (Spearman + ICC), rather than assuming it. Result over a synthetic population with independent latent factors: - convergent: syn↔angle ρ=1.000, sem↔residue ρ=1.000 - discriminant: off-diagonal ρ≈0.015 — angle ⊥ residue (genuinely orthogonal) - phase independent of content (ρ≈0); golden phase 32× better min-gap than random - RELIABILITY: angle ICC=0.998, phase=1.0, but residue ICC=0.471 (FAIL @ noise 0.18) The probe earns its keep by falsifying one cell: location-residue is a small-signal measure (deviation a−centroid, norm ~0–1.5) vs angle on full vectors (norm ~6), so √D L2 noise swamps it — empirical I-NOISE-FLOOR-JIRAK. Next: directional/per-dim residue + Belichtungsmesser σ-gate to recover ICC. Synthetic instrument-shape test only (orthogonality/reliability/separation); the linguistic mapping still needs a real-text (deepnsm/COCA) probe. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
…te on the instrument "Can we measure CLAM vs CAKES?" — yes. Runs real ndarray CLAM search (knn_repeated_rho, knn_dfs_sieve) vs an independent brute Hamming k-NN and puts both on the reliability instrument (recall / Spearman order / ICC distance / speedup). Measured (N=4000, 128-bit, 32 clusters, k=10, 300 queries): - exact by DISTANCE: both 1.000 (ρ=ICC=1.000 confirm identical returned distances) - accelerated: dfs-sieve 2.33×; repeated-ρ 0.74× (SLOWER than brute on tight Hamming clusters — its radius schedule overshoots) Two findings: (1) k-NN recall by INDEX undercounts on tie-heavy integer (Hamming) distances (idx-recall 0.79) — distance-recall is the correct exactness metric (1.000); (2) the instrument adjudicates between CAKES algorithms — dfs-sieve is the right one here, repeated-ρ is exact-but-mistuned. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
…amples & decode asserts - reliability: FidelityReport::compute returns a DEGENERATE report (zeros, rel_l2=1.0) on length mismatch instead of silently truncating to the shorter prefix (codex P2 + coderabbit Major) — consistent with the module's return-degenerate-not-panic convention; a dropped tail can't read as high fidelity. - edge_codec: added runnable /// examples to the 6 remaining public methods (Codebook::assign/centroid, CoarseResidueCodec::encode/reconstruct, ProductQuantizer::encode/reconstruct) per all-public-APIs-need-examples. - edge_codec: both reconstruct() decoders now assert the packed-code length (dim/2, m/2) so malformed input fails with a clear message instead of an out-of-bounds panic deep in unpack (coderabbit Major). edge_codec doctests 4→10; reliability 5; unit tests green; clippy -D warnings clean. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
…the instrument Tests the 16-byte spec's load-bearing claim (PLACE/HHTL ⊥ RESIDUE/HELIX) via a spherical decomposition of points around their cluster centroid, encoding the residue direction to the 256-sample golden-spiral hemisphere palette + a 1-bit sign. Measured: orthogonal (ρ(place,helix)=−0.001, ρ(radial,helix)=−0.013); adds info (centroid+radial+helix cuts reconstruction error 13.6× at 4.2° angular quant); golden separation 28× > random; reliable (ICC 0.88 under noise). All PASS. Finding: helix is a HEMISPHERE code (axis, ±v equivalent). A full SIGNED direction needs the palette index PLUS a 1-bit sign — without it reconstruction fails for half the sphere (fine_err ≈ coarse_err). The sign bit is the difference between adds-info FAIL and 13.6× PASS. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
…ay, migrate later) Operator decision on the PR #218 coderabbit layering thread: keep entropy_ladder in ndarray::hpc for the validation phase, migrate to the lance-graph thinking layer once the ladder stabilizes. Module header now states the rationale (no thinking/protocol crate dependency — pure fns on (u8,f64)+enums, decompose_spo takes raw indices not a CausalEdge64; sits beside existing hpc::nars and hpc::causal_diff) and the planned migration. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
Measures golden-spiral hemisphere quantization error vs palette size to settle the bit-depth question. 8/16-bit enumerated + measured; the equal-area law err≈c/√N (fit on 8-bit) predicts 16-bit within ~3% → validated → extrapolates 24/32/48-bit. Result (mean angular error): - 8-bit (256, 1B): 4.39° — routing / HHTL buckets only - 16-bit (65k, 2B): 0.283° — matches bf16 source precision (0.22°) - 24-bit (3B): 0.017° — below f16 (0.056°) ⇒ LOSSLESS for ≤f16 ← max fidelity / one place - 32-bit (4B): 0.001° — diminishing returns (below any ≤f16 source) - 48-bit (6B): 4e-6° — ≈ f32 floor; single-direction OVERKILL Confirms the original helix recommendation: 24-bit = max fidelity for ONE PLACE (a single direction); 48-bit (6B = CAM-PQ/splat budget) is for SPATIAL ATTENTION — an oriented anisotropic region (2 axes ≈ 2×24-bit, a Gaussian-splat Σ), not a point. 16-bit = bf16-economical, 8-bit = routing. (+1 sign bit for a full signed direction.) https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
…s the whole pyramid Tests whether a single 48-bit spatial-attention frame (principal 24-bit helix + 24-bit spatial gradient) earns its keep by reconstructing a coherent perturbation field over a Morton quadtree on-demand. Measured (256² finest, 8 levels): - amortization: 48 bits / 65536 cells = 0.0007 bits/cell — 32768× vs per-cell 24-bit - on-demand reconstruction exact at EVERY level (max 0.046°, the 24-bit quant floor; analytic, never materialized) - coherent-field fidelity 0.014° (quant floor); non-parametric jitter is the residual Two honest limits surfaced: (a) naive 2×2 mean-pool loses coherence at COARSE levels (7.7° at the root) because averaging rotating unit vectors isn't orientation-preserving — use on-demand eval (free) or an orientation-aware pool; (b) per-cell jitter needs its own per-cell code. ⇒ 48-bit earns its keep for COHERENT spatial-attention perturbation over a pyramid; it is a parametric field code, not a per-cell store. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
…eature under SD drift Tests "the coarse loss is a feature" claim: a self-calibrating floor (μ+3σ) rides the distribution so the 256 buckets stay well-exposed as σ drifts, rather than reconstructing. Three floors over a 4-phase drifting stream, on the real Cascade. Measured: - EWMA rolling floor TRACKS perfectly (ρ=1.000 vs true μ+3σ; reject stable ~0.1% across all phases, spread 0.05pp; bucket exposure 29–77%). - FIXED floor mis-exposes: 97.6% reject after an up-shift (recall collapse), 0% after a down-shift; reject-rate spread 42pp. FINDING (real bug surfaced): the shipped Cascade (global Welford + ShiftAlert) behaved IDENTICALLY to FIXED — ShiftAlert NEVER fired, because cumulative-Welford per-sample Δμ is always ≪ 2σ. The drift check is inert on a per-sample observe() feed; it would only fire on batch means. The EWMA (per-step) floor is what tracks. ⇒ coarse loss is usable for routing via a rolling floor; reconstruction fidelity is irrelevant — only the floor's calibration is. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/hpc/reliability.rs (1)
283-283:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
rel_l2can report perfect error on a wrong reconstruction whentruthis all zeros.At Line 283,
rel_l2is forced to0.0whenever‖truth‖≈0, even ifestimateis non-zero. That yields a false “no error” signal for a clearly bad reconstruction.Suggested fix
- let rel_l2 = if st > 1e-24 { se.sqrt() / st.sqrt() } else { 0.0 }; + let rel_l2 = if st > 1e-24 { + se.sqrt() / st.sqrt() + } else if se > 1e-24 { + 1.0 + } else { + 0.0 + };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/hpc/reliability.rs` at line 283, The `rel_l2` calculation at the line with the assignment to `rel_l2` incorrectly forces the result to 0.0 whenever the truth norm (st) is near zero, regardless of whether the estimate norm (se) is non-zero. This masks reconstruction errors when the true values are close to zero. Fix this by checking both se and st: only return 0.0 when both norms are near zero (indicating a correct near-zero reconstruction), otherwise compute the relative error appropriately or return a value that reflects the discrepancy between estimate and truth even when truth is near zero.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/morton_perturbation_probe.rs`:
- Line 154: The variable fine_pool_coherent is hard-coded to true instead of
being derived from the computed coherence metrics, causing the coherence check
to always pass regardless of actual measurements. Replace the hard-coded true
assignment with a condition that evaluates the computed worst and worst_overall
coherence metrics against the coherence threshold (based on the ≤0.01°
measurement criteria mentioned in the comment) to correctly determine whether
fine-scale coherence is actually acceptable.
In `@examples/rolling_floor_probe.rs`:
- Around line 157-163: The println! statements at lines 157–163 contain
hard-coded narrative text that asserts fixed outcomes (such as ρ=1.0 and
specific reject percentages) instead of reporting the actual computed metrics
from the probe run. Replace these hard-coded assertions with variable references
or computed values that reflect the actual measured results, so the probe
self-reports its true findings rather than pre-determined conclusions. For
example, reference computed variables holding values like the actual ρ value,
measured rejection rates, and drift check results instead of embedding literal
numbers and claims in the narrative strings.
---
Outside diff comments:
In `@src/hpc/reliability.rs`:
- Line 283: The `rel_l2` calculation at the line with the assignment to `rel_l2`
incorrectly forces the result to 0.0 whenever the truth norm (st) is near zero,
regardless of whether the estimate norm (se) is non-zero. This masks
reconstruction errors when the true values are close to zero. Fix this by
checking both se and st: only return 0.0 when both norms are near zero
(indicating a correct near-zero reconstruction), otherwise compute the relative
error appropriately or return a value that reflects the discrepancy between
estimate and truth even when truth is near zero.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c0c989d1-14c1-4b21-8b61-a26fe51ffb6b
📒 Files selected for processing (10)
Cargo.tomlexamples/cakes_grail_probe.rsexamples/helix_bitdepth_probe.rsexamples/helix_orthogonality_probe.rsexamples/instrument_mtmm_probe.rsexamples/morton_perturbation_probe.rsexamples/rolling_floor_probe.rssrc/hpc/edge_codec.rssrc/hpc/entropy_ladder.rssrc/hpc/reliability.rs
🚧 Files skipped from review as they are similar to previous changes (2)
- src/hpc/edge_codec.rs
- src/hpc/entropy_ladder.rs
| // ── verdict ── | ||
| let amortized = 48.0 / (n_fine as f64) < 0.01; | ||
| let on_demand_exact = on_demand_max.to_degrees() < 0.05; // at the 24-bit quant floor | ||
| let fine_pool_coherent = true; // levels 6–8 measured ≤0.01° above |
Contributor
There was a problem hiding this comment.
fine_pool_coherent is hard-coded, so this verdict can pass even when coherence fails.
At Line 154, the probe computes coherence metrics (worst, worst_overall) but ignores them and always marks fine-scale coherence as true.
Suggested fix
- let mut worst_overall = 0.0f64;
+ let mut worst_overall = 0.0f64;
+ let mut worst_fine = 0.0f64;
@@
worst_overall = worst_overall.max(worst);
+ if lvl >= levels - 2 {
+ worst_fine = worst_fine.max(worst);
+ }
@@
- let fine_pool_coherent = true; // levels 6–8 measured ≤0.01° above
+ let fine_pool_coherent = worst_fine.to_degrees() <= 0.01;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@examples/morton_perturbation_probe.rs` at line 154, The variable
fine_pool_coherent is hard-coded to true instead of being derived from the
computed coherence metrics, causing the coherence check to always pass
regardless of actual measurements. Replace the hard-coded true assignment with a
condition that evaluates the computed worst and worst_overall coherence metrics
against the coherence threshold (based on the ≤0.01° measurement criteria
mentioned in the comment) to correctly determine whether fine-scale coherence is
actually acceptable.
Comment on lines
+157
to
+163
| println!("\n ⇒ coarse loss IS a usable feature: the EWMA rolling floor rides the distribution, keeping the"); | ||
| println!(" 256 buckets well-exposed and the reject tail at ~0.1% as σ drifts (ρ=1.0 vs true μ+3σ). The"); | ||
| println!(" FIXED floor over-prunes after an up-shift (recall collapse, ~98% reject) and under-prunes"); | ||
| println!(" after a down-shift (0% reject). FINDING: the shipped Cascade behaved IDENTICALLY to FIXED —"); | ||
| println!(" its ShiftAlert never fired, because cumulative-Welford per-sample Δμ is always ≪ 2σ, so the"); | ||
| println!(" drift check is inert on a per-sample feed (fire it on batch means, or add an EWMA floor)."); | ||
| println!(" Reconstruction fidelity is irrelevant here — only the floor's calibration is."); |
Contributor
There was a problem hiding this comment.
Final verdict narrative is hard-coded and can contradict measured metrics.
Lines 157–163 assert fixed outcomes (including ρ=1.0 and shipped-vs-fixed behavior) instead of printing the computed run results, so the probe can self-report incorrect conclusions.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@examples/rolling_floor_probe.rs` around lines 157 - 163, The println!
statements at lines 157–163 contain hard-coded narrative text that asserts fixed
outcomes (such as ρ=1.0 and specific reject percentages) instead of reporting
the actual computed metrics from the probe run. Replace these hard-coded
assertions with variable references or computed values that reflect the actual
measured results, so the probe self-reports its true findings rather than
pre-determined conclusions. For example, reference computed variables holding
values like the actual ρ value, measured rejection rates, and drift check
results instead of embedding literal numbers and claims in the narrative
strings.
…on COCA high-D Answers "can one 48-bit helix preserve what CAM-PQ squeezes into 6 bytes?" Measured on a COCA-like high-D population (4096 words, 256 clusters, D=120), all 6-byte budgets: - CAM-PQ-6 (6×u8, tiles all D): recall@10 0.657, dist ρ 0.714 - Helix-48 (rank-3 PCA ceiling): recall@10 0.245, dist ρ 0.274 ⇒ helix CANNOT subsume CAM-PQ for high-D points. A 48-bit helix is a 3-DOF codec (orientation + magnitude); its best reconstruction of a high-D word is the top-3 PCA projection — ~⅓ the recall. Overlap exists only at ≤3D (orientations). 3×8 SPO + 2³ is a different category: a relational triple (= 3 palette/CAM-PQ codes + the Pearl active-mask) neither point codec can hold. No single 48-bit vector subsumes all three — the I-VSA-IDENTITIES / Correction-6 category boundary again: helix = orientation, CAM-PQ = high-D position, SPO = relation. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
… lossless Measures what the Morton-cascade machinery lends CAM-PQ. The coarse prefilter (first 2 of 6 subquantizers) is a partial-ADC LOWER BOUND, so pruning by it is admissible. Measured (N=8192, D=120, PQ-6, 300 queries): - recall@10 vs flat full-ADC = 1.000 at every survivor budget (64..512) — the prune is LOSSLESS relative to CAM-PQ's own results. - full-ADC evals cut 16×–128× (full distance computed only on coarse survivors). - recall@10 vs TRUE full-D ≈ 0.35 — CAM-PQ-6's own quantization ceiling, which the cascade matches exactly (1.0 vs flat) but cannot exceed. ⇒ Morton cascade adds SPEED (16–128× fewer full evals, lossless) + EFFICIENCY (free IVF coarse index / on-demand non-materialized blocks), NOT fidelity — fidelity is the orthogonal coarse+residue plane's job (edge_codec, ICC 0.97–0.99). Speed and fidelity are separable knobs. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
…, composable knobs Capstone map of the PR #218 probe family (10 probes). The holy grail = ONE SoA where every facet composes for accuracy AND speed, established mechanistically: - No single vector subsumes the facets (Correction-6 / I-VSA-IDENTITIES boundary) — the representation is a STRUCT of orthogonal facets, one column per category (HHTL place, helix orientation, CAM-PQ position, CausalEdge64 relation+truth, rolling-floor episodic basin, residue value, EpisodicWitness64 time). - SPEED knob = the cascade (coarse→fine admissible prune + tiling + rolling floor + Morton order): 16–128× fewer full evals at recall 1.000 vs flat — lossless. - FIDELITY knob = the residue plane (coarse + 4-bit/SVD): ICC 0.97–0.99, 14× — +bytes. - They compose: cascade-prune the coarse code, residue-refine the survivors. Includes the per-facet table with measured numbers, the category-boundary iron rules, the reproducible probe inventory, and an explicit WHITE-PATCHES list (EpisodicWitness64 unprobed, end-to-end compose unbuilt, cam_pq_cascade_search/AMX-assign not wired, Cascade Welford-inert bug, real-COCA not run, full SoA assembly pending). https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plateau — entropy-ladder SPO rung decomposition (ndarray foundation)
Matched pair with the AdaWorldAPI/lance-graph PR (the selector + plan). This is the hardware/math foundation: the measurement layer, the codec flavors, and the entropy coordinate that unifies SPO + NARS truth + the syntax→semantics→pragmatics ladder.
Delivered
hpc::reliability— Pearson r, Spearman ρ, Cronbach α, ICC(2,1) absolute-agreement, +FidelityReport. General&[f64]→f64primitives (jc/pillar-consumable); the validate/invalidate measurement layer.hpc::edge_codec— three commensurable quantizers sharing the canonical 16-byte edge block by interpretation, not layout:CoarseOnly(1 B palette index),CoarseResidue(1 + D/2, value-slab signed-4-bit residue),Pq32x4(16 B = 32×4-bit, the turbovec PQ model). Compact seeded k-means; nibble packinglo=code[2t]/hi=code[2t+1].hpc::entropy_ladder—nars_entropy(f,c) = 1 − c·|2f−1|(Staunen↔Wisdom),EntropyRung {Syntax,Semantics,Pragmatics},Quadrant {Staunen,Confusion,Boredom,Wisdom},PEARL_SUBSETS(2³),decompose_spo(reads the 3×palette-256 SPO from a CausalEdge64 — no re-quant),entropy_class(2-bit class for CausalEdge64 spare bits [63:61]).edge_codec_compare(measure all flavors × {blob, continuous}) andentropy_ladder_probe(rung/quadrant partition + Pearl-2³ SPO decomposition).Measured (AMX host)
Tests
28 new unit + 14 doctests;
clippy -D warningsclean (lib + new examples). Also silences a pre-existingexcessive_precisionlint ingolden_helix_probe.Full plan + R2–R6 roadmap lives in
lance-graph/.claude/plans/entropy-ladder-spo-rung-v1.md.https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
Generated by Claude Code
Summary by CodeRabbit
Release Notes