102 lines
8.0 KiB
Markdown
102 lines
8.0 KiB
Markdown
# STARK-verify port 0x0AE8, component (c): MMCS (FieldMerkleTreeMmcs) - IMPLEMENTED + CONFORMANCE-CONFIRMED
|
|
|
|
> **Note added 2026-10-30 (consensus).** Since block 25,080,000 (AIP-22 stage 5) headers no longer carry ECDSA commit seals: each carries the Falcon-512 seal of its proposer instead, and a commit still carries the ECDSA commit seal of its validator as an additional check. Where the note of 2026-09-24 below describes the headers, it describes chain 2800 below that block.
|
|
|
|
> **Note added 2026-09-24 (consensus).** Where this document says that consensus is classical secp256k1 ECDSA and not post-quantum, read it as the per-block finality statement it was until block 24,300,000 (AIP-22 stage 4, 2026-10-25). Since that block every validator attributes a consensus vote only through its verified post-quantum seal, and a block is post-quantum final once the next anchor, at most 32 blocks later, carries the certificate that covers it; headers still carry classical ECDSA commit seals until AIP-22 stage 5 (this sentence corrected on 2026-10-25). It no longer describes all of consensus on chain 2800: since 2026-08-14 every anchor block must carry a certificate of validator post-quantum seals under its hash (Falcon-512, with SLH-DSA as well since 2026-09-04), and since 2026-09-05 (block 17,250,000) the nine AERE-client validators (nine of the ten) refuse a commit message without a valid Falcon-512 seal of its author, and since 2026-09-08 (block 17,700,000) any QBFT consensus message without one, while the tenth, the second client, enforced that seal only on commit messages until block 24,300,000 and attributes every consensus message only through it since (this clause corrected on 2026-10-12, 2026-10-25 and 2026-10-28; until 2026-10-28 it dated the refusal of every message type to 2026-09-05, the date of the commit layer alone). (Note applied to this document on 2026-10-28.)
|
|
|
|
> **Scope caveat (2026-07-19 finding).** This component is conformance-confirmed against Plonky3
|
|
> `0.4.3-succinct` (Aere's OWN BabyBear + FRI STARK stack), NOT against SP1 6.1.0. SP1 6.1.0 is a
|
|
> **Hypercube** release (KoalaBear multilinear: BaseFold + Jagged + sumcheck-zerocheck + LogUp-GKR),
|
|
> so FRI and DEEP-ALI do not apply to it and this skeleton does not verify SP1 6.1.0 proofs. Replacing
|
|
> the SP1 BN254 Groth16 wrap is a separate ~22 to 32 person-week retarget. See
|
|
> `AERE-STARK-SP1-RECURSION-AIR-PORT-SPEC-SUMMARY.md`.
|
|
|
|
Date: 2026-07-19. Status: real known-answer test PASSES. Top-level precompile stays fail-closed.
|
|
|
|
## What was done
|
|
|
|
Implemented and conformance-confirmed component (c) of the SP1/Plonky3 inner STARK-verify port: the
|
|
Mixed Matrix Commitment Scheme (MMCS), i.e. Plonky3's `FieldMerkleTreeMmcs`, the Merkle-tree vector
|
|
commitment that FRI (d) and the trace commitment are built on. Same rigorous ground-truth method that
|
|
worked for Poseidon2: compiled and executed the exact pinned Plonky3 crates to emit authoritative
|
|
reference vectors, then reproduced them byte-for-byte.
|
|
|
|
### 1. Confirmed the exact construction from the pinned crates
|
|
|
|
Read the source of the pinned `0.4.3-succinct` crates (checksums matched the repo Cargo.lock):
|
|
`p3-merkle-tree` (d5703d92...), `p3-symmetric` (9047ce85...), `p3-commit` (50acacc7...),
|
|
`p3-matrix` (75c3f150...). The SP1 inner config is pinned VERBATIM in `p3-merkle-tree`'s own
|
|
`mmcs.rs` tests:
|
|
- leaf hasher = `PaddingFreeSponge<Perm, WIDTH=16, RATE=8, OUT=8>` (overwrite-mode, padding-free)
|
|
- 2-to-1 compr. = `TruncatedPermutation<Perm, N=2, CHUNK=8, WIDTH=16>` (permute(l||r)[0..8])
|
|
- MMCS = `FieldMerkleTreeMmcs<Packing, Packing, MyHash, MyCompress, DIGEST_ELEMS=8>`
|
|
So a digest = 8 BabyBear elements (32 bytes). Leaf hash / node compression run on the CONFIRMED
|
|
Poseidon2-BabyBear permutation (component (b)). Traced the tree build (`first_digest_layer` /
|
|
`compress_and_inject`), `open_batch`, and `verify_batch` from source; documented the mixed-height rule
|
|
(tallest-first, power-of-two padding with the zero digest, injection at the matching padded layer,
|
|
index reduced by `index >> (log_max_height - log2_ceil(height))`) and the height property that makes
|
|
the tree-build and verify grouping select the same set.
|
|
|
|
### 2. Ground-truth extractor (Rust, scratch dir only)
|
|
|
|
Wrote `mmcs-extractor` depending on the pinned crates, built the exact `MyMmcs`, and used the SAME
|
|
deterministic permutation as the Poseidon2 confirmation (`new_from_rng_128(seed_from_u64(1))`). It
|
|
commits to six known matrix batches and emits the root + an `open_batch` opening (opened rows + sibling
|
|
path) for each, printing every field element as canonical u32, and asserts the library's OWN
|
|
`verify_batch` accepts each. Its `permute([0;16])` equals the confirmed Poseidon2 "zeros" vector, so
|
|
the extractor's permutation is byte-identical to the reference's (guards the "two wrong copies" trap).
|
|
Ran on this box; all six library `verify_batch` calls returned Ok.
|
|
|
|
The six cases exercise: single power-of-two height (8x2), single non-power-of-two height (6x2, padded,
|
|
zero-digest padding), a column vector (8x1), and three mixed-height batches (8x2+4x3, 8x1+4x2+2x2,
|
|
5x2+3x1). One case (mixed_5x2_3x1) has a default (all-zero) sibling digest in its proof.
|
|
|
|
### 3. References + precompile wiring (matching the existing convention)
|
|
|
|
- `pqc-fork/pq-stark/mmcs_babybear_reference.py` (Python), `.mjs` (Node), `MmcsBabyBearSelfTest.java`
|
|
(standalone Java): hash_iter (PaddingFreeSponge), compress2to1 (TruncatedPermutation), full
|
|
mixed-height tree build + commit + open_batch + verify_batch, over the confirmed Poseidon2.
|
|
- `pqc-fork/precompiles/Sp1StarkVerifierPrecompiledContract.java`: new `Mmcs` static class
|
|
(`hashIter`, `compress2to1`, mixed-height `verifyBatch`) wired behind the still-fail-closed top level;
|
|
`Mmcs.available = true`. Referenced from `Fri.checkQuery` (still returns UNAVAILABLE). Compiles clean.
|
|
- `pqc-fork/pq-stark/test_mmcs_babybear.py` harness + `../results/kat-results-mmcs-babybear.json`.
|
|
- `pqc-fork/pq-stark/spec-mmcs-babybear.md` provenance doc.
|
|
- Updated `docs/AERE-STARK-VERIFIER-PORT-SPEC.md` (section 4 -> CONFIRMED, status table, intro, files)
|
|
and `pqc-fork/pq-stark/README.md` (new (c) section, layout, reproduce).
|
|
|
|
### 4. CONFORMANCE KAT: PASSED
|
|
|
|
`python test_mmcs_babybear.py` -> **PASS=300 FAIL=0**. CONFORMANCE 37/37 (perm sanity + 6 primitive
|
|
hash/compress KATs + 6 MMCS cases: root + openings + proof + verify-accept + tamper-reject) reproduced
|
|
EXACTLY from the pinned crates. Cross-language byte-identical across Python + Node + Java. Standalone
|
|
`java MmcsBabyBearSelfTest` -> PASS=80 FAIL=0 (includes open-every-valid-leaf self-consistency).
|
|
Poseidon2 regression (`test_poseidon2_babybear.py`) still PASS. No em-dashes / `--` in prose docs.
|
|
|
|
### Multi-matrix generalization
|
|
|
|
Handled beyond the single-matrix case: three genuine mixed-height batches (2 and 3 matrices) are in the
|
|
KAT and pass, so the tallest-first grouping, power-of-two padding, and injection logic are exercised,
|
|
not just documented.
|
|
|
|
## Fail-closed confirmation (unchanged)
|
|
|
|
The top-level 0x0AE8 still verifies NOTHING and returns EMPTY for every input:
|
|
- `StarkConstraints.evaluateAtZeta` returns UNAVAILABLE (stage 4 -> UNAVAILABLE -> EMPTY).
|
|
- `Challenger.spongePorted = false` (absorb no-op; `checkProofOfWork` false; `sampleQueryIndices` empty).
|
|
- `Fri.checkQuery` returns UNAVAILABLE.
|
|
- `computePrecompile` returns `VERIFIED` only if `verify(...) == ACCEPT`; the single `return ACCEPT`
|
|
line is unreachable. `Mmcs.available = true` flips NO top-level gate.
|
|
- DO NOT ACTIVATE banner intact.
|
|
|
|
## Honesty caveat ([MEASURE] remaining)
|
|
|
|
Conformance is against the pinned crate library (KAT reproduced byte-for-byte). Matching a REAL exported
|
|
SP1 v6.1.0 proof's actual commitment root (its specific trace / FRI matrices and heights) is a further
|
|
[MEASURE] step needing an exported proof + the challenger; flagged honestly, not faked.
|
|
|
|
## Scope boundary (unchanged)
|
|
|
|
On-chain ZK verifiers on Aere remain the classical BN254 Groth16 gateway until the whole STARK verifier
|
|
is real, audited, and founder-activated. This is one more component of a multi-week port. Consensus on
|
|
chain 2800 is unaffected (classical secp256k1 ECDSA QBFT).
|