aere-research/pq-stark/spec-mmcs-babybear.md

MMCS (FieldMerkleTreeMmcs) over BabyBear: construction + conformance KAT (component (c), 0x0AE8)

Scope caveat (2026-07-19 finding). "Plonky3/SP1" wording below means Plonky3 0.4.3-succinct (Aere's OWN BabyBear + FRI STARK stack). It does NOT mean the pinned SP1 6.1.0, which is a Hypercube release (KoalaBear multilinear: BaseFold + sumcheck-zerocheck + LogUp-GKR) and does not use this BabyBear FRI MMCS config. This component is conformance-confirmed for a BabyBear + FRI STARK verifier, not for an SP1 6.1.0 verifier. See ../../docs/AERE-STARK-SP1-RECURSION-AIR-PORT-SPEC-SUMMARY.md.

Date: 2026-07-19. Status: construction CONFIRMED-FROM-SOURCE; real conformance KAT PASSED.

This records exactly which Mixed Matrix Commitment Scheme (MMCS) Plonky3/SP1 use over BabyBear, where every part of the construction comes from, and the real known-answer test that confirms conformance. The top-level precompile 0x0AE8 stays fail-closed (returns EMPTY for every input) regardless: this component being confirmed does not port the sponge challenger, FRI folding, or the recursion AIR.

Pinned target (exact, checksum-matched)

The MMCS is FieldMerkleTreeMmcs from p3-merkle-tree, with the leaf hasher and node compressor from p3-symmetric and the Mmcs trait (verify_batch shape) from p3-commit, all crates.io version 0.4.3-succinct (the Succinct Plonky3 fork), the revision pinned in the repo's Cargo.lock files:

• p3-merkle-tree 0.4.3-succinct, sha256 d5703d9229d52a8c09970e4d722c3a8b4d37e688c306c3a1c03b872efcd204e6
• p3-symmetric 0.4.3-succinct, sha256 9047ce85c086a9b3f118e10078f10636f7bfeed5da871a04da0b61400af8793a
• p3-commit 0.4.3-succinct, sha256 50acacc7219fce6c01db938f82c1b21b5e7133990b7fff861f91534aeb569419
• p3-matrix 0.4.3-succinct, sha256 75c3f150ceb90e09539413bf481e618d05ee19210b4e467d2902eb82d2e15281

Leaf hashing / node compression run on the CONFIRMED Poseidon2-BabyBear permutation (component (b), p3-baby-bear / p3-poseidon2 0.4.3-succinct). Found in aerenew/zk-circuits/*/Cargo.lock and aerenew/rollup-evm-validity/*/Cargo.lock. A fresh cargo build of the extractor (below) resolved the SAME checksums, so the crates executed are byte-identical to the ones the AERE prover pins.

The exact SP1 inner config (verbatim from p3-merkle-tree's own tests)

p3-merkle-tree's src/mmcs.rs test module pins the type aliases the SP1 inner STARK uses:

type Perm       = Poseidon2<BabyBear, Poseidon2ExternalMatrixGeneral, DiffusionMatrixBabyBear, 16, 7>;
type MyHash     = PaddingFreeSponge<Perm, 16, 8, 8>;      // WIDTH 16, RATE 8, OUT 8
type MyCompress = TruncatedPermutation<Perm, 2, 8, 16>;   // N 2, CHUNK 8, WIDTH 16
type MyMmcs     = FieldMerkleTreeMmcs<Packing, Packing, MyHash, MyCompress, 8>;  // DIGEST_ELEMS 8

So a digest is 8 BabyBear elements (32 bytes). The same aliases appear in the SP1 BabyBearPoseidon2 inner config, so this is the commitment the shrink/wrap ShardProof's trace and FRI matrices use.

Confirmed construction (from source)

• Leaf hasher PaddingFreeSponge<Perm,16,8,8> (p3-symmetric sponge.rs): overwrite-mode, padding-free sponge. State starts all-zero; for each chunk of RATE=8 input elements, OVERWRITE the first chunk-length lanes (leaving the remaining lanes at their prior value) and permute; the digest is the first OUT=8 lanes. An empty input yields the all-zero digest with no permutation. hash_slice/hash_item are the trait defaults: hash the concatenation.
• Node compressor TruncatedPermutation<Perm,2,8,16> (p3-symmetric compression.rs): compress([l, r]) = permute(l || r)[0..8] (l, r each 8 elements padded into a width-16 state).
• Tree build FieldMerkleTree::new (p3-merkle-tree merkle_tree.rs): sort matrices tallest-first (stable); the first digest layer hashes the concatenated rows of ALL matrices at the max height, padded up to a power-of-two length with the zero digest; then repeatedly compress adjacent pairs up a layer, and where matrices of the next (padded) height exist, inject them via compress([compress([left, right]), hash(their concatenated row)]) (compress_and_inject). Rows beyond the injected matrices' height (but within the padded layer) are compressed against the zero digest. The commitment is the single top digest (the root).
• Height property (asserted by Plonky3): matrix heights that round up to the same power of two must be equal, so every padded-power-of-two bucket holds matrices of a single exact height. This is what makes height == max_height (tree build) and height.next_power_of_two() == curr_padded (verify) select the same set.
• Opening open_batch (p3-merkle-tree mmcs.rs): opened rows are each matrix's row at index >> (log_max_height - log2_ceil(matrix.height)), in ORIGINAL matrix order; the proof is the sibling digest digest_layers[i][(index >> i) ^ 1] for each of the log_max_height layers.
• Verify verify_batch (p3-merkle-tree mmcs.rs, the only method the on-chain verifier needs): group opened rows by padded height (tallest first, by original index); seed the running root with the tallest group's hashed concatenated rows; then walk the proof, at each step ordering (root, sibling) by index & 1 (0 -> root is left), compressing, and (when the next group's padded height is reached) compressing in that group's hashed rows; accept iff the final root equals the commitment.

Montgomery note

Field elements are serialized as canonical u32 (PrimeField32::as_canonical_u32), the same canonical form the Poseidon2 confirmation used. The permutation carries the Montgomery R^{-1} internal-layer factor internally (component (b), resolved there); the MMCS layer above it is pure field data plus hashing, with no additional Montgomery subtlety. As a cross-check, the extractor's permute([0;16]) (emitted as perm_zeros) equals the confirmed Poseidon2 "zeros" vector [1787823396, 953829438, 89382455, ...], proving the extractor's permutation is byte-identical to the one the reference uses.

How the KAT vectors were obtained (primary source)

A small Rust extractor depending on the pinned crates was compiled and run (cargo 1.97.0). It builds the exact MyMmcs above with the permutation from Poseidon2::new_from_rng_128(..., Xoroshiro128Plus::seed_from_u64(1)) (the SAME deterministic construction the Poseidon2 confirmation used), commits to several known matrix batches, and for each emits the root plus an open_batch opening (opened rows + sibling path). It also asserts the library's OWN verify_batch accepts every emitted opening. Each field element is printed as its canonical u32. The extractor and its output (mmcs_ground_truth.json) are the authoritative source for the numbers below.

Conformance KAT (real known-answer vectors, executed from the pinned crates)

Six MMCS cases were emitted and are reproduced EXACTLY by all three references (Python / Node / standalone Java) and by Mmcs in the precompile:

case	matrices (height x width)	open index	exercises
single_8x2	8x2	3	single power-of-two height
single_6x2	6x2 (padded to 8)	5	non-power-of-two height (zero-digest padding)
single_8x1	8x1	6	column vector (commit_vec shape)
mixed_8x2_4x3	8x2, 4x3	5	two heights, injection at layer 1
mixed_8x1_4x2_2x2	8x1, 4x2, 2x2	6	three heights, injection at two layers
mixed_5x2_3x1	5x2, 3x1	4	non-power-of-two mixed; a default (zero) sibling in the proof

Example (single_8x2, matrix rows [10,11],[12,13],...,[24,25], open index 3):

• root = [35761595, 1133593632, 733114748, 517674920, 1449914397, 74512820, 381712048, 469819303]
• opened row = [16, 17]
• proof (3 sibling digests) = 1513856679, ...], [57846071, ...], [1143223726, ...

Primitive KATs (also reproduced exactly):

• hash_item(5) = [881553380, 703286570, 452412164, ...]
• hash_slice([1,2]) = [1843359319, 912981492, 1448073574, ...]
• hash_slice([1,2,3]) = [1831345102, 1426305082, 956789587, ...]
• hash_slice([1..8]) = [8999572, 1033765830, 347083905, ...] (one full RATE block)
• hash_slice([1..9]) = [1134257664, 40304233, 1823880005, ...] (two blocks, tests the second absorb)
• compress(hash[1,2], hash[1,2,3]) = [1968576159, 1450511489, 1750728079, ...]

Full vectors are embedded in mmcs_babybear_reference.py (CONFORMANCE_CASES / PRIM_KATS) and mirrored in the .mjs and Java references.

Result (ran 2026-07-19):

• python test_mmcs_babybear.py -> PASS=300 FAIL=0 (self-consistency: every valid leaf opens to a path that recomputes the root, and tampered openings / proof siblings / roots all fail; CONFORMANCE 37/37; Python + Node + Java byte-identical; recorded in ../results/kat-results-mmcs-babybear.json).
• java MmcsBabyBearSelfTest -> PASS=80 FAIL=0 (includes the conformance cases + open-every-leaf).

Status summary

• Construction: CONFIRMED-FROM-SOURCE (the exact SP1 inner config from p3-merkle-tree's own tests; tree build / open / verify traced against p3-merkle-tree + p3-symmetric + p3-commit 0.4.3-succinct; checksums matched Cargo.lock).
• Conformance KAT: PASSED (6 MMCS cases + 6 primitive KATs + perm sanity reproduced byte-for-byte, and verify_batch accepts the emitted openings and rejects tampered ones).
• Mmcs.available = true (the commitment scheme is confirmed conformant).
• [MEASURE] remaining: conformance against a REAL exported SP1 v6.1.0 commitment root (the trace / FRI matrices from an actual proof) is a further step needing an exported proof + the challenger.
• Top-level 0x0AE8: still FAIL-CLOSED (EMPTY for every input). The duplex-sponge challenger (Challenger.spongePorted = false, component (f)), FRI folding (component (d)), and the SP1 recursion-AIR (StarkConstraints = UNAVAILABLE, component (e)) are un-ported. ACCEPT is unreachable. DO NOT ACTIVATE.