aere-research/pq-stark/spec-mmcs-babybear.md

# MMCS (FieldMerkleTreeMmcs) over BabyBear: construction + conformance KAT (component (c), 0x0AE8)

> **Scope caveat (2026-07-19 finding).** "Plonky3/SP1" wording below means Plonky3 `0.4.3-succinct`
> (Aere's OWN BabyBear + FRI STARK stack). It does NOT mean the pinned SP1 6.1.0, which is a Hypercube
> release (KoalaBear multilinear: BaseFold + sumcheck-zerocheck + LogUp-GKR) and does not use this
> BabyBear FRI MMCS config. This component is conformance-confirmed for a BabyBear + FRI STARK
> verifier, not for an SP1 6.1.0 verifier. See `../../docs/AERE-STARK-SP1-RECURSION-AIR-PORT-SPEC-SUMMARY.md`.

Date: 2026-07-19. Status: construction CONFIRMED-FROM-SOURCE; real conformance KAT PASSED.

This records exactly which Mixed Matrix Commitment Scheme (MMCS) Plonky3/SP1 use over BabyBear, where
every part of the construction comes from, and the real known-answer test that confirms conformance.
The top-level precompile 0x0AE8 stays fail-closed (returns EMPTY for every input) regardless: this
component being confirmed does not port the sponge challenger, FRI folding, or the recursion AIR.

## Pinned target (exact, checksum-matched)

The MMCS is `FieldMerkleTreeMmcs` from `p3-merkle-tree`, with the leaf hasher and node compressor from
`p3-symmetric` and the `Mmcs` trait (verify_batch shape) from `p3-commit`, all crates.io version
`0.4.3-succinct` (the Succinct Plonky3 fork), the revision pinned in the repo's Cargo.lock files:
- `p3-merkle-tree` 0.4.3-succinct, sha256 `d5703d9229d52a8c09970e4d722c3a8b4d37e688c306c3a1c03b872efcd204e6`
- `p3-symmetric`   0.4.3-succinct, sha256 `9047ce85c086a9b3f118e10078f10636f7bfeed5da871a04da0b61400af8793a`
- `p3-commit`      0.4.3-succinct, sha256 `50acacc7219fce6c01db938f82c1b21b5e7133990b7fff861f91534aeb569419`
- `p3-matrix`      0.4.3-succinct, sha256 `75c3f150ceb90e09539413bf481e618d05ee19210b4e467d2902eb82d2e15281`

Leaf hashing / node compression run on the CONFIRMED Poseidon2-BabyBear permutation (component (b),
`p3-baby-bear` / `p3-poseidon2` 0.4.3-succinct). Found in `aerenew/zk-circuits/*/Cargo.lock` and
`aerenew/rollup-evm-validity/*/Cargo.lock`. A fresh cargo build of the extractor (below) resolved the
SAME checksums, so the crates executed are byte-identical to the ones the AERE prover pins.

## The exact SP1 inner config (verbatim from p3-merkle-tree's own tests)

`p3-merkle-tree`'s `src/mmcs.rs` test module pins the type aliases the SP1 inner STARK uses:

```
type Perm       = Poseidon2<BabyBear, Poseidon2ExternalMatrixGeneral, DiffusionMatrixBabyBear, 16, 7>;
type MyHash     = PaddingFreeSponge<Perm, 16, 8, 8>;      // WIDTH 16, RATE 8, OUT 8
type MyCompress = TruncatedPermutation<Perm, 2, 8, 16>;   // N 2, CHUNK 8, WIDTH 16
type MyMmcs     = FieldMerkleTreeMmcs<Packing, Packing, MyHash, MyCompress, 8>;  // DIGEST_ELEMS 8
```

So a digest is 8 BabyBear elements (32 bytes). The same aliases appear in the SP1 `BabyBearPoseidon2`
inner config, so this is the commitment the shrink/wrap ShardProof's trace and FRI matrices use.

## Confirmed construction (from source)

- Leaf hasher `PaddingFreeSponge<Perm,16,8,8>` (p3-symmetric `sponge.rs`): overwrite-mode, padding-free
  sponge. State starts all-zero; for each chunk of RATE=8 input elements, OVERWRITE the first
  chunk-length lanes (leaving the remaining lanes at their prior value) and permute; the digest is the
  first OUT=8 lanes. An empty input yields the all-zero digest with no permutation.
  `hash_slice`/`hash_item` are the trait defaults: hash the concatenation.
- Node compressor `TruncatedPermutation<Perm,2,8,16>` (p3-symmetric `compression.rs`):
  `compress([l, r]) = permute(l || r)[0..8]` (l, r each 8 elements padded into a width-16 state).
- Tree build `FieldMerkleTree::new` (p3-merkle-tree `merkle_tree.rs`): sort matrices tallest-first
  (stable); the first digest layer hashes the concatenated rows of ALL matrices at the max height,
  padded up to a power-of-two length with the zero digest; then repeatedly compress adjacent pairs up a
  layer, and where matrices of the next (padded) height exist, inject them via
  `compress([compress([left, right]), hash(their concatenated row)])` (`compress_and_inject`). Rows
  beyond the injected matrices' height (but within the padded layer) are compressed against the zero
  digest. The commitment is the single top digest (the root).
- Height property (asserted by Plonky3): matrix heights that round up to the same power of two must be
  equal, so every padded-power-of-two bucket holds matrices of a single exact height. This is what makes
  `height == max_height` (tree build) and `height.next_power_of_two() == curr_padded` (verify) select
  the same set.
- Opening `open_batch` (p3-merkle-tree `mmcs.rs`): opened rows are each matrix's row at
  `index >> (log_max_height - log2_ceil(matrix.height))`, in ORIGINAL matrix order; the proof is the
  sibling digest `digest_layers[i][(index >> i) ^ 1]` for each of the `log_max_height` layers.
- Verify `verify_batch` (p3-merkle-tree `mmcs.rs`, the only method the on-chain verifier needs): group
  opened rows by padded height (tallest first, by original index); seed the running root with the
  tallest group's hashed concatenated rows; then walk the proof, at each step ordering `(root, sibling)`
  by `index & 1` (0 -> root is left), compressing, and (when the next group's padded height is reached)
  compressing in that group's hashed rows; accept iff the final root equals the commitment.

## Montgomery note

Field elements are serialized as canonical u32 (`PrimeField32::as_canonical_u32`), the same canonical
form the Poseidon2 confirmation used. The permutation carries the Montgomery R^{-1} internal-layer
factor internally (component (b), resolved there); the MMCS layer above it is pure field data plus
hashing, with no additional Montgomery subtlety. As a cross-check, the extractor's `permute([0;16])`
(emitted as `perm_zeros`) equals the confirmed Poseidon2 "zeros" vector
`[1787823396, 953829438, 89382455, ...]`, proving the extractor's permutation is byte-identical to the
one the reference uses.

## How the KAT vectors were obtained (primary source)

A small Rust extractor depending on the pinned crates was compiled and run (cargo 1.97.0). It builds
the exact `MyMmcs` above with the permutation from `Poseidon2::new_from_rng_128(...,
Xoroshiro128Plus::seed_from_u64(1))` (the SAME deterministic construction the Poseidon2 confirmation
used), commits to several known matrix batches, and for each emits the root plus an `open_batch`
opening (opened rows + sibling path). It also asserts the library's OWN `verify_batch` accepts every
emitted opening. Each field element is printed as its canonical u32. The extractor and its output
(`mmcs_ground_truth.json`) are the authoritative source for the numbers below.

## Conformance KAT (real known-answer vectors, executed from the pinned crates)

Six MMCS cases were emitted and are reproduced EXACTLY by all three references (Python / Node /
standalone Java) and by `Mmcs` in the precompile:

| case | matrices (height x width) | open index | exercises |
|---|---|---|---|
| single_8x2 | 8x2 | 3 | single power-of-two height |
| single_6x2 | 6x2 (padded to 8) | 5 | non-power-of-two height (zero-digest padding) |
| single_8x1 | 8x1 | 6 | column vector (commit_vec shape) |
| mixed_8x2_4x3 | 8x2, 4x3 | 5 | two heights, injection at layer 1 |
| mixed_8x1_4x2_2x2 | 8x1, 4x2, 2x2 | 6 | three heights, injection at two layers |
| mixed_5x2_3x1 | 5x2, 3x1 | 4 | non-power-of-two mixed; a default (zero) sibling in the proof |

Example (single_8x2, matrix rows [10,11],[12,13],...,[24,25], open index 3):
- root  = [35761595, 1133593632, 733114748, 517674920, 1449914397, 74512820, 381712048, 469819303]
- opened row = [16, 17]
- proof (3 sibling digests) = [[1513856679, ...], [57846071, ...], [1143223726, ...]]

Primitive KATs (also reproduced exactly):
- hash_item(5)        = [881553380, 703286570, 452412164, ...]
- hash_slice([1,2])   = [1843359319, 912981492, 1448073574, ...]
- hash_slice([1,2,3]) = [1831345102, 1426305082, 956789587, ...]
- hash_slice([1..8])  = [8999572, 1033765830, 347083905, ...]     (one full RATE block)
- hash_slice([1..9])  = [1134257664, 40304233, 1823880005, ...]   (two blocks, tests the second absorb)
- compress(hash[1,2], hash[1,2,3]) = [1968576159, 1450511489, 1750728079, ...]

Full vectors are embedded in `mmcs_babybear_reference.py` (CONFORMANCE_CASES / PRIM_KATS) and mirrored
in the `.mjs` and Java references.

Result (ran 2026-07-19):
- `python test_mmcs_babybear.py` -> PASS=300 FAIL=0 (self-consistency: every valid leaf opens to a path
  that recomputes the root, and tampered openings / proof siblings / roots all fail; CONFORMANCE 37/37;
  Python + Node + Java byte-identical; recorded in `../results/kat-results-mmcs-babybear.json`).
- `java MmcsBabyBearSelfTest` -> PASS=80 FAIL=0 (includes the conformance cases + open-every-leaf).

## Status summary

- Construction: CONFIRMED-FROM-SOURCE (the exact SP1 inner config from p3-merkle-tree's own tests;
  tree build / open / verify traced against p3-merkle-tree + p3-symmetric + p3-commit 0.4.3-succinct;
  checksums matched Cargo.lock).
- Conformance KAT: PASSED (6 MMCS cases + 6 primitive KATs + perm sanity reproduced byte-for-byte, and
  verify_batch accepts the emitted openings and rejects tampered ones).
- `Mmcs.available` = true (the commitment scheme is confirmed conformant).
- `[MEASURE]` remaining: conformance against a REAL exported SP1 v6.1.0 commitment root (the trace / FRI
  matrices from an actual proof) is a further step needing an exported proof + the challenger.
- Top-level 0x0AE8: still FAIL-CLOSED (EMPTY for every input). The duplex-sponge challenger
  (`Challenger.spongePorted = false`, component (f)), FRI folding (component (d)), and the SP1
  recursion-AIR (`StarkConstraints` = UNAVAILABLE, component (e)) are un-ported. ACCEPT is unreachable.
  DO NOT ACTIVATE.