aere-research/pq-stark/spec-fri-babybear.md
Aere Network 6cb0140fae Republished from a clean root: the compiled artifact is gone from history, and the local line of work joins the sanitized public line
The public history carried kat/__pycache__/mlkem768_reference.cpython-314.pyc,
a compiled Python artifact embedding the operator's absolute local path. Text
secret scanners do not read compiled binaries, which is exactly how it slipped
through, and removing it from the tip would have left it reachable through the
old root commits. So this repository is republished from a single clean root.

This root also carries, from the previously unpublished line of work:
- corrected LICENSE year, LICENSING.md, VERIFY-POLICY.md, and
  CITATIONS-UNRESOLVED.md remeasured 2026-08-11 (101 paths, README aligned)
- O-018: run_consensus_verification.py ran 19 of 29 models and reported PASS;
  it now runs all 29, and computemarket_smt.py gains resolveByTimeout /
  reclaimUnsettled cases plus a negative control
- O-006: the word 'audited' removed from next to Bouncy Castle, twice, after a
  concurrent edit resurrected it
- O-014: prior art named and dated - Algorand's native falcon_verify shipped
  about ten months before AERE's precompiles; the primacy claim is withdrawn
  where it was implied
- bench/ scripts parametrized so they actually run for an outsider (the
  earlier textual sanitization left $STAGING unexpanded inside Python strings)
- AIP-2/AIP-3 errata with measured figures, spec remeasurements at 2026-08-01,
  and the spec-zk-stack retractions (owner is an operational key, not the
  Foundation; 'maximally sound' withdrawn; aggregator V1 deprecated)
The redacted bench-host environment files from the sanitized line are kept
exactly as published; the unredacted local variants are not carried.
2026-08-15 13:52:14 +03:00

159 lines
11 KiB
Markdown

# FRI verifier over BabyBear: config + fold/opening relation + conformance KAT (component (d), 0x0AE8)
> **Scope caveat (2026-07-19 finding).** "Plonky3/SP1" wording below means Plonky3 `0.4.3-succinct`
> (Aere's OWN BabyBear + FRI STARK stack). It does NOT mean the pinned SP1 6.1.0, which is a Hypercube
> release whose inner recursion proof uses BaseFold, NOT FRI. This FRI low-degree test does not apply
> to SP1 6.1.0 at all; it is conformance-confirmed for a BabyBear + FRI STARK verifier (Aere's own
> Plonky3 circuits), not for an SP1 6.1.0 verifier. See
> `../../docs/AERE-STARK-SP1-RECURSION-AIR-PORT-SPEC-SUMMARY.md`.
Date: 2026-07-19. Status: FRI config CONFIRMED-FROM-SOURCE; fold + opening relation (verify_query)
IMPLEMENTED; real conformance KAT PASSED (accept genuine + reject tampered).
This records the exact FRI low-degree test Plonky3/SP1 use over BabyBear, traced from the pinned
`p3-fri` source, the fold + opening relation the verifier checks, and the real known-answer test that
confirms conformance. The top-level precompile 0x0AE8 stays FAIL-CLOSED (returns EMPTY for every input)
regardless: this component being confirmed does not port the duplex-sponge Fiat-Shamir challenger
(component (f)) that binds the transcript, nor the SP1 recursion-AIR (component (e)).
## Pinned target (exact, checksum-matched)
The FRI verifier is `p3_fri::verifier` from `p3-fri`, its commit-phase MMCS is `ExtensionMmcs` from
`p3-commit` wrapping the CONFIRMED `FieldMerkleTreeMmcs` (component (c)), and the challenger used to
emit the ground truth is `DuplexChallenger` from `p3-challenger`. All crates.io version
`0.4.3-succinct` (the Succinct Plonky3 fork), the revision pinned in the repo's Cargo.lock files:
- `p3-fri` 0.4.3-succinct, sha256 `5cbc4965ee488f3247867b7ec4bb005b8afa72cb0d461a4dcb1387ecab6426d5`
- `p3-challenger` 0.4.3-succinct, sha256 `b6a908924d43e4cfb93fb41c8346cac211b70314385a9037e9241f5b7f3eaf77`
- `p3-dft` 0.4.3-succinct, sha256 `be6408b10a2c27eb13a7d5580c546c2179a8dc7dbc10a990657311891f9b41c0`
- `p3-commit` 0.4.3-succinct, sha256 `50acacc7219fce6c01db938f82c1b21b5e7133990b7fff861f91534aeb569419`
Found in `aerenew/zk-circuits/*/Cargo.lock` and `aerenew/rollup-evm-validity/*/Cargo.lock`. A fresh
cargo build of the extractor (below) resolved the SAME checksums, so the crates executed are
byte-identical to the ones the AERE prover pins.
## The confirmed FRI config (traced from p3-fri source)
`FriConfig` (p3-fri `config.rs`) has exactly four fields:
`{ log_blowup, num_queries, proof_of_work_bits, mmcs }`. Traced from `verifier.rs`:
- Folding arity is 2 (hard-coded): each layer opens the sibling pair `(index, index ^ 1)` and folds
`index_pair = index >> 1`. The comment in `proof.rs` says a non-2 arity would need multiple siblings;
this pinned version is arity 2.
- `log_max_height = commit_phase_commits.len() + log_blowup`. The commit phase runs for
`log_max_height - log_blowup` layers (`for log_folded_height in (log_blowup..log_max_height).rev()` in
`prover.rs` commit_phase), so `num_fold_rounds = log_max_height - log_blowup`.
- The FINAL polynomial is a single CONSTANT `final_poly: F` (F = the challenge field EF). Folding runs
all the way down; after the last fold the remaining `blowup()` evaluations are all equal, and the
verifier checks `folded_eval == final_poly` for EVERY query (`verify_challenges`). (The source
comment notes it "could become Vec<...> if generalized to support non-constant final polynomials"; in
this pinned version it is a constant, so `log_final_poly_len = 0`.)
- The commit-phase MMCS is `ExtensionMmcs<Val, Challenge, ValMmcs>` (the FRI codewords live in the
degree-4 extension EF = `BinomialExtensionField<BabyBear, 4>`). Each commit-phase leaf is a PAIR of EF
evaluations; `ExtensionMmcs::verify_batch` flattens each EF element to its 4 base coords
(`as_base_slice`) and concatenates, so a leaf is 8 BabyBear coords, a single width-8 row of a
height-`2^log_folded_height` tree, checked by the CONFIRMED base `FieldMerkleTreeMmcs::verify_batch`
(component (c)).
- Proof-of-work / grinding (`proof_of_work_bits`, `check_witness`) is part of the transcript phase
(`verify_shape_and_sample_challenges`), i.e. component (f); it is NOT part of the per-query fold +
opening relation this component ports.
- `num_queries`, `log_blowup`, `proof_of_work_bits` are proximity/soundness parameters read from the
frozen SP1 config; the values are a [MEASURE] against a real exported proof (see the port spec). The
KAT here exercises several (log_blowup, log_max_height, num_queries) shapes to validate the relation
independently of the exact frozen numbers.
## The fold + opening relation (p3-fri verify_query, ported)
For one query at `index` (from `verifier.rs verify_query`, reproduced exactly by the references and by
`Fri.verifyQuery` in the precompile):
```
folded_eval = 0 in EF
x = two_adic_generator(log_max_height)^reverse_bits_len(index, log_max_height) # coset point, in EF
for each layer (log_folded_height = log_max_height-1 down to log_blowup):
folded_eval += reduced_openings[log_folded_height + 1] # the injected input opening at this height
index_sibling = index ^ 1; index_pair = index >> 1
evals = [folded_eval, folded_eval]; evals[index_sibling % 2] = sibling_value # the opened sibling
MMCS.verify_batch(commit_layer, dims={width:2(EF)->8(base), height:2^log_folded_height},
index_pair, [evals flattened to 8 base coords], opening_proof) # component (c)
xs = [x, x]; xs[index_sibling % 2] *= two_adic_generator(1) # the sibling point is -x
folded_eval = evals[0] + (beta - xs[0]) * (evals[1] - evals[0]) / (xs[1] - xs[0]) # interpolate at beta
index = index_pair; x = x^2
return folded_eval # verify_challenges checks this == final_poly for every query
```
- `two_adic_generator(bits)` for `bits <= 27` is the BabyBear base generator embedded in EF (confirmed
from `p3-baby-bear extension.rs ext_two_adic_generator`), so `x` is a base-field element embedded in
EF; `two_adic_generator(1) = -1` (the order-2 root). The arithmetic uses F_{p^4} (component (a)).
- The reduced openings come from the input polynomials in a full STARK (component (e)); for a
standalone single-codeword low-degree test they are zero except `reduced_openings[log_max_height] =
codeword_value_at_index`, which the extractor emits as `ro_top`.
## Montgomery / canonical note
Field elements are serialized as canonical u32 (`PrimeField32::as_canonical_u32` for base coords). The
F_{p^4} fold arithmetic is canonical (component (a)); the Montgomery `R^{-1}` factor lives entirely
inside the Poseidon2 internal layer (component (b), resolved there) and does not surface in the FRI or
MMCS layers. As a cross-check, the extractor's `permute([0;16])` (emitted as `perm_zeros`) equals the
confirmed Poseidon2 "zeros" vector, and the extractor's `two_adic_generator(bits)` for `bits 0..=27`
equals component (a)'s `twoAdicGenerator(bits)` (this is what makes the fold's coset points come out
right, closing the old `[VERIFY]` on the generator).
## How the KAT vectors were obtained (primary source)
A small Rust extractor (`pq-stark/fri-extractor`, `main.rs` + `Cargo.toml` + `Cargo.lock`) depending on
the pinned crates was compiled and run (cargo 1.97.0). It builds the exact SP1 inner FRI config
(BabyBear + `BinomialExtensionField<_,4>` + the `seed_from_u64(1)` Poseidon2 permutation +
`FieldMerkleTreeMmcs` + `ExtensionMmcs` + `DuplexChallenger<_,_,16,8>`), constructs genuinely
low-degree codewords (an RS/LDE of a random low-degree polynomial via `Radix2Dit`, bit-reversed), runs
`p3_fri::prover::prove` to emit a real `FriProof`, then runs
`p3_fri::verifier::{verify_shape_and_sample_challenges, verify_challenges}` to confirm the pinned
LIBRARY itself accepts. It emits everything the reference verifier needs (commit-phase roots, betas,
query indices, reduced openings, per-query per-layer sibling values + MMCS opening proofs, final poly),
each field element as canonical u32. The output (`pq-stark/fri_ground_truth.json`) is the authoritative
source. Three FRI cases are emitted:
| case | log_blowup | log_max_height | num_queries | fold layers |
|---|---|---|---|---|
| blowup1_h6_q4 | 1 | 6 | 4 | 5 |
| blowup2_h7_q5 | 2 | 7 | 5 | 5 |
| blowup1_h8_q6 | 1 | 8 | 6 | 7 |
## Conformance KAT (real known-answer test, executed from the pinned crates)
The reference (`fri_verify_reference.py`, `.mjs`, standalone `FriVerifySelfTest.java`, and
`Fri.verifyQuery` in the precompile) re-verifies each emitted proof:
- ACCEPT: every query's MMCS openings pass AND every query's folded constant equals `final_poly`.
- REJECT (four tamper variants per case): a corrupted opened value (breaks the MMCS opening), a
corrupted MMCS sibling digest (breaks the opening), a wrong fold challenge `beta` (breaks the
interpolation so `folded != final_poly`, i.e. a wrong fold), and a non-matching `final_poly` (the
final-poly check, the mechanism that catches a non-low-degree witness).
Result (ran 2026-07-19):
- `python test_fri_verify.py` -> PASS=55 FAIL=0 (sanity perm + 28 generators; 3 cases x (accept + 4
rejects); Python + Node + Java byte-identical per-query folded values; recorded in
`../results/kat-results-fri-verify.json`).
- `java FriVerifySelfTest fri_ground_truth.json` -> PASS=44 FAIL=0.
- The PRECOMPILE's own `Sp1StarkVerifierPrecompiledContract.Fri.verifyQuery` (driven directly)
reproduces accept + reject for all 3 cases (15/15), so the wired code, not just the parallel
references, is conformant.
## Status summary
- FRI config: CONFIRMED-FROM-SOURCE (arity 2; num_fold_rounds = log_max_height - log_blowup; constant
final poly; ExtensionMmcs-over-FieldMerkleTreeMmcs commit phase; traced from p3-fri 0.4.3-succinct;
checksums matched Cargo.lock).
- Fold + opening relation (verify_query): IMPLEMENTED (Python/Node/Java + `Fri.verifyQuery`).
- Conformance KAT: PASSED (3 real FRI proofs accepted, 4 tamper variants each rejected; three-language
byte-identical; the precompile's own method also conformant).
- Closed by this KAT (were `[VERIFY]`): arity-2 fold, num_fold_rounds, reverse-bits index-to-coset-point
mapping, `two_adic_generator(bits)` = Plonky3's, and the F_{p^4} non-residue `W = 11` = Plonky3's (the
EF fold arithmetic reproduces the interpolation).
- `Fri.foldRelationConfirmed` = true (the fold + opening relation is confirmed conformant).
- `[MEASURE]` remaining: conformance against a REAL exported SP1 v6.1.0 inner proof's FRI section (needs
the exported proof + the challenger, component (f)) and the AIR reduced openings (component (e)).
- Top-level 0x0AE8: still FAIL-CLOSED (EMPTY for every input). The duplex-sponge challenger
(`Challenger.spongePorted = false`, component (f)) that derives the betas/indices/grinding, and the
SP1 recursion-AIR (`StarkConstraints` = UNAVAILABLE, component (e)) that supplies the reduced
openings, are un-ported, so `Fri.checkQuery` cannot be driven and returns UNAVAILABLE. ACCEPT is
unreachable. DO NOT ACTIVATE.