6cb0140fae
The public history carried kat/__pycache__/mlkem768_reference.cpython-314.pyc, a compiled Python artifact embedding the operator's absolute local path. Text secret scanners do not read compiled binaries, which is exactly how it slipped through, and removing it from the tip would have left it reachable through the old root commits. So this repository is republished from a single clean root. This root also carries, from the previously unpublished line of work: - corrected LICENSE year, LICENSING.md, VERIFY-POLICY.md, and CITATIONS-UNRESOLVED.md remeasured 2026-08-11 (101 paths, README aligned) - O-018: run_consensus_verification.py ran 19 of 29 models and reported PASS; it now runs all 29, and computemarket_smt.py gains resolveByTimeout / reclaimUnsettled cases plus a negative control - O-006: the word 'audited' removed from next to Bouncy Castle, twice, after a concurrent edit resurrected it - O-014: prior art named and dated - Algorand's native falcon_verify shipped about ten months before AERE's precompiles; the primacy claim is withdrawn where it was implied - bench/ scripts parametrized so they actually run for an outsider (the earlier textual sanitization left $STAGING unexpanded inside Python strings) - AIP-2/AIP-3 errata with measured figures, spec remeasurements at 2026-08-01, and the spec-zk-stack retractions (owner is an operational key, not the Foundation; 'maximally sound' withdrawn; aggregator V1 deprecated) The redacted bench-host environment files from the sanitized line are kept exactly as published; the unredacted local variants are not carried.
119 lines
7.3 KiB
Markdown
119 lines
7.3 KiB
Markdown
# Poseidon2-BabyBear (width 16) constants: provenance + conformance KAT (component (b), 0x0AE8)
|
|
|
|
> **Scope caveat (2026-07-19 finding).** These BabyBear Poseidon2 constants come from Plonky3
|
|
> `0.4.3-succinct` (Aere's OWN BabyBear + FRI STARK stack). They are NOT the pinned SP1 6.1.0's
|
|
> constants: SP1 6.1.0 is a Hypercube release over the KoalaBear field, so it uses KoalaBear Poseidon2
|
|
> constants, not these. This component is conformance-confirmed for a BabyBear + FRI STARK verifier,
|
|
> not for an SP1 6.1.0 verifier. See `../../docs/AERE-STARK-SP1-RECURSION-AIR-PORT-SPEC-SUMMARY.md`.
|
|
|
|
Date: 2026-07-19. Status: constants CONFIRMED-FROM-SOURCE; real conformance KAT PASSED.
|
|
|
|
This records exactly where every Poseidon2-BabyBear width-16 constant used by the PQ STARK-verify
|
|
reference (component (b)) comes from, and the real known-answer test that confirms conformance. The
|
|
top-level precompile 0x0AE8 stays fail-closed (returns EMPTY for every input) regardless: this
|
|
component being confirmed does not port the sponge/FRI/MMCS/AIR.
|
|
|
|
## Pinned target (exact, checksum-matched)
|
|
|
|
`p3-baby-bear` and `p3-poseidon2`, crates.io version `0.4.3-succinct` (the Succinct Plonky3 fork),
|
|
the revision pinned in the repo's Cargo.lock files:
|
|
- `p3-poseidon2` 0.4.3-succinct, sha256 `522986377b2164c5f94f2dae88e0e0a3d169cc6239202ef4aeb4322d60feffd0`
|
|
- `p3-baby-bear` 0.4.3-succinct, sha256 `d69e6e9af4eaaaa60f7bb9f0e0f73ebcbaefe7e00974d97ad0fa542d6a4f0890`
|
|
|
|
Found in `aerenew/zk-circuits/*/Cargo.lock` and `aerenew/rollup-evm-validity/*/Cargo.lock`. A fresh
|
|
cargo build of the extractor (below) resolved the SAME two checksums, so the crates executed are
|
|
byte-identical to the ones the AERE prover pins.
|
|
|
|
## How the constants and KAT vectors were obtained (primary source)
|
|
|
|
The constants were NOT hand-transcribed. A small Rust extractor depending on the pinned crates was
|
|
compiled and run (cargo 1.97.0), executing the actual library code:
|
|
- `Xoroshiro128Plus::seed_from_u64(1)` (rand_xoshiro), then
|
|
- `Poseidon2::new_from_rng_128(Poseidon2ExternalMatrixGeneral, DiffusionMatrixBabyBear, &mut rng)`
|
|
which samples `external_constants: Vec<[BabyBear;16]>` (8 rounds) then `internal_constants:
|
|
Vec<BabyBear>` (13) via the library's own `Distribution<BabyBear> for Standard`,
|
|
- and `perm.permute_mut(...)` on fixed inputs to emit known-answer vectors.
|
|
|
|
Each field element was printed as its canonical u32 via `PrimeField32::as_canonical_u32()`. The
|
|
extractor also recovered the two linear-layer 16x16 matrices via basis-vector probes (see the
|
|
Montgomery note). Machine-readable output: `p3_ground_truth.json` (constants + KATs) and
|
|
`p3_matrices.json` (the two matrices) from the extractor run.
|
|
|
|
Corroborating source pages (structure only; the executed crates are authoritative for the numbers):
|
|
- p3-baby-bear poseidon2.rs (internal diagonal, RNG comment):
|
|
https://docs.rs/crate/p3-baby-bear/0.4.3-succinct/source/src/poseidon2.rs
|
|
- p3-monty-31 monty_31.rs (`Distribution<MontyField31> for Standard`: `next_u32()>>1`, reject if
|
|
`>= PRIME`, `new_monty(next_u31)`):
|
|
https://docs.rs/crate/p3-monty-31/0.4.3-succinct/source/src/monty_31.rs
|
|
- p3-poseidon2 lib.rs (`new_from_rng_128`: external-then-internal sampling):
|
|
https://docs.rs/crate/p3-poseidon2/0.4.3-succinct/source/src/lib.rs
|
|
- poseidon2 round_numbers.rs (`poseidon2_round_numbers_128(16, 7) = (8, 13)`) and external.rs
|
|
(`apply_mat4`, giving M4 = [[2,3,1,1],[1,2,3,1],[1,1,2,3],[3,1,1,2]]):
|
|
https://github.com/Plonky3/Plonky3 (poseidon2/src/round_numbers.rs, poseidon2/src/external.rs)
|
|
- rand_xoshiro (SplitMix64 seeding, `next_u32 = (next_u64() >> 32)`):
|
|
https://github.com/rust-random/rngs (rand_xoshiro/src/{xoroshiro128plus.rs,splitmix64.rs,common.rs})
|
|
|
|
## Confirmed parameters
|
|
|
|
- Field: BabyBear p = 2^31 - 2^27 + 1 = 2013265921.
|
|
- Width t = 16; S-box x^7 (7 = smallest d>1 with gcd(d, p-1)=1); ROUNDS_F = 8 (4+4), ROUNDS_P = 13.
|
|
- External layer M_E: MDS-light block form from M4 = [[2,3,1,1],[1,2,3,1],[1,1,2,3],[3,1,1,2]]
|
|
(diagonal blocks 2*M4, off-diagonal M4). Confirmed by recovering the exact 16x16 matrix from
|
|
`Poseidon2ExternalMatrixGeneral`.
|
|
- Internal diagonal D = INTERNAL_DIAG_M1_16 (canonical) =
|
|
[p-2, 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 32768].
|
|
- 141 round constants (128 external + 13 internal), canonical; see the reference files
|
|
(`poseidon2_babybear_reference.py/.mjs`, `Poseidon2BabyBearSelfTest.java`, `Poseidon2Bb` in the
|
|
precompile) for the full literal arrays.
|
|
|
|
## The Montgomery subtlety (the trap, resolved)
|
|
|
|
The internal linear layer `DiffusionMatrixBabyBear`, as a map on CANONICAL vectors, is NOT the
|
|
textbook `state[i] = state[i]*D[i] + sum`. Recovering its exact 16x16 canonical matrix from the
|
|
pinned library (basis-vector probes) showed every off-diagonal entry = 943718400 = R^{-1} =
|
|
(2^32)^{-1} mod p, and diagonal I[i][i] = R^{-1}*(1 + D[i]) exactly. So:
|
|
|
|
M_I (canonical) = R^{-1} * (J + diag(D)), i.e. out[i] = R_INV * (sum + D[i]*state[i]),
|
|
R_INV = 943718400.
|
|
|
|
This R^{-1} is the Montgomery-form artifact (the library stores elements in Montgomery form, R =
|
|
2^32). A prior offline pass used the textbook `state[i]*D[i] + sum` (no R^{-1}); that is
|
|
self-consistent and cross-language identical yet DISAGREES with the prover, exactly the
|
|
"two wrong copies agree" trap. It is fixed. The external layer has no such factor (clean integer
|
|
matrix), and round-constant addition + the x^7 S-box are unaffected by Montgomery form.
|
|
|
|
## Conformance KAT (real known-answer vectors, executed from the pinned crates)
|
|
|
|
Emitted by `perm.permute_mut(...)` on the pinned library; reproduced EXACTLY by all three references
|
|
(Python / Node / standalone Java) and by `Poseidon2Bb` in the precompile.
|
|
|
|
zeros
|
|
in = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
|
|
out = [1787823396, 953829438, 89382455, 347481625, 1754527224, 916217775, 1056029082, 410644796,
|
|
1169123478, 1854704276, 1195829987, 1485264906, 1824644035, 1948268315, 847945433, 190591038]
|
|
|
|
iota
|
|
in = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
|
|
out = [157639285, 1851003038, 1852457045, 1920360618, 779990819, 1080011039, 585017685, 1093051731,
|
|
249426030, 967262243, 623744062, 280332881, 1995600430, 1751988435, 317724737, 1895035071]
|
|
|
|
testvec (the p3-baby-bear width-16 test input)
|
|
in = [894848333, 1437655012, 1200606629, 1690012884, 71131202, 1749206695, 1717947831, 120589055,
|
|
19776022, 42382981, 1831865506, 724844064, 171220207, 1299207443, 227047920, 1783754913]
|
|
out = [512585766, 975869435, 1921378527, 1238606951, 899635794, 132650430, 1426417547, 1734425242,
|
|
57415409, 67173027, 1535042492, 1318033394, 1070659233, 17258943, 856719028, 1500534995]
|
|
|
|
Result (ran 2026-07-19):
|
|
- `python test_poseidon2_babybear.py` -> PASS=47017 FAIL=0 (includes 3/3 conformance KATs; Python +
|
|
Node + Java byte-identical; recorded in `../results/kat-results-poseidon2-babybear.json`).
|
|
- `java Poseidon2BabyBearSelfTest` -> PASS=13536 FAIL=0 (includes the 3 conformance KATs).
|
|
|
|
## Status summary
|
|
|
|
- Constants: CONFIRMED-FROM-SOURCE (executed the pinned crates; checksums matched Cargo.lock).
|
|
- Conformance KAT: PASSED (3/3 real known-answer vectors reproduced byte-for-byte).
|
|
- `Poseidon2Bb.available` = true (the permutation is confirmed conformant).
|
|
- Top-level 0x0AE8: still FAIL-CLOSED (EMPTY for every input). The duplex-sponge challenger
|
|
(`Challenger.spongePorted = false`), FRI folding, MMCS/Merkle openings, and the SP1 recursion-AIR
|
|
are un-ported. DO NOT ACTIVATE.
|