aere-research/research/specs/spec-parallel-execution.md

AERE Parallel Execution: Block-STM Executor and On-Chain Hint Registry

Technical specification. Chain 2800 (AERE Network mainnet).

Grounding: parallel-executor/ (Rust), rollup-sequencer/src/blockstm/ (TypeScript), contracts/contracts/parallel/AereBlockSTMRegistry.sol, contracts/contracts/parallel/AereRollupValidity.sol, contracts/deployments/block-stm-registry.json, and sdk-js/src/addresses.ts.

---

1. Executive summary and honest scope

AERE's parallel-execution work has three distinct pieces at three different levels of maturity. Stating them plainly up front, because the boundary between "shipped" and "roadmap" is the whole point of this document:

1. A real, self-verifying Block-STM parallel executor (shipped, benchmarked). aere-block-stm is a from-scratch Rust implementation of the Block-STM optimistic-concurrency algorithm (Gelashvili et al., arXiv:2203.06871, 2022, the same design behind Aptos and the family Sui and Monad build on). It executes a batch of transactions across many CPU cores and commits a single keccak256 state root. Every run also executes the batch sequentially and refuses to emit a root unless the two results match, so a concurrency bug cannot silently produce a bad commitment. Correctness is proven over 6000 randomized adversarial comparisons with zero mismatches, and measured speedup on a 16-core box is roughly 8.6x to 9.3x on low and medium-conflict batches.

2. An on-chain advisory registry (shipped, live). AereBlockSTMRegistry at 0x98E2C3e615841919d173D8FF642514c5902E8A28 is an opt-in registry where a contract publishes the storage-slot read/write hints for its own function selectors. It is advisory metadata for a scheduler, holds no funds, changes no execution semantics, and has no admin over anyone else.

3. Moving parallelism into base consensus (roadmap, not built). AERE's Layer-1 base layer runs Hyperledger Besu and executes transactions sequentially. Nothing here changes that. Making Besu execute in parallel is a fork of a Java execution client, a multi-month effort, and is explicitly out of scope for what exists today. The registry is metadata for that eventual fork; the concrete parallel path that runs today lives in the rollup layer, off-chain in the sequencer.

The honest one-line framing, taken verbatim from the executor README, is: "Real Block-STM parallel executor, wired into the AERE rollup sequencer; the L1 base layer stays Besu-sequential."

Everything in Section 2 through Section 5 is code that exists and runs. Section 6 (base-consensus promotion) and the fraud-proof / full-EVM items in Section 7 are roadmap and are labeled as such.

---

2. The parallel executor (aere-block-stm)

The executor is a standalone Rust crate under parallel-executor/ with zero external dependencies: the scheduler, multi-version memory, PRNG and keccak256 are all built on std. The build is fully offline and reproducible.

2.1 State and transaction model

The state model is intentionally EVM-shaped but minimal (src/types.rs):

• Key::Balance(account): the native-token balance of an account.
• Key::Storage(contract, slot): a single key/value contract storage slot.

Values are u128; an absent key reads as 0. A transaction version is (txn_index, incarnation), where incarnation counts how many times a transaction has been re-executed after an abort.

The VM understands four transaction kinds (TxKind), each chosen because its control flow or its written values depend on what it reads, so a stale read is silently wrong unless caught:

• Transfer {from, to, amount}: move amount if from can afford it. Control flow depends on reading the sender balance.
• Sweep {from, to}: move the entire balance of from. The written values depend directly on the read balance, a strong serializability probe.
• Increment {contract, slot}: storage[slot] += 1. When many transactions hit the same slot this is the classic all-conflict counter: the sequential result is start + count, and a correct parallel executor must reproduce exactly that despite reordering.
• AmmSwap {contract, x_slot, y_slot, dx}: a constant-product swap against a two-slot pool, modelling a hot DeFi pool that conflicts across concurrent swaps.

Each Txn also carries a gas field, a simulated per-transaction CPU cost measured in keccak rounds. It does not affect the resulting state (correctness is independent of gas); it exists so benchmarks charge each transaction a realistic amount of interpreter-like work instead of measuring only scheduling overhead. This is discussed honestly in Section 4.

2.2 One transaction semantics, two drivers

The single most important design decision is in src/vm.rs: there is exactly one implementation of what a transaction does, execute_txn, and it runs against an abstract VmView trait with two methods, read(key) and write(key, value). All arithmetic is saturating so both paths behave identically and never panic on overflow.

• The sequential oracle (src/sequential.rs) drives execute_txn with a view backed directly by a live HashMap state, applying transactions in strict index order. This is the definition of the correct answer.
• The parallel executor (src/parallel.rs) drives the same execute_txn with a view backed by multi-version memory that records the read set and can signal an abort.

Because both paths share the exact same semantics function, the parallel result can differ from the sequential result only if the concurrency control is wrong. That is precisely the property the correctness harness (Section 3) exercises.

2.3 Multi-version memory (MVMemory)

src/mvmemory.rs keeps, for every Key, an ordered BTreeMap<txn_index, cell>, where a cell is either a concrete Value(incarnation, value) written by a transaction, or an Estimate placeholder (a marker that a lower-indexed transaction previously wrote here but is currently being re-executed, so its value is not trustworthy).

A read by transaction i for key k returns the entry of the highest transaction j < i that wrote k (versions.range(..txn_idx).next_back()). That is what makes speculative parallel execution serial-equivalent: transaction i always observes the writes of all lower-indexed transactions that have run, and nothing from higher ones. If the highest lower entry is an Estimate, the read returns Blocked(j) and the reader must abort and park on j.

Storage is sharded across 256 independent mutexes keyed by an FNV-1a hash of the Key, so non-conflicting transactions touch disjoint locks and run without contention. Hot keys (the shared counter, the AMM pool) serialize on one shard, which is the honest source of the high-conflict slowdown, not an implementation defect.

Three operations matter:

• apply_write_set installs a completed incarnation's writes, deletes stale entries the previous incarnation wrote but this one did not, and returns whether the set of written keys changed (Block-STM calls this "wrote a new path" and uses it to trigger suffix revalidation).
• mark_estimates converts an aborted transaction's writes into Estimate placeholders the instant it is validation-aborted, so any concurrent higher transaction that reads one of those keys blocks on it instead of consuming a soon-to-be-wrong value.
• snapshot_final materializes the committed state once the block is fully executed and validated: base overlaid with, for each key, the value written by the highest transaction index that wrote a concrete value.

2.4 The collaborative scheduler

src/scheduler.rs implements the Block-STM collaborative scheduler (Algorithm 3 of the paper). It hands worker threads two kinds of task:

• Execution tasks: run incarnation k of transaction i.
• Validation tasks: re-check that an already-executed transaction's recorded read set is still valid against current MVMemory.

Two atomic cursors, execution_idx and validation_idx, drive it. Validation of a lower index always takes priority over execution of a higher one. When a transaction aborts (validation fails) or writes a new path, the relevant cursor is decreased so the affected range is revisited, and a decrease_cnt counter is bumped. The block is done only when both cursors have passed the end, no task is in flight, and no decrease happened during the check.

Dependencies are handled without condvars: when execution of i reads an Estimate from j, i is parked on j's dependency list and its task dropped; when j finishes it re-arms i (back to ReadyToExecute) and decreases execution_idx so a worker re-picks it. add_dependency holds the blocking transaction's status lock across the whole check-then-park critical section, so j cannot transition to Executed and drain its dependency list between the check and the push (which would lose the wakeup). try_validation_abort uses a compare-and-set on status so only one validator can abort a given incarnation.

The src/parallel.rs driver spawns num_threads workers, each looping next_task -> run -> feed result back -> repeat, until the scheduler reports Done. It tracks aggregate stats: total_executions (>= num_txns; the excess is re-executions), total_validations, and total_aborts.

2.5 keccak256 state root

src/keccak.rs is a from-scratch keccak256 (standard Keccak-f[1600], rate 1088 bits, original Keccak 0x01 .. 0x80 padding, i.e. keccak256 not SHA3-256), verified in tests against the known vectors for "" and "abc". state_root folds a materialized state into one 32-byte root by sorting non-zero entries into a canonical order (balances tagged 0x00, storage tagged 0x01, big-endian fields) and hashing. The canonical sort makes the root deterministic and independent of hashmap iteration order, which is exactly what a rollup state commitment requires, and it is EVM-compatible with zero external crates.

---

3. Correctness guarantee (proven, not asserted)

The whole point of Block-STM is that the parallel result must equal sequential execution. The harness (aere-block-stm harness, in src/main.rs) proves it empirically.

It runs 6 workload profiles x 200 randomized batches x {1,2,4,8,16} threads = 6000 comparisons (1200 distinct randomized batches, each compared at five thread counts) and asserts the Block-STM committed state equals the sequential oracle for every single case. The profiles include deliberately adversarial fully-conflicting cases: all-hot (400 transactions, 200 hot accounts, conflict probability 1.0) and brutal-single-counter (300 transactions on 64 accounts, one contract, one slot, conflict probability 0.9). Reported result:

  total (batch x threads) comparisons: 6000
  mismatches            : 0
  RESULT: PASS -- parallel Block-STM == sequential on ALL cases

Workloads are generated by a seedable SplitMix64 PRNG (src/workload.rs) with no external RNG crate, so any failing case can be replayed exactly. The default batch mix is 60% transfers, 15% sweeps, 15% shared-counter increments, and 10% AMM swaps.

cargo test --release adds targeted unit checks:

• shared_counter_is_serialized: 500 transactions all incrementing one slot yield exactly 500 at 1, 2, 4, 8, and 16 threads (naive parallelism would lose updates).
• parallel_equals_sequential_random: 40 seeds x 4 thread counts.
• keccak_known_answer: keccak256 of "" and "abc" match the standard vectors.

---

4. Measured speedup (honest numbers)

Measured on the AERE infra box: 16 physical cores, Linux 6.8, Rust 1.96. 20,000 transactions per batch, each carrying a simulated EVM execution cost of gas = 80 keccak-rounds. Speedup is versus the single-threaded sequential oracle, median of 7 runs. Reproduce with cargo run --release -- bench.

Workload	2 cores	4 cores	8 cores	16 cores	abort % @16
Low-conflict (~2% hot-touch)	1.78x	3.22x	5.71x	8.59x	0.3%
Medium-conflict (~15%)	1.87x	3.54x	6.24x	9.31x	1.9%
High-conflict (single hot counter, 15% of txns)	1.70x	3.04x	4.87x	4.62x	24%
Pathological (every tx hits ONE slot)	0.85x	0.85x	0.82x	0.73x	19%

The "8 to 10x" headline is the low and medium-conflict range (8.59x and 9.31x at 16 cores). The honest reading, including the cases that do not look good, is stated in the README and repeated here because it is a credibility asset:

• Low and medium-conflict batches scale nearly linearly.
• The high-conflict batch still reaches roughly 4.6x because only about 15% of its transactions touch the single hot slot; the rest are independent transfers.
• The pathological batch (every transaction increments the same slot) has zero available parallelism. Block-STM correctly serializes it and is actually slower than sequential (0.73x) because it pays for aborted speculation. This is the expected worst case: no correct parallel executor can beat sequential when there is nothing to parallelize.
• There is also a floor case: if transactions do near-zero compute (gas = 0), concurrency-control overhead dominates and the parallel executor is slower than sequential. Real EVM transactions are compute-heavy (opcode interpretation, keccak, ecrecover), which is the regime Block-STM targets, which is why the benchmark charges a realistic per-transaction cost. Run bench 0 to see the floor and bench 200 for heavier transactions.

These figures are single-machine microbenchmarks of the executor. They are not a network throughput claim, not a TPS figure, and say nothing about AERE L1, whose base layer remains Besu-sequential.

---

5. Integration into the rollup sequencer

The concrete place this executor runs today is AERE's rollup layer, off-chain in a sequencer.

5.1 The wrapper

rollup-sequencer/src/blockstm/BlockSTMExecutor.ts exposes:

executeBatch(base: Map<StateKey, bigint>, txns: RollupTx[], opts?) : BlockSTMResult

It has no npm dependencies (only Node's built-in child_process). It serializes the prior L2 state slice and the ordered batch into the executor's line protocol and shells out to the aere-block-stm execute subcommand, which runs the batch across cores, computes the new state, and returns a keccak256 state root as JSON. The execute path runs the batch both in parallel and sequentially and sets parallelEqualsSequential; the binary exits non-zero and the wrapper throws if they diverge, so a bad root can never reach the chain. BlockSTMResult carries { stateRoot, numTxns, threads, executions, validations, aborts, parallelEqualsSequential, state }.

The execute stdin wire protocol (src/main.rs) is one record per line: THREADS n, GAS n, BASE <key> <value> (key is B:<acct> or S:<contract>:<slot>), then TX TRANSFER|SWEEP|INC|AMM .... Output is a single JSON object with the committed root and self-check.

5.2 The on-chain commitment flow

The rollup layer is AERE's Rollup-as-a-Service stack. The canonical factory is AereRaaSFactoryV2 at 0xB7F8c754AC3155197d76f01857172bBd5a5F39Ab (bond token WAERE 0x7e84d7d66d5da4cfE46Da67CDEeB05B323e1f5e8, sink AereSink 0x69581B86A48161b067Ff4E01544780625B231676), which deploys a fixed AereRollupSettlementV2 instance per registered rollup. That per-rollup settlement contract, not a single global address, is where a sequencer proposes roots. Its call is proposeStateRoot(uint256 epoch, bytes32 root, bytes32 l2BlockHash), callable only by the rollup's sequencer, with epoch == latestEpoch + 1, followed by an optimistic challenge window.

The flow is therefore:

batch of L2 txns -> BlockSTMExecutor.executeBatch() -> { stateRoot, ... }
                                                          |
                                                          v
                    settlement.proposeStateRoot(epoch, stateRoot, l2BlockHash)

Because the executor guarantees the parallel result equals sequential execution, the committed root is deterministic and reproducible, which is what makes it safe as an optimistic-rollup commitment: a Phase-2 fraud proof would re-derive it by re-running the same deterministic executor over the disputed batch. See rollup-sequencer/src/blockstm/example.ts for an end-to-end run.

5.3 Integration status (honest)

• Executor: real Block-STM, correctness-proven, benchmarked. Shipped.
• Integration: the execute JSON boundary is live and tested; the TypeScript wrapper drives it and returns a root ready for proposeStateRoot. Shipped as the batch-execution core.
• Not yet built: the wiring from a live L2 mempool into executeBatch (this module is the batch-execution core the full sequencer loop plugs into, not a running production sequencer), and an on-chain optimistic fraud-proof verifier (settlement is optimistic Phase 1 today). These are roadmap.

---

6. The on-chain hint registry: AereBlockSTMRegistry

6.1 Purpose

For Block-STM to schedule efficiently it benefits from hints about which storage slots a transaction is likely to read or write; a good hint reduces the abort rate. Solidity does not expose this metadata. AereBlockSTMRegistry (contracts/contracts/parallel/AereBlockSTMRegistry.sol) is an opt-in registry where a contract publishes its likely read/write set at the function-selector grain, so an eventual scheduler can consult it. It is advisory only: hints cannot change execution semantics, and a scheduler always re-validates the actual read/write set after execution. A wrong hint costs at most one extra abort; it can never make a transaction succeed when it should fail.

6.2 Data model and API

• SlotHint { bytes32 slotPrefix; Mode mode; } where Mode is READ, WRITE, or READWRITE.
• HintSet { address publisher; SlotHint[] hints; uint64 publishedAt; }, stored per (target, selector) key keccak256(abi.encodePacked(target, selector)).

Functions:

• publish(address target, bytes4 selector, SlotHint[] hints): publish or replace the hint set. It caps hints at 64 (TooManyHints above that) and replaces rather than appends.
• clear(address target, bytes4 selector): clear the set; publisher only.
• hintsFor(target, selector) -> (publisher, hints, publishedAt): the view a scheduler reads.
• publisherOf(target, selector) -> address.

6.3 Access control (audit fix HIGH #27)

The first publisher for a (target, selector) must be the target itself or the target's Ownable owner, checked best-effort via a staticcall to owner() (_tryOwner). This closes a front-run squatting vector: without it, an attacker could publish poisoned hints for a victim contract's selector before the real owner did. After the first publisher claims a key, only that publisher can update or clear it (NotPublisher otherwise). There is no admin override and no Foundation gate: the contract author knows its own access pattern best. The registry test (contracts/test/block-stm-registry.test.js) exercises the claim rule, the replace-not-append behavior, the 64-hint cap, and publisher-only clear.

6.4 Deployment facts (live)

• Address: AereBlockSTMRegistry 0x98E2C3e615841919d173D8FF642514c5902E8A28 (roadmap item #33).
• Deploy tx: 0x5646445f5a6c8a3d9d06db7a6becd95f6227608b64291319fd19d7d928beee72, block 8877969.
• Deployer: 0xbeB33D20dFBBD49eC7AC1F617667f1f02dfd6465. Code size 1753 bytes. Deployed 2026-07-10.
• Read-verified at deploy: hintsFor(unpublished) returns (0x0, [], 0) and publisherOf(unpublished) returns the zero address.
• No owner, no admin, holds no funds.

6.5 What the registry does NOT do (honest scope)

The AERE L1 (Besu) executes sequentially today. The registry is metadata for the eventual Block-STM Besu fork; it does not make L1 parallel, and publishing hints has no effect on how any transaction executes on mainnet right now. Its hintsFor(target, selector) view is intentionally shaped to be the interface a future consensus precompile (planned at 0x103 in the Phase-2 Besu fork) would read. That precompile and fork do not exist. The contract is deliberately inert-but-ready: real, correct metadata storage today, waiting for a consumer that is roadmap.

---

7. Related on-chain anchor: AereRollupValidity (validity proofs of the same executor)

There is a second, stronger settlement path already anchored on-chain for the exact same executor. AereRollupValidity at 0x38772063572DF94E90351e44ccbBEefD5F497fbd verifies SP1 zkVM Groth16 validity proofs that AERE's deterministic rollup executor transitioned a prior state root to a new one by applying a specific batch. Its PROGRAM_VKEY binds the SP1 verification key of the guest ELF that ports execute_txn, keccak256 and state-root folding verbatim from parallel-executor/src/{vm,keccak}.rs, so the proven computation is byte-identical to what the production aere-block-stm binary self-checks and what BlockSTMExecutor commits. Public values are exactly 160 bytes: abi.encode(chainId, prevRoot, postRoot, batchHash, numTxns). Submission is permissionless; an epoch is final the instant its proof verifies (no challenge window), gated by a prevRoot == latestRoot continuity check. It routes through the SP1 gateway SP1VerifierGateway 0x9ca479C8c52C0EbB4599319a36a5a017BCC70628.

Per the SDK registry, epoch 0 has already been recorded from a real proof (submitEpoch 0xd4f8d858...12a4b0), advancing latestRoot to 0x256fa277...25e3ac with epochCount = 1, under an immutable vkey 0x00c39737...84ad (these three values are cited in the abbreviated form they appear in sdk-js/src/addresses.ts).

Honest scope, stated in the contract itself: this proves the state transition of AERE's current rollup executor, whose VM implements a bounded set of transaction kinds (Transfer, Sweep, Increment, AmmSwap) over a balance/storage map. It is a real validity proof of that executor. It is not a proof of arbitrary EVM bytecode. The path to a full validity rollup is replacing the executor's VM with a real EVM (revm) inside the zkVM, a separate multi-quarter effort. No claim built on this contract should imply it proves a full EVM.

---

8. Roadmap: moving parallelism into base consensus

What exists today is a proven off-chain executor plus a live on-chain hint registry. Promoting Block-STM into AERE's base consensus is the roadmap, and it needs work that is not done:

1. The Besu execution-client fork (the hard blocker). AERE L1 is Hyperledger Besu, a Java client that executes block transactions sequentially. Running Block-STM at L1 requires forking Besu's transaction-processing pipeline to execute speculatively with multi-version memory, validation, and abort/retry, then reconcile with QBFT block production. This is a multi-month Java engineering effort and is not mainnet-active. A first prototype now exists and is benchmarked: a parallel state-root commit was added to the forked Besu client's Bonsai world-state (aerenew/parallel/, parallel_commit.patch), self-checked to produce a commit root bit-identical to the sequential-commit root across independent-transfer, commit-heavy SSTORE, deletion, and mixed workloads. The honest finding is a boundary, not a win: the commit phase is allocation and memory-bandwidth bound, so adding commit worker threads (widths 2 to 16) is slower than a single worker, not faster. The improvement that does hold is a single-threaded commit rewrite that is roughly 2x cheaper than the stock parallel-trie commit (for example, on medium tries about 316.98 ms stock versus 142.38 ms at width 1); end-to-end, Block-STM execution plus the width-1 commit gives about 1.27x to 1.42x on commit-heavy load versus stock sequential exec plus sequential commit, and the real execution-side speedup remains the ~8.6x to 9.3x of Section 4. A network throughput win is gated on real execution-heavy load. This is a research/benchmark control surface (a process-global flag), not a mainnet-active feature; until the full pipeline lands, the registry's 0x103 precompile consumer does not exist and hints have no runtime effect on L1.

2. Fraud proofs for the optimistic rollup path. The per-rollup AereRollupSettlementV2 settlement is optimistic Phase 1: propose plus challenge window, no on-chain fraud-proof verifier yet. The deterministic executor makes a fraud proof constructible (re-run the same binary over the disputed batch), but the on-chain verifier is not built.

3. Full-EVM validity. AereRollupValidity proves the bounded four-kind VM, not arbitrary EVM. A full validity rollup needs a real EVM inside the zkVM.

4. Production sequencer loop. The live-mempool -> executeBatch -> proposeStateRoot loop is not wired end-to-end; today the module is the batch-execution core, not a running sequencer.

---

9. Honest limitations summary

• The AERE L1 mainnet executes sequentially on Besu. No parallel execution runs at L1 today, and the hint registry, while live, has no L1 consumer yet.
• The parallel executor's VM is a bounded four-kind model (Transfer, Sweep, Increment, AmmSwap) over a balance/storage map, not a full EVM. Its correctness proofs and validity proofs are proofs of that executor.
• The measured speedup figures are single-machine microbenchmarks (16 cores, gas=80), not a network throughput or TPS claim.
• The rollup settlement path is optimistic Phase 1 with no on-chain fraud proof yet; the validity-proof anchor covers only the bounded VM.
• Broader chain caveats apply and are not hidden: AERE runs seven validators under one operator, a single client, and has had no external audit; usage is thin. The self-verifying executor and the advisory registry are real and proven, but they sit on that foundation.

---

10. Address reference (verbatim, chain 2800)

Contract / entity	Address
AereBlockSTMRegistry	0x98E2C3e615841919d173D8FF642514c5902E8A28
AereRollupValidity	0x38772063572DF94E90351e44ccbBEefD5F497fbd
AereRaaSFactoryV2	0xB7F8c754AC3155197d76f01857172bBd5a5F39Ab
AereSink	0x69581B86A48161b067Ff4E01544780625B231676
WAERE (RaaS bond token)	0x7e84d7d66d5da4cfE46Da67CDEeB05B323e1f5e8
SP1VerifierGateway	0x9ca479C8c52C0EbB4599319a36a5a017BCC70628
Foundation (owner of Ownable contracts)	0x0243A4f47D44b40b65D33f20329dE20D00c6f3C3
Deployer (registry / validity)	0xbeB33D20dFBBD49eC7AC1F617667f1f02dfd6465

AereRollupSettlementV2 instances are deployed per rollup by AereRaaSFactoryV2, so no single global settlement address exists; the sequencer commits to the per-rollup instance's proposeStateRoot.