aere-network/aere-node
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Patch 0009: the chain-2800 fleet distribution of 2026-10-02 (heard-seal judge and pending-commit buffer, both acting only from the AIP-22 stage 4 height; snap-request cap, GHSA-pcv4-pxhv-99m7); fifteen 4(b) notices of patch 0008 now name patch 0008; run-node.sh applies 0009; anchor corpus restaged from the recipe; measured: the recipe build equals the fleet distribution class for class

Browse Source
This commit is contained in:
Aere Network 2026-10-09 14:25:20 +03:00
parent c05b2ba97b
commit 76a70f6422
36 changed files with 10338 additions and 55 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
LICENSING.md
View File

@ -123,7 +123,7 @@ the section 4(b) notice of changed files), and each keeps its original notice pe
<!-- GENERAT: lista de mai jos se rescrie de unealta de licentiere din depozitul de constructie -->
## anchor/: every file that carries someone else's copyright (generated 2026-10-05)
## anchor/: every file that carries someone else's copyright (generated 2026-10-09)
This list is **generated from the files this package actually ships**, not written by hand: the
previous hand-written table fell behind the tree on 2026-09-12 and seven upstream files went
@ -132,9 +132,9 @@ repository, and the publisher refuses to ship a package whose list disagrees wit
You do not need that tool to check the list: every row below names a file in this package, and
`grep -i copyright` on it shows the same line.
Measured 2026-10-05 over `anchor/`: **288 files**. **138** carry someone
else's copyright alone, **68** carry a shared notice naming both upstream and us, and **38** name only
`contributors to the AERE Network`; 44 carry no copyright line at all.
Measured 2026-10-09 over `anchor/`: **299 files**. **145** carry someone
else's copyright alone, **68** carry a shared notice naming both upstream and us, and **41** name only
`contributors to the AERE Network`; 45 carry no copyright line at all.
The first two groups are the table below. A shared notice is listed too: those are precisely the
upstream files we changed, so section 4(b) has something to say about each of them.
Every file below is Apache-2.0 material redistributed under section 4: it keeps its original
@ -145,6 +145,8 @@ notice (4(c)), and where we changed it the words **modified by Aere Network** ar
|---|---|---|
| `anchor/app/src/main/java/org/hyperledger/besu/cli/BesuCommand.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/app/src/main/java/org/hyperledger/besu/cli/options/AerePqEmergencyOptions.java` | `Copyright contributors to Besu / AERE Network.` | upstream, **modified by Aere Network** |
| `anchor/app/src/main/java/org/hyperledger/besu/cli/options/SynchronizerOptions.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/app/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/app/src/main/java/org/hyperledger/besu/controller/PqTipAdoptionGate.java` | `Copyright contributors to Besu / AERE Network.` | upstream, **modified by Aere Network** |
| `anchor/app/src/main/java/org/hyperledger/besu/controller/QbftBesuControllerBuilder.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/app/src/test/java/org/hyperledger/besu/cli/options/AerePqEmergencyOptionsTest.java` | `Copyright contributors to Besu / AERE Network.` | upstream, **modified by Aere Network** |
@ -334,12 +336,17 @@ notice (4(c)), and where we changed it the words **modified by Aere Network** ar
| `anchor/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/AereD433BftWithdrawalsTest.java` | `Copyright contributors to Besu.` | upstream |
| `anchor/ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/feemarket/LondonFeeMarketAereFloorTest.java` | `Copyright contributors to Hyperledger Besu.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManager.java` | `Copyright contributors to Hyperledger Besu.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapServer.java` | `Copyright contributors to Hyperledger Besu.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/BlockPropagationManager.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/DownloadHeadersStep.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/PropagatedBlockAdoptionGate.java` | `Copyright contributors to Besu / AERE Network.` | upstream, **modified by Aere Network** |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapSyncConfiguration.java` | `Copyright contributors to Hyperledger Besu.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransaction.java` | `Copyright contributors to Hyperledger Besu.` | upstream |
| `anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/AereD434SnapCapTest.java` | `Copyright contributors to Besu.` | upstream |
| `anchor/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManagerTest.java` | `Copyright contributors to Hyperledger Besu.` | upstream |
| `anchor/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java` | `Copyright ConsenSys AG.` | upstream |
| `anchor/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseTransactionPoolTest.java` | `Copyright contributors to Hyperledger Besu.` | upstream |
| `anchor/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayersTest.java` | `Copyright contributors to Hyperledger Besu.` | upstream |

6
README.md
View File

@ -103,9 +103,11 @@ Both patches target upstream commit `d2032017bb3b8cb215a97303980a1e4a643f7180`,
| `patches/0006-aere-pq-transactions.patch` | AIP-20, transaction type `0x50` authorized by ML-DSA | the tree after 0005 |
| `patches/0007-aere-pq-finality.patch` | AIP-21, `aere_getPqFinality` and the on-disk seal window | the tree after 0006 |
| `patches/0008-aere-pq-aip22.patch` | AIP-22 stages 1 to 3 (post-quantum validator ids from block 20,836,228) and the rest of the fleet distribution of 2026-09-29 | the tree after 0007 |
| `patches/0009-aere-pq-seal-judge-snap-cap.patch` | the fleet distribution of 2026-10-02: the heard-seal judge and the pending-commit buffer (both act only from the AIP-22 stage 4 height, not set on chain 2800), the snap-request cap | the tree after 0008 |
`run-node.sh` applies 0001 and 0003 to 0008, in that order; 0002 is for the testnet only. A node
built without 0008 stops at block 20,836,227.
`run-node.sh` applies 0001 and 0003 to 0009, in that order; 0002 is for the testnet only. A node
built without 0008 stops at block 20,836,227. (Note added 2026-10-09: 0009 is not needed to follow
the chain today; it keeps the recipe equal to what the fleet runs, see `RUN-A-NODE.md`.)
Patch 0003 is a different kind of thing from the first two and is described in `anchor/README.md`.
It changes twenty upstream files and adds fifty-five, and it changes one build file by one line.

28
RUN-A-NODE.md
View File

@ -267,6 +267,34 @@ above `ANCHOR_MIN_SEALS_CEILING` in the script and the section on the configurat
refusal. A refusal names the block and the
rule that refused it.
## Patch 0009: the fleet distribution of 2026-10-02 (2026-10-09)
Since 2026-10-05 the Besu nodes of chain 2800 run a distribution built from the tree of patch 0008
plus two changes; `patches/0009-aere-pq-seal-judge-snap-cap.patch` brings the recipe to that exact
tree. **A node built from patches 0001 and 0003 to 0008 still follows the chain**: neither change
alters what a node accepts below the AIP-22 stage 4 height (`aere.pq.finalityBlock`), which is not
set on chain 2800 and which `run-node.sh` does not set. The two changes:
- the heard-seal store and commit messages that arrive before the proposal: from the stage 4 height
a seal that does not verify never displaces one that does, and an early commit waits in a bounded
buffer until it can be validated against the proposal; below that height both paths run the code
of patch 0008;
- a cap on the snap-protocol requests a node serves at once, per peer (default 8) and in total
(default 200), ported from upstream Besu (advisory GHSA-pcv4-pxhv-99m7).
The patch also corrects the licence notices that patch 0008 brought into fifteen files: they named
patch files that are not in this package, and now name patch 0008 (only that line changes).
Measured 2026-10-09: patches 0001 and 0003 to 0009 applied with plain `git apply` on a pristine
`d2032017` checkout reproduce the patch generator's tree with zero differences; the build has the
same 3,557 classes as the fleet distribution of 2026-10-02, none only on one side, and **no class with
different code** (`javap -p -c` on the 45 pairs whose bytes differ: line-number tables only). The
build without 0009 lacks the classes of the first change, so the comparison tells the two apart. A
follower built from this recipe, restarted on its own database at the head of the chain, kept pace
with it: four readings a minute apart, the height rising each time, at most one block behind the
public endpoint, ten peers. **If you built a node before, `run-node.sh` rebuilds it**, as with every
change of the patch set.
---
## Sizing the JVM heap: bigger is not faster

2
anchor/BASE.txt
View File

@ -6,7 +6,7 @@ against a newer base, 26.7.1: 97 lines would be lost, among them the registratio
and /readiness probes in BesuCommand.java, and nothing would fail.
Since 2026-10-01 this directory holds every source file that the published patches (patches/0001 and
0003 to 0008, in the order run-node.sh applies them) change or add, taken from the recipe itself:
0003 to 0009, in the order run-node.sh applies them; 0009 added 2026-10-09) change or add, taken from the recipe itself:
the base tree with those patches applied. It is a reading copy; the patches remain the authoritative
form, and a check in the operator repository compares the two file by file.

47
anchor/MANIFEST-sha256.txt
View File

@ -5,6 +5,8 @@ computed them, and the same 276 files once produced two different aggregates her
329e4e2f93143cc8af092f0a951fa23619c12c4ec1793b053654f8e26e088da0 app/src/main/java/org/hyperledger/besu/cli/BesuCommand.java
3f289d74bb1878280a99742320d7a27c19146a2f19cdf6a0857f2aad4361ad5e app/src/main/java/org/hyperledger/besu/cli/options/AerePqEmergencyOptions.java
100dc02ffb21e3d1946f55f49d3c505da5a2d8f49e1e151c044ac249e7151cf9 app/src/main/java/org/hyperledger/besu/cli/options/SynchronizerOptions.java
e034c726fe15be38fe85a9a2543ae5b61ca31fe39df5046d27f6f600ca567900 app/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java
3676cf2ef881b81437ddec15bf6b8f8c393a5a6459f77179898703c289c9f8bc app/src/main/java/org/hyperledger/besu/controller/PqTipAdoptionGate.java
feaaa439fc80af14d766683231d46e5b2aa12aa6e651ed83f7ad8e9507fa1413 app/src/main/java/org/hyperledger/besu/controller/QbftBesuControllerBuilder.java
129493150c0b13e8020bdf50c0c1ae07268d4f73a69a821b303bf4894451fa5e app/src/test/java/org/hyperledger/besu/cli/options/AerePqEmergencyOptionsTest.java
@ -38,7 +40,7 @@ f67a205036550f7d0f1bfed2abe7b89abd73e80614adff8d696b33c580269a6f consensus/comm
660ba13ad60c49e86cf9ee4ced2b6932a5422d988f85c5c23098a92b029f4d25 consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqRegistryHashTool.java
11f1c2a97a40df999471977cb3eab9af4e816081b100e345e8c962ef3bbf76f5 consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqRegistryV3.java
2ee139d21f73ff3417f6ad61b233e5ca3a95f92e68a4284eb99a47b1f8f4b58b consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSchemeSchedule.java
80104994f8e5f02353c4bcb25084c451fd422c51c49e6891b574d76c7adf135f consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealCache.java
a6abe7ed223749bec04c21053bb4b99bf9d82d2d972ba62355ef1a8059375e4a consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealCache.java
73abbaddcec1533a7a758f1c4a26ae68a83ddbb96fd98ad147e74d7386bbbf3f consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealStore.java
776d9a422bcd306ad92d561ddebbbc1835d4ef5b147aa19d781556298bb27be4 consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealWindowStore.java
fad72fb52b6f6198113b80f5f2729002066a10decf596cfe26e602b058d9c603 consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealed.java
@ -47,11 +49,11 @@ addba40c0d931a3ecfa3b2f0179311dacaa604884e3ebb1d79f958bd913f72e2 consensus/comm
285f6a7c1188a387fb1ac21f5ab809595ef48d936b40aced94047525e6def0a6 consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/SealScheme.java
3b57840dc103063b3797697feb78b90af93ca123587e9278cfb43b84edbd995a consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/SealSchemes.java
444961503d515cb9f045729577e190941e59baa4bd90438a2da05f41fed7750a consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/SlhDsaSealScheme.java
9021d608f62e1330cf9b421013a06f2141bae469ba9162a24ce66f749aa4b21a consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftBlockCreator.java
c2fc665fc0eeab95efbeb6de445971c5efd1be7f6c34a009d27754645913a56d consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftBlockCreator.java
454eeca487505e70c7f0000f5401b6bf4530da1d5ce6bcb803427f1d08e3cb1c consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftBlockCreatorFactory.java
610275a18c2519d84ffb91982d7b9006a42605bbb330071572859df49f183c2a consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftMiningCoordinator.java
f6c9f8567486d212f83f324c013e52239ec4e0c494d69daa17e36efafa1bf5fd consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftMiningCoordinator.java
e89deb6f346c221affa717eafa464d497401b7b5a01c84c144dace42032879a9 consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/PqAnchorProducer.java
d8efb4d625233e18ef4764eaa29650a97b56e75af18d07607eda1dd484187f76 consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/messagewrappers/BftMessage.java
9ee4a0fce4ebd6da3bfb0f530a06435562002985ec456a148a21266c230eb40d consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/messagewrappers/BftMessage.java
4b0731ec053e313a9ea9afc78592f25b6e97e6d230ebb273532437f08da92e5a consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/network/ValidatorPeers.java
478f9be60c1259481bfd7b5a687e23ada29b4b07d4cdf6f9d3bbc13584d384f8 consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/payload/SignedData.java
1e0113154d79a91a07104bcc67a9a843847751b921c5f9b7cc96d7bc30a1452f consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/slhdsa/ADRS.java
@ -81,6 +83,7 @@ ed1cc6f617dc5a2f7801b6e8f6f039a035a2ffc9542b116262dac39bb70563dd consensus/comm
8aff8079c93d1463111e303831dbef886568790e2cff8ade77801a2864ae355b consensus/common/src/main/java/org/hyperledger/besu/consensus/common/validator/blockbased/VoteTally.java
5a1014676efeea6e29d0603b4e7dac2d7334840c2f1ba3885bdc1b13e337cd7c consensus/common/src/main/java/org/hyperledger/besu/consensus/common/validator/blockbased/VoteTallyUpdater.java
cf0dcc008ce2d040590329088abdcbfc0829bf529445422deee8022d4856e08a consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AereAnchorProposalDelayTest.java
76f01dce55a7816c1526d15855b0aa55ae725162278c0e31221c0981d73bdbc4 consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AereD454SealCacheTest.java
f5d6d26b9cd3babad8f7d16b9595497908b1f854d166837fbecd485d721ae856 consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AereKeysFromV3Test.java
1d7e3a30d3f4653b778a1bcbb81203ed8855d904017fa438a06e26baad8958c3 consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AereLocalKeysV3Test.java
27e5782c1742a361db7a22269ac44c1af574e25b07a87c76c10792bc9741b65c consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AerePqFinalityAuthorTest.java
@ -130,19 +133,20 @@ ae37d3525697bf84defd796bb8813193c78f81e7fa24a72f8f048285d52399e2 consensus/comm
d0ba00d95427ae4fb1c0a1eeb37bcd039fbfa5412957dc4b2da0725c0dea5028 consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/slhdsa/SlhDsaParallelSigningTest.java
bcdd5c3c267f38eee9700bc6ba07d41b487c7889224820868e1839044108451f consensus/common/src/test/java/org/hyperledger/besu/consensus/common/validator/blockbased/VoteTallyIdentityTest.java
6b6ecfcfb9b25bb009cb5d4df74648c729abf62626e5034af126771825b7418e consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/Commit.java
5ed47d976d4aafbb55369e1b3e96e6ad5efc3bf7c68be99ddb0e0ef9dbbc2f32 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/Proposal.java
394c557de273fa9fa9a5ad024e8a1982575ad756736aa045ffeb0bdc7dc8016c consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/RoundChange.java
556b7004b8ddb9aab4fc5132b3ffdaced949ab25b8549464378e5f07009383b4 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/Proposal.java
1e513b520e86e5c8251ec5453a2c7d459b883b75de1621ea44d6571e0ddd927e consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/RoundChange.java
a1405402e891189909a436abd9f83c863349fdcf01d3011542d805e2bb3911ed consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/network/QbftMessageTransmitter.java
35d38accf0d22b6fdadeab4f2bce0cdac708f8115591594b1bb1fa385b227c5c consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/payload/CommitPayload.java
536375fa2d9f27107d9907f34391413e052f2a8cd89611033fefe29b9edc7891 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/payload/MessageFactory.java
7914cc3f4a8a85dc7e2e8dad1ea9add86c4df00b04c2f7d822d4fed60a3aadcb consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/payload/PreparePayload.java
8f8b041e72cfaee43d2f4e61a790e603ed35b329cf431de68fc6907a18c3b824 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/payload/ProposalPayload.java
e19b12e6db33cfb9bb1c2f22e321815130dfbcc5730bb505320f08a45a371093 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/payload/RoundChangePayload.java
406a70cc38b5e22c84da3803781baa920223c7ebe8ca44b4ef1ec6b40e416f2c consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454PendingCommits.java
51eddf3cb8e37489abd35e6f4c36294a13ec76337bbbbfd54c07584085dceaac consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/QbftBlockHeightManager.java
0a450699c702104dcef6bf8912b4d180f05d647288c75a9f78da6af3bd3d90f1 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/QbftController.java
4a87188f209904c19f2b393fd50559a05b2be3161b4582364bc1641002bb4dff consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/QbftRound.java
324a5607313f25adb377d633be1f221588f8dafaa47a3c9d1e84fc5697323090 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/QbftRoundFactory.java
4494f33ca028b60eaa093e6c24dd13706461e228f7f5a29e40e8917d1ef3b373 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/RoundState.java
13c43bc993a4501357d5f54c3d406cd4433a9f0645c2e71cf190f0a6be6d6c9f consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/RoundState.java
eadc4a14e2f0fdf7c1d9667d67c59f709af7a86fccf51923de0248b73e223c21 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/types/QbftBlockCreator.java
4c0811d182f5252bdde31560f514a518db58c8d0c82c748a6c0f5d68472470c6 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/validation/CommitValidator.java
a40621537e2b2024ab5586c9001f96fd81ffe3c3d5ce1e867a07a77584cf2173 consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/validation/MessageValidator.java
@ -161,10 +165,12 @@ e2107737579c16b73cbd2c9b50e90abcd6c910309286fb862cef780555ca43a5 consensus/qbft
fe25255de4b5fa6d86412fa4c48218180f549f0a4ffea5b5cc0a7d19a5be4cd9 consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/payload/PreparePayloadPqTest.java
a994d5638940ea90f37ed6f3ed5eebc24781707f2f4555c7a079d1251ddbf893 consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/payload/ProposalPayloadPqTest.java
df5d7d2119cf0864a8c2dce50f4f69bd8b64804cfda7f7366c013b304af0a6e7 consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/payload/RoundChangePayloadPqTest.java
d3f3315851a80e4e910de56e085e737b12cda7e8d5597cc704157986af23bfcd consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454RoundStateTest.java
0c78dcd73a795f9d2736910ee695af35616c7bf57f3463fdd9eccb82f7e12322 consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/PqCommitSealsSignedOnceTest.java
97ac1e46ab37238a2b6232eccdd4636ac6fa928c8e3d4f8a473c083c5d5c3b39 consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/PqHybridExtrasNeverThrowTest.java
0bb2c1e31a7f29e9b07fb7030567746fc6d79460ed4739d5ecafac44501f7bb5 consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/PqLateSealSalvageTest.java
127de5fbe8014768a657f31bc77d7e66557273febdf3412a5536e8628fd70c5c consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/PqSealsWithoutProposalTest.java
35da5f6cc2d2585aa4afc26e0774b92a0247ae9fc6dd9934b60db08397053ba0 consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/validation/AereD412Fixture.java
b1a6fdc177ccca180afdb20a7946ee33437d47a12cd6f5aade6e02238213bcf9 consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/validation/AereD434RoundCapTest.java
c1b31f5c357a1966cdd1e963369a1eb3078a381679c6adc8fb5206f754eca84b consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/validation/AereFinalityMessageTest.java
70bc4d5242e5f5560333c862bef0b2d5167649aabb3cd66fe61a101678d30b4d consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/validation/AereFinalityVectorsWriter.java
@ -231,7 +237,7 @@ dfceadc9d220ef9b028ef30e09e12a2c44caffee1922b6aa98c95937862bc9f9 ethereum/api/s
cd94a4c059dd5e79136544f9dc09163df722ecedd41b8552cd0cdc22ba6328d3 ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetTransactionCount.java
ce03332291e5a575cede22e2885e561ae5d27e815551d7d4af3a93045949eabc ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/results/TransactionBaseResult.java
6a54e53e6f1569b0bc00617188b8d82c6d69fa2de13db6285af0c5e959d1eae7 ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/websocket/subscription/pending/PendingTransactionDetailResult.java
2de0590a917378d7cd1cb2a4ab1a4f9d0ac27545c0e080a83417244718606ba5 ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java
baf53a0731da0d56aebdc87fee8571238de3b054de47abadcb356f18e2867de5 ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java
3ee087fc34b455e113ec9d2c53b5dbdb3574429fcf166f4c968a80120f70f0eb ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/AereStateUnavailableIsAnErrorTest.java
6e99e9f399558abbd1d4fdb4b176ba413bf31cabb3c8aba1701c17ce5828ce8a ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/query/AereHeadStateRetryTest.java
bf6617807ee0dd03ed98e6f1a73d6d11366880c2f742f34e73ef4d74a4f01180 ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/Transaction.java
@ -242,8 +248,8 @@ bf6617807ee0dd03ed98e6f1a73d6d11366880c2f742f34e73ef4d74a4f01180 ethereum/core/
b3f5feddcae2e74f3b2517785a51e61998817fba4977d5f5a3fb9fbb48038af8 ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/AerePqTransactionSupport.java
10df69ba73a9847ea5b93d2fc2bc1c2dbf238b8b76d713568678403ea6edbd87 ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetProtocolSpecs.java
73f66fae03826866c74e87bd262e7bac79ce3db814de782dc52b9263ce156059 ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionProcessor.java
c19d72f93d09f945a6c5a0144125f8501cc0cb0d0749cd08e8d8ca091ad40dcc ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java
f2257781bf894f96f0fd67bc8f5d97fb8df181a95029147a9e594b068ca04cf2 ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/WithdrawalsValidator.java
3ddc8c9e2e26ab3261e79b7adec6546f35d6bd3639d91656be58c8b9ccd2b37d ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java
5d6fbb572cb66fbc39cd63109dd51f566a0dfd43f13b1eb0bbe3cf463f88ce33 ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/WithdrawalsValidator.java
b3c6811983da5ce74f75a0fccdd87a2385881fa99210cdb0484d357ac0d844ab ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/blockstm/AereBlockStmEngine.java
20d842d8413034e506c6ab22288a9e6870f0fb8a83362ec5adecb70b49f6af07 ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/blockstm/AereBlockStmPreprocessing.java
bd5d69c4ec63c212f364fa7fbb3ad658983654e772148bedb17c8028ce24540e ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/blockstm/AereBlockStmProcessor.java
@ -265,25 +271,30 @@ aa4999574b158ed3a9d738d7439033d543ed0a748792e0e60d1eb8ca8af84427 ethereum/core/
816c47aba04623a15e48ece16b440d6d9e16d23031841c22f9159830862d6dd6 ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/feemarket/LondonFeeMarketAereFloorTest.java
d34dd8c46ad4a394e8b15a2155c32f89883a0879c1cf7f1976f081938a3f56d3 ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/parallelization/AereBlockStmBenchmarkTest.java
6f0119bcafb5247995b4399f766d4adc9275669fe3ca08fbe2df5b01800624e3 ethereum/core/src/test/java/org/hyperledger/besu/ethereum/mainnet/parallelization/AereBlockStmSmokeTest.java
f94acc1cd8aac6679c39c74f7d23375ba19781e9bfd1d14b8c8a9e4314384cc1 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java
5b0694c7e5494bded806dc87e7721d05cf81533b28a0792d24ac9fca77606806 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java
3cda590f51dd0234191953f46b6a4365446af0e54f7128c6461240dcb90c60e4 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManager.java
9bfd4580c5e046a8e910e8a2e2a7e538ab3b5ccf59f4ecc9e10f3909cc85fe61 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapServer.java
62abf412b67808f0648b1af363a376e567ba7796cb1c4aa93617adc3f2484b8e ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/BlockPropagationManager.java
3b690b72e80a0f9eaf868a3fa3f99ace42dad74dc30b2c98b9f3ca5a023917c0 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/DownloadHeadersStep.java
7f611966d74f52ef2f6d55bb5882ffb72b10abf1985df2b01f03aad4a6e2baf1 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/PropagatedBlockAdoptionGate.java
5a8187cc12bb1a16a9b77956c4e1c5d8c088a4e65164179c427cd7b2c2f24b7f ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java
9d248b393f952a6d5f16c92c3f12283c899e9ccb2b4f29ec68ca7d795a6b3539 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapSyncConfiguration.java
89822723ef66d5fdc752d6a9be98786023d02fd8170a8261626ac6d68f55d47d ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java
e068fc39b4f645d1ccdd778acdf77983869a73428513c11ce587547391e36ce3 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/PendingTransaction.java
eb8d2305070adcf0424ab9884c8f09d9bc95a1662a4cf8775a136e3a314fa181 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java
76ce97047c7988c106969f69fbdcba6941657c482ea1e060b995e7ce0556c108 ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java
d4709b7514cff40308631cbac52b80cc42406af2c72e1996493c15dadc8bf9e8 ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/AereD434SnapCapTest.java
90297d1f3c4eaf41137f6c13bbe0fbe0bd5f9d5313388d692108b974120ab6a1 ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManagerTest.java
ea521576bf52d75995e2c00941eb455eb65323ab653e7b33d6d506b05ef816e3 ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java
cc003a07cc4cf27e20ebf249ae8d84b8f514d1c24e55dfaab00fea3c345e1a0c ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/BaseTransactionPoolTest.java
55e6c022442a970411bb677a1d0b27a4a7f8174b4fabd200de5b10393731b201 ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/transactions/layered/LayersTest.java
e39a09c49cec66fa2766470ba29bbccced6f42ecb7f95df39ccf5086b666510c ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java
2190cd3df779c0b04ca1ca09098ea1a99df07dab6659a2d4cac058f34c27e495 evm/src/main/java/org/hyperledger/besu/evm/frame/MessageFrame.java
eeeb63160aba1f889725d628b8623630baf227a82e367844b6d24619d8e83e7d evm/src/main/java/org/hyperledger/besu/evm/frame/TxValues.java
918a6c0b8a3e0074af5c7e105f695eee53e0238efb9b2ad2a9dc9377e25a00b8 ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java
f53a2c2b619b9a453d6e810e9211d9d640a0b7a3fe3317e3ce260b4ad918a8b0 evm/src/main/java/org/hyperledger/besu/evm/frame/MessageFrame.java
c33f14d3dfc06a5de6d9da519a0060383806473e354b5716804fe72c4d7284e4 evm/src/main/java/org/hyperledger/besu/evm/frame/TxValues.java
72866e4744d34a120345a1ed3a0699292fd4a63d04d6256b3a7337738441e7bf evm/src/main/java/org/hyperledger/besu/evm/gascalculator/FrontierGasCalculator.java
417134787ef4f01af0674bb8e641a7eef3c701e1d9d4e8a1c807faab4bb97d99 evm/src/main/java/org/hyperledger/besu/evm/precompile/AereFalconSupport.java
c59d8c714155b347fe64bb378715e71670a8bd42180a97d08156e6c20caa359c evm/src/main/java/org/hyperledger/besu/evm/precompile/Falcon1024PrecompiledContract.java
6487810e9b167eb6c0a7046a5226e40936116771efb91013d6c2eb9559c60f97 evm/src/main/java/org/hyperledger/besu/evm/precompile/Falcon512PrecompiledContract.java
42e4d1442d3478eac8c3177e35318872c365a173d4a16f529356e52944901d50 evm/src/main/java/org/hyperledger/besu/evm/precompile/MLDSA44PrecompiledContract.java
d17d9fde238d5d6527380d9829d1b52cd3c7c76c9874c6172e4ddf1dcc26acba evm/src/main/java/org/hyperledger/besu/evm/precompile/MainnetPrecompiledContracts.java
c3b208adc9e81229eb9194fbedd40186aeccf43b8039827604d00169878d6690 evm/src/main/java/org/hyperledger/besu/evm/precompile/P256VerifyPrecompiledContract.java
db7241cbaf605e210d5bb62cf251df92639d7890d9fa683892bafe6c066cde05 evm/src/main/java/org/hyperledger/besu/evm/precompile/MainnetPrecompiledContracts.java
a2adb2d6a5b6b58bd056430664750c1b416c5ba769781833270be783288d083f evm/src/main/java/org/hyperledger/besu/evm/precompile/P256VerifyPrecompiledContract.java
c4ef4c11f13c7387b79c5d6532299f540d8c4cb82c98a865be2d01053bd5d35e evm/src/main/java/org/hyperledger/besu/evm/precompile/SHAKE256PrecompiledContract.java
976daf6471e817ca668942aa07adfc7ede7d59b4c941d0bbb91d02a25fa6fbb4 evm/src/main/java/org/hyperledger/besu/evm/precompile/SLHDSA128sPrecompiledContract.java

670
anchor/app/src/main/java/org/hyperledger/besu/cli/options/SynchronizerOptions.java Normal file
View File

@ -0,0 +1,670 @@
/*
* Copyright ConsenSys AG.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*
* Modifications Copyright contributors to the Aere Network.
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
*/
package org.hyperledger.besu.cli.options;
import static org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration.DEFAULT_SNAP_SYNC_SAVE_PRE_MERGE_HEADERS_ONLY_ENABLED;
import org.hyperledger.besu.ethereum.eth.sync.SynchronizerConfiguration;
import org.hyperledger.besu.ethereum.eth.sync.snapsync.ImmutableSnapSyncConfiguration;
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
import java.net.URI;
import java.util.Arrays;
import java.util.List;
import com.google.common.collect.Range;
import org.apache.tuweni.units.bigints.UInt256;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import picocli.CommandLine;
/** The Synchronizer Cli options. */
public class SynchronizerOptions implements CLIOptions<SynchronizerConfiguration.Builder> {
private static final Logger LOG = LoggerFactory.getLogger(SynchronizerOptions.class);
private static final String BLOCK_PROPAGATION_RANGE_FLAG =
"--Xsynchronizer-block-propagation-range";
private static final String DOWNLOADER_CHANGE_TARGET_THRESHOLD_BY_HEIGHT_FLAG =
"--Xsynchronizer-downloader-change-target-threshold-by-height";
private static final String DOWNLOADER_CHANGE_TARGET_THRESHOLD_BY_TD_FLAG =
"--Xsynchronizer-downloader-change-target-threshold-by-td";
private static final String DOWNLOADER_HEADER_REQUEST_SIZE_FLAG =
"--Xsynchronizer-downloader-header-request-size";
private static final String DOWNLOADER_BODIES_REQUEST_SIZE_FLAG =
"--Xsynchronizer-downloader-bodies-request-size";
private static final String DOWNLOADER_CHECKPOINT_TIMEOUTS_PERMITTED_FLAG =
"--Xsynchronizer-downloader-checkpoint-timeouts-permitted";
private static final String DOWNLOADER_CHECKPOINT_RETRIES_FLAG =
"--Xsynchronizer-downloader-checkpoint-RETRIES";
private static final String DOWNLOADER_CHAIN_SEGMENT_SIZE_FLAG =
"--Xsynchronizer-downloader-chain-segment-size";
private static final String DOWNLOADER_PARALLELISM_FLAG =
"--Xsynchronizer-downloader-parallelism";
private static final String HEADER_DOWNLOAD_PARALLELISM_FACTOR_FLAG =
"--Xsynchronizer-header-download-parallelism-factor";
private static final String TRANSACTIONS_PARALLELISM_FLAG =
"--Xsynchronizer-transactions-parallelism";
private static final String COMPUTATION_PARALLELISM_FLAG =
"--Xsynchronizer-computation-parallelism";
private static final String PIVOT_DISTANCE_FROM_HEAD_FLAG = "--Xsynchronizer-pivot-distance";
private static final String WORLD_STATE_HASH_COUNT_PER_REQUEST_FLAG =
"--Xsynchronizer-world-state-hash-count-per-request";
private static final String WORLD_STATE_REQUEST_PARALLELISM_FLAG =
"--Xsynchronizer-world-state-request-parallelism";
private static final String WORLD_STATE_MAX_REQUESTS_WITHOUT_PROGRESS_FLAG =
"--Xsynchronizer-world-state-max-requests-without-progress";
private static final String WORLD_STATE_MIN_MILLIS_BEFORE_STALLING_FLAG =
"--Xsynchronizer-world-state-min-millis-before-stalling";
private static final String WORLD_STATE_TASK_CACHE_SIZE_FLAG =
"--Xsynchronizer-world-state-task-cache-size";
private static final String RECEIPTS_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG =
"--Xsynchronizer-receipts-download-step-timeout-millis";
private static final String BACKWARD_HEADERS_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG =
"--Xsynchronizer-backward-headers-download-step-timeout-millis";
private static final String BODIES_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG =
"--Xsynchronizer-bodies-download-step-timeout-millis";
// Regular (stable) flag
private static final String SNAP_SERVER_ENABLED_FLAG = "--snapsync-server-enabled";
private static final String SNAP_PIVOT_BLOCK_WINDOW_VALIDITY_FLAG =
"--Xsnapsync-synchronizer-pivot-block-window-validity";
private static final String SNAP_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING_FLAG =
"--Xsnapsync-synchronizer-pivot-block-distance-before-caching";
private static final String SNAP_STORAGE_COUNT_PER_REQUEST_FLAG =
"--Xsnapsync-synchronizer-storage-count-per-request";
private static final String SNAP_BYTECODE_COUNT_PER_REQUEST_FLAG =
"--Xsnapsync-synchronizer-bytecode-count-per-request";
private static final String SNAP_TRIENODE_COUNT_PER_REQUEST_FLAG =
"--Xsnapsync-synchronizer-trienode-count-per-request";
private static final String SNAP_TRANSACTION_INDEXING_ENABLED_FLAG =
"--snapsync-synchronizer-transaction-indexing-enabled";
private static final String SNAP_FLAT_ACCOUNT_HEALED_COUNT_PER_REQUEST_FLAG =
"--Xsnapsync-synchronizer-flat-account-healed-count-per-request";
private static final String SNAP_FLAT_STORAGE_HEALED_COUNT_PER_REQUEST_FLAG =
"--Xsnapsync-synchronizer-flat-slot-healed-count-per-request";
private static final String SNAP_SERVER_MAX_CONCURRENT_REQUESTS_PER_PEER_FLAG =
"--Xsnapsync-server-max-concurrent-requests-per-peer";
private static final String SNAP_SERVER_MAX_CONCURRENT_REQUESTS_FLAG =
"--Xsnapsync-server-max-concurrent-requests";
private static final String CHECKPOINT_POST_MERGE_FLAG = "--Xcheckpoint-post-merge-enabled";
private static final String SNAP_SYNC_SAVE_PRE_CHECKPOINT_HEADERS_ONLY_FLAG =
"--snapsync-synchronizer-pre-checkpoint-headers-only-enabled";
private static final String ERA1_IMPORT_PREPIPELINE_ENABLED_FLAG =
"--era1-import-prepipeline-enabled";
private static final String ERA1_DATA_URI_FLAG = "--era1-data-uri";
private static final String ERA1_IMPORT_PREPIPELINE_CONCURRENCY_FLAG =
"--era1-import-prepipeline-concurrency";
/**
* Parse block propagation range.
*
* @param arg the range such as -10..30
*/
@CommandLine.Option(
names = BLOCK_PROPAGATION_RANGE_FLAG,
hidden = true,
defaultValue = "-10..30",
paramLabel = "<LONG>..<LONG>",
description =
"Range around chain head where inbound blocks are propagated (default: ${DEFAULT-VALUE})")
public void parseBlockPropagationRange(final String arg) {
blockPropagationRange = OptionParser.parseLongRange(arg);
}
private Range<Long> blockPropagationRange =
SynchronizerConfiguration.DEFAULT_BLOCK_PROPAGATION_RANGE;
@CommandLine.Option(
names = DOWNLOADER_CHANGE_TARGET_THRESHOLD_BY_HEIGHT_FLAG,
hidden = true,
paramLabel = "<LONG>",
description =
"Minimum height difference before switching fast sync download peers (default: ${DEFAULT-VALUE})")
private long downloaderChangeTargetThresholdByHeight =
SynchronizerConfiguration.DEFAULT_DOWNLOADER_CHANGE_TARGET_THRESHOLD_BY_HEIGHT;
@CommandLine.Option(
names = DOWNLOADER_CHANGE_TARGET_THRESHOLD_BY_TD_FLAG,
hidden = true,
paramLabel = "<UINT256>",
description =
"Minimum total difficulty difference before switching fast sync download peers (default: ${DEFAULT-VALUE})")
private UInt256 downloaderChangeTargetThresholdByTd =
SynchronizerConfiguration.DEFAULT_DOWNLOADER_CHANGE_TARGET_THRESHOLD_BY_TD;
@CommandLine.Option(
names = DOWNLOADER_HEADER_REQUEST_SIZE_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Number of headers to request per packet (default: ${DEFAULT-VALUE})")
private int downloaderHeaderRequestSize =
SynchronizerConfiguration.DEFAULT_DOWNLOADER_HEADER_REQUEST_SIZE;
@CommandLine.Option(
names = DOWNLOADER_BODIES_REQUEST_SIZE_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Number of bodies to request per packet (default: ${DEFAULT-VALUE})")
private int downloaderBodiesRequestSize =
SynchronizerConfiguration.DEFAULT_DOWNLOADER_BODIES_REQUEST_SIZE;
@CommandLine.Option(
names = {DOWNLOADER_CHECKPOINT_RETRIES_FLAG, DOWNLOADER_CHECKPOINT_TIMEOUTS_PERMITTED_FLAG},
hidden = true,
paramLabel = "<INTEGER>",
description =
"Number of tries to attempt to download checkpoints before stopping (default: ${DEFAULT-VALUE})")
private int downloaderCheckpointRetries =
SynchronizerConfiguration.DEFAULT_DOWNLOADER_CHECKPOINT_TIMEOUTS_PERMITTED;
@CommandLine.Option(
names = DOWNLOADER_CHAIN_SEGMENT_SIZE_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Distance between checkpoint headers (default: ${DEFAULT-VALUE})")
private int downloaderChainSegmentSize =
SynchronizerConfiguration.DEFAULT_DOWNLOADER_CHAIN_SEGMENT_SIZE;
@CommandLine.Option(
names = DOWNLOADER_PARALLELISM_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Number of threads to provide to chain downloader (default: ${DEFAULT-VALUE})")
private int downloaderParallelism = SynchronizerConfiguration.DEFAULT_DOWNLOADER_PARALLELISM;
@CommandLine.Option(
names = HEADER_DOWNLOAD_PARALLELISM_FACTOR_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Parallelism factor for header downloads relative to downloader parallelism (default: ${DEFAULT-VALUE})")
private int headerDownloadParallelismFactor =
SynchronizerConfiguration.DEFAULT_HEADER_DOWNLOAD_PARALLELISM_FACTOR;
@CommandLine.Option(
names = TRANSACTIONS_PARALLELISM_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Number of threads to commit to transaction processing (default: ${DEFAULT-VALUE})")
private int transactionsParallelism = SynchronizerConfiguration.DEFAULT_TRANSACTIONS_PARALLELISM;
@CommandLine.Option(
names = COMPUTATION_PARALLELISM_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Number of threads to make available for bulk hash computations during downloads (default: # of processors)")
private int computationParallelism = Runtime.getRuntime().availableProcessors();
@CommandLine.Option(
names = PIVOT_DISTANCE_FROM_HEAD_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Distance from initial chain head to sync target (default: ${DEFAULT-VALUE})")
private int pivotDistance = SynchronizerConfiguration.DEFAULT_PIVOT_DISTANCE_FROM_HEAD;
// Not exposed as CLI option anymore, but used internally
private float fastSyncFullValidationRate = SynchronizerConfiguration.DEFAULT_FULL_VALIDATION_RATE;
@CommandLine.Option(
names = WORLD_STATE_HASH_COUNT_PER_REQUEST_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Fast sync world state hashes queried per request (default: ${DEFAULT-VALUE})")
private int worldStateHashCountPerRequest =
SynchronizerConfiguration.DEFAULT_WORLD_STATE_HASH_COUNT_PER_REQUEST;
@CommandLine.Option(
names = WORLD_STATE_REQUEST_PARALLELISM_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Number of concurrent requests to use when downloading fast sync world state (default: ${DEFAULT-VALUE})")
private int worldStateRequestParallelism =
SynchronizerConfiguration.DEFAULT_WORLD_STATE_REQUEST_PARALLELISM;
@CommandLine.Option(
names = WORLD_STATE_MAX_REQUESTS_WITHOUT_PROGRESS_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Number of world state requests accepted without progress before considering the download stalled (default: ${DEFAULT-VALUE})")
private int worldStateMaxRequestsWithoutProgress =
SynchronizerConfiguration.DEFAULT_WORLD_STATE_MAX_REQUESTS_WITHOUT_PROGRESS;
@CommandLine.Option(
names = WORLD_STATE_MIN_MILLIS_BEFORE_STALLING_FLAG,
hidden = true,
paramLabel = "<LONG>",
description =
"Minimum time in ms without progress before considering a world state download as stalled (default: ${DEFAULT-VALUE})")
private long worldStateMinMillisBeforeStalling =
SynchronizerConfiguration.DEFAULT_WORLD_STATE_MIN_MILLIS_BEFORE_STALLING;
@CommandLine.Option(
names = WORLD_STATE_TASK_CACHE_SIZE_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"The max number of pending node data requests cached in-memory during fast sync world state download. (default: ${DEFAULT-VALUE})")
private int worldStateTaskCacheSize =
SynchronizerConfiguration.DEFAULT_WORLD_STATE_TASK_CACHE_SIZE;
@CommandLine.Option(
names = RECEIPTS_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG,
hidden = true,
paramLabel = "<LONG>",
description =
"Maximum time in milliseconds to wait for receipts download step including all retries (default: ${DEFAULT-VALUE})")
private long receiptsDownloadStepTimeoutMillis =
SynchronizerConfiguration.DEFAULT_RECEIPTS_DOWNLOAD_STEP_TIMEOUT_MILLIS;
@CommandLine.Option(
names = BACKWARD_HEADERS_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG,
hidden = true,
paramLabel = "<LONG>",
description =
"Maximum time in milliseconds to wait for backward headers download step including all retries (default: ${DEFAULT-VALUE})")
private long backwardHeadersDownloadStepTimeoutMillis =
SynchronizerConfiguration.DEFAULT_BACKWARD_HEADERS_DOWNLOAD_STEP_TIMEOUT_MILLIS;
@CommandLine.Option(
names = BODIES_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG,
hidden = true,
paramLabel = "<LONG>",
description =
"Maximum time in milliseconds to wait for bodies download step including all retries (default: ${DEFAULT-VALUE})")
private long bodiesDownloadStepTimeoutMillis =
SynchronizerConfiguration.DEFAULT_BODIES_DOWNLOAD_STEP_TIMEOUT_MILLIS;
@CommandLine.Option(
names = SNAP_PIVOT_BLOCK_WINDOW_VALIDITY_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"The size of the pivot block window before having to change it (default: ${DEFAULT-VALUE})")
private int snapsyncPivotBlockWindowValidity =
SnapSyncConfiguration.DEFAULT_PIVOT_BLOCK_WINDOW_VALIDITY;
@CommandLine.Option(
names = SNAP_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"The distance from the head before loading a pivot block into the cache to have a ready pivot block when the window is finished (default: ${DEFAULT-VALUE})")
private int snapsyncPivotBlockDistanceBeforeCaching =
SnapSyncConfiguration.DEFAULT_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING;
@CommandLine.Option(
names = SNAP_STORAGE_COUNT_PER_REQUEST_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Snap sync storage queried per request (default: ${DEFAULT-VALUE})")
private int snapsyncStorageCountPerRequest =
SnapSyncConfiguration.DEFAULT_STORAGE_COUNT_PER_REQUEST;
@CommandLine.Option(
names = SNAP_BYTECODE_COUNT_PER_REQUEST_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Snap sync bytecode queried per request (default: ${DEFAULT-VALUE})")
private int snapsyncBytecodeCountPerRequest =
SnapSyncConfiguration.DEFAULT_BYTECODE_COUNT_PER_REQUEST;
@CommandLine.Option(
names = SNAP_TRIENODE_COUNT_PER_REQUEST_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description = "Snap sync trie node queried per request (default: ${DEFAULT-VALUE})")
private int snapsyncTrieNodeCountPerRequest =
SnapSyncConfiguration.DEFAULT_TRIENODE_COUNT_PER_REQUEST;
@CommandLine.Option(
names = SNAP_FLAT_ACCOUNT_HEALED_COUNT_PER_REQUEST_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Snap sync flat accounts verified and healed per request (default: ${DEFAULT-VALUE})")
private int snapsyncFlatAccountHealedCountPerRequest =
SnapSyncConfiguration.DEFAULT_LOCAL_FLAT_ACCOUNT_COUNT_TO_HEAL_PER_REQUEST;
@CommandLine.Option(
names = SNAP_FLAT_STORAGE_HEALED_COUNT_PER_REQUEST_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Snap sync flat slots verified and healed per request (default: ${DEFAULT-VALUE})")
private int snapsyncFlatStorageHealedCountPerRequest =
SnapSyncConfiguration.DEFAULT_LOCAL_FLAT_STORAGE_COUNT_TO_HEAL_PER_REQUEST;
@CommandLine.Option(
names = {SNAP_SERVER_ENABLED_FLAG},
paramLabel = "<Boolean>",
arity = "0..1",
fallbackValue = "true",
description = "Enable snap sync server capability. (default: ${DEFAULT-VALUE})")
private Boolean snapsyncServerEnabled = SnapSyncConfiguration.DEFAULT_SNAP_SERVER_ENABLED;
@SuppressWarnings("unused")
@CommandLine.Option(
names = {CHECKPOINT_POST_MERGE_FLAG},
hidden = true,
paramLabel = "<Boolean>",
arity = "0..1",
fallbackValue = "true",
description =
"DEPRECATED: This option is deprecated as CHECKPOINT sync mode has been replaced by SNAP. This option no longer has any effect.")
private void setCheckpointPostMergeSyncEnabled(final Boolean enabled) {
LOG.warn(
"{} is deprecated and has no effect. CHECKPOINT sync mode has been replaced by SNAP.",
CHECKPOINT_POST_MERGE_FLAG);
}
@CommandLine.Option(
names = {"--Xpeertask-system-enabled"},
hidden = true,
description =
"Temporary feature toggle to enable using the new peertask system (default: ${DEFAULT-VALUE})")
private final Boolean isPeerTaskSystemEnabled = false;
@CommandLine.Option(
names = SNAP_SERVER_MAX_CONCURRENT_REQUESTS_PER_PEER_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Maximum number of snap sync GET_* requests from a single peer that may be concurrently scheduled for processing. 0 specifies no limit (default: ${DEFAULT-VALUE})")
private int snapsyncServerMaxConcurrentRequestsPerPeer =
SnapSyncConfiguration.DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_PER_PEER;
@CommandLine.Option(
names = SNAP_SERVER_MAX_CONCURRENT_REQUESTS_FLAG,
hidden = true,
paramLabel = "<INTEGER>",
description =
"Maximum total number of snap sync GET_* requests, across all peers, that may be concurrently scheduled for processing. 0 specifies no limit (default: ${DEFAULT-VALUE})")
private int snapsyncServerMaxConcurrentRequests =
SnapSyncConfiguration.DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_GLOBAL;
@CommandLine.Option(
names = SNAP_TRANSACTION_INDEXING_ENABLED_FLAG,
paramLabel = "<Boolean>",
arity = "0..1",
description =
"Enable transaction indexing during SNAP sync. Enabling will increase syncing time and disk usage. However, to support RPCs that use transaction hash for historical queries, you'll need to enable this. (default: ${DEFAULT-VALUE})")
private Boolean snapTransactionIndexingEnabled =
SnapSyncConfiguration.DEFAULT_SNAP_SYNC_TRANSACTION_INDEXING_ENABLED;
@CommandLine.Option(
names = {SNAP_SYNC_SAVE_PRE_CHECKPOINT_HEADERS_ONLY_FLAG},
paramLabel = "<Boolean>",
arity = "0..1",
description =
"Enable snap sync downloader to save only headers (not block bodies) for blocks before the checkpoint. (default: ${DEFAULT-VALUE})")
private Boolean snapSyncSavePreCheckpointHeadersOnlyEnabled =
DEFAULT_SNAP_SYNC_SAVE_PRE_MERGE_HEADERS_ONLY_ENABLED;
@CommandLine.Option(
names = ERA1_IMPORT_PREPIPELINE_ENABLED_FLAG,
paramLabel = "<Boolean>",
arity = "0..1",
description =
"Enable the ERA1 import prepipeline for FULL sync. ERA1 files will be imported from the URI specified by "
+ ERA1_DATA_URI_FLAG
+ " (currently only local files and http is supported). If a sync mode other than FULL is selected, this will have no affect. (default: ${DEFAULT-VALUE})")
private Boolean era1ImportPrepipelineEnabled =
SynchronizerConfiguration.DEFAULT_ERA1_IMPORT_PREPIPELINE_ENABLED;
@CommandLine.Option(
names = ERA1_DATA_URI_FLAG,
paramLabel = "<URI>",
description =
"The URI to attempt to load ERA1 files from. For local files, a simple path may be used (e.g. /home/user/era1). (default: ${DEFAULT-VALUE})")
private URI era1DataUri = SynchronizerConfiguration.DEFAULT_ERA1_DATA_URI;
@CommandLine.Option(
names = ERA1_IMPORT_PREPIPELINE_CONCURRENCY_FLAG,
paramLabel = "<INTEGER>",
description =
"How many parallel pipelines to use in the ERA1 import prepipeline (default: ${DEFAULT-VALUE})")
private Integer era1ImportPrepipelineConcurrency =
SynchronizerConfiguration.DEFAULT_ERA1_IMPORT_PREPIPELINE_CONCURRENCY;
private SynchronizerOptions() {}
/**
* Flag to know whether the Snap sync server feature is enabled or disabled.
*
* @return true if snap sync server is enabled
*/
public boolean isSnapsyncServerEnabled() {
return snapsyncServerEnabled;
}
/**
* Flag to indicate whether the peer task system should be used where available
*
* @return true if the peer task system should be used where available
*/
public boolean isPeerTaskSystemEnabled() {
return isPeerTaskSystemEnabled;
}
/**
* Create synchronizer options.
*
* @return the synchronizer options
*/
public static SynchronizerOptions create() {
return new SynchronizerOptions();
}
/**
* Create synchronizer options from Synchronizer Configuration.
*
* @param config the Synchronizer Configuration
* @return the synchronizer options
*/
public static SynchronizerOptions fromConfig(final SynchronizerConfiguration config) {
final SynchronizerOptions options = new SynchronizerOptions();
options.blockPropagationRange = config.getBlockPropagationRange();
options.downloaderChangeTargetThresholdByHeight =
config.getDownloaderChangeTargetThresholdByHeight();
options.downloaderChangeTargetThresholdByTd = config.getDownloaderChangeTargetThresholdByTd();
options.downloaderHeaderRequestSize = config.getDownloaderHeaderRequestSize();
options.downloaderBodiesRequestSize = config.getDownloaderBodiesRequestSize();
options.downloaderCheckpointRetries = config.getDownloaderCheckpointRetries();
options.downloaderChainSegmentSize = config.getDownloaderChainSegmentSize();
options.downloaderParallelism = config.getDownloaderParallelism();
options.headerDownloadParallelismFactor = config.getHeaderDownloadParallelismFactor();
options.transactionsParallelism = config.getTransactionsParallelism();
options.computationParallelism = config.getComputationParallelism();
options.pivotDistance = config.getSyncPivotDistance();
options.fastSyncFullValidationRate = config.getFastSyncFullValidationRate();
options.worldStateHashCountPerRequest = config.getWorldStateHashCountPerRequest();
options.worldStateRequestParallelism = config.getWorldStateRequestParallelism();
options.worldStateMaxRequestsWithoutProgress = config.getWorldStateMaxRequestsWithoutProgress();
options.worldStateMinMillisBeforeStalling = config.getWorldStateMinMillisBeforeStalling();
options.worldStateTaskCacheSize = config.getWorldStateTaskCacheSize();
options.snapsyncPivotBlockWindowValidity =
config.getSnapSyncConfiguration().getPivotBlockWindowValidity();
options.snapsyncPivotBlockDistanceBeforeCaching =
config.getSnapSyncConfiguration().getPivotBlockDistanceBeforeCaching();
options.snapsyncStorageCountPerRequest =
config.getSnapSyncConfiguration().getStorageCountPerRequest();
options.snapsyncBytecodeCountPerRequest =
config.getSnapSyncConfiguration().getBytecodeCountPerRequest();
options.snapsyncTrieNodeCountPerRequest =
config.getSnapSyncConfiguration().getTrienodeCountPerRequest();
options.snapsyncFlatAccountHealedCountPerRequest =
config.getSnapSyncConfiguration().getLocalFlatAccountCountToHealPerRequest();
options.snapsyncFlatStorageHealedCountPerRequest =
config.getSnapSyncConfiguration().getLocalFlatStorageCountToHealPerRequest();
options.snapsyncServerEnabled = config.getSnapSyncConfiguration().isSnapServerEnabled();
options.snapsyncServerMaxConcurrentRequestsPerPeer =
config.getSnapSyncConfiguration().getMaxConcurrentSnapRequestsPerPeer();
options.snapsyncServerMaxConcurrentRequests =
config.getSnapSyncConfiguration().getMaxConcurrentSnapRequestsGlobal();
options.snapTransactionIndexingEnabled =
config.getSnapSyncConfiguration().isSnapSyncTransactionIndexingEnabled();
options.snapSyncSavePreCheckpointHeadersOnlyEnabled =
config.isSnapSyncSavePreCheckpointHeadersOnlyEnabled();
options.era1ImportPrepipelineEnabled = config.era1ImportPrepipelineEnabled();
options.era1DataUri = config.era1DataUri();
options.era1ImportPrepipelineConcurrency = config.era1ImportPrepipelineConcurrency();
options.receiptsDownloadStepTimeoutMillis = config.getReceiptsDownloadStepTimeoutMillis();
options.backwardHeadersDownloadStepTimeoutMillis =
config.getBackwardHeadersDownloadStepTimeoutMillis();
options.bodiesDownloadStepTimeoutMillis = config.getBodiesDownloadStepTimeoutMillis();
return options;
}
@Override
public SynchronizerConfiguration.Builder toDomainObject() {
final SynchronizerConfiguration.Builder builder = SynchronizerConfiguration.builder();
builder.blockPropagationRange(blockPropagationRange);
builder.downloaderChangeTargetThresholdByHeight(downloaderChangeTargetThresholdByHeight);
builder.downloaderChangeTargetThresholdByTd(downloaderChangeTargetThresholdByTd);
builder.downloaderHeadersRequestSize(downloaderHeaderRequestSize);
builder.downloaderBodiesRequestSize(downloaderBodiesRequestSize);
builder.downloaderCheckpointRetries(downloaderCheckpointRetries);
builder.downloaderChainSegmentSize(downloaderChainSegmentSize);
builder.downloaderParallelism(downloaderParallelism);
builder.headerDownloadParallelismFactor(headerDownloadParallelismFactor);
builder.transactionsParallelism(transactionsParallelism);
builder.computationParallelism(computationParallelism);
builder.syncPivotDistance(pivotDistance);
builder.fastSyncFullValidationRate(fastSyncFullValidationRate);
builder.worldStateHashCountPerRequest(worldStateHashCountPerRequest);
builder.worldStateRequestParallelism(worldStateRequestParallelism);
builder.worldStateMaxRequestsWithoutProgress(worldStateMaxRequestsWithoutProgress);
builder.worldStateMinMillisBeforeStalling(worldStateMinMillisBeforeStalling);
builder.worldStateTaskCacheSize(worldStateTaskCacheSize);
builder.snapSyncConfiguration(
ImmutableSnapSyncConfiguration.builder()
.pivotBlockWindowValidity(snapsyncPivotBlockWindowValidity)
.pivotBlockDistanceBeforeCaching(snapsyncPivotBlockDistanceBeforeCaching)
.storageCountPerRequest(snapsyncStorageCountPerRequest)
.bytecodeCountPerRequest(snapsyncBytecodeCountPerRequest)
.trienodeCountPerRequest(snapsyncTrieNodeCountPerRequest)
.localFlatAccountCountToHealPerRequest(snapsyncFlatAccountHealedCountPerRequest)
.localFlatStorageCountToHealPerRequest(snapsyncFlatStorageHealedCountPerRequest)
.isSnapServerEnabled(snapsyncServerEnabled)
.maxConcurrentSnapRequestsPerPeer(snapsyncServerMaxConcurrentRequestsPerPeer)
.maxConcurrentSnapRequestsGlobal(snapsyncServerMaxConcurrentRequests)
.isSnapSyncTransactionIndexingEnabled(snapTransactionIndexingEnabled)
.build());
builder.isPeerTaskSystemEnabled(isPeerTaskSystemEnabled);
builder.snapSyncSavePreCheckpointHeadersOnlyEnabled(
snapSyncSavePreCheckpointHeadersOnlyEnabled);
builder.receiptsDownloadStepTimeoutMillis(receiptsDownloadStepTimeoutMillis);
builder.backwardHeadersDownloadStepTimeoutMillis(backwardHeadersDownloadStepTimeoutMillis);
builder.bodiesDownloadStepTimeoutMillis(bodiesDownloadStepTimeoutMillis);
builder.era1ImportPrepipelineEnabled(era1ImportPrepipelineEnabled);
builder.era1DataUri(era1DataUri);
builder.era1ImportPrepipelineConcurrency(era1ImportPrepipelineConcurrency);
return builder;
}
@Override
public List<String> getCLIOptions() {
List<String> value =
Arrays.asList(
BLOCK_PROPAGATION_RANGE_FLAG,
OptionParser.format(blockPropagationRange),
DOWNLOADER_CHANGE_TARGET_THRESHOLD_BY_HEIGHT_FLAG,
OptionParser.format(downloaderChangeTargetThresholdByHeight),
DOWNLOADER_CHANGE_TARGET_THRESHOLD_BY_TD_FLAG,
OptionParser.format(downloaderChangeTargetThresholdByTd),
DOWNLOADER_HEADER_REQUEST_SIZE_FLAG,
OptionParser.format(downloaderHeaderRequestSize),
DOWNLOADER_BODIES_REQUEST_SIZE_FLAG,
OptionParser.format(downloaderBodiesRequestSize),
DOWNLOADER_CHECKPOINT_TIMEOUTS_PERMITTED_FLAG,
OptionParser.format(downloaderCheckpointRetries),
DOWNLOADER_CHAIN_SEGMENT_SIZE_FLAG,
OptionParser.format(downloaderChainSegmentSize),
DOWNLOADER_PARALLELISM_FLAG,
OptionParser.format(downloaderParallelism),
HEADER_DOWNLOAD_PARALLELISM_FACTOR_FLAG,
OptionParser.format(headerDownloadParallelismFactor),
TRANSACTIONS_PARALLELISM_FLAG,
OptionParser.format(transactionsParallelism),
COMPUTATION_PARALLELISM_FLAG,
OptionParser.format(computationParallelism),
PIVOT_DISTANCE_FROM_HEAD_FLAG,
OptionParser.format(pivotDistance),
WORLD_STATE_HASH_COUNT_PER_REQUEST_FLAG,
OptionParser.format(worldStateHashCountPerRequest),
WORLD_STATE_REQUEST_PARALLELISM_FLAG,
OptionParser.format(worldStateRequestParallelism),
WORLD_STATE_MAX_REQUESTS_WITHOUT_PROGRESS_FLAG,
OptionParser.format(worldStateMaxRequestsWithoutProgress),
WORLD_STATE_MIN_MILLIS_BEFORE_STALLING_FLAG,
OptionParser.format(worldStateMinMillisBeforeStalling),
WORLD_STATE_TASK_CACHE_SIZE_FLAG,
OptionParser.format(worldStateTaskCacheSize),
RECEIPTS_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG,
OptionParser.format(receiptsDownloadStepTimeoutMillis),
BACKWARD_HEADERS_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG,
OptionParser.format(backwardHeadersDownloadStepTimeoutMillis),
BODIES_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG,
OptionParser.format(bodiesDownloadStepTimeoutMillis),
SNAP_PIVOT_BLOCK_WINDOW_VALIDITY_FLAG,
OptionParser.format(snapsyncPivotBlockWindowValidity),
SNAP_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING_FLAG,
OptionParser.format(snapsyncPivotBlockDistanceBeforeCaching),
SNAP_STORAGE_COUNT_PER_REQUEST_FLAG,
OptionParser.format(snapsyncStorageCountPerRequest),
SNAP_BYTECODE_COUNT_PER_REQUEST_FLAG,
OptionParser.format(snapsyncBytecodeCountPerRequest),
SNAP_TRIENODE_COUNT_PER_REQUEST_FLAG,
OptionParser.format(snapsyncTrieNodeCountPerRequest),
SNAP_FLAT_ACCOUNT_HEALED_COUNT_PER_REQUEST_FLAG,
OptionParser.format(snapsyncFlatAccountHealedCountPerRequest),
SNAP_FLAT_STORAGE_HEALED_COUNT_PER_REQUEST_FLAG,
OptionParser.format(snapsyncFlatStorageHealedCountPerRequest),
SNAP_SERVER_ENABLED_FLAG,
OptionParser.format(snapsyncServerEnabled),
SNAP_SERVER_MAX_CONCURRENT_REQUESTS_PER_PEER_FLAG,
OptionParser.format(snapsyncServerMaxConcurrentRequestsPerPeer),
SNAP_SERVER_MAX_CONCURRENT_REQUESTS_FLAG,
OptionParser.format(snapsyncServerMaxConcurrentRequests),
SNAP_TRANSACTION_INDEXING_ENABLED_FLAG,
OptionParser.format(snapTransactionIndexingEnabled),
SNAP_SYNC_SAVE_PRE_CHECKPOINT_HEADERS_ONLY_FLAG,
OptionParser.format(snapSyncSavePreCheckpointHeadersOnlyEnabled),
ERA1_IMPORT_PREPIPELINE_ENABLED_FLAG,
OptionParser.format(era1ImportPrepipelineEnabled),
ERA1_DATA_URI_FLAG,
OptionParser.format(era1DataUri),
ERA1_IMPORT_PREPIPELINE_CONCURRENCY_FLAG,
OptionParser.format(era1ImportPrepipelineConcurrency));
return value;
}
}

1460
anchor/app/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java Normal file
View File

File diff suppressed because it is too large Load Diff

275
anchor/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealCache.java
View File

@ -14,16 +14,22 @@
*/
package org.hyperledger.besu.consensus.common.bft;
import org.hyperledger.besu.consensus.common.bft.blockcreation.PqAnchorProducer;
import org.hyperledger.besu.datatypes.Hash;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashSet;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import org.apache.tuweni.bytes.Bytes;
import org.apache.tuweni.bytes.Bytes32;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -59,11 +65,33 @@ import org.slf4j.LoggerFactory;
* the empty cache an absent file already gives. Persistence is OFF unless a caller enables it, and
* the only caller that does is the QBFT controller builder, only when the anchor is actually armed.
*
* <p>WHAT IT DOES NOT DO. The in-memory path verifies nothing. Whether a seal is valid, whether its
* index maps to an eligible validator, and whether there are enough of them, are decided at
* SELECTION time by the producer, against the message M of the specific parent. A seal that was
* signed under the pre-fork message simply fails that check and is dropped, which is what makes the
* height switch safe without any coordination beyond the shared activation height.
* <p>WHAT IT DOES NOT DO. Whether a seal index maps to an eligible validator, and whether there are
* enough seals, are decided at SELECTION time by the producer, against the message M of the specific
* parent. A seal that was signed under the pre-fork message simply fails that check and is dropped,
* which is what makes the height switch safe without any coordination beyond the shared activation
* height.
*
* <p>D-454 FAMILY (adversarial review of 2026-09-30): ONE SLOT PER INDEX, AND AN UNVERIFIED SEAL NEVER
* HOLDS IT AGAINST A VERIFIED ONE. Until this date the in-memory path verified nothing and the FIRST
* seal heard for an index won for good. The Commits feeding it are not all judged first: the D-339
* path keeps the seal of any Commit heard before the proposal, the late-seal salvage keeps any
* validator's late Commit, and below the commit enforcement no Commit's post-quantum seal is judged
* at all. So a garbage seal naming A's index, heard before A's own (below H_fin any validator can
* send one; from H_fin, when the author of a Commit is read from its seal, a copy of A's public
* commit seal in anyone's envelope), kept A's genuine seal out of every certificate this node could
* build - and enough of them, at the parent of an anchor, left no proposer able to reach K. Now, from
* H_fin ({@link AerePqFinality#PROPERTY}, AIP-22 stage 4, {@link #judgeGoverns}) and once the anchor
* is armed on this node ({@link #enablePersistence}, the one production entry point), a seal is
* JUDGED as it arrives, with the producer's own check ({@link #verifiesAsTheProducerDoes}): a seal
* that verifies replaces one that does not, a seal that does not verify never replaces anything, and
* a verified seal is never replaced. Without the judge (anchor not armed, or a height whose seal is
* not in the anchor form) every seal is unverified, which is exactly the first-seal-wins store of
* before. BELOW H_fin (and on every node where H_fin is unset) the store is the one of before, call
* by call: first seal per index wins, nothing is verified, the restore and the head file follow the
* rules of before (second review of D-454, 2026-10-01: armed from enablePersistence alone, the judge
* changed the live chain below the stage that needs it and put post-quantum verifications on its
* consensus thread). The producer still re-verifies every seal it carries; the judge decides only
* which seal is KEPT.
*/
public final class PqSealCache {
@ -128,8 +156,130 @@ public final class PqSealCache {
/** Heights restored at startup at most, newest first, whatever the configured window. */
private static final long MAX_RESTORED_HEIGHTS = 4096L;
/** D-454 family: the seals last written to the head file (compared, not only counted). */
private List<FalconSeal> persistedSeals = List.of();
/**
* D-454 FAMILY (2026-09-30): whether a heard seal VERIFIES. Must never throw; false means "not
* shown valid", never "shown invalid".
*/
@FunctionalInterface
public interface SealJudge {
/**
* Judge one heard seal.
*
* @param blockNumber the height of the block the seal attests
* @param onchainBlockHash the ON-CHAIN hash of that block
* @param schemeWireId the certificate-v2 wire tag of the seal's scheme (Falcon-512 for the v1 list)
* @param validatorIndex the registry index the seal names
* @param signature the seal's signature bytes
* @return true only when the signature verifies under the key the index names at that height
*/
boolean verifies(
long blockNumber,
Hash onchainBlockHash,
byte schemeWireId,
int validatorIndex,
Bytes signature);
/** Judges nothing: every seal is unverified, which is exactly the first-seal-wins store of before. */
SealJudge NONE = (h, hash, wire, index, sig) -> false;
}
private volatile SealJudge judge = SealJudge.NONE;
private PqSealCache() {}
/**
* D-454 FAMILY, BOUND TO H_FIN (second review, 2026-10-01): whether the judge of heard seals governs
* a height. Only from H_fin ({@link AerePqFinality#PROPERTY}, {@code aere.pq.finalityBlock}, AIP-22
* stage 4: the height from which the author of a Commit is read from its seal, the one
* RoundState's pending commits are governed by too); below it, and on every node where H_fin is
* unset, the store is the one of before, call by call, and no seal is verified on hearing.
*
* @param blockNumber the height of the block the seals attest
* @return true from H_fin
*/
static boolean judgeGoverns(final long blockNumber) {
return AerePqFinality.active(blockNumber);
}
/**
* Arms the judge of heard seals ({@link SealJudge#NONE} or null disarms it). The one way in:
* production arms it from {@link #enablePersistence}; the cost measurement gives a counting judge.
*
* @param sealJudge the judge
*/
void judgeWith(final SealJudge sealJudge) {
this.judge = sealJudge == null ? SealJudge.NONE : sealJudge;
}
/**
* D-454 FAMILY (2026-09-30): THE PRODUCER'S OWN CHECK, applied to a seal as it is heard. The message
* is the anchor form over the block's on-chain hash ({@link PqAnchor#commitMessage}, what the
* producer verifies against at selection time); a Falcon-512 seal is verified by the registry's
* own-head door at the seal's height, exactly as the producer verifies it at the parent's height; an
* extra scheme seal under the key the producer resolves (the v3 epoch from H_k, the hybrid registry
* before). Below the anchor form (the committed-seal digest, which needs the block) nothing is
* verified here and the store keeps its first-seal-wins rule. Never throws.
*
* @param blockNumber the height of the block the seal attests
* @param onchainBlockHash the ON-CHAIN hash of that block
* @param schemeWireId the wire tag of the seal's scheme
* @param validatorIndex the registry index the seal names
* @param signature the seal's signature bytes
* @return true only when the seal verifies as the producer would verify it
*/
static boolean verifiesAsTheProducerDoes(
final long blockNumber,
final Hash onchainBlockHash,
final byte schemeWireId,
final int validatorIndex,
final Bytes signature) {
try {
if (!PqAnchorProducer.sealMessageIsAnchorForm(blockNumber)) {
return false;
}
final Bytes32 message =
PqAnchor.commitMessage(
PqAnchorProducer.config().chainId(), blockNumber, onchainBlockHash.getBytes());
if (schemeWireId == SealSchemes.FALCON_512.wireId()) {
return PqSignerRegistry.falconSealSupport()
.verifyAtOwnHead(blockNumber, validatorIndex, message, signature);
}
final Optional<SealScheme> scheme = SealSchemes.byWireId(schemeWireId);
if (scheme.isEmpty()) {
return false;
}
final String schemeId = scheme.get().id();
final Optional<byte[]> publicKey =
AerePqIdentity.keysFromV3(blockNumber)
? AerePqIdentity.v3KeyAt(blockNumber, validatorIndex, schemeId)
: HybridSealSupport.instance()
.registry()
.flatMap(r -> r.publicKey(validatorIndex, schemeId));
return publicKey.isPresent()
&& scheme.get().verifyRaw(publicKey.get(), message.toArray(), signature.toArray());
} catch (final RuntimeException e) {
return false;
}
}
/**
* Whether the seal an index already holds for a block stays when another one arrives: always when it
* was verified, and when the newcomer is not verified. One place, so no caller can half-apply it (the
* second client's KeepsHeardSeal is the same rule).
*
* @param held whether the index holds a seal
* @param heldVerified whether that seal was verified
* @param newVerified whether the newcomer verified
* @return true when the held seal stays
*/
static boolean keepsHeldSeal(
final boolean held, final boolean heldVerified, final boolean newVerified) {
return held && (heldVerified || !newVerified);
}
/**
* The process-wide cache.
*
@ -146,11 +296,19 @@ public final class PqSealCache {
* separately: a second configurable path is a second thing that can be pointed at the wrong node's
* state, and this file is per-node by construction.
*
* <p>D-454 FAMILY (2026-09-30): this is the one production entry point reached only with the anchor
* armed (the QBFT controller builder, after the registry is active), so it also arms the judge of
* heard seals ({@link #verifiesAsTheProducerDoes}), with or without a data directory. Arming it
* here, and never lazily from the store itself, keeps a node or a test without the anchor from
* initialising the registry singletons as a side effect of hearing a seal. Armed is not governing:
* the judge is consulted only from H_fin ({@link #judgeGoverns}, bound 2026-10-01).
*
* @param dataDirectory the node's data directory, or null to leave persistence off
* @param chainId the chain id used in the M pre-image, which must be the SAME value the producer
* uses or every restored seal would fail verification
*/
public void enablePersistence(final Path dataDirectory, final long chainId) {
judgeWith(PqSealCache::verifiesAsTheProducerDoes);
if (dataDirectory == null) {
LOG.warn(
"AERE PERSISTENTA-SIGILII: no data directory was supplied, so heard seals stay in memory "
@ -170,10 +328,12 @@ public final class PqSealCache {
}
/**
* Turn persistence off again. For tests, and for an operator who wants the previous behaviour.
* Turn persistence off again. For tests, and for an operator who wants the previous behaviour. It
* also disarms the judge of heard seals (D-454 family), back to the first-seal-wins store.
*/
public void disablePersistence() {
this.persistenceFile = null;
this.judge = SealJudge.NONE;
this.windowDirectory = null;
synchronized (writeLock) {
persistedBlock = -1L;
@ -247,7 +407,17 @@ public final class PqSealCache {
final Entry entry =
byHash.computeIfAbsent(onchainBlockHash, unused -> new Entry(blockNumber));
for (final FalconSeal seal : verified) {
if (!judgeGoverns(blockNumber)) {
// below H_fin: the restore of before, call by call (D-454 family, bound to H_fin)
entry.seals.putIfAbsent(seal.getValidatorIndex(), seal);
continue;
}
// re-verified by readVerified: it displaces an unverified seal held for its index (D-454 family)
final int index = seal.getValidatorIndex();
if (!keepsHeldSeal(entry.seals.containsKey(index), entry.verified.contains(index), true)) {
entry.seals.put(index, seal);
entry.verified.add(index);
}
}
if (blockNumber > highestSeen) {
highestSeen = blockNumber;
@ -258,8 +428,11 @@ public final class PqSealCache {
}
/**
* Remember the seals heard for a block. Idempotent per validator index: the first seal seen for
* an index wins, so a later duplicate cannot displace it.
* Remember the seals heard for a block. One slot per validator index: a later duplicate cannot
* displace the seal held, and (D-454 family, 2026-09-30) from H_fin neither can any seal that does
* not verify; a seal that verifies displaces one that did not ({@link #keepsHeldSeal}). Below H_fin,
* or without the judge armed, nothing is verified and the first seal seen for an index wins, as
* before.
*
* @param blockNumber the height of the block the seals attest
* @param onchainBlockHash the ON-CHAIN hash of that block
@ -277,7 +450,33 @@ public final class PqSealCache {
byHash.computeIfAbsent(onchainBlockHash, unused -> new Entry(blockNumber));
for (final FalconSeal seal : seals) {
if (seal != null && seal.getValidatorIndex() >= 0 && seal.getSignature() != null) {
if (!judgeGoverns(blockNumber)) {
// below H_fin: the store of before, call by call (first seal per index wins, nothing judged)
entry.seals.putIfAbsent(seal.getValidatorIndex(), seal);
continue;
}
final int index = seal.getValidatorIndex();
final FalconSeal held = entry.seals.get(index);
// the same bytes again (every commit re-offers the round's seals), or a verified seal held:
// nothing to judge, the held seal stays
if (held != null
&& (entry.verified.contains(index) || held.getSignature().equals(seal.getSignature()))) {
continue;
}
final boolean verified =
judge.verifies(
blockNumber,
onchainBlockHash,
SealSchemes.FALCON_512.wireId(),
index,
seal.getSignature());
if (keepsHeldSeal(held != null, false, verified)) {
continue;
}
entry.seals.put(index, seal);
if (verified) {
entry.verified.add(index);
}
}
}
if (blockNumber > highestSeen) {
@ -301,6 +500,20 @@ public final class PqSealCache {
persistWindow(windowToJournal);
}
/** Same indices and the same signature bytes, in the same (canonical) order. */
private static boolean sameSeals(final List<FalconSeal> a, final List<FalconSeal> b) {
if (a.size() != b.size()) {
return false;
}
for (int i = 0; i < a.size(); i++) {
if (a.get(i).getValidatorIndex() != b.get(i).getValidatorIndex()
|| !a.get(i).getSignature().equals(b.get(i).getSignature())) {
return false;
}
}
return true;
}
private void persist(
final long blockNumber, final Hash onchainBlockHash, final List<FalconSeal> seals) {
final Path file = persistenceFile;
@ -311,9 +524,16 @@ public final class PqSealCache {
if (blockNumber < persistedBlock) {
return;
}
// D-454 family (2026-09-30): from H_fin a verified seal can REPLACE an unverified one without
// changing the count, so "no more seals than the file holds" is no longer "nothing new" there: the
// file is kept only when it holds more seals, or the very same ones. Below H_fin nothing is ever
// replaced and the rule of before stands, call by call (bound to H_fin, 2026-10-01).
if (blockNumber == persistedBlock
&& onchainBlockHash.equals(persistedHash)
&& seals.size() <= persistedCount) {
&& (judgeGoverns(blockNumber)
? (seals.size() < persistedCount
|| (seals.size() == persistedCount && sameSeals(seals, persistedSeals)))
: seals.size() <= persistedCount)) {
return;
}
try {
@ -333,6 +553,9 @@ public final class PqSealCache {
persistedBlock = blockNumber;
persistedHash = onchainBlockHash;
persistedCount = seals.size();
if (judgeGoverns(blockNumber)) {
persistedSeals = List.copyOf(seals);
}
} catch (final Exception e) {
// A disk fault must not be able to stop this node from taking part in consensus. It costs
// the restart protection, which is what the log line says, and nothing else.
@ -356,8 +579,10 @@ public final class PqSealCache {
*/
/**
* AERE ANCHOR V2: remember the extra (non-Falcon) scheme seals heard on the commits of a block.
* Same key as {@link #record}, same window, same idempotence (first seal per validator and
* scheme wins). NOT persisted across a restart: a node that restarts holds no extras until it
* Same key as {@link #record}, same window, same slot rule per validator and scheme (D-454 family:
* from H_fin an unverified seal never displaces a verified one and a verified one displaces an
* unverified one; below H_fin, or without the judge, the first seal wins, as before). NOT persisted
* across a restart: a node that restarts holds no extras until it
* takes part in one commit, exactly the Falcon situation before PqSealStore, and the producer
* then refuses the v2 certificate for one proposer turn rather than writing a short one.
*
@ -381,7 +606,31 @@ public final class PqSealCache {
continue;
}
final long key = ((long) seal.getValidatorIndex() << 8) | (seal.getSchemeWireId() & 0xffL);
if (!judgeGoverns(blockNumber)) {
// below H_fin: the store of before, call by call (first seal per index and scheme wins)
entry.extras.putIfAbsent(key, seal);
continue;
}
// D-454 family (2026-09-30): the same rule as the Falcon slot, per (index, scheme)
final SchemeSeal held = entry.extras.get(key);
if (held != null
&& (entry.extrasVerified.contains(key) || held.getSignature().equals(seal.getSignature()))) {
continue;
}
final boolean verified =
judge.verifies(
blockNumber,
onchainBlockHash,
seal.getSchemeWireId(),
seal.getValidatorIndex(),
seal.getSignature());
if (keepsHeldSeal(held != null, false, verified)) {
continue;
}
entry.extras.put(key, seal);
if (verified) {
entry.extrasVerified.add(key);
}
}
if (blockNumber > highestSeen) {
highestSeen = blockNumber;
@ -623,6 +872,10 @@ public final class PqSealCache {
private final Map<Integer, FalconSeal> seals = new LinkedHashMap<>();
/** AERE ANCHOR V2: heard extra (non-Falcon) scheme seals, keyed (index << 8 | wire). */
private final Map<Long, SchemeSeal> extras = new LinkedHashMap<>();
/** D-454 family: the indices (and the extra-seal keys) whose held seal VERIFIED. */
private final Set<Integer> verified = new HashSet<>();
private final Set<Long> extrasVerified = new HashSet<>();
private Entry(final long blockNumber) {
this.blockNumber = blockNumber;

2
anchor/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftBlockCreator.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-bft-prevrandao-block-creation.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.consensus.common.bft.blockcreation;

2
anchor/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftMiningCoordinator.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-bft-coordinator-lifecycle-sync.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.consensus.common.bft.blockcreation;

2
anchor/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/messagewrappers/BftMessage.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-bft-bounded-message-lists.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.consensus.common.bft.messagewrappers;

434
anchor/consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AereD454SealCacheTest.java Normal file
View File

@ -0,0 +1,434 @@
/*
* Copyright contributors to Aere Network.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
* compliance with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is
* distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing permissions and limitations under the
* License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.consensus.common.bft;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import static org.mockito.Mockito.withSettings;
import org.hyperledger.besu.consensus.common.bft.blockcreation.PqAnchorProducer;
import org.hyperledger.besu.consensus.common.validator.ValidatorProvider;
import org.hyperledger.besu.crypto.SecureRandomProvider;
import org.hyperledger.besu.datatypes.Address;
import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.core.BlockHeader;
import org.hyperledger.besu.ethereum.core.BlockHeaderTestFixture;
import java.lang.reflect.Field;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.SecureRandom;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.OptionalInt;
import org.apache.tuweni.bytes.Bytes;
import org.apache.tuweni.bytes.Bytes32;
import org.bouncycastle.crypto.digests.KeccakDigest;
import org.bouncycastle.pqc.crypto.falcon.FalconPrivateKeyParameters;
import org.bouncycastle.pqc.crypto.falcon.FalconSigner;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.mockito.quality.Strictness;
/**
* AERE D-454 FAMILY (adversarial review of 2026-09-30): the store of HEARD seals no longer lets an
* unverified seal hold an index against the genuine one.
*
* <p>The defect, measured by this class on the code before the repair: {@code PqSealCache.record} kept
* the FIRST seal heard for an index, verifying nothing, and it is fed by paths that judge nothing (the
* D-339 path keeps the seal of any Commit heard before the proposal; the late-seal salvage keeps any
* validator's late Commit; below the commit enforcement no Commit's post-quantum seal is judged). So a
* garbage seal naming A's index, heard before A's own (below H_fin any validator can send it; from
* H_fin a copy of A's public commit seal in anyone's envelope is "A's"), kept A's genuine seal out of
* the certificate this node builds, and at the parent of an anchor enough of them leave no proposer
* able to reach K. Same for the extra scheme seals (SLH-DSA) of a v2 certificate.
*
* <p>The fixture is the REAL one of {@code PqAnchorV2ProducerTest}: the genesis-anchored Falcon registry
* loaded by {@code FalconSealSupport}, a hybrid-1 registry with SLH-DSA-SHA2-128s probe keys loaded
* through the system configuration, and the PRODUCER itself ({@code PqAnchorProducer.apply}) as the
* judge of the outcome: the question is not what the store holds, it is whether this node can still
* build the certificate. The judge of heard seals is armed by {@code enablePersistence}, the one
* production entry point reached with the anchor armed.
*
* <p>Six proofs, each red under a named planting (consensus-pqc/d412-reparatie/cap-decis-besu/
* planteaza-d454.py, harness d454-roundstate.sh): three are the defect (red on the code before), two
* say that nothing else moved (a verified seal is never displaced; without the judge the first seal
* still wins), one that two unverified seals do not trade places.
*
* <p>BOUND TO H_FIN (second review of D-454, 2026-10-01): the judge governs a height only from H_fin
* ({@code AerePqFinality.PROPERTY}, AIP-22 stage 4). The six proofs above run from H_fin (H_fin is set
* to the registry's height, so every parent here is governed); three more say that BELOW H_fin the
* store is the one of today, with the judge armed by the production entry point: the first Falcon
* seal stays, the first SLH-DSA seal stays, and a restore displaces nothing. They are green on the
* code before the repair (that is today's behaviour) and red under the planting
* "judecator-armat-sub-hfin". H_fin is set directly on AerePqFinality: the store reads only {@code
* AerePqFinality.active}; the startup coherence of identity and keys (AERE-PQC-FIN-01) is not this
* class's question.
*/
public class AereD454SealCacheTest {
private static final long CHAIN_ID = 220_879L;
private static final long H = 1_000L;
/** H_fin for the six proofs from H_fin: every parent of this class is at or above it. */
private static final long H_FIN = H;
/** H_fin for the three proofs below H_fin: above every parent of this class. */
private static final long H_FIN_ABOVE = 5_000L;
private static final long V2 = 1_005L;
private static final int N = 7;
private static final int K = 3;
private static final String SLH = "slh-dsa-sha2-128s";
@TempDir private Path tmp;
private final List<FalconPrivateKeyParameters> falconKeys = new ArrayList<>();
private final List<SealScheme.GeneratedPair> slhKeys = new ArrayList<>();
private final List<Address> validators = new ArrayList<>();
private final SecureRandom random = SecureRandomProvider.createSecureRandom();
private Path dataDirectory;
@BeforeEach
public void setUp() throws Exception {
final KeccakDigest kd = new KeccakDigest(256);
final StringBuilder manifest = new StringBuilder();
manifest
.append("{\"config\":{\"aereFalconRegistry\":{")
.append(PqV2Fixture.manifestHeader(N, CHAIN_ID, H));
for (int i = 0; i < N; i++) {
falconKeys.add(PqV2Fixture.privateKey(i));
validators.add(PqV2Fixture.address(i));
final byte[] row = PqV2Fixture.anchorPreimageRow(i);
kd.update(row, 0, row.length);
manifest.append(',').append(PqV2Fixture.manifestEntry(i, N, CHAIN_ID, H));
}
manifest
.append("}},\"alloc\":{\"0000000000000000000000000000000000000fa1\":{\"storage\":{\"0x")
.append("0".repeat(64))
.append("\":\"0x");
final byte[] anchoredHash = new byte[32];
kd.doFinal(anchoredHash, 0);
manifest.append(Bytes.wrap(anchoredHash).toUnprefixedHexString()).append("\"}}}}");
final Path genesis = tmp.resolve("genesis-registry.json");
Files.writeString(genesis, manifest.toString());
System.setProperty("aere.falcon.genesis", genesis.toAbsolutePath().toString());
resetFalconSingleton();
PqSealCache.instance().disablePersistence();
PqSealCache.instance().clear();
armHybrid("0:falcon-512+" + SLH);
PqAnchorProducer.useConfigForTesting(
new PqAnchorConfig(
CHAIN_ID, H, Map.of(H, K), OptionalInt.empty(), false, OptionalInt.of(K),
OptionalInt.empty(), V2));
dataDirectory = Files.createDirectories(tmp.resolve("besu-data"));
// the anchor armed on this node: the production entry point, which also arms the judge
PqSealCache.instance().enablePersistence(dataDirectory, CHAIN_ID);
// and H_fin reached: the judge governs every parent of the six proofs from H_fin
setFinalityBlock(H_FIN);
}
@AfterEach
public void tearDown() throws Exception {
AerePqFinality.uninstall();
PqSealCache.instance().disablePersistence();
PqSealCache.instance().clear();
System.clearProperty("aere.falcon.genesis");
System.clearProperty(HybridSealSupport.PROPERTY_SCHEDULE);
System.clearProperty(HybridSealSupport.PROPERTY_REGISTRY);
HybridSealSupport.resetForTesting();
resetFalconSingleton();
PqAnchorProducer.useConfigForTesting(null);
}
// ------------------------------------------------------------------------------------ the defect
/** The certificate this node can build still carries A (index 0), whose genuine seal came second. */
@Test
public void aGarbageFalconSealHeardFirstNoLongerKeepsTheGenuineOneOut() {
final BlockHeader parent = parentOf(V2 - 2L); // a v1 anchor parent: Falcon only
hearGarbageFalcon(parent, (byte) 0x31);
hearFalcon(parent, 0, K);
assertThat(heldFalcon(parent, 0))
.describedAs("index 0 holds A's genuine seal, not the garbage heard before it")
.isEqualTo(genuine.get(0));
final BftExtraData produced = PqAnchorProducer.apply(base(), parent, contextWith(validators));
assertThat(produced.getFalconSeals())
.describedAs("K verified seals, A's among them: the proposer is not short of A")
.hasSize(K)
.anyMatch(s -> s.getValidatorIndex() == 0);
}
/** The replacement reaches the file: a node restarted onto this head restores A's genuine seal. */
@Test
public void theFileHoldsTheGenuineSealThatDisplacedAGarbageOne() {
final BlockHeader head = parentOf(V2 - 2L);
hearFalcon(head, 1, K); // the others first
hearGarbageFalcon(head, (byte) 0x32);
hearFalcon(head, 0, 1); // A's genuine seal is the LAST change: same count, other bytes
PqSealCache.instance().clear(); // the restart: memory gone, the file stays
final int restored =
PqSealCache.instance()
.restoreFromDisk(head.getNumber(), head.getHash(), PqSignerRegistry.falconSealSupport());
assertThat(restored).describedAs("the K genuine seals come back from the file, A's included").isEqualTo(K);
assertThat(PqAnchorProducer.apply(base(), head, contextWith(validators)).getFalconSeals())
.hasSize(K)
.anyMatch(s -> s.getValidatorIndex() == 0);
}
/** The same rule for the extra scheme of a v2 certificate. */
@Test
public void aGarbageSlhDsaSealHeardFirstNoLongerKeepsTheGenuineOneOut() {
final BlockHeader parent = parentOf(V2 + 9L); // a v2 anchor parent: Falcon and SLH-DSA
hearFalcon(parent, 0, K);
PqSealCache.instance()
.recordExtras(
parent.getNumber(),
parent.getHash(),
List.of(new SchemeSeal(SealSchemes.SLH_DSA_128S.wireId(), 0, garbage(7856, (byte) 0x33))));
hearSlh(parent, 0, K);
final List<SchemeSeal> tagged =
PqAnchorProducer.apply(base(), parent, contextWith(validators)).getHybridSeals();
assertThat(PqAnchorV2.distinctValidatorsWith(tagged, SealSchemes.SLH_DSA_128S.wireId()))
.describedAs("K SLH-DSA seals, A's genuine one among them")
.isEqualTo(K);
assertThat(tagged)
.anyMatch(
s -> s.getSchemeWireId() == SealSchemes.SLH_DSA_128S.wireId() && s.getValidatorIndex() == 0);
}
// ------------------------------------------------------------------------------------ nothing else moved
/** A verified seal is never displaced, by anything heard after it. */
@Test
public void aGarbageFalconSealAfterTheGenuineOneDisplacesNothing() {
final BlockHeader parent = parentOf(V2 - 2L);
hearFalcon(parent, 0, K);
hearGarbageFalcon(parent, (byte) 0x34);
assertThat(heldFalcon(parent, 0)).isEqualTo(genuine.get(0));
assertThat(PqAnchorProducer.apply(base(), parent, contextWith(validators)).getFalconSeals())
.hasSize(K)
.anyMatch(s -> s.getValidatorIndex() == 0);
}
/** Without the judge (a node without the anchor armed) the store is the one of before: first wins. */
@Test
public void withoutTheJudgeTheFirstSealStaysAsBefore() {
PqSealCache.instance().disablePersistence();
final BlockHeader parent = parentOf(V2 - 2L);
hearGarbageFalcon(parent, (byte) 0x35);
hearFalcon(parent, 0, K);
assertThat(heldFalcon(parent, 0))
.describedAs("unjudged, the first seal of index 0 stays")
.isEqualTo(garbage(666, (byte) 0x35));
}
/** Two seals neither of which verifies do not trade places. */
@Test
public void anUnverifiedSealNeverDisplacesAnother() {
final BlockHeader parent = parentOf(V2 - 2L);
hearGarbageFalcon(parent, (byte) 0x36);
hearGarbageFalcon(parent, (byte) 0x37);
assertThat(heldFalcon(parent, 0)).isEqualTo(garbage(666, (byte) 0x36));
}
// ------------------------------------------------------------------------------------ below H_fin: today
/** Below H_fin, with the judge armed, the first Falcon seal of an index stays: judged by nobody. */
@Test
public void belowHFinAGarbageFalconSealHeardFirstStaysAsToday() throws Exception {
setFinalityBlock(H_FIN_ABOVE);
final BlockHeader parent = parentOf(V2 - 2L);
hearGarbageFalcon(parent, (byte) 0x38);
hearFalcon(parent, 0, K);
assertThat(heldFalcon(parent, 0))
.describedAs("below H_fin the first seal of index 0 stays, as in today's binary")
.isEqualTo(garbage(666, (byte) 0x38));
assertThat(heldFalcon(parent, 1)).isEqualTo(genuine.get(1));
}
/** Below H_fin the first SLH-DSA seal of an index stays too. */
@Test
public void belowHFinAGarbageSlhDsaSealHeardFirstStaysAsToday() throws Exception {
setFinalityBlock(H_FIN_ABOVE);
final BlockHeader parent = parentOf(V2 + 9L);
PqSealCache.instance()
.recordExtras(
parent.getNumber(),
parent.getHash(),
List.of(new SchemeSeal(SealSchemes.SLH_DSA_128S.wireId(), 0, garbage(7856, (byte) 0x39))));
hearSlh(parent, 0, K);
final List<SchemeSeal> held = PqSealCache.instance().extrasFor(parent.getNumber(), parent.getHash());
assertThat(held).describedAs("one SLH-DSA slot per index, K indices").hasSize(K);
assertThat(held)
.filteredOn(s -> s.getValidatorIndex() == 0)
.extracting(SchemeSeal::getSignature)
.describedAs("below H_fin the first SLH-DSA seal of index 0 stays, as in today's binary")
.containsExactly(garbage(7856, (byte) 0x39));
}
/** Below H_fin a restore from the head file displaces nothing held (putIfAbsent, as today). */
@Test
public void belowHFinTheRestoreDisplacesNothingAsToday() throws Exception {
setFinalityBlock(H_FIN_ABOVE);
final BlockHeader head = parentOf(V2 - 2L);
hearGarbageFalcon(head, (byte) 0x3a); // index 0 holds garbage in memory
final List<FalconSeal> onFile = new ArrayList<>();
for (int i = 0; i < K; i++) {
onFile.add(new FalconSeal(i, genuineFalcon(head, i)));
}
// the head file holds the K genuine seals (as after a restart onto a head whose seals were written)
PqSealStore.writeAtomically(
PqSealCache.instance().persistenceFile(),
PqSealStore.encode(CHAIN_ID, head.getNumber(), head.getHash().getBytes(), onFile));
final int restored =
PqSealCache.instance()
.restoreFromDisk(head.getNumber(), head.getHash(), PqSignerRegistry.falconSealSupport());
assertThat(restored).describedAs("the K seals on file verify, as today").isEqualTo(K);
assertThat(heldFalcon(head, 0))
.describedAs("below H_fin the restored seal does not displace the one held, as in today's binary")
.isEqualTo(garbage(666, (byte) 0x3a));
assertThat(heldFalcon(head, 1)).isEqualTo(onFile.get(1).getSignature());
}
// ------------------------------------------------------------------------------------ helpers
/** Sets H_fin directly: the store reads only AerePqFinality.active (see the class comment). */
private static void setFinalityBlock(final long hFin) throws Exception {
final Field f = AerePqFinality.class.getDeclaredField("finalityBlock");
f.setAccessible(true);
f.setLong(null, hFin);
assertThat(AerePqFinality.finalityBlock()).isEqualTo(hFin);
}
private void armHybrid(final String schedule) throws Exception {
if (slhKeys.isEmpty()) {
for (int i = 0; i < N; i++) {
slhKeys.add(SealSchemes.SLH_DSA_128S.generate(random));
}
}
final StringBuilder p = new StringBuilder();
p.append("formatVersion=hybrid-1\nchainId=").append(CHAIN_ID).append("\ncount=").append(N).append('\n');
for (int i = 0; i < N; i++) {
p.append(i).append(".addr=").append(validators.get(i).toHexString()).append('\n');
p.append(i)
.append(".key.")
.append(SLH)
.append('=')
.append(Bytes.wrap(slhKeys.get(i).publicRegistryForm()).toHexString())
.append('\n');
}
final Path reg = tmp.resolve("hybrid-registry.properties");
Files.writeString(reg, p.toString());
System.setProperty(HybridSealSupport.PROPERTY_SCHEDULE, schedule);
System.setProperty(HybridSealSupport.PROPERTY_REGISTRY, reg.toAbsolutePath().toString());
HybridSealSupport.resetForTesting();
assertThat(HybridSealSupport.instance().registry()).describedAs("hybrid registry loaded").isPresent();
}
private static BlockHeader parentOf(final long number) {
return new BlockHeaderTestFixture().number(number).buildHeader();
}
private BftExtraData base() {
return new BftExtraData(
Bytes32.ZERO, Collections.emptyList(), Optional.empty(), 0, validators, Collections.emptyList());
}
private static Bytes32 message(final BlockHeader parent) {
return PqAnchor.commitMessage(CHAIN_ID, parent.getNumber(), parent.getHash().getBytes());
}
private final Map<Integer, Bytes> genuine = new HashMap<>();
private Bytes genuineFalcon(final BlockHeader parent, final int i) {
final FalconSigner signer = new FalconSigner();
signer.init(true, falconKeys.get(i));
return Bytes.wrap(signer.generateSignature(message(parent).toArray()));
}
/** Records the genuine Falcon seals of indices [from, to); a Falcon signature is randomised, so the
* seal of an index is the one recorded here ({@link #genuine}, one parent per test). */
private void hearFalcon(final BlockHeader parent, final int from, final int to) {
final List<FalconSeal> heard = new ArrayList<>();
for (int i = from; i < to; i++) {
final Bytes sig = genuineFalcon(parent, i);
genuine.put(i, sig);
heard.add(new FalconSeal(i, sig));
}
PqSealCache.instance().record(parent.getNumber(), parent.getHash(), heard);
}
private void hearGarbageFalcon(final BlockHeader parent, final byte fill) {
PqSealCache.instance()
.record(parent.getNumber(), parent.getHash(), List.of(new FalconSeal(0, garbage(666, fill))));
}
private void hearSlh(final BlockHeader parent, final int from, final int to) {
final List<SchemeSeal> extras = new ArrayList<>();
for (int i = from; i < to; i++) {
extras.add(
new SchemeSeal(
SealSchemes.SLH_DSA_128S.wireId(),
i,
Bytes.wrap(
SealSchemes.SLH_DSA_128S
.sign(slhKeys.get(i).privateKey(), message(parent).toArray())
.orElseThrow())));
}
PqSealCache.instance().recordExtras(parent.getNumber(), parent.getHash(), extras);
}
private static Bytes heldFalcon(final BlockHeader parent, final int index) {
return PqSealCache.instance().sealsFor(parent.getNumber(), parent.getHash()).stream()
.filter(s -> s.getValidatorIndex() == index)
.map(FalconSeal::getSignature)
.findFirst()
.orElse(Bytes.EMPTY);
}
private static Bytes garbage(final int length, final byte fill) {
final byte[] b = new byte[length];
Arrays.fill(b, fill);
return Bytes.wrap(b);
}
private static ProtocolContext contextWith(final Collection<Address> vs) {
final ValidatorProvider validatorProvider =
mock(ValidatorProvider.class, withSettings().strictness(Strictness.LENIENT));
when(validatorProvider.getValidatorsForBlock(any())).thenReturn(vs);
when(validatorProvider.getValidatorsAfterBlock(any())).thenReturn(vs);
final BftContext bftContext =
mock(BftContext.class, withSettings().strictness(Strictness.LENIENT));
when(bftContext.getValidatorProvider()).thenReturn(validatorProvider);
when(bftContext.as(any())).thenReturn(bftContext);
return new ProtocolContext.Builder().withConsensusContext(bftContext).build();
}
private static void resetFalconSingleton() throws Exception {
final Field f = FalconSealSupport.class.getDeclaredField("instance");
f.setAccessible(true);
f.set(null, null);
}
}

2
anchor/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/Proposal.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-bft-bounded-message-lists.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.consensus.qbft.core.messagewrappers;

2
anchor/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/RoundChange.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-bft-bounded-message-lists.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.consensus.qbft.core.messagewrappers;

203
anchor/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454PendingCommits.java Normal file
View File

@ -0,0 +1,203 @@
/*
* Copyright contributors to the Aere Network.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.consensus.qbft.core.statemachine;
import org.hyperledger.besu.consensus.common.bft.AerePqFinality;
import org.hyperledger.besu.consensus.qbft.core.messagewrappers.Commit;
import org.hyperledger.besu.crypto.SECPSignature;
import org.hyperledger.besu.crypto.SignatureAlgorithmFactory;
import org.hyperledger.besu.datatypes.Address;
import org.hyperledger.besu.datatypes.Hash;
import org.hyperledger.besu.ethereum.core.Util;
import org.hyperledger.besu.ethereum.rlp.BytesValueRLPInput;
import org.hyperledger.besu.ethereum.rlp.RLPInput;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Optional;
import java.util.function.Predicate;
import org.apache.tuweni.bytes.Bytes;
/**
* AERE D-454 (2026-09-30): the Commit messages a {@link RoundState} hears BEFORE its proposal, from
* H_fin (AIP-22 stage 4).
*
* <p>WHY. From H_fin the author of a Commit is the id its post-quantum seal index names, when that
* id is the signer of the commit seal; the Falcon seal itself is NOT verified to give the author
* ({@code AerePqFinality.authorFromSeal}), and the envelope's ECDSA key is not read. So a copy of
* validator A's commit seal in anyone's envelope, with a garbage Falcon seal, is "A's". The
* upstream RoundState kept the first Commit of each author before the proposal
* ({@code putIfAbsent}, no validation possible without the proposal) and removed the invalid ones at
* the proposal: the copy took A's place, A's own message was dropped, the copy was then removed, and
* A's vote was missing from the round for as long as gossip deduplication keeps A's message from
* coming back. Measured by the probe AereD412RoundStateProbe (consensus-pqc/d412-reparatie).
*
* <p>WHAT IT DOES. Each message waits here keyed by its bytes (a message heard twice waits once),
* within a quota per (claimed author, envelope class): SELF when the envelope is signed by the same
* ECDSA key as the commit seal (what Besu and Nethermind emit), FOREIGN otherwise. A copy made by
* anyone other than A is FOREIGN, so copies can never fill the quota A's own message needs; and a
* FOREIGN message may still be an honest vote (from H_fin the envelope does not decide the author),
* so it is not refused, only kept apart. When the proposal arrives, {@link #release} validates the
* waiting messages, SELF first, each class in arrival order, and a message takes its author's slot
* only if it VALIDATES and the slot is still free; a message whose author already holds a slot is
* not validated at all. The quota is checked before the envelope is classified (two ECDSA
* recoveries), so once both classes of an author are full nothing more is recovered for it.
*
* <p>BOUNDS. At most {@code 2 * MAX_PER_CLASS} messages per claimed author, and the authors that
* reach a RoundState are the validators of its height (QbftController.processMessage refuses any
* other author, and from H_fin a Commit without a bound index has NO_AUTHOR, never in a set).
*
* <p>Below H_fin nothing waits here and RoundState is exactly the upstream one ({@link #governs}).
*/
final class AereD454PendingCommits {
/** Per (claimed author, envelope class): how many distinct messages may wait for the proposal. */
static final int MAX_PER_CLASS = 2;
/** The envelope is signed by the commit seal's signer. */
static final int SELF = 0;
/** Any other envelope (another key, or one that does not recover). */
static final int FOREIGN = 1;
private static final class Held {
private final Commit commit;
private final int envelope;
private Held(final Commit commit, final int envelope) {
this.commit = commit;
this.envelope = envelope;
}
}
private final Map<Hash, Held> byFingerprint = new LinkedHashMap<>();
private final Map<Address, int[]> perAuthor = new HashMap<>();
private long classifications;
/**
* Whether a Commit of this height waits here before the proposal: from H_fin only.
*
* @param height the height of the round
* @return true from H_fin
*/
static boolean governs(final long height) {
return AerePqFinality.active(height);
}
/**
* Keeps a Commit heard before the proposal, within its author's quota for its envelope class.
*
* @param commit the commit
* @return true when this message is newly kept; false when it was already kept or its quota is full
*/
boolean hold(final Commit commit) {
final Address author = commit.getAuthor();
final int[] counted = perAuthor.get(author);
if (counted != null && counted[SELF] >= MAX_PER_CLASS && counted[FOREIGN] >= MAX_PER_CLASS) {
// both classes of this author are full: dropped before the two ECDSA recoveries
return false;
}
final Bytes signed = commit.getSignedPayload().encode();
final Hash fingerprint = Hash.hash(signed);
if (byFingerprint.containsKey(fingerprint)) {
return false;
}
final int envelope = envelopeOf(commit, signed);
classifications++;
final int[] quota = perAuthor.computeIfAbsent(author, a -> new int[2]);
if (quota[envelope] >= MAX_PER_CLASS) {
return false;
}
quota[envelope]++;
byFingerprint.put(fingerprint, new Held(commit, envelope));
return true;
}
/**
* The proposal arrived: every waiting message is judged, SELF first then FOREIGN, each class in
* arrival order, and takes its author's slot only when {@code validates} passes and the slot is
* free. Nothing waits afterwards.
*
* @param validates the round's Commit validation (MessageValidator.validateCommit)
* @param slots the round's counted commits, by author
*/
void release(final Predicate<Commit> validates, final Map<Address, Commit> slots) {
for (final int envelope : new int[] {SELF, FOREIGN}) {
for (final Held held : byFingerprint.values()) {
if (held.envelope != envelope) {
continue;
}
final Address author = held.commit.getAuthor();
if (slots.containsKey(author)) {
continue;
}
if (validates.test(held.commit)) {
slots.put(author, held.commit);
}
}
}
byFingerprint.clear();
perAuthor.clear();
}
/**
* How many messages wait.
*
* @return the number of waiting messages
*/
int size() {
return byFingerprint.size();
}
/**
* How many envelopes were classified (two ECDSA recoveries each), for the work bound.
*
* @return the number of classifications
*/
long classifications() {
return classifications;
}
/**
* SELF when the envelope is signed by the same ECDSA key as the commit seal, FOREIGN otherwise
* (also when either does not recover).
*
* @param commit the commit
* @param signed its encoded signed payload
* @return {@link #SELF} or {@link #FOREIGN}
*/
static int envelopeOf(final Commit commit, final Bytes signed) {
try {
final Optional<Address> sealSigner =
commit.getSignedPayload().getPayload().commitSealSigner();
if (sealSigner.isEmpty()) {
return FOREIGN;
}
final RLPInput in = new BytesValueRLPInput(signed, false);
in.enterList();
in.skipNext();
final SECPSignature envelope =
in.readBytes(SignatureAlgorithmFactory.getInstance()::decodeSignature);
final Address envelopeSigner =
Util.signatureToAddress(
envelope, commit.getSignedPayload().getPayload().hashForSignature());
return sealSigner.get().equals(envelopeSigner) ? SELF : FOREIGN;
} catch (final RuntimeException e) {
return FOREIGN;
}
}
}

21
anchor/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/RoundState.java
View File

@ -58,6 +58,9 @@ public class RoundState {
// to send out in a PrepareCertificate.
private final Map<Address, Prepare> prepareMessages = new LinkedHashMap<>();
private final Map<Address, Commit> commitMessages = new LinkedHashMap<>();
// AERE D-454 (2026-09-30): from H_fin a Commit heard before the proposal waits here, keyed by its bytes, within a quota
// per (claimed author, envelope class); it takes its author's slot in commitMessages only once it validates.
private final AereD454PendingCommits aereD454Pending = new AereD454PendingCommits();
private boolean prepared = false;
private boolean committed = false;
@ -109,6 +112,9 @@ public class RoundState {
proposalMessage = Optional.of(msg);
prepareMessages.entrySet().removeIf(e -> !validator.validatePrepare(e.getValue()));
commitMessages.entrySet().removeIf(e -> !validator.validateCommit(e.getValue()));
// AERE D-454: the commits kept aside before the proposal (from H_fin) are judged now, SELF-enveloped first; each
// takes its author's slot only if it validates. Below H_fin nothing is kept aside, so this calls nothing.
aereD454Pending.release(validator::validateCommit, commitMessages);
updateState();
return true;
}
@ -136,6 +142,21 @@ public class RoundState {
* @param msg the msg
*/
public void addCommitMessage(final Commit msg) {
// AERE D-454 (2026-09-30): from H_fin (AIP-22 stage 4) the author of a Commit is read from its seal index WITHOUT the
// Falcon check, so a copy of A's commit seal in anyone's envelope is "A's". Before the proposal it took A's slot here, A's
// own message was dropped by putIfAbsent, and the copy was removed at the proposal: A's vote was lost for as long as
// gossip deduplication keeps A's message from coming back. From H_fin a slot is taken only after validation. Below H_fin
// this method is exactly the upstream one.
if (AereD454PendingCommits.governs(roundIdentifier.getSequenceNumber())) {
if (proposalMessage.isEmpty()) {
aereD454Pending.hold(msg);
} else if (!commitMessages.containsKey(msg.getAuthor()) && validator.validateCommit(msg)) {
commitMessages.putIfAbsent(msg.getAuthor(), msg);
LOG.trace("Round state added commit message commit={}", msg);
}
updateState();
return;
}
if (proposalMessage.isEmpty() || validator.validateCommit(msg)) {
commitMessages.putIfAbsent(msg.getAuthor(), msg);
LOG.trace("Round state added commit message commit={}", msg);

317
anchor/consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454RoundStateTest.java Normal file
View File

@ -0,0 +1,317 @@
/*
* Copyright contributors to the Aere Network.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.consensus.qbft.core.statemachine;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import org.hyperledger.besu.consensus.common.bft.ConsensusRoundIdentifier;
import org.hyperledger.besu.consensus.common.bft.FalconSeal;
import org.hyperledger.besu.consensus.qbft.core.messagewrappers.Commit;
import org.hyperledger.besu.consensus.qbft.core.messagewrappers.Proposal;
import org.hyperledger.besu.consensus.qbft.core.validation.AereD412Fixture;
import org.hyperledger.besu.consensus.qbft.core.validation.AereD412Fixture.TBlock;
import org.hyperledger.besu.consensus.qbft.core.validation.CommitValidator;
import org.hyperledger.besu.consensus.qbft.core.validation.MessageValidator;
import org.hyperledger.besu.datatypes.Hash;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import org.apache.tuweni.bytes.Bytes;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
/**
* AERE D-454 (2026-09-30): the regression proof of RoundState from H_fin (AIP-22 stage 4), the
* probe AereD412RoundStateProbe turned around. The fixture is the D-412 one (AereD412Fixture): real
* Falcon keys, the finality regime installed, the REAL CommitValidator with the post-quantum
* enforcement armed. Two actors, two contents, two positions: A (node 0) with its own commit, X
* (node 3) with a copy of A's commit seal in X's envelope, and the copy before or after A's own
* message and the proposal. Every test has a planted twin (planteaza-d454.py) that must turn it red;
* the harness d454-roundstate.sh checks that each plant reddens exactly the tests it names.
*/
public class AereD454RoundStateTest {
private static final long H = 100L;
private static final int ROUND = 1;
private static final Hash P = Hash.hash(Bytes.of(9));
// static regimes (finality, identity, anchor form) are reset before AND after each test, so no
// other class sharing the JVM can decide which side of H_fin a test runs on (lesson D-200)
@BeforeEach
public void clean() {
AereD412Fixture.uninstallAll();
}
@AfterEach
public void disarm() {
AereD412Fixture.uninstallAll();
}
/** A RoundState on block b whose commit validation is the REAL CommitValidator, every call recorded. */
private static RoundState round(final AereD412Fixture fx, final TBlock b, final List<Commit> validated) {
final TBlock proposal = b.atRound(ROUND);
final Hash commitHash = fx.blockInterface.replaceRoundForCommitBlock(proposal, ROUND).getHash();
final CommitValidator real =
new CommitValidator(
fx.addresses(0, 1, 2, 3),
new ConsensusRoundIdentifier(H, ROUND),
proposal.getHash(),
commitHash,
commitHash,
fx.armed());
final MessageValidator mv = mock(MessageValidator.class);
when(mv.validateProposal(any())).thenReturn(true);
when(mv.validateCommit(any()))
.thenAnswer(
inv -> {
final Commit c = inv.getArgument(0);
validated.add(c);
return real.validate(c);
});
return new RoundState(new ConsensusRoundIdentifier(H, ROUND), 3, mv);
}
/** X's copy of A's commit seal, with a garbage Falcon seal claiming A's index. */
private static Commit garbageCopy(final AereD412Fixture fx, final Commit ofA, final int variant) {
return fx.copyOf(
ofA, 3, ROUND, Optional.of(new FalconSeal(0, Bytes.repeat((byte) (0x31 + variant), 600))));
}
private static AereD412Fixture finality() {
final AereD412Fixture fx = new AereD412Fixture(4, true);
fx.installFinality();
return fx;
}
// ------------------------------------------------------------------------------------------ T1
@Test
public void fromHFinACopyBeforeTheProposalNoLongerTakesTheHonestSlot() {
final AereD412Fixture fx = finality();
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
final Commit copy = garbageCopy(fx, ofA, 0);
assertThat(copy.getAuthor()).as("precondition: from H_fin the copy is authored by A").isEqualTo(fx.address(0));
final RoundState rs = round(fx, b, new ArrayList<>());
rs.addCommitMessage(copy);
rs.addCommitMessage(ofA);
rs.addCommitMessage(fx.honest(1, b, ROUND));
rs.addCommitMessage(fx.honest(2, b, ROUND));
assertThat(rs.isCommitted()).as("nothing is committed before the proposal").isFalse();
assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
assertThat(rs.getCommitSeals()).as("A's vote is counted: the copy took no slot").hasSize(3);
assertThat(rs.isCommitted()).as("three honest commits heard: the round is committed").isTrue();
assertThat(rs.getFalconSeals())
.as("the copy's garbage Falcon seal is not among the round's seals")
.doesNotContain(copy.getFalconSeal().orElseThrow())
.contains(ofA.getFalconSeal().orElseThrow());
}
// ------------------------------------------------------------------------------------------ T2
@Test
public void fromHFinCopiesFillOnlyTheForeignQuota() {
final AereD412Fixture fx = finality();
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
final RoundState rs = round(fx, b, new ArrayList<>());
for (int i = 0; i < 20; i++) {
rs.addCommitMessage(garbageCopy(fx, ofA, i));
}
rs.addCommitMessage(ofA);
rs.addCommitMessage(fx.honest(1, b, ROUND));
rs.addCommitMessage(fx.honest(2, b, ROUND));
assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
assertThat(rs.isCommitted())
.as("twenty copies before A's own message: A's vote still counts")
.isTrue();
assertThat(rs.getFalconSeals()).hasSize(3).contains(ofA.getFalconSeal().orElseThrow());
}
// ------------------------------------------------------------------------------------------ T3
@Test
public void fromHFinAnUnvalidatedCopyNeverCounts() {
final AereD412Fixture fx = finality();
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
final RoundState rs = round(fx, b, new ArrayList<>());
rs.addCommitMessage(garbageCopy(fx, ofA, 0));
rs.addCommitMessage(fx.honest(1, b, ROUND));
rs.addCommitMessage(fx.honest(2, b, ROUND));
assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
assertThat(rs.getCommitSeals()).as("the copy is not A's vote").hasSize(2);
assertThat(rs.isCommitted()).as("two honest commits and a copy: not committed").isFalse();
rs.addCommitMessage(ofA);
assertThat(rs.isCommitted()).as("A's own message after the proposal completes the round").isTrue();
}
// ------------------------------------------------------------------------------------------ T4
@Test
public void fromHFinTheReleaseValidatesSelfFirstAndSkipsCountedAuthors() {
final AereD412Fixture fx = finality();
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
final Commit ofB = fx.honest(1, b, ROUND);
final List<Commit> validated = new ArrayList<>();
final RoundState rs = round(fx, b, validated);
rs.addCommitMessage(garbageCopy(fx, ofA, 0));
rs.addCommitMessage(garbageCopy(fx, ofA, 1));
rs.addCommitMessage(ofA);
rs.addCommitMessage(ofB);
assertThat(validated).as("nothing is validated before the proposal").isEmpty();
assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
// BftMessage.equals compares the whole signed payload: each copy differs from ofA (envelope, seal)
assertThat(validated)
.as("A's and B's own messages validated, the copies never (A's slot was taken)")
.containsExactly(ofA, ofB);
assertThat(rs.getCommitSeals()).hasSize(2);
rs.addCommitMessage(fx.honest(2, b, ROUND));
assertThat(rs.isCommitted()).isTrue();
}
// ------------------------------------------------------------------------------------------ T5
@Test
public void fromHFinACountedAuthorIsNotValidatedAgainAfterTheProposal() {
final AereD412Fixture fx = finality();
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
final Commit ofB = fx.honest(1, b, ROUND);
final Commit ofC = fx.honest(2, b, ROUND);
final List<Commit> validated = new ArrayList<>();
final RoundState rs = round(fx, b, validated);
assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
rs.addCommitMessage(ofA);
for (int i = 0; i < 3; i++) {
rs.addCommitMessage(garbageCopy(fx, ofA, i));
}
rs.addCommitMessage(ofB);
rs.addCommitMessage(ofC);
assertThat(validated)
.as("copies of an author already counted are not validated")
.containsExactly(ofA, ofB, ofC);
assertThat(rs.isCommitted()).isTrue();
}
// ------------------------------------------------------------------------------------------ T6
@Test
public void fromHFinTheCheapGuardDropsBeforeAnyRecovery() {
final AereD412Fixture fx = finality();
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
// three distinct SELF messages of A (its envelope, its commit seal; Falcon by three different keys)
final Commit s1 = fx.commit(0, b, ROUND, 0, 0);
final Commit s2 = fx.commit(0, b, ROUND, 0, 1);
final Commit s3 = fx.commit(0, b, ROUND, 0, 2);
final AereD454PendingCommits pending = new AereD454PendingCommits();
assertThat(pending.hold(s1)).isTrue();
assertThat(pending.hold(s2)).isTrue();
assertThat(pending.hold(s3)).as("A's SELF quota is full").isFalse();
int held = 0;
for (int i = 0; i < 50; i++) {
if (pending.hold(garbageCopy(fx, ofA, i))) {
held++;
}
}
assertThat(held).as("the FOREIGN quota").isEqualTo(AereD454PendingCommits.MAX_PER_CLASS);
assertThat(pending.size()).isEqualTo(2 * AereD454PendingCommits.MAX_PER_CLASS);
assertThat(pending.classifications())
.as("once both classes of A are full, nothing more is recovered for A")
.isEqualTo(3L + AereD454PendingCommits.MAX_PER_CLASS);
}
// ------------------------------------------------------------------------------------------ T7
@Test
public void fromHFinTheSameMessageTwiceTakesOnePlace() {
final AereD412Fixture fx = finality();
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
final Commit c1 = garbageCopy(fx, ofA, 1);
final AereD454PendingCommits pending = new AereD454PendingCommits();
assertThat(pending.hold(ofA)).isTrue();
assertThat(pending.hold(ofA)).as("the same bytes again").isFalse();
assertThat(pending.hold(c1)).isTrue();
assertThat(pending.hold(c1)).as("the same copy again").isFalse();
assertThat(pending.size()).isEqualTo(2);
assertThat(pending.hold(garbageCopy(fx, ofA, 2)))
.as("a second, distinct copy still fits the FOREIGN quota: repeats took no place")
.isTrue();
assertThat(pending.hold(garbageCopy(fx, ofA, 3))).as("the FOREIGN quota is full").isFalse();
assertThat(pending.size()).isEqualTo(3);
assertThat(pending.hold(fx.honest(1, b, ROUND))).as("another author has its own quota").isTrue();
}
// ------------------------------------------------------------------------------------------ T8
@Test
public void belowHFinTheUpstreamPathIsKeptCallByCall() {
final AereD412Fixture fx = new AereD412Fixture(4, true);
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
final Commit ofA2 = fx.commit(0, b, ROUND, 0, 1);
final Commit copy = garbageCopy(fx, ofA, 0);
final Commit ofB = fx.honest(1, b, ROUND);
final Commit ofC = fx.honest(2, b, ROUND);
assertThat(copy.getAuthor()).as("precondition: below H_fin the copy is X's").isEqualTo(fx.address(3));
final List<Commit> validated = new ArrayList<>();
final RoundState rs = round(fx, b, validated);
rs.addCommitMessage(copy);
rs.addCommitMessage(ofA);
rs.addCommitMessage(ofA2);
rs.addCommitMessage(ofB);
assertThat(validated).as("upstream: nothing is validated before the proposal").isEmpty();
assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
rs.addCommitMessage(ofC);
rs.addCommitMessage(ofA);
// upstream: the first message of each author is kept before the proposal (putIfAbsent; A's second
// dropped), validated at the proposal in insertion order, then every later message is validated
assertThat(validated)
.as("the upstream sequence of validations, call by call")
.containsExactly(copy, ofA, ofB, ofC, ofA);
assertThat(rs.isCommitted()).isTrue();
assertThat(rs.getCommitSeals()).hasSize(3);
}
// ------------------------------------------------------------------------------------------ T9
@Test
public void fromHFinAValidForeignCopyIsAnHonestVote() {
final AereD412Fixture fx = finality();
final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
final Commit ofA = fx.honest(0, b, ROUND);
// X re-envelopes A's whole vote: A's commit seal AND A's valid Falcon seal
final Commit reEnveloped = fx.copyOf(ofA, 3, ROUND, ofA.getFalconSeal());
assertThat(reEnveloped.getAuthor()).isEqualTo(fx.address(0));
final RoundState rs = round(fx, b, new ArrayList<>());
rs.addCommitMessage(reEnveloped);
rs.addCommitMessage(fx.honest(1, b, ROUND));
rs.addCommitMessage(fx.honest(2, b, ROUND));
assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
assertThat(rs.isCommitted())
.as("from H_fin the envelope does not decide the author: A's vote in X's envelope counts")
.isTrue();
assertThat(rs.getCommitSeals()).hasSize(3);
}
}

590
anchor/consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/validation/AereD412Fixture.java Normal file
View File

@ -0,0 +1,590 @@
/* AERE D-412 / capul decis, pasul 1: fixtura comuna a probelor predicatului unic si a colectorului.
*
* Doi actori, doua continuturi, doua pozitii (regula casei): noduri QBFT reale (chei ECDSA reale,
* mesaje semnate cu MessageFactory din amonte), chei Falcon-512 REALE prin stratul de scheme, un
* registru de proba care leaga indexul i de nodul i si NUMARA verificarile, si un model de bloc in
* care amprenta depinde de runda exact ca in QBFT: digestul mesajului pastreaza runda, amprenta de pe
* lant (runda 0) nu. Nimic mockuit pe drumul criptografic.
*
* 2026-09-30 (revizuirea pasului 1):
* - commit() nu mai trece prin AereVoteCounts.Target (functia masurata): urmeaza, rand cu rand,
* drumul EMITATORULUI din QbftRound (commitHashFor -> createCommitBlock, pqOnchainHashOf,
* pqSealMessageFor -> PqAnchorProducer.commitSealMessage, createCommit(round, block.getHash(),
* ...)), pe blocul PROPUS al rundei; cu ancora armata (PqAnchorProducer.useConfigForTesting)
* sigiliul PQ semneaza forma de ancora. Legatura cu blocurile si mesajele reale o masoara
* AereD412RealHeadersTest (consensus/qbft) si AereD412RealRoundTest (integrare);
* - installFinality(): regimul REAL de la H_fin (AerePqIdentity + AerePqFinality instalate, ca in
* AereFinalityMessageTest), in care autorul unui Commit e id-ul indexului din sigiliu cand acela e
* semnatarul sigiliului de commit, deci plicul nu mai decide autorul;
* - registrul poate fi pus PENDING (nimic legat, ca un late-anchor neactivat, D-358) si are o
* generatie care se schimba la fiecare trecere;
* - blocul poate declara o marime (sizeOf), iar interfata numara inlocuirile de runda.
* 2026-09-30, a doua revizuire: hybridArmed() (Falcon + SLH-DSA cerut la orice inaltime, ca la un parinte de ancora) si
* commit(..., extras), ca o cheie sa poata cere predicatului refuzul cel mai scump: sigiliul Falcon verificat, apoi refuzul
* mesajului pentru SLH-DSA. */
package org.hyperledger.besu.consensus.qbft.core.validation;
import static org.mockito.Mockito.mock;
import org.hyperledger.besu.consensus.common.bft.AerePqFinality;
import org.hyperledger.besu.consensus.common.bft.AerePqIdentity;
import org.hyperledger.besu.consensus.common.bft.ConsensusRoundIdentifier;
import org.hyperledger.besu.consensus.common.bft.FalconSeal;
import org.hyperledger.besu.consensus.common.bft.HybridSignerRegistry;
import org.hyperledger.besu.consensus.common.bft.PqRegistryV3;
import org.hyperledger.besu.consensus.common.bft.PqSchemeSchedule;
import org.hyperledger.besu.consensus.common.bft.PqSignerRegistry;
import org.hyperledger.besu.consensus.common.bft.SchemeSeal;
import org.hyperledger.besu.consensus.common.bft.SealScheme;
import org.hyperledger.besu.consensus.common.bft.SealSchemes;
import org.hyperledger.besu.consensus.common.bft.blockcreation.PqAnchorProducer;
import org.hyperledger.besu.consensus.qbft.core.messagewrappers.Commit;
import org.hyperledger.besu.consensus.qbft.core.types.QbftBlock;
import org.hyperledger.besu.consensus.qbft.core.types.QbftBlockCodec;
import org.hyperledger.besu.consensus.qbft.core.types.QbftBlockHeader;
import org.hyperledger.besu.consensus.qbft.core.types.QbftBlockInterface;
import org.hyperledger.besu.crypto.SECPSignature;
import org.hyperledger.besu.crypto.SecureRandomProvider;
import org.hyperledger.besu.cryptoservices.NodeKey;
import org.hyperledger.besu.datatypes.Address;
import org.hyperledger.besu.datatypes.Hash;
import java.security.SecureRandom;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Properties;
import java.util.TreeMap;
import java.util.concurrent.atomic.AtomicLong;
import org.apache.tuweni.bytes.Bytes;
import org.apache.tuweni.bytes.Bytes32;
/** Shared fixture for the D-412 step-1 proofs. */
public final class AereD412Fixture {
/** The chain id of the post-quantum messages in the finality regime. */
public static final long CHAIN = 28001L;
/** H_id = H_k = the bind height of the one v3 epoch of the finality regime. */
public static final long H_ID = 10L;
/** H_fin of the finality regime. */
public static final long H_FIN = 20L;
/** A block model whose hash depends on the round, like a QBFT committed-seal hash. */
public static final class TBlock implements QbftBlock {
private final long number;
private final Hash parent;
private final Bytes content;
private final long declaredBytes;
// the round is not kept as a field: it only enters the hash, as in a QBFT committed-seal hash
private final Hash hash;
/**
* @param number the height
* @param parent the on-chain hash of the parent
* @param content what distinguishes siblings
* @param round the round written in the block
*/
public TBlock(final long number, final Hash parent, final Bytes content, final int round) {
this(number, parent, content, round, 0L);
}
/**
* @param number the height
* @param parent the on-chain hash of the parent
* @param content what distinguishes siblings
* @param round the round written in the block
* @param declaredBytes the size the block declares (charged by sizeOf, not allocated)
*/
public TBlock(
final long number,
final Hash parent,
final Bytes content,
final int round,
final long declaredBytes) {
this.number = number;
this.parent = parent;
this.content = content;
this.declaredBytes = declaredBytes;
this.hash =
Hash.hash(
Bytes.concatenate(
Bytes.ofUnsignedLong(number),
parent.getBytes(),
content,
Bytes.ofUnsignedLong(declaredBytes),
Bytes.ofUnsignedInt(round)));
}
/**
* The same block at another round.
*
* @param r the round
* @return the block at round r
*/
public TBlock atRound(final int r) {
return new TBlock(number, parent, content, r, declaredBytes);
}
/**
* The on-chain hash, round 0.
*
* @return the hash
*/
public Hash onchain() {
return atRound(0).getHash();
}
/**
* The height.
*
* @return the number
*/
public long number() {
return number;
}
@Override
public QbftBlockHeader getHeader() {
final TBlock self = this;
return new QbftBlockHeader() {
@Override
public long getNumber() {
return self.number;
}
@Override
public long getTimestamp() {
return 0L;
}
@Override
public Address getCoinbase() {
return Address.ZERO;
}
@Override
public Hash getHash() {
return self.hash;
}
// QbftBlockHeader.getParentHash() is D-437's (lab and delivery tree); the FLEET tree, from which the
// AIP-22 stage 4 binary is built, does not declare it. Without @Override the fixture compiles on both
// (t4-livrare.sh, 2026-10-01: with it, the fleet tree refused to compile the fixture).
@SuppressWarnings("MissingOverride")
public Hash getParentHash() {
return self.parent;
}
};
}
@Override
public boolean isEmpty() {
return true;
}
@Override
public Hash getHash() {
return hash;
}
}
/** Replaces the round, as the QBFT block interface does, and counts the replacements. */
public static final class TInterface implements QbftBlockInterface {
private final AtomicLong replacements = new AtomicLong();
@Override
public QbftBlock replaceRoundForCommitBlock(final QbftBlock proposalBlock, final int roundNumber) {
replacements.incrementAndGet();
return ((TBlock) proposalBlock).atRound(roundNumber);
}
@Override
public QbftBlock replaceRoundAndProposerForProposalBlock(
final QbftBlock proposalBlock, final int roundNumber, final Address proposer) {
return replaceRoundForCommitBlock(proposalBlock, roundNumber);
}
/**
* How many round replacements were made (the recoding work of the collector).
*
* @return the count
*/
public long replacements() {
return replacements.get();
}
}
private final SecureRandom random = SecureRandomProvider.createSecureRandom();
private final Map<Integer, Address> bindings = new HashMap<>();
private final Map<Integer, byte[]> falconPublic = new HashMap<>();
private final Map<Integer, SealScheme.PrivateHandle> falconPrivate = new HashMap<>();
private final AtomicLong pqVerifyCalls = new AtomicLong();
private final AtomicLong generation = new AtomicLong();
private volatile boolean pending;
private volatile boolean v3;
private final List<Address> ids = new ArrayList<>();
/** The nodes; node i owns Falcon index i. */
public final QbftNodeList nodes;
/** The block interface. */
public final TInterface blockInterface = new TInterface();
/**
* Registry: index i bound to node i (to id i in the finality regime), verification by the REAL
* Falcon scheme, counted. While PENDING nothing is bound and nothing verifies, as a late-anchor
* registry before activation.
*/
public final PqSignerRegistry registry =
new PqSignerRegistry() {
@Override
public Address addressForIndexAtHistoric(final long blockNumber, final int validatorIndex) {
return addressForIndexAtOwnHead(blockNumber, validatorIndex);
}
@Override
public Address addressForIndexAtOwnHead(final long blockNumber, final int validatorIndex) {
if (pending) {
return null;
}
if (v3) {
return AerePqIdentity.v3IdAt(blockNumber, validatorIndex).orElse(null);
}
return bindings.get(validatorIndex);
}
@Override
public boolean verifyAtHistoric(
final long blockNumber, final int validatorIndex, final Bytes message, final Bytes signature) {
return verifyAtOwnHead(blockNumber, validatorIndex, message, signature);
}
@Override
public boolean verifyAtOwnHead(
final long blockNumber, final int validatorIndex, final Bytes message, final Bytes signature) {
pqVerifyCalls.incrementAndGet();
if (pending) {
return false;
}
final byte[] pk = falconPublic.get(validatorIndex);
return pk != null
&& SealSchemes.FALCON_512.verifyRaw(pk, message.toArray(), signature.toArray());
}
};
/**
* @param count how many nodes (and Falcon keys)
* @param falconKeys whether to generate Falcon keys (the flood proof does not need them)
*/
public AereD412Fixture(final int count, final boolean falconKeys) {
nodes = QbftNodeList.createNodes(count, mock(QbftBlockCodec.class));
for (int i = 0; i < count; i++) {
bindings.put(i, nodes.getNode(i).getAddress());
if (falconKeys) {
final SealScheme.GeneratedPair pair = SealSchemes.FALCON_512.generate(random);
falconPublic.put(i, pair.publicRegistryForm());
falconPrivate.put(i, pair.privateKey());
}
}
}
/**
* How many Falcon verifications the registry performed.
*
* @return the count
*/
public long pqVerifyCalls() {
return pqVerifyCalls.get();
}
/**
* The verifier generation: changes every time the registry changes state.
*
* @return the generation
*/
public long generation() {
return generation.get();
}
/**
* Put the registry PENDING (nothing bound, nothing verifies) or active; either way the generation
* moves.
*
* @param isPending the new state
*/
public void setRegistryPending(final boolean isPending) {
pending = isPending;
generation.incrementAndGet();
}
/**
* The bytes a block is charged: 100 for the header plus what a TBlock declares.
*
* @param block the block
* @return the bytes
*/
public long sizeOf(final QbftBlock block) {
return 100L + (block instanceof TBlock t ? t.declaredBytes : 0L);
}
/**
* The author of node i: its address, or its post-quantum id in the finality regime.
*
* @param i the node
* @return the author
*/
public Address address(final int i) {
return v3 ? ids.get(i) : nodes.getNode(i).getAddress();
}
/**
* The authors of the given nodes.
*
* @param idx the nodes
* @return the authors, in that order
*/
public List<Address> addresses(final int... idx) {
final List<Address> out = new ArrayList<>();
for (final int i : idx) {
out.add(address(i));
}
return out;
}
/**
* An armed enforcement over the counted registry.
*
* @return the enforcement, armed from height 0
*/
public PqCommitEnforcement armed() {
return new PqCommitEnforcement(0L, registry);
}
/**
* An armed HYBRID enforcement (2026-09-30, second review of step 1): Falcon through the counted
* registry, plus SLH-DSA demanded at every height (as at an anchor parent, D-329), with a hybrid
* registry binding index i to node i. A commit with a valid Falcon seal and no valid SLH-DSA seal is
* refused BY THE MESSAGE after its Falcon seal was verified: the most expensive refusal a key can
* make the predicate pay for.
*
* @return the enforcement, armed from height 0
*/
public PqCommitEnforcement hybridArmed() {
final Properties p = new Properties();
p.setProperty("formatVersion", HybridSignerRegistry.FORMAT_VERSION);
p.setProperty("chainId", Long.toString(CHAIN));
p.setProperty("count", Integer.toString(falconPublic.size()));
for (final Map.Entry<Integer, byte[]> e : falconPublic.entrySet()) {
p.setProperty(e.getKey() + ".addr", nodes.getNode(e.getKey()).getAddress().toHexString());
p.setProperty(
e.getKey() + ".key." + SealSchemes.FALCON_512.id(), Bytes.wrap(e.getValue()).toHexString());
}
return new PqCommitEnforcement(
0L,
registry,
PqSchemeSchedule.parse(
"0:" + SealSchemes.FALCON_512.id() + "+" + SealSchemes.SLH_DSA_128S.id()),
HybridSignerRegistry.fromProperties(p, "aere-d412-fixture"),
h -> true);
}
/**
* Installs the finality regime (AIP-22 stage 4) for these nodes, as AereFinalityMessageTest does: one
* v3 epoch bound at H_ID with each node's Falcon key and ECDSA claim, identity from H_ID (keys from
* H_ID), H_FIN with the production-shaped verifier. From then on authors are ids.
*/
public void installFinality() {
final SealScheme falcon = SealSchemes.FALCON_512;
final int n = falconPrivate.size();
if (n == 0) {
throw new IllegalStateException("the finality regime needs the Falcon keys");
}
final List<PqRegistryV3.Row> rows = new ArrayList<>();
for (int i = 0; i < n; i++) {
final NodeKey nk = nodes.getNode(i).getNodeKey();
final byte[] std = new byte[897];
std[0] = 0x09;
System.arraycopy(falconPublic.get(i), 0, std, 1, 896);
final byte[] id = PqRegistryV3.idOf(std);
final Bytes32 popMsg = PqRegistryV3.popMessage(CHAIN, H_ID, n, i, id, falcon.wireId(), std);
final byte[] pop = falcon.sign(falconPrivate.get(i), popMsg.toArrayUnsafe()).orElseThrow();
final TreeMap<Integer, PqRegistryV3.Key> keys = new TreeMap<>();
keys.put(falcon.wireId() & 0xff, new PqRegistryV3.Key(falcon, std, pop));
final PqRegistryV3.Row bare = new PqRegistryV3.Row(i, id, keys, null, Map.of(), null, null);
final Bytes32 rh = PqRegistryV3.rowHash(CHAIN, H_ID, n, bare);
final SECPSignature claim = nk.sign(PqRegistryV3.claimDigest(CHAIN, rh));
rows.add(
new PqRegistryV3.Row(
i,
id,
keys,
null,
Map.of(),
nodes.getNode(i).getAddress().getBytes().toArrayUnsafe(),
claim.encodedBytes().toArrayUnsafe()));
ids.add(Address.wrap(Bytes.wrap(id)));
}
final List<Bytes32> hashes = new ArrayList<>();
for (final PqRegistryV3.Row r : rows) {
hashes.add(PqRegistryV3.rowHash(CHAIN, H_ID, n, r));
}
final Bytes32 root = PqRegistryV3.merkleRoot(hashes);
final PqRegistryV3.Registry reg =
new PqRegistryV3.Registry(
CHAIN, H_ID, n, rows, hashes, root, PqRegistryV3.registryHash(CHAIN, H_ID, n, root));
final TreeMap<Long, PqRegistryV3.Registry> epochs = new TreeMap<>();
epochs.put(H_ID, reg);
AerePqFinality.uninstall();
AerePqIdentity.uninstall();
AerePqIdentity.install(new AerePqIdentity.Config(H_ID, epochs, H_ID));
AerePqFinality.install(
H_FIN,
CHAIN,
(h, i, m, s) ->
AerePqIdentity.v3KeyAt(h, i, "falcon-512")
.map(k -> falcon.verifyRaw(k, m.toArrayUnsafe(), s.toArrayUnsafe()))
.orElse(false));
v3 = true;
}
/** Disarms the finality regime (and the anchor form). */
public static void uninstallAll() {
AerePqFinality.uninstall();
AerePqIdentity.uninstall();
PqAnchorProducer.useConfigForTesting(null);
}
/**
* A commit of node {@code author} for {@code block} at round {@code round}, built on the EMITTER's
* path (QbftRound), not on AereVoteCounts.Target: the proposal of the round is the block at round r;
* the ECDSA commit seal signs commitHashFor(proposal) = replaceRoundForCommitBlock(proposal, r);
* the post-quantum message is PqAnchorProducer.commitSealMessage(number, round-0 hash of the
* proposal, commitHash), as pqSealMessageFor; the payload carries proposal.getHash(). The Falcon
* seal CLAIMS index {@code claimedIndex} and is signed by the key of {@code signerIndex} (no seal
* when claimedIndex < 0).
*
* @param author the ECDSA author
* @param block the block voted, any round form
* @param round the round
* @param claimedIndex the index written in the seal, or -1 for no seal
* @param signerIndex whose Falcon key signs
* @return the commit
*/
public Commit commit(
final int author, final TBlock block, final int round, final int claimedIndex, final int signerIndex) {
return commit(author, block, round, claimedIndex, signerIndex, List.of());
}
/**
* As {@link #commit(int, TBlock, int, int, int)}, carrying extra scheme seals beside the Falcon one
* (the whole payload, extras included, signed by the author's ECDSA key, as MessageFactory does).
*
* @param author the ECDSA author
* @param block the block voted, any round form
* @param round the round
* @param claimedIndex the index written in the seal, or -1 for no seal
* @param signerIndex whose Falcon key signs
* @param extras the extra scheme seals
* @return the commit
*/
public Commit commit(
final int author,
final TBlock block,
final int round,
final int claimedIndex,
final int signerIndex,
final List<SchemeSeal> extras) {
final TBlock proposal = block.atRound(round);
final Hash commitHash = blockInterface.replaceRoundForCommitBlock(proposal, round).getHash();
final SECPSignature ecdsa =
nodes.getNode(author).getNodeKey().sign(Bytes32.wrap(commitHash.getBytes()));
final Bytes32 pqMessage =
PqAnchorProducer.commitSealMessage(
proposal.number(),
() -> blockInterface.replaceRoundForCommitBlock(proposal, 0).getHash().getBytes(),
commitHash);
Optional<FalconSeal> seal = Optional.empty();
if (claimedIndex >= 0) {
seal = Optional.of(falconSeal(claimedIndex, signerIndex, pqMessage));
}
return nodes
.getMessageFactory(author)
.createCommit(
new ConsensusRoundIdentifier(proposal.number(), round), proposal.getHash(), ecdsa, seal, extras);
}
/**
* A Falcon seal claiming {@code claimedIndex}, signed by the key of {@code signerIndex} over
* {@code message}.
*
* @param claimedIndex the index written in the seal
* @param signerIndex whose key signs
* @param message the signed message
* @return the seal
*/
public FalconSeal falconSeal(final int claimedIndex, final int signerIndex, final Bytes32 message) {
final byte[] sig =
SealSchemes.FALCON_512.sign(falconPrivate.get(signerIndex), message.toArray()).orElseThrow();
return new FalconSeal(claimedIndex, Bytes.wrap(sig));
}
/**
* A copy of {@code original} as another key can make it: the original commit seal and digest, the
* given round and seal, the envelope signed by {@code envelopeNode}.
*
* @param original the commit copied
* @param envelopeNode the node whose key signs the envelope
* @param round the round written in the copy
* @param seal the post-quantum seal written in the copy
* @return the copy
*/
public Commit copyOf(
final Commit original, final int envelopeNode, final int round, final Optional<FalconSeal> seal) {
return nodes
.getMessageFactory(envelopeNode)
.createCommit(
new ConsensusRoundIdentifier(original.getRoundIdentifier().getSequenceNumber(), round),
original.getDigest(),
original.getCommitSeal(),
seal);
}
/**
* An honest commit: own ECDSA, own Falcon index and key.
*
* @param author the node
* @param block the block
* @param round the round
* @return the commit
*/
public Commit honest(final int author, final TBlock block, final int round) {
return commit(author, block, round, author, author);
}
/**
* A commit with an arbitrary digest (a flood message): valid message signature of the author, a
* fixed commit seal and a garbage Falcon seal.
*
* @param author the node
* @param height the height
* @param round the round
* @param digest the digest
* @param commitSeal a commit seal (not checked by anything a flood reaches)
* @return the commit
*/
public Commit flood(
final int author, final long height, final int round, final Hash digest, final SECPSignature commitSeal) {
return nodes
.getMessageFactory(author)
.createCommit(
new ConsensusRoundIdentifier(height, round),
digest,
commitSeal,
Optional.of(new FalconSeal(author, Bytes.repeat((byte) 7, 64))));
}
}

2
anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-drop-per-request-console-writes.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.ethereum.api.query;

2
anchor/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-tx-gas-cap-unsigned.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.ethereum.mainnet;

2
anchor/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/WithdrawalsValidator.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-bft-withdrawals-not-applicable.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.ethereum.mainnet;

2
anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-cap-prestatus-rlpx-connections.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.ethereum.eth.manager;

363
anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManager.java Normal file
View File

@ -0,0 +1,363 @@
/*
* Copyright contributors to Hyperledger Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*
* Modifications Copyright contributors to the Aere Network.
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
*/
package org.hyperledger.besu.ethereum.eth.manager.snap;
import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.core.Synchronizer;
import org.hyperledger.besu.ethereum.eth.SnapProtocol;
import org.hyperledger.besu.ethereum.eth.manager.EthMessage;
import org.hyperledger.besu.ethereum.eth.manager.EthMessages;
import org.hyperledger.besu.ethereum.eth.manager.EthPeer;
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV1;
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV2;
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
import org.hyperledger.besu.ethereum.p2p.network.ProtocolManager;
import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnection;
import org.hyperledger.besu.ethereum.p2p.rlpx.framing.FramingException;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.AbstractSnapMessageData;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.Capability;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.Message;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.MessageData;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.messages.DisconnectMessage.DisconnectReason;
import org.hyperledger.besu.ethereum.rlp.RLPException;
import org.hyperledger.besu.ethereum.worldstate.WorldStateStorageCoordinator;
import org.hyperledger.besu.metrics.BesuMetricCategory;
import org.hyperledger.besu.plugin.services.MetricsSystem;
import org.hyperledger.besu.plugin.services.metrics.Counter;
import java.math.BigInteger;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.CancellationException;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
import com.google.common.collect.ImmutableList;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class SnapProtocolManager implements ProtocolManager {
private static final Logger LOG = LoggerFactory.getLogger(SnapProtocolManager.class);
private final List<Capability> supportedCapabilities;
private final EthPeers ethPeers;
private final EthMessages snapMessages;
private final EthScheduler ethScheduler;
// AERE D-434 (683880baa6, GHSA-pcv4-pxhv-99m7): per-peer and global caps on snap GET_* requests
// concurrently scheduled on the EthScheduler services pool; 0 disables a cap.
private final int maxConcurrentRequestsPerPeer;
private final int maxConcurrentRequestsGlobal;
private final AtomicInteger globalInFlightRequests = new AtomicInteger(0);
private final Map<PeerConnection, AtomicInteger> perPeerInFlightRequests =
new ConcurrentHashMap<>();
private final Counter rejectedRequestsCounter;
public SnapProtocolManager(
final WorldStateStorageCoordinator worldStateStorageCoordinator,
final SnapSyncConfiguration snapConfig,
final EthPeers ethPeers,
final EthMessages snapMessages,
final EthScheduler ethScheduler,
final ProtocolSchedule protocolSchedule,
final ProtocolContext protocolContext,
final Synchronizer synchronizer,
final MetricsSystem metricsSystem) {
this.ethPeers = ethPeers;
this.snapMessages = snapMessages;
this.ethScheduler = ethScheduler;
this.supportedCapabilities = calculateCapabilities(protocolSchedule);
// AERE D-455 (2026-10-01): snapConfig can be null (the upstream controller-builder tests hand in an
// unset SynchronizerConfiguration; SnapServer already reads it through Optional.ofNullable). A
// null configuration takes the DEFAULT caps, never "no cap" and never a NullPointerException.
final SnapSyncConfiguration snapCaps =
snapConfig != null ? snapConfig : SnapSyncConfiguration.getDefault();
this.maxConcurrentRequestsPerPeer = snapCaps.getMaxConcurrentSnapRequestsPerPeer();
this.maxConcurrentRequestsGlobal = snapCaps.getMaxConcurrentSnapRequestsGlobal();
new SnapServer(
snapConfig, snapMessages, worldStateStorageCoordinator, protocolContext, synchronizer);
metricsSystem.createIntegerGauge(
BesuMetricCategory.PEERS,
"snap_service_requests_in_flight_current",
"The current number of snap sync GET_* requests concurrently scheduled for processing",
globalInFlightRequests::get);
this.rejectedRequestsCounter =
metricsSystem.createCounter(
BesuMetricCategory.PEERS,
"snap_service_requests_rejected_total",
"Total number of snap sync GET_* requests answered with an empty response because a concurrency cap was reached");
}
private List<Capability> calculateCapabilities(final ProtocolSchedule protocolSchedule) {
final ImmutableList.Builder<Capability> capabilities = ImmutableList.builder();
capabilities.add(SnapProtocol.SNAP1);
if (protocolSchedule.anyMatch(spec -> spec.spec().isBlockAccessListEnabled())) {
capabilities.add(SnapProtocol.SNAP2);
}
return capabilities.build();
}
@Override
public String getSupportedProtocol() {
return SnapProtocol.NAME;
}
@Override
public List<Capability> getSupportedCapabilities() {
return supportedCapabilities;
}
@Override
public void stop() {}
@Override
public void awaitStop() throws InterruptedException {}
/**
* This function is called by the P2P framework when a SNAP message has been received.
*
* @param cap The capability under which the message was transmitted.
* @param message The message to be decoded.
*/
@Override
public void processMessage(final Capability cap, final Message message) {
final int code = message.getData().getCode();
LOG.trace("Process snap message {}, {}", cap, code);
final EthPeer ethPeer = ethPeers.peer(message.getConnection());
if (ethPeer == null) {
LOG.debug(
"Ignoring message received from unknown peer connection: {}", message.getConnection());
return;
}
final EthMessage ethMessage = new EthMessage(ethPeer, message.getData());
if (!ethPeer.validateReceivedMessage(ethMessage, getSupportedProtocol())) {
LOG.debug("Unsolicited message {} received from, disconnecting: {}", code, ethPeer);
ethPeer.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_UNSOLICITED_MESSAGE_RECEIVED);
return;
}
// Decode the snap message. FramingException (decompression failure) is a protocol violation.
final MessageData messageData;
try {
messageData = AbstractSnapMessageData.create(message);
} catch (final FramingException e) {
LOG.atDebug()
.setMessage("Disconnecting peer {} due to decompression failure for message code {}")
.addArgument(ethPeer::getLoggableId)
.addArgument(code)
.setCause(e)
.log();
ethPeer.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
return;
}
final EthMessage decodedEthMessage = new EthMessage(ethPeer, messageData);
// Dispatch to pending response handlers (no-op for inbound requests).
ethPeers.dispatchMessage(ethPeer, decodedEthMessage, getSupportedProtocol());
// GET_* requests are handled off the Netty event loop to avoid blocking ETH protocol traffic.
if (SnapV1.REQUEST_CODES.contains(code) || SnapV2.REQUEST_CODES.contains(code)) {
scheduleSnapRequest(ethPeer, decodedEthMessage, cap, code);
}
}
private void scheduleSnapRequest(
final EthPeer ethPeer,
final EthMessage decodedEthMessage,
final Capability cap,
final int code) {
if (!reserveSnapRequestSlot(ethPeer)) {
respondEmptyDueToOverload(ethPeer, decodedEthMessage, code);
return;
}
try {
scheduleReservedSnapRequest(ethPeer, decodedEthMessage, cap, code);
} catch (final RuntimeException e) {
// Only reachable during shutdown: the services executor rejects synchronously once shut
// down. Release the slot so it isn't leaked.
releaseSnapRequestSlot(ethPeer);
throw e;
}
}
/** Cap was hit; reply empty instead of leaving the peer to time out. */
private void respondEmptyDueToOverload(
final EthPeer ethPeer, final EthMessage decodedEthMessage, final int code) {
final BigInteger requestId;
try {
requestId = decodedEthMessage.getData().unwrapMessageData().getKey();
} catch (final RLPException e) {
LOG.debug(
"Received malformed snap message code={} (BREACH_OF_PROTOCOL), disconnecting: {}",
code,
ethPeer,
e);
ethPeer.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
return;
}
sendSnapResponse(ethPeer, emptyResponseFor(code).wrapMessageData(requestId));
}
private static MessageData emptyResponseFor(final int code) {
return switch (code) {
case SnapV1.GET_ACCOUNT_RANGE -> SnapServer.EMPTY_ACCOUNT_RANGE;
case SnapV1.GET_STORAGE_RANGE -> SnapServer.EMPTY_STORAGE_RANGE;
case SnapV1.GET_BYTECODES -> SnapServer.EMPTY_BYTE_CODES_MESSAGE;
case SnapV1.GET_TRIE_NODES -> SnapServer.EMPTY_TRIE_NODES_MESSAGE;
case SnapV2.GET_BLOCK_ACCESS_LISTS -> SnapServer.EMPTY_BLOCK_ACCESS_LISTS;
default -> throw new IllegalStateException("Unhandled snap GET_* code: " + code);
};
}
private void scheduleReservedSnapRequest(
final EthPeer ethPeer,
final EthMessage decodedEthMessage,
final Capability cap,
final int code) {
ethScheduler
.scheduleServiceTask(
() -> {
Optional<MessageData> maybeResponseData = Optional.empty();
try {
final Map.Entry<BigInteger, MessageData> requestIdAndEthMessage =
decodedEthMessage.getData().unwrapMessageData();
maybeResponseData =
snapMessages
.dispatch(new EthMessage(ethPeer, requestIdAndEthMessage.getValue()), cap)
.map(
responseData ->
responseData.wrapMessageData(requestIdAndEthMessage.getKey()));
} catch (final FramingException | RLPException e) {
LOG.debug(
"Received malformed snap message code={} (BREACH_OF_PROTOCOL), disconnecting: {}",
code,
ethPeer,
e);
ethPeer.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
}
maybeResponseData.ifPresent(responseData -> sendSnapResponse(ethPeer, responseData));
})
.exceptionally(
e -> {
if (!(e instanceof CancellationException)) {
LOG.atWarn()
.setMessage("Unexpected error handling snap request code={} from peer {}")
.addArgument(code)
.addArgument(ethPeer::getLoggableId)
.setCause(e)
.log();
}
return null;
})
.whenComplete((result, error) -> releaseSnapRequestSlot(ethPeer));
}
/**
* Reserves a global and per-peer slot before scheduling; increment-then-check avoids a TOCTOU
* race.
*
* @return true if reserved; false if a cap was hit (request answered empty instead).
*/
private boolean reserveSnapRequestSlot(final EthPeer ethPeer) {
if (maxConcurrentRequestsGlobal > 0) {
final int reservedGlobal = globalInFlightRequests.incrementAndGet();
if (reservedGlobal > maxConcurrentRequestsGlobal) {
globalInFlightRequests.decrementAndGet();
rejectSnapRequest(ethPeer, "global");
return false;
}
}
if (maxConcurrentRequestsPerPeer > 0) {
final AtomicInteger perPeerCount =
perPeerInFlightRequests.computeIfAbsent(
ethPeer.getConnection(), unused -> new AtomicInteger(0));
final int reservedPerPeer = perPeerCount.incrementAndGet();
if (reservedPerPeer > maxConcurrentRequestsPerPeer) {
perPeerCount.decrementAndGet();
if (maxConcurrentRequestsGlobal > 0) {
globalInFlightRequests.decrementAndGet();
}
rejectSnapRequest(ethPeer, "per-peer");
return false;
}
}
return true;
}
private void releaseSnapRequestSlot(final EthPeer ethPeer) {
if (maxConcurrentRequestsGlobal > 0) {
globalInFlightRequests.decrementAndGet();
}
if (maxConcurrentRequestsPerPeer > 0) {
final AtomicInteger perPeerCount = perPeerInFlightRequests.get(ethPeer.getConnection());
if (perPeerCount != null) {
perPeerCount.decrementAndGet();
}
}
}
private void rejectSnapRequest(final EthPeer ethPeer, final String scope) {
rejectedRequestsCounter.inc();
LOG.atDebug()
.setMessage("Answering snap request from peer {} empty: {} concurrency cap reached")
.addArgument(ethPeer::getLoggableId)
.addArgument(scope)
.log();
}
private void sendSnapResponse(final EthPeer ethPeer, final MessageData responseData) {
try {
ethPeer.send(responseData, getSupportedProtocol());
} catch (final PeerConnection.PeerNotConnected e) {
LOG.atTrace()
.setMessage("Peer disconnected before we could respond - nothing to do {}")
.addArgument(e.getMessage())
.log();
}
}
@Override
public void handleNewConnection(final PeerConnection connection) {}
@Override
public void handleDisconnect(
final PeerConnection connection,
final DisconnectReason reason,
final boolean initiatedByPeer) {
perPeerInFlightRequests.remove(connection);
}
@Override
public int getHighestProtocolVersion() {
return getSupportedCapabilities().stream()
.max(Comparator.comparing(Capability::getVersion))
.map(Capability::getVersion)
.orElse(0);
}
}

800
anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapServer.java Normal file
View File

@ -0,0 +1,800 @@
/*
* Copyright contributors to Hyperledger Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*
* Modifications Copyright contributors to the Aere Network.
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
*/
package org.hyperledger.besu.ethereum.eth.manager.snap;
import org.hyperledger.besu.datatypes.Hash;
import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.chain.Blockchain;
import org.hyperledger.besu.ethereum.core.Synchronizer;
import org.hyperledger.besu.ethereum.core.encoding.BlockAccessListEncoder;
import org.hyperledger.besu.ethereum.eth.manager.EthMessages;
import org.hyperledger.besu.ethereum.eth.messages.snap.AccountRangeMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.BlockAccessListsMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.ByteCodesMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.GetAccountRangeMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.GetBlockAccessListsMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.GetByteCodesMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.GetStorageRangeMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.GetTrieNodesMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV1;
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV2;
import org.hyperledger.besu.ethereum.eth.messages.snap.StorageRangeMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.TrieNodesMessage;
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
import org.hyperledger.besu.ethereum.mainnet.block.access.list.BlockAccessList;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.MessageData;
import org.hyperledger.besu.ethereum.proof.WorldStateProofProvider;
import org.hyperledger.besu.ethereum.rlp.BytesValueRLPOutput;
import org.hyperledger.besu.ethereum.rlp.RLP;
import org.hyperledger.besu.ethereum.trie.CompactEncoding;
import org.hyperledger.besu.ethereum.trie.MerkleTrie;
import org.hyperledger.besu.ethereum.trie.pathbased.bonsai.BonsaiWorldStateProvider;
import org.hyperledger.besu.ethereum.trie.pathbased.bonsai.storage.BonsaiWorldStateKeyValueStorage;
import org.hyperledger.besu.ethereum.worldstate.FlatDbMode;
import org.hyperledger.besu.ethereum.worldstate.WorldStateStorageCoordinator;
import org.hyperledger.besu.plugin.services.BesuEvents;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.NavigableMap;
import java.util.Optional;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;
import com.google.common.annotations.VisibleForTesting;
import kotlin.Pair;
import kotlin.collections.ArrayDeque;
import org.apache.commons.lang3.time.StopWatch;
import org.apache.tuweni.bytes.Bytes;
import org.apache.tuweni.bytes.Bytes32;
import org.apache.tuweni.units.bigints.UInt256;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/** See <a href="https://github.com/ethereum/devp2p/blob/master/caps/snap.md">snap</a> */
class SnapServer implements BesuEvents.InitialSyncCompletionListener {
private static final Logger LOGGER = LoggerFactory.getLogger(SnapServer.class);
private static final int PRIME_STATE_ROOT_CACHE_LIMIT = 128;
private static final int MAX_ENTRIES_PER_REQUEST = 100000;
private static final int MAX_RESPONSE_SIZE = 2 * 1024 * 1024;
private static final int MAX_CODE_LOOKUPS_PER_REQUEST = 1024;
private static final int MAX_TRIE_LOOKUPS_PER_REQUEST = 1024;
static final AccountRangeMessage EMPTY_ACCOUNT_RANGE =
AccountRangeMessage.create(new HashMap<>(), new ArrayDeque<>());
static final StorageRangeMessage EMPTY_STORAGE_RANGE =
StorageRangeMessage.create(new ArrayDeque<>(), Collections.emptyList());
static final TrieNodesMessage EMPTY_TRIE_NODES_MESSAGE =
TrieNodesMessage.create(new ArrayList<>());
static final ByteCodesMessage EMPTY_BYTE_CODES_MESSAGE =
ByteCodesMessage.create(new ArrayDeque<>());
static final BlockAccessListsMessage EMPTY_BLOCK_ACCESS_LISTS =
BlockAccessListsMessage.create(List.of());
static final Hash HASH_LAST = Hash.wrap(Bytes32.leftPad(Bytes.fromHexString("FF"), (byte) 0xFF));
private final AtomicBoolean isStarted = new AtomicBoolean(false);
private final EthMessages snapMessages;
private final WorldStateStorageCoordinator worldStateStorageCoordinator;
private final Optional<ProtocolContext> protocolContext;
// whether snap server is enabled
private final boolean snapServerEnabled;
// max time per snap request
private final long maxMillisPerRequest;
// provide worldstate storage by root hash
private Function<Hash, Optional<BonsaiWorldStateKeyValueStorage>> worldStateStorageProvider =
__ -> Optional.empty();
SnapServer(
final SnapSyncConfiguration snapConfig,
final EthMessages snapMessages,
final WorldStateStorageCoordinator worldStateStorageCoordinator,
final ProtocolContext protocolContext,
final Synchronizer synchronizer) {
this.snapServerEnabled =
Optional.ofNullable(snapConfig)
.map(SnapSyncConfiguration::isSnapServerEnabled)
.orElse(false);
this.snapMessages = snapMessages;
this.worldStateStorageCoordinator = worldStateStorageCoordinator;
this.protocolContext = Optional.of(protocolContext);
this.maxMillisPerRequest = ResponseSizePredicate.DEFAULT_MAX_MILLIS_PER_REQUEST;
registerResponseConstructors();
// subscribe to initial sync completed events to start/stop snap server,
// not saving the listenerId since we never need to unsubscribe.
synchronizer.subscribeInitialSync(this);
}
/**
* Create a snap server without registering a listener for worldstate initial sync events or
* priming worldstates by root hash. Used by unit tests.
*/
@VisibleForTesting
SnapServer(
final EthMessages snapMessages,
final WorldStateStorageCoordinator worldStateStorageCoordinator,
final Function<Hash, Optional<BonsaiWorldStateKeyValueStorage>> worldStateStorageProvider) {
this(
snapMessages,
worldStateStorageCoordinator,
worldStateStorageProvider,
ResponseSizePredicate.DEFAULT_MAX_MILLIS_PER_REQUEST);
}
@VisibleForTesting
SnapServer(
final EthMessages snapMessages,
final WorldStateStorageCoordinator worldStateStorageCoordinator,
final Function<Hash, Optional<BonsaiWorldStateKeyValueStorage>> worldStateStorageProvider,
final long maxMillisPerRequest) {
this.snapServerEnabled = true;
this.snapMessages = snapMessages;
this.worldStateStorageCoordinator = worldStateStorageCoordinator;
this.worldStateStorageProvider = worldStateStorageProvider;
this.protocolContext = Optional.empty();
this.maxMillisPerRequest = maxMillisPerRequest;
}
@Override
public void onInitialSyncCompleted() {
start();
}
@Override
public void onInitialSyncRestart() {
stop();
}
public synchronized SnapServer start() {
if (!isStarted.get() && snapServerEnabled) {
// if we are bonsai and full flat, we can provide a worldstate storage:
var worldStateKeyValueStorage = worldStateStorageCoordinator.worldStateKeyValueStorage();
if (worldStateKeyValueStorage.getDataStorageFormat().isBonsaiFormat()
&& (worldStateStorageCoordinator.isMatchingFlatMode(FlatDbMode.FULL)
|| worldStateStorageCoordinator.isMatchingFlatMode(FlatDbMode.ARCHIVE))) {
LOGGER.info("Starting SnapServer with Bonsai full flat db");
var bonsaiArchive =
protocolContext
.map(ProtocolContext::getWorldStateArchive)
.map(BonsaiWorldStateProvider.class::cast);
var cachedStorageManagerOpt =
bonsaiArchive.map(archive -> archive.getCachedWorldStorageManager());
if (cachedStorageManagerOpt.isPresent()) {
var cachedStorageManager = cachedStorageManagerOpt.get();
this.worldStateStorageProvider =
rootHash ->
cachedStorageManager
.getStorageByRootHash(rootHash)
.map(BonsaiWorldStateKeyValueStorage.class::cast);
// when we start we need to build the cache of latest 128 worldstates
// trielogs-to-root-hash:
var blockchain = protocolContext.map(ProtocolContext::getBlockchain).orElse(null);
// at startup, prime the latest worldstates by roothash:
cachedStorageManager.primeRootToBlockHashCache(blockchain, PRIME_STATE_ROOT_CACHE_LIMIT);
var flatDbStrategy =
((BonsaiWorldStateKeyValueStorage)
worldStateStorageCoordinator.worldStateKeyValueStorage())
.getFlatDbStrategy();
if (!flatDbStrategy.isCodeByCodeHash()) {
LOGGER.warn("SnapServer requires code stored by codehash, but it is not enabled");
}
} else {
LOGGER.warn(
"SnapServer started without cached storage manager, this should only happen in tests");
}
isStarted.set(true);
}
}
return this;
}
public synchronized SnapServer stop() {
isStarted.set(false);
return this;
}
private void registerResponseConstructors() {
snapMessages.registerResponseConstructor(
SnapV1.GET_ACCOUNT_RANGE,
(peer, messageData, capability) -> constructGetAccountRangeResponse(messageData));
snapMessages.registerResponseConstructor(
SnapV1.GET_STORAGE_RANGE,
(peer, messageData, capability) -> constructGetStorageRangeResponse(messageData));
snapMessages.registerResponseConstructor(
SnapV1.GET_BYTECODES,
(peer, messageData, capability) -> constructGetBytecodesResponse(messageData));
snapMessages.registerResponseConstructor(
SnapV1.GET_TRIE_NODES,
(peer, messageData, capability) -> constructGetTrieNodesResponse(messageData));
snapMessages.registerResponseConstructor(
SnapV2.GET_BLOCK_ACCESS_LISTS,
(peer, messageData, capability) -> constructGetBlockAccessListsResponse(messageData));
}
MessageData constructGetBlockAccessListsResponse(final MessageData message) {
if (!isStarted.get()) {
return EMPTY_BLOCK_ACCESS_LISTS;
}
final GetBlockAccessListsMessage getBlockAccessLists =
GetBlockAccessListsMessage.readFrom(message);
final Iterable<Hash> blockHashes = getBlockAccessLists.blockHashes(true);
int responseSizeEstimate = RLP.MAX_PREFIX_SIZE;
final BytesValueRLPOutput rlp = new BytesValueRLPOutput();
rlp.startList();
final Optional<Blockchain> maybeBlockchain =
protocolContext.map(ProtocolContext::getBlockchain);
if (maybeBlockchain.isPresent()) {
final var blockchain = maybeBlockchain.get();
int count = 0;
for (final Hash blockHash : blockHashes) {
if (count >= MAX_ENTRIES_PER_REQUEST) {
break;
}
count++;
final Optional<BlockAccessList> maybeBlockAccessList =
blockchain.getBlockAccessList(blockHash);
final BytesValueRLPOutput balOutput = new BytesValueRLPOutput();
if (maybeBlockAccessList.isPresent()) {
BlockAccessListEncoder.encode(maybeBlockAccessList.get(), balOutput);
} else {
// Empty lists are returned for blocks where the BAL is unavailable.
balOutput.startList();
balOutput.endList();
}
final int encodedSize = balOutput.encodedSize();
if (responseSizeEstimate + encodedSize > MAX_RESPONSE_SIZE) {
break;
}
responseSizeEstimate += encodedSize;
rlp.writeRaw(balOutput.encoded());
}
}
rlp.endList();
return BlockAccessListsMessage.createUnsafe(rlp.encoded());
}
MessageData constructGetAccountRangeResponse(final MessageData message) {
if (!isStarted.get()) {
return EMPTY_ACCOUNT_RANGE;
}
StopWatch stopWatch = StopWatch.createStarted();
final GetAccountRangeMessage getAccountRangeMessage = GetAccountRangeMessage.readFrom(message);
final GetAccountRangeMessage.Range range = getAccountRangeMessage.range(true);
final int maxResponseBytes = Math.min(range.responseBytes().intValue(), MAX_RESPONSE_SIZE);
LOGGER
.atTrace()
.setMessage("Received getAccountRangeMessage for {} from {} to {}")
.addArgument(() -> asLogHash(Bytes32.wrap(range.worldStateRootHash().getBytes())))
.addArgument(() -> asLogHash(Bytes32.wrap(range.startKeyHash().getBytes())))
.addArgument(() -> asLogHash(Bytes32.wrap(range.endKeyHash().getBytes())))
.log();
try {
if (range.worldStateRootHash().equals(Hash.EMPTY_TRIE_HASH)) {
return AccountRangeMessage.create(new HashMap<>(), List.of(MerkleTrie.EMPTY_TRIE_NODE));
}
return worldStateStorageProvider
.apply(range.worldStateRootHash())
.map(
storage -> {
LOGGER.trace("obtained worldstate in {}", stopWatch);
ResponseSizePredicate responseSizePredicate =
new ResponseSizePredicate(
"account",
stopWatch,
maxResponseBytes,
maxMillisPerRequest,
(pair) -> {
Bytes bytes =
AccountRangeMessage.toSlimAccount(RLP.input(pair.getSecond()));
return Bytes32.SIZE + bytes.size();
});
final Bytes32 endKeyBytes = Bytes32.wrap(range.endKeyHash().getBytes());
var shouldContinuePredicate =
new ExceedingPredicate(
new EndKeyExceedsPredicate(endKeyBytes).and(responseSizePredicate));
NavigableMap<Bytes32, Bytes> accounts =
storage.streamFlatAccounts(
range.startKeyHash().getBytes(), shouldContinuePredicate);
if (accounts.isEmpty() && shouldContinuePredicate.shouldContinue.get()) {
var fromNextHash =
range.endKeyHash().compareTo(range.startKeyHash()) >= 0
? range.endKeyHash()
: range.startKeyHash();
// fetch next account after range, if it exists
LOGGER.debug(
"found no accounts in range, taking first value starting from {}",
asLogHash(Bytes32.wrap(fromNextHash.getBytes())));
accounts =
storage.streamFlatAccounts(fromNextHash.getBytes(), UInt256.MAX_VALUE, 1L);
}
final var worldStateProof =
new WorldStateProofProvider(new WorldStateStorageCoordinator(storage));
final List<Bytes> proof =
worldStateProof.getAccountProofRelatedNodes(
range.worldStateRootHash(), Bytes32.wrap(range.startKeyHash().getBytes()));
if (!accounts.isEmpty()) {
proof.addAll(
worldStateProof.getAccountProofRelatedNodes(
range.worldStateRootHash(), accounts.lastKey()));
}
var resp = AccountRangeMessage.create(accounts, proof);
if (accounts.isEmpty()) {
LOGGER.debug(
"returned empty account range message for {} to {}, proof count {}",
asLogHash(Bytes32.wrap(range.startKeyHash().getBytes())),
asLogHash(Bytes32.wrap(range.endKeyHash().getBytes())),
proof.size());
}
LOGGER.debug(
"returned in {} account range {} to {} with {} accounts and {} proofs, resp size {} of max {}",
stopWatch,
asLogHash(Bytes32.wrap(range.startKeyHash().getBytes())),
asLogHash(Bytes32.wrap(range.endKeyHash().getBytes())),
accounts.size(),
proof.size(),
resp.getSize(),
maxResponseBytes);
return resp;
})
.orElseGet(
() -> {
LOGGER.debug("returned empty account range due to worldstate not present");
return EMPTY_ACCOUNT_RANGE;
});
} catch (Exception ex) {
LOGGER.error("Unexpected exception serving account range request", ex);
}
return EMPTY_ACCOUNT_RANGE;
}
MessageData constructGetStorageRangeResponse(final MessageData message) {
if (!isStarted.get()) {
return EMPTY_STORAGE_RANGE;
}
StopWatch stopWatch = StopWatch.createStarted();
final GetStorageRangeMessage getStorageRangeMessage = GetStorageRangeMessage.readFrom(message);
final GetStorageRangeMessage.StorageRange range = getStorageRangeMessage.range(true);
final int maxResponseBytes = Math.min(range.responseBytes().intValue(), MAX_RESPONSE_SIZE);
LOGGER
.atTrace()
.setMessage("Receive get storage range message size {} from {} to {} for {}")
.addArgument(message::getSize)
.addArgument(() -> asLogHash(Bytes32.wrap(range.startKeyHash().getBytes())))
.addArgument(
() ->
Optional.ofNullable(range.endKeyHash())
.map(h -> asLogHash(Bytes32.wrap(h.getBytes())))
.orElse("''"))
.addArgument(
() ->
range.hashes().stream()
.map(SnapServer::asLogHash)
.collect(Collectors.joining(",", "[", "]")))
.log();
try {
return worldStateStorageProvider
.apply(range.worldStateRootHash())
.map(
storage -> {
LOGGER.trace("obtained worldstate in {}", stopWatch);
// reusable predicate to limit by rec count and bytes:
var responsePredicate =
new ResponseSizePredicate(
"storage",
stopWatch,
maxResponseBytes,
maxMillisPerRequest,
(pair) -> {
var slotRlpOutput = new BytesValueRLPOutput();
slotRlpOutput.startList();
slotRlpOutput.writeBytes(pair.getFirst());
slotRlpOutput.writeBytes(pair.getSecond());
slotRlpOutput.endList();
return slotRlpOutput.encodedSize();
});
// only honor start and end hash if request is for a single account's storage:
Bytes32 startKeyBytes, endKeyBytes;
boolean isPartialRange = false;
if (range.hashes().size() > 1) {
startKeyBytes = Bytes32.ZERO;
endKeyBytes = Bytes32.wrap(HASH_LAST.getBytes());
} else {
startKeyBytes = Bytes32.wrap(range.startKeyHash().getBytes());
endKeyBytes = Bytes32.wrap(range.endKeyHash().getBytes());
isPartialRange =
!(startKeyBytes.equals(Bytes32.wrap(Hash.ZERO.getBytes()))
&& endKeyBytes.equals(Bytes32.wrap(HASH_LAST.getBytes())));
}
ArrayDeque<NavigableMap<Bytes32, Bytes>> collectedStorages = new ArrayDeque<>();
List<Bytes> proofNodes = new ArrayList<>();
final var worldStateProof =
new WorldStateProofProvider(new WorldStateStorageCoordinator(storage));
for (var forAccountHash : range.hashes()) {
var predicate =
new ExceedingPredicate(
new EndKeyExceedsPredicate(endKeyBytes).and(responsePredicate));
var accountStorages =
storage.streamFlatStorages(
Hash.wrap(forAccountHash), startKeyBytes, predicate);
//// address partial range queries that return empty
if (accountStorages.isEmpty() && isPartialRange) {
// fetch next slot after range, if it exists
LOGGER.debug(
"found no slots in range, taking first value starting from {}",
asLogHash(Bytes32.wrap(range.endKeyHash().getBytes())));
accountStorages =
storage.streamFlatStorages(
Hash.wrap(forAccountHash),
Bytes32.wrap(range.endKeyHash().getBytes()),
UInt256.MAX_VALUE,
1L);
}
// don't send empty storage ranges
if (!accountStorages.isEmpty()) {
collectedStorages.add(accountStorages);
}
// if a partial storage range was requested, or we interrupted storage due to
// request limits, send proofs:
if (isPartialRange || !predicate.shouldGetMore()) {
// send a proof for the left side range origin
proofNodes.addAll(
worldStateProof.getStorageProofRelatedNodes(
Bytes32.wrap(getAccountStorageRoot(forAccountHash, storage).getBytes()),
forAccountHash,
startKeyBytes));
if (!accountStorages.isEmpty()) {
// send a proof for the last key on the right
proofNodes.addAll(
worldStateProof.getStorageProofRelatedNodes(
Bytes32.wrap(
getAccountStorageRoot(forAccountHash, storage).getBytes()),
forAccountHash,
accountStorages.lastKey()));
}
}
if (!predicate.shouldGetMore()) {
break;
}
}
var resp = StorageRangeMessage.create(collectedStorages, proofNodes);
LOGGER.debug(
"returned in {} storage {} to {} range {} to {} with {} storages and {} proofs, resp size {} of max {}",
stopWatch,
asLogHash(range.hashes().first()),
asLogHash(range.hashes().last()),
asLogHash(Bytes32.wrap(range.startKeyHash().getBytes())),
asLogHash(Bytes32.wrap(range.endKeyHash().getBytes())),
collectedStorages.size(),
proofNodes.size(),
resp.getSize(),
maxResponseBytes);
return resp;
})
.orElseGet(
() -> {
LOGGER.debug("returned empty storage range due to missing worldstate");
return EMPTY_STORAGE_RANGE;
});
} catch (Exception ex) {
LOGGER.error("Unexpected exception serving storage range request", ex);
return EMPTY_STORAGE_RANGE;
}
}
MessageData constructGetBytecodesResponse(final MessageData message) {
if (!isStarted.get()) {
return EMPTY_BYTE_CODES_MESSAGE;
}
StopWatch stopWatch = StopWatch.createStarted();
final GetByteCodesMessage getByteCodesMessage = GetByteCodesMessage.readFrom(message);
final GetByteCodesMessage.CodeHashes codeHashes = getByteCodesMessage.codeHashes(true);
final int maxResponseBytes = Math.min(codeHashes.responseBytes().intValue(), MAX_RESPONSE_SIZE);
LOGGER
.atTrace()
.setMessage("Received get bytecodes message for {} hashes")
.addArgument(codeHashes.hashes()::size)
.log();
try {
List<Bytes> codeBytes = new ArrayDeque<>();
var codeHashList =
(codeHashes.hashes().size() < MAX_CODE_LOOKUPS_PER_REQUEST)
? codeHashes.hashes()
: codeHashes.hashes().subList(0, MAX_CODE_LOOKUPS_PER_REQUEST);
for (Bytes32 codeHash : codeHashList) {
if (Hash.EMPTY.getBytes().equals(codeHash)) {
codeBytes.add(Bytes.EMPTY);
} else {
Optional<Bytes> optCode = worldStateStorageCoordinator.getCode(Hash.wrap(codeHash), null);
if (optCode.isPresent()) {
if (!codeBytes.isEmpty()
&& (sumListBytes(codeBytes) + optCode.get().size() > maxResponseBytes
|| stopWatch.getTime() > maxMillisPerRequest)) {
break;
}
codeBytes.add(optCode.get());
}
}
}
var resp = ByteCodesMessage.create(codeBytes);
LOGGER.debug(
"returned in {} code bytes message with {} entries, resp size {} of max {}",
stopWatch,
codeBytes.size(),
resp.getSize(),
maxResponseBytes);
return resp;
} catch (Exception ex) {
LOGGER.error("Unexpected exception serving bytecodes request", ex);
return EMPTY_BYTE_CODES_MESSAGE;
}
}
MessageData constructGetTrieNodesResponse(final MessageData message) {
if (!isStarted.get()) {
return EMPTY_TRIE_NODES_MESSAGE;
}
StopWatch stopWatch = StopWatch.createStarted();
final GetTrieNodesMessage getTrieNodesMessage = GetTrieNodesMessage.readFrom(message);
final GetTrieNodesMessage.TrieNodesPaths triePaths = getTrieNodesMessage.paths(true);
final int maxResponseBytes = Math.min(triePaths.responseBytes().intValue(), MAX_RESPONSE_SIZE);
LOGGER
.atTrace()
.setMessage("Received get trie nodes message of size {}")
.addArgument(() -> triePaths.paths().size())
.log();
try {
return worldStateStorageProvider
.apply(triePaths.worldStateRootHash())
.map(
storage -> {
LOGGER.trace("obtained worldstate in {}", stopWatch);
ArrayList<Bytes> trieNodes = new ArrayList<>();
var triePathList =
triePaths.paths().size() < MAX_TRIE_LOOKUPS_PER_REQUEST
? triePaths.paths()
: triePaths.paths().subList(0, MAX_TRIE_LOOKUPS_PER_REQUEST);
for (var triePath : triePathList) {
// first element in paths is account
if (triePath.size() == 1) {
// if there is only one path, presume it should be compact encoded account path
final Bytes location = CompactEncoding.decode(triePath.get(0));
var optStorage = storage.getTrieNodeUnsafe(location);
if (optStorage.isEmpty() && location.isEmpty()) {
optStorage = Optional.of(MerkleTrie.EMPTY_TRIE_NODE);
}
var trieNode = optStorage.orElse(Bytes.EMPTY);
if (!trieNodes.isEmpty()
&& (sumListBytes(trieNodes) + trieNode.size() > maxResponseBytes
|| stopWatch.getTime(TimeUnit.MILLISECONDS) > maxMillisPerRequest)) {
break;
}
trieNodes.add(trieNode);
} else {
// There must be at least one element in the path otherwise it is invalid
if (triePath.isEmpty()) {
LOGGER.debug("returned empty trie nodes message due to invalid path");
return EMPTY_TRIE_NODES_MESSAGE;
}
// otherwise the first element should be account hash, and subsequent paths
// are compact encoded account storage paths
final Bytes32 accountPrefix = Bytes32.leftPad(triePath.getFirst());
var optAccount = storage.getAccount(Hash.wrap(accountPrefix));
if (optAccount.isEmpty()) {
continue;
}
List<Bytes> storagePaths = triePath.subList(1, triePath.size());
for (var path : storagePaths) {
final Bytes location = CompactEncoding.decode(path);
var optStorage =
storage.getTrieNodeUnsafe(Bytes.concatenate(accountPrefix, location));
if (optStorage.isEmpty() && location.isEmpty()) {
optStorage = Optional.of(MerkleTrie.EMPTY_TRIE_NODE);
}
var trieNode = optStorage.orElse(Bytes.EMPTY);
if (!trieNodes.isEmpty()
&& sumListBytes(trieNodes) + trieNode.size() > maxResponseBytes) {
break;
}
trieNodes.add(trieNode);
}
}
}
var resp = TrieNodesMessage.create(trieNodes);
LOGGER.debug(
"returned in {} trie nodes message with {} entries, resp size {} of max {}",
stopWatch,
trieNodes.size(),
resp.getCode(),
maxResponseBytes);
return resp;
})
.orElseGet(
() -> {
LOGGER.debug("returned empty trie nodes message due to missing worldstate");
return EMPTY_TRIE_NODES_MESSAGE;
});
} catch (Exception ex) {
LOGGER.error("Unexpected exception serving trienodes request", ex);
return EMPTY_TRIE_NODES_MESSAGE;
}
}
/**
* Predicate that doesn't immediately stop when the delegate predicate returns false, but instead
* sets a flag to stop after the current element is processed.
*/
static class ExceedingPredicate implements Predicate<Pair<Bytes32, Bytes>> {
private final Predicate<Pair<Bytes32, Bytes>> delegate;
final AtomicBoolean shouldContinue = new AtomicBoolean(true);
public ExceedingPredicate(final Predicate<Pair<Bytes32, Bytes>> delegate) {
this.delegate = delegate;
}
@Override
public boolean test(final Pair<Bytes32, Bytes> pair) {
final boolean result = delegate.test(pair);
return shouldContinue.getAndSet(result);
}
public boolean shouldGetMore() {
return shouldContinue.get();
}
}
/** Predicate that stops when the end key is exceeded. */
record EndKeyExceedsPredicate(Bytes endKey) implements Predicate<Pair<Bytes32, Bytes>> {
@Override
public boolean test(final Pair<Bytes32, Bytes> pair) {
return endKey.compareTo(Bytes.wrap(pair.getFirst())) > 0;
}
}
static class ResponseSizePredicate implements Predicate<Pair<Bytes32, Bytes>> {
// default to a max of 4 seconds per request
static final long DEFAULT_MAX_MILLIS_PER_REQUEST = 4000;
final AtomicInteger byteLimit = new AtomicInteger(0);
final AtomicInteger recordLimit = new AtomicInteger(0);
final AtomicBoolean shouldContinue = new AtomicBoolean(true);
final Function<Pair<Bytes32, Bytes>, Integer> encodingSizeAccumulator;
final StopWatch stopWatch;
final int maxResponseBytes;
final long maxMillisPerRequest;
final String forWhat;
ResponseSizePredicate(
final String forWhat,
final StopWatch stopWatch,
final int maxResponseBytes,
final long maxMillisPerRequest,
final Function<Pair<Bytes32, Bytes>, Integer> encodingSizeAccumulator) {
this.stopWatch = stopWatch;
this.maxResponseBytes = maxResponseBytes;
this.maxMillisPerRequest = maxMillisPerRequest;
this.forWhat = forWhat;
this.encodingSizeAccumulator = encodingSizeAccumulator;
}
@Override
public boolean test(final Pair<Bytes32, Bytes> pair) {
LOGGER
.atTrace()
.setMessage("{} pre-accumulate limits, bytes: {} , stream count: {}")
.addArgument(() -> forWhat)
.addArgument(byteLimit::get)
.addArgument(recordLimit::get)
.log();
if (stopWatch.getTime() > maxMillisPerRequest) {
shouldContinue.set(false);
LOGGER.warn(
"{} took too long, stopped at {} ms with {} records and {} bytes",
forWhat,
stopWatch.formatTime(),
recordLimit.get(),
byteLimit.get());
return false;
}
var underRecordLimit = recordLimit.addAndGet(1) <= MAX_ENTRIES_PER_REQUEST;
var underByteLimit =
byteLimit.accumulateAndGet(0, (cur, __) -> cur + encodingSizeAccumulator.apply(pair))
< maxResponseBytes;
if (underRecordLimit && underByteLimit) {
return true;
} else {
shouldContinue.set(false);
LOGGER
.atDebug()
.setMessage("{} post-accumulate limits, bytes: {} , stream count: {}")
.addArgument(() -> forWhat)
.addArgument(byteLimit::get)
.addArgument(recordLimit::get)
.log();
return false;
}
}
}
Hash getAccountStorageRoot(
final Bytes32 accountHash, final BonsaiWorldStateKeyValueStorage storage) {
return storage
.getTrieNodeUnsafe(Bytes.concatenate(accountHash, Bytes.EMPTY))
.map(Hash::hash)
.orElse(Hash.EMPTY_TRIE_HASH);
}
private static int sumListBytes(final List<Bytes> listOfBytes) {
// TODO: remove hack, 10% is a fudge factor to account for the overhead of rlp encoding
return listOfBytes.stream().map(Bytes::size).reduce((a, b) -> a + b).orElse(0) * 11 / 10;
}
private static String asLogHash(final Bytes32 hash) {
var str = hash.toHexString();
return str.substring(0, 4) + ".." + str.substring(59, 63);
}
}

132
anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapSyncConfiguration.java Normal file
View File

@ -0,0 +1,132 @@
/*
* Copyright contributors to Hyperledger Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*
* Modifications Copyright contributors to the Aere Network.
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
*/
package org.hyperledger.besu.ethereum.eth.sync.snapsync;
import org.immutables.value.Value;
@Value.Immutable
public class SnapSyncConfiguration {
// we use 126 and not the max value (128) to avoid sending requests that will be refused
public static final int DEFAULT_PIVOT_BLOCK_WINDOW_VALIDITY = 126;
public static final int DEFAULT_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING = 60;
public static final int DEFAULT_STORAGE_COUNT_PER_REQUEST =
384; // The default number of storage entries to download from peers per request.
public static final int DEFAULT_BYTECODE_COUNT_PER_REQUEST =
84; // The default number of code entries to download from peers per request.
public static final int DEFAULT_TRIENODE_COUNT_PER_REQUEST =
384; // The default number of trienode entries to download from peers per request.
public static final int DEFAULT_BLOCK_ACCESS_LIST_COUNT_PER_REQUEST =
84; // The default number of block access lists to download from peers per request.
public static final int DEFAULT_LOCAL_FLAT_ACCOUNT_COUNT_TO_HEAL_PER_REQUEST =
128; // The default number of flat accounts entries to verify and heal per request.
public static final int DEFAULT_LOCAL_FLAT_STORAGE_COUNT_TO_HEAL_PER_REQUEST =
1024; // The default number of flat slots entries to verify and heal per request.
public static final Boolean DEFAULT_SNAP_SERVER_ENABLED = Boolean.FALSE;
/**
* Default cap on the number of snap/1-2 GET_* requests from a single peer that may have a service
* task concurrently scheduled on the EthScheduler services pool. Sized with modest headroom above
* the number of outstanding requests we ourselves send a peer ({@code
* EthPeer.MAX_OUTSTANDING_REQUESTS = 5}). 0 disables the check. (AERE D-434, 683880baa6)
*/
public static final int DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_PER_PEER = 8;
/**
* Default cap on the total number of snap/1-2 GET_* requests, across all peers, that may have a
* service task concurrently scheduled on the EthScheduler services pool, sized as {@code
* DEFAULT_MAX_PEERS (25) * DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_PER_PEER (8) = 200}. 0 disables the
* check. Operators who raise {@code --max-peers} well above the default should raise this value
* proportionally. (AERE D-434, 683880baa6)
*/
public static final int DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_GLOBAL = 200;
public static final Boolean DEFAULT_SNAP_SYNC_TRANSACTION_INDEXING_ENABLED = Boolean.FALSE;
public static final Boolean DEFAULT_SNAP_SYNC_SAVE_PRE_MERGE_HEADERS_ONLY_ENABLED = Boolean.TRUE;
public static SnapSyncConfiguration getDefault() {
return ImmutableSnapSyncConfiguration.builder().build();
}
@Value.Default
public int getPivotBlockWindowValidity() {
return DEFAULT_PIVOT_BLOCK_WINDOW_VALIDITY;
}
@Value.Default
public int getPivotBlockDistanceBeforeCaching() {
return DEFAULT_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING;
}
@Value.Default
public int getStorageCountPerRequest() {
return DEFAULT_STORAGE_COUNT_PER_REQUEST;
}
@Value.Default
public int getBytecodeCountPerRequest() {
return DEFAULT_BYTECODE_COUNT_PER_REQUEST;
}
@Value.Default
public int getTrienodeCountPerRequest() {
return DEFAULT_TRIENODE_COUNT_PER_REQUEST;
}
@Value.Default
public int getBlockAccessListCountPerRequest() {
return DEFAULT_BLOCK_ACCESS_LIST_COUNT_PER_REQUEST;
}
@Value.Default
public int getLocalFlatAccountCountToHealPerRequest() {
return DEFAULT_LOCAL_FLAT_ACCOUNT_COUNT_TO_HEAL_PER_REQUEST;
}
@Value.Default
public int getLocalFlatStorageCountToHealPerRequest() {
return DEFAULT_LOCAL_FLAT_STORAGE_COUNT_TO_HEAL_PER_REQUEST;
}
@Value.Default
public Boolean isSnapServerEnabled() {
return DEFAULT_SNAP_SERVER_ENABLED;
}
@Value.Default
public int getMaxConcurrentSnapRequestsPerPeer() {
return DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_PER_PEER;
}
@Value.Default
public int getMaxConcurrentSnapRequestsGlobal() {
return DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_GLOBAL;
}
@Value.Default
public Boolean isSnapSyncTransactionIndexingEnabled() {
return DEFAULT_SNAP_SYNC_TRANSACTION_INDEXING_ENABLED;
}
}

2
anchor/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-syncstate-in-sync-lock.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.ethereum.eth.sync.state;

341
anchor/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/AereD434SnapCapTest.java Normal file
View File

@ -0,0 +1,341 @@
/*
* Copyright contributors to Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*
* Modified by Aere Network (2026-09-29, D-434): the snap concurrency-cap tests of hyperledger/besu 683880baa6
* (GHSA-pcv4-pxhv-99m7, #11101), ported as a separate class onto this fork's SnapProtocolManager constructor, which
* keeps ProtocolSchedule (capabilities are derived from it here) and takes MetricsSystem last.
*/
package org.hyperledger.besu.ethereum.eth.manager.snap;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import org.hyperledger.besu.datatypes.Hash;
import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.core.Synchronizer;
import org.hyperledger.besu.ethereum.eth.SnapProtocol;
import org.hyperledger.besu.ethereum.eth.manager.EthMessages;
import org.hyperledger.besu.ethereum.eth.manager.EthPeer;
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.manager.MockPeerConnection;
import org.hyperledger.besu.ethereum.eth.messages.snap.GetTrieNodesMessage;
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV1;
import org.hyperledger.besu.ethereum.eth.messages.snap.TrieNodesMessage;
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnection;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.DefaultMessage;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.Message;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.MessageData;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.RawMessage;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.messages.DisconnectMessage.DisconnectReason;
import org.hyperledger.besu.ethereum.worldstate.WorldStateStorageCoordinator;
import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
import java.math.BigInteger;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.RejectedExecutionException;
import org.apache.tuweni.bytes.Bytes;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class AereD434SnapCapTest {
@Mock private WorldStateStorageCoordinator worldStateStorageCoordinator;
@Mock private SnapSyncConfiguration snapConfig;
@Mock private EthPeers ethPeers;
@Mock private EthMessages snapMessages;
@Mock private ProtocolSchedule protocolSchedule;
@Mock private ProtocolContext protocolContext;
@Mock private Synchronizer synchronizer;
@Mock private EthScheduler ethScheduler;
@Mock private EthPeer ethPeer;
private SnapProtocolManager snapProtocolManager;
@BeforeEach
void setUp() {
when(snapConfig.isSnapServerEnabled()).thenReturn(false);
}
/** The defaults are the upstream ones, and both caps are on unless configured to 0. */
@Test
void defaultsAreEightPerPeerAndTwoHundredGlobal() {
final SnapSyncConfiguration defaults = SnapSyncConfiguration.getDefault();
assertThat(defaults.getMaxConcurrentSnapRequestsPerPeer()).isEqualTo(8);
assertThat(defaults.getMaxConcurrentSnapRequestsGlobal()).isEqualTo(200);
}
@Test
void limitsConcurrentSnapRequestsPerPeer() {
when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(2);
snapProtocolManager = createSnapProtocolManager();
final MockPeerConnection peerConnection = snapPeerConnection();
stubPeer(ethPeer, peerConnection);
stubPendingServiceTasks();
for (int i = 0; i < 5; i++) {
snapProtocolManager.processMessage(
SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, i));
}
// Only the first 2 requests are scheduled; the remaining 3 are answered empty once the
// per-peer cap is reached.
verify(ethScheduler, times(2)).scheduleServiceTask(any(Runnable.class));
}
@Test
void perPeerCapDoesNotBlockADistinctPeer() {
when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(2);
snapProtocolManager = createSnapProtocolManager();
final MockPeerConnection peerConnectionA = snapPeerConnection();
stubPeer(ethPeer, peerConnectionA);
final EthPeer ethPeerB = mock(EthPeer.class);
final MockPeerConnection peerConnectionB = snapPeerConnection();
stubPeer(ethPeerB, peerConnectionB);
stubPendingServiceTasks();
for (int i = 0; i < 2; i++) {
snapProtocolManager.processMessage(
SnapProtocol.SNAP1, getTrieNodesMessage(peerConnectionA, i));
snapProtocolManager.processMessage(
SnapProtocol.SNAP1, getTrieNodesMessage(peerConnectionB, i));
}
verify(ethScheduler, times(4)).scheduleServiceTask(any(Runnable.class));
}
@Test
void limitsConcurrentSnapRequestsGlobally() {
when(snapConfig.getMaxConcurrentSnapRequestsGlobal()).thenReturn(3);
snapProtocolManager = createSnapProtocolManager();
stubPendingServiceTasks();
for (int peerIndex = 0; peerIndex < 3; peerIndex++) {
final EthPeer peer = mock(EthPeer.class);
final MockPeerConnection peerConnection = snapPeerConnection();
stubPeer(peer, peerConnection);
for (int i = 0; i < 2; i++) {
snapProtocolManager.processMessage(
SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, i));
}
}
verify(ethScheduler, times(3)).scheduleServiceTask(any(Runnable.class));
}
@Test
void releasesSlotWhenScheduledTaskCompletes() {
when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
snapProtocolManager = createSnapProtocolManager();
final MockPeerConnection peerConnection = snapPeerConnection();
stubPeer(ethPeer, peerConnection);
final List<CompletableFuture<Void>> scheduledTasks = new ArrayList<>();
when(ethScheduler.scheduleServiceTask(any(Runnable.class)))
.thenAnswer(
invocation -> {
final CompletableFuture<Void> future = new CompletableFuture<>();
scheduledTasks.add(future);
return future;
});
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0));
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 1));
verify(ethScheduler, times(1)).scheduleServiceTask(any(Runnable.class));
scheduledTasks.get(0).complete(null);
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 2));
verify(ethScheduler, times(2)).scheduleServiceTask(any(Runnable.class));
}
@Test
void releasesSlotWhenSchedulingThrowsSynchronously() {
when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
snapProtocolManager = createSnapProtocolManager();
final MockPeerConnection peerConnection = snapPeerConnection();
stubPeer(ethPeer, peerConnection);
when(ethScheduler.scheduleServiceTask(any(Runnable.class)))
.thenThrow(new RejectedExecutionException("executor is shutting down"))
.thenAnswer(invocation -> new CompletableFuture<Void>());
assertThatThrownBy(
() ->
snapProtocolManager.processMessage(
SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0)))
.isInstanceOf(RejectedExecutionException.class);
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 1));
verify(ethScheduler, times(2)).scheduleServiceTask(any(Runnable.class));
}
@Test
void clearsPeerSlotsOnDisconnectPreventingALeak() {
when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
snapProtocolManager = createSnapProtocolManager();
final MockPeerConnection peerConnection = snapPeerConnection();
stubPeer(ethPeer, peerConnection);
stubPendingServiceTasks();
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0));
verify(ethScheduler, times(1)).scheduleServiceTask(any(Runnable.class));
snapProtocolManager.handleDisconnect(
peerConnection, DisconnectReason.TCP_SUBSYSTEM_ERROR, false);
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 1));
verify(ethScheduler, times(2)).scheduleServiceTask(any(Runnable.class));
}
@Test
void respondsEmptyWhenPerPeerCapReached() throws PeerConnection.PeerNotConnected {
when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
snapProtocolManager = createSnapProtocolManager();
final MockPeerConnection peerConnection = snapPeerConnection();
stubPeer(ethPeer, peerConnection);
stubPendingServiceTasks();
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0));
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 1));
verify(ethScheduler, times(1)).scheduleServiceTask(any(Runnable.class));
final ArgumentCaptor<MessageData> sent = ArgumentCaptor.forClass(MessageData.class);
verify(ethPeer).send(sent.capture(), eq(SnapProtocol.NAME));
assertThat(sent.getValue().getCode()).isEqualTo(SnapV1.TRIE_NODES);
assertThat(TrieNodesMessage.readFrom(sent.getValue()).nodes(true)).isEmpty();
assertThat(sent.getValue().unwrapMessageData().getKey()).isEqualTo(BigInteger.ONE);
}
@Test
void respondsEmptyWhenGlobalCapReached() throws PeerConnection.PeerNotConnected {
when(snapConfig.getMaxConcurrentSnapRequestsGlobal()).thenReturn(1);
snapProtocolManager = createSnapProtocolManager();
stubPendingServiceTasks();
final MockPeerConnection peerConnectionA = snapPeerConnection();
stubPeer(ethPeer, peerConnectionA);
final EthPeer ethPeerB = mock(EthPeer.class);
final MockPeerConnection peerConnectionB = snapPeerConnection();
stubPeer(ethPeerB, peerConnectionB);
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnectionA, 0));
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnectionB, 7));
verify(ethScheduler, times(1)).scheduleServiceTask(any(Runnable.class));
final ArgumentCaptor<MessageData> sent = ArgumentCaptor.forClass(MessageData.class);
verify(ethPeerB).send(sent.capture(), eq(SnapProtocol.NAME));
assertThat(sent.getValue().unwrapMessageData().getKey()).isEqualTo(BigInteger.valueOf(7));
verify(ethPeer, never()).send(any(), any());
}
@Test
void disconnectsPeerWhenRejectedRequestIsMalformed() throws PeerConnection.PeerNotConnected {
when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
snapProtocolManager = createSnapProtocolManager();
final MockPeerConnection peerConnection = snapPeerConnection();
stubPeer(ethPeer, peerConnection);
stubPendingServiceTasks();
snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0));
final MessageData malformed = new RawMessage(SnapV1.GET_TRIE_NODES, Bytes.of(0x01));
snapProtocolManager.processMessage(
SnapProtocol.SNAP1, new DefaultMessage(peerConnection, malformed));
verify(ethPeer).disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
verify(ethPeer, never()).send(any(), any());
}
/** With both caps at 0 (disabled) every request is scheduled, as before the fix. */
@Test
void zeroDisablesBothCaps() {
when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(0);
when(snapConfig.getMaxConcurrentSnapRequestsGlobal()).thenReturn(0);
snapProtocolManager = createSnapProtocolManager();
final MockPeerConnection peerConnection = snapPeerConnection();
stubPeer(ethPeer, peerConnection);
stubPendingServiceTasks();
for (int i = 0; i < 20; i++) {
snapProtocolManager.processMessage(
SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, i));
}
verify(ethScheduler, times(20)).scheduleServiceTask(any(Runnable.class));
}
private void stubPeer(final EthPeer peer, final PeerConnection connection) {
when(ethPeers.peer(connection)).thenReturn(peer);
when(peer.getConnection()).thenReturn(connection);
when(peer.validateReceivedMessage(any(), any())).thenReturn(true);
}
private void stubPendingServiceTasks() {
when(ethScheduler.scheduleServiceTask(any(Runnable.class)))
.thenAnswer(invocation -> new CompletableFuture<Void>());
}
private MockPeerConnection snapPeerConnection() {
return new MockPeerConnection(
new HashSet<>(Collections.singletonList(SnapProtocol.SNAP1)), (cap, msg, conn) -> {});
}
private Message getTrieNodesMessage(final PeerConnection peerConnection, final int requestId) {
final MessageData data =
GetTrieNodesMessage.create(Hash.ZERO, List.of(List.of(Bytes.EMPTY)))
.wrapMessageData(BigInteger.valueOf(requestId));
return new DefaultMessage(peerConnection, data);
}
private SnapProtocolManager createSnapProtocolManager() {
return new SnapProtocolManager(
worldStateStorageCoordinator,
snapConfig,
ethPeers,
snapMessages,
ethScheduler,
protocolSchedule,
protocolContext,
synchronizer,
new NoOpMetricsSystem());
}
}

103
anchor/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManagerTest.java Normal file
View File

@ -0,0 +1,103 @@
/*
* Copyright contributors to Hyperledger Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*
* Modifications Copyright contributors to the Aere Network.
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
*/
package org.hyperledger.besu.ethereum.eth.manager.snap;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.core.Synchronizer;
import org.hyperledger.besu.ethereum.eth.SnapProtocol;
import org.hyperledger.besu.ethereum.eth.manager.EthMessages;
import org.hyperledger.besu.ethereum.eth.manager.EthPeer;
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.manager.MockPeerConnection;
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.DefaultMessage;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.RawMessage;
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.messages.DisconnectMessage.DisconnectReason;
import org.hyperledger.besu.ethereum.worldstate.WorldStateStorageCoordinator;
import java.util.Collections;
import java.util.HashSet;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class SnapProtocolManagerTest {
@Mock private WorldStateStorageCoordinator worldStateStorageCoordinator;
@Mock private SnapSyncConfiguration snapConfig;
@Mock private EthPeers ethPeers;
@Mock private EthMessages snapMessages;
@Mock private ProtocolSchedule protocolSchedule;
@Mock private ProtocolContext protocolContext;
@Mock private Synchronizer synchronizer;
@Mock private EthScheduler ethScheduler;
@Mock private EthPeer ethPeer;
private SnapProtocolManager snapProtocolManager;
@BeforeEach
void setUp() {
when(snapConfig.isSnapServerEnabled()).thenReturn(false);
snapProtocolManager =
new SnapProtocolManager(
worldStateStorageCoordinator,
snapConfig,
ethPeers,
snapMessages,
ethScheduler,
protocolSchedule,
protocolContext,
synchronizer,
new org.hyperledger.besu.metrics.noop.NoOpMetricsSystem());
}
@Test
void disconnectsPeerOnDecompressionFailure() {
final MockPeerConnection peerConnection =
new MockPeerConnection(
new HashSet<>(Collections.singletonList(SnapProtocol.SNAP1)), (cap, msg, conn) -> {});
when(ethPeers.peer(peerConnection)).thenReturn(ethPeer);
when(ethPeer.validateReceivedMessage(any(), any())).thenReturn(true);
// Create a RawMessage with invalid compressed data that will throw FramingException
final RawMessage badMessage = new RawMessage(0x00, new byte[] {0x01, 0x02, 0x03});
snapProtocolManager.processMessage(
SnapProtocol.SNAP1, new DefaultMessage(peerConnection, badMessage));
assertThat(peerConnection.isDisconnected()).isFalse();
// ethPeer (mock) receives the disconnect call
org.mockito.Mockito.verify(ethPeer)
.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
}
}

2
anchor/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-syncstate-in-sync-lock.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.ethereum.eth.sync.state;

2
anchor/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-bft-bounded-message-lists.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.ethereum.rlp;

2
anchor/evm/src/main/java/org/hyperledger/besu/evm/frame/MessageFrame.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-warm-access-hashdos-treeset.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.evm.frame;

2
anchor/evm/src/main/java/org/hyperledger/besu/evm/frame/TxValues.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-warm-access-hashdos-treeset.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.evm.frame;

2
anchor/evm/src/main/java/org/hyperledger/besu/evm/precompile/MainnetPrecompiledContracts.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/0001-aere-pqc-precompiles-mainnet.patch.patch.
* found, as section 4(c) requires. The change itself is in patches/0001-aere-pqc-precompiles-mainnet.patch.
*/
package org.hyperledger.besu.evm.precompile;

2
anchor/evm/src/main/java/org/hyperledger/besu/evm/precompile/P256VerifyPrecompiledContract.java
View File

@ -16,7 +16,7 @@
*
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
* found, as section 4(c) requires. The change itself is in patches/aere-drop-per-request-console-writes.patch.
* found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
*/
package org.hyperledger.besu.evm.precompile;

4547
patches/0009-aere-pq-seal-judge-snap-cap.patch Normal file
View File

File diff suppressed because it is too large Load Diff

1
run-node.sh
View File

@ -50,6 +50,7 @@ PATCHES=(
"0006-aere-pq-transactions.patch"
"0007-aere-pq-finality.patch"
"0008-aere-pq-aip22.patch"
"0009-aere-pq-seal-judge-snap-cap.patch"
)
# --- locate self -------------------------------------------------------------------------------