4548 lines
213 KiB
Diff
4548 lines
213 KiB
Diff
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
|
|
From: Aere Network <node@aere.network>
|
|
Date: Fri, 09 Oct 2026 11:10:10 +0000
|
|
Subject: [PATCH] Aere Network: the chain-2800 fleet distribution of 2026-10-02 (heard-seal judge, snap request cap)
|
|
|
|
AERE patch 0009: the two changes the chain-2800 fleet has run since 2026-10-05 on top of patch 0008
|
|
|
|
Why this patch exists: since 2026-10-05 the Besu nodes of chain 2800 run a distribution
|
|
built from the exact tree of patch 0008 plus two changes. Neither changes what a node
|
|
accepts below the AIP-22 stage 4 height (aere.pq.finalityBlock, not set on chain 2800),
|
|
so a node built from patches 0001 and 0003 to 0008 follows the chain today. This patch
|
|
brings the recipe to the exact source tree of that distribution, so that the published
|
|
recipe and the fleet are the same code again, and so that a node built from the recipe
|
|
behaves as the fleet does on the day stage 4 is activated.
|
|
|
|
What it changes, as one delta over the tree produced by patches 0001 and 0003 to 0008:
|
|
- the heard-seal store (PqSealCache) and commit messages that arrive before the proposal
|
|
(RoundState, with the new class AereD454PendingCommits). From the stage 4 height, a seal
|
|
that does not verify never displaces one that does, a verified seal is never displaced,
|
|
and a commit that arrives before the proposal waits in a bounded buffer and takes its
|
|
author's place only after it has been validated against the proposal. Below that height,
|
|
and on any node without aere.pq.finalityBlock, both paths execute exactly the code of
|
|
patch 0008;
|
|
- a cap on the snap-protocol requests a node serves concurrently, per peer (default 8) and
|
|
in total (default 200), ported from upstream Besu (commit 683880baa6, advisory
|
|
GHSA-pcv4-pxhv-99m7). A request over the cap receives an empty response with the same
|
|
request id; 0 disables the cap; a missing snap configuration takes the default caps,
|
|
never "no cap";
|
|
- the tests of both changes (AereD454SealCacheTest, AereD454RoundStateTest with its
|
|
fixture AereD412Fixture, AereD434SnapCapTest) and the adjusted upstream test
|
|
SnapProtocolManagerTest;
|
|
- the licence notices of section 4(b) (see below): in files modified by patch 0008, fifteen
|
|
notices named patch files that are not part of this package; they now name patch 0008,
|
|
which carries those changes. One notice from patch 0001 named its own patch file with a
|
|
doubled extension; it now names it exactly. Only the patch name on that line changes, so
|
|
the compiled classes are unchanged.
|
|
|
|
Nothing else is left out: patches 0001 and 0003 to 0009 applied on a pristine d2032017
|
|
checkout reproduce the fleet source tree file for file, apart from the in-file licence
|
|
notices described below and the three lines that patch 0008 already describes.
|
|
|
|
The upstream copyright header ("Copyright contributors to Besu" or the ConsenSys header) of
|
|
every modified file is kept unchanged, per Apache License 2.0 section 4(c). Every upstream
|
|
Java file this patch modifies for the first time receives, in this diff, the in-file
|
|
modification notice that section 4(b) requires, naming this patch; files already modified
|
|
by patches 0001 and 0003 to 0008 keep the notice naming the patch that first modified them.
|
|
The changed files are also listed above the diff, which is the statement of modification
|
|
that section 4(b) requires.
|
|
---
|
|
app/src/main/java/org/hyperledger/besu/cli/options/SynchronizerOptions.java | 40 +++
|
|
.../main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java | 9 +-
|
|
.../main/java/org/hyperledger/besu/consensus/common/bft/PqSealCache.java | 281 ++++++++++++++++-
|
|
.../besu/consensus/common/bft/blockcreation/BftBlockCreator.java | 2 +-
|
|
.../besu/consensus/common/bft/blockcreation/BftMiningCoordinator.java | 2 +-
|
|
.../hyperledger/besu/consensus/common/bft/messagewrappers/BftMessage.java | 2 +-
|
|
.../org/hyperledger/besu/consensus/common/bft/AereD454SealCacheTest.java | 434 ++++++++++++++++++++++++++
|
|
.../org/hyperledger/besu/consensus/qbft/core/messagewrappers/Proposal.java | 2 +-
|
|
.../hyperledger/besu/consensus/qbft/core/messagewrappers/RoundChange.java | 2 +-
|
|
.../besu/consensus/qbft/core/statemachine/AereD454PendingCommits.java | 203 +++++++++++++
|
|
.../org/hyperledger/besu/consensus/qbft/core/statemachine/RoundState.java | 21 ++
|
|
.../besu/consensus/qbft/core/statemachine/AereD454RoundStateTest.java | 317 +++++++++++++++++++
|
|
.../hyperledger/besu/consensus/qbft/core/validation/AereD412Fixture.java | 590 ++++++++++++++++++++++++++++++++++++
|
|
.../java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java | 2 +-
|
|
.../org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java | 2 +-
|
|
.../java/org/hyperledger/besu/ethereum/mainnet/WithdrawalsValidator.java | 2 +-
|
|
.../src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java | 2 +-
|
|
.../org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManager.java | 149 ++++++++-
|
|
.../java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapServer.java | 18 +-
|
|
.../hyperledger/besu/ethereum/eth/sync/snapsync/SnapSyncConfiguration.java | 33 ++
|
|
.../main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java | 2 +-
|
|
.../org/hyperledger/besu/ethereum/eth/manager/snap/AereD434SnapCapTest.java | 341 +++++++++++++++++++++
|
|
.../hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManagerTest.java | 9 +-
|
|
.../java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java | 2 +-
|
|
ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java | 2 +-
|
|
evm/src/main/java/org/hyperledger/besu/evm/frame/MessageFrame.java | 2 +-
|
|
evm/src/main/java/org/hyperledger/besu/evm/frame/TxValues.java | 2 +-
|
|
.../org/hyperledger/besu/evm/precompile/MainnetPrecompiledContracts.java | 2 +-
|
|
.../org/hyperledger/besu/evm/precompile/P256VerifyPrecompiledContract.java | 2 +-
|
|
29 files changed, 2437 insertions(+), 40 deletions(-)
|
|
|
|
diff --git a/app/src/main/java/org/hyperledger/besu/cli/options/SynchronizerOptions.java b/app/src/main/java/org/hyperledger/besu/cli/options/SynchronizerOptions.java
|
|
index d4a2eb048a..0d3f642e56 100644
|
|
--- a/app/src/main/java/org/hyperledger/besu/cli/options/SynchronizerOptions.java
|
|
+++ b/app/src/main/java/org/hyperledger/besu/cli/options/SynchronizerOptions.java
|
|
@@ -1,33 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
+ *
|
|
+ * Modifications Copyright contributors to the Aere Network.
|
|
+ *
|
|
+ * This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
+ * Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
|
|
*/
|
|
package org.hyperledger.besu.cli.options;
|
|
|
|
import static org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration.DEFAULT_SNAP_SYNC_SAVE_PRE_MERGE_HEADERS_ONLY_ENABLED;
|
|
|
|
import org.hyperledger.besu.ethereum.eth.sync.SynchronizerConfiguration;
|
|
import org.hyperledger.besu.ethereum.eth.sync.snapsync.ImmutableSnapSyncConfiguration;
|
|
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
|
|
|
|
import java.net.URI;
|
|
import java.util.Arrays;
|
|
import java.util.List;
|
|
|
|
import com.google.common.collect.Range;
|
|
import org.apache.tuweni.units.bigints.UInt256;
|
|
import org.slf4j.Logger;
|
|
import org.slf4j.LoggerFactory;
|
|
import picocli.CommandLine;
|
|
|
|
/** The Synchronizer Cli options. */
|
|
@@ -80,40 +86,46 @@ public class SynchronizerOptions implements CLIOptions<SynchronizerConfiguration
|
|
|
|
private static final String SNAP_PIVOT_BLOCK_WINDOW_VALIDITY_FLAG =
|
|
"--Xsnapsync-synchronizer-pivot-block-window-validity";
|
|
private static final String SNAP_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING_FLAG =
|
|
"--Xsnapsync-synchronizer-pivot-block-distance-before-caching";
|
|
private static final String SNAP_STORAGE_COUNT_PER_REQUEST_FLAG =
|
|
"--Xsnapsync-synchronizer-storage-count-per-request";
|
|
private static final String SNAP_BYTECODE_COUNT_PER_REQUEST_FLAG =
|
|
"--Xsnapsync-synchronizer-bytecode-count-per-request";
|
|
private static final String SNAP_TRIENODE_COUNT_PER_REQUEST_FLAG =
|
|
"--Xsnapsync-synchronizer-trienode-count-per-request";
|
|
private static final String SNAP_TRANSACTION_INDEXING_ENABLED_FLAG =
|
|
"--snapsync-synchronizer-transaction-indexing-enabled";
|
|
|
|
private static final String SNAP_FLAT_ACCOUNT_HEALED_COUNT_PER_REQUEST_FLAG =
|
|
"--Xsnapsync-synchronizer-flat-account-healed-count-per-request";
|
|
|
|
private static final String SNAP_FLAT_STORAGE_HEALED_COUNT_PER_REQUEST_FLAG =
|
|
"--Xsnapsync-synchronizer-flat-slot-healed-count-per-request";
|
|
|
|
+ private static final String SNAP_SERVER_MAX_CONCURRENT_REQUESTS_PER_PEER_FLAG =
|
|
+ "--Xsnapsync-server-max-concurrent-requests-per-peer";
|
|
+
|
|
+ private static final String SNAP_SERVER_MAX_CONCURRENT_REQUESTS_FLAG =
|
|
+ "--Xsnapsync-server-max-concurrent-requests";
|
|
+
|
|
private static final String CHECKPOINT_POST_MERGE_FLAG = "--Xcheckpoint-post-merge-enabled";
|
|
|
|
private static final String SNAP_SYNC_SAVE_PRE_CHECKPOINT_HEADERS_ONLY_FLAG =
|
|
"--snapsync-synchronizer-pre-checkpoint-headers-only-enabled";
|
|
|
|
private static final String ERA1_IMPORT_PREPIPELINE_ENABLED_FLAG =
|
|
"--era1-import-prepipeline-enabled";
|
|
private static final String ERA1_DATA_URI_FLAG = "--era1-data-uri";
|
|
private static final String ERA1_IMPORT_PREPIPELINE_CONCURRENCY_FLAG =
|
|
"--era1-import-prepipeline-concurrency";
|
|
|
|
/**
|
|
* Parse block propagation range.
|
|
*
|
|
* @param arg the range such as -10..30
|
|
*/
|
|
@CommandLine.Option(
|
|
names = BLOCK_PROPAGATION_RANGE_FLAG,
|
|
hidden = true,
|
|
defaultValue = "-10..30",
|
|
@@ -364,40 +376,58 @@ public class SynchronizerOptions implements CLIOptions<SynchronizerConfiguration
|
|
names = {CHECKPOINT_POST_MERGE_FLAG},
|
|
hidden = true,
|
|
paramLabel = "<Boolean>",
|
|
arity = "0..1",
|
|
fallbackValue = "true",
|
|
description =
|
|
"DEPRECATED: This option is deprecated as CHECKPOINT sync mode has been replaced by SNAP. This option no longer has any effect.")
|
|
private void setCheckpointPostMergeSyncEnabled(final Boolean enabled) {
|
|
LOG.warn(
|
|
"{} is deprecated and has no effect. CHECKPOINT sync mode has been replaced by SNAP.",
|
|
CHECKPOINT_POST_MERGE_FLAG);
|
|
}
|
|
|
|
@CommandLine.Option(
|
|
names = {"--Xpeertask-system-enabled"},
|
|
hidden = true,
|
|
description =
|
|
"Temporary feature toggle to enable using the new peertask system (default: ${DEFAULT-VALUE})")
|
|
private final Boolean isPeerTaskSystemEnabled = false;
|
|
|
|
+ @CommandLine.Option(
|
|
+ names = SNAP_SERVER_MAX_CONCURRENT_REQUESTS_PER_PEER_FLAG,
|
|
+ hidden = true,
|
|
+ paramLabel = "<INTEGER>",
|
|
+ description =
|
|
+ "Maximum number of snap sync GET_* requests from a single peer that may be concurrently scheduled for processing. 0 specifies no limit (default: ${DEFAULT-VALUE})")
|
|
+ private int snapsyncServerMaxConcurrentRequestsPerPeer =
|
|
+ SnapSyncConfiguration.DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_PER_PEER;
|
|
+
|
|
+ @CommandLine.Option(
|
|
+ names = SNAP_SERVER_MAX_CONCURRENT_REQUESTS_FLAG,
|
|
+ hidden = true,
|
|
+ paramLabel = "<INTEGER>",
|
|
+ description =
|
|
+ "Maximum total number of snap sync GET_* requests, across all peers, that may be concurrently scheduled for processing. 0 specifies no limit (default: ${DEFAULT-VALUE})")
|
|
+ private int snapsyncServerMaxConcurrentRequests =
|
|
+ SnapSyncConfiguration.DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_GLOBAL;
|
|
+
|
|
@CommandLine.Option(
|
|
names = SNAP_TRANSACTION_INDEXING_ENABLED_FLAG,
|
|
paramLabel = "<Boolean>",
|
|
arity = "0..1",
|
|
description =
|
|
"Enable transaction indexing during SNAP sync. Enabling will increase syncing time and disk usage. However, to support RPCs that use transaction hash for historical queries, you'll need to enable this. (default: ${DEFAULT-VALUE})")
|
|
private Boolean snapTransactionIndexingEnabled =
|
|
SnapSyncConfiguration.DEFAULT_SNAP_SYNC_TRANSACTION_INDEXING_ENABLED;
|
|
|
|
@CommandLine.Option(
|
|
names = {SNAP_SYNC_SAVE_PRE_CHECKPOINT_HEADERS_ONLY_FLAG},
|
|
paramLabel = "<Boolean>",
|
|
arity = "0..1",
|
|
description =
|
|
"Enable snap sync downloader to save only headers (not block bodies) for blocks before the checkpoint. (default: ${DEFAULT-VALUE})")
|
|
private Boolean snapSyncSavePreCheckpointHeadersOnlyEnabled =
|
|
DEFAULT_SNAP_SYNC_SAVE_PRE_MERGE_HEADERS_ONLY_ENABLED;
|
|
|
|
@CommandLine.Option(
|
|
names = ERA1_IMPORT_PREPIPELINE_ENABLED_FLAG,
|
|
@@ -479,40 +509,44 @@ public class SynchronizerOptions implements CLIOptions<SynchronizerConfiguration
|
|
options.worldStateHashCountPerRequest = config.getWorldStateHashCountPerRequest();
|
|
options.worldStateRequestParallelism = config.getWorldStateRequestParallelism();
|
|
options.worldStateMaxRequestsWithoutProgress = config.getWorldStateMaxRequestsWithoutProgress();
|
|
options.worldStateMinMillisBeforeStalling = config.getWorldStateMinMillisBeforeStalling();
|
|
options.worldStateTaskCacheSize = config.getWorldStateTaskCacheSize();
|
|
options.snapsyncPivotBlockWindowValidity =
|
|
config.getSnapSyncConfiguration().getPivotBlockWindowValidity();
|
|
options.snapsyncPivotBlockDistanceBeforeCaching =
|
|
config.getSnapSyncConfiguration().getPivotBlockDistanceBeforeCaching();
|
|
options.snapsyncStorageCountPerRequest =
|
|
config.getSnapSyncConfiguration().getStorageCountPerRequest();
|
|
options.snapsyncBytecodeCountPerRequest =
|
|
config.getSnapSyncConfiguration().getBytecodeCountPerRequest();
|
|
options.snapsyncTrieNodeCountPerRequest =
|
|
config.getSnapSyncConfiguration().getTrienodeCountPerRequest();
|
|
options.snapsyncFlatAccountHealedCountPerRequest =
|
|
config.getSnapSyncConfiguration().getLocalFlatAccountCountToHealPerRequest();
|
|
options.snapsyncFlatStorageHealedCountPerRequest =
|
|
config.getSnapSyncConfiguration().getLocalFlatStorageCountToHealPerRequest();
|
|
options.snapsyncServerEnabled = config.getSnapSyncConfiguration().isSnapServerEnabled();
|
|
+ options.snapsyncServerMaxConcurrentRequestsPerPeer =
|
|
+ config.getSnapSyncConfiguration().getMaxConcurrentSnapRequestsPerPeer();
|
|
+ options.snapsyncServerMaxConcurrentRequests =
|
|
+ config.getSnapSyncConfiguration().getMaxConcurrentSnapRequestsGlobal();
|
|
options.snapTransactionIndexingEnabled =
|
|
config.getSnapSyncConfiguration().isSnapSyncTransactionIndexingEnabled();
|
|
options.snapSyncSavePreCheckpointHeadersOnlyEnabled =
|
|
config.isSnapSyncSavePreCheckpointHeadersOnlyEnabled();
|
|
options.era1ImportPrepipelineEnabled = config.era1ImportPrepipelineEnabled();
|
|
options.era1DataUri = config.era1DataUri();
|
|
options.era1ImportPrepipelineConcurrency = config.era1ImportPrepipelineConcurrency();
|
|
options.receiptsDownloadStepTimeoutMillis = config.getReceiptsDownloadStepTimeoutMillis();
|
|
options.backwardHeadersDownloadStepTimeoutMillis =
|
|
config.getBackwardHeadersDownloadStepTimeoutMillis();
|
|
options.bodiesDownloadStepTimeoutMillis = config.getBodiesDownloadStepTimeoutMillis();
|
|
return options;
|
|
}
|
|
|
|
@Override
|
|
public SynchronizerConfiguration.Builder toDomainObject() {
|
|
final SynchronizerConfiguration.Builder builder = SynchronizerConfiguration.builder();
|
|
builder.blockPropagationRange(blockPropagationRange);
|
|
builder.downloaderChangeTargetThresholdByHeight(downloaderChangeTargetThresholdByHeight);
|
|
builder.downloaderChangeTargetThresholdByTd(downloaderChangeTargetThresholdByTd);
|
|
@@ -524,40 +558,42 @@ public class SynchronizerOptions implements CLIOptions<SynchronizerConfiguration
|
|
builder.headerDownloadParallelismFactor(headerDownloadParallelismFactor);
|
|
builder.transactionsParallelism(transactionsParallelism);
|
|
builder.computationParallelism(computationParallelism);
|
|
builder.syncPivotDistance(pivotDistance);
|
|
builder.fastSyncFullValidationRate(fastSyncFullValidationRate);
|
|
builder.worldStateHashCountPerRequest(worldStateHashCountPerRequest);
|
|
builder.worldStateRequestParallelism(worldStateRequestParallelism);
|
|
builder.worldStateMaxRequestsWithoutProgress(worldStateMaxRequestsWithoutProgress);
|
|
builder.worldStateMinMillisBeforeStalling(worldStateMinMillisBeforeStalling);
|
|
builder.worldStateTaskCacheSize(worldStateTaskCacheSize);
|
|
builder.snapSyncConfiguration(
|
|
ImmutableSnapSyncConfiguration.builder()
|
|
.pivotBlockWindowValidity(snapsyncPivotBlockWindowValidity)
|
|
.pivotBlockDistanceBeforeCaching(snapsyncPivotBlockDistanceBeforeCaching)
|
|
.storageCountPerRequest(snapsyncStorageCountPerRequest)
|
|
.bytecodeCountPerRequest(snapsyncBytecodeCountPerRequest)
|
|
.trienodeCountPerRequest(snapsyncTrieNodeCountPerRequest)
|
|
.localFlatAccountCountToHealPerRequest(snapsyncFlatAccountHealedCountPerRequest)
|
|
.localFlatStorageCountToHealPerRequest(snapsyncFlatStorageHealedCountPerRequest)
|
|
.isSnapServerEnabled(snapsyncServerEnabled)
|
|
+ .maxConcurrentSnapRequestsPerPeer(snapsyncServerMaxConcurrentRequestsPerPeer)
|
|
+ .maxConcurrentSnapRequestsGlobal(snapsyncServerMaxConcurrentRequests)
|
|
.isSnapSyncTransactionIndexingEnabled(snapTransactionIndexingEnabled)
|
|
.build());
|
|
builder.isPeerTaskSystemEnabled(isPeerTaskSystemEnabled);
|
|
builder.snapSyncSavePreCheckpointHeadersOnlyEnabled(
|
|
snapSyncSavePreCheckpointHeadersOnlyEnabled);
|
|
builder.receiptsDownloadStepTimeoutMillis(receiptsDownloadStepTimeoutMillis);
|
|
builder.backwardHeadersDownloadStepTimeoutMillis(backwardHeadersDownloadStepTimeoutMillis);
|
|
builder.bodiesDownloadStepTimeoutMillis(bodiesDownloadStepTimeoutMillis);
|
|
builder.era1ImportPrepipelineEnabled(era1ImportPrepipelineEnabled);
|
|
builder.era1DataUri(era1DataUri);
|
|
builder.era1ImportPrepipelineConcurrency(era1ImportPrepipelineConcurrency);
|
|
return builder;
|
|
}
|
|
|
|
@Override
|
|
public List<String> getCLIOptions() {
|
|
List<String> value =
|
|
Arrays.asList(
|
|
BLOCK_PROPAGATION_RANGE_FLAG,
|
|
OptionParser.format(blockPropagationRange),
|
|
@@ -598,33 +634,37 @@ public class SynchronizerOptions implements CLIOptions<SynchronizerConfiguration
|
|
BACKWARD_HEADERS_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG,
|
|
OptionParser.format(backwardHeadersDownloadStepTimeoutMillis),
|
|
BODIES_DOWNLOAD_STEP_TIMEOUT_MILLIS_FLAG,
|
|
OptionParser.format(bodiesDownloadStepTimeoutMillis),
|
|
SNAP_PIVOT_BLOCK_WINDOW_VALIDITY_FLAG,
|
|
OptionParser.format(snapsyncPivotBlockWindowValidity),
|
|
SNAP_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING_FLAG,
|
|
OptionParser.format(snapsyncPivotBlockDistanceBeforeCaching),
|
|
SNAP_STORAGE_COUNT_PER_REQUEST_FLAG,
|
|
OptionParser.format(snapsyncStorageCountPerRequest),
|
|
SNAP_BYTECODE_COUNT_PER_REQUEST_FLAG,
|
|
OptionParser.format(snapsyncBytecodeCountPerRequest),
|
|
SNAP_TRIENODE_COUNT_PER_REQUEST_FLAG,
|
|
OptionParser.format(snapsyncTrieNodeCountPerRequest),
|
|
SNAP_FLAT_ACCOUNT_HEALED_COUNT_PER_REQUEST_FLAG,
|
|
OptionParser.format(snapsyncFlatAccountHealedCountPerRequest),
|
|
SNAP_FLAT_STORAGE_HEALED_COUNT_PER_REQUEST_FLAG,
|
|
OptionParser.format(snapsyncFlatStorageHealedCountPerRequest),
|
|
SNAP_SERVER_ENABLED_FLAG,
|
|
OptionParser.format(snapsyncServerEnabled),
|
|
+ SNAP_SERVER_MAX_CONCURRENT_REQUESTS_PER_PEER_FLAG,
|
|
+ OptionParser.format(snapsyncServerMaxConcurrentRequestsPerPeer),
|
|
+ SNAP_SERVER_MAX_CONCURRENT_REQUESTS_FLAG,
|
|
+ OptionParser.format(snapsyncServerMaxConcurrentRequests),
|
|
SNAP_TRANSACTION_INDEXING_ENABLED_FLAG,
|
|
OptionParser.format(snapTransactionIndexingEnabled),
|
|
SNAP_SYNC_SAVE_PRE_CHECKPOINT_HEADERS_ONLY_FLAG,
|
|
OptionParser.format(snapSyncSavePreCheckpointHeadersOnlyEnabled),
|
|
ERA1_IMPORT_PREPIPELINE_ENABLED_FLAG,
|
|
OptionParser.format(era1ImportPrepipelineEnabled),
|
|
ERA1_DATA_URI_FLAG,
|
|
OptionParser.format(era1DataUri),
|
|
ERA1_IMPORT_PREPIPELINE_CONCURRENCY_FLAG,
|
|
OptionParser.format(era1ImportPrepipelineConcurrency));
|
|
return value;
|
|
}
|
|
}
|
|
diff --git a/app/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java b/app/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java
|
|
index 7d6a548be1..90eb90f718 100644
|
|
--- a/app/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java
|
|
+++ b/app/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java
|
|
@@ -1,33 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
+ *
|
|
+ * Modifications Copyright contributors to the Aere Network.
|
|
+ *
|
|
+ * This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
+ * Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
|
|
*/
|
|
package org.hyperledger.besu.controller;
|
|
|
|
import static com.google.common.base.Preconditions.checkNotNull;
|
|
|
|
import org.hyperledger.besu.chainimport.BlockHeadersCachePreload;
|
|
import org.hyperledger.besu.components.BesuComponent;
|
|
import org.hyperledger.besu.config.GenesisConfig;
|
|
import org.hyperledger.besu.config.GenesisConfigOptions;
|
|
import org.hyperledger.besu.consensus.merge.MergeContext;
|
|
import org.hyperledger.besu.consensus.merge.UnverifiedForkchoiceSupplier;
|
|
import org.hyperledger.besu.consensus.qbft.BFTPivotSelectorFromPeers;
|
|
import org.hyperledger.besu.cryptoservices.NodeKey;
|
|
import org.hyperledger.besu.datatypes.Hash;
|
|
import org.hyperledger.besu.ethereum.ConsensusContext;
|
|
import org.hyperledger.besu.ethereum.ProtocolContext;
|
|
import org.hyperledger.besu.ethereum.api.ApiConfiguration;
|
|
import org.hyperledger.besu.ethereum.api.jsonrpc.methods.JsonRpcMethods;
|
|
import org.hyperledger.besu.ethereum.blockcreation.MiningCoordinator;
|
|
import org.hyperledger.besu.ethereum.chain.BadBlockManager;
|
|
@@ -1318,41 +1324,42 @@ public abstract class BesuControllerBuilder implements MiningConfigurationOverri
|
|
.build();
|
|
}
|
|
|
|
private Optional<SnapProtocolManager> createSnapProtocolManager(
|
|
final ProtocolSchedule protocolSchedule,
|
|
final ProtocolContext protocolContext,
|
|
final WorldStateStorageCoordinator worldStateStorageCoordinator,
|
|
final EthPeers ethPeers,
|
|
final EthMessages snapMessages,
|
|
final EthScheduler ethScheduler,
|
|
final Synchronizer synchronizer) {
|
|
return Optional.of(
|
|
new SnapProtocolManager(
|
|
worldStateStorageCoordinator,
|
|
syncConfig.getSnapSyncConfiguration(),
|
|
ethPeers,
|
|
snapMessages,
|
|
ethScheduler,
|
|
protocolSchedule,
|
|
protocolContext,
|
|
- synchronizer));
|
|
+ synchronizer,
|
|
+ metricsSystem));
|
|
}
|
|
|
|
WorldStateArchive createWorldStateArchive(
|
|
final WorldStateStorageCoordinator worldStateStorageCoordinator,
|
|
final Blockchain blockchain,
|
|
final BonsaiCachedMerkleTrieLoader bonsaiCachedMerkleTrieLoader,
|
|
final Supplier<WorldStateHealer> worldStateHealerSupplier) {
|
|
return switch (dataStorageConfiguration.getDataStorageFormat()) {
|
|
case BONSAI -> {
|
|
final BonsaiWorldStateKeyValueStorage worldStateKeyValueStorage =
|
|
worldStateStorageCoordinator.getStrategy(BonsaiWorldStateKeyValueStorage.class);
|
|
|
|
yield new BonsaiWorldStateProvider(
|
|
worldStateKeyValueStorage,
|
|
blockchain,
|
|
dataStorageConfiguration.getPathBasedExtraStorageConfiguration(),
|
|
bonsaiCachedMerkleTrieLoader,
|
|
besuComponent.map(BesuComponent::getBesuPluginContext).orElse(null),
|
|
evmConfiguration,
|
|
worldStateHealerSupplier,
|
|
diff --git a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealCache.java b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealCache.java
|
|
index 716c67e0f5..e3ddc481f2 100755
|
|
--- a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealCache.java
|
|
+++ b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/PqSealCache.java
|
|
@@ -1,86 +1,114 @@
|
|
/*
|
|
* Copyright contributors to Besu / AERE Network.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*/
|
|
package org.hyperledger.besu.consensus.common.bft;
|
|
|
|
+import org.hyperledger.besu.consensus.common.bft.blockcreation.PqAnchorProducer;
|
|
import org.hyperledger.besu.datatypes.Hash;
|
|
|
|
import java.nio.file.Path;
|
|
import java.util.ArrayList;
|
|
import java.util.Collection;
|
|
+import java.util.HashSet;
|
|
import java.util.Iterator;
|
|
import java.util.LinkedHashMap;
|
|
import java.util.List;
|
|
import java.util.Map;
|
|
+import java.util.Optional;
|
|
+import java.util.Set;
|
|
|
|
+import org.apache.tuweni.bytes.Bytes;
|
|
+import org.apache.tuweni.bytes.Bytes32;
|
|
import org.slf4j.Logger;
|
|
import org.slf4j.LoggerFactory;
|
|
|
|
/**
|
|
* AERE ANCHOR V2: the bounded store of Falcon seals this node HEARD for recent blocks, in memory and
|
|
* - for the chain head only - on disk.
|
|
*
|
|
* <p>WHAT IT IS FOR. The proposer of block N must carry a certificate over block N-1. Under
|
|
* V2 that certificate is chosen by the proposer out of what it heard, so the seals gossiped on
|
|
* Commit messages have to survive from the height where they arrive to the height where the next
|
|
* block is built. That is all this class does.
|
|
*
|
|
* <p>KEYED ON THE ON-CHAIN BLOCK HASH, NEVER ON THE COMMIT DIGEST. The digest carried by a Commit
|
|
* message is the committed-seal hash, which INCLUDES the round number, so it differs between rounds
|
|
* for one and the same block. The producer looks the certificate up by {@code
|
|
* parentHeader.getHash()}, which is the round-independent on-chain hash. Storing under anything
|
|
* else would silently produce empty certificates at every round change.
|
|
*
|
|
* <p>AERE PERSISTENTA-SIGILII (2026-08-05): PERSISTENT FOR THE CHAIN HEAD, AND THE OLD REASONING FOR
|
|
* NOT BEING SO WAS MEASURED WRONG. This class used to say, in this exact place, that losing the map
|
|
* on restart costs "at most one proposer turn per restart". That reasoning is correct for ONE node
|
|
* restarted inside a LIVE fleet, which is the case it was written for. For a fleet restarted
|
|
* ENTIRELY it is false, and the falsehood was measured on a seven-node network: with K>0 every
|
|
* node came back holding zero seals, no certificate could reach K, nobody could propose, so no
|
|
* Commit was ever sent, so no node ever heard a seal again. The chain was dead, permanently, and no
|
|
* amount of waiting healed it. The seals over M(head) exist nowhere else: they travel only on the
|
|
* Commit messages of the head block, and those are never replayed.
|
|
*
|
|
* <p>AND IT IS NOT DEFECT A8 IN ANOTHER COAT. A8 was a registry of public keys read from a file and
|
|
* BELIEVED. Every seal read back here is re-verified, cryptographically, against the anchored
|
|
* registry over M rebuilt from the header this process just loaded - see {@link PqSealStore}. A
|
|
* forged file cannot inject a seal without forging a Falcon-512 signature; the worst it achieves is
|
|
* the empty cache an absent file already gives. Persistence is OFF unless a caller enables it, and
|
|
* the only caller that does is the QBFT controller builder, only when the anchor is actually armed.
|
|
*
|
|
- * <p>WHAT IT DOES NOT DO. The in-memory path verifies nothing. Whether a seal is valid, whether its
|
|
- * index maps to an eligible validator, and whether there are enough of them, are decided at
|
|
- * SELECTION time by the producer, against the message M of the specific parent. A seal that was
|
|
- * signed under the pre-fork message simply fails that check and is dropped, which is what makes the
|
|
- * height switch safe without any coordination beyond the shared activation height.
|
|
+ * <p>WHAT IT DOES NOT DO. Whether a seal index maps to an eligible validator, and whether there are
|
|
+ * enough seals, are decided at SELECTION time by the producer, against the message M of the specific
|
|
+ * parent. A seal that was signed under the pre-fork message simply fails that check and is dropped,
|
|
+ * which is what makes the height switch safe without any coordination beyond the shared activation
|
|
+ * height.
|
|
+ *
|
|
+ * <p>D-454 FAMILY (adversarial review of 2026-09-30): ONE SLOT PER INDEX, AND AN UNVERIFIED SEAL NEVER
|
|
+ * HOLDS IT AGAINST A VERIFIED ONE. Until this date the in-memory path verified nothing and the FIRST
|
|
+ * seal heard for an index won for good. The Commits feeding it are not all judged first: the D-339
|
|
+ * path keeps the seal of any Commit heard before the proposal, the late-seal salvage keeps any
|
|
+ * validator's late Commit, and below the commit enforcement no Commit's post-quantum seal is judged
|
|
+ * at all. So a garbage seal naming A's index, heard before A's own (below H_fin any validator can
|
|
+ * send one; from H_fin, when the author of a Commit is read from its seal, a copy of A's public
|
|
+ * commit seal in anyone's envelope), kept A's genuine seal out of every certificate this node could
|
|
+ * build - and enough of them, at the parent of an anchor, left no proposer able to reach K. Now, from
|
|
+ * H_fin ({@link AerePqFinality#PROPERTY}, AIP-22 stage 4, {@link #judgeGoverns}) and once the anchor
|
|
+ * is armed on this node ({@link #enablePersistence}, the one production entry point), a seal is
|
|
+ * JUDGED as it arrives, with the producer's own check ({@link #verifiesAsTheProducerDoes}): a seal
|
|
+ * that verifies replaces one that does not, a seal that does not verify never replaces anything, and
|
|
+ * a verified seal is never replaced. Without the judge (anchor not armed, or a height whose seal is
|
|
+ * not in the anchor form) every seal is unverified, which is exactly the first-seal-wins store of
|
|
+ * before. BELOW H_fin (and on every node where H_fin is unset) the store is the one of before, call
|
|
+ * by call: first seal per index wins, nothing is verified, the restore and the head file follow the
|
|
+ * rules of before (second review of D-454, 2026-10-01: armed from enablePersistence alone, the judge
|
|
+ * changed the live chain below the stage that needs it and put post-quantum verifications on its
|
|
+ * consensus thread). The producer still re-verifies every seal it carries; the judge decides only
|
|
+ * which seal is KEPT.
|
|
*/
|
|
public final class PqSealCache {
|
|
|
|
private static final Logger LOG = LoggerFactory.getLogger(PqSealCache.class);
|
|
|
|
/**
|
|
* AIP-21 (AERE, 2026-09-18): how far below the highest seen height entries are kept, in blocks. The
|
|
* consensus default stays 256 (what the producer needs is the parent); a node that serves
|
|
* aere_getPqFinality may keep more through the system property, at ~7 kB per block for ten seals.
|
|
* Read once, never below 256, never a consensus value: two nodes with different windows import
|
|
* the same chain.
|
|
*/
|
|
public static final String PROPERTY_FINALITY_WINDOW = "aere.pq.finality.window";
|
|
|
|
private static final long DEFAULT_HEIGHT_WINDOW = 256L;
|
|
|
|
/** How far below the highest seen height entries are kept, in blocks. */
|
|
private static final long HEIGHT_WINDOW = readWindow();
|
|
|
|
/** How many distinct block hashes are retained. Several rounds at one height each add one. */
|
|
@@ -111,86 +139,218 @@ public final class PqSealCache {
|
|
/** Guards the file, never the map, so a slow disk cannot hold up the consensus thread's map. */
|
|
private final Object writeLock = new Object();
|
|
|
|
private volatile Path persistenceFile;
|
|
private volatile long persistenceChainId;
|
|
|
|
private long persistedBlock = -1L;
|
|
private Hash persistedHash;
|
|
private int persistedCount = -1;
|
|
|
|
// AIP-21, SECOND HALF (2026-09-18): the finality window survives a restart. Heard heights are
|
|
// appended to a segmented journal (PqSealWindowStore) when the NEXT height begins, and read back,
|
|
// re-verified seal by seal against the canonical chain, at startup. Null = journal off.
|
|
private volatile Path windowDirectory;
|
|
private volatile long windowPrunedSegment = -1L;
|
|
private long windowJournaledThrough = -1L;
|
|
|
|
/** Heights restored at startup at most, newest first, whatever the configured window. */
|
|
private static final long MAX_RESTORED_HEIGHTS = 4096L;
|
|
|
|
+ /** D-454 family: the seals last written to the head file (compared, not only counted). */
|
|
+ private List<FalconSeal> persistedSeals = List.of();
|
|
+
|
|
+ /**
|
|
+ * D-454 FAMILY (2026-09-30): whether a heard seal VERIFIES. Must never throw; false means "not
|
|
+ * shown valid", never "shown invalid".
|
|
+ */
|
|
+ @FunctionalInterface
|
|
+ public interface SealJudge {
|
|
+ /**
|
|
+ * Judge one heard seal.
|
|
+ *
|
|
+ * @param blockNumber the height of the block the seal attests
|
|
+ * @param onchainBlockHash the ON-CHAIN hash of that block
|
|
+ * @param schemeWireId the certificate-v2 wire tag of the seal's scheme (Falcon-512 for the v1 list)
|
|
+ * @param validatorIndex the registry index the seal names
|
|
+ * @param signature the seal's signature bytes
|
|
+ * @return true only when the signature verifies under the key the index names at that height
|
|
+ */
|
|
+ boolean verifies(
|
|
+ long blockNumber,
|
|
+ Hash onchainBlockHash,
|
|
+ byte schemeWireId,
|
|
+ int validatorIndex,
|
|
+ Bytes signature);
|
|
+
|
|
+ /** Judges nothing: every seal is unverified, which is exactly the first-seal-wins store of before. */
|
|
+ SealJudge NONE = (h, hash, wire, index, sig) -> false;
|
|
+ }
|
|
+
|
|
+ private volatile SealJudge judge = SealJudge.NONE;
|
|
+
|
|
private PqSealCache() {}
|
|
|
|
+ /**
|
|
+ * D-454 FAMILY, BOUND TO H_FIN (second review, 2026-10-01): whether the judge of heard seals governs
|
|
+ * a height. Only from H_fin ({@link AerePqFinality#PROPERTY}, {@code aere.pq.finalityBlock}, AIP-22
|
|
+ * stage 4: the height from which the author of a Commit is read from its seal, the one
|
|
+ * RoundState's pending commits are governed by too); below it, and on every node where H_fin is
|
|
+ * unset, the store is the one of before, call by call, and no seal is verified on hearing.
|
|
+ *
|
|
+ * @param blockNumber the height of the block the seals attest
|
|
+ * @return true from H_fin
|
|
+ */
|
|
+ static boolean judgeGoverns(final long blockNumber) {
|
|
+ return AerePqFinality.active(blockNumber);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * Arms the judge of heard seals ({@link SealJudge#NONE} or null disarms it). The one way in:
|
|
+ * production arms it from {@link #enablePersistence}; the cost measurement gives a counting judge.
|
|
+ *
|
|
+ * @param sealJudge the judge
|
|
+ */
|
|
+ void judgeWith(final SealJudge sealJudge) {
|
|
+ this.judge = sealJudge == null ? SealJudge.NONE : sealJudge;
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * D-454 FAMILY (2026-09-30): THE PRODUCER'S OWN CHECK, applied to a seal as it is heard. The message
|
|
+ * is the anchor form over the block's on-chain hash ({@link PqAnchor#commitMessage}, what the
|
|
+ * producer verifies against at selection time); a Falcon-512 seal is verified by the registry's
|
|
+ * own-head door at the seal's height, exactly as the producer verifies it at the parent's height; an
|
|
+ * extra scheme seal under the key the producer resolves (the v3 epoch from H_k, the hybrid registry
|
|
+ * before). Below the anchor form (the committed-seal digest, which needs the block) nothing is
|
|
+ * verified here and the store keeps its first-seal-wins rule. Never throws.
|
|
+ *
|
|
+ * @param blockNumber the height of the block the seal attests
|
|
+ * @param onchainBlockHash the ON-CHAIN hash of that block
|
|
+ * @param schemeWireId the wire tag of the seal's scheme
|
|
+ * @param validatorIndex the registry index the seal names
|
|
+ * @param signature the seal's signature bytes
|
|
+ * @return true only when the seal verifies as the producer would verify it
|
|
+ */
|
|
+ static boolean verifiesAsTheProducerDoes(
|
|
+ final long blockNumber,
|
|
+ final Hash onchainBlockHash,
|
|
+ final byte schemeWireId,
|
|
+ final int validatorIndex,
|
|
+ final Bytes signature) {
|
|
+ try {
|
|
+ if (!PqAnchorProducer.sealMessageIsAnchorForm(blockNumber)) {
|
|
+ return false;
|
|
+ }
|
|
+ final Bytes32 message =
|
|
+ PqAnchor.commitMessage(
|
|
+ PqAnchorProducer.config().chainId(), blockNumber, onchainBlockHash.getBytes());
|
|
+ if (schemeWireId == SealSchemes.FALCON_512.wireId()) {
|
|
+ return PqSignerRegistry.falconSealSupport()
|
|
+ .verifyAtOwnHead(blockNumber, validatorIndex, message, signature);
|
|
+ }
|
|
+ final Optional<SealScheme> scheme = SealSchemes.byWireId(schemeWireId);
|
|
+ if (scheme.isEmpty()) {
|
|
+ return false;
|
|
+ }
|
|
+ final String schemeId = scheme.get().id();
|
|
+ final Optional<byte[]> publicKey =
|
|
+ AerePqIdentity.keysFromV3(blockNumber)
|
|
+ ? AerePqIdentity.v3KeyAt(blockNumber, validatorIndex, schemeId)
|
|
+ : HybridSealSupport.instance()
|
|
+ .registry()
|
|
+ .flatMap(r -> r.publicKey(validatorIndex, schemeId));
|
|
+ return publicKey.isPresent()
|
|
+ && scheme.get().verifyRaw(publicKey.get(), message.toArray(), signature.toArray());
|
|
+ } catch (final RuntimeException e) {
|
|
+ return false;
|
|
+ }
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * Whether the seal an index already holds for a block stays when another one arrives: always when it
|
|
+ * was verified, and when the newcomer is not verified. One place, so no caller can half-apply it (the
|
|
+ * second client's KeepsHeardSeal is the same rule).
|
|
+ *
|
|
+ * @param held whether the index holds a seal
|
|
+ * @param heldVerified whether that seal was verified
|
|
+ * @param newVerified whether the newcomer verified
|
|
+ * @return true when the held seal stays
|
|
+ */
|
|
+ static boolean keepsHeldSeal(
|
|
+ final boolean held, final boolean heldVerified, final boolean newVerified) {
|
|
+ return held && (heldVerified || !newVerified);
|
|
+ }
|
|
+
|
|
/**
|
|
* The process-wide cache.
|
|
*
|
|
* @return the singleton
|
|
*/
|
|
public static PqSealCache instance() {
|
|
return INSTANCE;
|
|
}
|
|
|
|
/**
|
|
* Turn on writing the chain head's seals to the node's data directory.
|
|
*
|
|
* <p>The path is DERIVED from the data directory Besu was started with, never configured
|
|
* separately: a second configurable path is a second thing that can be pointed at the wrong node's
|
|
* state, and this file is per-node by construction.
|
|
*
|
|
+ * <p>D-454 FAMILY (2026-09-30): this is the one production entry point reached only with the anchor
|
|
+ * armed (the QBFT controller builder, after the registry is active), so it also arms the judge of
|
|
+ * heard seals ({@link #verifiesAsTheProducerDoes}), with or without a data directory. Arming it
|
|
+ * here, and never lazily from the store itself, keeps a node or a test without the anchor from
|
|
+ * initialising the registry singletons as a side effect of hearing a seal. Armed is not governing:
|
|
+ * the judge is consulted only from H_fin ({@link #judgeGoverns}, bound 2026-10-01).
|
|
+ *
|
|
* @param dataDirectory the node's data directory, or null to leave persistence off
|
|
* @param chainId the chain id used in the M pre-image, which must be the SAME value the producer
|
|
* uses or every restored seal would fail verification
|
|
*/
|
|
public void enablePersistence(final Path dataDirectory, final long chainId) {
|
|
+ judgeWith(PqSealCache::verifiesAsTheProducerDoes);
|
|
if (dataDirectory == null) {
|
|
LOG.warn(
|
|
"AERE PERSISTENTA-SIGILII: no data directory was supplied, so heard seals stay in memory "
|
|
+ "only. A simultaneous restart of the whole fleet with K>0 stops the chain.");
|
|
return;
|
|
}
|
|
this.persistenceChainId = chainId;
|
|
this.persistenceFile = PqSealStore.fileIn(dataDirectory);
|
|
this.windowPrunedSegment = -1L;
|
|
this.windowDirectory =
|
|
PqSealWindowStore.enabled() ? PqSealWindowStore.directoryIn(dataDirectory) : null;
|
|
LOG.info(
|
|
"AERE PERSISTENTA-SIGILII: heard seals for the chain head are kept in {} (chainId={}). "
|
|
+ "Every seal read back is re-verified against the anchored registry before it is used.",
|
|
this.persistenceFile,
|
|
chainId);
|
|
}
|
|
|
|
/**
|
|
- * Turn persistence off again. For tests, and for an operator who wants the previous behaviour.
|
|
+ * Turn persistence off again. For tests, and for an operator who wants the previous behaviour. It
|
|
+ * also disarms the judge of heard seals (D-454 family), back to the first-seal-wins store.
|
|
*/
|
|
public void disablePersistence() {
|
|
this.persistenceFile = null;
|
|
+ this.judge = SealJudge.NONE;
|
|
this.windowDirectory = null;
|
|
synchronized (writeLock) {
|
|
persistedBlock = -1L;
|
|
persistedHash = null;
|
|
persistedCount = -1;
|
|
}
|
|
}
|
|
|
|
/**
|
|
* The file heard seals are written to, or null when persistence is off.
|
|
*
|
|
* @return the seal file, or null
|
|
*/
|
|
public Path persistenceFile() {
|
|
return persistenceFile;
|
|
}
|
|
|
|
/**
|
|
* Refill the cache from disk for a specific block, normally this node's chain head at startup.
|
|
*
|
|
@@ -230,175 +390,264 @@ public final class PqSealCache {
|
|
previous);
|
|
}
|
|
}
|
|
} catch (final RuntimeException e) {
|
|
// readVerified is written not to throw; this is the belt on top of the braces, because an
|
|
// exception escaping here would turn a lost cache into a node that refuses to start.
|
|
LOG.warn(
|
|
"AERE PERSISTENTA-SIGILII: restoring seals from {} failed unexpectedly ({}); the cache "
|
|
+ "starts empty, which is exactly the behaviour before this file existed.",
|
|
file,
|
|
e.toString());
|
|
return 0;
|
|
}
|
|
if (verified.isEmpty()) {
|
|
return 0;
|
|
}
|
|
synchronized (this) {
|
|
final Entry entry =
|
|
byHash.computeIfAbsent(onchainBlockHash, unused -> new Entry(blockNumber));
|
|
for (final FalconSeal seal : verified) {
|
|
- entry.seals.putIfAbsent(seal.getValidatorIndex(), seal);
|
|
+ if (!judgeGoverns(blockNumber)) {
|
|
+ // below H_fin: the restore of before, call by call (D-454 family, bound to H_fin)
|
|
+ entry.seals.putIfAbsent(seal.getValidatorIndex(), seal);
|
|
+ continue;
|
|
+ }
|
|
+ // re-verified by readVerified: it displaces an unverified seal held for its index (D-454 family)
|
|
+ final int index = seal.getValidatorIndex();
|
|
+ if (!keepsHeldSeal(entry.seals.containsKey(index), entry.verified.contains(index), true)) {
|
|
+ entry.seals.put(index, seal);
|
|
+ entry.verified.add(index);
|
|
+ }
|
|
}
|
|
if (blockNumber > highestSeen) {
|
|
highestSeen = blockNumber;
|
|
}
|
|
prune();
|
|
}
|
|
return verified.size();
|
|
}
|
|
|
|
/**
|
|
- * Remember the seals heard for a block. Idempotent per validator index: the first seal seen for
|
|
- * an index wins, so a later duplicate cannot displace it.
|
|
+ * Remember the seals heard for a block. One slot per validator index: a later duplicate cannot
|
|
+ * displace the seal held, and (D-454 family, 2026-09-30) from H_fin neither can any seal that does
|
|
+ * not verify; a seal that verifies displaces one that did not ({@link #keepsHeldSeal}). Below H_fin,
|
|
+ * or without the judge armed, nothing is verified and the first seal seen for an index wins, as
|
|
+ * before.
|
|
*
|
|
* @param blockNumber the height of the block the seals attest
|
|
* @param onchainBlockHash the ON-CHAIN hash of that block
|
|
* @param seals the seals heard, possibly empty
|
|
*/
|
|
public void record(
|
|
final long blockNumber, final Hash onchainBlockHash, final Collection<FalconSeal> seals) {
|
|
final List<FalconSeal> toPersist;
|
|
List<PqSealWindowStore.Heard> windowToJournal = List.of();
|
|
synchronized (this) {
|
|
if (onchainBlockHash == null || seals == null || seals.isEmpty()) {
|
|
return;
|
|
}
|
|
final Entry entry =
|
|
byHash.computeIfAbsent(onchainBlockHash, unused -> new Entry(blockNumber));
|
|
for (final FalconSeal seal : seals) {
|
|
if (seal != null && seal.getValidatorIndex() >= 0 && seal.getSignature() != null) {
|
|
- entry.seals.putIfAbsent(seal.getValidatorIndex(), seal);
|
|
+ if (!judgeGoverns(blockNumber)) {
|
|
+ // below H_fin: the store of before, call by call (first seal per index wins, nothing judged)
|
|
+ entry.seals.putIfAbsent(seal.getValidatorIndex(), seal);
|
|
+ continue;
|
|
+ }
|
|
+ final int index = seal.getValidatorIndex();
|
|
+ final FalconSeal held = entry.seals.get(index);
|
|
+ // the same bytes again (every commit re-offers the round's seals), or a verified seal held:
|
|
+ // nothing to judge, the held seal stays
|
|
+ if (held != null
|
|
+ && (entry.verified.contains(index) || held.getSignature().equals(seal.getSignature()))) {
|
|
+ continue;
|
|
+ }
|
|
+ final boolean verified =
|
|
+ judge.verifies(
|
|
+ blockNumber,
|
|
+ onchainBlockHash,
|
|
+ SealSchemes.FALCON_512.wireId(),
|
|
+ index,
|
|
+ seal.getSignature());
|
|
+ if (keepsHeldSeal(held != null, false, verified)) {
|
|
+ continue;
|
|
+ }
|
|
+ entry.seals.put(index, seal);
|
|
+ if (verified) {
|
|
+ entry.verified.add(index);
|
|
+ }
|
|
}
|
|
}
|
|
if (blockNumber > highestSeen) {
|
|
// AIP-21 second half: a NEW height begins, so what was heard for the heights below it is as
|
|
// complete as this node will ever hold it. It goes to the window journal, outside this lock.
|
|
windowToJournal = windowSnapshotBelow(blockNumber);
|
|
highestSeen = blockNumber;
|
|
}
|
|
prune();
|
|
if (persistenceFile == null || blockNumber < highestSeen) {
|
|
// Only the HIGHEST block is worth a file: the producer of the next block asks for the head's
|
|
// seals and nothing else, and writing older heights would just churn the disk.
|
|
return;
|
|
}
|
|
toPersist = PqAnchor.sortedByIndex(new ArrayList<>(entry.seals.values()));
|
|
}
|
|
// Deliberately OUTSIDE the map lock: a file write must never be able to stall a caller that only
|
|
// wants to read the cache.
|
|
persist(blockNumber, onchainBlockHash, toPersist);
|
|
// the head file first (liveness depends on it), the window journal after (best effort)
|
|
persistWindow(windowToJournal);
|
|
}
|
|
|
|
+ /** Same indices and the same signature bytes, in the same (canonical) order. */
|
|
+ private static boolean sameSeals(final List<FalconSeal> a, final List<FalconSeal> b) {
|
|
+ if (a.size() != b.size()) {
|
|
+ return false;
|
|
+ }
|
|
+ for (int i = 0; i < a.size(); i++) {
|
|
+ if (a.get(i).getValidatorIndex() != b.get(i).getValidatorIndex()
|
|
+ || !a.get(i).getSignature().equals(b.get(i).getSignature())) {
|
|
+ return false;
|
|
+ }
|
|
+ }
|
|
+ return true;
|
|
+ }
|
|
+
|
|
private void persist(
|
|
final long blockNumber, final Hash onchainBlockHash, final List<FalconSeal> seals) {
|
|
final Path file = persistenceFile;
|
|
if (file == null) {
|
|
return;
|
|
}
|
|
synchronized (writeLock) {
|
|
if (blockNumber < persistedBlock) {
|
|
return;
|
|
}
|
|
+ // D-454 family (2026-09-30): from H_fin a verified seal can REPLACE an unverified one without
|
|
+ // changing the count, so "no more seals than the file holds" is no longer "nothing new" there: the
|
|
+ // file is kept only when it holds more seals, or the very same ones. Below H_fin nothing is ever
|
|
+ // replaced and the rule of before stands, call by call (bound to H_fin, 2026-10-01).
|
|
if (blockNumber == persistedBlock
|
|
&& onchainBlockHash.equals(persistedHash)
|
|
- && seals.size() <= persistedCount) {
|
|
+ && (judgeGoverns(blockNumber)
|
|
+ ? (seals.size() < persistedCount
|
|
+ || (seals.size() == persistedCount && sameSeals(seals, persistedSeals)))
|
|
+ : seals.size() <= persistedCount)) {
|
|
return;
|
|
}
|
|
try {
|
|
// MAGAZIA TINE VARFUL (D-330, 2026-09-03): before the main file moves to a HIGHER height, what it
|
|
// held becomes the previous slot. The main file may hold a proposal that never imports; the
|
|
// previous slot then still holds the head, and a restart can restore the head's seals from it.
|
|
if (persistedBlock >= 0 && blockNumber > persistedBlock && java.nio.file.Files.isRegularFile(file)) {
|
|
java.nio.file.Files.move(
|
|
file,
|
|
file.resolveSibling(PqSealStore.PREVIOUS_FILE_NAME),
|
|
java.nio.file.StandardCopyOption.REPLACE_EXISTING);
|
|
}
|
|
PqSealStore.writeAtomically(
|
|
file,
|
|
PqSealStore.encode(
|
|
persistenceChainId, blockNumber, onchainBlockHash.getBytes(), seals));
|
|
persistedBlock = blockNumber;
|
|
persistedHash = onchainBlockHash;
|
|
persistedCount = seals.size();
|
|
+ if (judgeGoverns(blockNumber)) {
|
|
+ persistedSeals = List.copyOf(seals);
|
|
+ }
|
|
} catch (final Exception e) {
|
|
// A disk fault must not be able to stop this node from taking part in consensus. It costs
|
|
// the restart protection, which is what the log line says, and nothing else.
|
|
LOG.warn(
|
|
"AERE PERSISTENTA-SIGILII: could not write the {} heard seal(s) for block {} to {} "
|
|
+ "({}). This node keeps working; it just loses its seals if it restarts now.",
|
|
seals.size(),
|
|
blockNumber,
|
|
file,
|
|
e.toString());
|
|
}
|
|
}
|
|
}
|
|
|
|
/**
|
|
* The seals heard for a block, in canonical order.
|
|
*
|
|
* @param blockNumber the height of the block the seals must attest
|
|
* @param onchainBlockHash the ON-CHAIN hash of that block
|
|
* @return the seals, sorted by strictly increasing validator index, possibly empty
|
|
*/
|
|
/**
|
|
* AERE ANCHOR V2: remember the extra (non-Falcon) scheme seals heard on the commits of a block.
|
|
- * Same key as {@link #record}, same window, same idempotence (first seal per validator and
|
|
- * scheme wins). NOT persisted across a restart: a node that restarts holds no extras until it
|
|
+ * Same key as {@link #record}, same window, same slot rule per validator and scheme (D-454 family:
|
|
+ * from H_fin an unverified seal never displaces a verified one and a verified one displaces an
|
|
+ * unverified one; below H_fin, or without the judge, the first seal wins, as before). NOT persisted
|
|
+ * across a restart: a node that restarts holds no extras until it
|
|
* takes part in one commit, exactly the Falcon situation before PqSealStore, and the producer
|
|
* then refuses the v2 certificate for one proposer turn rather than writing a short one.
|
|
*
|
|
* @param blockNumber the height of the block the seals are over
|
|
* @param onchainBlockHash the round-independent on-chain hash of that block
|
|
* @param extraSeals the heard scheme seals, unverified
|
|
*/
|
|
public void recordExtras(
|
|
final long blockNumber, final Hash onchainBlockHash, final Collection<SchemeSeal> extraSeals) {
|
|
synchronized (this) {
|
|
if (onchainBlockHash == null || extraSeals == null || extraSeals.isEmpty()) {
|
|
return;
|
|
}
|
|
final Entry entry =
|
|
byHash.computeIfAbsent(onchainBlockHash, unused -> new Entry(blockNumber));
|
|
for (final SchemeSeal seal : extraSeals) {
|
|
if (seal == null
|
|
|| seal.getValidatorIndex() < 0
|
|
|| seal.getSignature() == null
|
|
|| seal.getSchemeWireId() == SealSchemes.FALCON_512.wireId()) {
|
|
continue;
|
|
}
|
|
final long key = ((long) seal.getValidatorIndex() << 8) | (seal.getSchemeWireId() & 0xffL);
|
|
- entry.extras.putIfAbsent(key, seal);
|
|
+ if (!judgeGoverns(blockNumber)) {
|
|
+ // below H_fin: the store of before, call by call (first seal per index and scheme wins)
|
|
+ entry.extras.putIfAbsent(key, seal);
|
|
+ continue;
|
|
+ }
|
|
+ // D-454 family (2026-09-30): the same rule as the Falcon slot, per (index, scheme)
|
|
+ final SchemeSeal held = entry.extras.get(key);
|
|
+ if (held != null
|
|
+ && (entry.extrasVerified.contains(key) || held.getSignature().equals(seal.getSignature()))) {
|
|
+ continue;
|
|
+ }
|
|
+ final boolean verified =
|
|
+ judge.verifies(
|
|
+ blockNumber,
|
|
+ onchainBlockHash,
|
|
+ seal.getSchemeWireId(),
|
|
+ seal.getValidatorIndex(),
|
|
+ seal.getSignature());
|
|
+ if (keepsHeldSeal(held != null, false, verified)) {
|
|
+ continue;
|
|
+ }
|
|
+ entry.extras.put(key, seal);
|
|
+ if (verified) {
|
|
+ entry.extrasVerified.add(key);
|
|
+ }
|
|
}
|
|
if (blockNumber > highestSeen) {
|
|
highestSeen = blockNumber;
|
|
}
|
|
prune();
|
|
}
|
|
}
|
|
|
|
/**
|
|
* AERE ANCHOR V2: the extra scheme seals heard for a block, canonical order, unverified.
|
|
*
|
|
* @param blockNumber the height
|
|
* @param onchainBlockHash the round-independent on-chain hash
|
|
* @return the seals, or empty when nothing was heard under that key
|
|
*/
|
|
public synchronized List<SchemeSeal> extrasFor(
|
|
final long blockNumber, final Hash onchainBlockHash) {
|
|
final Entry entry = byHash.get(onchainBlockHash);
|
|
if (entry == null || entry.blockNumber != blockNumber) {
|
|
return List.of();
|
|
@@ -606,26 +855,30 @@ public final class PqSealCache {
|
|
private void prune() {
|
|
final Iterator<Map.Entry<Hash, Entry>> it = byHash.entrySet().iterator();
|
|
while (it.hasNext()) {
|
|
final Map.Entry<Hash, Entry> e = it.next();
|
|
if (highestSeen - e.getValue().blockNumber > HEIGHT_WINDOW) {
|
|
it.remove();
|
|
}
|
|
}
|
|
while (byHash.size() > MAX_ENTRIES) {
|
|
final Iterator<Hash> oldest = byHash.keySet().iterator();
|
|
oldest.next();
|
|
oldest.remove();
|
|
}
|
|
}
|
|
|
|
private static final class Entry {
|
|
private final long blockNumber;
|
|
private final Map<Integer, FalconSeal> seals = new LinkedHashMap<>();
|
|
/** AERE ANCHOR V2: heard extra (non-Falcon) scheme seals, keyed (index << 8 | wire). */
|
|
private final Map<Long, SchemeSeal> extras = new LinkedHashMap<>();
|
|
+ /** D-454 family: the indices (and the extra-seal keys) whose held seal VERIFIED. */
|
|
+ private final Set<Integer> verified = new HashSet<>();
|
|
+
|
|
+ private final Set<Long> extrasVerified = new HashSet<>();
|
|
|
|
private Entry(final long blockNumber) {
|
|
this.blockNumber = blockNumber;
|
|
}
|
|
}
|
|
}
|
|
diff --git a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftBlockCreator.java b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftBlockCreator.java
|
|
index e43afaca83..ff1a09cf83 100644
|
|
--- a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftBlockCreator.java
|
|
+++ b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftBlockCreator.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-bft-prevrandao-block-creation.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.consensus.common.bft.blockcreation;
|
|
|
|
import org.hyperledger.besu.config.BftConfigOptions;
|
|
import org.hyperledger.besu.consensus.common.ForksSchedule;
|
|
import org.hyperledger.besu.consensus.common.bft.BftBlockHeaderFunctions;
|
|
import org.hyperledger.besu.consensus.common.bft.BftExtraDataCodec;
|
|
import org.hyperledger.besu.consensus.common.bft.BftHelpers;
|
|
import org.hyperledger.besu.consensus.common.bft.BftProtocolSchedule;
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.ethereum.ProtocolContext;
|
|
import org.hyperledger.besu.ethereum.blockcreation.AbstractBlockCreator;
|
|
import org.hyperledger.besu.ethereum.core.BlockHeader;
|
|
import org.hyperledger.besu.ethereum.core.BlockHeaderBuilder;
|
|
import org.hyperledger.besu.ethereum.core.MiningConfiguration;
|
|
import org.hyperledger.besu.ethereum.core.SealableBlockHeader;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
|
|
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
|
|
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
|
|
import org.hyperledger.besu.ethereum.mainnet.ProtocolSpec;
|
|
diff --git a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftMiningCoordinator.java b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftMiningCoordinator.java
|
|
index da4eeecd8c..fa9f480ed1 100644
|
|
--- a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftMiningCoordinator.java
|
|
+++ b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/blockcreation/BftMiningCoordinator.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-bft-coordinator-lifecycle-sync.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.consensus.common.bft.blockcreation;
|
|
|
|
import org.hyperledger.besu.consensus.common.bft.BftEventQueue;
|
|
import org.hyperledger.besu.consensus.common.bft.BftExecutors;
|
|
import org.hyperledger.besu.consensus.common.bft.BftProcessor;
|
|
import org.hyperledger.besu.consensus.common.bft.events.NewChainHead;
|
|
import org.hyperledger.besu.consensus.common.bft.statemachine.BftEventHandler;
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.datatypes.Wei;
|
|
import org.hyperledger.besu.ethereum.blockcreation.MiningCoordinator;
|
|
import org.hyperledger.besu.ethereum.chain.BlockAddedEvent;
|
|
import org.hyperledger.besu.ethereum.chain.BlockAddedObserver;
|
|
import org.hyperledger.besu.ethereum.chain.Blockchain;
|
|
import org.hyperledger.besu.ethereum.core.Block;
|
|
import org.hyperledger.besu.ethereum.core.BlockHeader;
|
|
import org.hyperledger.besu.ethereum.core.Transaction;
|
|
import org.hyperledger.besu.ethereum.eth.sync.state.SyncState;
|
|
import org.hyperledger.besu.plugin.services.BesuEvents;
|
|
|
|
diff --git a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/messagewrappers/BftMessage.java b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/messagewrappers/BftMessage.java
|
|
index e52c52905e..c99dfc45a5 100644
|
|
--- a/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/messagewrappers/BftMessage.java
|
|
+++ b/consensus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/messagewrappers/BftMessage.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-bft-bounded-message-lists.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.consensus.common.bft.messagewrappers;
|
|
|
|
import org.hyperledger.besu.consensus.common.bft.ConsensusRoundIdentifier;
|
|
import org.hyperledger.besu.consensus.common.bft.payload.Authored;
|
|
import org.hyperledger.besu.consensus.common.bft.payload.Payload;
|
|
import org.hyperledger.besu.consensus.common.bft.payload.RoundSpecific;
|
|
import org.hyperledger.besu.consensus.common.bft.payload.SignedData;
|
|
import org.hyperledger.besu.crypto.SECPSignature;
|
|
import org.hyperledger.besu.crypto.SignatureAlgorithmFactory;
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.ethereum.rlp.BytesValueRLPOutput;
|
|
import org.hyperledger.besu.ethereum.rlp.RLPInput;
|
|
|
|
import java.util.Objects;
|
|
import java.util.StringJoiner;
|
|
import java.util.function.Function;
|
|
|
|
import org.apache.tuweni.bytes.Bytes;
|
|
|
|
diff --git a/consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AereD454SealCacheTest.java b/consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AereD454SealCacheTest.java
|
|
new file mode 100644
|
|
index 0000000000..ac0f0ea657
|
|
--- /dev/null
|
|
+++ b/consensus/common/src/test/java/org/hyperledger/besu/consensus/common/bft/AereD454SealCacheTest.java
|
|
@@ -0,0 +1,434 @@
|
|
+/*
|
|
+ * Copyright contributors to Aere Network.
|
|
+ *
|
|
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
|
|
+ * compliance with the License. You may obtain a copy of the License at
|
|
+ *
|
|
+ * http://www.apache.org/licenses/LICENSE-2.0
|
|
+ *
|
|
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is
|
|
+ * distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
+ * implied. See the License for the specific language governing permissions and limitations under the
|
|
+ * License.
|
|
+ *
|
|
+ * SPDX-License-Identifier: Apache-2.0
|
|
+ */
|
|
+package org.hyperledger.besu.consensus.common.bft;
|
|
+
|
|
+import static org.assertj.core.api.Assertions.assertThat;
|
|
+import static org.mockito.ArgumentMatchers.any;
|
|
+import static org.mockito.Mockito.mock;
|
|
+import static org.mockito.Mockito.when;
|
|
+import static org.mockito.Mockito.withSettings;
|
|
+
|
|
+import org.hyperledger.besu.consensus.common.bft.blockcreation.PqAnchorProducer;
|
|
+import org.hyperledger.besu.consensus.common.validator.ValidatorProvider;
|
|
+import org.hyperledger.besu.crypto.SecureRandomProvider;
|
|
+import org.hyperledger.besu.datatypes.Address;
|
|
+import org.hyperledger.besu.ethereum.ProtocolContext;
|
|
+import org.hyperledger.besu.ethereum.core.BlockHeader;
|
|
+import org.hyperledger.besu.ethereum.core.BlockHeaderTestFixture;
|
|
+
|
|
+import java.lang.reflect.Field;
|
|
+import java.nio.file.Files;
|
|
+import java.nio.file.Path;
|
|
+import java.security.SecureRandom;
|
|
+import java.util.ArrayList;
|
|
+import java.util.Arrays;
|
|
+import java.util.Collection;
|
|
+import java.util.Collections;
|
|
+import java.util.HashMap;
|
|
+import java.util.List;
|
|
+import java.util.Map;
|
|
+import java.util.Optional;
|
|
+import java.util.OptionalInt;
|
|
+
|
|
+import org.apache.tuweni.bytes.Bytes;
|
|
+import org.apache.tuweni.bytes.Bytes32;
|
|
+import org.bouncycastle.crypto.digests.KeccakDigest;
|
|
+import org.bouncycastle.pqc.crypto.falcon.FalconPrivateKeyParameters;
|
|
+import org.bouncycastle.pqc.crypto.falcon.FalconSigner;
|
|
+import org.junit.jupiter.api.AfterEach;
|
|
+import org.junit.jupiter.api.BeforeEach;
|
|
+import org.junit.jupiter.api.Test;
|
|
+import org.junit.jupiter.api.io.TempDir;
|
|
+import org.mockito.quality.Strictness;
|
|
+
|
|
+/**
|
|
+ * AERE D-454 FAMILY (adversarial review of 2026-09-30): the store of HEARD seals no longer lets an
|
|
+ * unverified seal hold an index against the genuine one.
|
|
+ *
|
|
+ * <p>The defect, measured by this class on the code before the repair: {@code PqSealCache.record} kept
|
|
+ * the FIRST seal heard for an index, verifying nothing, and it is fed by paths that judge nothing (the
|
|
+ * D-339 path keeps the seal of any Commit heard before the proposal; the late-seal salvage keeps any
|
|
+ * validator's late Commit; below the commit enforcement no Commit's post-quantum seal is judged). So a
|
|
+ * garbage seal naming A's index, heard before A's own (below H_fin any validator can send it; from
|
|
+ * H_fin a copy of A's public commit seal in anyone's envelope is "A's"), kept A's genuine seal out of
|
|
+ * the certificate this node builds, and at the parent of an anchor enough of them leave no proposer
|
|
+ * able to reach K. Same for the extra scheme seals (SLH-DSA) of a v2 certificate.
|
|
+ *
|
|
+ * <p>The fixture is the REAL one of {@code PqAnchorV2ProducerTest}: the genesis-anchored Falcon registry
|
|
+ * loaded by {@code FalconSealSupport}, a hybrid-1 registry with SLH-DSA-SHA2-128s probe keys loaded
|
|
+ * through the system configuration, and the PRODUCER itself ({@code PqAnchorProducer.apply}) as the
|
|
+ * judge of the outcome: the question is not what the store holds, it is whether this node can still
|
|
+ * build the certificate. The judge of heard seals is armed by {@code enablePersistence}, the one
|
|
+ * production entry point reached with the anchor armed.
|
|
+ *
|
|
+ * <p>Six proofs, each red under a named planting (consensus-pqc/d412-reparatie/cap-decis-besu/
|
|
+ * planteaza-d454.py, harness d454-roundstate.sh): three are the defect (red on the code before), two
|
|
+ * say that nothing else moved (a verified seal is never displaced; without the judge the first seal
|
|
+ * still wins), one that two unverified seals do not trade places.
|
|
+ *
|
|
+ * <p>BOUND TO H_FIN (second review of D-454, 2026-10-01): the judge governs a height only from H_fin
|
|
+ * ({@code AerePqFinality.PROPERTY}, AIP-22 stage 4). The six proofs above run from H_fin (H_fin is set
|
|
+ * to the registry's height, so every parent here is governed); three more say that BELOW H_fin the
|
|
+ * store is the one of today, with the judge armed by the production entry point: the first Falcon
|
|
+ * seal stays, the first SLH-DSA seal stays, and a restore displaces nothing. They are green on the
|
|
+ * code before the repair (that is today's behaviour) and red under the planting
|
|
+ * "judecator-armat-sub-hfin". H_fin is set directly on AerePqFinality: the store reads only {@code
|
|
+ * AerePqFinality.active}; the startup coherence of identity and keys (AERE-PQC-FIN-01) is not this
|
|
+ * class's question.
|
|
+ */
|
|
+public class AereD454SealCacheTest {
|
|
+
|
|
+ private static final long CHAIN_ID = 220_879L;
|
|
+ private static final long H = 1_000L;
|
|
+ /** H_fin for the six proofs from H_fin: every parent of this class is at or above it. */
|
|
+ private static final long H_FIN = H;
|
|
+ /** H_fin for the three proofs below H_fin: above every parent of this class. */
|
|
+ private static final long H_FIN_ABOVE = 5_000L;
|
|
+ private static final long V2 = 1_005L;
|
|
+ private static final int N = 7;
|
|
+ private static final int K = 3;
|
|
+ private static final String SLH = "slh-dsa-sha2-128s";
|
|
+
|
|
+ @TempDir private Path tmp;
|
|
+
|
|
+ private final List<FalconPrivateKeyParameters> falconKeys = new ArrayList<>();
|
|
+ private final List<SealScheme.GeneratedPair> slhKeys = new ArrayList<>();
|
|
+ private final List<Address> validators = new ArrayList<>();
|
|
+ private final SecureRandom random = SecureRandomProvider.createSecureRandom();
|
|
+ private Path dataDirectory;
|
|
+
|
|
+ @BeforeEach
|
|
+ public void setUp() throws Exception {
|
|
+ final KeccakDigest kd = new KeccakDigest(256);
|
|
+ final StringBuilder manifest = new StringBuilder();
|
|
+ manifest
|
|
+ .append("{\"config\":{\"aereFalconRegistry\":{")
|
|
+ .append(PqV2Fixture.manifestHeader(N, CHAIN_ID, H));
|
|
+ for (int i = 0; i < N; i++) {
|
|
+ falconKeys.add(PqV2Fixture.privateKey(i));
|
|
+ validators.add(PqV2Fixture.address(i));
|
|
+ final byte[] row = PqV2Fixture.anchorPreimageRow(i);
|
|
+ kd.update(row, 0, row.length);
|
|
+ manifest.append(',').append(PqV2Fixture.manifestEntry(i, N, CHAIN_ID, H));
|
|
+ }
|
|
+ manifest
|
|
+ .append("}},\"alloc\":{\"0000000000000000000000000000000000000fa1\":{\"storage\":{\"0x")
|
|
+ .append("0".repeat(64))
|
|
+ .append("\":\"0x");
|
|
+ final byte[] anchoredHash = new byte[32];
|
|
+ kd.doFinal(anchoredHash, 0);
|
|
+ manifest.append(Bytes.wrap(anchoredHash).toUnprefixedHexString()).append("\"}}}}");
|
|
+ final Path genesis = tmp.resolve("genesis-registry.json");
|
|
+ Files.writeString(genesis, manifest.toString());
|
|
+ System.setProperty("aere.falcon.genesis", genesis.toAbsolutePath().toString());
|
|
+ resetFalconSingleton();
|
|
+ PqSealCache.instance().disablePersistence();
|
|
+ PqSealCache.instance().clear();
|
|
+ armHybrid("0:falcon-512+" + SLH);
|
|
+ PqAnchorProducer.useConfigForTesting(
|
|
+ new PqAnchorConfig(
|
|
+ CHAIN_ID, H, Map.of(H, K), OptionalInt.empty(), false, OptionalInt.of(K),
|
|
+ OptionalInt.empty(), V2));
|
|
+ dataDirectory = Files.createDirectories(tmp.resolve("besu-data"));
|
|
+ // the anchor armed on this node: the production entry point, which also arms the judge
|
|
+ PqSealCache.instance().enablePersistence(dataDirectory, CHAIN_ID);
|
|
+ // and H_fin reached: the judge governs every parent of the six proofs from H_fin
|
|
+ setFinalityBlock(H_FIN);
|
|
+ }
|
|
+
|
|
+ @AfterEach
|
|
+ public void tearDown() throws Exception {
|
|
+ AerePqFinality.uninstall();
|
|
+ PqSealCache.instance().disablePersistence();
|
|
+ PqSealCache.instance().clear();
|
|
+ System.clearProperty("aere.falcon.genesis");
|
|
+ System.clearProperty(HybridSealSupport.PROPERTY_SCHEDULE);
|
|
+ System.clearProperty(HybridSealSupport.PROPERTY_REGISTRY);
|
|
+ HybridSealSupport.resetForTesting();
|
|
+ resetFalconSingleton();
|
|
+ PqAnchorProducer.useConfigForTesting(null);
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------ the defect
|
|
+
|
|
+ /** The certificate this node can build still carries A (index 0), whose genuine seal came second. */
|
|
+ @Test
|
|
+ public void aGarbageFalconSealHeardFirstNoLongerKeepsTheGenuineOneOut() {
|
|
+ final BlockHeader parent = parentOf(V2 - 2L); // a v1 anchor parent: Falcon only
|
|
+ hearGarbageFalcon(parent, (byte) 0x31);
|
|
+ hearFalcon(parent, 0, K);
|
|
+ assertThat(heldFalcon(parent, 0))
|
|
+ .describedAs("index 0 holds A's genuine seal, not the garbage heard before it")
|
|
+ .isEqualTo(genuine.get(0));
|
|
+ final BftExtraData produced = PqAnchorProducer.apply(base(), parent, contextWith(validators));
|
|
+ assertThat(produced.getFalconSeals())
|
|
+ .describedAs("K verified seals, A's among them: the proposer is not short of A")
|
|
+ .hasSize(K)
|
|
+ .anyMatch(s -> s.getValidatorIndex() == 0);
|
|
+ }
|
|
+
|
|
+ /** The replacement reaches the file: a node restarted onto this head restores A's genuine seal. */
|
|
+ @Test
|
|
+ public void theFileHoldsTheGenuineSealThatDisplacedAGarbageOne() {
|
|
+ final BlockHeader head = parentOf(V2 - 2L);
|
|
+ hearFalcon(head, 1, K); // the others first
|
|
+ hearGarbageFalcon(head, (byte) 0x32);
|
|
+ hearFalcon(head, 0, 1); // A's genuine seal is the LAST change: same count, other bytes
|
|
+ PqSealCache.instance().clear(); // the restart: memory gone, the file stays
|
|
+ final int restored =
|
|
+ PqSealCache.instance()
|
|
+ .restoreFromDisk(head.getNumber(), head.getHash(), PqSignerRegistry.falconSealSupport());
|
|
+ assertThat(restored).describedAs("the K genuine seals come back from the file, A's included").isEqualTo(K);
|
|
+ assertThat(PqAnchorProducer.apply(base(), head, contextWith(validators)).getFalconSeals())
|
|
+ .hasSize(K)
|
|
+ .anyMatch(s -> s.getValidatorIndex() == 0);
|
|
+ }
|
|
+
|
|
+ /** The same rule for the extra scheme of a v2 certificate. */
|
|
+ @Test
|
|
+ public void aGarbageSlhDsaSealHeardFirstNoLongerKeepsTheGenuineOneOut() {
|
|
+ final BlockHeader parent = parentOf(V2 + 9L); // a v2 anchor parent: Falcon and SLH-DSA
|
|
+ hearFalcon(parent, 0, K);
|
|
+ PqSealCache.instance()
|
|
+ .recordExtras(
|
|
+ parent.getNumber(),
|
|
+ parent.getHash(),
|
|
+ List.of(new SchemeSeal(SealSchemes.SLH_DSA_128S.wireId(), 0, garbage(7856, (byte) 0x33))));
|
|
+ hearSlh(parent, 0, K);
|
|
+ final List<SchemeSeal> tagged =
|
|
+ PqAnchorProducer.apply(base(), parent, contextWith(validators)).getHybridSeals();
|
|
+ assertThat(PqAnchorV2.distinctValidatorsWith(tagged, SealSchemes.SLH_DSA_128S.wireId()))
|
|
+ .describedAs("K SLH-DSA seals, A's genuine one among them")
|
|
+ .isEqualTo(K);
|
|
+ assertThat(tagged)
|
|
+ .anyMatch(
|
|
+ s -> s.getSchemeWireId() == SealSchemes.SLH_DSA_128S.wireId() && s.getValidatorIndex() == 0);
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------ nothing else moved
|
|
+
|
|
+ /** A verified seal is never displaced, by anything heard after it. */
|
|
+ @Test
|
|
+ public void aGarbageFalconSealAfterTheGenuineOneDisplacesNothing() {
|
|
+ final BlockHeader parent = parentOf(V2 - 2L);
|
|
+ hearFalcon(parent, 0, K);
|
|
+ hearGarbageFalcon(parent, (byte) 0x34);
|
|
+ assertThat(heldFalcon(parent, 0)).isEqualTo(genuine.get(0));
|
|
+ assertThat(PqAnchorProducer.apply(base(), parent, contextWith(validators)).getFalconSeals())
|
|
+ .hasSize(K)
|
|
+ .anyMatch(s -> s.getValidatorIndex() == 0);
|
|
+ }
|
|
+
|
|
+ /** Without the judge (a node without the anchor armed) the store is the one of before: first wins. */
|
|
+ @Test
|
|
+ public void withoutTheJudgeTheFirstSealStaysAsBefore() {
|
|
+ PqSealCache.instance().disablePersistence();
|
|
+ final BlockHeader parent = parentOf(V2 - 2L);
|
|
+ hearGarbageFalcon(parent, (byte) 0x35);
|
|
+ hearFalcon(parent, 0, K);
|
|
+ assertThat(heldFalcon(parent, 0))
|
|
+ .describedAs("unjudged, the first seal of index 0 stays")
|
|
+ .isEqualTo(garbage(666, (byte) 0x35));
|
|
+ }
|
|
+
|
|
+ /** Two seals neither of which verifies do not trade places. */
|
|
+ @Test
|
|
+ public void anUnverifiedSealNeverDisplacesAnother() {
|
|
+ final BlockHeader parent = parentOf(V2 - 2L);
|
|
+ hearGarbageFalcon(parent, (byte) 0x36);
|
|
+ hearGarbageFalcon(parent, (byte) 0x37);
|
|
+ assertThat(heldFalcon(parent, 0)).isEqualTo(garbage(666, (byte) 0x36));
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------ below H_fin: today
|
|
+
|
|
+ /** Below H_fin, with the judge armed, the first Falcon seal of an index stays: judged by nobody. */
|
|
+ @Test
|
|
+ public void belowHFinAGarbageFalconSealHeardFirstStaysAsToday() throws Exception {
|
|
+ setFinalityBlock(H_FIN_ABOVE);
|
|
+ final BlockHeader parent = parentOf(V2 - 2L);
|
|
+ hearGarbageFalcon(parent, (byte) 0x38);
|
|
+ hearFalcon(parent, 0, K);
|
|
+ assertThat(heldFalcon(parent, 0))
|
|
+ .describedAs("below H_fin the first seal of index 0 stays, as in today's binary")
|
|
+ .isEqualTo(garbage(666, (byte) 0x38));
|
|
+ assertThat(heldFalcon(parent, 1)).isEqualTo(genuine.get(1));
|
|
+ }
|
|
+
|
|
+ /** Below H_fin the first SLH-DSA seal of an index stays too. */
|
|
+ @Test
|
|
+ public void belowHFinAGarbageSlhDsaSealHeardFirstStaysAsToday() throws Exception {
|
|
+ setFinalityBlock(H_FIN_ABOVE);
|
|
+ final BlockHeader parent = parentOf(V2 + 9L);
|
|
+ PqSealCache.instance()
|
|
+ .recordExtras(
|
|
+ parent.getNumber(),
|
|
+ parent.getHash(),
|
|
+ List.of(new SchemeSeal(SealSchemes.SLH_DSA_128S.wireId(), 0, garbage(7856, (byte) 0x39))));
|
|
+ hearSlh(parent, 0, K);
|
|
+ final List<SchemeSeal> held = PqSealCache.instance().extrasFor(parent.getNumber(), parent.getHash());
|
|
+ assertThat(held).describedAs("one SLH-DSA slot per index, K indices").hasSize(K);
|
|
+ assertThat(held)
|
|
+ .filteredOn(s -> s.getValidatorIndex() == 0)
|
|
+ .extracting(SchemeSeal::getSignature)
|
|
+ .describedAs("below H_fin the first SLH-DSA seal of index 0 stays, as in today's binary")
|
|
+ .containsExactly(garbage(7856, (byte) 0x39));
|
|
+ }
|
|
+
|
|
+ /** Below H_fin a restore from the head file displaces nothing held (putIfAbsent, as today). */
|
|
+ @Test
|
|
+ public void belowHFinTheRestoreDisplacesNothingAsToday() throws Exception {
|
|
+ setFinalityBlock(H_FIN_ABOVE);
|
|
+ final BlockHeader head = parentOf(V2 - 2L);
|
|
+ hearGarbageFalcon(head, (byte) 0x3a); // index 0 holds garbage in memory
|
|
+ final List<FalconSeal> onFile = new ArrayList<>();
|
|
+ for (int i = 0; i < K; i++) {
|
|
+ onFile.add(new FalconSeal(i, genuineFalcon(head, i)));
|
|
+ }
|
|
+ // the head file holds the K genuine seals (as after a restart onto a head whose seals were written)
|
|
+ PqSealStore.writeAtomically(
|
|
+ PqSealCache.instance().persistenceFile(),
|
|
+ PqSealStore.encode(CHAIN_ID, head.getNumber(), head.getHash().getBytes(), onFile));
|
|
+ final int restored =
|
|
+ PqSealCache.instance()
|
|
+ .restoreFromDisk(head.getNumber(), head.getHash(), PqSignerRegistry.falconSealSupport());
|
|
+ assertThat(restored).describedAs("the K seals on file verify, as today").isEqualTo(K);
|
|
+ assertThat(heldFalcon(head, 0))
|
|
+ .describedAs("below H_fin the restored seal does not displace the one held, as in today's binary")
|
|
+ .isEqualTo(garbage(666, (byte) 0x3a));
|
|
+ assertThat(heldFalcon(head, 1)).isEqualTo(onFile.get(1).getSignature());
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------ helpers
|
|
+
|
|
+ /** Sets H_fin directly: the store reads only AerePqFinality.active (see the class comment). */
|
|
+ private static void setFinalityBlock(final long hFin) throws Exception {
|
|
+ final Field f = AerePqFinality.class.getDeclaredField("finalityBlock");
|
|
+ f.setAccessible(true);
|
|
+ f.setLong(null, hFin);
|
|
+ assertThat(AerePqFinality.finalityBlock()).isEqualTo(hFin);
|
|
+ }
|
|
+
|
|
+ private void armHybrid(final String schedule) throws Exception {
|
|
+ if (slhKeys.isEmpty()) {
|
|
+ for (int i = 0; i < N; i++) {
|
|
+ slhKeys.add(SealSchemes.SLH_DSA_128S.generate(random));
|
|
+ }
|
|
+ }
|
|
+ final StringBuilder p = new StringBuilder();
|
|
+ p.append("formatVersion=hybrid-1\nchainId=").append(CHAIN_ID).append("\ncount=").append(N).append('\n');
|
|
+ for (int i = 0; i < N; i++) {
|
|
+ p.append(i).append(".addr=").append(validators.get(i).toHexString()).append('\n');
|
|
+ p.append(i)
|
|
+ .append(".key.")
|
|
+ .append(SLH)
|
|
+ .append('=')
|
|
+ .append(Bytes.wrap(slhKeys.get(i).publicRegistryForm()).toHexString())
|
|
+ .append('\n');
|
|
+ }
|
|
+ final Path reg = tmp.resolve("hybrid-registry.properties");
|
|
+ Files.writeString(reg, p.toString());
|
|
+ System.setProperty(HybridSealSupport.PROPERTY_SCHEDULE, schedule);
|
|
+ System.setProperty(HybridSealSupport.PROPERTY_REGISTRY, reg.toAbsolutePath().toString());
|
|
+ HybridSealSupport.resetForTesting();
|
|
+ assertThat(HybridSealSupport.instance().registry()).describedAs("hybrid registry loaded").isPresent();
|
|
+ }
|
|
+
|
|
+ private static BlockHeader parentOf(final long number) {
|
|
+ return new BlockHeaderTestFixture().number(number).buildHeader();
|
|
+ }
|
|
+
|
|
+ private BftExtraData base() {
|
|
+ return new BftExtraData(
|
|
+ Bytes32.ZERO, Collections.emptyList(), Optional.empty(), 0, validators, Collections.emptyList());
|
|
+ }
|
|
+
|
|
+ private static Bytes32 message(final BlockHeader parent) {
|
|
+ return PqAnchor.commitMessage(CHAIN_ID, parent.getNumber(), parent.getHash().getBytes());
|
|
+ }
|
|
+
|
|
+ private final Map<Integer, Bytes> genuine = new HashMap<>();
|
|
+
|
|
+ private Bytes genuineFalcon(final BlockHeader parent, final int i) {
|
|
+ final FalconSigner signer = new FalconSigner();
|
|
+ signer.init(true, falconKeys.get(i));
|
|
+ return Bytes.wrap(signer.generateSignature(message(parent).toArray()));
|
|
+ }
|
|
+
|
|
+ /** Records the genuine Falcon seals of indices [from, to); a Falcon signature is randomised, so the
|
|
+ * seal of an index is the one recorded here ({@link #genuine}, one parent per test). */
|
|
+ private void hearFalcon(final BlockHeader parent, final int from, final int to) {
|
|
+ final List<FalconSeal> heard = new ArrayList<>();
|
|
+ for (int i = from; i < to; i++) {
|
|
+ final Bytes sig = genuineFalcon(parent, i);
|
|
+ genuine.put(i, sig);
|
|
+ heard.add(new FalconSeal(i, sig));
|
|
+ }
|
|
+ PqSealCache.instance().record(parent.getNumber(), parent.getHash(), heard);
|
|
+ }
|
|
+
|
|
+ private void hearGarbageFalcon(final BlockHeader parent, final byte fill) {
|
|
+ PqSealCache.instance()
|
|
+ .record(parent.getNumber(), parent.getHash(), List.of(new FalconSeal(0, garbage(666, fill))));
|
|
+ }
|
|
+
|
|
+ private void hearSlh(final BlockHeader parent, final int from, final int to) {
|
|
+ final List<SchemeSeal> extras = new ArrayList<>();
|
|
+ for (int i = from; i < to; i++) {
|
|
+ extras.add(
|
|
+ new SchemeSeal(
|
|
+ SealSchemes.SLH_DSA_128S.wireId(),
|
|
+ i,
|
|
+ Bytes.wrap(
|
|
+ SealSchemes.SLH_DSA_128S
|
|
+ .sign(slhKeys.get(i).privateKey(), message(parent).toArray())
|
|
+ .orElseThrow())));
|
|
+ }
|
|
+ PqSealCache.instance().recordExtras(parent.getNumber(), parent.getHash(), extras);
|
|
+ }
|
|
+
|
|
+ private static Bytes heldFalcon(final BlockHeader parent, final int index) {
|
|
+ return PqSealCache.instance().sealsFor(parent.getNumber(), parent.getHash()).stream()
|
|
+ .filter(s -> s.getValidatorIndex() == index)
|
|
+ .map(FalconSeal::getSignature)
|
|
+ .findFirst()
|
|
+ .orElse(Bytes.EMPTY);
|
|
+ }
|
|
+
|
|
+ private static Bytes garbage(final int length, final byte fill) {
|
|
+ final byte[] b = new byte[length];
|
|
+ Arrays.fill(b, fill);
|
|
+ return Bytes.wrap(b);
|
|
+ }
|
|
+
|
|
+ private static ProtocolContext contextWith(final Collection<Address> vs) {
|
|
+ final ValidatorProvider validatorProvider =
|
|
+ mock(ValidatorProvider.class, withSettings().strictness(Strictness.LENIENT));
|
|
+ when(validatorProvider.getValidatorsForBlock(any())).thenReturn(vs);
|
|
+ when(validatorProvider.getValidatorsAfterBlock(any())).thenReturn(vs);
|
|
+ final BftContext bftContext =
|
|
+ mock(BftContext.class, withSettings().strictness(Strictness.LENIENT));
|
|
+ when(bftContext.getValidatorProvider()).thenReturn(validatorProvider);
|
|
+ when(bftContext.as(any())).thenReturn(bftContext);
|
|
+ return new ProtocolContext.Builder().withConsensusContext(bftContext).build();
|
|
+ }
|
|
+
|
|
+ private static void resetFalconSingleton() throws Exception {
|
|
+ final Field f = FalconSealSupport.class.getDeclaredField("instance");
|
|
+ f.setAccessible(true);
|
|
+ f.set(null, null);
|
|
+ }
|
|
+}
|
|
diff --git a/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/Proposal.java b/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/Proposal.java
|
|
index 64ece7de19..7d9ee23bb0 100644
|
|
--- a/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/Proposal.java
|
|
+++ b/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/Proposal.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-bft-bounded-message-lists.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.consensus.qbft.core.messagewrappers;
|
|
|
|
import org.hyperledger.besu.consensus.common.bft.messagewrappers.BftMessage;
|
|
import org.hyperledger.besu.consensus.common.bft.payload.SignedData;
|
|
import org.hyperledger.besu.consensus.qbft.core.payload.PreparePayload;
|
|
import org.hyperledger.besu.consensus.qbft.core.payload.ProposalPayload;
|
|
import org.hyperledger.besu.consensus.qbft.core.payload.RoundChangePayload;
|
|
import org.hyperledger.besu.consensus.qbft.core.types.QbftBlock;
|
|
import org.hyperledger.besu.consensus.qbft.core.types.QbftBlockCodec;
|
|
import org.hyperledger.besu.ethereum.mainnet.block.access.list.BlockAccessList;
|
|
import org.hyperledger.besu.ethereum.rlp.BytesValueRLPOutput;
|
|
import org.hyperledger.besu.ethereum.rlp.RLP;
|
|
import org.hyperledger.besu.ethereum.rlp.RLPInput;
|
|
|
|
import java.util.List;
|
|
import java.util.Optional;
|
|
|
|
import org.apache.tuweni.bytes.Bytes;
|
|
|
|
diff --git a/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/RoundChange.java b/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/RoundChange.java
|
|
index 5d7cfce849..35efb6ae75 100644
|
|
--- a/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/RoundChange.java
|
|
+++ b/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/messagewrappers/RoundChange.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-bft-bounded-message-lists.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.consensus.qbft.core.messagewrappers;
|
|
|
|
import org.hyperledger.besu.consensus.common.bft.messagewrappers.BftMessage;
|
|
import org.hyperledger.besu.consensus.common.bft.payload.SignedData;
|
|
import org.hyperledger.besu.consensus.qbft.core.payload.PreparePayload;
|
|
import org.hyperledger.besu.consensus.qbft.core.payload.PreparedRoundMetadata;
|
|
import org.hyperledger.besu.consensus.qbft.core.payload.RoundChangePayload;
|
|
import org.hyperledger.besu.consensus.qbft.core.types.QbftBlock;
|
|
import org.hyperledger.besu.consensus.qbft.core.types.QbftBlockCodec;
|
|
import org.hyperledger.besu.ethereum.core.encoding.BlockAccessListDecoder;
|
|
import org.hyperledger.besu.ethereum.mainnet.block.access.list.BlockAccessList;
|
|
import org.hyperledger.besu.ethereum.rlp.BytesValueRLPOutput;
|
|
import org.hyperledger.besu.ethereum.rlp.RLP;
|
|
import org.hyperledger.besu.ethereum.rlp.RLPInput;
|
|
|
|
import java.util.List;
|
|
import java.util.Optional;
|
|
|
|
import org.apache.tuweni.bytes.Bytes;
|
|
diff --git a/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454PendingCommits.java b/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454PendingCommits.java
|
|
new file mode 100644
|
|
index 0000000000..fe01ac4a7d
|
|
--- /dev/null
|
|
+++ b/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454PendingCommits.java
|
|
@@ -0,0 +1,203 @@
|
|
+/*
|
|
+ * Copyright contributors to the Aere Network.
|
|
+ *
|
|
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
+ * the License. You may obtain a copy of the License at
|
|
+ *
|
|
+ * http://www.apache.org/licenses/LICENSE-2.0
|
|
+ *
|
|
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
+ * specific language governing permissions and limitations under the License.
|
|
+ *
|
|
+ * SPDX-License-Identifier: Apache-2.0
|
|
+ */
|
|
+package org.hyperledger.besu.consensus.qbft.core.statemachine;
|
|
+
|
|
+import org.hyperledger.besu.consensus.common.bft.AerePqFinality;
|
|
+import org.hyperledger.besu.consensus.qbft.core.messagewrappers.Commit;
|
|
+import org.hyperledger.besu.crypto.SECPSignature;
|
|
+import org.hyperledger.besu.crypto.SignatureAlgorithmFactory;
|
|
+import org.hyperledger.besu.datatypes.Address;
|
|
+import org.hyperledger.besu.datatypes.Hash;
|
|
+import org.hyperledger.besu.ethereum.core.Util;
|
|
+import org.hyperledger.besu.ethereum.rlp.BytesValueRLPInput;
|
|
+import org.hyperledger.besu.ethereum.rlp.RLPInput;
|
|
+
|
|
+import java.util.HashMap;
|
|
+import java.util.LinkedHashMap;
|
|
+import java.util.Map;
|
|
+import java.util.Optional;
|
|
+import java.util.function.Predicate;
|
|
+
|
|
+import org.apache.tuweni.bytes.Bytes;
|
|
+
|
|
+/**
|
|
+ * AERE D-454 (2026-09-30): the Commit messages a {@link RoundState} hears BEFORE its proposal, from
|
|
+ * H_fin (AIP-22 stage 4).
|
|
+ *
|
|
+ * <p>WHY. From H_fin the author of a Commit is the id its post-quantum seal index names, when that
|
|
+ * id is the signer of the commit seal; the Falcon seal itself is NOT verified to give the author
|
|
+ * ({@code AerePqFinality.authorFromSeal}), and the envelope's ECDSA key is not read. So a copy of
|
|
+ * validator A's commit seal in anyone's envelope, with a garbage Falcon seal, is "A's". The
|
|
+ * upstream RoundState kept the first Commit of each author before the proposal
|
|
+ * ({@code putIfAbsent}, no validation possible without the proposal) and removed the invalid ones at
|
|
+ * the proposal: the copy took A's place, A's own message was dropped, the copy was then removed, and
|
|
+ * A's vote was missing from the round for as long as gossip deduplication keeps A's message from
|
|
+ * coming back. Measured by the probe AereD412RoundStateProbe (consensus-pqc/d412-reparatie).
|
|
+ *
|
|
+ * <p>WHAT IT DOES. Each message waits here keyed by its bytes (a message heard twice waits once),
|
|
+ * within a quota per (claimed author, envelope class): SELF when the envelope is signed by the same
|
|
+ * ECDSA key as the commit seal (what Besu and Nethermind emit), FOREIGN otherwise. A copy made by
|
|
+ * anyone other than A is FOREIGN, so copies can never fill the quota A's own message needs; and a
|
|
+ * FOREIGN message may still be an honest vote (from H_fin the envelope does not decide the author),
|
|
+ * so it is not refused, only kept apart. When the proposal arrives, {@link #release} validates the
|
|
+ * waiting messages, SELF first, each class in arrival order, and a message takes its author's slot
|
|
+ * only if it VALIDATES and the slot is still free; a message whose author already holds a slot is
|
|
+ * not validated at all. The quota is checked before the envelope is classified (two ECDSA
|
|
+ * recoveries), so once both classes of an author are full nothing more is recovered for it.
|
|
+ *
|
|
+ * <p>BOUNDS. At most {@code 2 * MAX_PER_CLASS} messages per claimed author, and the authors that
|
|
+ * reach a RoundState are the validators of its height (QbftController.processMessage refuses any
|
|
+ * other author, and from H_fin a Commit without a bound index has NO_AUTHOR, never in a set).
|
|
+ *
|
|
+ * <p>Below H_fin nothing waits here and RoundState is exactly the upstream one ({@link #governs}).
|
|
+ */
|
|
+final class AereD454PendingCommits {
|
|
+
|
|
+ /** Per (claimed author, envelope class): how many distinct messages may wait for the proposal. */
|
|
+ static final int MAX_PER_CLASS = 2;
|
|
+
|
|
+ /** The envelope is signed by the commit seal's signer. */
|
|
+ static final int SELF = 0;
|
|
+
|
|
+ /** Any other envelope (another key, or one that does not recover). */
|
|
+ static final int FOREIGN = 1;
|
|
+
|
|
+ private static final class Held {
|
|
+ private final Commit commit;
|
|
+ private final int envelope;
|
|
+
|
|
+ private Held(final Commit commit, final int envelope) {
|
|
+ this.commit = commit;
|
|
+ this.envelope = envelope;
|
|
+ }
|
|
+ }
|
|
+
|
|
+ private final Map<Hash, Held> byFingerprint = new LinkedHashMap<>();
|
|
+ private final Map<Address, int[]> perAuthor = new HashMap<>();
|
|
+ private long classifications;
|
|
+
|
|
+ /**
|
|
+ * Whether a Commit of this height waits here before the proposal: from H_fin only.
|
|
+ *
|
|
+ * @param height the height of the round
|
|
+ * @return true from H_fin
|
|
+ */
|
|
+ static boolean governs(final long height) {
|
|
+ return AerePqFinality.active(height);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * Keeps a Commit heard before the proposal, within its author's quota for its envelope class.
|
|
+ *
|
|
+ * @param commit the commit
|
|
+ * @return true when this message is newly kept; false when it was already kept or its quota is full
|
|
+ */
|
|
+ boolean hold(final Commit commit) {
|
|
+ final Address author = commit.getAuthor();
|
|
+ final int[] counted = perAuthor.get(author);
|
|
+ if (counted != null && counted[SELF] >= MAX_PER_CLASS && counted[FOREIGN] >= MAX_PER_CLASS) {
|
|
+ // both classes of this author are full: dropped before the two ECDSA recoveries
|
|
+ return false;
|
|
+ }
|
|
+ final Bytes signed = commit.getSignedPayload().encode();
|
|
+ final Hash fingerprint = Hash.hash(signed);
|
|
+ if (byFingerprint.containsKey(fingerprint)) {
|
|
+ return false;
|
|
+ }
|
|
+ final int envelope = envelopeOf(commit, signed);
|
|
+ classifications++;
|
|
+ final int[] quota = perAuthor.computeIfAbsent(author, a -> new int[2]);
|
|
+ if (quota[envelope] >= MAX_PER_CLASS) {
|
|
+ return false;
|
|
+ }
|
|
+ quota[envelope]++;
|
|
+ byFingerprint.put(fingerprint, new Held(commit, envelope));
|
|
+ return true;
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * The proposal arrived: every waiting message is judged, SELF first then FOREIGN, each class in
|
|
+ * arrival order, and takes its author's slot only when {@code validates} passes and the slot is
|
|
+ * free. Nothing waits afterwards.
|
|
+ *
|
|
+ * @param validates the round's Commit validation (MessageValidator.validateCommit)
|
|
+ * @param slots the round's counted commits, by author
|
|
+ */
|
|
+ void release(final Predicate<Commit> validates, final Map<Address, Commit> slots) {
|
|
+ for (final int envelope : new int[] {SELF, FOREIGN}) {
|
|
+ for (final Held held : byFingerprint.values()) {
|
|
+ if (held.envelope != envelope) {
|
|
+ continue;
|
|
+ }
|
|
+ final Address author = held.commit.getAuthor();
|
|
+ if (slots.containsKey(author)) {
|
|
+ continue;
|
|
+ }
|
|
+ if (validates.test(held.commit)) {
|
|
+ slots.put(author, held.commit);
|
|
+ }
|
|
+ }
|
|
+ }
|
|
+ byFingerprint.clear();
|
|
+ perAuthor.clear();
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * How many messages wait.
|
|
+ *
|
|
+ * @return the number of waiting messages
|
|
+ */
|
|
+ int size() {
|
|
+ return byFingerprint.size();
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * How many envelopes were classified (two ECDSA recoveries each), for the work bound.
|
|
+ *
|
|
+ * @return the number of classifications
|
|
+ */
|
|
+ long classifications() {
|
|
+ return classifications;
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * SELF when the envelope is signed by the same ECDSA key as the commit seal, FOREIGN otherwise
|
|
+ * (also when either does not recover).
|
|
+ *
|
|
+ * @param commit the commit
|
|
+ * @param signed its encoded signed payload
|
|
+ * @return {@link #SELF} or {@link #FOREIGN}
|
|
+ */
|
|
+ static int envelopeOf(final Commit commit, final Bytes signed) {
|
|
+ try {
|
|
+ final Optional<Address> sealSigner =
|
|
+ commit.getSignedPayload().getPayload().commitSealSigner();
|
|
+ if (sealSigner.isEmpty()) {
|
|
+ return FOREIGN;
|
|
+ }
|
|
+ final RLPInput in = new BytesValueRLPInput(signed, false);
|
|
+ in.enterList();
|
|
+ in.skipNext();
|
|
+ final SECPSignature envelope =
|
|
+ in.readBytes(SignatureAlgorithmFactory.getInstance()::decodeSignature);
|
|
+ final Address envelopeSigner =
|
|
+ Util.signatureToAddress(
|
|
+ envelope, commit.getSignedPayload().getPayload().hashForSignature());
|
|
+ return sealSigner.get().equals(envelopeSigner) ? SELF : FOREIGN;
|
|
+ } catch (final RuntimeException e) {
|
|
+ return FOREIGN;
|
|
+ }
|
|
+ }
|
|
+}
|
|
diff --git a/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/RoundState.java b/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/RoundState.java
|
|
index b7ba44b5a1..95e7fc7b3e 100644
|
|
--- a/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/RoundState.java
|
|
+++ b/consensus/qbft-core/src/main/java/org/hyperledger/besu/consensus/qbft/core/statemachine/RoundState.java
|
|
@@ -41,40 +41,43 @@ import java.util.stream.Collectors;
|
|
import org.slf4j.Logger;
|
|
import org.slf4j.LoggerFactory;
|
|
|
|
/**
|
|
* The Round state holds all the messages for a given round and tracks whether quorum has been
|
|
* reached for the round to be prepared or committed.
|
|
*/
|
|
public class RoundState {
|
|
private static final Logger LOG = LoggerFactory.getLogger(RoundState.class);
|
|
|
|
private final ConsensusRoundIdentifier roundIdentifier;
|
|
private final MessageValidator validator;
|
|
private final long quorum;
|
|
|
|
private Optional<Proposal> proposalMessage = Optional.empty();
|
|
|
|
// Must track the actual Prepare message, not just the sender, as these may need to be reused
|
|
// to send out in a PrepareCertificate.
|
|
private final Map<Address, Prepare> prepareMessages = new LinkedHashMap<>();
|
|
private final Map<Address, Commit> commitMessages = new LinkedHashMap<>();
|
|
+ // AERE D-454 (2026-09-30): from H_fin a Commit heard before the proposal waits here, keyed by its bytes, within a quota
|
|
+ // per (claimed author, envelope class); it takes its author's slot in commitMessages only once it validates.
|
|
+ private final AereD454PendingCommits aereD454Pending = new AereD454PendingCommits();
|
|
|
|
private boolean prepared = false;
|
|
private boolean committed = false;
|
|
|
|
/**
|
|
* Instantiates a new Round state.
|
|
*
|
|
* @param roundIdentifier the round identifier
|
|
* @param quorum the quorum
|
|
* @param validator the validator
|
|
*/
|
|
public RoundState(
|
|
final ConsensusRoundIdentifier roundIdentifier,
|
|
final int quorum,
|
|
final MessageValidator validator) {
|
|
this.roundIdentifier = roundIdentifier;
|
|
this.quorum = quorum;
|
|
this.validator = validator;
|
|
}
|
|
|
|
@@ -92,67 +95,85 @@ public class RoundState {
|
|
*
|
|
* @return the message validator
|
|
*/
|
|
public MessageValidator getValidator() {
|
|
return validator;
|
|
}
|
|
|
|
/**
|
|
* Sets proposed block.
|
|
*
|
|
* @param msg the Proposal payload msg
|
|
* @return the proposed block
|
|
*/
|
|
public boolean setProposedBlock(final Proposal msg) {
|
|
|
|
if (proposalMessage.isEmpty()) {
|
|
if (validator.validateProposal(msg)) {
|
|
proposalMessage = Optional.of(msg);
|
|
prepareMessages.entrySet().removeIf(e -> !validator.validatePrepare(e.getValue()));
|
|
commitMessages.entrySet().removeIf(e -> !validator.validateCommit(e.getValue()));
|
|
+ // AERE D-454: the commits kept aside before the proposal (from H_fin) are judged now, SELF-enveloped first; each
|
|
+ // takes its author's slot only if it validates. Below H_fin nothing is kept aside, so this calls nothing.
|
|
+ aereD454Pending.release(validator::validateCommit, commitMessages);
|
|
updateState();
|
|
return true;
|
|
}
|
|
}
|
|
|
|
return false;
|
|
}
|
|
|
|
/**
|
|
* Add prepare message.
|
|
*
|
|
* @param msg the msg
|
|
*/
|
|
public void addPrepareMessage(final Prepare msg) {
|
|
if (proposalMessage.isEmpty() || validator.validatePrepare(msg)) {
|
|
prepareMessages.putIfAbsent(msg.getAuthor(), msg);
|
|
LOG.trace("Round state added prepare message prepare={}", msg);
|
|
}
|
|
updateState();
|
|
}
|
|
|
|
/**
|
|
* Add commit message.
|
|
*
|
|
* @param msg the msg
|
|
*/
|
|
public void addCommitMessage(final Commit msg) {
|
|
+ // AERE D-454 (2026-09-30): from H_fin (AIP-22 stage 4) the author of a Commit is read from its seal index WITHOUT the
|
|
+ // Falcon check, so a copy of A's commit seal in anyone's envelope is "A's". Before the proposal it took A's slot here, A's
|
|
+ // own message was dropped by putIfAbsent, and the copy was removed at the proposal: A's vote was lost for as long as
|
|
+ // gossip deduplication keeps A's message from coming back. From H_fin a slot is taken only after validation. Below H_fin
|
|
+ // this method is exactly the upstream one.
|
|
+ if (AereD454PendingCommits.governs(roundIdentifier.getSequenceNumber())) {
|
|
+ if (proposalMessage.isEmpty()) {
|
|
+ aereD454Pending.hold(msg);
|
|
+ } else if (!commitMessages.containsKey(msg.getAuthor()) && validator.validateCommit(msg)) {
|
|
+ commitMessages.putIfAbsent(msg.getAuthor(), msg);
|
|
+ LOG.trace("Round state added commit message commit={}", msg);
|
|
+ }
|
|
+ updateState();
|
|
+ return;
|
|
+ }
|
|
if (proposalMessage.isEmpty() || validator.validateCommit(msg)) {
|
|
commitMessages.putIfAbsent(msg.getAuthor(), msg);
|
|
LOG.trace("Round state added commit message commit={}", msg);
|
|
}
|
|
|
|
updateState();
|
|
}
|
|
|
|
private void updateState() {
|
|
prepared = (prepareMessages.size() >= quorum) && proposalMessage.isPresent();
|
|
committed = (commitMessages.size() >= quorum) && proposalMessage.isPresent();
|
|
LOG.trace(
|
|
"Round state updated prepared={} committed={} preparedQuorum={}/{} committedQuorum={}/{}",
|
|
prepared,
|
|
committed,
|
|
prepareMessages.size(),
|
|
quorum,
|
|
commitMessages.size(),
|
|
quorum);
|
|
}
|
|
diff --git a/consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454RoundStateTest.java b/consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454RoundStateTest.java
|
|
new file mode 100644
|
|
index 0000000000..0201df057c
|
|
--- /dev/null
|
|
+++ b/consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/statemachine/AereD454RoundStateTest.java
|
|
@@ -0,0 +1,317 @@
|
|
+/*
|
|
+ * Copyright contributors to the Aere Network.
|
|
+ *
|
|
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
+ * the License. You may obtain a copy of the License at
|
|
+ *
|
|
+ * http://www.apache.org/licenses/LICENSE-2.0
|
|
+ *
|
|
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
+ * specific language governing permissions and limitations under the License.
|
|
+ *
|
|
+ * SPDX-License-Identifier: Apache-2.0
|
|
+ */
|
|
+package org.hyperledger.besu.consensus.qbft.core.statemachine;
|
|
+
|
|
+import static org.assertj.core.api.Assertions.assertThat;
|
|
+import static org.mockito.ArgumentMatchers.any;
|
|
+import static org.mockito.Mockito.mock;
|
|
+import static org.mockito.Mockito.when;
|
|
+
|
|
+import org.hyperledger.besu.consensus.common.bft.ConsensusRoundIdentifier;
|
|
+import org.hyperledger.besu.consensus.common.bft.FalconSeal;
|
|
+import org.hyperledger.besu.consensus.qbft.core.messagewrappers.Commit;
|
|
+import org.hyperledger.besu.consensus.qbft.core.messagewrappers.Proposal;
|
|
+import org.hyperledger.besu.consensus.qbft.core.validation.AereD412Fixture;
|
|
+import org.hyperledger.besu.consensus.qbft.core.validation.AereD412Fixture.TBlock;
|
|
+import org.hyperledger.besu.consensus.qbft.core.validation.CommitValidator;
|
|
+import org.hyperledger.besu.consensus.qbft.core.validation.MessageValidator;
|
|
+import org.hyperledger.besu.datatypes.Hash;
|
|
+
|
|
+import java.util.ArrayList;
|
|
+import java.util.List;
|
|
+import java.util.Optional;
|
|
+
|
|
+import org.apache.tuweni.bytes.Bytes;
|
|
+import org.junit.jupiter.api.AfterEach;
|
|
+import org.junit.jupiter.api.BeforeEach;
|
|
+import org.junit.jupiter.api.Test;
|
|
+
|
|
+/**
|
|
+ * AERE D-454 (2026-09-30): the regression proof of RoundState from H_fin (AIP-22 stage 4), the
|
|
+ * probe AereD412RoundStateProbe turned around. The fixture is the D-412 one (AereD412Fixture): real
|
|
+ * Falcon keys, the finality regime installed, the REAL CommitValidator with the post-quantum
|
|
+ * enforcement armed. Two actors, two contents, two positions: A (node 0) with its own commit, X
|
|
+ * (node 3) with a copy of A's commit seal in X's envelope, and the copy before or after A's own
|
|
+ * message and the proposal. Every test has a planted twin (planteaza-d454.py) that must turn it red;
|
|
+ * the harness d454-roundstate.sh checks that each plant reddens exactly the tests it names.
|
|
+ */
|
|
+public class AereD454RoundStateTest {
|
|
+
|
|
+ private static final long H = 100L;
|
|
+ private static final int ROUND = 1;
|
|
+ private static final Hash P = Hash.hash(Bytes.of(9));
|
|
+
|
|
+ // static regimes (finality, identity, anchor form) are reset before AND after each test, so no
|
|
+ // other class sharing the JVM can decide which side of H_fin a test runs on (lesson D-200)
|
|
+ @BeforeEach
|
|
+ public void clean() {
|
|
+ AereD412Fixture.uninstallAll();
|
|
+ }
|
|
+
|
|
+ @AfterEach
|
|
+ public void disarm() {
|
|
+ AereD412Fixture.uninstallAll();
|
|
+ }
|
|
+
|
|
+ /** A RoundState on block b whose commit validation is the REAL CommitValidator, every call recorded. */
|
|
+ private static RoundState round(final AereD412Fixture fx, final TBlock b, final List<Commit> validated) {
|
|
+ final TBlock proposal = b.atRound(ROUND);
|
|
+ final Hash commitHash = fx.blockInterface.replaceRoundForCommitBlock(proposal, ROUND).getHash();
|
|
+ final CommitValidator real =
|
|
+ new CommitValidator(
|
|
+ fx.addresses(0, 1, 2, 3),
|
|
+ new ConsensusRoundIdentifier(H, ROUND),
|
|
+ proposal.getHash(),
|
|
+ commitHash,
|
|
+ commitHash,
|
|
+ fx.armed());
|
|
+ final MessageValidator mv = mock(MessageValidator.class);
|
|
+ when(mv.validateProposal(any())).thenReturn(true);
|
|
+ when(mv.validateCommit(any()))
|
|
+ .thenAnswer(
|
|
+ inv -> {
|
|
+ final Commit c = inv.getArgument(0);
|
|
+ validated.add(c);
|
|
+ return real.validate(c);
|
|
+ });
|
|
+ return new RoundState(new ConsensusRoundIdentifier(H, ROUND), 3, mv);
|
|
+ }
|
|
+
|
|
+ /** X's copy of A's commit seal, with a garbage Falcon seal claiming A's index. */
|
|
+ private static Commit garbageCopy(final AereD412Fixture fx, final Commit ofA, final int variant) {
|
|
+ return fx.copyOf(
|
|
+ ofA, 3, ROUND, Optional.of(new FalconSeal(0, Bytes.repeat((byte) (0x31 + variant), 600))));
|
|
+ }
|
|
+
|
|
+ private static AereD412Fixture finality() {
|
|
+ final AereD412Fixture fx = new AereD412Fixture(4, true);
|
|
+ fx.installFinality();
|
|
+ return fx;
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T1
|
|
+
|
|
+ @Test
|
|
+ public void fromHFinACopyBeforeTheProposalNoLongerTakesTheHonestSlot() {
|
|
+ final AereD412Fixture fx = finality();
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ final Commit copy = garbageCopy(fx, ofA, 0);
|
|
+ assertThat(copy.getAuthor()).as("precondition: from H_fin the copy is authored by A").isEqualTo(fx.address(0));
|
|
+ final RoundState rs = round(fx, b, new ArrayList<>());
|
|
+ rs.addCommitMessage(copy);
|
|
+ rs.addCommitMessage(ofA);
|
|
+ rs.addCommitMessage(fx.honest(1, b, ROUND));
|
|
+ rs.addCommitMessage(fx.honest(2, b, ROUND));
|
|
+ assertThat(rs.isCommitted()).as("nothing is committed before the proposal").isFalse();
|
|
+ assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
|
|
+ assertThat(rs.getCommitSeals()).as("A's vote is counted: the copy took no slot").hasSize(3);
|
|
+ assertThat(rs.isCommitted()).as("three honest commits heard: the round is committed").isTrue();
|
|
+ assertThat(rs.getFalconSeals())
|
|
+ .as("the copy's garbage Falcon seal is not among the round's seals")
|
|
+ .doesNotContain(copy.getFalconSeal().orElseThrow())
|
|
+ .contains(ofA.getFalconSeal().orElseThrow());
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T2
|
|
+
|
|
+ @Test
|
|
+ public void fromHFinCopiesFillOnlyTheForeignQuota() {
|
|
+ final AereD412Fixture fx = finality();
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ final RoundState rs = round(fx, b, new ArrayList<>());
|
|
+ for (int i = 0; i < 20; i++) {
|
|
+ rs.addCommitMessage(garbageCopy(fx, ofA, i));
|
|
+ }
|
|
+ rs.addCommitMessage(ofA);
|
|
+ rs.addCommitMessage(fx.honest(1, b, ROUND));
|
|
+ rs.addCommitMessage(fx.honest(2, b, ROUND));
|
|
+ assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
|
|
+ assertThat(rs.isCommitted())
|
|
+ .as("twenty copies before A's own message: A's vote still counts")
|
|
+ .isTrue();
|
|
+ assertThat(rs.getFalconSeals()).hasSize(3).contains(ofA.getFalconSeal().orElseThrow());
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T3
|
|
+
|
|
+ @Test
|
|
+ public void fromHFinAnUnvalidatedCopyNeverCounts() {
|
|
+ final AereD412Fixture fx = finality();
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ final RoundState rs = round(fx, b, new ArrayList<>());
|
|
+ rs.addCommitMessage(garbageCopy(fx, ofA, 0));
|
|
+ rs.addCommitMessage(fx.honest(1, b, ROUND));
|
|
+ rs.addCommitMessage(fx.honest(2, b, ROUND));
|
|
+ assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
|
|
+ assertThat(rs.getCommitSeals()).as("the copy is not A's vote").hasSize(2);
|
|
+ assertThat(rs.isCommitted()).as("two honest commits and a copy: not committed").isFalse();
|
|
+ rs.addCommitMessage(ofA);
|
|
+ assertThat(rs.isCommitted()).as("A's own message after the proposal completes the round").isTrue();
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T4
|
|
+
|
|
+ @Test
|
|
+ public void fromHFinTheReleaseValidatesSelfFirstAndSkipsCountedAuthors() {
|
|
+ final AereD412Fixture fx = finality();
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ final Commit ofB = fx.honest(1, b, ROUND);
|
|
+ final List<Commit> validated = new ArrayList<>();
|
|
+ final RoundState rs = round(fx, b, validated);
|
|
+ rs.addCommitMessage(garbageCopy(fx, ofA, 0));
|
|
+ rs.addCommitMessage(garbageCopy(fx, ofA, 1));
|
|
+ rs.addCommitMessage(ofA);
|
|
+ rs.addCommitMessage(ofB);
|
|
+ assertThat(validated).as("nothing is validated before the proposal").isEmpty();
|
|
+ assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
|
|
+ // BftMessage.equals compares the whole signed payload: each copy differs from ofA (envelope, seal)
|
|
+ assertThat(validated)
|
|
+ .as("A's and B's own messages validated, the copies never (A's slot was taken)")
|
|
+ .containsExactly(ofA, ofB);
|
|
+ assertThat(rs.getCommitSeals()).hasSize(2);
|
|
+ rs.addCommitMessage(fx.honest(2, b, ROUND));
|
|
+ assertThat(rs.isCommitted()).isTrue();
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T5
|
|
+
|
|
+ @Test
|
|
+ public void fromHFinACountedAuthorIsNotValidatedAgainAfterTheProposal() {
|
|
+ final AereD412Fixture fx = finality();
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ final Commit ofB = fx.honest(1, b, ROUND);
|
|
+ final Commit ofC = fx.honest(2, b, ROUND);
|
|
+ final List<Commit> validated = new ArrayList<>();
|
|
+ final RoundState rs = round(fx, b, validated);
|
|
+ assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
|
|
+ rs.addCommitMessage(ofA);
|
|
+ for (int i = 0; i < 3; i++) {
|
|
+ rs.addCommitMessage(garbageCopy(fx, ofA, i));
|
|
+ }
|
|
+ rs.addCommitMessage(ofB);
|
|
+ rs.addCommitMessage(ofC);
|
|
+ assertThat(validated)
|
|
+ .as("copies of an author already counted are not validated")
|
|
+ .containsExactly(ofA, ofB, ofC);
|
|
+ assertThat(rs.isCommitted()).isTrue();
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T6
|
|
+
|
|
+ @Test
|
|
+ public void fromHFinTheCheapGuardDropsBeforeAnyRecovery() {
|
|
+ final AereD412Fixture fx = finality();
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ // three distinct SELF messages of A (its envelope, its commit seal; Falcon by three different keys)
|
|
+ final Commit s1 = fx.commit(0, b, ROUND, 0, 0);
|
|
+ final Commit s2 = fx.commit(0, b, ROUND, 0, 1);
|
|
+ final Commit s3 = fx.commit(0, b, ROUND, 0, 2);
|
|
+ final AereD454PendingCommits pending = new AereD454PendingCommits();
|
|
+ assertThat(pending.hold(s1)).isTrue();
|
|
+ assertThat(pending.hold(s2)).isTrue();
|
|
+ assertThat(pending.hold(s3)).as("A's SELF quota is full").isFalse();
|
|
+ int held = 0;
|
|
+ for (int i = 0; i < 50; i++) {
|
|
+ if (pending.hold(garbageCopy(fx, ofA, i))) {
|
|
+ held++;
|
|
+ }
|
|
+ }
|
|
+ assertThat(held).as("the FOREIGN quota").isEqualTo(AereD454PendingCommits.MAX_PER_CLASS);
|
|
+ assertThat(pending.size()).isEqualTo(2 * AereD454PendingCommits.MAX_PER_CLASS);
|
|
+ assertThat(pending.classifications())
|
|
+ .as("once both classes of A are full, nothing more is recovered for A")
|
|
+ .isEqualTo(3L + AereD454PendingCommits.MAX_PER_CLASS);
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T7
|
|
+
|
|
+ @Test
|
|
+ public void fromHFinTheSameMessageTwiceTakesOnePlace() {
|
|
+ final AereD412Fixture fx = finality();
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ final Commit c1 = garbageCopy(fx, ofA, 1);
|
|
+ final AereD454PendingCommits pending = new AereD454PendingCommits();
|
|
+ assertThat(pending.hold(ofA)).isTrue();
|
|
+ assertThat(pending.hold(ofA)).as("the same bytes again").isFalse();
|
|
+ assertThat(pending.hold(c1)).isTrue();
|
|
+ assertThat(pending.hold(c1)).as("the same copy again").isFalse();
|
|
+ assertThat(pending.size()).isEqualTo(2);
|
|
+ assertThat(pending.hold(garbageCopy(fx, ofA, 2)))
|
|
+ .as("a second, distinct copy still fits the FOREIGN quota: repeats took no place")
|
|
+ .isTrue();
|
|
+ assertThat(pending.hold(garbageCopy(fx, ofA, 3))).as("the FOREIGN quota is full").isFalse();
|
|
+ assertThat(pending.size()).isEqualTo(3);
|
|
+ assertThat(pending.hold(fx.honest(1, b, ROUND))).as("another author has its own quota").isTrue();
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T8
|
|
+
|
|
+ @Test
|
|
+ public void belowHFinTheUpstreamPathIsKeptCallByCall() {
|
|
+ final AereD412Fixture fx = new AereD412Fixture(4, true);
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ final Commit ofA2 = fx.commit(0, b, ROUND, 0, 1);
|
|
+ final Commit copy = garbageCopy(fx, ofA, 0);
|
|
+ final Commit ofB = fx.honest(1, b, ROUND);
|
|
+ final Commit ofC = fx.honest(2, b, ROUND);
|
|
+ assertThat(copy.getAuthor()).as("precondition: below H_fin the copy is X's").isEqualTo(fx.address(3));
|
|
+ final List<Commit> validated = new ArrayList<>();
|
|
+ final RoundState rs = round(fx, b, validated);
|
|
+ rs.addCommitMessage(copy);
|
|
+ rs.addCommitMessage(ofA);
|
|
+ rs.addCommitMessage(ofA2);
|
|
+ rs.addCommitMessage(ofB);
|
|
+ assertThat(validated).as("upstream: nothing is validated before the proposal").isEmpty();
|
|
+ assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
|
|
+ rs.addCommitMessage(ofC);
|
|
+ rs.addCommitMessage(ofA);
|
|
+ // upstream: the first message of each author is kept before the proposal (putIfAbsent; A's second
|
|
+ // dropped), validated at the proposal in insertion order, then every later message is validated
|
|
+ assertThat(validated)
|
|
+ .as("the upstream sequence of validations, call by call")
|
|
+ .containsExactly(copy, ofA, ofB, ofC, ofA);
|
|
+ assertThat(rs.isCommitted()).isTrue();
|
|
+ assertThat(rs.getCommitSeals()).hasSize(3);
|
|
+ }
|
|
+
|
|
+ // ------------------------------------------------------------------------------------------ T9
|
|
+
|
|
+ @Test
|
|
+ public void fromHFinAValidForeignCopyIsAnHonestVote() {
|
|
+ final AereD412Fixture fx = finality();
|
|
+ final TBlock b = new TBlock(H, P, Bytes.of(0x0b), 0);
|
|
+ final Commit ofA = fx.honest(0, b, ROUND);
|
|
+ // X re-envelopes A's whole vote: A's commit seal AND A's valid Falcon seal
|
|
+ final Commit reEnveloped = fx.copyOf(ofA, 3, ROUND, ofA.getFalconSeal());
|
|
+ assertThat(reEnveloped.getAuthor()).isEqualTo(fx.address(0));
|
|
+ final RoundState rs = round(fx, b, new ArrayList<>());
|
|
+ rs.addCommitMessage(reEnveloped);
|
|
+ rs.addCommitMessage(fx.honest(1, b, ROUND));
|
|
+ rs.addCommitMessage(fx.honest(2, b, ROUND));
|
|
+ assertThat(rs.setProposedBlock(mock(Proposal.class))).isTrue();
|
|
+ assertThat(rs.isCommitted())
|
|
+ .as("from H_fin the envelope does not decide the author: A's vote in X's envelope counts")
|
|
+ .isTrue();
|
|
+ assertThat(rs.getCommitSeals()).hasSize(3);
|
|
+ }
|
|
+}
|
|
diff --git a/consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/validation/AereD412Fixture.java b/consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/validation/AereD412Fixture.java
|
|
new file mode 100644
|
|
index 0000000000..7d2a8d97e6
|
|
--- /dev/null
|
|
+++ b/consensus/qbft-core/src/test/java/org/hyperledger/besu/consensus/qbft/core/validation/AereD412Fixture.java
|
|
@@ -0,0 +1,590 @@
|
|
+/* AERE D-412 / capul decis, pasul 1: fixtura comuna a probelor predicatului unic si a colectorului.
|
|
+ *
|
|
+ * Doi actori, doua continuturi, doua pozitii (regula casei): noduri QBFT reale (chei ECDSA reale,
|
|
+ * mesaje semnate cu MessageFactory din amonte), chei Falcon-512 REALE prin stratul de scheme, un
|
|
+ * registru de proba care leaga indexul i de nodul i si NUMARA verificarile, si un model de bloc in
|
|
+ * care amprenta depinde de runda exact ca in QBFT: digestul mesajului pastreaza runda, amprenta de pe
|
|
+ * lant (runda 0) nu. Nimic mockuit pe drumul criptografic.
|
|
+ *
|
|
+ * 2026-09-30 (revizuirea pasului 1):
|
|
+ * - commit() nu mai trece prin AereVoteCounts.Target (functia masurata): urmeaza, rand cu rand,
|
|
+ * drumul EMITATORULUI din QbftRound (commitHashFor -> createCommitBlock, pqOnchainHashOf,
|
|
+ * pqSealMessageFor -> PqAnchorProducer.commitSealMessage, createCommit(round, block.getHash(),
|
|
+ * ...)), pe blocul PROPUS al rundei; cu ancora armata (PqAnchorProducer.useConfigForTesting)
|
|
+ * sigiliul PQ semneaza forma de ancora. Legatura cu blocurile si mesajele reale o masoara
|
|
+ * AereD412RealHeadersTest (consensus/qbft) si AereD412RealRoundTest (integrare);
|
|
+ * - installFinality(): regimul REAL de la H_fin (AerePqIdentity + AerePqFinality instalate, ca in
|
|
+ * AereFinalityMessageTest), in care autorul unui Commit e id-ul indexului din sigiliu cand acela e
|
|
+ * semnatarul sigiliului de commit, deci plicul nu mai decide autorul;
|
|
+ * - registrul poate fi pus PENDING (nimic legat, ca un late-anchor neactivat, D-358) si are o
|
|
+ * generatie care se schimba la fiecare trecere;
|
|
+ * - blocul poate declara o marime (sizeOf), iar interfata numara inlocuirile de runda.
|
|
+ * 2026-09-30, a doua revizuire: hybridArmed() (Falcon + SLH-DSA cerut la orice inaltime, ca la un parinte de ancora) si
|
|
+ * commit(..., extras), ca o cheie sa poata cere predicatului refuzul cel mai scump: sigiliul Falcon verificat, apoi refuzul
|
|
+ * mesajului pentru SLH-DSA. */
|
|
+package org.hyperledger.besu.consensus.qbft.core.validation;
|
|
+
|
|
+import static org.mockito.Mockito.mock;
|
|
+
|
|
+import org.hyperledger.besu.consensus.common.bft.AerePqFinality;
|
|
+import org.hyperledger.besu.consensus.common.bft.AerePqIdentity;
|
|
+import org.hyperledger.besu.consensus.common.bft.ConsensusRoundIdentifier;
|
|
+import org.hyperledger.besu.consensus.common.bft.FalconSeal;
|
|
+import org.hyperledger.besu.consensus.common.bft.HybridSignerRegistry;
|
|
+import org.hyperledger.besu.consensus.common.bft.PqRegistryV3;
|
|
+import org.hyperledger.besu.consensus.common.bft.PqSchemeSchedule;
|
|
+import org.hyperledger.besu.consensus.common.bft.PqSignerRegistry;
|
|
+import org.hyperledger.besu.consensus.common.bft.SchemeSeal;
|
|
+import org.hyperledger.besu.consensus.common.bft.SealScheme;
|
|
+import org.hyperledger.besu.consensus.common.bft.SealSchemes;
|
|
+import org.hyperledger.besu.consensus.common.bft.blockcreation.PqAnchorProducer;
|
|
+import org.hyperledger.besu.consensus.qbft.core.messagewrappers.Commit;
|
|
+import org.hyperledger.besu.consensus.qbft.core.types.QbftBlock;
|
|
+import org.hyperledger.besu.consensus.qbft.core.types.QbftBlockCodec;
|
|
+import org.hyperledger.besu.consensus.qbft.core.types.QbftBlockHeader;
|
|
+import org.hyperledger.besu.consensus.qbft.core.types.QbftBlockInterface;
|
|
+import org.hyperledger.besu.crypto.SECPSignature;
|
|
+import org.hyperledger.besu.crypto.SecureRandomProvider;
|
|
+import org.hyperledger.besu.cryptoservices.NodeKey;
|
|
+import org.hyperledger.besu.datatypes.Address;
|
|
+import org.hyperledger.besu.datatypes.Hash;
|
|
+
|
|
+import java.security.SecureRandom;
|
|
+import java.util.ArrayList;
|
|
+import java.util.HashMap;
|
|
+import java.util.List;
|
|
+import java.util.Map;
|
|
+import java.util.Optional;
|
|
+import java.util.Properties;
|
|
+import java.util.TreeMap;
|
|
+import java.util.concurrent.atomic.AtomicLong;
|
|
+
|
|
+import org.apache.tuweni.bytes.Bytes;
|
|
+import org.apache.tuweni.bytes.Bytes32;
|
|
+
|
|
+/** Shared fixture for the D-412 step-1 proofs. */
|
|
+public final class AereD412Fixture {
|
|
+
|
|
+ /** The chain id of the post-quantum messages in the finality regime. */
|
|
+ public static final long CHAIN = 28001L;
|
|
+
|
|
+ /** H_id = H_k = the bind height of the one v3 epoch of the finality regime. */
|
|
+ public static final long H_ID = 10L;
|
|
+
|
|
+ /** H_fin of the finality regime. */
|
|
+ public static final long H_FIN = 20L;
|
|
+
|
|
+ /** A block model whose hash depends on the round, like a QBFT committed-seal hash. */
|
|
+ public static final class TBlock implements QbftBlock {
|
|
+ private final long number;
|
|
+ private final Hash parent;
|
|
+ private final Bytes content;
|
|
+ private final long declaredBytes;
|
|
+ // the round is not kept as a field: it only enters the hash, as in a QBFT committed-seal hash
|
|
+ private final Hash hash;
|
|
+
|
|
+ /**
|
|
+ * @param number the height
|
|
+ * @param parent the on-chain hash of the parent
|
|
+ * @param content what distinguishes siblings
|
|
+ * @param round the round written in the block
|
|
+ */
|
|
+ public TBlock(final long number, final Hash parent, final Bytes content, final int round) {
|
|
+ this(number, parent, content, round, 0L);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * @param number the height
|
|
+ * @param parent the on-chain hash of the parent
|
|
+ * @param content what distinguishes siblings
|
|
+ * @param round the round written in the block
|
|
+ * @param declaredBytes the size the block declares (charged by sizeOf, not allocated)
|
|
+ */
|
|
+ public TBlock(
|
|
+ final long number,
|
|
+ final Hash parent,
|
|
+ final Bytes content,
|
|
+ final int round,
|
|
+ final long declaredBytes) {
|
|
+ this.number = number;
|
|
+ this.parent = parent;
|
|
+ this.content = content;
|
|
+ this.declaredBytes = declaredBytes;
|
|
+ this.hash =
|
|
+ Hash.hash(
|
|
+ Bytes.concatenate(
|
|
+ Bytes.ofUnsignedLong(number),
|
|
+ parent.getBytes(),
|
|
+ content,
|
|
+ Bytes.ofUnsignedLong(declaredBytes),
|
|
+ Bytes.ofUnsignedInt(round)));
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * The same block at another round.
|
|
+ *
|
|
+ * @param r the round
|
|
+ * @return the block at round r
|
|
+ */
|
|
+ public TBlock atRound(final int r) {
|
|
+ return new TBlock(number, parent, content, r, declaredBytes);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * The on-chain hash, round 0.
|
|
+ *
|
|
+ * @return the hash
|
|
+ */
|
|
+ public Hash onchain() {
|
|
+ return atRound(0).getHash();
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * The height.
|
|
+ *
|
|
+ * @return the number
|
|
+ */
|
|
+ public long number() {
|
|
+ return number;
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public QbftBlockHeader getHeader() {
|
|
+ final TBlock self = this;
|
|
+ return new QbftBlockHeader() {
|
|
+ @Override
|
|
+ public long getNumber() {
|
|
+ return self.number;
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public long getTimestamp() {
|
|
+ return 0L;
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public Address getCoinbase() {
|
|
+ return Address.ZERO;
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public Hash getHash() {
|
|
+ return self.hash;
|
|
+ }
|
|
+
|
|
+ // QbftBlockHeader.getParentHash() is D-437's (lab and delivery tree); the FLEET tree, from which the
|
|
+ // AIP-22 stage 4 binary is built, does not declare it. Without @Override the fixture compiles on both
|
|
+ // (t4-livrare.sh, 2026-10-01: with it, the fleet tree refused to compile the fixture).
|
|
+ @SuppressWarnings("MissingOverride")
|
|
+ public Hash getParentHash() {
|
|
+ return self.parent;
|
|
+ }
|
|
+ };
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public boolean isEmpty() {
|
|
+ return true;
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public Hash getHash() {
|
|
+ return hash;
|
|
+ }
|
|
+ }
|
|
+
|
|
+ /** Replaces the round, as the QBFT block interface does, and counts the replacements. */
|
|
+ public static final class TInterface implements QbftBlockInterface {
|
|
+ private final AtomicLong replacements = new AtomicLong();
|
|
+
|
|
+ @Override
|
|
+ public QbftBlock replaceRoundForCommitBlock(final QbftBlock proposalBlock, final int roundNumber) {
|
|
+ replacements.incrementAndGet();
|
|
+ return ((TBlock) proposalBlock).atRound(roundNumber);
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public QbftBlock replaceRoundAndProposerForProposalBlock(
|
|
+ final QbftBlock proposalBlock, final int roundNumber, final Address proposer) {
|
|
+ return replaceRoundForCommitBlock(proposalBlock, roundNumber);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * How many round replacements were made (the recoding work of the collector).
|
|
+ *
|
|
+ * @return the count
|
|
+ */
|
|
+ public long replacements() {
|
|
+ return replacements.get();
|
|
+ }
|
|
+ }
|
|
+
|
|
+ private final SecureRandom random = SecureRandomProvider.createSecureRandom();
|
|
+ private final Map<Integer, Address> bindings = new HashMap<>();
|
|
+ private final Map<Integer, byte[]> falconPublic = new HashMap<>();
|
|
+ private final Map<Integer, SealScheme.PrivateHandle> falconPrivate = new HashMap<>();
|
|
+ private final AtomicLong pqVerifyCalls = new AtomicLong();
|
|
+ private final AtomicLong generation = new AtomicLong();
|
|
+ private volatile boolean pending;
|
|
+ private volatile boolean v3;
|
|
+ private final List<Address> ids = new ArrayList<>();
|
|
+
|
|
+ /** The nodes; node i owns Falcon index i. */
|
|
+ public final QbftNodeList nodes;
|
|
+
|
|
+ /** The block interface. */
|
|
+ public final TInterface blockInterface = new TInterface();
|
|
+
|
|
+ /**
|
|
+ * Registry: index i bound to node i (to id i in the finality regime), verification by the REAL
|
|
+ * Falcon scheme, counted. While PENDING nothing is bound and nothing verifies, as a late-anchor
|
|
+ * registry before activation.
|
|
+ */
|
|
+ public final PqSignerRegistry registry =
|
|
+ new PqSignerRegistry() {
|
|
+ @Override
|
|
+ public Address addressForIndexAtHistoric(final long blockNumber, final int validatorIndex) {
|
|
+ return addressForIndexAtOwnHead(blockNumber, validatorIndex);
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public Address addressForIndexAtOwnHead(final long blockNumber, final int validatorIndex) {
|
|
+ if (pending) {
|
|
+ return null;
|
|
+ }
|
|
+ if (v3) {
|
|
+ return AerePqIdentity.v3IdAt(blockNumber, validatorIndex).orElse(null);
|
|
+ }
|
|
+ return bindings.get(validatorIndex);
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public boolean verifyAtHistoric(
|
|
+ final long blockNumber, final int validatorIndex, final Bytes message, final Bytes signature) {
|
|
+ return verifyAtOwnHead(blockNumber, validatorIndex, message, signature);
|
|
+ }
|
|
+
|
|
+ @Override
|
|
+ public boolean verifyAtOwnHead(
|
|
+ final long blockNumber, final int validatorIndex, final Bytes message, final Bytes signature) {
|
|
+ pqVerifyCalls.incrementAndGet();
|
|
+ if (pending) {
|
|
+ return false;
|
|
+ }
|
|
+ final byte[] pk = falconPublic.get(validatorIndex);
|
|
+ return pk != null
|
|
+ && SealSchemes.FALCON_512.verifyRaw(pk, message.toArray(), signature.toArray());
|
|
+ }
|
|
+ };
|
|
+
|
|
+ /**
|
|
+ * @param count how many nodes (and Falcon keys)
|
|
+ * @param falconKeys whether to generate Falcon keys (the flood proof does not need them)
|
|
+ */
|
|
+ public AereD412Fixture(final int count, final boolean falconKeys) {
|
|
+ nodes = QbftNodeList.createNodes(count, mock(QbftBlockCodec.class));
|
|
+ for (int i = 0; i < count; i++) {
|
|
+ bindings.put(i, nodes.getNode(i).getAddress());
|
|
+ if (falconKeys) {
|
|
+ final SealScheme.GeneratedPair pair = SealSchemes.FALCON_512.generate(random);
|
|
+ falconPublic.put(i, pair.publicRegistryForm());
|
|
+ falconPrivate.put(i, pair.privateKey());
|
|
+ }
|
|
+ }
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * How many Falcon verifications the registry performed.
|
|
+ *
|
|
+ * @return the count
|
|
+ */
|
|
+ public long pqVerifyCalls() {
|
|
+ return pqVerifyCalls.get();
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * The verifier generation: changes every time the registry changes state.
|
|
+ *
|
|
+ * @return the generation
|
|
+ */
|
|
+ public long generation() {
|
|
+ return generation.get();
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * Put the registry PENDING (nothing bound, nothing verifies) or active; either way the generation
|
|
+ * moves.
|
|
+ *
|
|
+ * @param isPending the new state
|
|
+ */
|
|
+ public void setRegistryPending(final boolean isPending) {
|
|
+ pending = isPending;
|
|
+ generation.incrementAndGet();
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * The bytes a block is charged: 100 for the header plus what a TBlock declares.
|
|
+ *
|
|
+ * @param block the block
|
|
+ * @return the bytes
|
|
+ */
|
|
+ public long sizeOf(final QbftBlock block) {
|
|
+ return 100L + (block instanceof TBlock t ? t.declaredBytes : 0L);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * The author of node i: its address, or its post-quantum id in the finality regime.
|
|
+ *
|
|
+ * @param i the node
|
|
+ * @return the author
|
|
+ */
|
|
+ public Address address(final int i) {
|
|
+ return v3 ? ids.get(i) : nodes.getNode(i).getAddress();
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * The authors of the given nodes.
|
|
+ *
|
|
+ * @param idx the nodes
|
|
+ * @return the authors, in that order
|
|
+ */
|
|
+ public List<Address> addresses(final int... idx) {
|
|
+ final List<Address> out = new ArrayList<>();
|
|
+ for (final int i : idx) {
|
|
+ out.add(address(i));
|
|
+ }
|
|
+ return out;
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * An armed enforcement over the counted registry.
|
|
+ *
|
|
+ * @return the enforcement, armed from height 0
|
|
+ */
|
|
+ public PqCommitEnforcement armed() {
|
|
+ return new PqCommitEnforcement(0L, registry);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * An armed HYBRID enforcement (2026-09-30, second review of step 1): Falcon through the counted
|
|
+ * registry, plus SLH-DSA demanded at every height (as at an anchor parent, D-329), with a hybrid
|
|
+ * registry binding index i to node i. A commit with a valid Falcon seal and no valid SLH-DSA seal is
|
|
+ * refused BY THE MESSAGE after its Falcon seal was verified: the most expensive refusal a key can
|
|
+ * make the predicate pay for.
|
|
+ *
|
|
+ * @return the enforcement, armed from height 0
|
|
+ */
|
|
+ public PqCommitEnforcement hybridArmed() {
|
|
+ final Properties p = new Properties();
|
|
+ p.setProperty("formatVersion", HybridSignerRegistry.FORMAT_VERSION);
|
|
+ p.setProperty("chainId", Long.toString(CHAIN));
|
|
+ p.setProperty("count", Integer.toString(falconPublic.size()));
|
|
+ for (final Map.Entry<Integer, byte[]> e : falconPublic.entrySet()) {
|
|
+ p.setProperty(e.getKey() + ".addr", nodes.getNode(e.getKey()).getAddress().toHexString());
|
|
+ p.setProperty(
|
|
+ e.getKey() + ".key." + SealSchemes.FALCON_512.id(), Bytes.wrap(e.getValue()).toHexString());
|
|
+ }
|
|
+ return new PqCommitEnforcement(
|
|
+ 0L,
|
|
+ registry,
|
|
+ PqSchemeSchedule.parse(
|
|
+ "0:" + SealSchemes.FALCON_512.id() + "+" + SealSchemes.SLH_DSA_128S.id()),
|
|
+ HybridSignerRegistry.fromProperties(p, "aere-d412-fixture"),
|
|
+ h -> true);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * Installs the finality regime (AIP-22 stage 4) for these nodes, as AereFinalityMessageTest does: one
|
|
+ * v3 epoch bound at H_ID with each node's Falcon key and ECDSA claim, identity from H_ID (keys from
|
|
+ * H_ID), H_FIN with the production-shaped verifier. From then on authors are ids.
|
|
+ */
|
|
+ public void installFinality() {
|
|
+ final SealScheme falcon = SealSchemes.FALCON_512;
|
|
+ final int n = falconPrivate.size();
|
|
+ if (n == 0) {
|
|
+ throw new IllegalStateException("the finality regime needs the Falcon keys");
|
|
+ }
|
|
+ final List<PqRegistryV3.Row> rows = new ArrayList<>();
|
|
+ for (int i = 0; i < n; i++) {
|
|
+ final NodeKey nk = nodes.getNode(i).getNodeKey();
|
|
+ final byte[] std = new byte[897];
|
|
+ std[0] = 0x09;
|
|
+ System.arraycopy(falconPublic.get(i), 0, std, 1, 896);
|
|
+ final byte[] id = PqRegistryV3.idOf(std);
|
|
+ final Bytes32 popMsg = PqRegistryV3.popMessage(CHAIN, H_ID, n, i, id, falcon.wireId(), std);
|
|
+ final byte[] pop = falcon.sign(falconPrivate.get(i), popMsg.toArrayUnsafe()).orElseThrow();
|
|
+ final TreeMap<Integer, PqRegistryV3.Key> keys = new TreeMap<>();
|
|
+ keys.put(falcon.wireId() & 0xff, new PqRegistryV3.Key(falcon, std, pop));
|
|
+ final PqRegistryV3.Row bare = new PqRegistryV3.Row(i, id, keys, null, Map.of(), null, null);
|
|
+ final Bytes32 rh = PqRegistryV3.rowHash(CHAIN, H_ID, n, bare);
|
|
+ final SECPSignature claim = nk.sign(PqRegistryV3.claimDigest(CHAIN, rh));
|
|
+ rows.add(
|
|
+ new PqRegistryV3.Row(
|
|
+ i,
|
|
+ id,
|
|
+ keys,
|
|
+ null,
|
|
+ Map.of(),
|
|
+ nodes.getNode(i).getAddress().getBytes().toArrayUnsafe(),
|
|
+ claim.encodedBytes().toArrayUnsafe()));
|
|
+ ids.add(Address.wrap(Bytes.wrap(id)));
|
|
+ }
|
|
+ final List<Bytes32> hashes = new ArrayList<>();
|
|
+ for (final PqRegistryV3.Row r : rows) {
|
|
+ hashes.add(PqRegistryV3.rowHash(CHAIN, H_ID, n, r));
|
|
+ }
|
|
+ final Bytes32 root = PqRegistryV3.merkleRoot(hashes);
|
|
+ final PqRegistryV3.Registry reg =
|
|
+ new PqRegistryV3.Registry(
|
|
+ CHAIN, H_ID, n, rows, hashes, root, PqRegistryV3.registryHash(CHAIN, H_ID, n, root));
|
|
+ final TreeMap<Long, PqRegistryV3.Registry> epochs = new TreeMap<>();
|
|
+ epochs.put(H_ID, reg);
|
|
+ AerePqFinality.uninstall();
|
|
+ AerePqIdentity.uninstall();
|
|
+ AerePqIdentity.install(new AerePqIdentity.Config(H_ID, epochs, H_ID));
|
|
+ AerePqFinality.install(
|
|
+ H_FIN,
|
|
+ CHAIN,
|
|
+ (h, i, m, s) ->
|
|
+ AerePqIdentity.v3KeyAt(h, i, "falcon-512")
|
|
+ .map(k -> falcon.verifyRaw(k, m.toArrayUnsafe(), s.toArrayUnsafe()))
|
|
+ .orElse(false));
|
|
+ v3 = true;
|
|
+ }
|
|
+
|
|
+ /** Disarms the finality regime (and the anchor form). */
|
|
+ public static void uninstallAll() {
|
|
+ AerePqFinality.uninstall();
|
|
+ AerePqIdentity.uninstall();
|
|
+ PqAnchorProducer.useConfigForTesting(null);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * A commit of node {@code author} for {@code block} at round {@code round}, built on the EMITTER's
|
|
+ * path (QbftRound), not on AereVoteCounts.Target: the proposal of the round is the block at round r;
|
|
+ * the ECDSA commit seal signs commitHashFor(proposal) = replaceRoundForCommitBlock(proposal, r);
|
|
+ * the post-quantum message is PqAnchorProducer.commitSealMessage(number, round-0 hash of the
|
|
+ * proposal, commitHash), as pqSealMessageFor; the payload carries proposal.getHash(). The Falcon
|
|
+ * seal CLAIMS index {@code claimedIndex} and is signed by the key of {@code signerIndex} (no seal
|
|
+ * when claimedIndex < 0).
|
|
+ *
|
|
+ * @param author the ECDSA author
|
|
+ * @param block the block voted, any round form
|
|
+ * @param round the round
|
|
+ * @param claimedIndex the index written in the seal, or -1 for no seal
|
|
+ * @param signerIndex whose Falcon key signs
|
|
+ * @return the commit
|
|
+ */
|
|
+ public Commit commit(
|
|
+ final int author, final TBlock block, final int round, final int claimedIndex, final int signerIndex) {
|
|
+ return commit(author, block, round, claimedIndex, signerIndex, List.of());
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * As {@link #commit(int, TBlock, int, int, int)}, carrying extra scheme seals beside the Falcon one
|
|
+ * (the whole payload, extras included, signed by the author's ECDSA key, as MessageFactory does).
|
|
+ *
|
|
+ * @param author the ECDSA author
|
|
+ * @param block the block voted, any round form
|
|
+ * @param round the round
|
|
+ * @param claimedIndex the index written in the seal, or -1 for no seal
|
|
+ * @param signerIndex whose Falcon key signs
|
|
+ * @param extras the extra scheme seals
|
|
+ * @return the commit
|
|
+ */
|
|
+ public Commit commit(
|
|
+ final int author,
|
|
+ final TBlock block,
|
|
+ final int round,
|
|
+ final int claimedIndex,
|
|
+ final int signerIndex,
|
|
+ final List<SchemeSeal> extras) {
|
|
+ final TBlock proposal = block.atRound(round);
|
|
+ final Hash commitHash = blockInterface.replaceRoundForCommitBlock(proposal, round).getHash();
|
|
+ final SECPSignature ecdsa =
|
|
+ nodes.getNode(author).getNodeKey().sign(Bytes32.wrap(commitHash.getBytes()));
|
|
+ final Bytes32 pqMessage =
|
|
+ PqAnchorProducer.commitSealMessage(
|
|
+ proposal.number(),
|
|
+ () -> blockInterface.replaceRoundForCommitBlock(proposal, 0).getHash().getBytes(),
|
|
+ commitHash);
|
|
+ Optional<FalconSeal> seal = Optional.empty();
|
|
+ if (claimedIndex >= 0) {
|
|
+ seal = Optional.of(falconSeal(claimedIndex, signerIndex, pqMessage));
|
|
+ }
|
|
+ return nodes
|
|
+ .getMessageFactory(author)
|
|
+ .createCommit(
|
|
+ new ConsensusRoundIdentifier(proposal.number(), round), proposal.getHash(), ecdsa, seal, extras);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * A Falcon seal claiming {@code claimedIndex}, signed by the key of {@code signerIndex} over
|
|
+ * {@code message}.
|
|
+ *
|
|
+ * @param claimedIndex the index written in the seal
|
|
+ * @param signerIndex whose key signs
|
|
+ * @param message the signed message
|
|
+ * @return the seal
|
|
+ */
|
|
+ public FalconSeal falconSeal(final int claimedIndex, final int signerIndex, final Bytes32 message) {
|
|
+ final byte[] sig =
|
|
+ SealSchemes.FALCON_512.sign(falconPrivate.get(signerIndex), message.toArray()).orElseThrow();
|
|
+ return new FalconSeal(claimedIndex, Bytes.wrap(sig));
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * A copy of {@code original} as another key can make it: the original commit seal and digest, the
|
|
+ * given round and seal, the envelope signed by {@code envelopeNode}.
|
|
+ *
|
|
+ * @param original the commit copied
|
|
+ * @param envelopeNode the node whose key signs the envelope
|
|
+ * @param round the round written in the copy
|
|
+ * @param seal the post-quantum seal written in the copy
|
|
+ * @return the copy
|
|
+ */
|
|
+ public Commit copyOf(
|
|
+ final Commit original, final int envelopeNode, final int round, final Optional<FalconSeal> seal) {
|
|
+ return nodes
|
|
+ .getMessageFactory(envelopeNode)
|
|
+ .createCommit(
|
|
+ new ConsensusRoundIdentifier(original.getRoundIdentifier().getSequenceNumber(), round),
|
|
+ original.getDigest(),
|
|
+ original.getCommitSeal(),
|
|
+ seal);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * An honest commit: own ECDSA, own Falcon index and key.
|
|
+ *
|
|
+ * @param author the node
|
|
+ * @param block the block
|
|
+ * @param round the round
|
|
+ * @return the commit
|
|
+ */
|
|
+ public Commit honest(final int author, final TBlock block, final int round) {
|
|
+ return commit(author, block, round, author, author);
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * A commit with an arbitrary digest (a flood message): valid message signature of the author, a
|
|
+ * fixed commit seal and a garbage Falcon seal.
|
|
+ *
|
|
+ * @param author the node
|
|
+ * @param height the height
|
|
+ * @param round the round
|
|
+ * @param digest the digest
|
|
+ * @param commitSeal a commit seal (not checked by anything a flood reaches)
|
|
+ * @return the commit
|
|
+ */
|
|
+ public Commit flood(
|
|
+ final int author, final long height, final int round, final Hash digest, final SECPSignature commitSeal) {
|
|
+ return nodes
|
|
+ .getMessageFactory(author)
|
|
+ .createCommit(
|
|
+ new ConsensusRoundIdentifier(height, round),
|
|
+ digest,
|
|
+ commitSeal,
|
|
+ Optional.of(new FalconSeal(author, Bytes.repeat((byte) 7, 64))));
|
|
+ }
|
|
+}
|
|
diff --git a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java
|
|
index b7e4c4524e..02ce695006 100644
|
|
--- a/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java
|
|
+++ b/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-drop-per-request-console-writes.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.api.query;
|
|
|
|
import static com.google.common.base.Preconditions.checkArgument;
|
|
import static org.hyperledger.besu.ethereum.api.query.cache.TransactionLogBloomCacher.BLOCKS_PER_BLOOM_CACHE;
|
|
import static org.hyperledger.besu.ethereum.mainnet.feemarket.ExcessBlobGasCalculator.calculateExcessBlobGasForParent;
|
|
import static org.hyperledger.besu.ethereum.trie.pathbased.common.provider.WorldStateQueryParams.withBlockHeaderAndNoUpdateNodeHead;
|
|
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.datatypes.Hash;
|
|
import org.hyperledger.besu.datatypes.LogsBloomFilter;
|
|
import org.hyperledger.besu.datatypes.Wei;
|
|
import org.hyperledger.besu.ethereum.api.ApiConfiguration;
|
|
import org.hyperledger.besu.ethereum.api.ImmutableApiConfiguration;
|
|
import org.hyperledger.besu.ethereum.api.query.cache.TransactionLogBloomCacher;
|
|
import org.hyperledger.besu.ethereum.chain.Blockchain;
|
|
import org.hyperledger.besu.ethereum.chain.TransactionLocation;
|
|
import org.hyperledger.besu.ethereum.core.Block;
|
|
import org.hyperledger.besu.ethereum.core.BlockBody;
|
|
import org.hyperledger.besu.ethereum.core.BlockHeader;
|
|
diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java
|
|
index 804f4bed0a..9f86268d42 100755
|
|
--- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java
|
|
+++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/MainnetTransactionValidator.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright contributors to Hyperledger Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-tx-gas-cap-unsigned.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.mainnet;
|
|
|
|
import static org.hyperledger.besu.evm.account.Account.MAX_NONCE;
|
|
import static org.hyperledger.besu.evm.internal.Words.clampedAdd;
|
|
import static org.hyperledger.besu.evm.worldstate.CodeDelegationHelper.hasCodeDelegation;
|
|
|
|
import org.hyperledger.besu.crypto.SECPSignature;
|
|
import org.hyperledger.besu.crypto.SignatureAlgorithmFactory;
|
|
import org.hyperledger.besu.datatypes.BlobType;
|
|
import org.hyperledger.besu.datatypes.CodeDelegation;
|
|
import org.hyperledger.besu.datatypes.Hash;
|
|
import org.hyperledger.besu.datatypes.TransactionType;
|
|
import org.hyperledger.besu.datatypes.Wei;
|
|
import org.hyperledger.besu.ethereum.GasLimitCalculator;
|
|
import org.hyperledger.besu.ethereum.core.Transaction;
|
|
import org.hyperledger.besu.ethereum.mainnet.feemarket.FeeMarket;
|
|
import org.hyperledger.besu.ethereum.transaction.TransactionInvalidReason;
|
|
import org.hyperledger.besu.evm.account.Account;
|
|
import org.hyperledger.besu.evm.gascalculator.GasCalculator;
|
|
diff --git a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/WithdrawalsValidator.java b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/WithdrawalsValidator.java
|
|
index fc04a7c55b..2f4bff8200 100755
|
|
--- a/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/WithdrawalsValidator.java
|
|
+++ b/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/WithdrawalsValidator.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright contributors to Hyperledger Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-bft-withdrawals-not-applicable.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.mainnet;
|
|
|
|
import static com.google.common.base.Preconditions.checkArgument;
|
|
|
|
import org.hyperledger.besu.datatypes.Hash;
|
|
import org.hyperledger.besu.ethereum.core.Block;
|
|
import org.hyperledger.besu.ethereum.core.Withdrawal;
|
|
|
|
import java.util.List;
|
|
import java.util.Optional;
|
|
|
|
import org.slf4j.Logger;
|
|
import org.slf4j.LoggerFactory;
|
|
|
|
public interface WithdrawalsValidator {
|
|
|
|
boolean validateWithdrawals(Optional<List<Withdrawal>> withdrawals);
|
|
|
|
boolean validateWithdrawalsRoot(Block block);
|
|
diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java
|
|
index 92ec415271..11cc885291 100644
|
|
--- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java
|
|
+++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/EthPeers.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-cap-prestatus-rlpx-connections.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.eth.manager;
|
|
|
|
import org.hyperledger.besu.ethereum.core.BlockHeader;
|
|
import org.hyperledger.besu.ethereum.eth.EthProtocol;
|
|
import org.hyperledger.besu.ethereum.eth.SnapProtocol;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthPeer.DisconnectCallback;
|
|
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerSelector;
|
|
import org.hyperledger.besu.ethereum.eth.peervalidation.PeerValidator;
|
|
import org.hyperledger.besu.ethereum.eth.sync.ChainHeadTracker;
|
|
import org.hyperledger.besu.ethereum.eth.sync.SnapServerChecker;
|
|
import org.hyperledger.besu.ethereum.eth.sync.SyncMode;
|
|
import org.hyperledger.besu.ethereum.eth.sync.TrailingPeerRequirements;
|
|
import org.hyperledger.besu.ethereum.forkid.ForkId;
|
|
import org.hyperledger.besu.ethereum.forkid.ForkIdManager;
|
|
import org.hyperledger.besu.ethereum.mainnet.ProtocolSpec;
|
|
import org.hyperledger.besu.ethereum.p2p.peers.Peer;
|
|
import org.hyperledger.besu.ethereum.p2p.peers.PeerId;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.RlpxAgent;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnection;
|
|
diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManager.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManager.java
|
|
index 4078bebd2c..4954e7c2cc 100644
|
|
--- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManager.java
|
|
+++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManager.java
|
|
@@ -1,93 +1,132 @@
|
|
/*
|
|
* Copyright contributors to Hyperledger Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
+ *
|
|
+ * Modifications Copyright contributors to the Aere Network.
|
|
+ *
|
|
+ * This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
+ * Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.eth.manager.snap;
|
|
|
|
import org.hyperledger.besu.ethereum.ProtocolContext;
|
|
import org.hyperledger.besu.ethereum.core.Synchronizer;
|
|
import org.hyperledger.besu.ethereum.eth.SnapProtocol;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthMessage;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthMessages;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthPeer;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV1;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV2;
|
|
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
|
|
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
|
|
import org.hyperledger.besu.ethereum.p2p.network.ProtocolManager;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnection;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.framing.FramingException;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.AbstractSnapMessageData;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.Capability;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.Message;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.MessageData;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.messages.DisconnectMessage.DisconnectReason;
|
|
import org.hyperledger.besu.ethereum.rlp.RLPException;
|
|
import org.hyperledger.besu.ethereum.worldstate.WorldStateStorageCoordinator;
|
|
+import org.hyperledger.besu.metrics.BesuMetricCategory;
|
|
+import org.hyperledger.besu.plugin.services.MetricsSystem;
|
|
+import org.hyperledger.besu.plugin.services.metrics.Counter;
|
|
|
|
import java.math.BigInteger;
|
|
import java.util.Comparator;
|
|
import java.util.List;
|
|
import java.util.Map;
|
|
import java.util.Optional;
|
|
import java.util.concurrent.CancellationException;
|
|
+import java.util.concurrent.ConcurrentHashMap;
|
|
+import java.util.concurrent.atomic.AtomicInteger;
|
|
|
|
import com.google.common.collect.ImmutableList;
|
|
import org.slf4j.Logger;
|
|
import org.slf4j.LoggerFactory;
|
|
|
|
public class SnapProtocolManager implements ProtocolManager {
|
|
private static final Logger LOG = LoggerFactory.getLogger(SnapProtocolManager.class);
|
|
|
|
private final List<Capability> supportedCapabilities;
|
|
private final EthPeers ethPeers;
|
|
private final EthMessages snapMessages;
|
|
private final EthScheduler ethScheduler;
|
|
|
|
+ // AERE D-434 (683880baa6, GHSA-pcv4-pxhv-99m7): per-peer and global caps on snap GET_* requests
|
|
+ // concurrently scheduled on the EthScheduler services pool; 0 disables a cap.
|
|
+ private final int maxConcurrentRequestsPerPeer;
|
|
+ private final int maxConcurrentRequestsGlobal;
|
|
+ private final AtomicInteger globalInFlightRequests = new AtomicInteger(0);
|
|
+ private final Map<PeerConnection, AtomicInteger> perPeerInFlightRequests =
|
|
+ new ConcurrentHashMap<>();
|
|
+ private final Counter rejectedRequestsCounter;
|
|
+
|
|
public SnapProtocolManager(
|
|
final WorldStateStorageCoordinator worldStateStorageCoordinator,
|
|
final SnapSyncConfiguration snapConfig,
|
|
final EthPeers ethPeers,
|
|
final EthMessages snapMessages,
|
|
final EthScheduler ethScheduler,
|
|
final ProtocolSchedule protocolSchedule,
|
|
final ProtocolContext protocolContext,
|
|
- final Synchronizer synchronizer) {
|
|
+ final Synchronizer synchronizer,
|
|
+ final MetricsSystem metricsSystem) {
|
|
this.ethPeers = ethPeers;
|
|
this.snapMessages = snapMessages;
|
|
this.ethScheduler = ethScheduler;
|
|
this.supportedCapabilities = calculateCapabilities(protocolSchedule);
|
|
+ // AERE D-455 (2026-10-01): snapConfig can be null (the upstream controller-builder tests hand in an
|
|
+ // unset SynchronizerConfiguration; SnapServer already reads it through Optional.ofNullable). A
|
|
+ // null configuration takes the DEFAULT caps, never "no cap" and never a NullPointerException.
|
|
+ final SnapSyncConfiguration snapCaps =
|
|
+ snapConfig != null ? snapConfig : SnapSyncConfiguration.getDefault();
|
|
+ this.maxConcurrentRequestsPerPeer = snapCaps.getMaxConcurrentSnapRequestsPerPeer();
|
|
+ this.maxConcurrentRequestsGlobal = snapCaps.getMaxConcurrentSnapRequestsGlobal();
|
|
new SnapServer(
|
|
snapConfig, snapMessages, worldStateStorageCoordinator, protocolContext, synchronizer);
|
|
+
|
|
+ metricsSystem.createIntegerGauge(
|
|
+ BesuMetricCategory.PEERS,
|
|
+ "snap_service_requests_in_flight_current",
|
|
+ "The current number of snap sync GET_* requests concurrently scheduled for processing",
|
|
+ globalInFlightRequests::get);
|
|
+ this.rejectedRequestsCounter =
|
|
+ metricsSystem.createCounter(
|
|
+ BesuMetricCategory.PEERS,
|
|
+ "snap_service_requests_rejected_total",
|
|
+ "Total number of snap sync GET_* requests answered with an empty response because a concurrency cap was reached");
|
|
}
|
|
|
|
private List<Capability> calculateCapabilities(final ProtocolSchedule protocolSchedule) {
|
|
final ImmutableList.Builder<Capability> capabilities = ImmutableList.builder();
|
|
capabilities.add(SnapProtocol.SNAP1);
|
|
if (protocolSchedule.anyMatch(spec -> spec.spec().isBlockAccessListEnabled())) {
|
|
capabilities.add(SnapProtocol.SNAP2);
|
|
}
|
|
|
|
return capabilities.build();
|
|
}
|
|
|
|
@Override
|
|
public String getSupportedProtocol() {
|
|
return SnapProtocol.NAME;
|
|
}
|
|
|
|
@Override
|
|
public List<Capability> getSupportedCapabilities() {
|
|
return supportedCapabilities;
|
|
@@ -136,85 +175,189 @@ public class SnapProtocolManager implements ProtocolManager {
|
|
.log();
|
|
ethPeer.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
|
|
return;
|
|
}
|
|
final EthMessage decodedEthMessage = new EthMessage(ethPeer, messageData);
|
|
|
|
// Dispatch to pending response handlers (no-op for inbound requests).
|
|
ethPeers.dispatchMessage(ethPeer, decodedEthMessage, getSupportedProtocol());
|
|
|
|
// GET_* requests are handled off the Netty event loop to avoid blocking ETH protocol traffic.
|
|
if (SnapV1.REQUEST_CODES.contains(code) || SnapV2.REQUEST_CODES.contains(code)) {
|
|
scheduleSnapRequest(ethPeer, decodedEthMessage, cap, code);
|
|
}
|
|
}
|
|
|
|
private void scheduleSnapRequest(
|
|
final EthPeer ethPeer,
|
|
final EthMessage decodedEthMessage,
|
|
final Capability cap,
|
|
final int code) {
|
|
+ if (!reserveSnapRequestSlot(ethPeer)) {
|
|
+ respondEmptyDueToOverload(ethPeer, decodedEthMessage, code);
|
|
+ return;
|
|
+ }
|
|
+ try {
|
|
+ scheduleReservedSnapRequest(ethPeer, decodedEthMessage, cap, code);
|
|
+ } catch (final RuntimeException e) {
|
|
+ // Only reachable during shutdown: the services executor rejects synchronously once shut
|
|
+ // down. Release the slot so it isn't leaked.
|
|
+ releaseSnapRequestSlot(ethPeer);
|
|
+ throw e;
|
|
+ }
|
|
+ }
|
|
+
|
|
+ /** Cap was hit; reply empty instead of leaving the peer to time out. */
|
|
+ private void respondEmptyDueToOverload(
|
|
+ final EthPeer ethPeer, final EthMessage decodedEthMessage, final int code) {
|
|
+ final BigInteger requestId;
|
|
+ try {
|
|
+ requestId = decodedEthMessage.getData().unwrapMessageData().getKey();
|
|
+ } catch (final RLPException e) {
|
|
+ LOG.debug(
|
|
+ "Received malformed snap message code={} (BREACH_OF_PROTOCOL), disconnecting: {}",
|
|
+ code,
|
|
+ ethPeer,
|
|
+ e);
|
|
+ ethPeer.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
|
|
+ return;
|
|
+ }
|
|
+ sendSnapResponse(ethPeer, emptyResponseFor(code).wrapMessageData(requestId));
|
|
+ }
|
|
+
|
|
+ private static MessageData emptyResponseFor(final int code) {
|
|
+ return switch (code) {
|
|
+ case SnapV1.GET_ACCOUNT_RANGE -> SnapServer.EMPTY_ACCOUNT_RANGE;
|
|
+ case SnapV1.GET_STORAGE_RANGE -> SnapServer.EMPTY_STORAGE_RANGE;
|
|
+ case SnapV1.GET_BYTECODES -> SnapServer.EMPTY_BYTE_CODES_MESSAGE;
|
|
+ case SnapV1.GET_TRIE_NODES -> SnapServer.EMPTY_TRIE_NODES_MESSAGE;
|
|
+ case SnapV2.GET_BLOCK_ACCESS_LISTS -> SnapServer.EMPTY_BLOCK_ACCESS_LISTS;
|
|
+ default -> throw new IllegalStateException("Unhandled snap GET_* code: " + code);
|
|
+ };
|
|
+ }
|
|
+
|
|
+ private void scheduleReservedSnapRequest(
|
|
+ final EthPeer ethPeer,
|
|
+ final EthMessage decodedEthMessage,
|
|
+ final Capability cap,
|
|
+ final int code) {
|
|
ethScheduler
|
|
.scheduleServiceTask(
|
|
() -> {
|
|
Optional<MessageData> maybeResponseData = Optional.empty();
|
|
try {
|
|
final Map.Entry<BigInteger, MessageData> requestIdAndEthMessage =
|
|
decodedEthMessage.getData().unwrapMessageData();
|
|
maybeResponseData =
|
|
snapMessages
|
|
.dispatch(new EthMessage(ethPeer, requestIdAndEthMessage.getValue()), cap)
|
|
.map(
|
|
responseData ->
|
|
responseData.wrapMessageData(requestIdAndEthMessage.getKey()));
|
|
} catch (final FramingException | RLPException e) {
|
|
LOG.debug(
|
|
"Received malformed snap message code={} (BREACH_OF_PROTOCOL), disconnecting: {}",
|
|
code,
|
|
ethPeer,
|
|
e);
|
|
ethPeer.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
|
|
}
|
|
maybeResponseData.ifPresent(responseData -> sendSnapResponse(ethPeer, responseData));
|
|
})
|
|
.exceptionally(
|
|
e -> {
|
|
if (!(e instanceof CancellationException)) {
|
|
LOG.atWarn()
|
|
.setMessage("Unexpected error handling snap request code={} from peer {}")
|
|
.addArgument(code)
|
|
.addArgument(ethPeer::getLoggableId)
|
|
.setCause(e)
|
|
.log();
|
|
}
|
|
return null;
|
|
- });
|
|
+ })
|
|
+ .whenComplete((result, error) -> releaseSnapRequestSlot(ethPeer));
|
|
+ }
|
|
+
|
|
+ /**
|
|
+ * Reserves a global and per-peer slot before scheduling; increment-then-check avoids a TOCTOU
|
|
+ * race.
|
|
+ *
|
|
+ * @return true if reserved; false if a cap was hit (request answered empty instead).
|
|
+ */
|
|
+ private boolean reserveSnapRequestSlot(final EthPeer ethPeer) {
|
|
+ if (maxConcurrentRequestsGlobal > 0) {
|
|
+ final int reservedGlobal = globalInFlightRequests.incrementAndGet();
|
|
+ if (reservedGlobal > maxConcurrentRequestsGlobal) {
|
|
+ globalInFlightRequests.decrementAndGet();
|
|
+ rejectSnapRequest(ethPeer, "global");
|
|
+ return false;
|
|
+ }
|
|
+ }
|
|
+ if (maxConcurrentRequestsPerPeer > 0) {
|
|
+ final AtomicInteger perPeerCount =
|
|
+ perPeerInFlightRequests.computeIfAbsent(
|
|
+ ethPeer.getConnection(), unused -> new AtomicInteger(0));
|
|
+ final int reservedPerPeer = perPeerCount.incrementAndGet();
|
|
+ if (reservedPerPeer > maxConcurrentRequestsPerPeer) {
|
|
+ perPeerCount.decrementAndGet();
|
|
+ if (maxConcurrentRequestsGlobal > 0) {
|
|
+ globalInFlightRequests.decrementAndGet();
|
|
+ }
|
|
+ rejectSnapRequest(ethPeer, "per-peer");
|
|
+ return false;
|
|
+ }
|
|
+ }
|
|
+ return true;
|
|
+ }
|
|
+
|
|
+ private void releaseSnapRequestSlot(final EthPeer ethPeer) {
|
|
+ if (maxConcurrentRequestsGlobal > 0) {
|
|
+ globalInFlightRequests.decrementAndGet();
|
|
+ }
|
|
+ if (maxConcurrentRequestsPerPeer > 0) {
|
|
+ final AtomicInteger perPeerCount = perPeerInFlightRequests.get(ethPeer.getConnection());
|
|
+ if (perPeerCount != null) {
|
|
+ perPeerCount.decrementAndGet();
|
|
+ }
|
|
+ }
|
|
+ }
|
|
+
|
|
+ private void rejectSnapRequest(final EthPeer ethPeer, final String scope) {
|
|
+ rejectedRequestsCounter.inc();
|
|
+ LOG.atDebug()
|
|
+ .setMessage("Answering snap request from peer {} empty: {} concurrency cap reached")
|
|
+ .addArgument(ethPeer::getLoggableId)
|
|
+ .addArgument(scope)
|
|
+ .log();
|
|
}
|
|
|
|
private void sendSnapResponse(final EthPeer ethPeer, final MessageData responseData) {
|
|
try {
|
|
ethPeer.send(responseData, getSupportedProtocol());
|
|
} catch (final PeerConnection.PeerNotConnected e) {
|
|
LOG.atTrace()
|
|
.setMessage("Peer disconnected before we could respond - nothing to do {}")
|
|
.addArgument(e.getMessage())
|
|
.log();
|
|
}
|
|
}
|
|
|
|
@Override
|
|
public void handleNewConnection(final PeerConnection connection) {}
|
|
|
|
@Override
|
|
public void handleDisconnect(
|
|
final PeerConnection connection,
|
|
final DisconnectReason reason,
|
|
- final boolean initiatedByPeer) {}
|
|
+ final boolean initiatedByPeer) {
|
|
+ perPeerInFlightRequests.remove(connection);
|
|
+ }
|
|
|
|
@Override
|
|
public int getHighestProtocolVersion() {
|
|
return getSupportedCapabilities().stream()
|
|
.max(Comparator.comparing(Capability::getVersion))
|
|
.map(Capability::getVersion)
|
|
.orElse(0);
|
|
}
|
|
}
|
|
diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapServer.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapServer.java
|
|
index 0f9e796199..94a66f7bba 100644
|
|
--- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapServer.java
|
|
+++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapServer.java
|
|
@@ -1,33 +1,39 @@
|
|
/*
|
|
* Copyright contributors to Hyperledger Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
+ *
|
|
+ * Modifications Copyright contributors to the Aere Network.
|
|
+ *
|
|
+ * This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
+ * Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.eth.manager.snap;
|
|
|
|
import org.hyperledger.besu.datatypes.Hash;
|
|
import org.hyperledger.besu.ethereum.ProtocolContext;
|
|
import org.hyperledger.besu.ethereum.chain.Blockchain;
|
|
import org.hyperledger.besu.ethereum.core.Synchronizer;
|
|
import org.hyperledger.besu.ethereum.core.encoding.BlockAccessListEncoder;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthMessages;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.AccountRangeMessage;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.BlockAccessListsMessage;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.ByteCodesMessage;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.GetAccountRangeMessage;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.GetBlockAccessListsMessage;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.GetByteCodesMessage;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.GetStorageRangeMessage;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.GetTrieNodesMessage;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV1;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV2;
|
|
import org.hyperledger.besu.ethereum.eth.messages.snap.StorageRangeMessage;
|
|
@@ -60,48 +66,50 @@ import java.util.function.Predicate;
|
|
import java.util.stream.Collectors;
|
|
|
|
import com.google.common.annotations.VisibleForTesting;
|
|
import kotlin.Pair;
|
|
import kotlin.collections.ArrayDeque;
|
|
import org.apache.commons.lang3.time.StopWatch;
|
|
import org.apache.tuweni.bytes.Bytes;
|
|
import org.apache.tuweni.bytes.Bytes32;
|
|
import org.apache.tuweni.units.bigints.UInt256;
|
|
import org.slf4j.Logger;
|
|
import org.slf4j.LoggerFactory;
|
|
|
|
/** See <a href="https://github.com/ethereum/devp2p/blob/master/caps/snap.md">snap</a> */
|
|
class SnapServer implements BesuEvents.InitialSyncCompletionListener {
|
|
private static final Logger LOGGER = LoggerFactory.getLogger(SnapServer.class);
|
|
private static final int PRIME_STATE_ROOT_CACHE_LIMIT = 128;
|
|
private static final int MAX_ENTRIES_PER_REQUEST = 100000;
|
|
private static final int MAX_RESPONSE_SIZE = 2 * 1024 * 1024;
|
|
private static final int MAX_CODE_LOOKUPS_PER_REQUEST = 1024;
|
|
private static final int MAX_TRIE_LOOKUPS_PER_REQUEST = 1024;
|
|
- private static final AccountRangeMessage EMPTY_ACCOUNT_RANGE =
|
|
+ static final AccountRangeMessage EMPTY_ACCOUNT_RANGE =
|
|
AccountRangeMessage.create(new HashMap<>(), new ArrayDeque<>());
|
|
- private static final StorageRangeMessage EMPTY_STORAGE_RANGE =
|
|
+ static final StorageRangeMessage EMPTY_STORAGE_RANGE =
|
|
StorageRangeMessage.create(new ArrayDeque<>(), Collections.emptyList());
|
|
- private static final TrieNodesMessage EMPTY_TRIE_NODES_MESSAGE =
|
|
+ static final TrieNodesMessage EMPTY_TRIE_NODES_MESSAGE =
|
|
TrieNodesMessage.create(new ArrayList<>());
|
|
- private static final ByteCodesMessage EMPTY_BYTE_CODES_MESSAGE =
|
|
+ static final ByteCodesMessage EMPTY_BYTE_CODES_MESSAGE =
|
|
ByteCodesMessage.create(new ArrayDeque<>());
|
|
+ static final BlockAccessListsMessage EMPTY_BLOCK_ACCESS_LISTS =
|
|
+ BlockAccessListsMessage.create(List.of());
|
|
|
|
static final Hash HASH_LAST = Hash.wrap(Bytes32.leftPad(Bytes.fromHexString("FF"), (byte) 0xFF));
|
|
|
|
private final AtomicBoolean isStarted = new AtomicBoolean(false);
|
|
private final EthMessages snapMessages;
|
|
|
|
private final WorldStateStorageCoordinator worldStateStorageCoordinator;
|
|
private final Optional<ProtocolContext> protocolContext;
|
|
|
|
// whether snap server is enabled
|
|
private final boolean snapServerEnabled;
|
|
|
|
// max time per snap request
|
|
private final long maxMillisPerRequest;
|
|
|
|
// provide worldstate storage by root hash
|
|
private Function<Hash, Optional<BonsaiWorldStateKeyValueStorage>> worldStateStorageProvider =
|
|
__ -> Optional.empty();
|
|
|
|
SnapServer(
|
|
@@ -220,41 +228,41 @@ class SnapServer implements BesuEvents.InitialSyncCompletionListener {
|
|
private void registerResponseConstructors() {
|
|
snapMessages.registerResponseConstructor(
|
|
SnapV1.GET_ACCOUNT_RANGE,
|
|
(peer, messageData, capability) -> constructGetAccountRangeResponse(messageData));
|
|
snapMessages.registerResponseConstructor(
|
|
SnapV1.GET_STORAGE_RANGE,
|
|
(peer, messageData, capability) -> constructGetStorageRangeResponse(messageData));
|
|
snapMessages.registerResponseConstructor(
|
|
SnapV1.GET_BYTECODES,
|
|
(peer, messageData, capability) -> constructGetBytecodesResponse(messageData));
|
|
snapMessages.registerResponseConstructor(
|
|
SnapV1.GET_TRIE_NODES,
|
|
(peer, messageData, capability) -> constructGetTrieNodesResponse(messageData));
|
|
snapMessages.registerResponseConstructor(
|
|
SnapV2.GET_BLOCK_ACCESS_LISTS,
|
|
(peer, messageData, capability) -> constructGetBlockAccessListsResponse(messageData));
|
|
}
|
|
|
|
MessageData constructGetBlockAccessListsResponse(final MessageData message) {
|
|
if (!isStarted.get()) {
|
|
- return BlockAccessListsMessage.create(List.of());
|
|
+ return EMPTY_BLOCK_ACCESS_LISTS;
|
|
}
|
|
|
|
final GetBlockAccessListsMessage getBlockAccessLists =
|
|
GetBlockAccessListsMessage.readFrom(message);
|
|
final Iterable<Hash> blockHashes = getBlockAccessLists.blockHashes(true);
|
|
|
|
int responseSizeEstimate = RLP.MAX_PREFIX_SIZE;
|
|
final BytesValueRLPOutput rlp = new BytesValueRLPOutput();
|
|
rlp.startList();
|
|
|
|
final Optional<Blockchain> maybeBlockchain =
|
|
protocolContext.map(ProtocolContext::getBlockchain);
|
|
|
|
if (maybeBlockchain.isPresent()) {
|
|
final var blockchain = maybeBlockchain.get();
|
|
int count = 0;
|
|
for (final Hash blockHash : blockHashes) {
|
|
if (count >= MAX_ENTRIES_PER_REQUEST) {
|
|
break;
|
|
}
|
|
diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapSyncConfiguration.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapSyncConfiguration.java
|
|
index 5be69780fd..4cf279c8e5 100644
|
|
--- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapSyncConfiguration.java
|
|
+++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/snapsync/SnapSyncConfiguration.java
|
|
@@ -1,62 +1,85 @@
|
|
/*
|
|
* Copyright contributors to Hyperledger Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
+ *
|
|
+ * Modifications Copyright contributors to the Aere Network.
|
|
+ *
|
|
+ * This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
+ * Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.eth.sync.snapsync;
|
|
|
|
import org.immutables.value.Value;
|
|
|
|
@Value.Immutable
|
|
public class SnapSyncConfiguration {
|
|
|
|
// we use 126 and not the max value (128) to avoid sending requests that will be refused
|
|
public static final int DEFAULT_PIVOT_BLOCK_WINDOW_VALIDITY = 126;
|
|
public static final int DEFAULT_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING = 60;
|
|
|
|
public static final int DEFAULT_STORAGE_COUNT_PER_REQUEST =
|
|
384; // The default number of storage entries to download from peers per request.
|
|
public static final int DEFAULT_BYTECODE_COUNT_PER_REQUEST =
|
|
84; // The default number of code entries to download from peers per request.
|
|
public static final int DEFAULT_TRIENODE_COUNT_PER_REQUEST =
|
|
384; // The default number of trienode entries to download from peers per request.
|
|
public static final int DEFAULT_BLOCK_ACCESS_LIST_COUNT_PER_REQUEST =
|
|
84; // The default number of block access lists to download from peers per request.
|
|
|
|
public static final int DEFAULT_LOCAL_FLAT_ACCOUNT_COUNT_TO_HEAL_PER_REQUEST =
|
|
128; // The default number of flat accounts entries to verify and heal per request.
|
|
|
|
public static final int DEFAULT_LOCAL_FLAT_STORAGE_COUNT_TO_HEAL_PER_REQUEST =
|
|
1024; // The default number of flat slots entries to verify and heal per request.
|
|
|
|
public static final Boolean DEFAULT_SNAP_SERVER_ENABLED = Boolean.FALSE;
|
|
|
|
+ /**
|
|
+ * Default cap on the number of snap/1-2 GET_* requests from a single peer that may have a service
|
|
+ * task concurrently scheduled on the EthScheduler services pool. Sized with modest headroom above
|
|
+ * the number of outstanding requests we ourselves send a peer ({@code
|
|
+ * EthPeer.MAX_OUTSTANDING_REQUESTS = 5}). 0 disables the check. (AERE D-434, 683880baa6)
|
|
+ */
|
|
+ public static final int DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_PER_PEER = 8;
|
|
+
|
|
+ /**
|
|
+ * Default cap on the total number of snap/1-2 GET_* requests, across all peers, that may have a
|
|
+ * service task concurrently scheduled on the EthScheduler services pool, sized as {@code
|
|
+ * DEFAULT_MAX_PEERS (25) * DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_PER_PEER (8) = 200}. 0 disables the
|
|
+ * check. Operators who raise {@code --max-peers} well above the default should raise this value
|
|
+ * proportionally. (AERE D-434, 683880baa6)
|
|
+ */
|
|
+ public static final int DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_GLOBAL = 200;
|
|
+
|
|
public static final Boolean DEFAULT_SNAP_SYNC_TRANSACTION_INDEXING_ENABLED = Boolean.FALSE;
|
|
public static final Boolean DEFAULT_SNAP_SYNC_SAVE_PRE_MERGE_HEADERS_ONLY_ENABLED = Boolean.TRUE;
|
|
|
|
public static SnapSyncConfiguration getDefault() {
|
|
return ImmutableSnapSyncConfiguration.builder().build();
|
|
}
|
|
|
|
@Value.Default
|
|
public int getPivotBlockWindowValidity() {
|
|
return DEFAULT_PIVOT_BLOCK_WINDOW_VALIDITY;
|
|
}
|
|
|
|
@Value.Default
|
|
public int getPivotBlockDistanceBeforeCaching() {
|
|
return DEFAULT_PIVOT_BLOCK_DISTANCE_BEFORE_CACHING;
|
|
}
|
|
|
|
@Value.Default
|
|
public int getStorageCountPerRequest() {
|
|
return DEFAULT_STORAGE_COUNT_PER_REQUEST;
|
|
@@ -75,25 +98,35 @@ public class SnapSyncConfiguration {
|
|
@Value.Default
|
|
public int getBlockAccessListCountPerRequest() {
|
|
return DEFAULT_BLOCK_ACCESS_LIST_COUNT_PER_REQUEST;
|
|
}
|
|
|
|
@Value.Default
|
|
public int getLocalFlatAccountCountToHealPerRequest() {
|
|
return DEFAULT_LOCAL_FLAT_ACCOUNT_COUNT_TO_HEAL_PER_REQUEST;
|
|
}
|
|
|
|
@Value.Default
|
|
public int getLocalFlatStorageCountToHealPerRequest() {
|
|
return DEFAULT_LOCAL_FLAT_STORAGE_COUNT_TO_HEAL_PER_REQUEST;
|
|
}
|
|
|
|
@Value.Default
|
|
public Boolean isSnapServerEnabled() {
|
|
return DEFAULT_SNAP_SERVER_ENABLED;
|
|
}
|
|
|
|
+ @Value.Default
|
|
+ public int getMaxConcurrentSnapRequestsPerPeer() {
|
|
+ return DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_PER_PEER;
|
|
+ }
|
|
+
|
|
+ @Value.Default
|
|
+ public int getMaxConcurrentSnapRequestsGlobal() {
|
|
+ return DEFAULT_MAX_CONCURRENT_SNAP_REQUESTS_GLOBAL;
|
|
+ }
|
|
+
|
|
@Value.Default
|
|
public Boolean isSnapSyncTransactionIndexingEnabled() {
|
|
return DEFAULT_SNAP_SYNC_TRANSACTION_INDEXING_ENABLED;
|
|
}
|
|
}
|
|
diff --git a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java
|
|
index 9d5cdadf39..40256dbcb0 100755
|
|
--- a/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java
|
|
+++ b/ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncState.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-syncstate-in-sync-lock.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.eth.sync.state;
|
|
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.ethereum.chain.Blockchain;
|
|
import org.hyperledger.besu.ethereum.chain.ChainHead;
|
|
import org.hyperledger.besu.ethereum.core.BlockHeader;
|
|
import org.hyperledger.besu.ethereum.core.DefaultSyncStatus;
|
|
import org.hyperledger.besu.ethereum.core.Synchronizer;
|
|
import org.hyperledger.besu.ethereum.core.Synchronizer.InSyncListener;
|
|
import org.hyperledger.besu.ethereum.eth.manager.ChainHeadEstimate;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthPeer;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
|
|
import org.hyperledger.besu.ethereum.eth.sync.common.checkpoint.Checkpoint;
|
|
import org.hyperledger.besu.ethereum.eth.sync.worldstate.WorldStateDownloadStatus;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.messages.DisconnectMessage.DisconnectReason;
|
|
import org.hyperledger.besu.plugin.data.SyncStatus;
|
|
import org.hyperledger.besu.plugin.services.BesuEvents.InitialSyncCompletionListener;
|
|
import org.hyperledger.besu.plugin.services.BesuEvents.SyncStatusListener;
|
|
import org.hyperledger.besu.plugin.services.BesuEvents.TTDReachedListener;
|
|
diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/AereD434SnapCapTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/AereD434SnapCapTest.java
|
|
new file mode 100644
|
|
index 0000000000..fb1dc7cea8
|
|
--- /dev/null
|
|
+++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/AereD434SnapCapTest.java
|
|
@@ -0,0 +1,341 @@
|
|
+/*
|
|
+ * Copyright contributors to Besu.
|
|
+ *
|
|
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
+ * the License. You may obtain a copy of the License at
|
|
+ *
|
|
+ * http://www.apache.org/licenses/LICENSE-2.0
|
|
+ *
|
|
+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
+ * specific language governing permissions and limitations under the License.
|
|
+ *
|
|
+ * SPDX-License-Identifier: Apache-2.0
|
|
+ *
|
|
+ * Modified by Aere Network (2026-09-29, D-434): the snap concurrency-cap tests of hyperledger/besu 683880baa6
|
|
+ * (GHSA-pcv4-pxhv-99m7, #11101), ported as a separate class onto this fork's SnapProtocolManager constructor, which
|
|
+ * keeps ProtocolSchedule (capabilities are derived from it here) and takes MetricsSystem last.
|
|
+ */
|
|
+package org.hyperledger.besu.ethereum.eth.manager.snap;
|
|
+
|
|
+import static org.assertj.core.api.Assertions.assertThat;
|
|
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
|
+import static org.mockito.ArgumentMatchers.any;
|
|
+import static org.mockito.ArgumentMatchers.eq;
|
|
+import static org.mockito.Mockito.mock;
|
|
+import static org.mockito.Mockito.never;
|
|
+import static org.mockito.Mockito.times;
|
|
+import static org.mockito.Mockito.verify;
|
|
+import static org.mockito.Mockito.when;
|
|
+
|
|
+import org.hyperledger.besu.datatypes.Hash;
|
|
+import org.hyperledger.besu.ethereum.ProtocolContext;
|
|
+import org.hyperledger.besu.ethereum.core.Synchronizer;
|
|
+import org.hyperledger.besu.ethereum.eth.SnapProtocol;
|
|
+import org.hyperledger.besu.ethereum.eth.manager.EthMessages;
|
|
+import org.hyperledger.besu.ethereum.eth.manager.EthPeer;
|
|
+import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
|
|
+import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
|
|
+import org.hyperledger.besu.ethereum.eth.manager.MockPeerConnection;
|
|
+import org.hyperledger.besu.ethereum.eth.messages.snap.GetTrieNodesMessage;
|
|
+import org.hyperledger.besu.ethereum.eth.messages.snap.SnapV1;
|
|
+import org.hyperledger.besu.ethereum.eth.messages.snap.TrieNodesMessage;
|
|
+import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
|
|
+import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
|
|
+import org.hyperledger.besu.ethereum.p2p.rlpx.connections.PeerConnection;
|
|
+import org.hyperledger.besu.ethereum.p2p.rlpx.wire.DefaultMessage;
|
|
+import org.hyperledger.besu.ethereum.p2p.rlpx.wire.Message;
|
|
+import org.hyperledger.besu.ethereum.p2p.rlpx.wire.MessageData;
|
|
+import org.hyperledger.besu.ethereum.p2p.rlpx.wire.RawMessage;
|
|
+import org.hyperledger.besu.ethereum.p2p.rlpx.wire.messages.DisconnectMessage.DisconnectReason;
|
|
+import org.hyperledger.besu.ethereum.worldstate.WorldStateStorageCoordinator;
|
|
+import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
|
|
+
|
|
+import java.math.BigInteger;
|
|
+import java.util.ArrayList;
|
|
+import java.util.Collections;
|
|
+import java.util.HashSet;
|
|
+import java.util.List;
|
|
+import java.util.concurrent.CompletableFuture;
|
|
+import java.util.concurrent.RejectedExecutionException;
|
|
+
|
|
+import org.apache.tuweni.bytes.Bytes;
|
|
+import org.junit.jupiter.api.BeforeEach;
|
|
+import org.junit.jupiter.api.Test;
|
|
+import org.junit.jupiter.api.extension.ExtendWith;
|
|
+import org.mockito.ArgumentCaptor;
|
|
+import org.mockito.Mock;
|
|
+import org.mockito.junit.jupiter.MockitoExtension;
|
|
+import org.mockito.junit.jupiter.MockitoSettings;
|
|
+import org.mockito.quality.Strictness;
|
|
+
|
|
+@ExtendWith(MockitoExtension.class)
|
|
+@MockitoSettings(strictness = Strictness.LENIENT)
|
|
+class AereD434SnapCapTest {
|
|
+
|
|
+ @Mock private WorldStateStorageCoordinator worldStateStorageCoordinator;
|
|
+ @Mock private SnapSyncConfiguration snapConfig;
|
|
+ @Mock private EthPeers ethPeers;
|
|
+ @Mock private EthMessages snapMessages;
|
|
+ @Mock private ProtocolSchedule protocolSchedule;
|
|
+ @Mock private ProtocolContext protocolContext;
|
|
+ @Mock private Synchronizer synchronizer;
|
|
+ @Mock private EthScheduler ethScheduler;
|
|
+ @Mock private EthPeer ethPeer;
|
|
+
|
|
+ private SnapProtocolManager snapProtocolManager;
|
|
+
|
|
+ @BeforeEach
|
|
+ void setUp() {
|
|
+ when(snapConfig.isSnapServerEnabled()).thenReturn(false);
|
|
+ }
|
|
+
|
|
+ /** The defaults are the upstream ones, and both caps are on unless configured to 0. */
|
|
+ @Test
|
|
+ void defaultsAreEightPerPeerAndTwoHundredGlobal() {
|
|
+ final SnapSyncConfiguration defaults = SnapSyncConfiguration.getDefault();
|
|
+ assertThat(defaults.getMaxConcurrentSnapRequestsPerPeer()).isEqualTo(8);
|
|
+ assertThat(defaults.getMaxConcurrentSnapRequestsGlobal()).isEqualTo(200);
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void limitsConcurrentSnapRequestsPerPeer() {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(2);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+
|
|
+ final MockPeerConnection peerConnection = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnection);
|
|
+ stubPendingServiceTasks();
|
|
+
|
|
+ for (int i = 0; i < 5; i++) {
|
|
+ snapProtocolManager.processMessage(
|
|
+ SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, i));
|
|
+ }
|
|
+
|
|
+ // Only the first 2 requests are scheduled; the remaining 3 are answered empty once the
|
|
+ // per-peer cap is reached.
|
|
+ verify(ethScheduler, times(2)).scheduleServiceTask(any(Runnable.class));
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void perPeerCapDoesNotBlockADistinctPeer() {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(2);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+
|
|
+ final MockPeerConnection peerConnectionA = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnectionA);
|
|
+ final EthPeer ethPeerB = mock(EthPeer.class);
|
|
+ final MockPeerConnection peerConnectionB = snapPeerConnection();
|
|
+ stubPeer(ethPeerB, peerConnectionB);
|
|
+ stubPendingServiceTasks();
|
|
+
|
|
+ for (int i = 0; i < 2; i++) {
|
|
+ snapProtocolManager.processMessage(
|
|
+ SnapProtocol.SNAP1, getTrieNodesMessage(peerConnectionA, i));
|
|
+ snapProtocolManager.processMessage(
|
|
+ SnapProtocol.SNAP1, getTrieNodesMessage(peerConnectionB, i));
|
|
+ }
|
|
+
|
|
+ verify(ethScheduler, times(4)).scheduleServiceTask(any(Runnable.class));
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void limitsConcurrentSnapRequestsGlobally() {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsGlobal()).thenReturn(3);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+ stubPendingServiceTasks();
|
|
+
|
|
+ for (int peerIndex = 0; peerIndex < 3; peerIndex++) {
|
|
+ final EthPeer peer = mock(EthPeer.class);
|
|
+ final MockPeerConnection peerConnection = snapPeerConnection();
|
|
+ stubPeer(peer, peerConnection);
|
|
+ for (int i = 0; i < 2; i++) {
|
|
+ snapProtocolManager.processMessage(
|
|
+ SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, i));
|
|
+ }
|
|
+ }
|
|
+
|
|
+ verify(ethScheduler, times(3)).scheduleServiceTask(any(Runnable.class));
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void releasesSlotWhenScheduledTaskCompletes() {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+
|
|
+ final MockPeerConnection peerConnection = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnection);
|
|
+ final List<CompletableFuture<Void>> scheduledTasks = new ArrayList<>();
|
|
+ when(ethScheduler.scheduleServiceTask(any(Runnable.class)))
|
|
+ .thenAnswer(
|
|
+ invocation -> {
|
|
+ final CompletableFuture<Void> future = new CompletableFuture<>();
|
|
+ scheduledTasks.add(future);
|
|
+ return future;
|
|
+ });
|
|
+
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0));
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 1));
|
|
+ verify(ethScheduler, times(1)).scheduleServiceTask(any(Runnable.class));
|
|
+
|
|
+ scheduledTasks.get(0).complete(null);
|
|
+
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 2));
|
|
+ verify(ethScheduler, times(2)).scheduleServiceTask(any(Runnable.class));
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void releasesSlotWhenSchedulingThrowsSynchronously() {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+
|
|
+ final MockPeerConnection peerConnection = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnection);
|
|
+ when(ethScheduler.scheduleServiceTask(any(Runnable.class)))
|
|
+ .thenThrow(new RejectedExecutionException("executor is shutting down"))
|
|
+ .thenAnswer(invocation -> new CompletableFuture<Void>());
|
|
+
|
|
+ assertThatThrownBy(
|
|
+ () ->
|
|
+ snapProtocolManager.processMessage(
|
|
+ SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0)))
|
|
+ .isInstanceOf(RejectedExecutionException.class);
|
|
+
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 1));
|
|
+ verify(ethScheduler, times(2)).scheduleServiceTask(any(Runnable.class));
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void clearsPeerSlotsOnDisconnectPreventingALeak() {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+
|
|
+ final MockPeerConnection peerConnection = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnection);
|
|
+ stubPendingServiceTasks();
|
|
+
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0));
|
|
+ verify(ethScheduler, times(1)).scheduleServiceTask(any(Runnable.class));
|
|
+
|
|
+ snapProtocolManager.handleDisconnect(
|
|
+ peerConnection, DisconnectReason.TCP_SUBSYSTEM_ERROR, false);
|
|
+
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 1));
|
|
+ verify(ethScheduler, times(2)).scheduleServiceTask(any(Runnable.class));
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void respondsEmptyWhenPerPeerCapReached() throws PeerConnection.PeerNotConnected {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+
|
|
+ final MockPeerConnection peerConnection = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnection);
|
|
+ stubPendingServiceTasks();
|
|
+
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0));
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 1));
|
|
+
|
|
+ verify(ethScheduler, times(1)).scheduleServiceTask(any(Runnable.class));
|
|
+ final ArgumentCaptor<MessageData> sent = ArgumentCaptor.forClass(MessageData.class);
|
|
+ verify(ethPeer).send(sent.capture(), eq(SnapProtocol.NAME));
|
|
+ assertThat(sent.getValue().getCode()).isEqualTo(SnapV1.TRIE_NODES);
|
|
+ assertThat(TrieNodesMessage.readFrom(sent.getValue()).nodes(true)).isEmpty();
|
|
+ assertThat(sent.getValue().unwrapMessageData().getKey()).isEqualTo(BigInteger.ONE);
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void respondsEmptyWhenGlobalCapReached() throws PeerConnection.PeerNotConnected {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsGlobal()).thenReturn(1);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+ stubPendingServiceTasks();
|
|
+
|
|
+ final MockPeerConnection peerConnectionA = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnectionA);
|
|
+ final EthPeer ethPeerB = mock(EthPeer.class);
|
|
+ final MockPeerConnection peerConnectionB = snapPeerConnection();
|
|
+ stubPeer(ethPeerB, peerConnectionB);
|
|
+
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnectionA, 0));
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnectionB, 7));
|
|
+
|
|
+ verify(ethScheduler, times(1)).scheduleServiceTask(any(Runnable.class));
|
|
+ final ArgumentCaptor<MessageData> sent = ArgumentCaptor.forClass(MessageData.class);
|
|
+ verify(ethPeerB).send(sent.capture(), eq(SnapProtocol.NAME));
|
|
+ assertThat(sent.getValue().unwrapMessageData().getKey()).isEqualTo(BigInteger.valueOf(7));
|
|
+ verify(ethPeer, never()).send(any(), any());
|
|
+ }
|
|
+
|
|
+ @Test
|
|
+ void disconnectsPeerWhenRejectedRequestIsMalformed() throws PeerConnection.PeerNotConnected {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(1);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+
|
|
+ final MockPeerConnection peerConnection = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnection);
|
|
+ stubPendingServiceTasks();
|
|
+
|
|
+ snapProtocolManager.processMessage(SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, 0));
|
|
+
|
|
+ final MessageData malformed = new RawMessage(SnapV1.GET_TRIE_NODES, Bytes.of(0x01));
|
|
+ snapProtocolManager.processMessage(
|
|
+ SnapProtocol.SNAP1, new DefaultMessage(peerConnection, malformed));
|
|
+
|
|
+ verify(ethPeer).disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
|
|
+ verify(ethPeer, never()).send(any(), any());
|
|
+ }
|
|
+
|
|
+ /** With both caps at 0 (disabled) every request is scheduled, as before the fix. */
|
|
+ @Test
|
|
+ void zeroDisablesBothCaps() {
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsPerPeer()).thenReturn(0);
|
|
+ when(snapConfig.getMaxConcurrentSnapRequestsGlobal()).thenReturn(0);
|
|
+ snapProtocolManager = createSnapProtocolManager();
|
|
+
|
|
+ final MockPeerConnection peerConnection = snapPeerConnection();
|
|
+ stubPeer(ethPeer, peerConnection);
|
|
+ stubPendingServiceTasks();
|
|
+
|
|
+ for (int i = 0; i < 20; i++) {
|
|
+ snapProtocolManager.processMessage(
|
|
+ SnapProtocol.SNAP1, getTrieNodesMessage(peerConnection, i));
|
|
+ }
|
|
+ verify(ethScheduler, times(20)).scheduleServiceTask(any(Runnable.class));
|
|
+ }
|
|
+
|
|
+ private void stubPeer(final EthPeer peer, final PeerConnection connection) {
|
|
+ when(ethPeers.peer(connection)).thenReturn(peer);
|
|
+ when(peer.getConnection()).thenReturn(connection);
|
|
+ when(peer.validateReceivedMessage(any(), any())).thenReturn(true);
|
|
+ }
|
|
+
|
|
+ private void stubPendingServiceTasks() {
|
|
+ when(ethScheduler.scheduleServiceTask(any(Runnable.class)))
|
|
+ .thenAnswer(invocation -> new CompletableFuture<Void>());
|
|
+ }
|
|
+
|
|
+ private MockPeerConnection snapPeerConnection() {
|
|
+ return new MockPeerConnection(
|
|
+ new HashSet<>(Collections.singletonList(SnapProtocol.SNAP1)), (cap, msg, conn) -> {});
|
|
+ }
|
|
+
|
|
+ private Message getTrieNodesMessage(final PeerConnection peerConnection, final int requestId) {
|
|
+ final MessageData data =
|
|
+ GetTrieNodesMessage.create(Hash.ZERO, List.of(List.of(Bytes.EMPTY)))
|
|
+ .wrapMessageData(BigInteger.valueOf(requestId));
|
|
+ return new DefaultMessage(peerConnection, data);
|
|
+ }
|
|
+
|
|
+ private SnapProtocolManager createSnapProtocolManager() {
|
|
+ return new SnapProtocolManager(
|
|
+ worldStateStorageCoordinator,
|
|
+ snapConfig,
|
|
+ ethPeers,
|
|
+ snapMessages,
|
|
+ ethScheduler,
|
|
+ protocolSchedule,
|
|
+ protocolContext,
|
|
+ synchronizer,
|
|
+ new NoOpMetricsSystem());
|
|
+ }
|
|
+}
|
|
diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManagerTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManagerTest.java
|
|
index 997fcb43b9..caa89d1e1a 100644
|
|
--- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManagerTest.java
|
|
+++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/manager/snap/SnapProtocolManagerTest.java
|
|
@@ -1,33 +1,39 @@
|
|
/*
|
|
* Copyright contributors to Hyperledger Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
+ *
|
|
+ * Modifications Copyright contributors to the Aere Network.
|
|
+ *
|
|
+ * This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
+ * Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0009-aere-pq-seal-judge-snap-cap.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.eth.manager.snap;
|
|
|
|
import static org.assertj.core.api.Assertions.assertThat;
|
|
import static org.mockito.ArgumentMatchers.any;
|
|
import static org.mockito.Mockito.when;
|
|
|
|
import org.hyperledger.besu.ethereum.ProtocolContext;
|
|
import org.hyperledger.besu.ethereum.core.Synchronizer;
|
|
import org.hyperledger.besu.ethereum.eth.SnapProtocol;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthMessages;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthPeer;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
|
|
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
|
|
import org.hyperledger.besu.ethereum.eth.manager.MockPeerConnection;
|
|
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapSyncConfiguration;
|
|
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.DefaultMessage;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.RawMessage;
|
|
import org.hyperledger.besu.ethereum.p2p.rlpx.wire.messages.DisconnectMessage.DisconnectReason;
|
|
@@ -55,41 +61,42 @@ class SnapProtocolManagerTest {
|
|
@Mock private ProtocolSchedule protocolSchedule;
|
|
@Mock private ProtocolContext protocolContext;
|
|
@Mock private Synchronizer synchronizer;
|
|
@Mock private EthScheduler ethScheduler;
|
|
@Mock private EthPeer ethPeer;
|
|
|
|
private SnapProtocolManager snapProtocolManager;
|
|
|
|
@BeforeEach
|
|
void setUp() {
|
|
when(snapConfig.isSnapServerEnabled()).thenReturn(false);
|
|
snapProtocolManager =
|
|
new SnapProtocolManager(
|
|
worldStateStorageCoordinator,
|
|
snapConfig,
|
|
ethPeers,
|
|
snapMessages,
|
|
ethScheduler,
|
|
protocolSchedule,
|
|
protocolContext,
|
|
- synchronizer);
|
|
+ synchronizer,
|
|
+ new org.hyperledger.besu.metrics.noop.NoOpMetricsSystem());
|
|
}
|
|
|
|
@Test
|
|
void disconnectsPeerOnDecompressionFailure() {
|
|
final MockPeerConnection peerConnection =
|
|
new MockPeerConnection(
|
|
new HashSet<>(Collections.singletonList(SnapProtocol.SNAP1)), (cap, msg, conn) -> {});
|
|
when(ethPeers.peer(peerConnection)).thenReturn(ethPeer);
|
|
when(ethPeer.validateReceivedMessage(any(), any())).thenReturn(true);
|
|
|
|
// Create a RawMessage with invalid compressed data that will throw FramingException
|
|
final RawMessage badMessage = new RawMessage(0x00, new byte[] {0x01, 0x02, 0x03});
|
|
snapProtocolManager.processMessage(
|
|
SnapProtocol.SNAP1, new DefaultMessage(peerConnection, badMessage));
|
|
|
|
assertThat(peerConnection.isDisconnected()).isFalse();
|
|
// ethPeer (mock) receives the disconnect call
|
|
org.mockito.Mockito.verify(ethPeer)
|
|
.disconnect(DisconnectReason.BREACH_OF_PROTOCOL_MALFORMED_MESSAGE_RECEIVED);
|
|
}
|
|
diff --git a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java
|
|
index 5b58648df1..d8498f37fd 100644
|
|
--- a/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java
|
|
+++ b/ethereum/eth/src/test/java/org/hyperledger/besu/ethereum/eth/sync/state/SyncStateTest.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-syncstate-in-sync-lock.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.eth.sync.state;
|
|
|
|
import static org.assertj.core.api.Assertions.assertThat;
|
|
import static org.mockito.ArgumentMatchers.any;
|
|
import static org.mockito.ArgumentMatchers.anyBoolean;
|
|
import static org.mockito.Mockito.doReturn;
|
|
import static org.mockito.Mockito.lenient;
|
|
import static org.mockito.Mockito.mock;
|
|
import static org.mockito.Mockito.never;
|
|
import static org.mockito.Mockito.spy;
|
|
import static org.mockito.Mockito.times;
|
|
import static org.mockito.Mockito.verify;
|
|
import static org.mockito.Mockito.verifyNoMoreInteractions;
|
|
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.datatypes.Hash;
|
|
import org.hyperledger.besu.ethereum.chain.MutableBlockchain;
|
|
import org.hyperledger.besu.ethereum.core.Block;
|
|
import org.hyperledger.besu.ethereum.core.BlockDataGenerator;
|
|
diff --git a/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java b/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java
|
|
index 695f088181..7401d1ae50 100755
|
|
--- a/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java
|
|
+++ b/ethereum/rlp/src/main/java/org/hyperledger/besu/ethereum/rlp/RLPInput.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-bft-bounded-message-lists.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.ethereum.rlp;
|
|
|
|
import java.math.BigInteger;
|
|
import java.net.InetAddress;
|
|
import java.util.ArrayList;
|
|
import java.util.List;
|
|
import java.util.function.Function;
|
|
|
|
import org.apache.tuweni.bytes.Bytes;
|
|
import org.apache.tuweni.bytes.Bytes32;
|
|
import org.apache.tuweni.bytes.Bytes48;
|
|
import org.apache.tuweni.units.bigints.UInt256;
|
|
import org.apache.tuweni.units.bigints.UInt64;
|
|
|
|
/**
|
|
* An input used to decode data in RLP encoding.
|
|
*
|
|
* <p>An RLP "value" is fundamentally an {@code Item} defined the following way:
|
|
*
|
|
diff --git a/evm/src/main/java/org/hyperledger/besu/evm/frame/MessageFrame.java b/evm/src/main/java/org/hyperledger/besu/evm/frame/MessageFrame.java
|
|
index eadf4b3c49..59c16a5215 100644
|
|
--- a/evm/src/main/java/org/hyperledger/besu/evm/frame/MessageFrame.java
|
|
+++ b/evm/src/main/java/org/hyperledger/besu/evm/frame/MessageFrame.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright ConsenSys AG.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-warm-access-hashdos-treeset.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.evm.frame;
|
|
|
|
import static com.google.common.base.Preconditions.checkState;
|
|
import static java.util.Collections.emptySet;
|
|
|
|
import org.hyperledger.besu.collections.undo.UndoSet;
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.datatypes.Log;
|
|
import org.hyperledger.besu.datatypes.VersionedHash;
|
|
import org.hyperledger.besu.datatypes.Wei;
|
|
import org.hyperledger.besu.evm.Code;
|
|
import org.hyperledger.besu.evm.blockhash.BlockHashLookup;
|
|
import org.hyperledger.besu.evm.internal.MemoryEntry;
|
|
import org.hyperledger.besu.evm.internal.OperandStack;
|
|
import org.hyperledger.besu.evm.internal.StorageEntry;
|
|
import org.hyperledger.besu.evm.internal.UnderflowException;
|
|
import org.hyperledger.besu.evm.operation.Operation;
|
|
import org.hyperledger.besu.evm.worldstate.WorldUpdater;
|
|
|
|
diff --git a/evm/src/main/java/org/hyperledger/besu/evm/frame/TxValues.java b/evm/src/main/java/org/hyperledger/besu/evm/frame/TxValues.java
|
|
index b13a96f5bb..f9ad92e1f1 100644
|
|
--- a/evm/src/main/java/org/hyperledger/besu/evm/frame/TxValues.java
|
|
+++ b/evm/src/main/java/org/hyperledger/besu/evm/frame/TxValues.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright contributors to Hyperledger Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-warm-access-hashdos-treeset.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.evm.frame;
|
|
|
|
import org.hyperledger.besu.collections.undo.UndoScalar;
|
|
import org.hyperledger.besu.collections.undo.UndoSet;
|
|
import org.hyperledger.besu.collections.undo.UndoTable;
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.datatypes.VersionedHash;
|
|
import org.hyperledger.besu.datatypes.Wei;
|
|
import org.hyperledger.besu.evm.blockhash.BlockHashLookup;
|
|
|
|
import java.util.ArrayDeque;
|
|
import java.util.Deque;
|
|
import java.util.TreeSet;
|
|
import java.util.List;
|
|
import java.util.Optional;
|
|
|
|
import com.google.common.collect.TreeBasedTable;
|
|
import org.apache.tuweni.bytes.Bytes32;
|
|
|
|
diff --git a/evm/src/main/java/org/hyperledger/besu/evm/precompile/MainnetPrecompiledContracts.java b/evm/src/main/java/org/hyperledger/besu/evm/precompile/MainnetPrecompiledContracts.java
|
|
index b4ae37bb95..fcb5cc87f9 100644
|
|
--- a/evm/src/main/java/org/hyperledger/besu/evm/precompile/MainnetPrecompiledContracts.java
|
|
+++ b/evm/src/main/java/org/hyperledger/besu/evm/precompile/MainnetPrecompiledContracts.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright contributors to Hyperledger Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/0001-aere-pqc-precompiles-mainnet.patch.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0001-aere-pqc-precompiles-mainnet.patch.
|
|
*/
|
|
package org.hyperledger.besu.evm.precompile;
|
|
|
|
import static org.hyperledger.besu.datatypes.Address.P256_VERIFY;
|
|
|
|
import org.hyperledger.besu.datatypes.Address;
|
|
import org.hyperledger.besu.evm.gascalculator.GasCalculator;
|
|
|
|
/** Provides the various precompiled contracts used on mainnet hard forks. */
|
|
public interface MainnetPrecompiledContracts {
|
|
|
|
/**
|
|
* Frontier precompile contract registry.
|
|
*
|
|
* @param gasCalculator the gas calculator
|
|
* @return the precompile contract registry
|
|
*/
|
|
static PrecompileContractRegistry frontier(final GasCalculator gasCalculator) {
|
|
PrecompileContractRegistry precompileContractRegistry = new PrecompileContractRegistry();
|
|
populateForFrontier(precompileContractRegistry, gasCalculator);
|
|
diff --git a/evm/src/main/java/org/hyperledger/besu/evm/precompile/P256VerifyPrecompiledContract.java b/evm/src/main/java/org/hyperledger/besu/evm/precompile/P256VerifyPrecompiledContract.java
|
|
index 220194b6ab..fe0b01a825 100644
|
|
--- a/evm/src/main/java/org/hyperledger/besu/evm/precompile/P256VerifyPrecompiledContract.java
|
|
+++ b/evm/src/main/java/org/hyperledger/besu/evm/precompile/P256VerifyPrecompiledContract.java
|
|
@@ -1,39 +1,39 @@
|
|
/*
|
|
* Copyright contributors to Besu.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
* the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
* specific language governing permissions and limitations under the License.
|
|
*
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
*
|
|
* Modifications Copyright contributors to the Aere Network.
|
|
*
|
|
* This file was modified by contributors to the Aere Network, as required by section 4(b) of the
|
|
* Apache License 2.0. The copyright header above is the upstream one and is left exactly as it was
|
|
- * found, as section 4(c) requires. The change itself is in patches/aere-drop-per-request-console-writes.patch.
|
|
+ * found, as section 4(c) requires. The change itself is in patches/0008-aere-pq-aip22.patch.
|
|
*/
|
|
package org.hyperledger.besu.evm.precompile;
|
|
|
|
import org.hyperledger.besu.crypto.SECP256R1;
|
|
import org.hyperledger.besu.crypto.SECPPublicKey;
|
|
import org.hyperledger.besu.crypto.SECPSignature;
|
|
import org.hyperledger.besu.crypto.SignatureAlgorithm;
|
|
import org.hyperledger.besu.evm.frame.MessageFrame;
|
|
import org.hyperledger.besu.evm.gascalculator.GasCalculator;
|
|
import org.hyperledger.besu.nativelib.boringssl.BoringSSLPrecompiles;
|
|
|
|
import java.math.BigInteger;
|
|
|
|
import com.github.benmanes.caffeine.cache.Cache;
|
|
import com.github.benmanes.caffeine.cache.Caffeine;
|
|
import jakarta.validation.constraints.NotNull;
|
|
import org.apache.tuweni.bytes.Bytes;
|
|
import org.apache.tuweni.bytes.Bytes32;
|
|
import org.bouncycastle.asn1.sec.SECNamedCurves;
|
|
import org.bouncycastle.asn1.x9.X9ECParameters;
|