diff --git a/RUN-A-NODE.md b/RUN-A-NODE.md index c24e3c7..c85997e 100644 --- a/RUN-A-NODE.md +++ b/RUN-A-NODE.md @@ -49,7 +49,7 @@ audit each step rather than run it. |---|---| | `genesis.json` | The genesis file, byte for byte as it sits on the Foundation nodes. sha256 `e5038e6a1437bd85fa51294609edf2126a997f14f4fa7a65a0d39bd0aa24d2e5` (of the published file with LF line endings, exactly as `curl` fetches it; a Windows checkout that converts line endings will hash differently) Updated 2026-09-10: the fleet genesis gained a third `config.pqRegistryHash` entry at 18,082,816 (block-0 hash unchanged; only `config` changed), so the file and its digest changed on that date. | | `registries/` | The public Falcon validator registries. Without them a node imports fine to block 13,014,000 (the first anchor) and then refuses every anchor block, because it cannot map a certificate's seal indices to validator addresses. See `registries/README.md`; sha256 of each below. | -| a bootnode enode | two published in the bootnodes section below; run-node.sh uses them by default | +| a bootnode enode | one published in the bootnodes section below; run-node.sh uses it by default | | `patches/`, `precompiles/` | The Besu fork, see `README.md`. Needed to sync past block 9,189,161 | The registries are public (Falcon public keys and validator addresses, each carrying a proof of @@ -292,29 +292,27 @@ besu \ --min-gas-price=0 ``` -Two bootnodes are published here since 2026-08-25. Both are public read hosts that are NOT in -the validator set, so taking them down cannot cost the network consensus; their addresses were -already public through the services they serve. The first runs Besu, the second runs Nethermind, -so your first two peers are also two independent implementations: +One bootnode is published here (two from 2026-08-25 to 2026-09-11; the second address was withdrawn on 2026-09-11 +because its host no longer fits the rule in the section below, and a replacement public read host will be +published when it exists). It is a public read host that is NOT in the validator set, so taking it down +cannot cost the network consensus; its address was already public through the service it serves. It runs Besu: ``` enode://7e8ff740b79bf28a6d7e46ea8d2317cea4a223e7af60be36e91e13dc7051f4cb29077631452e699a5e18e640a94fd6e370e9e51dd91ecba1f2ba9666e07ff01b@37.27.216.110:30303 -enode://a4b90d4f5fc7758814d41acfe00f6a2516683f3d3052410091b894d014dbb2184d6158da76f72e84ca268c2ccde3590d5686f978868f3b6ad6af87e398179fb8@157.180.67.185:30303 ``` -`run-node.sh` uses both by default when `--bootnode` is not given. A written address is not a -live address: only your own `net_peerCount` proves they are up. If both are unreachable, write +`run-node.sh` uses it by default when `--bootnode` is not given. A written address is not a +live address: only your own `net_peerCount` proves it is up. If it is unreachable, write to office@aere.network with the subject "bootnode" for a current enode by return. -Measured 2026-09-11: the second bootnode currently accepts connections only from its own configured -peers, so in practice your first peer is the first bootnode. A node that cannot receive inbound +Measured 2026-09-11: a node that cannot receive inbound connections (behind NAT without a forwarded port) keeps that single peer, and Besu does not fetch the bodies of blocks that carry transactions from a single busy peer: the log repeats `Unable to retrieve blocks for block numbers ... Restarting after short delay` and the import crawls. Forward your P2P port (30303 by default) and, when you run the manual command below, add `--p2p-host=` so that peers can bond with you; once discovery works the node reaches the rest of the network on its own (measured the same day with ten peers: about 1,300 blocks per second -from genesis, no body failures). +from genesis, no body failures). With a single published bootnode this matters more, not less. `--sync-mode=FULL` is deliberate. Chain 2800 runs QBFT with a half second block period, so the head was about 12.2 million blocks when this was written (2026-08-03), passed 14.6 million on 2026-08-19, @@ -377,15 +375,15 @@ the two. --- -## What the two bootnodes are, and what they are not +## What the bootnode is, and what it is not -They are two full nodes that are **not** in the validator set. Chain 2800 seals blocks with nine -QBFT validators (seven until 2026-08-09), and none of the nine is listed here, on purpose. Publishing a validator's p2p +It is a full node that is **not** in the validator set. Chain 2800 seals blocks with ten QBFT +validators (nine from 2026-08-09, seven before; ten since 2026-09-11), and none of them is listed here, on purpose. Publishing a validator's p2p address hands an attacker a target whose downtime costs the network consensus headroom, which is why -sentry style architectures exist. The two nodes above carry no consensus responsibility: if both are -taken down, new nodes cannot bootstrap until they are replaced, and block production is unaffected. +sentry style architectures exist. The node above carries no consensus responsibility: if it is +taken down, new nodes cannot bootstrap until it is replaced, and block production is unaffected. -**Update, 2026-08-28.** The first bootnode (37.27.216.110) now also serves the devp2p +**Update, 2026-08-28.** The bootnode (37.27.216.110) now also serves the devp2p `snap/1` protocol (`SnapServer ... Bonsai full flat db` in its startup log, capability list `[eth/68..71], [istanbul/100], [snap/1]`). A client that chooses `--sync-mode=SNAP` against it downloads headers plus the state at the pivot instead of replaying every historic block body, @@ -395,7 +393,7 @@ a complete stranger's snap sync to head against it has not yet been run, and the `run-node.sh` path deliberately stays `--sync-mode=FULL`, because a validating node that replayed every block is a stronger statement than one that trusted a pivot. -A written address is not a live address. This file cannot prove the bootnodes are up at the moment +A written address is not a live address. This file cannot prove the bootnode is up at the moment you read it; only your own `net_peerCount` can. Both were reachable and serving blocks when this file was written. Both are Foundation operated, so diff --git a/SPEC.md b/SPEC.md index 3677ac8..d4ed7a8 100644 --- a/SPEC.md +++ b/SPEC.md @@ -182,7 +182,7 @@ Aere adds, changes, or that a follower must check. - Validator set changes are made by the standard QBFT voting mechanism (the vote element of extraData, section 2.3). The set had 3 validators at genesis (source: `genesis.json`), and the live chain seals with ten validators since 2026-09-11 (nine from 2026-08-09, seven until then) (source: `RUN-A-NODE.md`, "What the - two bootnodes are"). The block-by-block vote history is on the chain itself and is not restated + bootnode is"). The block-by-block vote history is on the chain itself and is not restated here. ### 2.2 Quorum diff --git a/anchor/MANIFEST-sha256.txt b/anchor/MANIFEST-sha256.txt index 3ca6f04..2dfe97c 100644 --- a/anchor/MANIFEST-sha256.txt +++ b/anchor/MANIFEST-sha256.txt @@ -5,7 +5,7 @@ computed them, and the same 276 files once produced two different aggregates her 329e4e2f93143cc8af092f0a951fa23619c12c4ec1793b053654f8e26e088da0 app/src/main/java/org/hyperledger/besu/cli/BesuCommand.java 3f289d74bb1878280a99742320d7a27c19146a2f19cdf6a0857f2aad4361ad5e app/src/main/java/org/hyperledger/besu/cli/options/AerePqEmergencyOptions.java -05765f25da67f9b14d490121e73536a41c107edcc841c7303afaae93061ebf39 app/src/main/java/org/hyperledger/besu/controller/QbftBesuControllerBuilder.java +e17f5b548f5a0833aef11148f561f2854dacf501bbe7c98f926349707ef77429 app/src/main/java/org/hyperledger/besu/controller/QbftBesuControllerBuilder.java 129493150c0b13e8020bdf50c0c1ae07268d4f73a69a821b303bf4894451fa5e app/src/test/java/org/hyperledger/besu/cli/options/AerePqEmergencyOptionsTest.java 728c29c299edf6011f2c2798cbe930a043112e0e06b924e7edb54284ca44a758 config/src/main/java/org/hyperledger/besu/config/JsonGenesisConfigOptions.java a3b93f2602e9755d91358c9b3f473235f4d40ae23d912f797cdb03ed7e1348f9 consensus/common/build.gradle @@ -170,4 +170,11 @@ a81da71dd34dd111a0da43c22d22345dcc595c084a2c2d36aec1f45d8b762f80 consensus/qbft 321cb7e6923f77e078b523316650ac49136cf8d4df506b9bb235a32795c69611 consensus/qbft/src/test/java/org/hyperledger/besu/consensus/qbft/headervalidationrules/PqArmedWithoutRegistryTest.java 484bc74ff3d1e52d25631264d83af70c2b695e973c7360b0f3db471ab5c6613a consensus/qbft/src/test/java/org/hyperledger/besu/consensus/qbft/headervalidationrules/PqEmergencyShoutRuleTest.java 49c3fd97e7985966531928af173b6f38ac48ff9dc8b5be1106e5b7bfb9c7b2b6 consensus/qbft/src/test/java/org/hyperledger/besu/consensus/qbft/headervalidationrules/PqForkGateFeedTest.java +7ea99f24bc1c9131e8ef08f7a2c6f2a08af71ebee011a4cb88a367bd446b6c27 ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetBalance.java +5bf41ab0be06328fa1e18bf9c98bfd89b7b16384fe185d96b3385595dabe61b3 ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetCode.java +4c96b06073ff3830f4e0292fc732ec3fb9e0ed50ef741dceedf755b50f18d5ae ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetStorageAt.java +8e649d2a1c47cd912f0782408da3f3af5537e8c670c12cba72dbb502809c848d ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetTransactionCount.java +41b5f7b0066ef6d8af25491814eeacf2bcbb29415f407020affde05b7d1916b7 ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java +3ee087fc34b455e113ec9d2c53b5dbdb3574429fcf166f4c968a80120f70f0eb ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/AereStateUnavailableIsAnErrorTest.java +6e99e9f399558abbd1d4fdb4b176ba413bf31cabb3c8aba1701c17ce5828ce8a ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/query/AereHeadStateRetryTest.java 3b690b72e80a0f9eaf868a3fa3f99ace42dad74dc30b2c98b9f3ca5a023917c0 ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/DownloadHeadersStep.java diff --git a/anchor/app/src/main/java/org/hyperledger/besu/controller/QbftBesuControllerBuilder.java b/anchor/app/src/main/java/org/hyperledger/besu/controller/QbftBesuControllerBuilder.java index c9bd350..281add4 100644 --- a/anchor/app/src/main/java/org/hyperledger/besu/controller/QbftBesuControllerBuilder.java +++ b/anchor/app/src/main/java/org/hyperledger/besu/controller/QbftBesuControllerBuilder.java @@ -318,6 +318,69 @@ public class QbftBesuControllerBuilder extends BesuControllerBuilder { + "opens again.", e.toString()); } + + // AERE-D358 (2026-09-10, measured on validator 8 of chain 2800): the "import path" named above + // is FalconSealValidationRule, and on 2800 that rule is RETIRED from the anchor block + // (PqAnchorConfig.legacyFalconRuleRetirementBlock), so it never runs. The startup attempt + // above is therefore the ONLY attempt, and it is silent when the head world state is not yet + // readable at that instant: after a plain container restart at 15:41Z the manifest loaded as + // PENDING, no STARTUP line was logged at all, and the node stayed PENDING for 55 minutes - + // refusing every peer Commit and Prepare as "NO post-quantum seal", emitting nothing, timing + // out its proposer slot every rotation, and slowing the whole chain from 0.56 to 1.07 s/block. + // A second restart activated at once, which is exactly what a race looks like. + // + // The repair: while the late anchor is PENDING, retry the same read (same contract, same + // slot 0, same activateLateAnchor) on every block added to the chain, and stop retrying once + // it is activated or terminally failed. It invents nothing and weakens no check. + if (FalconSealSupport.instance().anchorAddress() != null + && FalconSealSupport.instance().lateAnchorPending()) { + final java.util.concurrent.atomic.AtomicLong aereD358Retries = + new java.util.concurrent.atomic.AtomicLong(); + blockchain.observeBlockAdded( + event -> { + final FalconSealSupport pqc = FalconSealSupport.instance(); + if (!pqc.lateAnchorPending()) { + return; + } + try { + final BlockHeader h = event.getBlock().getHeader(); + protocolContext + .getWorldStateArchive() + .get(h.getStateRoot(), h.getHash()) + .ifPresent( + ws -> { + final var cont = + ws.get(Address.fromHexString(pqc.anchorAddress())); + if (cont == null) { + return; + } + final var slot0 = + cont.getStorageValue( + org.apache.tuweni.units.bigints.UInt256.ZERO); + if (slot0 == null || slot0.isZero()) { + return; + } + final long n = aereD358Retries.incrementAndGet(); + final boolean armat = + pqc.activateLateAnchor(slot0.toBytes().toUnprefixedHexString()); + if (armat || n == 1 || n % 100 == 0) { + LOG.info( + "AERE-D358: late-anchor activation retried at block {} (attempt {}): {}", + h.getNumber(), + n, + armat + ? "SUCCEEDED" + : "still pending (contract present, hash mismatch or refused)"); + } + }); + } catch (final Exception e) { + LOG.debug("AERE-D358: retry could not read the head state ({})", e.toString()); + } + }); + LOG.info( + "AERE-D358: late anchor is PENDING after the startup attempt; activation will be " + + "retried on every imported block until it succeeds or terminally fails."); + } } // AERE A8 (2026-08-01): bind the Falcon registry to consensus. diff --git a/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetBalance.java b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetBalance.java new file mode 100644 index 0000000..0f43576 --- /dev/null +++ b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetBalance.java @@ -0,0 +1,80 @@ +/* + * Copyright ConsenSys AG. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods; + +import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.exception.InvalidJsonRpcParameters; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.BlockParameterOrBlockHash; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.JsonRpcParameter.JsonRpcParameterException; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.Quantity; +import org.hyperledger.besu.ethereum.api.query.BlockchainQueries; + +import java.util.function.Supplier; + +public class EthGetBalance extends AbstractBlockParameterOrBlockHashMethod { + public EthGetBalance(final BlockchainQueries blockchainQueries) { + super(blockchainQueries); + } + + public EthGetBalance(final Supplier blockchainQueries) { + super(blockchainQueries); + } + + @Override + public String getName() { + return RpcMethod.ETH_GET_BALANCE.getMethodName(); + } + + @Override + protected BlockParameterOrBlockHash blockParameterOrBlockHash( + final JsonRpcRequestContext request) { + try { + return request.getRequiredParameter(1, BlockParameterOrBlockHash.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid block or block hash parameter (index 1)", RpcErrorType.INVALID_BLOCK_PARAMS, e); + } + } + + // AERE D-FAZA2-04 (2026-09-10): an EMPTY answer from the state query means the world state for that + // block is not available (outside the node's state window, or not yet persisted); it never means + // "no such account" - fromAccount() fills a default for that. Upstream serialised the empty answer as + // a bare JSON null, which every client reads as a value. Here it is the same honest error that + // eth_getProof already returns: WORLD_STATE_UNAVAILABLE. + @Override + protected Object resultByBlockHash(final JsonRpcRequestContext request, final Hash blockHash) { + final Address address; + try { + address = request.getRequiredParameter(0, Address.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid address parameter (index 0)", RpcErrorType.INVALID_ADDRESS_PARAMS, e); + } + return blockchainQueries + .get() + .accountBalance(address, blockHash) + .map(Quantity::create) + .map(s -> s) + .orElseGet( + () -> + new JsonRpcErrorResponse( + request.getRequest().getId(), RpcErrorType.WORLD_STATE_UNAVAILABLE)); + } +} diff --git a/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetCode.java b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetCode.java new file mode 100644 index 0000000..b4be077 --- /dev/null +++ b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetCode.java @@ -0,0 +1,81 @@ +/* + * Copyright ConsenSys AG. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods; + +import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.exception.InvalidJsonRpcParameters; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.BlockParameterOrBlockHash; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.JsonRpcParameter.JsonRpcParameterException; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; +import org.hyperledger.besu.ethereum.api.query.BlockchainQueries; + +import java.util.function.Supplier; + +import org.apache.tuweni.bytes.Bytes; + +public class EthGetCode extends AbstractBlockParameterOrBlockHashMethod { + + public EthGetCode(final BlockchainQueries blockchainQueries) { + super(blockchainQueries); + } + + public EthGetCode(final Supplier blockchainQueries) { + super(blockchainQueries); + } + + @Override + public String getName() { + return RpcMethod.ETH_GET_CODE.getMethodName(); + } + + @Override + protected BlockParameterOrBlockHash blockParameterOrBlockHash( + final JsonRpcRequestContext request) { + try { + return request.getRequiredParameter(1, BlockParameterOrBlockHash.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid block or block hash parameter (index 1)", RpcErrorType.INVALID_BLOCK_PARAMS, e); + } + } + + // AERE D-FAZA2-04 (2026-09-10): an EMPTY answer from the state query means the world state for that + // block is not available (outside the node's state window, or not yet persisted); it never means + // "no such account" - fromAccount() fills a default for that. Upstream serialised the empty answer as + // a bare JSON null, which every client reads as a value. Here it is the same honest error that + // eth_getProof already returns: WORLD_STATE_UNAVAILABLE. + @Override + protected Object resultByBlockHash(final JsonRpcRequestContext request, final Hash blockHash) { + final Address address; + try { + address = request.getRequiredParameter(0, Address.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid address parameter (index 0)", RpcErrorType.INVALID_ADDRESS_PARAMS, e); + } + return getBlockchainQueries() + .getCode(address, blockHash) + .map(Bytes::toString) + .map(s -> s) + .orElseGet( + () -> + new JsonRpcErrorResponse( + request.getRequest().getId(), RpcErrorType.WORLD_STATE_UNAVAILABLE)); + } +} diff --git a/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetStorageAt.java b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetStorageAt.java new file mode 100644 index 0000000..fcd9909 --- /dev/null +++ b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetStorageAt.java @@ -0,0 +1,83 @@ +/* + * Copyright ConsenSys AG. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods; + +import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.datatypes.parameters.UInt256Parameter; +import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.exception.InvalidJsonRpcParameters; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.BlockParameterOrBlockHash; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.JsonRpcParameter.JsonRpcParameterException; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; +import org.hyperledger.besu.ethereum.api.query.BlockchainQueries; + +import org.apache.tuweni.units.bigints.UInt256; + +public class EthGetStorageAt extends AbstractBlockParameterOrBlockHashMethod { + public EthGetStorageAt(final BlockchainQueries blockchainQueries) { + super(blockchainQueries); + } + + @Override + public String getName() { + return RpcMethod.ETH_GET_STORAGE_AT.getMethodName(); + } + + @Override + protected BlockParameterOrBlockHash blockParameterOrBlockHash( + final JsonRpcRequestContext request) { + try { + return request.getRequiredParameter(2, BlockParameterOrBlockHash.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid block or block hash parameter (index 2)", RpcErrorType.INVALID_BLOCK_PARAMS, e); + } + } + + @Override + // AERE D-FAZA2-04 (2026-09-10): an EMPTY answer from the state query means the world state for that + // block is not available (outside the node's state window, or not yet persisted); it never means + // "no such account" - fromAccount() fills a default for that. Upstream serialised the empty answer as + // a bare JSON null, which every client reads as a value. Here it is the same honest error that + // eth_getProof already returns: WORLD_STATE_UNAVAILABLE. + protected Object resultByBlockHash(final JsonRpcRequestContext request, final Hash blockHash) { + final Address address; + try { + address = request.getRequiredParameter(0, Address.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid address parameter (index 0)", RpcErrorType.INVALID_ADDRESS_PARAMS, e); + } + final UInt256 position; + try { + position = request.getRequiredParameter(1, UInt256Parameter.class).getValue(); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid position parameter (index 1)", RpcErrorType.INVALID_POSITION_PARAMS, e); + } + return blockchainQueries + .get() + .storageAt(address, position, blockHash) + .map(UInt256::toHexString) + .map(s -> s) + .orElseGet( + () -> + new JsonRpcErrorResponse( + request.getRequest().getId(), RpcErrorType.WORLD_STATE_UNAVAILABLE)); + } +} diff --git a/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetTransactionCount.java b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetTransactionCount.java new file mode 100644 index 0000000..d8ba86c --- /dev/null +++ b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/EthGetTransactionCount.java @@ -0,0 +1,113 @@ +/* + * Copyright ConsenSys AG. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods; + +import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.ethereum.api.jsonrpc.RpcMethod; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.exception.InvalidJsonRpcParameters; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.BlockParameterOrBlockHash; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.JsonRpcParameter.JsonRpcParameterException; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.Quantity; +import org.hyperledger.besu.ethereum.api.query.BlockchainQueries; +import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool; + +import java.util.function.Supplier; + +import com.google.common.base.Suppliers; + +public class EthGetTransactionCount extends AbstractBlockParameterOrBlockHashMethod { + private final Supplier transactionPoolSupplier; + + public EthGetTransactionCount( + final BlockchainQueries blockchain, final TransactionPool transactionPoolSupplier) { + this(Suppliers.ofInstance(blockchain), Suppliers.ofInstance(transactionPoolSupplier)); + } + + public EthGetTransactionCount( + final Supplier blockchain, + final Supplier transactionPoolSupplier) { + super(blockchain); + this.transactionPoolSupplier = transactionPoolSupplier; + } + + @Override + public String getName() { + return RpcMethod.ETH_GET_TRANSACTION_COUNT.getMethodName(); + } + + @Override + protected BlockParameterOrBlockHash blockParameterOrBlockHash( + final JsonRpcRequestContext request) { + try { + return request.getRequiredParameter(1, BlockParameterOrBlockHash.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid block or block hash parameter (index 1)", RpcErrorType.INVALID_BLOCK_PARAMS, e); + } + } + + @Override + protected Object pendingResult(final JsonRpcRequestContext request) { + final Address address; + try { + address = request.getRequiredParameter(0, Address.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid address parameter (index 0)", RpcErrorType.INVALID_ADDRESS_PARAMS, e); + } + final long pendingNonce = + transactionPoolSupplier.get().getNextNonceForSender(address).orElse(0); + final long latestNonce = + getBlockchainQueries() + .getTransactionCount( + address, getBlockchainQueries().getBlockchain().getChainHead().getHash()); + + if (Long.compareUnsigned(pendingNonce, latestNonce) > 0) { + return Quantity.create(pendingNonce); + } + + return Quantity.create(latestNonce); + } + + // AERE D-FAZA2-04 (2026-09-10): an EMPTY answer from the state query means the world state for that + // block is not available (outside the node's state window, or not yet persisted); it never means + // "no such account" - fromAccount() fills a default for that. Upstream serialised the empty answer as + // a bare JSON null, which every client reads as a value. Here it is the same honest error that + // eth_getProof already returns: WORLD_STATE_UNAVAILABLE. + // Upstream folded the missing state into a nonce of 0 (`.orElse(0L)`), a FALSE 0x0 that a wallet + // signs with. Here the nonce is asked for as an Optional and the missing state is an error. + @Override + protected Object resultByBlockHash(final JsonRpcRequestContext request, final Hash blockHash) { + final Address address; + try { + address = request.getRequiredParameter(0, Address.class); + } catch (JsonRpcParameterException e) { + throw new InvalidJsonRpcParameters( + "Invalid address parameter (index 0)", RpcErrorType.INVALID_ADDRESS_PARAMS, e); + } + return getBlockchainQueries() + .getTransactionCountIfStateAvailable(address, blockHash) + .map(Quantity::create) + .map(s -> s) + .orElseGet( + () -> + new JsonRpcErrorResponse( + request.getRequest().getId(), RpcErrorType.WORLD_STATE_UNAVAILABLE)); + } +} diff --git a/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java new file mode 100644 index 0000000..caed487 --- /dev/null +++ b/anchor/ethereum/api/src/main/java/org/hyperledger/besu/ethereum/api/query/BlockchainQueries.java @@ -0,0 +1,1340 @@ +/* + * Copyright ConsenSys AG. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.query; + +import static com.google.common.base.Preconditions.checkArgument; +import static org.hyperledger.besu.ethereum.api.query.cache.TransactionLogBloomCacher.BLOCKS_PER_BLOOM_CACHE; +import static org.hyperledger.besu.ethereum.mainnet.feemarket.ExcessBlobGasCalculator.calculateExcessBlobGasForParent; +import static org.hyperledger.besu.ethereum.trie.pathbased.common.provider.WorldStateQueryParams.withBlockHeaderAndNoUpdateNodeHead; + +import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.datatypes.LogsBloomFilter; +import org.hyperledger.besu.datatypes.Wei; +import org.hyperledger.besu.ethereum.api.ApiConfiguration; +import org.hyperledger.besu.ethereum.api.ImmutableApiConfiguration; +import org.hyperledger.besu.ethereum.api.query.cache.TransactionLogBloomCacher; +import org.hyperledger.besu.ethereum.chain.Blockchain; +import org.hyperledger.besu.ethereum.chain.TransactionLocation; +import org.hyperledger.besu.ethereum.core.Block; +import org.hyperledger.besu.ethereum.core.BlockBody; +import org.hyperledger.besu.ethereum.core.BlockHeader; +import org.hyperledger.besu.ethereum.core.LogWithMetadata; +import org.hyperledger.besu.ethereum.core.MiningConfiguration; +import org.hyperledger.besu.ethereum.core.MutableWorldState; +import org.hyperledger.besu.ethereum.core.ProcessableBlockHeader; +import org.hyperledger.besu.ethereum.core.Transaction; +import org.hyperledger.besu.ethereum.core.TransactionReceipt; +import org.hyperledger.besu.ethereum.eth.manager.EthScheduler; +import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; +import org.hyperledger.besu.ethereum.mainnet.ProtocolSpec; +import org.hyperledger.besu.ethereum.mainnet.feemarket.BaseFeeMarket; +import org.hyperledger.besu.ethereum.mainnet.feemarket.FeeMarket; +import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive; +import org.hyperledger.besu.evm.account.Account; +import org.hyperledger.besu.util.OrderStatistics; + +import java.io.EOFException; +import java.io.IOException; +import java.io.RandomAccessFile; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.List; +import java.util.Optional; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.Function; +import java.util.function.Supplier; +import java.util.stream.Collectors; +import java.util.stream.IntStream; +import java.util.stream.LongStream; +import java.util.stream.Stream; + +import org.apache.tuweni.bytes.Bytes; +import org.apache.tuweni.units.bigints.UInt256; +import org.apache.tuweni.units.bigints.UInt256s; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class BlockchainQueries { + private static final Logger LOG = LoggerFactory.getLogger(BlockchainQueries.class); + + private final ProtocolSchedule protocolSchedule; + private final WorldStateArchive worldStateArchive; + private final Blockchain blockchain; + private final Optional cachePath; + private final Optional transactionLogBloomCacher; + private final Optional ethScheduler; + private final ApiConfiguration apiConfig; + private final MiningConfiguration miningConfiguration; + + public BlockchainQueries( + final ProtocolSchedule protocolSchedule, + final Blockchain blockchain, + final WorldStateArchive worldStateArchive, + final MiningConfiguration miningConfiguration) { + this( + protocolSchedule, + blockchain, + worldStateArchive, + Optional.empty(), + Optional.empty(), + miningConfiguration); + } + + public BlockchainQueries( + final ProtocolSchedule protocolSchedule, + final Blockchain blockchain, + final WorldStateArchive worldStateArchive, + final EthScheduler scheduler, + final MiningConfiguration miningConfiguration) { + this( + protocolSchedule, + blockchain, + worldStateArchive, + Optional.empty(), + Optional.ofNullable(scheduler), + miningConfiguration); + } + + public BlockchainQueries( + final ProtocolSchedule protocolSchedule, + final Blockchain blockchain, + final WorldStateArchive worldStateArchive, + final Optional cachePath, + final Optional scheduler, + final MiningConfiguration miningConfiguration) { + this( + protocolSchedule, + blockchain, + worldStateArchive, + cachePath, + scheduler, + ImmutableApiConfiguration.builder().build(), + miningConfiguration); + } + + public BlockchainQueries( + final ProtocolSchedule protocolSchedule, + final Blockchain blockchain, + final WorldStateArchive worldStateArchive, + final Optional cachePath, + final Optional scheduler, + final ApiConfiguration apiConfig, + final MiningConfiguration miningConfiguration) { + this.protocolSchedule = protocolSchedule; + this.blockchain = blockchain; + this.worldStateArchive = worldStateArchive; + this.cachePath = cachePath; + this.ethScheduler = scheduler; + this.transactionLogBloomCacher = + (cachePath.isPresent() && scheduler.isPresent()) + ? Optional.of( + new TransactionLogBloomCacher(blockchain, cachePath.get(), scheduler.get())) + : Optional.empty(); + this.apiConfig = apiConfig; + this.miningConfiguration = miningConfiguration; + } + + public Blockchain getBlockchain() { + return blockchain; + } + + public WorldStateArchive getWorldStateArchive() { + return worldStateArchive; + } + + public Optional getTransactionLogBloomCacher() { + return transactionLogBloomCacher; + } + + /** + * Retrieves the header hash of the block at the given height in the canonical chain. + * + * @param number The height of the block whose hash should be retrieved. + * @return The hash of the block at the given height. + */ + public Optional getBlockHashByNumber(final long number) { + return blockchain.getBlockHashByNumber(number); + } + + /** + * Return the block number of the head of the chain. + * + * @return The block number of the head of the chain. + */ + public long headBlockNumber() { + return blockchain.getChainHeadBlockNumber(); + } + + /** + * Return the header of the head of the chain. + * + * @return The header of the head of the chain. + */ + public BlockHeader headBlockHeader() { + return blockchain.getChainHeadHeader(); + } + + /** + * Return the header of the last finalized block. + * + * @return The header of the last finalized block. + */ + public Optional finalizedBlockHeader() { + return blockchain.getFinalized().flatMap(blockchain::getBlockHeader); + } + + /** + * Return the header of the last safe block. + * + * @return The header of the last safe block. + */ + public Optional safeBlockHeader() { + return blockchain.getSafeBlock().flatMap(blockchain::getBlockHeader); + } + + /** + * Determines the block header for the address associated with this storage index. + * + * @param address The address of the account that owns the storage being queried. + * @param storageIndex The storage index whose value is being retrieved. + * @param blockNumber The blockNumber that is being queried. + * @return The value at the storage index being queried. + */ + public Optional storageAt( + final Address address, final UInt256 storageIndex, final long blockNumber) { + final Hash blockHash = getBlockHashByNumber(blockNumber).orElse(Hash.EMPTY); + + return storageAt(address, storageIndex, blockHash); + } + + /** + * Determines the block header for the address associated with this storage index. + * + * @param address The address of the account that owns the storage being queried. + * @param storageIndex The storage index whose value is being retrieved. + * @param blockHash The blockHash that is being queried. + * @return The value at the storage index being queried. + */ + public Optional storageAt( + final Address address, final UInt256 storageIndex, final Hash blockHash) { + return fromAccount( + address, blockHash, account -> account.getStorageValue(storageIndex), UInt256.ZERO); + } + + /** + * Returns the balance of the given account at a specific block number. + * + * @param address The address of the account being queried. + * @param blockNumber The block number being queried. + * @return The balance of the account in Wei. + */ + public Optional accountBalance(final Address address, final long blockNumber) { + final Hash blockHash = getBlockHashByNumber(blockNumber).orElse(Hash.EMPTY); + + return accountBalance(address, blockHash); + } + + /** + * Returns the balance of the given account at a specific block hash. + * + * @param address The address of the account being queried. + * @param blockHash The block hash being queried. + * @return The balance of the account in Wei. + */ + public Optional accountBalance(final Address address, final Hash blockHash) { + return fromAccount(address, blockHash, Account::getBalance, Wei.ZERO); + } + + /** + * Retrieves the code associated with the given account at a particular block number. + * + * @param address The account address being queried. + * @param blockNumber The height of the block to be checked. + * @return The code associated with this address. + */ + public Optional getCode(final Address address, final long blockNumber) { + final Hash blockHash = getBlockHashByNumber(blockNumber).orElse(Hash.EMPTY); + + return getCode(address, blockHash); + } + + /** + * Retrieves the code associated with the given account at a particular block hash. + * + * @param address The account address being queried. + * @param blockHash The hash of the block to be checked. + * @return The code associated with this address. + */ + public Optional getCode(final Address address, final Hash blockHash) { + return fromAccount(address, blockHash, Account::getCode, Bytes.EMPTY); + } + + /** + * Returns the number of transactions in the block at the given height. + * + * @param blockNumber The height of the block being queried. + * @return The number of transactions contained in the referenced block. + */ + public Optional getTransactionCount(final long blockNumber) { + if (outsideBlockchainRange(blockNumber)) { + return Optional.empty(); + } + return blockchain.getBlockHashByNumber(blockNumber).flatMap(this::getTransactionCount); + } + + /** + * Returns the number of transactions in the block with the given hash. + * + * @param blockHeaderHash The hash of the block being queried. + * @return The number of transactions contained in the referenced block. + */ + public Optional getTransactionCount(final Hash blockHeaderHash) { + return blockchain.getBlockBody(blockHeaderHash).map(body -> body.getTransactions().size()); + } + + /** + * Returns the number of transactions sent from the given address in the block at the given + * height. + * + * @param address The address whose sent transactions we want to count. + * @param blockNumber The height of the block being queried. + * @return The number of transactions sent from the given address. + */ + public long getTransactionCount(final Address address, final long blockNumber) { + final Hash blockHash = + getBlockHeaderByNumber(blockNumber).map(BlockHeader::getHash).orElse(Hash.EMPTY); + + return getTransactionCount(address, blockHash); + } + + /** + * Returns the number of transactions sent from the given address in the block at the given hash. + * + * @param address The address whose sent transactions we want to count. + * @param blockHash The hash of the block being queried. + * @return The number of transactions sent from the given address. + */ + public long getTransactionCount(final Address address, final Hash blockHash) { + return getAndMapWorldState( + blockHash, worldState -> Optional.ofNullable(worldState.get(address))) + .map(Account::getNonce) + .orElse(0L); + } + + /** + * AERE D-FAZA2-04: the nonce as an Optional that is EMPTY when the world state for the block is not + * available, so a caller can tell "state missing" from "nonce zero". {@link #getTransactionCount(Address, + * Hash)} keeps upstream's behaviour (0 when missing) for the callers that rely on it. + */ + public Optional getTransactionCountIfStateAvailable(final Address address, final Hash blockHash) { + return getAndMapWorldState( + blockHash, + worldState -> + Optional.of( + Optional.ofNullable(worldState.get(address)).map(Account::getNonce).orElse(0L))); + } + + /** + * Returns the number of transactions sent from the given address in the latest block. + * + * @param address The address whose sent transactions we want to count. + * @return The number of transactions sent from the given address. + */ + public long getTransactionCount(final Address address) { + return getTransactionCount(address, headBlockNumber()); + } + + /** + * Returns the number of ommers in the block at the given height. + * + * @param blockNumber The height of the block being queried. + * @return The number of ommers in the referenced block. + */ + public Optional getOmmerCount(final long blockNumber) { + return blockchain.getBlockHashByNumber(blockNumber).flatMap(this::getOmmerCount); + } + + /** + * Returns the number of ommers in the block at the given height. + * + * @param blockHeaderHash The hash of the block being queried. + * @return The number of ommers in the referenced block. + */ + public Optional getOmmerCount(final Hash blockHeaderHash) { + return blockchain.getBlockBody(blockHeaderHash).map(b -> b.getOmmers().size()); + } + + /** + * Returns the number of ommers in the latest block. + * + * @return The number of ommers in the latest block. + */ + public Optional getOmmerCount() { + return getOmmerCount(blockchain.getChainHeadHash()); + } + + /** + * Returns the ommer at the given index for the referenced block. + * + * @param blockHeaderHash The hash of the block to be queried. + * @param index The index of the ommer in the blocks ommers list. + * @return The ommer at the given index belonging to the referenced block. + */ + public Optional getOmmer(final Hash blockHeaderHash, final int index) { + return blockchain.getBlockBody(blockHeaderHash).map(blockBody -> getOmmer(blockBody, index)); + } + + private BlockHeader getOmmer(final BlockBody blockBody, final int index) { + final List ommers = blockBody.getOmmers(); + if (ommers.size() > index) { + return ommers.get(index); + } else { + return null; + } + } + + /** + * Returns the ommer at the given index for the referenced block. + * + * @param blockNumber The block number identifying the block to be queried. + * @param index The index of the ommer in the blocks ommers list. + * @return The ommer at the given index belonging to the referenced block. + */ + public Optional getOmmer(final long blockNumber, final int index) { + return blockchain.getBlockHashByNumber(blockNumber).flatMap(hash -> getOmmer(hash, index)); + } + + /** + * Returns the ommer at the given index for the latest block. + * + * @param index The index of the ommer in the blocks ommers list. + * @return The ommer at the given index belonging to the latest block. + */ + public Optional getOmmer(final int index) { + return blockchain + .getBlockHashByNumber(blockchain.getChainHeadBlockNumber()) + .flatMap(hash -> getOmmer(hash, index)); + } + + /** + * Given a block hash, returns the associated block augmented with metadata. + * + * @param blockHeaderHash The hash of the target block's header. + * @return The referenced block. + */ + public Optional> blockByHash( + final Hash blockHeaderHash) { + return blockchain + .getBlockHeader(blockHeaderHash) + .flatMap( + header -> + blockchain + .getBlockBody(blockHeaderHash) + .flatMap( + body -> + blockchain + .getTotalDifficultyByHash(blockHeaderHash) + .map( + td -> { + final List txs = body.getTransactions(); + final List formattedTxs = + formatTransactions( + txs, + header.getNumber(), + header.getBaseFee(), + blockHeaderHash, + header.getTimestamp()); + final List ommers = + body.getOmmers().stream() + .map(BlockHeader::getHash) + .collect(Collectors.toList()); + final int size = new Block(header, body).getSize(); + return new BlockWithMetadata<>( + header, + formattedTxs, + ommers, + td, + size, + body.getWithdrawals()); + }))); + } + + /** + * Given a block number, returns the associated block augmented with metadata. + * + * @param number The height of the target block. + * @return The referenced block. + */ + public Optional> blockByNumber( + final long number) { + return blockchain.getBlockHashByNumber(number).flatMap(this::blockByHash); + } + + /** + * Returns the latest block augmented with metadata. + * + * @return The latest block. + */ + public Optional> latestBlock() { + return this.blockByHash(blockchain.getChainHeadHash()); + } + + /** + * Given a block hash, returns the associated block with metadata and a list of transaction hashes + * rather than full transactions. + * + * @param blockHeaderHash The hash of the target block's header. + * @return The referenced block. + */ + public Optional> blockByHashWithTxHashes( + final Hash blockHeaderHash) { + return blockchain + .getBlockHeader(blockHeaderHash) + .flatMap( + header -> + blockchain + .getBlockBody(blockHeaderHash) + .flatMap( + body -> + blockchain + .getTotalDifficultyByHash(blockHeaderHash) + .map( + td -> { + final List txs = + body.getTransactions().stream() + .map(Transaction::getHash) + .collect(Collectors.toList()); + final List ommers = + body.getOmmers().stream() + .map(BlockHeader::getHash) + .collect(Collectors.toList()); + final int size = new Block(header, body).getSize(); + return new BlockWithMetadata<>( + header, txs, ommers, td, size, body.getWithdrawals()); + }))); + } + + /** + * Given a block number, returns the associated block with metadata and a list of transaction + * hashes rather than full transactions. + * + * @param blockNumber The height of the target block's header. + * @return The referenced block. + */ + public Optional> blockByNumberWithTxHashes(final long blockNumber) { + return blockchain.getBlockHashByNumber(blockNumber).flatMap(this::blockByHashWithTxHashes); + } + + public Optional getBlockHeaderByHash(final Hash hash) { + return blockchain.getBlockHeader(hash); + } + + public Optional getBlockHeaderByNumber(final long number) { + return blockchain.getBlockHeader(number); + } + + public boolean isBlockAccessListSupported(final BlockHeader header) { + return protocolSchedule.getByBlockHeader(header).getBlockAccessListFactory().isPresent(); + } + + public boolean blockIsOnCanonicalChain(final Hash hash) { + return blockchain.blockIsOnCanonicalChain(hash); + } + + /** + * Returns the latest block with metadata and a list of transaction hashes rather than full + * transactions. + * + * @return The latest block. + */ + public Optional> latestBlockWithTxHashes() { + return this.blockByHashWithTxHashes(blockchain.getChainHeadHash()); + } + + /** + * Given a transaction hash, returns the associated transaction. + * + * @param transactionHash The hash of the target transaction. + * @return The transaction associated with the given hash. + */ + public Optional transactionByHash(final Hash transactionHash) { + final Optional maybeLocation = + blockchain.getTransactionLocation(transactionHash); + if (maybeLocation.isEmpty()) { + return Optional.empty(); + } + final TransactionLocation loc = maybeLocation.get(); + final Hash blockHash = loc.getBlockHash(); + // getTransactionLocation should not return if the TX or block doesn't exist, so throwing + // on a missing optional is appropriate. + final BlockHeader header = blockchain.getBlockHeader(blockHash).orElseThrow(); + final Transaction transaction = blockchain.getTransactionByHash(transactionHash).orElseThrow(); + return Optional.of( + new TransactionWithMetadata( + transaction, + header.getNumber(), + header.getBaseFee(), + blockHash, + loc.getTransactionIndex(), + header.getTimestamp())); + } + + /** + * Returns the transaction at the given index for the specified block. + * + * @param blockNumber The number of the block being queried. + * @param txIndex The index of the transaction to return. + * @return The transaction at the specified location. + */ + public Optional transactionByBlockNumberAndIndex( + final long blockNumber, final int txIndex) { + checkArgument(txIndex >= 0); + return blockchain + .getBlockHeader(blockNumber) + .map(header -> transactionByHeaderAndIndex(header, txIndex)); + } + + /** + * Returns the transaction at the given index for the specified block. + * + * @param blockHeaderHash The hash of the block being queried. + * @param txIndex The index of the transaction to return. + * @return The transaction at the specified location. + */ + public Optional transactionByBlockHashAndIndex( + final Hash blockHeaderHash, final int txIndex) { + checkArgument(txIndex >= 0); + return blockchain + .getBlockHeader(blockHeaderHash) + .map(header -> transactionByHeaderAndIndex(header, txIndex)); + } + + /** + * Helper method to return the transaction at the given index for the specified header, used by + * getTransactionByBlock*AndIndex methods. + * + * @param header The block header. + * @param txIndex The index of the transaction to return. + * @return The transaction at the specified location. + */ + private TransactionWithMetadata transactionByHeaderAndIndex( + final BlockHeader header, final int txIndex) { + final Hash blockHeaderHash = header.getHash(); + + return blockchain + .getBlockBody(blockHeaderHash) + .map(BlockBody::getTransactions) + .filter((txs) -> txIndex < txs.size()) + .map( + (txs) -> + new TransactionWithMetadata( + txs.get(txIndex), + header.getNumber(), + header.getBaseFee(), + blockHeaderHash, + txIndex, + header.getTimestamp())) + .orElse(null); + } + + public Optional transactionLocationByHash(final Hash transactionHash) { + return blockchain.getTransactionLocation(transactionHash); + } + + /** + * Returns the transaction receipts associated with the given block hash. + * + * @param blockHash The hash of the block that corresponds to the receipts to retrieve. + * @return The transaction receipts associated with the referenced block. + */ + public Optional> transactionReceiptsByBlockHash( + final Hash blockHash, final ProtocolSchedule protocolSchedule) { + final Optional block = blockchain.getBlockByHash(blockHash); + if (block.isEmpty()) { + return Optional.empty(); + } + final BlockHeader header = block.get().getHeader(); + final List transactions = block.get().getBody().getTransactions(); + + final List transactionReceipts = + blockchain.getTxReceipts(blockHash).orElseThrow(); + + long cumulativeGasUsedUntilTx = 0; + int logIndexOffset = 0; + + List receiptsResult = + new ArrayList(transactions.size()); + + for (int transactionIndex = 0; transactionIndex < transactions.size(); transactionIndex++) { + final Transaction transaction = transactions.get(transactionIndex); + final TransactionReceipt transactionReceipt = transactionReceipts.get(transactionIndex); + final Hash transactionHash = transaction.getHash(); + + long gasUsed = transactionReceipt.getCumulativeGasUsed() - cumulativeGasUsedUntilTx; + + Optional maybeBlobGasUsed = + getBlobGasUsed(transaction, protocolSchedule.getByBlockHeader(header)); + + Optional maybeBlobGasPrice = + getBlobGasPrice(transaction, header, protocolSchedule.getByBlockHeader(header)); + + receiptsResult.add( + TransactionReceiptWithMetadata.create( + transactionReceipt, + transaction, + transactionHash, + transactionIndex, + gasUsed, + header.getBaseFee(), + blockHash, + header.getTimestamp(), + header.getNumber(), + maybeBlobGasUsed, + maybeBlobGasPrice, + logIndexOffset)); + + cumulativeGasUsedUntilTx = transactionReceipt.getCumulativeGasUsed(); + logIndexOffset += transactionReceipt.getLogsList().size(); + } + return Optional.of(receiptsResult); + } + + /** + * Returns the transaction receipt associated with the given transaction hash. + * + * @param transactionHash The hash of the transaction that corresponds to the receipt to retrieve. + * @return The transaction receipt associated with the referenced transaction. + */ + public Optional transactionReceiptByTransactionHash( + final Hash transactionHash, final ProtocolSchedule protocolSchedule) { + final Optional maybeLocation = + blockchain.getTransactionLocation(transactionHash); + if (maybeLocation.isEmpty()) { + return Optional.empty(); + } + // getTransactionLocation should not return if the TX or block doesn't exist, so throwing + // on a missing optional is appropriate. + final TransactionLocation location = maybeLocation.get(); + final Hash blockhash = location.getBlockHash(); + final int transactionIndex = location.getTransactionIndex(); + + final Block block = blockchain.getBlockByHash(blockhash).orElseThrow(); + final Transaction transaction = block.getBody().getTransactions().get(transactionIndex); + + final BlockHeader header = block.getHeader(); + final List transactionReceipts = + blockchain.getTxReceipts(blockhash).orElseThrow(); + final TransactionReceipt transactionReceipt = transactionReceipts.get(transactionIndex); + + long gasUsed = transactionReceipt.getCumulativeGasUsed(); + int logIndexOffset = 0; + if (transactionIndex > 0) { + gasUsed -= transactionReceipts.get(transactionIndex - 1).getCumulativeGasUsed(); + logIndexOffset = + IntStream.range(0, transactionIndex) + .map(i -> transactionReceipts.get(i).getLogsList().size()) + .sum(); + } + + Optional maybeBlobGasUsed = + getBlobGasUsed(transaction, protocolSchedule.getByBlockHeader(header)); + + Optional maybeBlobGasPrice = + getBlobGasPrice(transaction, header, protocolSchedule.getByBlockHeader(header)); + + return Optional.of( + TransactionReceiptWithMetadata.create( + transactionReceipt, + transaction, + transactionHash, + transactionIndex, + gasUsed, + header.getBaseFee(), + blockhash, + header.getTimestamp(), + header.getNumber(), + maybeBlobGasUsed, + maybeBlobGasPrice, + logIndexOffset)); + } + + /** + * Calculates the blob gas used for data in a transaction. + * + * @param transaction the transaction to calculate the gas for + * @param protocolSpec the protocol specification to use for gas calculation + * @return an Optional containing the blob gas used for data if the transaction type supports + * blobs, otherwise returns an empty Optional + */ + private Optional getBlobGasUsed( + final Transaction transaction, final ProtocolSpec protocolSpec) { + return transaction.getType().supportsBlob() + ? Optional.of(protocolSpec.getGasCalculator().blobGasCost(transaction.getBlobCount())) + : Optional.empty(); + } + + public long getMinimumTransactionCost(final ProcessableBlockHeader header) { + return protocolSchedule.getByBlockHeader(header).getGasCalculator().getMinimumTransactionCost(); + } + + public long getTransactionGasLimitCap(final ProcessableBlockHeader header) { + return protocolSchedule + .getByBlockHeader(header) + .getGasLimitCalculator() + .transactionGasLimitCap(); + } + + /** + * Calculates the blob gas price for data in a transaction. + * + * @param transaction the transaction to calculate the gas price for + * @param header the block header of the current block + * @param protocolSpec the protocol specification to use for gas price calculation + * @return an Optional containing the blob gas price for data if the transaction type supports + * blobs, otherwise returns an empty Optional + */ + private Optional getBlobGasPrice( + final Transaction transaction, final BlockHeader header, final ProtocolSpec protocolSpec) { + if (transaction.getType().supportsBlob()) { + return blockchain + .getBlockHeader(header.getParentHash()) + .map( + parentHeader -> + protocolSpec + .getFeeMarket() + .blobGasPricePerGas( + calculateExcessBlobGasForParent(protocolSpec, parentHeader))); + } + return Optional.empty(); + } + + /** + * Retrieve logs from the range of blocks with optional filtering based on logger address and log + * topics. + * + * @param fromBlockNumber The block number defining the first block in the search range + * (inclusive). + * @param toBlockNumber The block number defining the last block in the search range (inclusive). + * @param query Constraints on required topics by topic index. For a given index if the set of + * topics is non-empty, the topic at this index must match one of the values in the set. + * @param isQueryAlive Whether the backend query should stay alive. + * @return The set of logs matching the given constraints. + */ + public List matchingLogs( + final long fromBlockNumber, + final long toBlockNumber, + final LogsQuery query, + final Supplier isQueryAlive) { + try { + final List result = new ArrayList<>(); + final long startSegment = fromBlockNumber / BLOCKS_PER_BLOOM_CACHE; + final long endSegment = toBlockNumber / BLOCKS_PER_BLOOM_CACHE; + long currentStep = fromBlockNumber; + for (long segment = startSegment; segment <= endSegment; segment++) { + final long thisSegment = segment; + final long thisStep = currentStep; + final long nextStep = (segment + 1) * BLOCKS_PER_BLOOM_CACHE; + BackendQuery.stopIfExpired(isQueryAlive); + result.addAll( + cachePath + .map(path -> path.resolve("logBloom-" + thisSegment + ".cache")) + .filter(Files::isRegularFile) + .map( + cacheFile -> { + try { + return matchingLogsCached( + thisSegment * BLOCKS_PER_BLOOM_CACHE, + thisStep % BLOCKS_PER_BLOOM_CACHE, + Math.min(toBlockNumber, nextStep - 1) % BLOCKS_PER_BLOOM_CACHE, + query, + cacheFile, + isQueryAlive); + } catch (final Exception e) { + throw new RuntimeException(e); + } + }) + .orElseGet( + () -> + matchingLogsUncached( + thisStep, + Math.min(toBlockNumber, Math.min(toBlockNumber, nextStep - 1)), + query, + isQueryAlive))); + currentStep = nextStep; + } + return result; + } catch (final Exception e) { + throw new IllegalStateException("Error retrieving matching logs", e); + } + } + + private List matchingLogsUncached( + final long fromBlockNumber, + final long toBlockNumber, + final LogsQuery query, + final Supplier isQueryAlive) { + // rangeClosed handles the inverted from/to situations automatically with zero results. + return LongStream.rangeClosed(fromBlockNumber, toBlockNumber) + .mapToObj(blockchain::getBlockHeader) + // Use takeWhile instead of clamping on toBlockNumber/headBlockNumber because it may get an + // extra block or two for a query that has a toBlockNumber past chain head. Similarly this + // handles the case when fromBlockNumber is past chain head. + .takeWhile(Optional::isPresent) + .map(Optional::get) + .filter(header -> query.couldMatch(header.getLogsBloom())) + .flatMap(header -> matchingLogs(header.getHash(), query, isQueryAlive).stream()) + .collect(Collectors.toList()); + } + + private List matchingLogsCached( + final long segmentStart, + final long offset, + final long endOffset, + final LogsQuery query, + final Path cacheFile, + final Supplier isQueryAlive) + throws Exception { + final List results = new ArrayList<>(); + try (final RandomAccessFile raf = new RandomAccessFile(cacheFile.toFile(), "r")) { + raf.seek(offset * 256); + final byte[] bloomBuff = new byte[256]; + final Bytes bytesValue = Bytes.wrap(bloomBuff); + for (long pos = offset; pos <= endOffset; pos++) { + BackendQuery.stopIfExpired(isQueryAlive); + try { + raf.readFully(bloomBuff); + } catch (final EOFException e) { + results.addAll( + matchingLogsUncached( + segmentStart + pos, segmentStart + endOffset, query, isQueryAlive)); + break; + } + final LogsBloomFilter logsBloom = new LogsBloomFilter(bytesValue); + if (query.couldMatch(logsBloom)) { + results.addAll( + matchingLogs( + blockchain.getBlockHashByNumber(segmentStart + pos).orElseThrow(), + query, + isQueryAlive)); + } + } + } catch (final IOException e) { + e.printStackTrace(System.out); + LOG.error("Error reading cached log blooms", e); + } + return results; + } + + public List matchingLogs( + final Hash blockHash, final LogsQuery query, final Supplier isQueryAlive) { + try { + final Optional blockHeader = getBlockHeader(blockHash, isQueryAlive); + if (blockHeader.isEmpty()) { + return Collections.emptyList(); + } + // receipts and transactions should exist if the header exists, so throwing is ok. + final List receipts = getReceipts(blockHash, isQueryAlive); + final List transactions = getTransactions(blockHash, isQueryAlive); + final long number = blockHeader.get().getNumber(); + final long blockTimestamp = blockHeader.get().getTimestamp(); + final boolean removed = getRemoved(blockHash, isQueryAlive); + + final AtomicInteger logIndexOffset = new AtomicInteger(); + return IntStream.range(0, receipts.size()) + .mapToObj( + i -> { + try { + BackendQuery.stopIfExpired(isQueryAlive); + final List result = + LogWithMetadata.generate( + logIndexOffset.intValue(), + receipts.get(i), + number, + blockHash, + blockTimestamp, + transactions.get(i).getHash(), + i, + removed); + logIndexOffset.addAndGet(receipts.get(i).getLogs().size()); + return result; + } catch (final Exception e) { + throw new RuntimeException(e); + } + }) + .flatMap(Collection::stream) + .filter(query::matches) + .collect(Collectors.toList()); + } catch (final Exception e) { + throw new RuntimeException(e); + } + } + + public List matchingLogs( + final Hash blockHash, + final TransactionWithMetadata transactionWithMetaData, + final Supplier isQueryAlive) { + if (transactionWithMetaData.getTransactionIndex().isEmpty()) { + throw new RuntimeException( + "Cannot find logs because transaction " + + transactionWithMetaData.getTransaction().getHash() + + " does not have a transaction index"); + } + + try { + final Optional blockHeader = getBlockHeader(blockHash, isQueryAlive); + if (blockHeader.isEmpty()) { + return Collections.emptyList(); + } + // receipts and transactions should exist if the header exists, so throwing is ok. + final List receipts = getReceipts(blockHash, isQueryAlive); + final List transactions = getTransactions(blockHash, isQueryAlive); + final long number = blockHeader.get().getNumber(); + final long blockTimestamp = blockHeader.get().getTimestamp(); + final boolean removed = getRemoved(blockHash, isQueryAlive); + + final int transactionIndex = transactionWithMetaData.getTransactionIndex().get(); + final int logIndexOffset = + logIndexOffset( + transactionWithMetaData.getTransaction().getHash(), receipts, transactions); + + return LogWithMetadata.generate( + logIndexOffset, + receipts.get(transactionIndex), + number, + blockHash, + blockTimestamp, + transactions.get(transactionIndex).getHash(), + transactionIndex, + removed); + + } catch (final Exception e) { + throw new RuntimeException(e); + } + } + + // AERE D-109 (2026-09-10): the world state of a block that has JUST become the chain head is not + // available for a few milliseconds (measured on the live readers: 2-96 ms, occasionally >300 ms) + // after its header is; a query resolved against 'latest' in that window used to return + // Optional.empty, which JSON-RPC serialises as a bare null - a value Besu never returns otherwise + // for eth_getCode/getBalance/getStorageAt/getTransactionCount. Measured directly on a reader: + // ~1 in 5,000 'latest' queries, always within a few ms of an import, never for older blocks. So + // when the lookup fails for a header at (or within HEAD_WINDOW blocks of) the chain head, we look + // again a bounded number of times, with short pauses, before giving up. Older blocks are not + // retried: for them an empty result is the state window closing, and a retry would only add latency. + private static final long AERE_HEAD_WINDOW_BLOCKS = 2; + private static final long[] AERE_HEAD_STATE_RETRY_PAUSES_MS = {50, 100, 150, 300}; + + + /** + * Wraps an operation on MutableWorldState with try-with-resources the corresponding block hash. + * This method provides access to the worldstate via a mapper function in order to ensure all uses + * of the MutableWorldState are subsequently closed, via the try-with-resources block. + * + * @param return type of the operation on the MutableWorldState + * @param blockHash the block hash + * @param mapper Function which performs an operation on a MutableWorldState + * @return the world state at the block number + */ + public Optional getAndMapWorldState( + final Hash blockHash, final Function> mapper) { + + return blockchain + .getBlockHeader(blockHash) + .flatMap( + blockHeader -> { + int attempts = 0; + while (true) { + try (var ws = + worldStateArchive + .getWorldState(withBlockHeaderAndNoUpdateNodeHead(blockHeader)) + .orElse(null)) { + if (ws != null) { + if (attempts > 0) { + LOG.info( + "AERE-D109: world state for {} became available after {} retr{}", + blockHeader.toLogString(), + attempts, + attempts == 1 ? "y" : "ies"); + } + return mapper.apply(ws); + } + } catch (Exception ex) { + LOG.error( + "failed worldstate query for " + blockHash.getBytes().toShortHexString(), ex); + return Optional.empty(); + } + if (!aereShouldRetryHeadState(blockHeader, attempts)) { + break; + } + try { + Thread.sleep(AERE_HEAD_STATE_RETRY_PAUSES_MS[attempts]); + } catch (InterruptedException ie) { + Thread.currentThread().interrupt(); + break; + } + attempts++; + } + LOG.atDebug() + .setMessage("Failed to find worldstate for {}") + .addArgument(blockHeader.toLogString()) + .log(); + return Optional.empty(); + }); + } + + // AERE D-109: retry only near the head, and only a bounded number of times. + boolean aereShouldRetryHeadState(final BlockHeader blockHeader, final int attempts) { + if (attempts >= AERE_HEAD_STATE_RETRY_PAUSES_MS.length) { + return false; + } + final long head = blockchain.getChainHeadBlockNumber(); + return head - blockHeader.getNumber() <= AERE_HEAD_WINDOW_BLOCKS; + } + + /** + * Wraps an operation on MutableWorldState with try-with-resources the corresponding block number + * + * @param return type of the operation on the MutableWorldState + * @param blockNumber the block number + * @param mapper Function which performs an operation on a MutableWorldState returning type U + * @return the world state at the block number + */ + public Optional getAndMapWorldState( + final long blockNumber, final Function> mapper) { + final Hash blockHash = + getBlockHeaderByNumber(blockNumber).map(BlockHeader::getHash).orElse(Hash.EMPTY); + return getAndMapWorldState(blockHash, mapper); + } + + public Wei gasPrice() { + final Block chainHeadBlock = blockchain.getChainHeadBlock(); + final var chainHeadHeader = chainHeadBlock.getHeader(); + final long blockHeight = chainHeadHeader.getNumber(); + + final var nextBlockProtocolSpec = + protocolSchedule.getForNextBlockHeader(chainHeadHeader, System.currentTimeMillis()); + final var nextBlockFeeMarket = nextBlockProtocolSpec.getFeeMarket(); + + final Wei[] gasCollection = + Stream.concat( + LongStream.range( + Math.max(0, blockHeight - apiConfig.getGasPriceBlocks() + 1), blockHeight) + .mapToObj( + l -> + blockchain + .getBlockByNumber(l) + .orElseThrow( + () -> + new IllegalStateException( + "Could not retrieve block #" + l))), + Stream.of(chainHeadBlock)) + .map(Block::getBody) + .map(BlockBody::getTransactions) + .flatMap(Collection::stream) + .filter(t -> t.getGasPrice().isPresent()) + .map(t -> t.getGasPrice().get()) + .toArray(Wei[]::new); + + return gasCollection.length == 0 + ? gasPriceLowerBound(chainHeadHeader, nextBlockFeeMarket) + : UInt256s.max( + gasPriceLowerBound(chainHeadHeader, nextBlockFeeMarket), + UInt256s.min( + apiConfig.getGasPriceMax(), + OrderStatistics.selectKthInPlace( + gasCollection, + Math.min( + gasCollection.length - 1, + (int) ((gasCollection.length) * apiConfig.getGasPriceFraction()))))); + } + + /** + * Return the min gas required for a tx to be mineable. On networks with gas price fee market it + * is just the minGasPrice, while on networks with base fee market it is the max between the + * minGasPrice and the baseFee for the next block. + * + * @return the min gas required for a tx to be mineable. + */ + public Wei gasPriceLowerBound() { + final var chainHeadHeader = blockchain.getChainHeadHeader(); + final var nextBlockProtocolSpec = + protocolSchedule.getForNextBlockHeader(chainHeadHeader, System.currentTimeMillis()); + final var nextBlockFeeMarket = nextBlockProtocolSpec.getFeeMarket(); + return gasPriceLowerBound(chainHeadHeader, nextBlockFeeMarket); + } + + private Wei gasPriceLowerBound( + final BlockHeader chainHeadHeader, final FeeMarket nextBlockFeeMarket) { + final var minGasPrice = miningConfiguration.getMinTransactionGasPrice(); + + if (nextBlockFeeMarket.implementsBaseFee()) { + return UInt256s.max( + getNextBlockBaseFee(chainHeadHeader, (BaseFeeMarket) nextBlockFeeMarket), minGasPrice); + } + + return minGasPrice; + } + + public Wei gasPriorityFee() { + final Block chainHeadBlock = blockchain.getChainHeadBlock(); + final long blockHeight = chainHeadBlock.getHeader().getNumber(); + + final Wei[] gasCollection = + Stream.concat( + LongStream.range( + Math.max(0, blockHeight - apiConfig.getGasPriceBlocks() + 1), blockHeight) + .mapToObj( + l -> + blockchain + .getBlockByNumber(l) + .orElseThrow( + () -> + new IllegalStateException( + "Could not retrieve block #" + l))), + Stream.of(chainHeadBlock)) + .map(Block::getBody) + .map(BlockBody::getTransactions) + .flatMap(Collection::stream) + .filter(t -> t.getMaxPriorityFeePerGas().isPresent()) + .map(t -> t.getMaxPriorityFeePerGas().get()) + .toArray(Wei[]::new); + + return gasCollection.length == 0 + ? miningConfiguration.getMinPriorityFeePerGas() + : UInt256s.max( + miningConfiguration.getMinPriorityFeePerGas(), + OrderStatistics.selectKthInPlace( + gasCollection, + Math.min( + gasCollection.length - 1, + (int) ((gasCollection.length) * apiConfig.getGasPriceFraction())))); + } + + /** + * Calculate and return the value of the base fee for the next block, if the network has a base + * fee market, otherwise return empty. + * + * @return the optional base fee + */ + public Optional getNextBlockBaseFee() { + final var chainHeadHeader = blockchain.getChainHeadHeader(); + final var nextBlockProtocolSpec = + protocolSchedule.getForNextBlockHeader(chainHeadHeader, System.currentTimeMillis()); + final var nextBlockFeeMarket = nextBlockProtocolSpec.getFeeMarket(); + return nextBlockFeeMarket.implementsBaseFee() + ? Optional.of(getNextBlockBaseFee(chainHeadHeader, (BaseFeeMarket) nextBlockFeeMarket)) + : Optional.empty(); + } + + private Wei getNextBlockBaseFee( + final BlockHeader chainHeadHeader, final BaseFeeMarket nextBlockFeeMarket) { + return nextBlockFeeMarket.computeBaseFee( + chainHeadHeader.getNumber() + 1, + chainHeadHeader.getBaseFee().orElse(Wei.ZERO), + chainHeadHeader.getGasUsed(), + nextBlockFeeMarket.targetGasUsed(chainHeadHeader)); + } + + private Optional fromAccount( + final Address address, + final Hash blockHash, + final Function getter, + final T noAccountValue) { + return getAndMapWorldState( + blockHash, + worldState -> + Optional.ofNullable(worldState.get(address)) + .map(getter) + .or(() -> Optional.ofNullable(noAccountValue))); + } + + private List formatTransactions( + final List txs, + final long blockNumber, + final Optional baseFee, + final Hash blockHash, + final long blockTimestamp) { + final int count = txs.size(); + final List result = new ArrayList<>(count); + for (int i = 0; i < count; i++) { + result.add( + new TransactionWithMetadata( + txs.get(i), blockNumber, baseFee, blockHash, i, blockTimestamp)); + } + return result; + } + + private boolean outsideBlockchainRange(final long blockNumber) { + return blockNumber > headBlockNumber() || blockNumber < BlockHeader.GENESIS_BLOCK_NUMBER; + } + + private Boolean getRemoved(final Hash blockHash, final Supplier isQueryAlive) + throws Exception { + return BackendQuery.runIfAlive( + "matchingLogs - blockIsOnCanonicalChain", + () -> !blockchain.blockIsOnCanonicalChain(blockHash), + isQueryAlive); + } + + private List getTransactions( + final Hash blockHash, final Supplier isQueryAlive) throws Exception { + return BackendQuery.runIfAlive( + "matchingLogs - getBlockBody", + () -> + blockchain + .getBlockBody(blockHash) + .map((bb) -> bb.getTransactions()) + .orElse(Collections.emptyList()), + isQueryAlive); + } + + private List getReceipts( + final Hash blockHash, final Supplier isQueryAlive) throws Exception { + return BackendQuery.runIfAlive( + "matchingLogs - getTxReceipts", + () -> blockchain.getTxReceipts(blockHash).orElse(Collections.emptyList()), + isQueryAlive); + } + + private Optional getBlockHeader( + final Hash blockHash, final Supplier isQueryAlive) throws Exception { + return BackendQuery.runIfAlive( + "matchingLogs - getBlockHeader", () -> blockchain.getBlockHeader(blockHash), isQueryAlive); + } + + private int logIndexOffset( + final Hash transactionHash, + final List receipts, + final List transactions) { + int logIndexOffset = 0; + for (int i = 0; i < receipts.size(); i++) { + if (transactions.get(i).getHash().equals(transactionHash)) { + break; + } + + logIndexOffset += receipts.get(i).getLogsList().size(); + } + + return logIndexOffset; + } + + public Optional getEthScheduler() { + return ethScheduler; + } +} diff --git a/anchor/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/AereStateUnavailableIsAnErrorTest.java b/anchor/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/AereStateUnavailableIsAnErrorTest.java new file mode 100644 index 0000000..d2caccd --- /dev/null +++ b/anchor/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/jsonrpc/internal/methods/AereStateUnavailableIsAnErrorTest.java @@ -0,0 +1,138 @@ +/* + * Copyright contributors to Hyperledger Besu. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +import org.hyperledger.besu.datatypes.Address; +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.datatypes.Wei; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequest; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse; +import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType; +import org.hyperledger.besu.ethereum.api.query.BlockchainQueries; +import org.hyperledger.besu.ethereum.core.BlockHeader; +import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool; + +import java.util.Optional; + +import org.apache.tuweni.bytes.Bytes; +import org.apache.tuweni.units.bigints.UInt256; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +/** + * AERE D-FAZA2-04: when the world state for the requested block is not available, the four state + * methods answer with the WORLD_STATE_UNAVAILABLE error, never with a bare null (or a nonce of 0). + * When the state IS available they answer as before. + */ +public class AereStateUnavailableIsAnErrorTest { + + private static final Address ADDR = Address.fromHexString("0x1111111111111111111111111111111111111111"); + private static final Hash BLOCK_HASH = Hash.fromHexStringLenient("0xabcd"); + private static final long HEAD = 1000L; + + private BlockchainQueries queries; + + @BeforeEach + public void setUp() { + queries = mock(BlockchainQueries.class); + // the numeric block parameter path: number -> hash + when(queries.headBlockNumber()).thenReturn(HEAD); + when(queries.getBlockHashByNumber(HEAD)).thenReturn(Optional.of(BLOCK_HASH)); + final BlockHeader header = mock(BlockHeader.class); + when(header.getBlockHash()).thenReturn(BLOCK_HASH); + when(queries.getBlockHeaderByNumber(HEAD)).thenReturn(Optional.of(header)); + when(queries.getBlockHeaderByHash(BLOCK_HASH)).thenReturn(Optional.of(header)); + } + + private static JsonRpcRequestContext request(final String method, final Object... params) { + return new JsonRpcRequestContext(new JsonRpcRequest("2.0", method, params)); + } + + private static void assertWorldStateUnavailable(final JsonRpcResponse response) { + assertThat(response).isInstanceOf(JsonRpcErrorResponse.class); + assertThat(((JsonRpcErrorResponse) response).getErrorType()) + .isEqualTo(RpcErrorType.WORLD_STATE_UNAVAILABLE); + } + + private static Object result(final JsonRpcResponse response) { + assertThat(response).isInstanceOf(JsonRpcSuccessResponse.class); + return ((JsonRpcSuccessResponse) response).getResult(); + } + + @Test + public void getCodeWithoutStateIsAnError() { + when(queries.getCode(any(Address.class), any(Hash.class))).thenReturn(Optional.empty()); + final EthGetCode method = new EthGetCode(queries); + assertWorldStateUnavailable(method.response(request("eth_getCode", ADDR.toHexString(), "0x3e8"))); + } + + @Test + public void getCodeWithStateAnswersAsBefore() { + when(queries.getCode(any(Address.class), any(Hash.class))).thenReturn(Optional.of(Bytes.fromHexString("0x6001"))); + final EthGetCode method = new EthGetCode(queries); + assertThat(result(method.response(request("eth_getCode", ADDR.toHexString(), "0x3e8")))).isEqualTo("0x6001"); + } + + @Test + public void getBalanceWithoutStateIsAnError() { + when(queries.accountBalance(any(Address.class), any(Hash.class))).thenReturn(Optional.empty()); + final EthGetBalance method = new EthGetBalance(queries); + assertWorldStateUnavailable(method.response(request("eth_getBalance", ADDR.toHexString(), "0x3e8"))); + } + + @Test + public void getBalanceWithStateAnswersAsBefore() { + when(queries.accountBalance(any(Address.class), any(Hash.class))).thenReturn(Optional.of(Wei.of(7))); + final EthGetBalance method = new EthGetBalance(queries); + assertThat(result(method.response(request("eth_getBalance", ADDR.toHexString(), "0x3e8")))).isEqualTo("0x7"); + } + + @Test + public void getStorageAtWithoutStateIsAnError() { + when(queries.storageAt(any(Address.class), any(UInt256.class), any(Hash.class))).thenReturn(Optional.empty()); + final EthGetStorageAt method = new EthGetStorageAt(queries); + assertWorldStateUnavailable(method.response(request("eth_getStorageAt", ADDR.toHexString(), "0x0", "0x3e8"))); + } + + @Test + public void getStorageAtWithStateAnswersAsBefore() { + when(queries.storageAt(any(Address.class), any(UInt256.class), any(Hash.class))).thenReturn(Optional.of(UInt256.valueOf(5))); + final EthGetStorageAt method = new EthGetStorageAt(queries); + assertThat(result(method.response(request("eth_getStorageAt", ADDR.toHexString(), "0x0", "0x3e8")))) + .isEqualTo(UInt256.valueOf(5).toHexString()); + } + + @Test + public void getTransactionCountWithoutStateIsAnErrorNotZero() { + when(queries.getTransactionCountIfStateAvailable(any(Address.class), any(Hash.class))).thenReturn(Optional.empty()); + final EthGetTransactionCount method = new EthGetTransactionCount(queries, mock(TransactionPool.class)); + assertWorldStateUnavailable(method.response(request("eth_getTransactionCount", ADDR.toHexString(), "0x3e8"))); + } + + @Test + public void getTransactionCountWithStateAnswersAsBefore() { + when(queries.getTransactionCountIfStateAvailable(any(Address.class), any(Hash.class))).thenReturn(Optional.of(3L)); + final EthGetTransactionCount method = new EthGetTransactionCount(queries, mock(TransactionPool.class)); + assertThat(result(method.response(request("eth_getTransactionCount", ADDR.toHexString(), "0x3e8")))).isEqualTo("0x3"); + } +} diff --git a/anchor/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/query/AereHeadStateRetryTest.java b/anchor/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/query/AereHeadStateRetryTest.java new file mode 100644 index 0000000..33be192 --- /dev/null +++ b/anchor/ethereum/api/src/test/java/org/hyperledger/besu/ethereum/api/query/AereHeadStateRetryTest.java @@ -0,0 +1,140 @@ +/* + * Copyright contributors to Hyperledger Besu. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on + * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the + * specific language governing permissions and limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ +package org.hyperledger.besu.ethereum.api.query; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import org.hyperledger.besu.datatypes.Hash; +import org.hyperledger.besu.ethereum.chain.Blockchain; +import org.hyperledger.besu.ethereum.core.BlockHeader; +import org.hyperledger.besu.ethereum.core.MiningConfiguration; +import org.hyperledger.besu.ethereum.core.MutableWorldState; +import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule; +import org.hyperledger.besu.ethereum.worldstate.WorldStateArchive; +import org.hyperledger.besu.ethereum.trie.pathbased.common.provider.WorldStateQueryParams; + +import java.util.Optional; + +import org.junit.jupiter.api.Test; + +/** + * AERE D-109: a query against a block that has just become the chain head can find no world state + * for a few milliseconds. {@link BlockchainQueries#getAndMapWorldState(Hash, java.util.function.Function)} + * must look again (a bounded number of times) near the head, and must NOT retry for old blocks, + * where an empty result is the state window closing. + */ +public class AereHeadStateRetryTest { + + private static final Hash HASH = Hash.fromHexStringLenient("0x1234"); + + private static BlockHeader headerAt(final long number) { + final BlockHeader header = mock(BlockHeader.class); + when(header.getNumber()).thenReturn(number); + when(header.getHash()).thenReturn(HASH); + when(header.getBlockHash()).thenReturn(HASH); + when(header.getStateRoot()).thenReturn(Hash.EMPTY_TRIE_HASH); + when(header.getParentHash()).thenReturn(Hash.ZERO); + when(header.toLogString()).thenReturn("#" + number); + return header; + } + + private static BlockchainQueries queries( + final Blockchain blockchain, final WorldStateArchive archive) { + return new BlockchainQueries( + mock(ProtocolSchedule.class), blockchain, archive, MiningConfiguration.newDefault()); + } + + @Test + public void headBlockStateThatAppearsOnSecondLookIsReturned() { + final Blockchain blockchain = mock(Blockchain.class); + final WorldStateArchive archive = mock(WorldStateArchive.class); + final BlockHeader head = headerAt(1000); + when(blockchain.getBlockHeader(HASH)).thenReturn(Optional.of(head)); + when(blockchain.getChainHeadBlockNumber()).thenReturn(1000L); + final MutableWorldState ws = mock(MutableWorldState.class); + when(archive.getWorldState(any(WorldStateQueryParams.class))) + .thenReturn(Optional.empty()) + .thenReturn(Optional.of(ws)); + + final Optional result = + queries(blockchain, archive).getAndMapWorldState(HASH, w -> Optional.of("ok")); + + assertThat(result).contains("ok"); + verify(archive, times(2)).getWorldState(any(WorldStateQueryParams.class)); + } + + @Test + public void oldBlockWithoutStateIsNotRetried() { + // negative control: an old block gets exactly ONE look, and an empty answer stays empty + final Blockchain blockchain = mock(Blockchain.class); + final WorldStateArchive archive = mock(WorldStateArchive.class); + final BlockHeader old = headerAt(1000); + when(blockchain.getBlockHeader(HASH)).thenReturn(Optional.of(old)); + when(blockchain.getChainHeadBlockNumber()).thenReturn(5000L); + when(archive.getWorldState(any(WorldStateQueryParams.class))).thenReturn(Optional.empty()); + + final Optional result = + queries(blockchain, archive).getAndMapWorldState(HASH, w -> Optional.of("ok")); + + assertThat(result).isEmpty(); + verify(archive, times(1)).getWorldState(any(WorldStateQueryParams.class)); + } + + @Test + public void headBlockThatNeverGetsStateStopsAfterBoundedRetries() { + // the retry is bounded: a head block that never gets its state is asked 1 + 4 times, then empty + final Blockchain blockchain = mock(Blockchain.class); + final WorldStateArchive archive = mock(WorldStateArchive.class); + final BlockHeader head = headerAt(1000); + when(blockchain.getBlockHeader(HASH)).thenReturn(Optional.of(head)); + when(blockchain.getChainHeadBlockNumber()).thenReturn(1001L); + when(archive.getWorldState(any(WorldStateQueryParams.class))).thenReturn(Optional.empty()); + + final long t0 = System.nanoTime(); + final Optional result = + queries(blockchain, archive).getAndMapWorldState(HASH, w -> Optional.of("ok")); + final long elapsedMs = (System.nanoTime() - t0) / 1_000_000; + + assertThat(result).isEmpty(); + verify(archive, times(5)).getWorldState(any(WorldStateQueryParams.class)); + // 50 + 100 + 150 + 300 ms of pauses, and nothing unbounded + assertThat(elapsedMs).isBetween(550L, 5000L); + } + + @Test + public void stateFoundOnFirstLookIsReturnedWithoutWaiting() { + final Blockchain blockchain = mock(Blockchain.class); + final WorldStateArchive archive = mock(WorldStateArchive.class); + final BlockHeader head = headerAt(1000); + when(blockchain.getBlockHeader(HASH)).thenReturn(Optional.of(head)); + when(blockchain.getChainHeadBlockNumber()).thenReturn(1000L); + when(archive.getWorldState(any(WorldStateQueryParams.class))) + .thenReturn(Optional.of(mock(MutableWorldState.class))); + + final long t0 = System.nanoTime(); + final Optional result = + queries(blockchain, archive).getAndMapWorldState(HASH, w -> Optional.of("ok")); + final long elapsedMs = (System.nanoTime() - t0) / 1_000_000; + + assertThat(result).contains("ok"); + verify(archive, times(1)).getWorldState(any(WorldStateQueryParams.class)); + assertThat(elapsedMs).isLessThan(50L); + } +} diff --git a/run-node.sh b/run-node.sh index 6bb70ed..5cb7a6b 100644 --- a/run-node.sh +++ b/run-node.sh @@ -15,7 +15,7 @@ # Re-running it does not rebuild what is already built or re-apply patches already applied. # # Usage: -# ./run-node.sh [options] (the two published bootnodes are used unless --bootnode is given) +# ./run-node.sh [options] (the published bootnode is used unless --bootnode is given) # # Options: # --bootnode Override the published bootnodes with your own peer for discovery. @@ -207,10 +207,10 @@ fi # --- 6. start ---------------------------------------------------------------------------------- say "6/6 Starting the node" if [ -z "$BOOTNODE" ]; then - # The two published bootnodes (RUN-A-NODE.md, bootnodes section): both public read hosts, - # neither in the validator set, one Besu and one Nethermind. --bootnode overrides them. - BOOTNODE="enode://7e8ff740b79bf28a6d7e46ea8d2317cea4a223e7af60be36e91e13dc7051f4cb29077631452e699a5e18e640a94fd6e370e9e51dd91ecba1f2ba9666e07ff01b@37.27.216.110:30303,enode://a4b90d4f5fc7758814d41acfe00f6a2516683f3d3052410091b894d014dbb2184d6158da76f72e84ca268c2ccde3590d5686f978868f3b6ad6af87e398179fb8@157.180.67.185:30303" - say " no --bootnode given; using the two published bootnodes from RUN-A-NODE.md" + # The published bootnode (RUN-A-NODE.md, bootnodes section): a public read host, not in the validator + # set. A second address was published from 2026-08-25 to 2026-09-11 and withdrawn. --bootnode overrides it. + BOOTNODE="enode://7e8ff740b79bf28a6d7e46ea8d2317cea4a223e7af60be36e91e13dc7051f4cb29077631452e699a5e18e640a94fd6e370e9e51dd91ecba1f2ba9666e07ff01b@37.27.216.110:30303" + say " no --bootnode given; using the published bootnode from RUN-A-NODE.md" fi mkdir -p "$DATA_PATH"