Glamsterdam devnet-8 — RPC interoperability after a seven-client re-test
Prepared for ACDT · 2026-08-24 10:14 UTC · geth, Besu, Nethermind, reth, Erigon, Ethrex and Nimbus-EL · live post-Amsterdam/Gloas network · identical requests sent to one Teku-paired node per execution client.
The EVM consensus surface remains clean; the RPC surface does not. The original devnet-7 differential found no execution-result divergence after 22 rounds, but catalogued 28 RPC, debug and method-support differences. Re-running every finding on devnet-8 shows that only three are fixed on their originally affected client, three are partially improved, and 21 still reproduce.
This report distinguishes unambiguous client defects, missing optional coverage, implementation conventions, and questions that need a cross-client rule. Missing methods and different debug output are not presented as consensus bugs.
tested live
still reproduce
improved
original client
in this RPC suite
Network control: at publication the network was synced and finalizing normally, at current epoch 2458 and finalized epoch 2456. The findings below are live RPC behavior, not symptoms of a stalled or partitioned chain.
1 · The ACDT asks
| Area | Observed split | Suggested outcome |
|---|---|---|
eth_simulateV1.traceTransfers × EIP-7708 |
Three clients return both the legacy synthetic 0xeeee…eeee log and the protocol
0xffff…fffe log; Besu substitutes the synthetic log; Nethermind has a third behavior;
two clients do not implement the method. |
Specify whether traceTransfers=true adds to, replaces, or suppresses the EIP-7708
protocol log. Add one shared fixture before every client independently freezes a different rule. |
eth_estimateGas after EIP-2780/8037 |
Up to three answers for simple transfers and five for a storage/refund probe. All observed answers are safe over-estimates, but wallets receive client-dependent gas limits. | Decide whether the API promises the executable minimum or only a safe bound. In either case, publish cross-client fixtures for the repriced paths. |
| Debug tracer normalization | Default tracer, tracer name, depth convention and per-opcode state-gas attribution differ. The same SSTORE step differs by 97,920 gas while total execution agrees. | Continue standardization in execution-apis; require total-gas fields to distinguish regular and state gas, while explicitly allowing or forbidding per-step attribution choices. |
| Filter and execution-error semantics | First historical eth_getFilterChanges poll is empty on four clients and full on reth;
memory OOG maps to three error codes, including code 3 (revert) on Ethrex. |
Define first-poll behavior for closed historical ranges and a portable error category for OOG versus REVERT, even if exact messages remain implementation-defined. |
2 · Clear client defects still present
| ID | Client | Live devnet-8 result | Impact / status |
|---|---|---|---|
| D4 analogue | Nimbus-EL | A value transfer to a fresh account returns success from eth_call at gas limits
21,000, 50,000 and 204,599. The other six return OOG; all succeed at 204,600. |
Simulation masks OOG and can cause tooling to underfund a transaction. The original Nethermind bug is fixed; the behavior moved to the seventh client newly included in this run. |
| D8 | Besu, Ethrex | eth_createAccessList returns no entry for account-only BALANCE and
EXTCODESIZE accesses, for both existing and non-existent accounts. Storage accesses work. |
The returned list does not pre-warm the accessed account. Besu: issue #11134. |
| D12 | Ethrex | Receipt/log timestamps are now fixed, but blockTimestamp is still absent from every
transaction returned by eth_getTransactionByHash and full-block queries. |
Consumers need a block lookup fallback for all Ethrex transaction objects. |
| D18 | Besu | eth_feeHistory accepts reward percentiles 150 and -5 even
though execution-apis constrains values to 0–100. |
Invalid input succeeds and cross-client validation diverges. Evidence added to issue #11055. |
| D22 | Erigon | eth_getProof succeeds at head and fails at head−1, −2, −4 and −16 with
root hash mismatch in account proof trie. |
Proofs at finalized blocks are unusable. The failure is safe—it errors rather than returning an invalid proof—but it is not a pruning/window response. |
| D23 | Erigon | For the same closed two-block range, eth_getLogs returned 474 logs while a filter
created with the same object returned zero from eth_getFilterLogs. |
Silent data loss for consumers using filter handles. |
| D24 | Ethrex | eth_getLogs and eth_newFilter reject a filter without topics
as missing a required parameter. The other six accept the field's omission. |
Rejects a valid filter object; breaks generic log tooling. |
Also unambiguously stale, but lower impact: Erigon still rejects short storage
keys such as 0x0 in eth_getProof (D11); Erigon returns a degenerate null-ratio
response for eth_feeHistory block counts ≥263 (D19); and Ethrex still requires
the optional filter field above. These are RPC input/edge handling, not execution defects.
3 · Repricing makes eth_estimateGas observably client-dependent
All seven clients agree on real block execution. They do not agree on the gas limit a caller should submit for the same transaction. Values below are decimal and were measured with identical live requests on devnet-8.
| Call shape | Spec-derived executable minimum | geth | Besu | Nethermind | reth | Erigon | Ethrex | Nimbus |
|---|---|---|---|---|---|---|---|---|
| self-send, value=1 | 12,000 | 21,000 | 12,000 | 12,000 | 21,000 | 12,000 | 12,000 | 12,000 |
| value=0 → other existing account | 15,000 | 21,000 | 15,159 | 15,000 | 21,000 | 15,000 | 15,000 | 15,000 |
| value=1 → existing account | 21,000 | 21,000 | 21,165 | 21,000 | 21,000 | 21,000 | 21,000 | 21,000 |
| value=1 → fresh account | 204,600 | 207,391 | 207,391 | 207,391 | 207,391 | 204,600 | 204,600 | 204,600 |
The geth/reth 21,000 floor predates the EIP-2780 cheap paths. Besu adds a small margin on existing-account access. Five clients add 2,791 on a fresh-account transfer. These are safe over-estimates in the tested cases, so this is an interoperability and wallet-pricing issue—not a consensus claim.
4 · eth_simulateV1.traceTransfers has four outcomes
A single plain value transfer now causes a protocol-native EIP-7708 log from
0xfffffffffffffffffffffffffffffffffffffffe. The older simulation option synthesizes a
second ERC-20-shaped log from 0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee.
| Client | traceTransfers=false | traceTransfers=true | Interpretation |
|---|---|---|---|
| geth | ffff | eeee + ffff | double-counts |
| reth | ffff | eeee + ffff | double-counts |
| Erigon | ffff | eeee + ffff | double-counts |
| Besu | ffff | eeee | substitutes; drops protocol log |
| Nethermind | −32603 Internal error | ffff | different broken combination |
| Ethrex | −32601 Method not found | not implemented | |
| Nimbus-EL | −32601 not registered | not implemented | |
This is the most useful ACDT discussion item. No implementation can infer whether
the legacy synthetic log should coexist with a protocol log that did not exist when
traceTransfers was designed. A shared rule is cheaper now than seven client-specific
compatibility branches later.
5 · Method and shape coverage
A dash means missing or unusable, not “consensus incorrect.” Debug and txpool methods are valuable for devnet differential testing, but several are not standardized Ethereum JSON-RPC.
| Surface | geth | Besu | Nethermind | reth | Erigon | Ethrex | Nimbus |
|---|---|---|---|---|---|---|---|
debug_traceCall default struct trace | yes | yes | yes | yes | yes | call frame | — |
eth_simulateV1 | yes | yes | partial | yes | yes | — | — |
eth_call state overrides | yes | yes | yes | yes | yes | — | — |
txpool_status | yes | yes | yes | yes | yes | — | — |
| raw transaction by hash / block index | yes | — | yes | yes | yes | — | — |
eth_newFilter | yes | yes | yes | yes | yes | topics required | — |
eth_getFilterLogs | yes | yes | yes | yes | returns 0 | — | — |
trace_block / trace_call | —* | yes | yes | yes | yes | — | — |
debug_storageRangeAt | nextKey | complete | — | null | nextKey | — | — |
debug_accountRange | object | addressMap | — | null | raw-address cursor | — | — |
eth_getUncleCountByBlockNumber | yes | yes | yes | yes | yes | — | yes |
* geth has never implemented Parity-style trace_*; this is expected,
not a regression. Besu raw-transaction support is already tracked in
issue #10477.
6 · Complete D1–D28 status ledger
| ID | Status | Devnet-8 result | Classification |
|---|---|---|---|
| D1 | OPEN | Nimbus still rejects omitted/null params; [] works. | interop strictness |
| D2 | FIXED | Nethermind trace succeeds without explicit gasPrice. | original client fixed |
| D3 | OPEN | Ethrex defaults to callTracer, rejects structLogger, names it opcodeTracer. | tracer convention |
| D4 | MOVED | Nethermind fixed; Nimbus now returns success below the runtime-OOG threshold. | client defect |
| D5 | OPEN | Estimate-gas spread remains, including legacy 21,000 floors. | needs API expectation |
| D6 | OPEN | 49,152-byte deployment: Besu/Nethermind/Ethrex proceed; geth/reth/Erigon cap/OOG. | node policy |
| D7 | OPEN | SSTORE per-step cost splits 3–3 by 97,920; no consensus claim. | tracer attribution |
| D8 | OPEN | Besu/Ethrex omit account-only accesses from generated lists. | client defect |
| D9 | OPEN | Memory OOG returns -32000, reth -32003, Ethrex 3; Nimbus returns empty success on the no-to probe. | error semantics |
| D10 | DECISION | Four distinct simulate/transfer-log outcomes. | spec interaction |
| D11 | OPEN | Erigon rejects short storage key 0x0. | input strictness |
| D12 | PARTIAL | Ethrex log timestamps fixed; transaction timestamps still absent. | client defect |
| D13 | OPEN | Besu/Ethrex add depth=2 step for empty sub-call; others do not. | tracer convention |
| D14 | OPEN | Ethrex and Nimbus cannot serve the tested eth_call state override. | coverage gap |
| D15 | OPEN | Ethrex and Nimbus lack txpool_status. | coverage gap |
| D16 | PARTIAL | Erigon added geth-shaped support; Besu/reth/others remain incompatible or absent. | debug API shape |
| D17 | PARTIAL | Besu added its own variant; geth/Erigon/Besu/reth still have incompatible shapes. | debug API shape |
| D18 | OPEN | Besu accepts 150 and −5; Nimbus also accepts −5. | client validation |
| D19 | OPEN | Erigon returns oldestBlock=0x0, gasUsedRatio=null for huge blockCount. | edge handling |
| D20 | FIXED | Ethrex simulated GASLIMIT now equals the pinned header exactly. | original client fixed |
| D21 | OPEN | Besu/Ethrex still lack raw-tx RPCs; Nimbus also lacks them. | coverage gap |
| D22 | OPEN | Erigon proof fails at every tested non-head depth. | client defect |
| D23 | OPEN | Erigon direct logs=474, filter logs=0 for identical range. | client defect |
| D24 | OPEN | Ethrex requires optional topics. | client defect |
| D25 | OPEN | Ethrex still lacks block filter and Parity trace methods; Nimbus also lacks them. | coverage gap |
| D26 | AMBIGUOUS | First historical changes poll: reth=474, four clients=0. | needs API expectation |
| D27 | OPEN | Ethrex lacks uncle-count RPC. | coverage gap |
| D28 | NOTE | Deprecated eth_coinbase/eth_accounts remain client-specific. | not a defect |
7 · What no longer needs ACDT time
- Nethermind trace defaulting:
debug_traceCallno longer requires an explicitgasPrice. - Nethermind runtime OOG: the original D4 now surfaces OOG correctly.
- Ethrex simulated GASLIMIT: returned 199,804,308 at pinned block
0x12a3e, byte-for-byte matching every header. - Erigon fee-history panic:
safeandfinalizednow return valid histories rather than crashing the handler. - geth callTracer panic: remains fixed; zero matching panic lines across devnet-8 geth hosts in the preceding 24 hours.
8 · Minimal reproductions
These commands use the ethPandaOps Panda CLI. Replace the network and instance names to run the same JSON-RPC against another endpoint.
Nimbus runtime OOG returns success
panda ethnode exec-rpc glamsterdam-devnet-8 teku-nimbusel-1 eth_call \
'[{"from":"0x2cdec15cb181def9f2c24c34dad73539015272be",
"to":"0x00000000000000000000000000000000deadbe02",
"value":"0x1","gas":"0x31f37","gasPrice":"0x77359400"},"latest"]'
# Nimbus: 0x
# Other six: OOG. gas=0x31f38 succeeds everywhere.Besu/Ethrex account-only access-list omission
panda ethnode exec-rpc glamsterdam-devnet-8 teku-besu-1 eth_createAccessList \
'[{"from":"0x2cdec15cb181def9f2c24c34dad73539015272be",
"data":"0x7300000000000000000000000000000000deadbe023100",
"gas":"0x100000","gasPrice":"0x77359400"},"latest"]'
# Besu/Ethrex: accessList=[]
# geth/reth/Erigon/Nimbus: [{address: 0x...deadbe02, storageKeys: []}]Besu fee-history bounds
panda ethnode exec-rpc glamsterdam-devnet-8 teku-besu-1 eth_feeHistory \
'["0x4","latest",[150]]'
panda ethnode exec-rpc glamsterdam-devnet-8 teku-besu-1 eth_feeHistory \
'["0x4","latest",[-5]]'
# Both succeed on Besu.Erigon non-head proof
head=$(panda ethnode exec-rpc glamsterdam-devnet-8 teku-geth-1 eth_blockNumber)
prior=$(printf '0x%x' $((head-1)))
panda ethnode exec-rpc glamsterdam-devnet-8 teku-erigon-1 eth_getProof \
"[\"0x0000BFF46984E3725691FA540A8C7589300D8282\",[],\"$prior\"]"
# -32000 root hash mismatch in account proof trieErigon filter self-inconsistency
filter='{"fromBlock":"0x12a3d","toBlock":"0x12a3e","topics":[]}'
panda ethnode exec-rpc glamsterdam-devnet-8 teku-erigon-1 eth_getLogs "[$filter]"
id=$(panda ethnode exec-rpc glamsterdam-devnet-8 teku-erigon-1 eth_newFilter "[$filter]")
panda ethnode exec-rpc glamsterdam-devnet-8 teku-erigon-1 eth_getFilterLogs "[\"$id\"]"
# Same node and range: 474 direct, 0 through the filter.Ethrex optional topics field
panda ethnode exec-rpc glamsterdam-devnet-8 teku-ethrex-1 eth_getLogs \
'[{"fromBlock":"0x12a3d","toBlock":"0x12a3e"}]'
# -32000 Expected parameter: topics is missing9 · Method, scope and versions
Each differential sent the identical request to the seven named nodes, pinned a block where head races could affect output, reduced responses to the field under test, and re-ran anything anomalous against a spec-derived boundary or an internal consistency check. Stateful filters were uninstalled after use. No transaction was broadcast.
| Client | Live web3_clientVersion |
|---|---|
| geth | Geth/v1.17.6-unstable-aa1f2fcf-20260813/linux-amd64/go1.26.5 |
| Besu | besu/v26.8-develop-6081aee/linux-x86_64/openjdk-java-25 |
| Nethermind | Nethermind/v1.40.0-unstable+3e3bd4bf-hpA/linux-x64/dotnet10.0.11 |
| reth | reth/v2.5.0-3d270d9/x86_64-unknown-linux-gnu |
| Erigon | erigon/3.7.0/linux-amd64/go1.26.7 |
| Ethrex | ethrex/v22.0.0-glamsterdam-devnet-8-86dd3154e5bf415d82d15149bba09adcdbf29481 |
| Nimbus-EL | Nimbus/v0.4.0-ed9be981/linux-amd64/Nim-2.2.10 |
Sources and prior art: original devnet-7 differential report · execution-apis fee-history schema · eth_simulateV1 notes · EIP-7708 · Besu access-list issue.
Scope guard: this report does not re-assert the original report's cryptographic root checks, opcode sweep or total-gas agreement; it re-tests the RPC findings. “Open” means the same observable difference still exists on devnet-8. It does not mean the method is standardized, required, or consensus-critical.
Generated from live glamsterdam-devnet-8 probes on 2026-08-24. Public Panda upload. The upload is intended for ACDT discussion and client-team triage; exact commands above are the portable evidence.