chore!: enable multiple L1 nodes to be used #11945

spypsy · 2025-02-12T14:24:13Z

Updates ETHEREUM_HOST env var to ETHEREUM_HOSTS. Using a single host should still work as it did before
Single ViemWalletClient & ViemPublicClient

BREAKING CHANGE:

env var ETHEREUM_HOST -> ETHEREUM_HOSTS
CLI arg --l1-rpc-url -> --l1-rpc-urls
TypeScript configs with l1RpcUrl -> l1RpcUrls
aztec.js functions with l1RpcUrl -> l1RpcUrls
DeployL1Contracts (type) -> DeployL1ContractsReturnType

Follow-up #12254

github-actions · 2025-02-13T11:09:46Z

Changes to public function bytecode sizes

Generated at commit: ad901e035ea4109b8e123a34da5b59ed9cae999d, compared to commit: 921d2cd9de54a9fd7cc49e4951c9fa65ab1526ef

🧾 Summary (100% most significant diffs)

Program	Bytecode size in bytes (+/-)	%
TokenBlacklist::public_dispatch	+42 ❌	+0.18%

Full diff report 👇

Program	Bytecode size in bytes (+/-)	%
TokenBlacklist::public_dispatch	23,831 (+42)	+0.18%

spalladino

Hell of a PR. Found a few instances that need attention, but aside from that, looks good.

spalladino · 2025-02-24T13:32:18Z

docker-compose.yml

@@ -64,7 +64,7 @@ services:
      - p2p-boot-node
    entrypoint: |
      /bin/sh -c '
-        export ETHEREUM_HOST=$$(cat /var/run/secrets/ethereum-host)
+        export ETHEREUM_HOSTS=$$(cat /var/run/secrets/ethereum-host)


Do we need to pluralize the secret as well? (I'm fine if we don't, just checking we didn't chan ge the secret but forgot to change here)

only for consistency's sake, will do

spalladino · 2025-02-24T13:34:31Z

spartan/aztec-network/templates/_helpers.tpl

 until curl -s -X POST -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":67}' \
-  ${ETHEREUM_HOST} | grep 0x; do
-  echo "Waiting for Ethereum node ${ETHEREUM_HOST}..."
+  ${ETHEREUM_HOSTS} | grep 0x; do
+  echo "Waiting for Ethereum node ${ETHEREUM_HOSTS}..."
  sleep 5


I think this will break if there's more than one value in ETHEREUM_HOSTS, we need to loop over all values and wait until all (or one?) is available.

I think one? point of this is redundancy so we should consider just one as 'ready'

Makes sense!

spalladino · 2025-02-24T13:36:38Z

yarn-project/archiver/src/archiver/archiver.ts

    const publicClient = createPublicClient({
      chain: chain.chainInfo,
-      transport: http(chain.rpcUrl),
+      transport: fallback(config.l1RpcUrls.map(url => http(url))),


spalladino · 2025-02-24T13:37:41Z

yarn-project/aztec-faucet/src/faucet.ts

    this.walletClient = createWalletClient({
      account: this.account,
      chain: chain.chainInfo,
-      transport: ViemHttp(chain.rpcUrl),
+      transport: fallback([ViemHttp(chain.rpcUrls[0])]),
    });

    this.publicClient = createPublicClient({
      chain: chain.chainInfo,
-      transport: ViemHttp(chain.rpcUrl),
+      transport: fallback([ViemHttp(chain.rpcUrls[0])]),
    });


Why rpcUrls[0] instead of mapping over all urls, as done in the archiver?

good question, I don't think I meant to do that. will update

spalladino · 2025-02-24T13:39:22Z

yarn-project/aztec.js/src/api/ethereum/portal_manager.ts

+    private publicClient: ViemPublicClient,
+    private walletClient: ViemWalletClient,


Not for this PR, but we should stop using the two clients simultaneously, and just use the wallet.extend(publicActions) to have one client with the capabilities of both.

spalladino · 2025-02-24T13:49:49Z

yarn-project/end-to-end/scripts/native-network/utils/get-chain-id.sh

 export L1_CHAIN_ID=$(curl -s -X POST -H "Content-Type: application/json" \
  --data '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' \
-  $ETHEREUM_HOST | grep -o '"result":"0x[^"]*"' | cut -d'"' -f4 | xargs printf "%d\n")
+  $ETHEREUM_HOSTS | grep -o '"result":"0x[^"]*"' | cut -d'"' -f4 | xargs printf "%d\n")


spalladino · 2025-02-24T13:50:00Z

yarn-project/end-to-end/scripts/native-network/validator.sh

 # Automatically detect if we're using Anvil
-if curl -s -H "Content-Type: application/json" -X POST --data '{"method":"web3_clientVersion","params":[],"id":49,"jsonrpc":"2.0"}' $ETHEREUM_HOST | jq .result | grep -q anvil; then
+if curl -s -H "Content-Type: application/json" -X POST --data '{"method":"web3_clientVersion","params":[],"id":49,"jsonrpc":"2.0"}' $ETHEREUM_HOSTS | jq .result | grep -q anvil; then


seems we don't actually use IS_ANVIL anymore, will remove

spalladino · 2025-02-24T13:51:00Z

yarn-project/end-to-end/src/devnet/e2e_smoke.test.ts

@@ -162,7 +162,7 @@ describe('End-to-end tests for devnet', () => {
      claimSecret: { value: string };
      messageLeafIndex: string;
    }>('bridge-fee-juice', [amount, l2Account.getAddress()], {
-      'l1-rpc-url': ETHEREUM_HOST!,
+      'l1-rpc-url': ETHEREUM_HOSTS!,


Suggested change

'l1-rpc-url': ETHEREUM_HOSTS!,

'l1-rpc-urls': ETHEREUM_HOSTS!,

I think?

good catch indeed!

spalladino · 2025-02-24T13:51:08Z

yarn-project/end-to-end/src/devnet/e2e_smoke.test.ts

@@ -279,7 +279,7 @@ describe('End-to-end tests for devnet', () => {

  async function getL1Balance(address: string, token?: EthAddress): Promise<bigint> {
    const { balance } = await cli<{ balance: string }>('get-l1-balance', [address], {
-      'l1-rpc-url': ETHEREUM_HOST!,
+      'l1-rpc-url': ETHEREUM_HOSTS!,


Same as above

spalladino · 2025-02-24T13:56:23Z

yarn-project/ethereum/src/test/tx_delayer.test.ts

+export type ViemClient = Client<
+  HttpTransport,
+  Chain,
+  PrivateKeyAccount,
+  [...PublicRpcSchema, ...WalletRpcSchema],
+  PublicActions<HttpTransport, Chain> & WalletActions<Chain, PrivateKeyAccount>
+>;


Curious why we had to re-define and re-export this type from here

ah this was initially the only place I ran into it so only had it here. then realized it's used in more places and added in types, will update 👍

spalladino · 2025-02-24T14:02:31Z

There are also a few mentions to ETHEREUM_HOST in the docs folder that will need update

spalladino · 2025-02-24T14:03:26Z

And it'd be nice if we could tweak the cheat codes to also handle multiple rpc urls, it's the only component left that still forces one rpc url.

spalladino · 2025-02-25T19:27:28Z

spartan/aztec-network/templates/_helpers.tpl

+      -d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":67}' \
+      ${HOST} | grep -q 0x; then
+      echo "Ethereum node ${HOST} is ready!"
+      exit 0


What's the scope of this exit 0? IIUC this template gets pasted as-is where it is included, so any operations after this helper would not run due to the early exit. This would affect eg deploy-l1-contracts:

- name: deploy-l1-contracts {{- include "aztec-network.image" . | nindent 10 }} command: - /bin/bash - -c - | cp /scripts/deploy-l1-contracts.sh /tmp/deploy-l1-contracts.sh chmod +x /tmp/deploy-l1-contracts.sh source /shared/config/service-addresses source /shared/config/validator-addresses {{- include "aztec-network.waitForEthereum" . | nindent 14 }} /tmp/deploy-l1-contracts.sh "{{ .Values.aztec.l1Salt }}" "{{ .Values.ethereum.chainId }}" "$VALIDATOR_ADDRESSES"

Sorry, reviewed this in an outdated commit and it was already fixed.

Still, I think the new version doesn't do what we expect. I understand it loops over every host, waiting for 5s inbetween them, and exits once the first one replies. But it doesn't go back to the first one to retry if all failed, it just goes through, right?

spalladino · 2025-02-25T19:31:11Z

yarn-project/end-to-end/scripts/native-network/deploy-l1-contracts.sh

+found_node=0
+for HOST in $(echo "${ETHEREUM_HOSTS}" | tr ',' '\n'); do
+  if curl -s -X POST -H 'Content-Type: application/json' \
+    --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
+    "$HOST" 2>/dev/null | grep -q 'result'; then
+    echo "Node $HOST is ready"
+    found_node=1
+    break
+  fi
  sleep 1
 done


Heads up this is not equivalent to the previous one: the new one will exit if the node is not immediately available, while the previous one will retry. Given this is called simultaneously to ETHEREUM_SCRIPT in scripts/run_native_testnet.sh, we should keep the behaviour of waiting for the node to be up.

yarn-project/end-to-end/scripts/native-network/deploy-l1-contracts.sh

...t/prover-client/src/proving_broker/proving_broker_database/broker_persisted_database.test.ts

yarn-project/ethereum/package.json

yarn-project/ethereum/src/eth_cheat_codes.ts

spalladino · 2025-02-26T15:21:02Z

yarn-project/end-to-end/scripts/native-network/deploy-l1-contracts.sh

+      "$HOST" 2>/dev/null | grep -q 'result'; then
+      echo "Node $HOST is ready"
+      found_node=1
+      break 2


spartan/aztec-network/templates/_helpers.tpl

* master: (31 commits) feat: Slack message to ci channel tagging owners on flakes. (#12284) fix: slack notify was broken by quoted commit titles revert: "chore: Fix and reenable fees-settings test (#12302)" fix: run arm64 on master (#12307) yolo fix chore: Fix and reenable fees-settings test (#12302) feat!: rename compute_nullifier_without_context (#12308) chore: Lazy loading artifacts everywhere (#12285) chore: Reenable dapp subscription test (#12304) chore: Run prover test with fake proofs when requested (#12305) chore: Do not set CI_FULL outside CI (#12300) chore: new mnemonic deployments on sepolia (#12076) chore!: enable multiple L1 nodes to be used (#11945) chore: remove no longer supported extension from vscode/extension.json (#12303) fix(e2e): p2p_reqresp (#12297) feat: Sync from noir (#12298) chore: enabling `e2e_contract_updates` in CI + nuking irrelevant test (#12293) feat: prepend based merge (#12093) feat: fetch addresses from registry (#12000) feat: live logs (#12271) ...

Follow-up fixes from merging #11945 & #12076

spypsy added 4 commits February 12, 2025 11:40

fix: support multiple L1 node URLs

1140b31

leftovers

b2faca5

more fixes

16146ca

Merge branch 'master' into spy/multi-rpc-urls

4f79daf

spypsy marked this pull request as ready for review February 12, 2025 14:26

spypsy added 8 commits February 12, 2025 14:43

fix --rpc-url usage

286954a

Merge branch 'master' into spy/multi-rpc-urls

f0e23a1

fix imports + tests

0fad63b

undo test change

09ea2b1

update --l1-rpc-urls

d6334bb

fix imports

dd1e0ef

another import

6d45015

externalHosts

4fb917e

spypsy added 2 commits February 20, 2025 12:57

Merge branch 'master' into spy/multi-rpc-urls

d129c99

post-merge fixes

4836dd9

spypsy requested a review from charlielye as a code owner February 20, 2025 14:05

spypsy changed the title ~~fix: enable multiple L1 nodes to be used~~ chore!: enable multiple L1 nodes to be used Feb 20, 2025

spypsy and others added 5 commits February 20, 2025 14:29

more post-merge fixes

bbacaaf

fix e2e test

4a91b45

Merge branch 'master' into spy/multi-rpc-urls

8714406

Merge branch 'master' into spy/multi-rpc-urls

29a1311

Merge branch 'master' into spy/multi-rpc-urls

24ccc68

spalladino requested changes Feb 24, 2025

View reviewed changes

spypsy added 4 commits February 24, 2025 17:18

PR Fixes

da0e3cb

update docs

842ae5d

update docs

685d0c5

Merge branch 'master' into spy/multi-rpc-urls

12cc436

spypsy added 5 commits February 25, 2025 10:01

Merge branch 'master' into spy/multi-rpc-urls

aa5fa6c

fix wallet client types

8e79c39

Merge branch 'master' into spy/multi-rpc-urls

e438592

use multi-urls in cheatcode

eca0f24

Merge branch 'master' into spy/multi-rpc-urls

889caad

spypsy requested a review from spalladino February 25, 2025 16:27

spypsy added 4 commits February 25, 2025 17:25

fix helper spartan script

fe3d7d3

fix e2e cheat_codes

c489937

Merge branch 'master' into spy/multi-rpc-urls

85c007f

rm unused import

1547a62

spalladino requested changes Feb 25, 2025

View reviewed changes

spypsy added 6 commits February 26, 2025 09:44

PR Fixes

67c81a9

Merge branch 'master' into spy/multi-rpc-urls

d58918e

deploy l1 contracts fixes

fc382de

fix type imports

bb6975c

update gh secret names

9405407

break, don't return

36c9e22

spypsy requested a review from spalladino February 26, 2025 12:14

no need to retry get-chain-id

b111170

spalladino approved these changes Feb 26, 2025

View reviewed changes

spypsy added 2 commits February 26, 2025 16:36

Merge branch 'master' into spy/multi-rpc-urls

da839b8

Merge branch 'master' into spy/multi-rpc-urls

4be11a4

spypsy enabled auto-merge (squash) February 26, 2025 17:49

merge fix

3bd421e

spypsy merged commit fcf6278 into master Feb 26, 2025
9 checks passed

spypsy deleted the spy/multi-rpc-urls branch February 26, 2025 19:23

spypsy mentioned this pull request Feb 27, 2025

fix: sepolia deployments #12329

Merged

just-mitch pushed a commit that referenced this pull request Mar 3, 2025

fix: sepolia deployments (#12329)

f72ccc2

Follow-up fixes from merging #11945 & #12076

AztecBot mentioned this pull request Mar 6, 2025

feat: Sync from aztec-packages noir-lang/noir#7606

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore!: enable multiple L1 nodes to be used #11945

chore!: enable multiple L1 nodes to be used #11945

spypsy commented Feb 12, 2025 •

edited

Loading

github-actions bot commented Feb 13, 2025

spalladino left a comment

spalladino Feb 24, 2025

spypsy Feb 24, 2025

spalladino Feb 24, 2025

spypsy Feb 24, 2025

spalladino Feb 24, 2025

spalladino Feb 24, 2025

spalladino Feb 24, 2025

spypsy Feb 24, 2025

spalladino Feb 24, 2025

spalladino Feb 24, 2025

spalladino Feb 24, 2025

spypsy Feb 24, 2025

spalladino Feb 24, 2025

spypsy Feb 24, 2025

spalladino Feb 24, 2025

spalladino Feb 24, 2025

spypsy Feb 24, 2025

spalladino commented Feb 24, 2025

spalladino commented Feb 24, 2025

spalladino Feb 25, 2025

spalladino Feb 25, 2025

spalladino Feb 25, 2025

spalladino Feb 25, 2025

spalladino Feb 26, 2025

		private publicClient: ViemPublicClient,
		private walletClient: ViemWalletClient,

	'l1-rpc-url': ETHEREUM_HOSTS!,
	'l1-rpc-urls': ETHEREUM_HOSTS!,

chore!: enable multiple L1 nodes to be used #11945

chore!: enable multiple L1 nodes to be used #11945

Conversation

spypsy commented Feb 12, 2025 • edited Loading

github-actions bot commented Feb 13, 2025

Changes to public function bytecode sizes

🧾 Summary (100% most significant diffs)

spalladino left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

spalladino commented Feb 24, 2025

spalladino commented Feb 24, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

spypsy commented Feb 12, 2025 •

edited

Loading