Initiative
Recently I was assigned a task to test out Zhejiang testnet, which is kind of a big deal for ETH ecosystem. Most docs provide simple setup methods like Docker or binary installation. I wanted to spice things up and try the "Cloud-Native" way -- Kubernetes, which I've never used as node infra before. This article is mainly about my setup and some thoughts after trying out the Zhejiang testnet.
K8s Provider and Node Spec
Since our organization uses GCP, using Google Kubernetes Engine is pretty much a no brainer. Question is what kind of node spec and storage do I use? Ethereum nodes tend to be I/O bound, so a dog slow hard drive definitely won't do it. Thankfully, Google provides a thorough information regarding VM spec and SSD speed.
Choice of EL, CL client
My GOTO combo of EL+CL client have always been Geth+Lighthouse. However, many have reported that there is a bug with this combo for the testnet. I've tried this combo personally too, Geth indeed always stop syncing at a specific block. Therefore, I switched my execution client to Besu. Besu works pretty well, I have nothing to complain about it. 🎉
Maybe I should consider give Besu a shot on the mainnet later.
Installation Guide
This part is kind of technical, readers that need this may have a look at my github repo: https://github.com/108356037/zhejiang-testnet-k8s. Lighthouse image has been modified to shipped with some custom data regarding the testnet, I've provided the Dockerfiles too.
Feel free to open up a PR if you find anything that could be improved.
Playing with ethdo
After the clients are all synced, it's time to play with the most important feature of Zhejiang testnet: Withdrawals. However, for a validator to be withdrawed from the beacon chain, its withdrawal credentials must start with 0x01 prefix. This article has a good amount of details about beacon chain keys. Now let's try to change our validator's withdrawal credentials. For the below section, you need a consensus client, ethdo-cli, a running validator on Zhejiang.
I. First, let's port-forward the Lighthouse pod's API port to our local machine:
# be sure modify the pod name according to yours
kubectl port-forward lighthouse-depl-69fb58b966-6hmpm 5052:5052
II. Check if ethdo works (ethdo default connects to 127.0.0.1:5052):
ethdo node info --verbose
## should see below output
#Version: Lighthouse/v3.4.0-e062a7c/x86_64-linux
#Syncing: false
III. Now we can use ethdo to modify withdrawal credentials prefix. For this example, I will use validator 61797.
Notice that it has a 0x00 prefix, the UI also tells us a 0x01 withdrawal credentials is required to received payouts. Let's change it with ethdo (mnemonic will be the 24 english words you got when creating the validator key):
ethdo validator credentials set --mnemonic="abandon abandon abandon … art" --validator=61797 --withdrawal-address=0x787818A78B7e0C8B684142e8F3c12A6b06061856
Let's check if it's successfully changed:
Moreover, if we check the page on beacon chain explorer, we have the credentials modified:
Notice that on the withdrawal tab, we can see a pending withdraw awaiting:
New payload in Execution client
According to the specs we have a new withdrawal payload in the EL client, let's explore this too.
I. Port-forward the Besu pod:
# be sure modify the pod name according to yours
kubectl port-forward besu-depl-7756c8cfcb-7l5vj 8545:8545
II. Do a simple curl to get block data:
curl -s -X POST -H "Content-Type: application/json" \
--data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["latest", false],"id":1}' \
http://127.0.0.1:8545
The json response as below:
{
"jsonrpc" : "2.0",
"id" : 1,
"result" : {
"number" : "0xf83c",
"hash" : "0xbfd9282d15905f32e485911c90d24abe8ef222bd7f1afbd8119b98e364bf1b4f",
"mixHash" : "0x71bae828c337e589a8d21d7a519174ef7f36522dda458557babad44330f7563b",
"parentHash" : "0xbeb9729064aaeb989447ae5e3c2f760cce05da451d753ab73a4555fb3007afd5",
"nonce" : "0x0000000000000000",
"sha3Uncles" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
"logsBloom" : "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"transactionsRoot" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
"stateRoot" : "0x0c3469f8e4dbfb654f6070180d4561a6a9666f2640948631ad43a1567e24d0fd",
"receiptsRoot" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
"miner" : "0xd9a5179f091d85051d3c982785efd1455cec8699",
"difficulty" : "0x0",
"totalDifficulty" : "0x1",
"extraData" : "0xd883010b00846765746888676f312e31392e35856c696e7578",
"baseFeePerGas" : "0x7",
"size" : "0x44d",
"gasLimit" : "0x1c9c380",
"gasUsed" : "0x0",
"timestamp" : "0x63e66d90",
"uncles" : [ ],
"transactions" : [ ],
"withdrawalsRoot" : "0x2e0ac7755fb91654e4f0df1e6d6f6b537e98766991160996bd30773de8d2502a",
"withdrawals" : [ {
"index" : "0x51604",
"validatorIndex" : "0xc1b0",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2694c"
}, {
"index" : "0x51605",
"validatorIndex" : "0xc1b1",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x287dd"
}, {
"index" : "0x51606",
"validatorIndex" : "0xc1b2",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x28900"
}, {
"index" : "0x51607",
"validatorIndex" : "0xc1b3",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x51608",
"validatorIndex" : "0xc1b4",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2694c"
}, {
"index" : "0x51609",
"validatorIndex" : "0xc1b5",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x5160a",
"validatorIndex" : "0xc1b6",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x5160b",
"validatorIndex" : "0xc1b7",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x5160c",
"validatorIndex" : "0xc1b8",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x5160d",
"validatorIndex" : "0xc1b9",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x5160e",
"validatorIndex" : "0xc1ba",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2694c"
}, {
"index" : "0x5160f",
"validatorIndex" : "0xc1bb",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x51610",
"validatorIndex" : "0xc1bc",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x28597"
}, {
"index" : "0x51611",
"validatorIndex" : "0xc1bd",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x51612",
"validatorIndex" : "0xc1be",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x2a791"
}, {
"index" : "0x51613",
"validatorIndex" : "0xc1bf",
"address" : "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
"amount" : "0x28597"
} ]
}
}
Notice that we got two new field: withdrawals & withdrawalsRoot. These fields are not presented at current mainnet EL clients.
Last thoughts
Overall, the experience is pretty smooth on Zhejiang testnet. However, this is just a testnet, it does not equal the real Shanghai upgrade. What I'm kind of concerned is the process of changing withdrawal credentials. The process requires talking with the CL client, I'm not sure whether the mnemonic is encrypted during the communication. The withdrawal address is also a one-time only change, if a hacker can trick stakers to set the withdrawal address at their favor, this will be a nightmare. Although setting the withdrawal address is only through a command line, in the future we definitely need more safety checks beforehand.