tl; dr
today i go over the ethereum merge, taking place at block 15,537,393, with the main purpose of transiting the blockchain from proof-of-work to proof-of-stake.
the event was perhaps one of the most important checkpoints in the story of blockchain technology, but most of all, a nerd ecstatic moment 😎.
🎶 today’s mood
PART I: THE MOTIVATION
🧱⛓ Consensus mechanisms
Consensus mechanisms define how a peer-to-peer network decides on its current status and on which blocks are to be added to the chain (and their order).
Block production is a general term that describes the act of scanning the mempool to pick up pending transitions (txs), ordering these transactions into blocks, and attaching this new block to the chain.
🔮 The mempool (a contraction of memory and pool) is the node’s mechanism for storing information on unconfirmed txs. It acts as a sort of waiting room for txs that have not yet been included in a block.
In a PoW model, the network is secured by miners and their expensive mining hardware and electricity bills. The work is done in exchange for a portion of transaction fees for block issuance.
In the PoS model, the network is secured by validators who have staked 32 ETH in order to validate the network (think as a collateral against bad behavior). Moving to PoS makes Ethereum much more energy efficient: the power consumption is projected to be around 99.95% (making Ethereum ESG compliant).
After the Merge, a validator is randomly selected in every slot (spaced 12 seconds apart) to be the block proposer. The validator bundles transactions together, execute them, and determine a new 'state' in the chain (passing it around to other validators). In this context, a finalized block is one that has been accepted as canonical by more than 2/3 of the validators.
🦇🔊 ETH issuance
Another motivation for the Merge is the reduction of ETH issuance.
The issuance of ETH is the process of creating ETH, and the burning of ETH (from transaction fees) is the process of removing existing ETH from circulation (destroying).
The rate of issuance and burning gets calculated on several parameters, and the balance between them determines ETH's resulting inflation or deflation rate.
EIP 1559 introduced the BASEFEE burning mechanism last August (shipped by the London hardfork), so the bulk of Ethereum transaction fees already has been burned for a few months. The remaining fees that are not burned (called "tips" or "priority fees") are now to be paid to the block proposer.
Ethereum's issuance will be deflationary during periods of high user activity. According to ultrasound.money, at the time of this post, 2.3k ETH has been burned to the date, and the total ETH supply is 120M, with 14M ETH skated.
🔮 The “Triple Halvening” is a term borrowed from Bitcoin to denominate the drop in ETH issuance after The Merge. While Bitcoin halves its issuance rate every 4 years, Ethereum will see its issuance rate reduced by roughly 90% at the Merge.
PART II: THE COMPONENTS
Since the Ethereum blockchain holds hundreds of billions of dollars of economic activity, instead of transitioning the chain to PoS in one large change, the Ethereum developers decided to first create a separate PoS blockchain, the Beacon Chain, in December 2020.
The Merge was the process of moving this chain, which has been running in parallel, to the EVM state of the Ethereum PoW chain. In this transition, the chain’s execution and consensus layers unbundle and run separately.
After the Merge, all applications running on Ethereum now run exactly as pre-Merge, with the state and transaction history maintained.
📡✨ The Beacon Chain
The Beacon Chain was created to ensure the PoS consensus logic was sound before enabling it on Ethereum, and also to aggregate enough validators (~400k by the time of the Merge).
The Beacon Chain was a ledger of accounts that conducted and coordinated the network of Ethereum, without really processing Mainnet transactions or handling smart contract interactions. Instead, the chain was reaching a consensus on its own state by agreeing on active validators and their account balances.
🌑🚜 Shadow forks and testnets
As one can expect, this transition has been through multiple rounds of tests that included merging shadow forks and testnets.
A shadow fork is a devnet created by forking a live network with a small number of nodes, keeping the chain’s state & history (and, therefore, able to replay transactions from the main network).
In the context of the Merge, shadow forks allowed developers to test how the upgrade would happen, but without the vast majority of the nodes being aware.
In addition, the Ethereum public testnets went to the Merge process and persisted as PoS networks, helping developers to test their applications. However, shadow forks provided a more realistic testing environment, as existing testnets already had transactions happening on them.
🪖🏁 Terminal total difficulty (TTD)
The trigger for the Merge switch was determined by a new variable, TERMINAL_TOTAL_DIFFICULTY, which represents the sum of the PoW difficulties of every block accumulated on the Ethereum PoW network (the sum of the Ethash mining difficulty for all blocks). The Merge would happen when TTD reached 58750000000000000000000.
🔮 The Ethash algorithm is used to dynamically adjust the mining difficulty of the blockchain that implements it.
When the chain accumulated enough PoW to exceed the total difficulty, nodes switched to following the PoS chain. It was the first time an Ethereum upgrade was triggered by this variable, as opposed to the block height. The reasoning was to avoid an attacker mining malicious forks that would satisfy the block height requirement at the time of the Merge.
⛲️⚡️ The execution layer
Before the transition to PoS, to validate the Ethereum network, node operators run one of the client implementations:
Before the Merge, these clients bundled the execution layer (EVM) and the consensus layer (PoW).
After the Merge, they transited to being strictly execution layer clients, working with the consensus layer clients to form a full Ethereum node.
Communication between the execution and consensus layers is now facilitated by the Engine API. This requires authentication using a JWT secret, which is provided to both clients.
🗳⚡️ The consensus layer
The Beacon chain was launched by decoupling the consensus layer from the execution client. In PoS, the validity of transactions contained within an execution payload depends on the validity of the "consensus block" it is contained within.
🔮
LMD-GHOSTis an acronym for "Latest Message Driven Greediest Heaviest Observed SubTree" and the fork-choice algorithm used by Ethereum's consensus clients to identify the head of the chain (the head of the chain is the block with the greatest accumulation of attestations in its history).
Some popular consensus clients are:
With multiple client implementations on both layers, there are several different combinations of execution and consensus clients. Here is a dashboard on the client statistics in the Ethereum blockchain.
🔮 Having multiple client implementations allows for quickly switching to different clients in case a bug or vulnerability is found, ensuring a better resilience of the network. Here is a dashboard conveying data on Ethereum client diversity.
PART III: THE MERGE
🪢⚙️ Node operators
The changes for non-validating Ethereum node operators were the requirement to run clients for both the execution layer and the consensus layer.
Stakers running their own node setup or infrastructure had to make sure:
-
both a consensus client and an execution client are running
-
both clients are authenticated with a shared JWT secret
-
a fee recipient address was set (to receive earned transaction fee tips/MEV)
🔮 If you are interested in running your own Ethereum mainnet or testnet node, check my comprehensive notes here.
🐼💾 The moment of the Merge
At the moment of the Merge, execution clients started listening to blocks coming from the PoS chain.
Once PoS took over from PoW, execution engines stopped gossiping blocks. This meant deprecating the NewBlockHashes and NewBlock handlers at the peer-to-peer layer.
The execution layer was still in charge of syncing the network state, gossiping transactions, handling transaction fees, and maintaining its transaction pool.
😎🧮 The new kids in the block
Since the Merge came with changes to consensus, this included changes related to block structure, slot/block timing, opcode changes, sources of on-chain randomness, and the concept of safe head and finalized blocks.
The ExecutionPayloads is the post-merge equivalent of blocks in the PoW chain, and the execution engine ensures that all transaction senders can pay at least the current baseFeePerGas and that any extra fees are sent to feeReceipient (a "legacy" Ethereum address).
Anything related to mining (difficulty, mixHash, nonce) or ommers (ommers, ommersHash) had to be changed. Several block fields were set to 0 (or their data structure's equivalent). The length of extraData was also capped to 32 bytes on mainnet.
This is a high-level overview of how blocks are processed after the Merge:
-
a validator is chosen to propose a block
-
this validator asks its execution layer, via the Engine API, to send them an
ExecutionPayload -
the execution layer returns a payload that contains the most profitable set of valid transactions
-
the consensus layer proposes a block including this payload and propagates it on the chain
⏰🥓 Block time and variance
While the block time on PoW averages around 13 seconds, on PoS the time is divided into fixed-duration slots of 12 seconds (however, some slots can miss a block making the average block time higher than 12 seconds but still lower than 13 seconds).
The merge also reduced the block time variance, and on average, we can expect faster transaction confirmations.
tl; dr: what’s next
-
The Shanghai update, predicted for 2023, will introduce EIP-3540 (also known as the EVM object format) and enable withdrawals.
-
The Surge is a massive scalability increase for rollups through sharding, planned for 2023. The main innovation of Danksharding is a merged fee market. Instead of a fixed number of shards with different blocks and proposers, one randomly-chosen proposer picks all transactions and data for each slot.
-
The Verge will be implementing Verkle trees (which are much more efficient in proof size) and stateless clients.
-
The Purge will be eliminating historical data (hence, reducing the amount of data needed to be stored by a validator).
-
The Splurge will be looking at consensus & MEV. Ethereum will implement Proposal-Builder Separation for the consensus layer, achieving centralized block production and decentralized block verification.
