This article was initially written as a run-up for the upcoming Dencun upgrade, but I didn’t have time to finish it before the upgrade. I decided that this article contains a lot of helpful information about how blobs work and what exactly they change in the rollup workflow, so I modified it a little and published post factum.

To understand the material below, it’s helpful to have a basic understanding of Merkle trees, Ethereum, and its rollup-centric scaling roadmap. My article “Dr. Dankshard or How I learned to stop worrying and love rollups” is a great introduction to the latter.

Special thanks to Proto for help with OP Stack

---

Data storage problem

If you try to summarise rollups in two sentences, you’ll get something like this:

Rollup is a separate blockchain network that verifies its own state using a smart contract on its parent, “layer one” chain rather than its own independent consensus.

In order to achieve this, rollup sequencers give the smart contract on L1 all necessary data to verify the state - for example, transaction data and (in case of ZK rollups) Zero-Knowledge cryptographic validity proof - and the smart contract uses it to generate blockchain-verified verdict about the canonical network state.

However, this definition lacks important details, because it’s worth noting how exactly optimistic and ZK rollups utilize L1 data.

• Optimistic rollups need to store transaction data on L1 so that people who want to challenge the state transition can always access everything necessary for challenging. The smart contract only needs a contested batch chunk to verify users’ fraud proofs.

• ZK rollups need to store transaction data on L1 so that people who want to generate the validity proof for the batch can always access the data for proving. The smart contract can verify the proof only by using a short cryptographic commitment to the data.

In fact, in both architectures, the “verifier” smart contract does not need all the data; it’s the off-chain “proving” entities that do. Data storage on L1 is preferred because it has the same trust assumptions as the output of any smart contract on it—in other words, the “verifier” smart contract verdict is as secure as all the data stored.

Interesting fact: L2 networks that store their data outside their L1 are called Validiums (ZK L2s) and Optimiums (optimistic L2s). Some people, including the L2BEAT team, do not consider them L2s because by using an external Data Availability system, they introduce additional trust assumptions into their system. Rollup is a universal term for L2s that store their data on their L1.

Another important thing you might already assume is that data is only needed until its batch is considered valid by the smart contract on L1. After that, all nodes will follow the Merkle root from the contract as the canonical one.

That is, we don’t need to store the rollup data forever as we do using calldata, so we can make some sort of temporary data storage, also secured by L1 but pruned after some time.

What’s a blob?

EIP-4844 introduces special consensus layer entities, called “blobs”.

At a high level, the blob is a 128kb binary object kept by the consensus layer. Its commitment’s hash is stored in the EVM, so anyone can reveal the commitment itself and use it to execute necessary computations. Blobs are pruned after 4096 epochs ~ 18 days, which is enough for both optimistic and ZK rollups to verify their batches.

How do rollups verify their state if the blob isn’t available in EVM? What’s the commitment? How come only its hash is stored in EVM? How does it all work? To answer all these questions, let’s explore the logic behind Protodanksharding.

KZG polynomial commitment scheme

Note: I won’t go into all cryptographic details to keep the article relatively simple. Rather, I will take a software-centric approach to explain solely where and how KZG is used within post-4844 Ethereum and rollups.

Technically, the blob is an array (list) of 4096 numbers from 0 to a very big number roughly equal to 2 to the power of 254.857. These numbers, called “field elements,” are packed in 32 bytes each, so the entire blob takes 128kb of disk space.

KZG commitment scheme allows conversion of an array of field elements (in our case, the blob) into a 48-byte short commitment which can be used to 1) authenticate blobs and 2) generate short proofs that any field element exists in the original array, which can be verified using only the commitment and thus without revealing the whole array.

The first feature is very similar to the logic of hash functions, but the second one is the most important for us (and is the reason why we didn’t just take any hash function!):

Remember how I said that only commitment hash is available in the EVM? Thanks to EIP-4844’s point evaluation precompile, we can verify KZG proofs on-chain, only using the commitment, supposed field element, and the proof itself. In the EIP-4844 implementation of KZG, all this data takes 32+48+32+32+48=192 bytes.

What is this needed for?

Even though all types of rollups can use this feature, it’s the most useful for optimistic rollups: fraud proofs only need one batch chunk which was executed incorrectly, and they can utilize KZG proofs to contest only necessary parts of the blob, leaving the rest outside the EVM (and thus allowing it to be pruned after challenge time passes).

ZK rollups can enshrine KZG proof verification inside the validity proof of their batch, but since the validity proof verifies the whole content of the blob, the more efficient solution should be enshrining blob proof verification instead, leaving the blob itself in the private input.

What’s a blob proof? It’s a cryptographic proof that some commitment corresponds to (that is, was derived from) a certain known blob. It’s not very useful with small 128kb blobs because we can simply perform an interpolation and compare the resulting commitment with the supposed one, but if (when?) we make blobs larger, blob proofs will win performance by a lot.

As you can see, the characteristics of the KZG commitment scheme allow for efficient blob utilization for both ZK and optimistic rollups.

Blob production

EIP-4844 introduces a new type of blob-carrying transactions. It’s almost the same as a normal (EIP-1559) transaction but has two new fields - max_fee_per_blob_gas and blob_versioned_hashes.

• max_fee_per_blob_gas - maximum fee per 1 blob gas (not to be confused with EVM gas!). The blob gas market follows EIP-1559-like logic, so the network won’t take more per 1 blob gas than the calculated blob base fee. Blobs have a fixed blob gas usage of 131072, that is, one blob gas per byte.

• blob_versioned_hashes - versioned hashes of blobs’ commitments. Each transaction can contain multiple blobs, hence “hashes.” When a validator decides whether to take a blob-carrying transaction, it asks its consensus layer client if the blob that has the commitment with a certain versioned hash is available (downloaded). The versioned hash is the simple SHA256 hash, but the first byte is version - 0x01.

Also, to send a blob-carrying transaction, you need to insert your blobs, their commitments, and corresponding blob proofs into the transaction envelope.

The common misconception is that blob production is expensive and, therefore, must be accomplished by a validator who takes the transaction. As we can see, the latter is impossible—the transaction creator has to set versioned hashes of their blobs, thus executing all interpolation into commitments themselves. Is it actually expensive to generate blobs, though?

To get a blob that can be published on-chain and which contents can be KZG proven, two steps are required:

• Convert a byte array into an array of field elements. Since the field element is slightly (~1.2 bits) less than the 32 bytes it’s packed into, we can’t just split our byte array into 32-byte chunks. However, it’s quite simple to implement necessary compression that, say, pads every 31 bytes with a zero byte to not exceed the limit. Such an operation shouldn’t be expensive, because the blobs are just 128kb each.

• Perform a KZG polynomial interpolation to generate the commitment to the blob which versioned hash needs to be set in your blob-carrying transaction.

Unless you’re a programmer, I guess you were confused with the first part. Let’s break it down!

Do you remember that field elements are packed in 32 bytes each? That’s because their limit, that large number called BLS modulus, roughly equals 2^254.857, so it fits in 255 bits. However, we can’t store bits, only bytes, so we add one more zero bit and fit it in 256 bits = 32 bytes.

Even though we pack field elements in 32 bytes, we don’t quite have all these 32 bytes available, because the first bit has to be set to zero and all remaining bits must be less than BLS modulus. Therefore, not all sequences of 32 bytes are suitable for field elements.

But what if we need to fit sequences larger than the BLS modulus? - We have to pad our values. This will result in slightly more bytes used and we’ll need to KZG prove two field elements to get a single 32-byte sequence, but our field elements won’t exceed the modulus.

Okay, making an array of field elements seems pretty trivial. What about interpolation?

I made blobs-benchmark, a Python script that generates 100 random blobs with their commitments and shows how fast this process was done. The script itself is pretty basic because the KZG Python library is available online. Let’s run it!

My M1 laptop generates 100 EIP-4844 blobs in ~8.9 seconds or about 89 milliseconds per blob. The actual random blob generation happens instantly, so it’s all interpolation.

This script is open source, so if you had any experience running them before, you can have a try and check how long it takes on your machine. As we can see, blob production is practically free, so sequencers won’t need to handle any additional costs.

---

Going inside the rollups

Let’s see how rollups utilize blobs, what their proving mechanism is, and how they pack batches into field elements. Thankfully, all rollups are open source, so we can check all the code ourselves!

I took OP Mainnet for our pseudo-research because it’s based on OP Stack, so all OP Stack rollups with blob support (Base, Mode, Zora, etc) have the same logic.

OP Mainnet

At the time of writing, the OP sequencer creates new blobs by sending blob-carrying transactions from a special Batcher account to the vanity address.

Notice that the destination address is almost fully zero. It’s not someone’s wallet; rather, it is a sort of demonstrative burn address, hence the “10“ at the end, which is the chain ID of OP Mainnet.

Previously all these transactions contained batch data in their calldata. Now, all transactions have empty calldata, so technically they’re the same as a simple 0 ETH transfer, but also carry blobs.

---

(The info below is taken from OP Stack specs, specifically this and this page. It’s slightly technical and not really needed for understanding)

OP Stack uses channel encoding to convert transaction batches into data streams that can be fit into blobs on L1.

1. Firstly, the sequencer constructs the span batch - the range of multiple consecutive L2 blocks, with all their transactions and necessary metadata to reconstruct the network state.

2. Then it gets encoded using RLP and compressed using zlib stream compression. This was the last step before the blob upgrade, as L1 transaction calldata can accept raw bytes. However, this is not the case with blobs, so now it also goes through blob encoding.

3. The resulting byte sequence is split into chunks of 127 bytes = 1016 bits each, and each chunk is encoded in 4 field elements (254 bits per element).

4. While OP Stack technically supports sending multiple blob transactions to store the single channel, I couldn’t find such examples in the mainnet, so let’s assume the channel takes up to 6 blobs. If the last used blob isn’t full, it’s padded with field elements equal to zero.

I chose a random sequencer transaction and made a Python script that decodes its blobs back into the span batch and reads its contents.

Using the same logic as described above, I decoded the blobs into the span batch and read its contents. The batch also contains transactions themselves, but the script doesn’t print them for the brevity of the output.

If you want to run it yourself, here’s the link. I won’t describe its internals because this part will become too complicated to comprehend, but if you have some experience in Python, it shouldn’t be difficult for you to understand the script.

---

About every hour, the OP proposer (in fact, the same entity, since the rollup is centralized) pushes a new state root based on the execution of newly posted batches. OP Stack doesn’t have fault proofs yet, so the validity of the blobs’ content is not proven in any way.

OP Rollup Node is connected to the L1 (Ethereum) node and OP Execution Engine. Rollup Node constantly scans L1 blocks for new proposals and batch sequences and passes them to the Execution Engine, where they’re used to reconstruct the L2 chain.

As all OP batches are considered final after 7 days of challenge time, only finalized batches can become unavailable (because their blobs are pruned by L1 only after ~18 days). Thus, the OP node can safely follow the latest finalized output root and leave historical batches untouched. Elements of the state trie can be fetched P2P, and because the node always has the state root, only one honest node is enough to reconstruct the entire trie.

---

People are always asking me if I know Alex Hook…

Do you know what’s the first rule of the fight club?

• What’s the “fight club”?

I won’t tell you the whole thing, only the first rule. KZG commitments are going to help me.

In my recent experiment, I posted the entire Fight Club script in two blobs on mainnet and sent a transaction to the point evaluation precompile with the proof that the “you do not talk about Fight Club” rule actually exists in one of those blobs. When I did this, there were no tools to build the blobs yourself, so I had to write a deployment script. Here’s how it works:

Firstly, I had to get a Fight Club script to put into the Python deployment script.

However, to put it into the blobs I needed to solve two problems.

1. The Fight Club script is too big to fit in the single blob (~160 KB, only text with all unnecessary symbols stripped)

2. It consists of ASCII printable characters that can be as big as 0111 1110 in binary form. If any field element starts with a character bigger than 0111 0011 (first bits of BLS modulus), the entire blob is invalid.

The first problem is easy - we can put up to 6 blobs in a single transaction. Here’s how I fixed the second one:

This part of the Python script splits the incoming data (in our case, the movie script) into chunks of 31 bytes each, pads every chunk with a null byte (thus, making them start with 0000 0000), and groups them into arrays of length 131072, essentially constructing the always-valid blobs.

Ok, now we’ve got blobs. How do we publish them?

This may seem weird. If you know no Python, this is what happens above:

• We build an array with all the data needed to construct the transaction. You may even notice familiar values - a gas limit of 21000 since we call no contracts, gas price, max priority fee, transaction count AKA nonce, but with versioned hashes of our blobs. Do you remember that, unlike blobs or commitments, they’re stored forever? Here’s why - they’re built in the transaction body, which is added to the block, which is added into the blockchain.

• We encode this array using RLP encoding and sign the result with our Ethereum private key. The signatures in Ethereum consist of three elements - two numbers (signature.r, signature.s) 32 bytes each, and a parity bit (signature.v).

  Notice the “03” number we add to the start of the array. This is the type of a blob-carrying (EIP-4844) transaction. Normal transactions have the type of 02 (EIP-1559).

• We build an array with our transaction, our blobs, its commitments, and proofs and pad them with the transaction type. This is a transaction envelope or packet, we should send it to our node provider so that it can be added to the mempool and distributed within the Ethereum P2P network.

Now we’re ready to send it. Ta-da!

The entire script took two blobs. The publication itself cost me 0.000000000000262144 ETH, which is virtually free. The main cost driver was sending the transaction to the network, which was about $5 at that moment.

You can send up to 6 blobs in the single blob-carrying transaction and, as it’s almost the same as the normal one, execute some other smart contract computations in the same transaction, distributing the costs.

The important thing to remember is that the EVM has no idea what the blobs were because I packed them into the networking packet. It only knows the hashes of the commitments, which is enough for me to verify blob proofs on it.

Here’s what the verification script looks like:

It’s very similar to the logic above. For a reason - what was changed is transaction type (now it’s 02, a normal EIP-1559 transaction), destination address (0x0A - point evaluation precompile), and the calldata (that is, transaction input) is not empty now. Everything else is the same.

The calldata is the sequence of the versioned hash, the field element existence of which we want to verify, the commitment, and the KZG proof. This is all the point evaluation precompile needs to verify our proof. As I already said, the blob is not needed to verify proofs.

Build, sign, pack, send. Voila.

The “Success“ label tells us that the precompile did not revert while verifying the proof, so it is valid. Yes, you got it right; this is just a smart contract call. Now the EVM knows the first rule of the fight club according to the scripts but no other rules or what the fight club is.

The source code of the scripts is available here.

---

Replace the first rule of the fight club with the forged batch element and the movie script with the batch of transactions, and that’s how optimistic rollups work. Make the proof a blob proof and wrap it into the validity proof so that it only needs the versioned hash as public input, and that’s how ZK rollups work. That’s it. That’s what the blobs changed in the rapidly moving Ethereum ecosystem.

In fact, the blob upgrade is nothing but using some cryptographic primitives to optimize the workflow of rollups, so that they can run more efficiently and give cheaper fees and smoother experience to end users. This is the whole crypto - huge engineering work taking a lot of man-hours that gives nothing but a better user experience in the completely trustless system slowly changing the entire world.

If you try to summarize this article, you’ll get completely different results based on who you’re trying to explain it to. For ordinary users, making the vast majority of the user base, “it makes the fees cheaper“ is enough. For curious minds, the concept of blobs being pruned after they’re not needed anymore should do the thing. If one is an engineer, however, you probably wouldn’t compress this material significantly much. I tried to cover everything you could be interested in if you wanted to know the blobs, but not going to write your own implementation of them. I hope this article gave you such kind of understanding and await your feedback on any socials or comments if they’re added to Mirror someday.

Thank you for reading.