Simple Serialize, or SSZ, is a serialization and deserialization protocol used extensively in Ethereum. From a .NET/C# developer's standpoint, think of SSZ as akin to serialization frameworks you might be familiar with, like JSON.NET or XML Serialization in .NET, but with specific optimizations and features tailored for blockchain technology.
• Data Types and Structures: SSZ is designed to handle standard data types (integers, byte arrays, etc.) and complex structures (akin to C# classes or structs). o Imagine serializing a C# class: SSZ does something similar but in a way, that’s optimized for Ethereum’s data structures.
• Deterministic Serialization: Determinism in serialization is crucial in a blockchain context to ensure consistency across different nodes. o In .NET terms, this would be like having a guarantee that binary serialization of an object always yields the exact same byte array.
• Efficiency and Size: SSZ is more efficient in terms of computational resources and storage space than text-based serialization formats like JSON. Think of the efficiency you get with binary serialization in .NET but fine-tuned for the kinds of data structures used in Ethereum.
• Merkle Trees and Hashing: SSZ is designed to be compatible with Merkle trees, a fundamental aspect of blockchain's integrity and security mechanism. This is similar to computing hashes of serialized objects for ensuring data integrity, a concept familiar in .NET with cryptographic hash libraries.
While .NET serialization focuses on general-purpose use and human readability (especially with JSON and XML), SSZ is specialized for blockchain’s efficiency and security requirements.
SSZ’s deterministic nature is somewhat like binary serialization in .NET but with added blockchain-specific features.
In .NET, you often choose between human-readable (JSON, XML) and efficient (binary) serialization. SSZ is more akin to the latter but is tailored for Ethereum’s blockchain.
BLS-384, part of the Boneh-Lynn-Shacham (BLS) signature scheme, is a cryptographic algorithm that uses elliptic curve pairings over a 384-bit curve. It's gaining traction in blockchain technologies, including Ethereum 2.0, for its unique properties in signature aggregation. Understanding BLS-384 is crucial for developers involved in blockchain and cryptographic applications.
Familiarize yourself with ECC in .NET, as BLS builds on these concepts. Classes under System.Security.Cryptography can provide foundational knowledge.
The Ethereum API Builder Specification is a set of guidelines and standards for external block builders in the Ethereum ecosystem. It's designed as an interface for consensus layer clients to source blocks built by external entities. This specification is particularly significant in the context of block building, which is a specialized activity that requires high fixed costs to be an efficient validator. This creates an advantage for staking pools as they can effectively distribute the cost across many validators.
The Builder API serves as a temporary solution that requires higher trust assumptions than Proposer-builder separation (PBS) but can be fully implemented without modification to the base protocol. It operates by providing the proposer with a "blind" execution layer header to incorporate into a block and a "value" amount, which will be transferred to the proposer once they create a block with the header. Once the proposer signs a block with the header, they are bound to the choice (or risk being slashed due to equivocation). This allows the builder to reveal the blinded transactions without the possibility of the proposer tampering with them.
Serialization of formats like JSON/XML has been a prevalent practice in the .NET environment for many years, resulting in a variety of frameworks available today. For instance, Newtonsoft, which has been widely used for a long time, and with the advent of .NET Core, Microsoft's own implementation under System.Text.Json has gained increasing popularity. The challenge here lies with SSZ, which is more specific to blockchain and lacks widespread implementation.
The 'System.Security.Cryptography' namespace in .NET provides a comprehensive range of cryptographic services. These services include secure encoding and decoding of data, hashing, random number generation, and message authentication. It offers implementations for a wide array of standard cryptographic algorithms, encompassing various facets of cryptography such as symmetric and asymmetric encryption, digital signatures, and hash algorithms. Examples of these algorithms include RSA, AES, TripleDES, and SHA512
To utilize tools that are widely recognized and used by many C# developers, I selected XUnit as the testing framework. It's important to note that this choice isn't exclusive to XUnit; you can easily opt for any other framework that suits your preferences. In addition to the conventional unit testing, I identified three specific components that required testing: SSZ, BLS, and the integration of all these elements.
A general challenge in testing is the handling of production source data. In this case, there is a relationship between the response we receive from the Builder API resources 'getheader' and 'submitpayload'. Based on my experience, I prefer working with test data derived from real-world production sources. Both payloads are rather big so instead of trying to mock these responses we set up the system, ran tests against the Testnet, and logged the winning bids along with the complete requests. This approach allowed us to save payloads as JSON text files for use in our test project as mocked data.
Addressing this took up a significant portion of the development time because we had to deploy this to run against Testnet and wait for a winning bid, which happens to us about once in a 24-hour window.
Now that we have everything in place regarding test data, we can establish a level of confidence sufficient to release the first version. We have some unit tests that primarily verify our test data contains the correct information. However, in my opinion, the more crucial aspect to focus on is the integration tests, namely testing a complete process, such as a winning scenario and one where something has gone wrong.
We concentrated on two distinct components, SSZ and BLS. Neither SSZ nor BLS is built into .NET, nor were they easy to find implementations of. Our test project references the API project, which, in its basic form, inherits all configurations. However, thanks to our dependency injection, we can override specific services. For example, the API calls that, in production, would have gone to a relay are now utilizing the information we obtained from the Testnet mentioned earlier. So, in simple terms, we have abstracted away the blockchain and can run our integration tests locally on our development machines, and in the current scenario, in our DevOps environments through GitHub Actions, without a connection to a blockchain.
To summarize, there have been three fundamental challenges in this project: SSZ, BLS, and sourcing test data from production.
SSZ: Unlike JSON, SSZ is not as widely used by developers, which requires us to go back to the drawing board multiple times. We considered developing something from scratch but eventually found a project that we could build upon to support the latest versions of Ethereum.
BLS: The scenario is similar to SSZ in terms of usage. We were unable to find any readily available package or implementation. Instead, our implementation is based on a C++ library that we can call from C#.
Blockchain Test Data: We're accustomed to having sample data provided to work with for testing purposes. However, as mentioned earlier, we ended up running the entire application on Testnet, waiting, and hoping for a winning block, and then logging the information to work with that data.
I would estimate that these three components have taken up about 70% of the development time. Now that they are in place, future development to support the next Ethereum hard fork will be significantly faster. With all this established, we are in a strong position to continue our development and very easily add additional features as requested by the community.