Building Sig, a New Read-Optimized Solana Validator

Summary

Syndica, a web3 native cloud platform, announced the development of SIG, a new read-optimized Solana validator. Ahmed, the CEO and co-founder, together with Brennan Gebotys, a Senior Protocol Engineer at Syndica, introduced the project and its significance in the context of blockchain technology and Solana’s ecosystem. Their presentation focused on the validator's unique approach to handling read operations, which constitute the majority of the requests to nodes. They highlighted the use of Zig, a general-purpose programming language, chosen for its optimization capabilities and readability, and outlined the roadmap towards a validator capable of efficiently following the blockchain.

Key Points:

Introduction to SIG and Syndica’s Vision

Ahmed from Syndica introduced SIG, a new Solana validator designed to optimize read operations on the blockchain. He emphasized the importance of creating a bridge for web2 developers to enter the web3 space, leveraging cloud technology. Syndica, through its RPC infrastructure and specialized APIs, supports this transition by enabling easy deployment of applications on their platform. Ahmed pointed out that while much attention is given to transactions per second (TPS), the reads per second (RPS) metric is critical for end users who frequently interact with the blockchain data. SIG is their response to improving Solana’s read performance and client diversity.

The Technical Advantages of Zig

Brennan explained their choice of Zig for developing the validator. Zig's clarity in memory allocation and explicit error handling simplifies code comprehension and enhances safety features like memory leak detection. This transparency facilitates performance optimization while avoiding common memory-related errors. Furthermore, Zig supports the easy integration of C code, which allows leveraging existing high-performance codebases. Brennan conveyed that Zig’s simplicity will encourage contributions and growth within the Solana validator community.

Roadmap and Current Progress

The SIG's roadmap was shared, detailing the phases involved in developing a comprehensive validator. Starting with the Gossip Protocol and advancing to AccountsDB and onward to consensus mechanisms and RPC infrastructure. The goal is to create a validator that excels in RPS, given that certain data queries are taxing on the network. As of the announcement, the Gossip Protocol has been completed and the team is actively working on AccountsDB.

Facts + Figures

• Syndica processes 100 to 200 terabytes of data each month.
• About 96% of all calls to nodes are read calls.
• There is a significant "slot lag" issue in Solana compared to Ethereum.
• Zig is chosen for its explicit error and memory allocation handling, increasing the robustness and readability of the validator software.
• The roadmap for SIG includes the development of Gossip Protocol, AccountsDB, state machine, consensus mechanisms, and finally RPC infrastructure optimized for RPS.
• Current progress has Syndica completing the Gossip Protocol.
• The team is focusing on AccountsDB, targeting optimization for read-heavy queries such as getProgramAccounts.

Top quotes

• "We process anywhere from 100 to 200 terabytes of data every single month."
• "96% of all calls made to nodes are actually read calls."
• "If you're excited about building the runtime, if you're excited about building AccountsDB or optimizing gossip, whatever it may be, just send in the application."
• "We have a lot of work to do to make sure we're maintaining readability and fine-grain understanding of what's going on in the code."
• "We are building literally the future of the decentralized finance, really, in the whole world."

Questions Answered

What is SIG?

SIG is a new read-optimized Solana validator being developed by Syndica. Its purpose is to address and improve the performance of read operations on the Solana blockchain, which are a significant majority of node requests. The use of SIG is expected to enhance the network's throughput concerning reads per second (RPS) and offer a more varied client ecosystem, which currently lags behind Ethereum’s.

Why is focusing on reads per second (RPS) important?

RPS is crucial because most end-user interactions with the blockchain are read operations, such as fetching account balances or contract states. Optimizing RPS will potentially lead to a faster and more efficient user experience, reducing lag and allowing for more rapid access to blockchain data.

Why did Syndica choose Zig for developing SIG?

Zig was chosen for its ability to write robust, optimal, and reusable software with no hidden control flow or memory allocations. This clarity allows developers to understand, maintain, and optimize the code more effectively. Zig's compatibility with existing C libraries also ensures that SIG can benefit from well-tested performance code already available.

How does Zig contribute to the safety and performance of the SIG validator?

Zig’s manual memory management allows for performance benefits through fine-grain control over expensive operations like memory allocation. Additionally, features like the defer statement prevent common issues like memory leaks and segmentation faults, which contributes to creating reliable and efficient validator software.

What progress has been made on SIG so far?

The team at Syndica has completed the development of the Gossip Protocol, which allows the Zig client to communicate with mainnet nodes. They are currently working on the AccountsDB to efficiently represent the state of the blockchain on the validator, with a particular focus on optimizing read-heavy operations.