Solana's Latest Developments: ZK Features, Core BPF Progress, and New Developer Tools

In the latest episode of the Solana Changelog, hosts Nick from the Solana Foundation DevRel team and Jonas, currently attending Mountain Dow in Greece, dive deep into the recent updates and exciting developments in the Solana ecosystem. From new ZK features to core BPF program conversions and innovative developer tools, this episode covers a wide range of topics that are shaping the future of Solana.

ZK Features Coming to Solana

One of the most exciting announcements in this episode is the introduction of new Zero-Knowledge (ZK) features to Solana. Jonas highlights the addition of functionality to the confidential transfers and token extension program, which leverages ZK technology.

"There's some new ZK stuff coming to Solana, actually, which I think is super exciting," Jonas enthuses. He explains that developers will soon be able to use Algum hash to validate public keys and verify that the owner possesses the corresponding private key. Additionally, this feature will allow validation that two messages were signed by the same key.

This development is significant as it expands Solana's capabilities in the realm of privacy and security. The use of Algum encryption, which Jonas notes is similar to RSA, brings advanced cryptographic features to the Solana ecosystem. These additions are expected to open up new possibilities for developers building privacy-focused applications on Solana.

Borsh 0.9 Deprecation

Nick brings attention to an important breaking change in the Solana ecosystem: the official deprecation and removal of Borsh 0.9. This serialization library has been a crucial component for many Solana developers, and its removal marks a significant milestone.

"Borsh 0.9 has been deprecated, it's been deprecated for a while, but it's actually been finally removed. So it's a breaking change, just so everyone knows," Nick explains. He emphasizes that version 0.10 is now the current version, urging developers to update their projects accordingly.

This change underscores the importance of staying up-to-date with the latest developments in the Solana ecosystem. Developers are encouraged to test their applications thoroughly to ensure compatibility with the new Borsh version.

Anchor CI Updates

The Changelog also covers important updates to the Anchor CI (Continuous Integration) process. Nick explains that the Agave repository, which is central to Solana's development, now includes CI jobs to ensure compatibility with Anchor, a popular framework for Solana development.

"There's actually a CI job within the Agave repo that actually checks to make sure that anchor will still work since anchor is so widely used," Nick elaborates. He highlights that the CI has been updated to include checks for both Anchor 0.29 and 0.30, acknowledging that not all developers have migrated to the latest version due to significant changes.

This dual-version checking approach demonstrates Solana's commitment to backwards compatibility and smooth transitions for developers. It ensures that changes to the core protocol don't inadvertently break widely-used development tools.

SyscallGetSysvar Improvements

Jonas brings attention to improvements in the Sysvar system, which is crucial for developers interacting with Solana's on-chain state. He explains that these changes will make it easier for developers to read large amounts of data from other accounts, such as stake accounts.

"Now they are improving on this. So it would be way easier, for example, to read like big amounts of data from other accounts," Jonas explains. He adds that developers will be able to read data directly from the buffer instead of loading entire accounts, which should lead to improved efficiency.

Nick adds that these changes are part of a broader initiative to streamline Sysvar access. He mentions that eventually, many of the current Sysvar syscalls may be deprecated in favor of this new, more generic approach.

This improvement showcases Solana's ongoing efforts to enhance developer experience and optimize on-chain operations. By making it easier to access and manipulate on-chain data, these changes could lead to more efficient and sophisticated applications on the Solana network.

Core BPF Initiative Progress

A significant portion of the Changelog is dedicated to discussing the progress of the Core BPF initiative. This ongoing project aims to convert core Solana programs from native code to BPF (Berkeley Packet Filter) programs, enhancing compatibility across different validator implementations.

Nick explains, "The core BPF initiative is ever ongoing. And there's actually the first two of those programs that were initially enshrined programs, they're being converted to BPF." He specifically mentions the config program and the feature gate program as the first two to receive feature gates for eventual deployment across all clusters.

This initiative is particularly important for the development of alternative validator clients like Firedancer. By converting core programs to BPF, Solana ensures that these programs can be easily implemented across different validator implementations without the need for each team to rewrite them from scratch.

Jonas adds context to this, explaining, "They rewriting all the core programs, or most of them to BPF programs. And this is mainly so that Firedancer doesn't need to re-implement everything because they can just use the same compiled programs."

SIMD 89 Amendment: Programify Feature Gates

The hosts discuss an important amendment to SIMD 89, which deals with the process of upgrading core BPF programs. Nick explains that the initial proposal involved a multi-sig approach with specific key holders at different organizations. However, the amendment changes this to a feature gate within the feature gate program itself.

"The proposed change is actually to make it so it's a feature gate within the feature gate program to actually upgrade the core BPF programs, which I personally think makes way more sense," Nick comments.

Jonas agrees, adding that this new approach allows for a more mature and transparent upgrade process. He suggests that validators might also be involved in voting on these upgrades, further decentralizing the decision-making process.

This change reflects Solana's commitment to decentralization and transparency in protocol upgrades. By moving away from a multi-sig approach to a feature gate system, Solana is creating a more robust and community-driven process for implementing critical upgrades.

Rent Collection Disabled: SIMD 84 Approval

One of the most significant announcements in this Changelog is the approval and merging of SIMD 84, which disables rent collection on Solana. Nick explains the background and implications of this change:

"SIMD 84 was officially approved and merged into the SIMD repo. And SIMD 84, for those of you who don't know, is to actually disable rent collection," Nick states. He goes on to explain that rent collection has been effectively deprecated within the runtime for quite some time, as all accounts have been required to be rent-exempt for a while now.

This change has several important implications:

1. Simplification of the codebase: The core protocol engineers can now remove a significant section of unused code, leading to a simpler and more maintainable codebase.

2. Faster Firedancer development: As Jonas points out, this change means the Firedancer team won't have to implement rent collection logic at all, potentially speeding up their development process.

3. Terminology shift: Jonas suggests that the term "rent" might be phased out in favor of "minimum balance," which more accurately reflects the current system.

This change exemplifies Solana's commitment to continuous improvement and optimization of its protocol. By removing unused features and simplifying the codebase, Solana is paving the way for more efficient and streamlined blockchain operations.

Trident: Open-Source Fuzzing Tool

The Changelog also highlights exciting new developer tools in the Solana ecosystem. One such tool is Trident, an open-source fuzzing tool developed by Aki Blockchain.

Jonas explains, "There's a new open source program from Aki Blockchain, which is an open source fuzzer. And if you don't know what a fuzzer is, like a fuzzer is basically something that you can use to test your programs."

He goes on to describe how Trident works: "And what it does, it just throws a bunch of random stuff at your program, basically, just all kinds of random bytes and so on. And sees if there's problems, if you get overflows or any data problems."

This tool represents a significant addition to the Solana developer toolkit, especially for those focused on security and robustness. By subjecting programs to randomized inputs, developers can uncover potential vulnerabilities or edge cases that might not be caught by traditional testing methods.

Nick emphasizes the importance of such tools, suggesting that developers should use Trident in addition to formal audits. This approach can help create a more comprehensive security strategy for Solana applications.

Web3.js Socket Issue Resolution

The hosts discuss a long-standing issue with web socket connections in web3.js, Solana's JavaScript library. Nick explains that this problem, which has persisted for about two years, caused developers to run out of connections when attempting to establish web socket connections.

"Turns out they would basically go to a really high connection count when you really only need one," Nick explains. He then describes the surprisingly simple solution proposed by Wilfred from Triton: setting the default connection pool size to just one.

This fix, while seemingly minor, has the potential to solve numerous problems that developers have been grappling with for years. It's a testament to the power of open-source development and community problem-solving.

Jonas adds that this issue was particularly challenging because it only manifested when connection limits were reached, making it difficult to reproduce consistently. The resolution of this problem is expected to significantly improve the developer experience when working with web3.js.

Unruggable: Open-Source CLI Wallet

Another exciting development highlighted in the Changelog is the creation of an open-source CLI wallet called Unruggable. Nick expresses enthusiasm about this project, noting its comprehensive documentation and user-friendly features.

"Someone created basically an open source CLI wallet, which is like incredibly cool. It's under the GPL3 license. And the docs guide here, the readme in this repo is actually like pretty detailed of like how to set it up, which is amazing," Nick explains.

He goes on to describe some of the wallet's features, including the ability to display QR codes in the terminal. This capability could be particularly useful for developers working on cross-device or mobile-desktop integrations.

Jonas points out the potential utility of such a tool in development environments: "It always helps like even with like development, like just look at your wallet that is safe somewhere and your local file and see what's actually going on in there."

The introduction of Unruggable demonstrates the vibrant and innovative nature of the Solana developer community. By creating open-source tools that address specific needs, developers are continually expanding and improving the Solana ecosystem.

Stack Exchange Activity

The hosts conclude the Changelog by highlighting the increasing activity on Solana's Stack Exchange. Jonas mentions several active contributors and notes the positive feedback from the community.

"We have Jimmy here, we have Sid, Biz Aktar and Christian is there, column of course. So yeah, it's super good. Like I heard from many people that they're actually getting their answers faster now and better answers," Jonas reports.

Nick adds that some questions are receiving multiple valid answers within minutes of being posted, indicating a highly engaged and knowledgeable community. This level of activity and quick response times can significantly improve the developer experience on Solana, making it easier for both newcomers and experienced developers to find solutions to their problems.

The growth of the Solana Stack Exchange is a positive indicator of the ecosystem's health and the strength of its developer community. As more developers contribute their knowledge and experience, the platform becomes an increasingly valuable resource for anyone building on Solana.

Conclusion

This episode of the Solana Changelog provides a comprehensive overview of the latest developments in the Solana ecosystem. From core protocol improvements like the conversion of native programs to BPF and the disabling of rent collection, to new developer tools like Trident and Unruggable, Solana continues to evolve and improve.

The introduction of new ZK features and the resolution of long-standing issues in web3.js demonstrate Solana's commitment to enhancing both the capabilities and the developer experience of the platform. Meanwhile, the growing activity on the Solana Stack Exchange and the continuous community contributions to open-source tools highlight the strength and engagement of the Solana developer community.

As Solana continues to mature and expand, these updates and improvements pave the way for more efficient, secure, and innovative applications on the network. Developers, users, and stakeholders in the Solana ecosystem have much to look forward to as these changes are implemented and new tools become available.

Facts + Figures

• New ZK features are coming to Solana, including Algum hash for public key validation and message signing verification.
• Borsh 0.9 has been officially deprecated and removed, with version 0.10 now being the current version.
• The Agave repository's CI now checks compatibility with both Anchor 0.29 and 0.30.
• SyscallGetSysvar improvements will make it easier to read large amounts of data from other accounts.
• The first two core programs (config and feature gate) have received feature gates for conversion to BPF.
• SIMD 89 amendment changes the upgrade process for core BPF programs from a multi-sig approach to a feature gate system.
• SIMD 84, which disables rent collection, has been approved and merged.
• Trident, an open-source fuzzing tool for Solana programs, has been released by Aki Blockchain.
• A long-standing web socket connection issue in web3.js has been resolved by setting the default connection pool size to one.
• Unruggable, an open-source CLI wallet for Solana, has been created with features like QR code display in the terminal.
• The Solana Stack Exchange is seeing increased activity, with some questions receiving multiple valid answers within minutes.

Questions Answered

What new ZK features are coming to Solana?

Solana is introducing new Zero-Knowledge (ZK) features, particularly in the realm of confidential transfers and token extension programs. These include the ability to use Algum hash for validating public keys and verifying that the owner possesses the corresponding private key. Additionally, developers will be able to validate that two messages were signed by the same key. These features leverage Algum encryption, which is similar to RSA, bringing advanced cryptographic capabilities to the Solana ecosystem.

Why is Solana converting core programs to BPF?

Solana is converting its core programs from native code to BPF (Berkeley Packet Filter) programs as part of the Core BPF initiative. This conversion is primarily aimed at enhancing compatibility across different validator implementations, particularly for alternative clients like Firedancer. By having core programs in BPF format, teams developing new validator implementations won't need to rewrite these programs from scratch. This initiative simplifies the development process for alternative clients and ensures consistency across the Solana ecosystem.

What changes are being made to rent collection in Solana?

Solana has approved SIMD 84, which effectively disables rent collection on the network. This change comes after a period where all accounts have been required to be rent-exempt, making the rent collection code unused. With this update, the core protocol engineers can remove a significant section of unused code, simplifying the codebase and potentially improving performance. This change also means that the Firedancer team won't need to implement rent collection logic, potentially speeding up their development process. The term "rent" might be phased out in favor of "minimum balance" to more accurately reflect the current system.

What is Trident and how can it help Solana developers?

Trident is a new open-source fuzzing tool developed by Aki Blockchain for testing Solana programs. It works by throwing random inputs at a program to uncover potential vulnerabilities, overflows, or data problems that might not be caught by traditional testing methods. Trident is designed to complement formal audits and can be an important part of a comprehensive security strategy for Solana applications. By using Trident, developers can improve the robustness and security of their Solana programs before deployment.

What is Unruggable and how can it be useful for Solana developers?

Unruggable is an open-source CLI (Command Line Interface) wallet for Solana. It provides a comprehensive set of wallet functionalities accessible through a command-line interface, making it particularly useful for developers and advanced users. One of its notable features is the ability to display QR codes in the terminal, which can be helpful for cross-device or mobile-desktop integrations. Unruggable can be especially valuable in development environments, allowing developers to easily check wallet states and transactions during the development process. Its open-source nature also means that it can be customized or integrated into other tools as needed.