Summary

At Breakpoint 2023, Nicolas Penny, COO and Co-Founder of Helius, presented a talk on scaling NFT compression from its inception to a stable production level. Penny provided insights into the challenges of implementing NFT compression algorithms on the blockchain, specifically Solana. He emphasized the significance of making NFTs more accessible and affordable through compression and shared the story of how Helius continually improved and scaled its infrastructure to support this innovative technology, including managing high-load events such as those caused by massive NFT airdrops like Drip.

Key Points:

NFT Compression Defined

Nicolas Penny provided a primer on NFT compression, a technology allowing users, especially those minting NFTs, to save on costs. By eliminating the need for separate mint, metadata, and token accounts, and instead storing data with transactions in a Merkle tree, the process costs significantly less—around $150 for one million NFTs. This efficiency is made possible through cryptographic hash functions that act as digital fingerprints verifying the data's correctness on chain. Penny highlighted that despite minor account involvement, compression is secure and cost-effective.

Scaling Journey and Challenges

The heart of Penny's talk was Helius' journey in scaling NFT compression to production. He described the painful early days of deployment, the process improvements made, and the operational challenges faced along the way, especially when handling high-volume NFT drops that could strain the system. Penny highlighted the importance of redundancy in data processing and the implementation of backfilling mechanisms to recover from any potential data losses. These steps were crucial to ensure reliable and continuous service, even during peak traffic times.

Operational Improvements and Solutions

Helius made several operational improvements to ensure the stability and robustness of their NFT compression system. These included creating verification and recovery tools, adding redundancy through multiple nodes, and implementing deduplication to manage high traffic loads more effectively. These improvements allowed Helius to provide reliable service to customers, supporting large-scale launches like those by Helium for hotspot management and Dialect for exchangeable stickers.

Future Developments and General Indexing

Looking ahead, Penny discussed upcoming enhancements such as updateable metadata for NFTs and broader applications of compression beyond NFTs. The latter entails developing more robust general indexing solutions to help other developers avoid the challenges Helius faced. He signaled that this work would help foster an ecosystem where developers can build custom compression solutions more easily, reducing costs for a range of blockchain-based services.

Facts + Figures

• Nicolas Penny is the COO and Co-Founder of Helius.
• Prior to Helius, Penny worked at AWS specifically on RDS.
• NFT compression significantly reduces costs for minting NFTs.
• Compressed NFTs use Merkle trees, eliminating the need for separate accounts and storing data with transactions.
• Helius tackled scaling from initial instability to reliable, low-latency production.
• Dialect successfully launched their exchangeable sticker program using NFT compression technology.
• Helium smoothly migrated their hotspot management to Solana using compressed NFTs.
• High traffic events, such as Drip's 350k NFT airdrop, highlighted the importance of deduplication in the system.
• Penny discussed the potential future of NFT compression technology, including updateable metadata.
• The overall goal is to develop general indexing solutions for the blockchain community.

Top quotes

• "Essentially it allows people to go and use accounts but a lot more cheap."
• "Traditionally, NFTs use three accounts...with compression NFTs you have none of these accounts, making it far cheaper."
• "The sequence number is a counter that increments every single time you do something on the tree."
• "The main point of my talk is just discussing the journey that we took at Helios to put all this stuff in production and run it operationally in a way that it becomes low latency, reliable."
• "The thing to think about is that when you're talking about hashes, they are by design irreversible."
• "This red here is locks...they're called exclusive locks."
• "The solution is pretty straightforward. It's basically doing a form of deduplication."

Questions Answered

What is NFT compression, and why does it matter?

NFT compression is a process that simplifies the creation and storage of non-fungible tokens by eliminating the need for separate accounts for minting, metadata, and the tokens themselves. Instead, it harnesses cryptographic data structures such as Merkle trees to streamline operations and minimize costs. For those minting NFTs, this means a significant reduction in expenses, leading to cheaper and more efficient token production. Compression is particularly impactful because it can democratize access to NFT creation, opening up the space to a broader range of creators and users by lowering financial barriers.

How did Helius scale NFT compression to production?

Helius began by wrestling with the early challenges of deploying NFT compression on the Solana blockchain, experiencing the growing pains common in new tech development. As issues arose, they developed various operational tools and added redundancy to their system, including multiple streaming nodes. Their approach also featured a complex and efficient backfilling system that allows the team to repair and recover data effortlessly. Throughout the process, they emphasized continuous improvement to reach a state of low latency and high reliability.

What operational solutions were critical in Helius’ journey with NFT compression?

Operational solutions that were instrumental in scaling NFT compression included verification tools to check the integrity of Merkle trees, scanning for and repairing missing data updates, and deduplication to handle high volumes of NFT minting operations more efficiently. These advancements collectively enhanced the robustness of Helius' infrastructural support for NFT compression, allowing for successful adoption by various companies and services seeking to utilize this technology at scale.

What complications arose with the introduction of large NFT airdrops, and how were they addressed?

When large NFT airdrops occurred, like those implemented by Drip, Helius' system experienced spikes in load, which stressed the infrastructure and created exclusive locks in their database. These locks occurred because multiple updates to the same table were waiting on access to write data. Helius' response was to introduce deduplication in their process, ensuring that each piece of data is processed only once, significantly reducing redundancy and the potential for system overload during these periods of intense activity.

What new features and developments are anticipated for NFT compression?

Future developments for NFT compression include allowing updateable metadata within NFTs, which would enable changes to an NFT's properties after its creation. More broadly, Helius aims to create general indexing solutions that will benefit any developer looking to leverage compression technology in their projects, regardless of whether they are directly related to NFTs. This expansive view will potentially lower costs and barriers for a wide variety of blockchain-based applications, paving the way for innovative uses of the technology across industries.