Solana's Revolutionary Compression Technology: Reshaping NFTs and Blockchain Data Storage

In a recent episode of Validated, host Austin moderates an insightful panel discussion about Solana's groundbreaking compression technology. The panel features Jon Wong from the Solana Foundation, Noah Prince from the Helium Foundation, and Ilmoi, co-founder and CEO of Tensor. These experts dive deep into the technical aspects of compression, its current applications, and its potential to drive mass adoption of Solana.

What is Compression?

Compression is a innovative method of storing data on the Solana blockchain that dramatically reduces costs by orders of magnitude. At its core, compression leverages a data structure called a concurrent Merkle tree, which creates a unique fingerprint from massive amounts of off-chain data while storing that fingerprint on-chain. This approach allows for significant cost savings and opens up new possibilities for blockchain applications.

Jon Wong explains the origin of compression:

"Part of this was a prompt around economics. It was a company you might have heard of called Instagram that was looking to figure out how they could mint millions of NFTs on Solana. They had the thoughtfulness to think about how they would sponsor gas fees, transaction fees, deposits, storage deposits, all that sort of stuff. And when doing the math at the current price of the SOL itself, just the tokens, it was going to be just way too inefficient and economically infeasible for them to provide the experience they wanted to provide for their consumers."

The Economics of Compression

The economic impact of compression cannot be overstated. With traditional NFT minting, the costs can be prohibitively expensive for large-scale projects. However, compression changes the game entirely. As Jon Wong points out:

"With compression, you can mint one hundred million NFTs for two thousand dollars."

This dramatic reduction in costs opens up a world of possibilities for creators, brands, and developers who previously may have been priced out of the NFT space.

Compressed NFTs: A New Paradigm

While compression has various applications, its most common use case currently is compressed NFTs. These NFTs maintain all the functionality of traditional NFTs but at a fraction of the cost. This cost-effectiveness is driving innovation and new use cases in the NFT space.

Ilmoi from Tensor shares his perspective on the impact of compressed NFTs:

"We're like super bullish in the compressed NFTs at Tensor. We just think that this is the moment when Solana really starts to diverge from EVM and starts to show its colors and do interesting things that are simply not possible there. We think that in five years from now, the average NFT project on Solana will be something very different to the average NFT project on EVM. Not better, not worse, just different. It'll be like the kind of things that you can only do with like quantities in the millions and billions."

Technical Implementation of Compression

The technical implementation of compression is complex but elegant. Jon Wong explains:

"We combine both bits of data that we both think are on-chain, and combining them using the Merkle tree structure. So for those of you don't know, like a Merkle tree, you basically take perfectly balanced binary tree, you hash it all the way up, and the very top node of this tree is a hash, and that hash uniquely identifies the entire state of the tree. That hash is in an account, and all the leaves, all of the NFTs, are in the Solana ledger."

This approach allows for efficient storage and verification of large amounts of data, making it possible to create and manage millions of NFTs at scale.

Compression Beyond NFTs

While compressed NFTs are the most visible application of compression technology, its potential extends far beyond digital collectibles. Noah Prince from the Helium Foundation shares how they've leveraged compression for their network:

"Helium, we've really played with this and the hotspots are NFTs. Helium mobile is coming out with mobile subscribers of people who have like SIM cards. And those people actually participate in the network by mapping so they can like kind of say, where coverage is. And those people, they also have an NFT that represents them as a subscriber."

This example illustrates how compression can be used to represent real-world assets and participation in decentralized networks, expanding the definition of what an NFT can be.

The Developer Experience

Implementing compression technology is not without its challenges. The panelists share their experiences of "chewing glass" – a term used in the Solana community to describe the difficulties of working with new, complex technologies.

Ilmoi recounts:

"It sort of really threw me back to the early days of Solana, when I remember joining as a dev, and there was like a single large discord channel where everybody would like ask questions, and whoever was the loudest got the most replies. And there was like barely any docs, if you wanted to know how something worked, you went source diving."

Despite these initial hurdles, the panelists agree that the ecosystem has made significant strides in improving the developer experience, with more tools and documentation becoming available.

Security and Decentralization

A common concern with new blockchain technologies is security and decentralization. The panelists address these concerns head-on, explaining that compressed NFTs are just as secure as traditional NFTs on Solana.

Jon Wong elaborates:

"The combination of the two means that anything that was minted, any of these compressed entities is as secure and decentralized as Solana itself. Even the smart contracts itself, there's actually two smart contracts. There's the state compression one and the bubblegum one. The state compression program was explicitly designed so that the person who was creating the storage box could not take away any NFT from other people."

This design ensures that compressed assets maintain the same level of security and decentralization that users expect from blockchain technology.

New Use Cases Unlocked by Compression

The low cost of compression is unlocking entirely new use cases for NFTs and digital assets. Jon Wong shares his perspective:

"I think that enterprises are going to start changing the kinds of products that they can build with blockchain technologies. And we're starting to see that already. Dialect with whatever, you know, 1.5 million stickers issued. They were able to, I think it's not about the scale that was interesting is that they were able to justify covering the minting and transaction costs for every single one of those stickers that gets minted. And that means that a consumer can go with a wallet with zero SOL and be able to receive and use digital assets."

This ability to create frictionless user experiences could be a game-changer for mass adoption of blockchain technology.

The Future of NFTs on Solana

The panelists agree that the future of NFTs on Solana will look very different from the 10,000 PFP (profile picture) collections that have dominated the space thus far. Noah Prince explains:

"Having had a chance to talk with plenty of these product companies that are thinking about NFTs, you know, coming back to like, what are these NFTs, they're not going to look like the, you know, digital collectibles where there's 10,000 of them, they're not going to look the same way, they're not going to be exactly PFPs. They're going to be digital assets, as Noah mentioned earlier, they're going to represent just digital identifiers."

This shift in thinking about NFTs as digital assets rather than just collectibles opens up a world of possibilities for real-world applications of blockchain technology.

Challenges and Future Developments

While compression offers many benefits, it also presents new challenges, particularly in terms of data storage and retrieval. The panelists discuss potential solutions to issues like the growing ledger size of Solana and transaction size limits.

Jon Wong mentions ongoing work in this area:

"We need to provide more tools. We need to provide snapshots. You know, it's still, you know, we're still maybe two months into the launch of the formal launch of the product. I expect in two years it's going to be a little arduous for folks to even who even know what's going on to pick this stuff up. But, you know, I don't think it's going to be impossible to be able to create those things."

Social Media on Solana with Compression

The discussion touches on the potential for building social media platforms on Solana using compression technology. While it's technically possible, the panelists agree that the main challenges are product-related rather than technical.

Noah Prince shares his thoughts:

"I think there's a guy on Solana Vixit that's building solar plex and that is basically, I think we're already there. Someone's doing it. I mean, it's Reddit, not Twitter, but... There's Vixit on a solar plex and then, you know, shake building gum, right? The fundamental primitives of social media can all be stored using compression."

This demonstrates that compression technology is already enabling new types of applications on Solana beyond just NFTs.

The Big Data Problem for Blockchains

As blockchain adoption grows, so does the amount of data that needs to be stored and processed. The panelists discuss how compression fits into this larger trend and the challenges it presents.

Ilmoi shares his perspective:

"I think a really good framework that I encountered some time ago was by multicoin, where they said like, the layer ones themselves are optimized for writes. Because that's so far what's been the bottleneck. And if you think about all the work, Solana has done, if you think about all the work that layer two have done it, so it's always been about through both TPS and how do you write more to the blockchain. And then what's probably going to happen is, as there's real apps built in the blockchain, that use that data, you now have this opportunity for a layer of startups to exist that are optimized for reads."

This insight suggests that compression is just one piece of a larger puzzle in addressing the big data challenges facing blockchain technology.

Compression and the Future of Blockchain Architecture

The discussion turns to how compression might influence the future architecture of blockchain networks. The panelists explore the idea of specialized chains versus general-purpose blockchains like Solana.

Noah Prince offers his thoughts:

"It's not clear to me yet that, you know, I had this thought originally coming into blockchains that it could be something like databases where, you know, you explicitly make different trade-offs and that leads to different consequences. So, for example, you know, you have Postgres, which is storing things in row wise format. And so, it's really, really quick for inserting, but querying is slower because you have to pull the entire row out of the hard disk every single time you query versus like redshift, which is a fork of Postgres that is columnar. And so, the inserts are slower, but the queries are faster. And so, I had this thought like, 'Oh, what if blockchains are like that?' You could just make these different fundamental decisions that, you know, make your transaction slower, but you read better. But I just haven't seen evidence of that."

This observation suggests that while specialization might seem appealing, general-purpose blockchains like Solana may continue to dominate due to their flexibility and ability to adapt to various use cases.

The Role of Off-Chain Solutions

As the discussion progresses, the panelists explore the role of off-chain solutions in conjunction with on-chain compression. They highlight that while the core security and integrity of the data remain on-chain, certain read-optimized operations can be handled off-chain without compromising the overall security model.

Noah Prince explains:

"In fact, I'm okay even with the web2 way of doing it where, you know, you're using geyser with a, you know, Kafka pipeline off of it and indexing it in a web2 way. Like, web2 is fine. It works. And as long as you've got enough people doing it, it is decentralized, because Triton's doing it, Helios is doing it, Hello Moon just showed up and is indexing the compressed NFTs. Like, at a certain point, when I've got 2000 different people running the indexer for the compressed NFT, does it matter if it's not stored on Arweave? Probably not."

This perspective highlights the potential for a hybrid approach that leverages the strengths of both on-chain and off-chain solutions to create scalable and efficient blockchain applications.

Compression and Developer Empowerment

One of the key themes that emerged from the discussion is how compression empowers developers to build applications that were previously impractical or impossible due to cost constraints. Ilmoi shares an example from Tensor's experience:

"We're doing a big airdrop that's coming in at Tensor. And we basically have these two options. Like, do we do it completely centralized off chain, which is like done by us? Or do we force everything to go through Solana? And we're going with a ladder because that leaves us no chance to mess it up. Because if we do it ourselves, there's a real chance that we airdrop somebody more than we should have or less than we should have. But if everything is going through the Solana's runtime, then the blockchain itself guarantees that everybody receives exactly the amount they should be receiving."

This example illustrates how compression, combined with Solana's capabilities, allows developers to leverage the blockchain's inherent guarantees to build more robust and trustworthy applications.

The Impact on User Experience

The panelists emphasize that the true power of compression lies not just in its technical capabilities, but in how it can transform user experiences. By dramatically reducing costs, compression enables developers to create frictionless onboarding experiences for users who may not even be aware they're interacting with a blockchain.

Jon Wong elaborates:

"I think that fundamentally changes the kinds of products. It changes the kind of audience that folks can build towards. And I think that's going to translate really well to games, where games definitely want to issue tons of digital assets on chain, allow them to be tradable and allow them to grow, that sort of stuff, as well as enterprises where, you know, coming back to this Instagram example, an enterprise, you know, everything is a million plus, right? And everything times a million gets pretty expensive, except for compression, right?"

This potential for seamless user experiences could be a key driver in bringing blockchain technology to the mainstream.

Addressing Scalability Concerns

As the discussion turns to the future of Solana and blockchain technology in general, the panelists address concerns about scalability, particularly in light of the increasing amount of data being stored on-chain through compression.

Jon Wong acknowledges the challenge:

"So, you know, while we're saving a lot of SOL tokens from moving this data from the account, the ledger size is growing at an incredibly high rate. Solana's already got, you know, multiple hundreds of terabytes worth of archival data. So, this is a problem that would have needed to get solved anyway. And this is only sort of throwing more fuel on the fire in terms of like what RPC providers have to do in order to serve all these queries."

However, the panelists also discuss potential solutions, including new incentive structures for archival node operators and innovative approaches to data storage and retrieval.

The Evolution of NFTs

Throughout the discussion, it becomes clear that compression is not just a technical innovation, but a catalyst for reimagining what NFTs can be. Noah Prince shares his perspective:

"I pretty much make anything a token that I want to show up in your wallet as something that you own. So even your positions in our, you know, in our governance, your positions, even though you've deposited tokens, actually come back as as NFTs. You know, we really kind of brought in the way that you look at NFTs past just, you know, JPEGs of images and collectibles and really into is why like the term digital asset a lot better than NFT, because it kind of fully describes what it is."

This evolution in thinking about NFTs as digital assets rather than just collectibles opens up new possibilities for representing ownership, participation, and value in the digital world.

Compression and Interoperability

While the discussion primarily focuses on Solana, the panelists also touch on how compression might impact interoperability between different blockchain networks. They suggest that as compression becomes more widely adopted, it could potentially become a standard feature across multiple chains, facilitating easier cross-chain interactions and asset transfers.

The Role of Compression in DeFi

Although the current focus of compression has been on NFTs, the panelists briefly explore its potential applications in decentralized finance (DeFi). They speculate that compression could enable more efficient on-chain order books, reduce the cost of complex financial instruments, and potentially even revolutionize how liquidity is managed on decentralized exchanges.

Compression and Privacy

An interesting point raised during the discussion is the potential intersection of compression technology with privacy solutions. While not explored in depth, the panelists hint at the possibility of using compression techniques to create more efficient zero-knowledge proofs or other privacy-preserving mechanisms on Solana.

The Environmental Impact of Compression

While not explicitly discussed, the implications of compression on the environmental impact of blockchain technology are significant. By reducing the amount of data that needs to be stored and processed by every node in the network, compression could potentially lead to lower energy consumption and a smaller carbon footprint for the Solana ecosystem.

Compression and Regulatory Compliance

As blockchain technology moves towards mainstream adoption, regulatory compliance becomes an increasingly important consideration. The panelists briefly touch on how compression might interact with regulatory requirements, particularly in the context of storing and managing large amounts of user data on-chain.

The Future of Compression Technology

Looking to the future, the panelists express excitement about the continued evolution of compression technology. They speculate on potential improvements, such as more efficient Merkle tree structures, advanced pruning techniques, and the integration of compression with other cutting-edge blockchain technologies.

Jon Wong shares his vision:

"I definitely think that we will see just an egregious amount. If Vibhu has his way with drip, we're going to see an egregious amount of NFTs showing up in people's wallets, he treats them like email, right? And I think it's a fascinating comp for something that I hadn't really processed. We're going to have to build products and services to manage how many things are happening and making sure that all of this stuff is efficient and you're not getting overloaded by it. But I think that's probably okay because that means that more things are on chain and composable at different pieces."

This perspective highlights the potential for compression to fundamentally change how we interact with blockchain technology, moving towards a future where digital assets are as ubiquitous and easy to use as email.

Conclusion

The discussion on compression technology in this episode of Validated provides a fascinating glimpse into the future of blockchain technology, particularly on the Solana network. By dramatically reducing costs and enabling new use cases, compression has the potential to drive mass adoption of blockchain technology and reshape our understanding of digital assets.

As Jon Wong, Noah Prince, and Ilmoi have demonstrated, compression is not just a technical innovation, but a catalyst for reimagining what's possible in the world of blockchain. From enabling millions of NFTs to be minted at minimal cost to facilitating new types of decentralized applications, compression is opening up a world of possibilities that were previously out of reach.

As the technology continues to evolve and mature, we can expect to see even more innovative applications leveraging compression on Solana. Whether it's revolutionizing social media, transforming gaming economies, or enabling new forms of digital ownership, compression is set to play a crucial role in the next phase of blockchain adoption.

The journey of compression technology on Solana is just beginning, and if this discussion is any indication, the future looks incredibly bright. As developers, enterprises, and users continue to explore and push the boundaries of what's possible with compression, we're likely to see a new wave of blockchain innovation that could reshape the digital landscape as we know it.

Facts + Figures

• Compression allows minting of 100 million NFTs for approximately $2,000, a dramatic cost reduction compared to traditional methods.
• Compression leverages a data structure called a concurrent Merkle tree to create a unique fingerprint from massive amounts of off-chain data.
• The hash at the top of the Merkle tree uniquely identifies the entire state of the tree and is stored in an account on Solana.
• All the leaves of the Merkle tree, representing the NFTs, are stored in the Solana ledger.
• Helium successfully migrated its entire state onto Solana in April, using compressed NFTs to represent hotspots and mobile subscribers.
• Tensor was the first marketplace to support compressed NFTs, taking about two months to build the initial implementation.
• Compressed NFT contracts have processed more transactions in 30 days than all of Tensor's existing contracts since the beginning.
• Solana's archival data has already reached multiple hundreds of terabytes.
• Compression enables enterprises to issue millions of digital assets at a feasible cost, opening up new possibilities for large-scale NFT projects.
• The panelists predict that in five years, the average NFT project on Solana will be significantly different from those on EVM-compatible chains, focusing on use cases involving millions or billions of NFTs.
• Compression is not limited to NFTs and can be used for various types of on-chain data storage and management.
• The adoption of compression technology has led to the need for new tools and approaches for indexing and querying compressed data on Solana.

Questions Answered

What is compression in the context of Solana?

Compression is a innovative method of storing data on the Solana blockchain that dramatically reduces costs by orders of magnitude. It leverages a data structure called a concurrent Merkle tree, which creates a unique fingerprint from massive amounts of off-chain data while storing that fingerprint on-chain. This approach allows for significant cost savings and enables the creation and management of millions of NFTs or other digital assets at a fraction of the cost of traditional methods.

How does compression reduce the cost of minting NFTs?

Compression reduces the cost of minting NFTs by storing only a small amount of data on-chain while keeping the bulk of the data off-chain in a verifiable manner. Instead of storing all the NFT data directly in accounts, which requires paying rent in SOL, compression uses a Merkle tree structure where only the root hash is stored on-chain. This allows for the minting of millions of NFTs at a tiny fraction of the cost of traditional methods. For example, with compression, you can mint 100 million NFTs for approximately $2,000, making large-scale NFT projects economically feasible.

Are compressed NFTs as secure as traditional NFTs on Solana?

Yes, compressed NFTs are just as secure as traditional NFTs on Solana. The panelists emphasize that the security of compressed entities is equivalent to the security of Solana itself. The smart contracts used for compression (state compression and bubblegum) are designed to ensure that the creator of the storage box cannot take away NFTs from other people. The combination of on-chain data and the Solana ledger means that all information can be verified and reconstructed, maintaining the same level of security and decentralization that users expect from blockchain technology.

What new use cases does compression enable for NFTs and digital assets?

Compression enables a wide range of new use cases for NFTs and digital assets by dramatically reducing costs. This allows for the creation of digital assets at scale, enabling use cases such as representing real-world assets (like Helium's hotspots), creating digital identifiers for network participants, issuing millions of digital stickers or collectibles, and implementing complex governance systems. It also makes it economically feasible for enterprises to issue large numbers of digital assets, potentially revolutionizing areas like gaming, social media, and customer loyalty programs.

How does compression impact the developer experience on Solana?

Compression introduces both challenges and opportunities for developers on Solana. Initially, working with compression technology was described as "chewing glass" due to limited documentation and the complexity of the new paradigm. However, as the ecosystem has matured, more tools and resources have become available. Compression empowers developers to build applications that were previously impractical due to cost constraints, allowing for more creative and ambitious projects. It also requires developers to think differently about data storage and retrieval, potentially leading to new patterns and best practices in Solana development.

What is the future outlook for compressed NFTs on Solana?

The panelists express great optimism for the future of compressed NFTs on Solana. They predict that in five years, the average NFT project on Solana will look very different from current projects, moving beyond the typical 10,000 PFP collections to more diverse and utilitarian use cases. Compressed NFTs are expected to enable new types of digital assets that represent real-world items, participation in networks, or complex financial instruments. The low cost and high scalability of compressed NFTs are likely to drive innovation and potentially lead to mainstream adoption of blockchain technology in various industries.

How does compression address scalability concerns on Solana?

While compression significantly reduces the cost of minting and managing large numbers of digital assets, it also introduces new challenges related to data storage and retrieval. The Solana ledger is growing rapidly, with hundreds of terabytes of archival data. Compression is part of the solution to scalability, as it allows for more efficient data storage. However, the panelists acknowledge that additional solutions will be needed, such as new incentive structures for archival node operators and innovative approaches to data indexing and querying. The discussion suggests that a combination of on-chain compression and off-chain indexing solutions will likely be necessary to address long-term scalability concerns.

Can compression be used for applications beyond NFTs on Solana?

Yes, compression has potential applications beyond NFTs on Solana. While NFTs are the most common use case currently, the panelists discuss how compression can be used for various types of on-chain data storage and management. This could include applications in decentralized finance (DeFi), social media platforms, gaming economies, and even potentially for improving privacy solutions. The general concept of state compression could be applied to any situation where large amounts of data need to be efficiently stored and verified on-chain, opening up possibilities for a wide range of decentralized applications.