How NEAR Is Building Crypto's Interoperability Layer | Illia Polosukhin

NEAR Protocol's Journey from AI to Crypto

NEAR Protocol, co-founded by Illia Polosukhin, has come a long way since its inception in 2017. Originally started as NEAR AI, a company focused on teaching machines to code, the project pivoted to blockchain technology when faced with the challenge of paying international contributors. This shift highlights the real-world problem that cryptocurrencies aim to solve - enabling seamless, borderless transactions without the constraints of traditional banking systems.

Polosukhin, who has a background in machine learning and AI, worked at Google Research on technology that now powers advancements like ChatGPT and Midjourney. This experience in cutting-edge AI development provided a unique perspective when entering the blockchain space. The transition from AI to crypto showcases the versatility and adaptability of the NEAR team, qualities that have contributed to the protocol's success and innovation in the blockchain industry.

NEAR's Unique Approach to Scaling: Dynamic Sharding

One of the key differentiators of NEAR Protocol is its approach to scaling through dynamic sharding. Sharding is a technique borrowed from traditional web-scale companies, where data and processing are divided across multiple machines to improve performance and scalability. NEAR's implementation of sharding in a blockchain context aims to solve the scalability challenges faced by many Layer 1 protocols.

Polosukhin explains the rationale behind NEAR's sharding strategy:

"If you worked in any web2 like scale companies, sharding is something that's just like, it's there. It's like you do everything in a sharded way, right? Like you cannot imagine processing internet, right? And training a model on one computer. You need to do it at, you know, thousands of machines in parallel at scale."

This approach allows NEAR to scale horizontally by adding more machines as needed, rather than relying solely on maximizing the performance of a single machine. The goal is to create a system that can handle billions of users without compromising on decentralization or user experience.

Nightshade: NEAR's Innovative Sharding Design

NEAR's sharding implementation, called Nightshade, takes a unique approach to balancing scalability and user experience. The core idea is to treat each account as an independent "rollup" or blockchain, allowing for parallel processing and storage. These individual accounts are then bundled together on machines that can handle their processing requirements.

Polosukhin describes the system:

"So let's say on this machine, you have a million accounts on this machine, you have a million accounts. And then on another machine, you have one account because it's actually big. It's very popular application. It's been used by a million people. Right. So you kind of have this mapping of the account space to machines and that's your actual like physical shards."

From a user perspective, this complex infrastructure is abstracted away, presenting a single blockchain that produces a block every second. This design allows NEAR to maintain low latency and high throughput while preserving a unified user experience.

Stateless Validation: The Next Evolution of NEAR's Sharding

NEAR is currently working on the next phase of its sharding implementation, which includes stateless validation. This upgrade, launched on an incentivized testnet, aims to further improve scalability and performance.

The key innovation of stateless validation is that block producers only need to store the entire state in memory, rather than on hard drives. This approach significantly reduces the bottleneck caused by state updates on hard drives. Polosukhin explains:

"Because we have sharding, we can limit the amount of size of state on each independent shard, and we just add more machines with kind of when it grows. So we can keep the whole state in memory, which obviously is way more performant."

This design allows for thousands of validators to receive and validate blocks without needing to have the entire state available to them. The new system is expected to be deployed on NEAR's mainnet in May, completing a significant portion of the sharding roadmap outlined in 2019.

NEAR's Flexibility and Continuous Improvement

One of the strengths of NEAR's architecture is its flexibility and ability to evolve without disrupting user experience. Polosukhin highlights that the protocol has undergone 48 upgrades without interruptions for users, simply by validators upgrading their software. This adaptability allows NEAR to continuously improve its performance and functionality while maintaining a stable environment for developers and users.

The sharding design also allows for the potential implementation of different virtual machines (VMs) on individual shards, providing even more flexibility for future developments and specific use cases.

Cross-Chain Compatibility: Aurora and EVM Support

NEAR's architecture allows for the implementation of other blockchain environments as smart contracts within its ecosystem. A prime example of this is Aurora, an Ethereum Virtual Machine (EVM) implementation running as a smart contract on NEAR. Aurora's success led to it being allocated a separate shard to handle its high traffic without affecting other parts of the network.

This approach to cross-chain compatibility demonstrates NEAR's commitment to interoperability and its ability to support diverse blockchain ecosystems within its framework. Polosukhin notes:

"You can launch actually has to be just you know take that dress code, compile it and launch it as as a smart contract on near. Obviously it will not be as performant because not running natively, but you will get ability to run all the contracts inside it."

This flexibility extends beyond Ethereum, with Polosukhin suggesting that even a Bitcoin-like environment could be implemented on NEAR, showcasing the protocol's potential as a multi-chain platform.

NEAR's Vision for Chain Abstraction

As the blockchain space continues to evolve with the proliferation of Layer 2 solutions and application-specific chains, NEAR is positioning itself as a unifying force through chain abstraction. The goal is to hide the complexity of underlying blockchain infrastructures from both users and developers, creating a seamless experience across multiple chains.

Polosukhin describes this vision:

"The overall vision is like we want to make it easy to be for people to use you know that's really self like have self sovereignty. And you know the blockchain is important part of that you know roll ups now offer kind of you know additional benefits and specific use cases."

This approach aims to solve the fragmentation issue in the current blockchain landscape, where users need to manage multiple wallets and navigate different blockchain environments. NEAR's solution involves creating a unified experience where users can interact with various blockchain applications without needing to understand the underlying infrastructure.

The Blockchain Operating System Concept

NEAR is developing what they call a "blockchain operating system" to realize their vision of chain abstraction. This system aims to provide a holistic experience for using Web3 applications without requiring users to think about blockchains, wallets, or complex addresses.

Key components of this blockchain operating system include:

1. Decentralized front-ends: NEAR has acquired technology to enable decentralized front-end experiences.
2. Multi-chain support: The system will combine multiple networks into a single user experience.
3. Account aggregation: NEAR accounts will be able to transact on various blockchains, including Bitcoin, Solana, and Ethereum-based networks.

Polosukhin emphasizes the importance of this approach:

"Now the cool thing is what this does is unifying security. So before we were like, you know, there's arguments on Twitter like, Oh, your chain doesn't have enough, you know, security, the stake is too low, whatever whatever, maybe validators, you know, too centralized to the essential, like, not decentralized enough. Like all of that is starting to go away because now like, Hey, here's a proof."

This unified security model aims to address concerns about the security and decentralization of individual chains by creating a cross-chain security framework.

NEAR's Data Availability Service

NEAR has positioned itself as a competitive player in the data availability (DA) space, offering a service that is reportedly more cost-effective and scalable than alternatives like Celestia. Polosukhin explains that NEAR's data availability solution is built into the consensus layer, making it an integral part of the protocol's architecture.

Key features of NEAR's data availability service include:

• Cost-effectiveness: Claimed to be 30 times cheaper than Celestia and 85,000 times cheaper than Ethereum for data availability.
• Scalability: Currently offers 16 megabytes per second of data availability, with the potential to increase as more shards are added.
• Proven reliability: The system has been live for three years without downtime.
• Integration with rollup technologies: NEAR has integrated with various rollup frameworks, including Arbitrum CDK and StarkWare stacks.

This data availability service is part of NEAR's strategy to become a comprehensive infrastructure provider for the broader blockchain ecosystem, supporting the development of Layer 2 solutions and application-specific chains.

Addressing the Challenges of Multi-Chain Development

NEAR recognizes the challenges developers face when building for multiple blockchain ecosystems. Polosukhin argues that limiting an application to a single chain restricts its potential audience. He states:

"You're you're choosing a specific ecosystem is actually limiting your target audience. And I mean, we've heard this right. Like many times that we see like, oh, you should build on that chain, you should build on that chain is like, like that logic is in general is broken because you should be building for all audience, like it should not be for a chain audience should be for every audience."

To address this, NEAR is developing tools and frameworks that allow developers to create applications that can seamlessly operate across multiple chains. This approach aims to expand the reach of decentralized applications and provide a more inclusive ecosystem for both developers and users.

The AI and Crypto Intersection

Given Polosukhin's background in AI, including his work on technology that powers modern language models, the discussion touched on the potential intersection of AI and blockchain technologies. Polosukhin identified several areas where these technologies could complement each other:

1. Cryptographic authentication for content: As AI-generated content becomes more prevalent, there's a need for a cryptographic layer to verify the authenticity and origin of information.

2. User-owned AI models: Polosukhin advocates for open-source AI models that can run locally on users' devices, ensuring privacy and reducing the risk of manipulation.

3. Provable AI computations: For larger models or those containing sensitive data, blockchain technology could be used to prove the results of AI computations without revealing the underlying data or model.

Polosukhin emphasizes the importance of these developments:

"So what we need is cryptographic reputation cryptographic authentication reputation layer under all of this so similarly how we had a transition from HTTP to HTTPS when Internet started."

This vision of a more secure and verifiable digital landscape highlights the potential for blockchain technology to address some of the challenges posed by advancing AI capabilities.

NEAR's Approach to Narratives and Market Positioning

Despite NEAR's technological advancements and growing user base, Polosukhin acknowledges that the protocol has faced challenges in maintaining a strong narrative in the crypto space. He attributes this partly to NEAR's focus on real-world adoption and working with projects that may not be as visible in the traditional crypto sphere.

Polosukhin explains:

"I think part of this is also maybe some things we were not doing very well. Now we did kind of you know adjusting cellular space I came back as a CEO for the near foundation in November and so part of this is also right now kind of building out this chain abstraction narrative right that's not just near but also the whole ecosystem and web space is moving toward."

The renewed focus on chain abstraction and interoperability aims to position NEAR as a key player in the evolving blockchain landscape. By emphasizing these aspects, NEAR is working to create a narrative that aligns with its technological capabilities and vision for the future of the industry.

Value Accrual in the NEAR Ecosystem

When discussing how value accrues to the NEAR token and ecosystem, Polosukhin outlined several components:

1. Governance: A portion of rewards (inflation) is allocated to a treasury for developing public goods and continuing protocol development.
2. Network security: The token is used for staking and securing the network.
3. Utility: NEAR tokens are used for transaction fees and storage costs.
4. Financial applications: The token can be used as collateral and utility in other protocols within the ecosystem.
5. Brand value: As the ecosystem grows, the recognition and brand value of NEAR increases.

Polosukhin emphasizes the importance of considering different timeframes when evaluating these components:

"I mean that's kind of a high level framework and then again depending on the timeframe you can kind of think of like well over time, for example, you know this governance part becomes really valuable because it directs and defines what this existing evolving."

This multi-faceted approach to value accrual aims to create a sustainable and growing ecosystem that benefits both short-term participants and long-term stakeholders.

Challenges and Future Directions for Crypto

Polosukhin identified several key challenges that the crypto industry needs to address to achieve mainstream adoption:

1. Security: Improving the security of user funds and addressing the frequent occurrence of hacks and thefts.
2. Image problem: Balancing the benefits of speculation for creating momentum with the need for sustainable growth and development.
3. Expanding use cases: Developing more real-world applications that connect blockchain technology to everyday life.

On security, Polosukhin states:

"I think the biggest biggest problem is security. I think the, like for a space that kind of town like talent security as like the main thing. And you know we like have this balance on Twitter whatever was decentralized and more secure and whatnot. Like we do very little to actually secure people's funds as a space and like that seemed like the most important thing in this whole in this whole endeavor."

Addressing these challenges will be crucial for the continued growth and adoption of blockchain technology across various sectors.

Conclusion: NEAR's Role in Shaping the Future of Blockchain

NEAR Protocol, under the leadership of Illia Polosukhin and the NEAR team, is positioning itself as a key player in the evolving blockchain landscape. By focusing on scalability through dynamic sharding, interoperability through chain abstraction, and user experience through a unified blockchain operating system, NEAR is addressing some of the most pressing challenges in the crypto industry.

The protocol's approach to balancing technological innovation with practical usability showcases a mature understanding of what's needed to bring blockchain technology to mainstream adoption. As the industry continues to evolve, NEAR's contributions to solving issues of scalability, interoperability, and user experience will likely play a significant role in shaping the future of decentralized technologies.

With ongoing developments in areas such as stateless validation, data availability services, and multi-chain support, NEAR is well-positioned to continue driving innovation in the blockchain space. As the ecosystem grows and matures, it will be interesting to see how NEAR's vision of a seamless, interconnected blockchain landscape unfolds and potentially reshapes the way we interact with decentralized technologies.

Facts + Figures

• NEAR Protocol originated from NEAR AI, a company focused on teaching machines to code, founded in 2017.
• Illia Polosukhin, co-founder of NEAR, worked at Google Research on technology now powering advancements like ChatGPT and Midjourney.
• NEAR's sharding implementation, called Nightshade, treats each account as an independent "rollup" or blockchain.
• NEAR has undergone 48 protocol upgrades without interruptions for users.
• Aurora, an Ethereum Virtual Machine (EVM) implementation, runs as a smart contract on NEAR and has been allocated a separate shard.
• NEAR's data availability service is claimed to be 30 times cheaper than Celestia and 85,000 times cheaper than Ethereum.
• NEAR currently offers 16 megabytes per second of data availability.
• The NEAR ecosystem has over 15 million monthly active users and 1.5-1.8 million daily active users.
• NEAR's blockchain operating system aims to combine 13 different networks in one user experience.
• NEAR's account aggregation feature will allow transactions on various blockchains, including Bitcoin, Solana, and Ethereum-based networks.
• NEAR has integrated with rollup frameworks including Arbitrum CDK and StarkWare stacks.
• The new stateless validation design is expected to be deployed on NEAR's mainnet in May.
• NEAR is working on implementing a prover for its blockchain to be compatible with Polygon's aggregation network.
• Catching app, a cosmos app on NEAR, has reached 20+ million monthly active users.

Questions Answered

What is NEAR Protocol's approach to scaling?

NEAR Protocol uses a dynamic sharding approach called Nightshade to scale its blockchain. This system treats each account as an independent "rollup" or blockchain, allowing for parallel processing and storage. Accounts are bundled together on machines based on their processing requirements, enabling horizontal scaling by adding more machines as needed. From a user perspective, this complex infrastructure appears as a single blockchain producing blocks every second.

How does NEAR's stateless validation work?

NEAR's stateless validation is a new upgrade to its sharding implementation. In this system, block producers only need to store the entire state in memory, rather than on hard drives. This approach significantly reduces the bottleneck caused by state updates on hard drives. Validators can receive and validate blocks without needing the entire state available to them, allowing for thousands of validators to participate in the network efficiently. This design is expected to improve scalability and performance when deployed on the mainnet.

What is NEAR's vision for chain abstraction?

NEAR's vision for chain abstraction aims to hide the complexity of underlying blockchain infrastructures from both users and developers. The goal is to create a seamless experience across multiple chains, solving the fragmentation issue in the current blockchain landscape. NEAR is developing a "blockchain operating system" that provides a unified interface for interacting with various blockchain applications without requiring users to understand the underlying infrastructure. This includes features like decentralized front-ends, multi-chain support, and account aggregation across different blockchains.

How does NEAR's data availability service compare to competitors?

NEAR's data availability service is reportedly more cost-effective and scalable than alternatives like Celestia. It is claimed to be 30 times cheaper than Celestia and 85,000 times cheaper than Ethereum for data availability. The service currently offers 16 megabytes per second of data availability, with the potential to increase as more shards are added. It has been live for three years without downtime and is integrated with various rollup technologies, including Arbitrum CDK and StarkWare stacks.

What are the main challenges NEAR identifies for the crypto industry?

Illia Polosukhin identifies three main challenges for the crypto industry: 1) Security - improving the protection of user funds and addressing frequent hacks and thefts. 2) Image problem - balancing the benefits of speculation for creating momentum with the need for sustainable growth and development. 3) Expanding use cases - developing more real-world applications that connect blockchain technology to everyday life. Addressing these challenges is crucial for the continued growth and adoption of blockchain technology across various sectors.

How does NEAR plan to improve interoperability between different blockchains?

NEAR plans to improve interoperability through its chain abstraction approach and blockchain operating system. This includes developing account aggregation features that allow NEAR accounts to transact on various blockchains, including Bitcoin, Solana, and Ethereum-based networks. NEAR is also working on implementing a prover for its blockchain to be compatible with Polygon's aggregation network, which will help unify security across different chains. The goal is to create a seamless experience where users can interact with applications across multiple blockchains without needing to understand the underlying infrastructure.

What is NEAR's perspective on the intersection of AI and blockchain?

NEAR sees several potential intersections between AI and blockchain technologies. These include: 1) Developing a cryptographic authentication layer for verifying the authenticity and origin of AI-generated content. 2) Promoting user-owned AI models that can run locally on devices, ensuring privacy and reducing manipulation risks. 3) Creating systems for provable AI computations, allowing verification of results from large or sensitive models without revealing underlying data. NEAR believes blockchain technology can address some of the challenges posed by advancing AI capabilities, particularly in areas of content verification and secure, decentralized AI processing.