Breakpoint 2024: Product Keynote: FrodoBots (Michael Cho)

FrodoBots is set to revolutionize the gaming industry with its groundbreaking fusion of robotics, AI, and real-world gameplay. At Solana's Breakpoint 2024, co-founder Michael Cho unveiled Earth Rovers, a game that allows players to control physical robots across the globe, collect NFTs, and contribute to cutting-edge AI research.

Summary

FrodoBots, a deep tech project at the intersection of robotics and gaming, has introduced Earth Rovers, a browser-based game that allows players to control real robots in various cities worldwide. The game combines elements of RC car control with real-world navigation missions, enabling players to interact with their environment through a 4G-connected robot.

The company has been testing this concept for two years in over 20 cities, with nearly 100 robots deployed. The game has gained traction, with some players purchasing game passes to access premium features. FrodoBots has also recently launched a scavenger hunt gameplay mode, where players can buy their own robot for $300 and collect NFTs in their neighborhood.

Beyond gaming, FrodoBots is making significant strides in AI research. They have open-sourced a dataset of 2,000 hours of real-world driving data, which is currently the largest robotic urban navigation dataset globally. The company is collaborating with UC Berkeley on an academic paper and organizing the Earth Rover Challenge, an e-sports style competition pitting human gamers against AI teams from top institutions.

FrodoBots' unique position in providing both extensive real-world data and a platform for model evaluation has attracted partnerships with leading robotic AI researchers. This combination of engaging gameplay and valuable research data positions FrodoBots at the forefront of embodied AI advancement.

Key Points:

Earth Rovers: A New Frontier in Gaming

Earth Rovers represents a revolutionary approach to gaming, blending the virtual and physical worlds in ways never before seen. Players can control real robots from anywhere in the world, navigating them through actual city streets and environments. This browser-based game requires no downloads, making it accessible to a wide audience.

The game's unique selling point is its use of 4G-connected robots that can be controlled remotely. This allows for an unprecedented level of immersion, as players can see and hear the real-world environment through the robot's cameras and microphones. The ability to interact with people around the robot adds a social dimension to the gameplay, creating a truly unique gaming experience.

Scavenger Hunt Mode and NFT Integration

FrodoBots has recently introduced a scavenger hunt gameplay mode, further enhancing the Earth Rovers experience. This new mode allows players to purchase their own robot for $300, which they can then use to explore their local area and collect NFTs. This Pokemon Go-style gameplay adds an exciting AR element to the game, encouraging players to engage with their real-world surroundings.

The NFT integration provides an additional layer of excitement and potential value for players. While some NFTs may be common (humorously referred to as "poop" in the game), others are rare and potentially valuable. This creates a collecting aspect to the game that can drive engagement and potentially provide financial incentives for dedicated players.

Contributions to AI Research

Beyond its gaming applications, FrodoBots is making significant contributions to the field of AI research. The company has open-sourced a dataset of 2,000 hours of real-world driving data, collected from a dozen cities. This two-terabyte dataset is currently the largest robotic urban navigation dataset in the world, providing invaluable information for researchers working on autonomous navigation systems.

FrodoBots is collaborating with an AI lab from UC Berkeley to publish an academic paper based on an even larger dataset of five terabytes. This partnership highlights the scientific value of the data being collected through the Earth Rovers project and demonstrates FrodoBots' commitment to advancing the field of robotics and AI.

The Earth Rover Challenge

One of the most exciting announcements from the presentation is the upcoming Earth Rover Challenge, set to take place in Abu Dhabi. This e-sports style competition will pit five top human gamers against five AI teams from prestigious institutions like Carnegie Mellon and UC Berkeley. The event will be livestreamed, showcasing the current capabilities of AI in real-world navigation tasks.

This challenge serves multiple purposes. It provides an engaging spectacle for viewers, demonstrating the capabilities of both human players and AI systems. For researchers, it offers a unique opportunity to evaluate their models in real-world conditions across multiple cities. This kind of large-scale, real-world testing is typically unavailable even to the most well-funded research labs, making FrodoBots' platform incredibly valuable to the scientific community.

Facts + Figures

• FrodoBots has deployed nearly 100 robots in over 20 cities worldwide
• The Earth Rovers game is browser-based and requires no downloads
• Players can purchase their own robot for $300 to participate in the scavenger hunt mode
• FrodoBots has open-sourced a 2,000-hour, 2-terabyte dataset of real-world driving data
• The company is working on a 5-terabyte dataset with UC Berkeley
• The Earth Rover Challenge will feature 5 human gamers vs 5 AI teams
• Top gaming guild YGG is partnering with FrodoBots to select the best human players
• Robotic labs from institutions like Carnegie Mellon and UC Berkeley are participating in the AI side of the challenge
• The Earth Rover Challenge will be livestreamed as an e-sports event
• Exhibit is providing compute support for the AI teams, including access to H100 GPUs

Top quotes

1. "Our goal is to advance robotics AI or basically embody AI by cross sourcing hopefully the world's largest real-world data sets with robotic gaming."
2. "You could be a gamer in Singapore and get control of the robot halfway across the world in New York while taking on real-world navigation missions."
3. "We think there's a lot of very interesting AR element that we want to continue to build in the coming months."
4. "This is already the largest robotic urban navigation data set in the world."
5. "The whole thing will be live stream as it is an e-sports event."
6. "We provide them with two things that is kind of game-changing for them. Number one, of course, the data. But more recently, we realized that this concept of evaluating the models in the real world, that's actually something that they never get to do."

Questions Answered

What is Earth Rovers?

Earth Rovers is a browser-based game developed by FrodoBots that allows players to control real robots in cities around the world. It combines elements of RC car control with real-world navigation missions, enabling players to interact with their environment through a 4G-connected robot. The game requires no downloads and includes features like voice communication, allowing players to talk to people near the robot.

How does the scavenger hunt mode work in Earth Rovers?

The scavenger hunt mode in Earth Rovers allows players to purchase their own robot for $300 and use it to collect NFTs in their local neighborhood. Similar to Pokemon Go, players can see loot box locations on a map and drive their robot towards them. Some NFTs are common (jokingly referred to as "poop"), while others are rare and potentially valuable, adding a collecting aspect to the gameplay.

How is FrodoBots contributing to AI research?

FrodoBots is contributing to AI research in several ways. They have open-sourced a 2,000-hour, 2-terabyte dataset of real-world driving data from a dozen cities, which is currently the largest robotic urban navigation dataset in the world. They are also collaborating with UC Berkeley on a 5-terabyte dataset and academic paper. Additionally, FrodoBots provides researchers with a unique platform to evaluate their AI models in real-world conditions across multiple cities.

What is the Earth Rover Challenge?

The Earth Rover Challenge is an e-sports style competition organized by FrodoBots, set to take place in Abu Dhabi. It will pit five top human gamers against five AI teams from prestigious institutions like Carnegie Mellon and UC Berkeley. The event will be livestreamed and showcases the current capabilities of AI in real-world navigation tasks. This challenge provides a unique opportunity for researchers to evaluate their models in real-world conditions across multiple cities.

Why are researchers interested in partnering with FrodoBots?

Researchers are interested in partnering with FrodoBots because the company provides two crucial elements: extensive real-world data and a platform for model evaluation. The data collected through Earth Rovers is invaluable for training and improving AI models for urban navigation. Moreover, FrodoBots' network of robots in over 20 cities allows researchers to evaluate their models in diverse real-world conditions, something that even well-funded labs typically don't have access to.