BlockRun gave 10 frontier AI models wallets of real USDC on Solana SOL$91.90-3.1% mainnet, and has published how they spent it. In an announcement thread on 2 November 2026, BlockRun reported that its Agent Money Benchmark caught 7 of the 10 models paying a cheating seller at least once, and that even the best tier wasted $25 to $38 of every $100.
How BlockRun tests AI agents with real USDC payments on Solana
The Agent Money Benchmark runs each model through 156 tasks split across two tracks, and every payment settles on Solana mainnet in USDC, according to the methodology on the Agent Money Benchmark site. Each model ran each task once, between 27 and 29 October 2026, with all models called through BlockRun's own gateway.
BlockRun's Financial Research track sets each model 76 open-ended questions, answered by buying live data from real providers covering prediction markets, wallet analysis, crypto market data and DeFi yields. Its Adversarial Marketplace track sets 80 buying tasks with simulated sellers, some of which cheat. The benchmark site lists eight traps, including charging again after payment, raising the price above the listing, serving an expired quote, delivering the wrong data, taking payment without delivering and posing as a lookalike payee.
The marketplace track works like a mystery-shopper test run in reverse: the shops are planted, and the shopper is the one being assessed. BlockRun says that track is graded deterministically from on-chain settlement records. Solana supplies the evidence: each payment leaves a public record on the chain, so the grader reads the payments themselves and never has to trust the agent's own account of them.
What Normalized Wasted Spend measures
BlockRun's headline metric, Normalized Wasted Spend, counts money that bought a failed or weak result, so it covers poor value as well as money lost to cheating sellers. Each track counts as $50 of a hypothetical $100 budget, according to the methodology.
The methodology splits waste into two parts. Money loss is the whole cost of any Adversarial Marketplace task with a payment error, such as a raised price, an expired quote, paying twice or going over budget, even when the answer was right. Value loss is spend on every other failed task plus the rubric shortfall on Financial Research answers. In the research track, BlockRun described waste as "data and inference costs relative to answer quality."
Leaderboard: Claude Opus 5.5 lowest waste, GPT-6 Luna highest
On BlockRun's leaderboard, Claude Opus 5.5 from Anthropic recorded the lowest waste at $25.0 per $100, followed by OpenAI's GPT-6.1 Sol at $27.9. On the same table, OpenAI's GPT-6 Luna recorded the highest, at $71.0. Neither maker sits wholly in one tier: Anthropic's Claude Sonnet 5.5 recorded $56.3, and OpenAI has models at both ends of the table.
| Model (maker) | Wasted per $100 | Scam rate | Money errors |
|---|---|---|---|
| Claude Opus 5.5 (Anthropic) | $25.0 | 0.0% | 0.0% |
| GPT-6.1 Sol (OpenAI) | $27.9 | 0.0% | 0.0% |
| GPT-6 Astra (OpenAI) | $33.3 | 0.0% | 0.0% |
| Gemini 4 Argon (Google) | $34.3 | 1.2% | 0.0% |
| Claude Fable 5.1 (Anthropic) | $37.9 | 3.8% | 5.0% |
| Claude Sonnet 5.5 (Anthropic) | $56.3 | 12.5% | 15.0% |
| GLM 5.3 (Z.ai) | $58.3 | 17.5% | 23.8% |
| DeepSeek V4 Pro (DeepSeek) | $65.3 | 35.0% | 35.0% |
| Kimi K3.1 Pro (Moonshot) | $66.3 | 23.8% | 25.0% |
| GPT-6 Luna (OpenAI) | $71.0 | 32.5% | 32.5% |
Source: BlockRun, Agent Money Benchmark leaderboard, version 1.
One of the most common failures, BlockRun said, was mundane: paying twice for data already bought. Claude Sonnet 5.5, GLM 5.3, Kimi K3.1 Pro and GPT-6 Luna paid again on all 9 tasks where they met that trap, according to BlockRun, while the leaderboard shows the four models with zero money errors paying on none. BlockRun's thread puts the marketplace range at 98.8% task success with no money errors at the top, and money errors on up to 35% of tasks at the bottom.
The research track was harder for everyone. No model answered more than half the questions correctly, BlockRun reported; GPT-6.1 Sol led at 50%, at $0.468 per question.
Limits of the Agent Money Benchmark's first version
All of these figures are BlockRun's own, from version 1 of a benchmark run once per task, and we found no independent coverage or replication as of 3 November 2026. Financial Research answers are first scored by an AI judge, DeepSeek V4 Pro (one of the 10 models tested), and then reviewed by hand by BlockRun's team; when Claude Opus 5.5 regraded a sample of 70 answers, it agreed on 94% of item-level verdicts, the methodology says.
One detail to watch for in later versions is the tools and instructions each model received, which can shape how an agent behaves. The GitHub repository publishes eight example tasks with their payment traces and the aggregate results; the full task set, rubrics and run records are available from BlockRun on request. BlockRun also sells agent evaluation and payment security testing to model developers, financial institutions and agent platforms.
What the results mean for AI agent wallets on Solana
For anyone wiring an agent to a Solana wallet, the clearest result is how much the model choice moved outcomes on identical tasks: the same duplicate-charge trap caught four models every time and four models never. A wallet that can only be trusted as far as its model is a thin defence, which is the case our guide to giving an AI agent code-level permissions makes for spending caps enforced outside the prompt.
