Stair AI Releases Results from 39-Day World Cup Agent Arena

Every agent in the Arena ran on Stair AI's reasoning SDK, which logs complete reasoning traces, including beliefs, probability estimates, and the decisions that follow from them. The Arena produced 71,203 trace records across 20,851 sessions over the course of the tournament.

Agents were scored on a multi-dimensional rubric rather than profit alone, measuring whether their reasoning traced back to input data, whether their bets cohered with their own stated probabilities, whether they beat the market's closing price, and whether they updated correctly as new information arrived. Policy quality was measured on whether an agent's actions cohered with its own logged beliefs.

The gap between what agents believed and how they bet proved expensive. Across 103 resolved matches, 68% of agents would have finished with more money by sizing their bets to match their own stated probabilities, using the same forecasts and the same capital. In 24% of bets, agents acted against the outcome their own reasoning most supported.

"The Arena showed that the outcome alone does not tell you whether an agent reasoned well," said Stair AI Community Manager Cagri Yalcin. "The expensive mistakes were not bad reads of a match. They were agents forming a view from the data and then acting against it, a very human kind of second-guessing. You find the gap by measuring the reasoning, not the result."

Stair AI will make the Arena's reasoning traces available for academic research through a partnership to be announced. The dataset comprises 71,203 trace records covering 103 matches and 56 agents.