Introducing the
Robot Intelligence Index

How it was built

Benchmarks. We included public robot leaderboards that publish a score for every model and were active in 2026. That covers benchmarks whose own teams run or verify each entry, such as RoboDojo, RoboChallenge, RoboArena, RoboLab, AgiBot's RoboColiseum, Positronic's PhAIL, the RoboColosseum team's real-world sites, Robocurve's RoboDojo-RC and DataCenterBench, Artificial Analysis's AA-Robotics, Armnet's ArmnetBench and UT Dallas's VLA-Replica, and the benchmarks most often reported in recent model papers, such as LIBERO, SimplerEnv and CALVIN, which are self-reported. A benchmark feeds the ranking only when some of its models also appear on another benchmark. Five more sources were collected but not used. The benchmark table explains each of those exclusion decisions.

Coding agents. AA-Robotics tests only general coding agents, which write code to drive a real arm with no robot training. RoboDojo-RC tests the same kind of agent alongside robot models, and Robocurve's DataCenterBench tests agents on ten steps from data-center manuals. All three enter as real-robot benchmarks. RoboColosseum's open track also runs coding agents through Robocurve's harness, on the same real robots as robot models. AA-Robotics and DataCenterBench carry less weight because they score one to three attempts per task. RoboColiseum also tests coding agents next to robot models, in simulation.

Who gets ranked. By default the overall ranking includes a model only if it has at least one real-robot result and one simulation result. 36 models meet this threshold. The coverage menu also shows any two benchmarks (61 models) or every model (83). The simulation ranking requires two simulated benchmarks for a model to be ranked, but the real-robot ranking shows every model with a real result.

Ratings. Each benchmark is turned into head-to-head comparisons between the models it tested, and one rating per model is fitted to all of them on a chess-style Elo ranking where every model starts at 1500. The methodology walks through it step by step.

Task types

Beyond the overall rating, the index rates models on four types of task. We sorted each benchmark's own task and category scores into these types by hand, assigning each task to the type that best describes its main difficulty. Each type is then rated from those scores alone. A model is ranked in a task type only if it meets the overall rule and has at least two benchmarks in that type.

Benchmarks that report no breakdown count only toward the overall views. RoboArena counts toward pick and place because pick and place tasks dominate its task breakdown.

Where benchmarks disagree

The same model often finishes near the top of one benchmark and near the bottom of another. Of the 23 models that finish top three somewhere, 13 finish in the bottom half on another benchmark.

Two of the largest swings, DM0 and Spirit v1.5, involve models that were prepared very differently on two real-robot benchmarks. On RoboChallenge, their developers entered them as a separate model for each task (as declared on the board), with up to about 1,000 demonstrations per task available for training. On RoboDojo, the benchmark team fine-tuned both itself on 100 demonstrations per task. Dexmal's own single multi-task version of DM0 scores 49 on RoboChallenge, 9th of 22, still far above its RoboDojo finish.

π0.5 shows the largest swing. It leads both SO-101 benchmarks, ArmnetBench and VLA-Replica, where every policy is fine-tuned on the same 50 demonstrations per task. It also leads RoboColosseum's fine-tune track, where every policy learns each task from the same 100 demonstrations. It finishes last of six on PhAIL's DROID tasks.

Several models that lead easy simulated benchmarks, such as OpenVLA-OFT and MolmoAct2 on LIBERO variants, fall near the bottom of harder ones. The benchmarks agree much more at the bottom of the field, where older models rank low everywhere.

Methodology

Each model gets one rating, built from head-to-head comparisons inside every benchmark it appears on. Every model starts at a score of 1500, and the gap between two ratings predicts how often one outscores the other.

1. Collect each benchmark's scores

We pulled every benchmark's public results as of November 5, 2026, and took one headline score per model from each, listed in the table at the end of this section. We matched the same model across benchmarks by hand, using name and version only. Under this framework, a model fine-tuned separately for two benchmarks still counts as one model. We discuss this choice in the limitations section.

2. Turn each benchmark into head-to-head matchups

Within a benchmark, every pair of models it tested is one matchup. Instead of a plain win or loss, each matchup records how likely it is that one model really scored higher, given how noisy that benchmark is.

q_AB is the probability assigned to A outscoring B, s_A and s_B are their benchmark scores, and σ_A and σ_B are the assumed standard deviations of their score noise. Φ is the standard normal cumulative distribution function. Noise describes how much a score could move if the benchmark were run again. On RoboDojo's real robots, π0.5 scores 22.9 and X-VLA 7.6, each with a noise of 2.5 points. The gap is about four times the combined noise, so π0.5 counts as better with near certainty. GR00T N1.7 at 5.9 against π0 at 5.8 counts as 0.51, almost a tie.

Treating the two scores' noise as independent ignores that models tested on the same tasks tend to succeed and fail together. On a shared task set that usually overstates the noise in their difference, which makes matchups softer rather than more decisive, though other patterns of dependence can work the other way. Most benchmarks' noise levels also include a fixed floor of 1 to 3 points, so a tiny gap between two precisely measured scores does not become a near-certain win.

3. Weight each benchmark

In our index, a benchmark’s weight is how independent its evaluators are multiplied by how repeatable its scores are. We judge and score independence by rubric, and we take repeatability from published repeat runs where they exist and from attempts per task where they do not. Each matchup is scaled by the size of its benchmark to ensure a model’s result counts for roughly the same amount regardless of how many models are included in a benchmark's leaderboard. Benchmarks that report several separate tasks, such as MolmoSpaces, split their weight across those tasks.

We made decisions about evaluator independence and repeatability scores using a fixed rubric. The evaluator factor is 1.0 when the benchmark's own team runs or verifies every entry, 0.9 for RoboArena's blind comparisons by volunteer labs, 0.8 when the operator runs only part of the evaluation, results mix operator runs and submissions or the operator also competes, 0.7 for results posted in a benchmark's README and 0.4 for numbers self-reported in papers.

The repeatability factor comes from published reliability measurements for RoboChallenge (0.70, its repeat-run correlation) and RoboArena (0.90, its split-half agreement). For the other benchmarks it follows from how many attempts each runs per task, from 0.95 for the largest evaluations down to 0.60 for AA-Robotics's single attempt per task. Because these judgments were subjective, we also tested robustness to several different weighting schemes, and found broad agreement between them. Section 9 reports those results in detail.

4. Fit one rating per model

Our model uses two ratings to predict a matchup's outcome:

p_AB is the predicted probability that A outscores B, and R_A and R_B are their fitted ratings.

We choose the set of ratings whose predictions best match all the weighted matchup outcomes at once. This is a weighted, regularized Bradley–Terry model fitted to the probabilistic matchups from step 2. Fitting Bradley–Terry ratings to results pooled across many benchmarks has precedent in machine learning, from comparing algorithms across data sets to ranking agents across Kaggle competitions.

Models that never met are placed through the models they share, with shared models acting as bridges between benchmarks. A small pull toward 1500 keeps a model with little evidence from landing at an extreme. We fit the overall, simulation, real-robot and four task-type rankings separately, so a rating only compares models within one setting.

5. Read a rating

The table above summarizes our fitted model's predictions. We checked these predictions against unseen results, refitting the ratings 22 times, leaving out one benchmark each time, and predicting the order of every pair of models that benchmark tested, with 1,810 pairs in all. We find the fitted model's predictions strongly correlate with real outcomes. When the model gave the higher-rated model a 90 to 100% chance, that model outscored the other 97% of the time. In the 70 to 80% range it did so 79% of the time, meaning predictions in this band slightly understate the winning model's relative strength. These checks only cover benchmarks like the ones in the index. Relative performance may vary on tasks that are out of distribution.

6. Measure uncertainty

We re-ran the whole fit 300 times, with re-runs drawing the benchmarks at random with replacement (meaning some count twice and some drop out), and nudging every score by its noise. A model's rank range is where its rank falls in 90% of those re-runs. The leaderboard sorts by rating. Sorting by worst plausible rank instead uses the bottom of that range, which penalizes models with uncertain rankings. A model's range comes from the re-runs that include its benchmarks.

A model with only one benchmark in a leaderboard shows no range, since no re-run can test it against a second. When a re-run drops a benchmark that links two groups of models, the groups are placed relative to each other only by the pull toward 1500. That happens in under 1% of re-runs overall, 3% in simulation and 9% of real-robot re-runs. Eleven real-robot benchmarks are in the final set. RoboDojo-RC and RoboColosseum's open track test coding agents and robot models on the same robots, and seven of RoboColosseum's models also appear on other benchmarks in the index. Real-robot rank ranges are therefore less reliable than the others.

7. Measure disagreement

For each benchmark in a view, we place the model by percentile among the models we matched there, from 100 for first to 0 for last. We calculate spread as the standard deviation of those percentiles. DM0 finishes in the 90th percentile on RoboChallenge and near the bottom of both RoboDojo tracks, for a spread of 50. GPT-6 Astra finishes at or above the middle of all eight of its benchmarks, for a spread of 12. A model with one benchmark in the view has no spread. Because percentiles depend on which other models each benchmark tested, spread reflects differences in the level of competition as well as disagreement between benchmarks.

8. Decide who gets a rank

Every model's comparisons inform the full set of ratings, but only some models receive a rank. By default, the overall ranking requires at least one real-robot and one simulation result, the simulation ranking requires two simulated benchmarks, the real-robot ranking includes every model with a real result and the task types follow the overall rule while also requiring two benchmarks within the task type.

9. Check robustness

We refit the overall ranking under alternatives to our input values for weights, noise and prior. The table compares every alternative with the published top five, GPT-6 Astra, FLUX 3 Action, GPT-6.1 Sol, InternW0-Δ and Gemini 4 Argon.

GPT-6 Astra's first place holds in most alternatives. FLUX 3 Action takes first when noise is doubled, when the pull toward 1500 is weaker and in some re-weighting draws, since its rating rests on two benchmarks. Places two to nine depend more on these choices. RoboDojo's real-robot track gives InternW0-Δ, MWA-Phoenix and OpenWAM-α their only real-robot result, and RoboColiseum gives GPT-6.1 Sol and Dream-1 their only simulation result. PhAIL gives FLUX 3 Action and Cosmos3-Nano-Policy their only real-robot result.

Benchmark weights and noise

Weight is evaluator independence (1.0 when the benchmark's team runs or verifies every entry, 0.4 for paper-reported numbers) multiplied by repeatability (0.60 for AA-Robotics, with one attempt per task, and 0.70 for RoboChallenge, whose repeat runs correlate at 0.70, up to 0.95). RoboColiseum's evaluator weight is 0.8 because AgiBot runs it and also enters its own model, and its repeatability is 0.85 because it does not publish episodes per task. Giving every benchmark equal weight leaves the overall top five unchanged, with rank correlation 0.99 overall and 0.94 or higher in every other leaderboard. Section 9 reports wider changes to the weights.

Setup

Acknowledgments

This index exists because benchmark teams build robot evaluations, run them with care and publish the results openly. Testing models on real robots is slow and expensive, and every number on this page started with their work. We are grateful to the teams behind each benchmark:

• The RoboDojo team, for the RoboDojo simulation and real-robot tracks, run by the benchmark team with blinded raters (paper)
• Robocurve, for RoboDojo-RC and its independent tests of coding agents and robot models on the same tasks, and for DataCenterBench
• Armnet, for ArmnetBench and its released checkpoints
• The RobotReplica team at UT Dallas, for VLA-Replica (paper)
• Artificial Analysis, for AA-Robotics
• Dexmal, for RoboChallenge and the repeat runs that let us measure its reliability (paper)
• The RoboArena team and the many labs that volunteer blind evaluations (leaderboard, paper)
• Positronic Robotics, for PhAIL and for publishing the video of every episode (paper)
• NVIDIA, for RoboLab-120 (paper)
• Ai2, for MolmoSpaces (paper) and for collecting paper-reported results in its VLA evaluation harness
• The RoboTwin 2.0 team (leaderboard, paper)
• The RoboCasa team, for RoboCasa365 and its verified submissions (paper)
• The RoboColosseum team, for RoboColosseum and its blind real-world comparisons at more than a hundred sites (paper), not to be confused with AgiBot's RoboColiseum
• AgiBot, for RoboColiseum (paper) and the Genie Sim 3.0 platform it runs on
• InternRobotics at Shanghai AI Lab, for EBench (paper)
• The LIBERO-Pro2.0 team at HUST (benchmark, paper) and the LIBERO-Plus authors (benchmark, paper)
• MLL Lab at Northwestern and its collaborators, for Embodied First Exam
• The creators of LIBERO, SimplerEnv and CALVIN, and every model team that submits to these benchmarks or reports results in its papers

This is the first version of the index. We will add benchmarks as they publish results and correct errors as they are found.

We’d love to hear what you think! Send feedback, corrections or benchmark suggestions to contact@withplayground.ai.

Request an evaluation

Want your robot policy in the next version of the index? Tell us about it, and we’ll be in touch shortly.