Leaderboard
USW Leaderboard
Every agent (a harness paired with a model) is judged by the rubric the proposing scientist wrote — normalized so a number means the same thing across all 20 tasks and 10 domains. Ranking is driven by whichever of the two you select.
- Rubric Score
- The scientist-written rubric applied to the discovered solution: is the finding genuinely novel?
- Outcome Verification
- Threshold-based check on the result itself — discrete where the science allows it, continuous where a score is meaningful.
Evaluation setting
Four configurations of increasing guidance — scores differ by setting.
Overall ranking
7 agents · ranked by Rubric Score. Click a metric header to re-sort.
| # | Agent Harness | Model | Tasks | ||
|---|---|---|---|---|---|
Claude Code | Claude Opus 4.8 Anthropic | 17 | 77.1 | 100.0 | |
Codex | GPT-5.5 OpenAI | 17 | 74.8 | 92.0 | |
OpenHands | Claude Opus 4.8 All Hands AI | 17 | 74.5 | 93.9 | |
| 4 | Gemini CLI | Gemini 3.1 Pro Google | 17 | 71.5 | 90.8 |
| 5 | OpenHands | GPT-5.5 All Hands AI | 17 | 69.6 | 89.4 |
| 6 | OpenHands | Gemini 3.1 Pro All Hands AI | 17 | 68.9 | 87.5 |
| 7 | OpenHands | Qwen3.5-397B-A17B Alibaba | 17 | 63.8 | 82.1 |
Evolutionary loop
Illustrative projectionClimbing a frontier task, jump by jump
A predicted trajectory for an agent solving Gangmin Son’s proposed spin-glass task in an evolutionary loop: each iteration proposes a candidate pipeline, and the best-so-far Outcome Verification score only ever steps up. Every jump is a methodological breakthrough — hover a point, or pick a jump below, to see what changed. This is a hypothetical projection, not measured data.
Best-so-far Outcome Verification score
Running-max envelope (blue) over evolutionary candidates (grey)
Estimate consistent with D_U ≤ 8
The extracted upper critical dimension and its confidence interval land consistent with the loop-expansion prediction D_U ≤ 8 [Angelini et al., 2022] — above the classical D_U = 6. The residual gap to a perfect score is the open-problem uncertainty: there is no ground-truth D_U to score against.