Argo-Bench’s best agent costs nearly 10× more per task than its closest budget rival

Argo-Bench’s headline is that enterprise data agents still struggle: its strongest run, Claude Opus 5.5 at extra-high reasoning effort, averages 59.5 points out of 100 across 210 tasks. But the more operational finding is lower on the same leaderboard. GPT-6.1 Sol averages 49.5 points at a reported $0.49 per task, while Opus costs $4.71. In other words, Sol delivers 83% of the leader’s mean score for about 10% of its model cost. Argo-Bench leaderboard

That does not make Sol the universal winner. Opus fully solves 34.8% of tasks under the benchmark’s threshold of 95 points, compared with 24.8% for Sol. A team buying the extra ten mean-score points also gets ten percentage points more fully solved tasks. The question is whether that gain is worth roughly $4.22 more per attempt before infrastructure and human-review costs. Argo-Bench leaderboard

Cost and capability do not rise together

The board reports more than score and price. Opus averages 72,276 tokens and 82 steps per task; Sol averages 17,073 tokens and 24 steps. GPT-6 Astra sits between them on quality, averaging 51.8 points at $2.71, 14,578 tokens and 23 steps. Claude Sonnet 5.5 reaches the same 51.8 mean score as Astra but uses 84,748 tokens, 77 steps and $3.74 per task. These are not controlled model-only comparisons—the systems’ behavior and trajectories differ—but they show why a single accuracy ranking is a poor procurement table. Argo-Bench leaderboard

For deployment, the useful frontier has at least three axes: mean outcome quality, probability of a near-complete solution, and cost per attempted task. Teams with high-volume, reviewable work may rationally choose Sol’s lower-cost operating point and reserve a stronger model for retries. Workflows where a missed fraud case or incorrect financial action is expensive may justify Opus’s higher full-solve rate.

The benchmark is inspectable, but official grading is not fully local

Argo-Bench releases the 210 tasks, reference agent and a 71.3 GiB compressed warehouse spanning 1,219 Parquet files. The simulated environment contains 235 tables and 7.49 billion rows, with setup paths for BigQuery, DuckDB, Snowflake, Trino, Delta Lake and Iceberg. That makes the workload substantially more inspectable than a leaderboard backed only by private prompts. Argo-Bench project page Argo-Bench repository

The official scores still depend on a private simulated world, however. The public site says submissions are scored against held-out simulator state, and it reports 95% uncertainty ranges of roughly ±2 to 8 points. Buyers should therefore treat close rankings cautiously and reproduce task-level cost, latency and failure patterns on their own warehouse before choosing a model. Argo-Bench methodology and submission note