live wire
▸Snowflake agent evals can pin judge versions; unpinned metrics may roll forward and shift scoresSnowflake▸ThoughtSpot makes Spotter 3 default for new instances; it requires download rights and always shares query data with an LLMThoughtSpot documentation▸MotherDuck adds image classification and object location to prompt_jev(), returning typed SQL outputs with confidence scoresMotherDuck▸BigQuery conversational analytics adds Montréal and Toronto; mixed-region agents still default to globalGoogle Cloud Documentation▸Fabric enables Azure OpenAI Copilot access by default, but OpenAI-subprocessor and cross-region processing/storage switches remain offMicrosoft Learn▸Snowflake Cortex Agents now continue by default when named tools are inaccessible, emitting warnings instead of failing the runSnowflake documentation▸Databricks Genie One can use group memberships to tailor answers, while Unity Catalog remains the access-control boundaryDatabricks documentation▸Fabric data agents can query Power BI models with Read alone; Build and workspace access are unnecessaryMicrosoft Learn▸Fabric data-agent Git configs expose column flags, but selecting a table still includes every columnMicrosoft Learn▸Google Data Agent Kit offers remote Spark kernels, but notebook exports can still move results into local Pandas or ArrowGoogle Cloud Documentation▸Cube says production analytics agents need explicit partial-result states, caller-filtered model search and durable artifactsCube▸Thinking Machines names Bridgewater behind ReViSQL-K2.6, citing 92.97% SC-16 accuracy at 15% of frontier-model costThinking Machines Lab▸AWS says a mismatched data/tool/prompt release cut its Field Sales agent pass rate from 78% to 38% in pre-productionAWS Artificial Intelligence Blog▸NL2SQL Multi-Agent ships evaluation artifacts across BIRD, CSpider, Archer and KDD DataSpace—but no software licenseGitHub▸Snowflake agent evals can pin judge versions; unpinned metrics may roll forward and shift scoresSnowflake▸ThoughtSpot makes Spotter 3 default for new instances; it requires download rights and always shares query data with an LLMThoughtSpot documentation▸MotherDuck adds image classification and object location to prompt_jev(), returning typed SQL outputs with confidence scoresMotherDuck▸BigQuery conversational analytics adds Montréal and Toronto; mixed-region agents still default to globalGoogle Cloud Documentation▸Fabric enables Azure OpenAI Copilot access by default, but OpenAI-subprocessor and cross-region processing/storage switches remain offMicrosoft Learn▸Snowflake Cortex Agents now continue by default when named tools are inaccessible, emitting warnings instead of failing the runSnowflake documentation▸Databricks Genie One can use group memberships to tailor answers, while Unity Catalog remains the access-control boundaryDatabricks documentation▸Fabric data agents can query Power BI models with Read alone; Build and workspace access are unnecessaryMicrosoft Learn▸Fabric data-agent Git configs expose column flags, but selecting a table still includes every columnMicrosoft Learn▸Google Data Agent Kit offers remote Spark kernels, but notebook exports can still move results into local Pandas or ArrowGoogle Cloud Documentation▸Cube says production analytics agents need explicit partial-result states, caller-filtered model search and durable artifactsCube▸Thinking Machines names Bridgewater behind ReViSQL-K2.6, citing 92.97% SC-16 accuracy at 15% of frontier-model costThinking Machines Lab▸AWS says a mismatched data/tool/prompt release cut its Field Sales agent pass rate from 78% to 38% in pre-productionAWS Artificial Intelligence Blog▸NL2SQL Multi-Agent ships evaluation artifacts across BIRD, CSpider, Archer and KDD DataSpace—but no software licenseGitHub
nl2sql.ai
rankingMETHODOLOGY

How we rank: the nl2sql.ai leaderboard methodology

Four pillars, evidence-diffed monthly editions, and no commercial relationships with anything we score.

By The Editorial Desk· Aug 10, 2026the quick take — two AI hosts go live when you do

The leaderboard scores systems 0–100 across four pillars:

PillarWeightWhat it measures
Capability40%SQL quality on hard, messy schemas; public benchmark evidence; self-correction
Trust & governance25%Semantic grounding, verified queries, permissions, auditability
Adoption signals20%Production usage, community momentum, integrations
Openness15%Open source/weights, self-hosting, extensibility, docs

Three rules make it a ranking worth trusting:

  1. Evidence-diffed editions. The leaderboard re-scores monthly. Every score change names its evidence — a release, a benchmark submission, an engineering post, a pricing change. Editions are permanent and linkable; you can always see what moved and why.
  2. Comparable, not identical, categories. Cloud-native services, OSS frameworks, and open-weight models compete on the same board because buyers actually choose between them — but each entry's verdict states what it is, so a rank-3 model isn't mistaken for a rank-3 managed service.
  3. No commercial relationships. Nothing on the board pays us; nothing can. If that ever changes, the board dies before the rule does.

The launch edition (August 2026) is the reviewed baseline. From September, the tools desk proposes score changes with citations, the benchmark desk supplies the numbers, and the editor signs off on every published edition. Disagree with a score? The newsroom reads its mail — every edition page says how to file a challenge, and challenges get answered in public.

Filed by The Editorial Desk. Corrections: desk@nl2sql.ai · Our standards →

comments · 0

    Comments are moderated before they appear. Your email is used once to confirm it is you — never shown, never sold. Corrections and questions get an answer from the desk when we have one.

    more from the desk