live wire
▸ThoughtSpot makes Spotter 3 default for new instances; it requires download rights and always shares query data with an LLMThoughtSpot documentation▸MotherDuck adds image classification and object location to prompt_jev(), returning typed SQL outputs with confidence scoresMotherDuck▸BigQuery conversational analytics adds Montréal and Toronto; mixed-region agents still default to globalGoogle Cloud Documentation▸Fabric enables Azure OpenAI Copilot access by default, but OpenAI-subprocessor and cross-region processing/storage switches remain offMicrosoft Learn▸Snowflake Cortex Agents now continue by default when named tools are inaccessible, emitting warnings instead of failing the runSnowflake documentation▸Databricks Genie One can use group memberships to tailor answers, while Unity Catalog remains the access-control boundaryDatabricks documentation▸Fabric data agents can query Power BI models with Read alone; Build and workspace access are unnecessaryMicrosoft Learn▸Fabric data-agent Git configs expose column flags, but selecting a table still includes every columnMicrosoft Learn▸Google Data Agent Kit offers remote Spark kernels, but notebook exports can still move results into local Pandas or ArrowGoogle Cloud Documentation▸Cube says production analytics agents need explicit partial-result states, caller-filtered model search and durable artifactsCube▸Thinking Machines names Bridgewater behind ReViSQL-K2.6, citing 92.97% SC-16 accuracy at 15% of frontier-model costThinking Machines Lab▸AWS says a mismatched data/tool/prompt release cut its Field Sales agent pass rate from 78% to 38% in pre-productionAWS Artificial Intelligence Blog▸NL2SQL Multi-Agent ships evaluation artifacts across BIRD, CSpider, Archer and KDD DataSpace—but no software licenseGitHub▸Snowflake says Cortex Analyst apps keep working, but recommends Cortex Agents; semantic views and verified queries carry overSnowflake Documentation▸ThoughtSpot makes Spotter 3 default for new instances; it requires download rights and always shares query data with an LLMThoughtSpot documentation▸MotherDuck adds image classification and object location to prompt_jev(), returning typed SQL outputs with confidence scoresMotherDuck▸BigQuery conversational analytics adds Montréal and Toronto; mixed-region agents still default to globalGoogle Cloud Documentation▸Fabric enables Azure OpenAI Copilot access by default, but OpenAI-subprocessor and cross-region processing/storage switches remain offMicrosoft Learn▸Snowflake Cortex Agents now continue by default when named tools are inaccessible, emitting warnings instead of failing the runSnowflake documentation▸Databricks Genie One can use group memberships to tailor answers, while Unity Catalog remains the access-control boundaryDatabricks documentation▸Fabric data agents can query Power BI models with Read alone; Build and workspace access are unnecessaryMicrosoft Learn▸Fabric data-agent Git configs expose column flags, but selecting a table still includes every columnMicrosoft Learn▸Google Data Agent Kit offers remote Spark kernels, but notebook exports can still move results into local Pandas or ArrowGoogle Cloud Documentation▸Cube says production analytics agents need explicit partial-result states, caller-filtered model search and durable artifactsCube▸Thinking Machines names Bridgewater behind ReViSQL-K2.6, citing 92.97% SC-16 accuracy at 15% of frontier-model costThinking Machines Lab▸AWS says a mismatched data/tool/prompt release cut its Field Sales agent pass rate from 78% to 38% in pre-productionAWS Artificial Intelligence Blog▸NL2SQL Multi-Agent ships evaluation artifacts across BIRD, CSpider, Archer and KDD DataSpace—but no software licenseGitHub▸Snowflake says Cortex Analyst apps keep working, but recommends Cortex Agents; semantic views and verified queries carry overSnowflake Documentation
nl2sql.ai

research · since 2026-09-20

NL2SQL Fine-tune Reliability

Mohan Vishe

Reproducible MIT evaluation of QLoRA fine-tuning for Qwen2.5-Coder-3B on BIRD Mini-Dev, measuring capability (pass@10), repeatability (pass^10), failure modes and comparison with a prompted 7B model.

website ↗repository ↗MIT
text-to-sqlbenchmarkfine-tuningreliability
desk notes

Final reproducible results published Sept. 26, 2026. On 496 scored questions × 10 attempts per arm, fine-tuning raised pass@10 by 4.2 points and pass^10 by 4.4, leaving the reliability gap statistically unchanged. Attempt-level SQL crashes fell 19.8 points while silent wrong-row executions rose 14.5 points. Repository publishes attempts, scorer, tests, run logs and confidence intervals; 0 stars/0 forks at verification.