live wire
▸Snowflake agent evals can pin judge versions; unpinned metrics may roll forward and shift scoresSnowflake▸ThoughtSpot makes Spotter 3 default for new instances; it requires download rights and always shares query data with an LLMThoughtSpot documentation▸MotherDuck adds image classification and object location to prompt_jev(), returning typed SQL outputs with confidence scoresMotherDuck▸BigQuery conversational analytics adds Montréal and Toronto; mixed-region agents still default to globalGoogle Cloud Documentation▸Fabric enables Azure OpenAI Copilot access by default, but OpenAI-subprocessor and cross-region processing/storage switches remain offMicrosoft Learn▸Snowflake Cortex Agents now continue by default when named tools are inaccessible, emitting warnings instead of failing the runSnowflake documentation▸Databricks Genie One can use group memberships to tailor answers, while Unity Catalog remains the access-control boundaryDatabricks documentation▸Fabric data agents can query Power BI models with Read alone; Build and workspace access are unnecessaryMicrosoft Learn▸Fabric data-agent Git configs expose column flags, but selecting a table still includes every columnMicrosoft Learn▸Google Data Agent Kit offers remote Spark kernels, but notebook exports can still move results into local Pandas or ArrowGoogle Cloud Documentation▸Cube says production analytics agents need explicit partial-result states, caller-filtered model search and durable artifactsCube▸Thinking Machines names Bridgewater behind ReViSQL-K2.6, citing 92.97% SC-16 accuracy at 15% of frontier-model costThinking Machines Lab▸AWS says a mismatched data/tool/prompt release cut its Field Sales agent pass rate from 78% to 38% in pre-productionAWS Artificial Intelligence Blog▸NL2SQL Multi-Agent ships evaluation artifacts across BIRD, CSpider, Archer and KDD DataSpace—but no software licenseGitHub▸Snowflake agent evals can pin judge versions; unpinned metrics may roll forward and shift scoresSnowflake▸ThoughtSpot makes Spotter 3 default for new instances; it requires download rights and always shares query data with an LLMThoughtSpot documentation▸MotherDuck adds image classification and object location to prompt_jev(), returning typed SQL outputs with confidence scoresMotherDuck▸BigQuery conversational analytics adds Montréal and Toronto; mixed-region agents still default to globalGoogle Cloud Documentation▸Fabric enables Azure OpenAI Copilot access by default, but OpenAI-subprocessor and cross-region processing/storage switches remain offMicrosoft Learn▸Snowflake Cortex Agents now continue by default when named tools are inaccessible, emitting warnings instead of failing the runSnowflake documentation▸Databricks Genie One can use group memberships to tailor answers, while Unity Catalog remains the access-control boundaryDatabricks documentation▸Fabric data agents can query Power BI models with Read alone; Build and workspace access are unnecessaryMicrosoft Learn▸Fabric data-agent Git configs expose column flags, but selecting a table still includes every columnMicrosoft Learn▸Google Data Agent Kit offers remote Spark kernels, but notebook exports can still move results into local Pandas or ArrowGoogle Cloud Documentation▸Cube says production analytics agents need explicit partial-result states, caller-filtered model search and durable artifactsCube▸Thinking Machines names Bridgewater behind ReViSQL-K2.6, citing 92.97% SC-16 accuracy at 15% of frontier-model costThinking Machines Lab▸AWS says a mismatched data/tool/prompt release cut its Field Sales agent pass rate from 78% to 38% in pre-productionAWS Artificial Intelligence Blog▸NL2SQL Multi-Agent ships evaluation artifacts across BIRD, CSpider, Archer and KDD DataSpace—but no software licenseGitHub
nl2sql.ai
benchmarkBenchmark

HakiCC gets large throughput gains from LLM-designed concurrency control—but verifies only observed schedules

A UCI pipeline generated ten workload-specific protocols and improved nine through evolutionary search; its correctness test is empirical, not a proof over every interleaving.

Throughput gains across two benchmarks, with one workload regressing.
Chart: figures from the story
By The Benchmark Desk· Oct 5, 2026the quick take — two AI hosts go live when you do

A new University of California, Irvine paper reports that an LLM pipeline can generate workload-specific database concurrency-control protocols and then optimize their implementations against measured throughput. The headline result is substantial: across five generated protocols for the paper’s TPC-C setup, the second-stage optimizer improved throughput by 50.6% on average; across five AuctionMark protocols, the average gain was 92.2%. Those are separate workload cohorts reported on October 1, 2026—not a single combined score.

What was actually tested

HakiCC has two stages. Stage 1 uses Gemini 2.5 Pro to propose and implement a protocol, with a repair loop capped at 50 iterations. Stage 2 uses Gemini 2.5 Flash in OpenEvolve for 1,000 serially evaluated iterations, allowing localized diffs rather than full rewrites. The authors report the first five protocols produced for each workload, ten in total.

The TPC-C evaluation is narrower than the full benchmark mix. It uses New Order and Payment only, split 50/50, with 1% random user aborts. The paper varies warehouses from 64 down to one across six contention levels, and separately tests four hotspot levels with eight warehouses. AuctionMark is tested at four contention levels by changing Zipfian skew over item and user access.

Every scenario runs for 30 seconds with 64 worker threads on a Java in-memory transactional engine. The machine has two Intel Xeon Gold 6242 CPUs, 96 virtual threads and 187 GB RAM. That makes the result a controlled systems experiment, not evidence about production database engines.

Optimization found bugs, not just new algorithms

The biggest gains often came from repairing implementation defects. For the TPC-C protocol SADL, the optimizer fixed duplicate sequence-number assignment, incorrectly shared schedulers and a missing wake-up that caused head-of-line blocking. SADL then improved between 52% and 406% across the six reported contention levels.

AuctionMark shows why the average needs context. TREAD regressed 21% at low contention, then gained 480%, 106% and 91% at medium, high and extreme contention. PVL gained 435% at extreme contention, while HSL declined by 4% to 14% at every tested level. The paper’s detailed result is therefore nine improved protocols out of ten, not universal improvement.

The correctness boundary matters

All ten final protocols passed the paper’s conflict-serializability checks, including banking, shared-counter and deadlock stress workloads. But the authors explicitly describe this as schedule-dependent empirical verification: precedence-graph cycle detection establishes serializability for the observed executions, not for every possible interleaving.

That distinction is the useful result. HakiCC shows that an LLM-guided search can discover and repair high-impact concurrency bottlenecks under a closed benchmark loop. It does not yet show that generated protocols can replace formal verification or production validation.

Filed by The Benchmark Desk. Corrections: desk@nl2sql.ai · Our standards →

comments · 0

    Comments are moderated before they appear. Your email is used once to confirm it is you — never shown, never sold. Corrections and questions get an answer from the desk when we have one.