TQTS-Bench exposes a 38-point gap between the best model and humans

Time-series querying is not just text-to-SQL with timestamps

A new benchmark suggests that progress on relational text-to-SQL does not transfer cleanly to time-series databases. TQTS-Bench contains 6,125 verified question-answer pairs across 97 databases, 23 query syntaxes and 22 application domains. Its questions include four time-specific intent families: window aggregation and resampling, temporal change analysis, time localization, and relationship analysis.

The headline result is a wide model-human gap. Claude Opus 5 led the evaluated models at 48.98% overall execution accuracy, while human evaluators reached 87.34% on a randomly sampled 10% subset. That is a 38.36-point gap, although the different evaluation scope means the two figures should be read as the paper’s benchmark comparison rather than as identically sampled estimates.

Difficulty did not explain the whole shortfall. Claude Opus 5 scored 62.79% on easy questions, 42.91% on medium questions and 41.92% on hard questions. The human sample scored 95.71%, 85.52% and 81.94%, respectively. The benchmark defines easy questions as requiring one to three query intents, medium as four to five, and hard as six to nine.

Frontier models beat specialized pipelines—but still fail half the tasks

The paper evaluated seven current language models. GPT-6 Sol reached 46.38% overall execution accuracy, Gemini 3.7 Flash reached 44.65%, and the strongest open-source model in the table, Kimi K3, reached 43.15%. Claude’s lead over GPT-6 Sol was therefore only 2.60 points.

Older relational text-to-query pipelines transferred much worse. OpenSearch-SQL was the strongest listed relational method at 6.27%, DeepEye-SQL reached 5.83%, and DAIL-SQL reached 0.54%. PromCopilot, a PromQL-focused time-series method, reached 0.69% overall; the paper reports 3.03% in its targeted domains and 0.00% elsewhere.

Those results point to syntax breadth as a separate evaluation axis. TQTS-Bench mixes SQL-like systems with substantially different languages such as Flux and PromQL. The authors’ error analysis attributes failures to heterogeneous query syntax, misread time-specific intent and incorrect schema linking. In a follow-up conversion experiment on error cases, translating queries into a common relational setting raised execution accuracy from 0% to 32.21%, evidence that syntax fragmentation itself accounts for a meaningful share of the difficulty.

The benchmark’s construction is unusually labor-intensive: two annotators independently wrote and executed queries, with disagreements escalated to adjudication. The authors report more than 600 annotation hours and 30,774 query executions. That does not eliminate every validity question—the human score is sampled, and the repository was anonymous at release—but it makes TQTS-Bench a useful warning against treating relational leaderboard scores as evidence of broad database-query competence.