SQLStage makes text-to-SQL tuning inspectable—but its best configuration is not a benchmark result

SQLStage, a new Apache-2.0 text-to-SQL workbench, is built around a useful premise: choosing a model is only one decision in a database assistant. Schema retrieval, examples, business guidance, sampled values, candidate count and repair steps can all change an answer, so the project exposes those choices as a configuration that teams can inspect and test.

The product is the experiment loop

The same Python engine powers SQLStage’s React workbench, evaluation search and separate prediction service. It supports SQLite, MySQL and PostgreSQL, plus OpenAI-compatible model endpoints. Retrieval options include all-schema, keyword, BM25, embedding, hybrid and model-based table selection; generation can produce direct SQL or an intermediate query object, with optional multiple candidates, voting, model selection, execution feedback and repair.

That breadth could easily become an unbounded model-call machine. Instead, the default search is capped at four configurations across 10 fixed questions, with a shared limit of 50 model requests. Retrieval calls, embeddings, candidate generation, selection, repairs and retries all consume the same budget. Reports retain per-question errors, token usage and timing, while SQLite result caching is keyed to the strategy, data and code fingerprint.

This is a better engineering artifact than a single launch-day accuracy number. Teams can ask whether adding examples helped, whether a database guide changed errors, or whether Pick-3 selection justified its extra calls—on their schema and their questions.

Search is not validation

SQLStage’s documentation draws an important line around the result. Its configuration ranking prioritizes AST accuracy, then error count and latency, but the selected strategy has already been optimized against the search questions. The maintainer therefore tells users to validate it on an untouched holdout set and explicitly says the repository publishes no universal accuracy score.

That caveat matters. A local grid-search winner is evidence that one configuration fit one evaluation set; it is not evidence that the same recipe leads across databases, workloads or business vocabularies. SQLStage ships no competition dataset, third-party question set, model weights or precomputed leaderboard result. Users must bring the database, reference questions, examples and guidance that make the comparison meaningful.

Inspectability is not a sandbox

The release also avoids overselling its safety layer. Read-only SQL checks, dangerous-function blocking, row limits and timeouts are described as guardrails rather than a security sandbox. The project recommends least-privilege database accounts and warns that schema, examples and retrieved context can be sent to the configured model provider. Its own checks cover software behavior—13 offline test scripts, frontend builds, smoke checks and CI—not model accuracy.

For teams evaluating text-to-SQL stacks, that makes SQLStage most useful as a controlled workbench, not an accuracy claim. It gives practitioners a place to measure the decisions surrounding the model, then leaves the hardest proof—an untouched holdout and safe deployment—to them.