Snowflake’s agent-evaluation product is GA—but two orchestration metrics are not

Snowflake made Cortex Agent Evaluations generally available on March 13, 2026. That label is easy to read too broadly. The first-party release note names three available measurement paths: answer correctness against an expected answer, reference-free logical consistency across instructions, plans and tool calls, and custom LLM-judge metrics defined by the customer.

The more detailed Snowflake engineering post describes a wider four-part Goal-Plan-Action framework. In that breakdown, answer correctness and logical consistency are presented without preview qualifiers. But tool-selection accuracy and tool-execution accuracy are explicitly marked private preview.

That distinction matters for analytics agents. A final answer can match an expected answer while the agent still chooses an unnecessary tool, calls the right tool at the wrong stage, or passes the wrong arguments and recovers later. Snowflake’s post says the preview metrics are intended to test those intermediate decisions: tool selection asks whether the right tools were chosen at the right stages; tool execution compares invocations and tool outputs with expected behavior.

What the GA label safely supports

Teams can build a test table of queries and expected behavior, run evaluations in Snowsight or through the EXECUTE_AI_EVALUATION SQL procedure, and inspect run summaries plus record-level traces. Snowflake also documents SQL access to evaluation data and record traces, and says evaluations can be scheduled with a Task.

Those are substantial production controls. They support regression testing of final answers, consistency checks without ground truth, domain-specific custom judges, and trace inspection when a score falls.

What it does not guarantee

The GA announcement does not make every metric in the surrounding framework generally available. The March release note omits tool-selection and tool-execution accuracy from its GA metric list; the engineering post labels both private preview. A buyer should therefore avoid turning “Cortex Agent Evaluations is GA” into “all orchestration scoring is GA.”

The practical acceptance test is straightforward: verify in the target account which metric toggles are actually exposed, confirm whether tool invocations can be encoded in the ground-truth object, and preserve trace review as a fallback when intermediate-step scoring is unavailable.

The useful reading is not that Snowflake’s GA claim is misleading. It is that product status applies at the product boundary, while individual controls inside that product can remain preview-only. For an agent that writes SQL or routes among governed data tools, deployment documentation should record both levels.