BigQuery’s AI.AGG can summarize beyond one context window—but its default accepts partial inference failure

BigQuery has made AI.AGG generally available, moving a useful piece of agent-style analysis into a native SQL aggregate. The function takes rows of text or multimodal references plus a natural-language instruction, invokes a Vertex AI Gemini model, and returns one value per input group. Google positions it for sentiment analysis, multimodal summarization, agent-performance analysis, log analysis, and error clustering. Google Cloud documentation

The important change is the execution model

This is not merely a row-by-row prompt wrapper. Google says AI.AGG automatically batches inputs and performs multi-level aggregation when the data exceeds a model’s standard context window. For smaller inputs it can skip that hierarchy. That makes the function materially different from AI.GENERATE, which returns a result for each input row and leaves comparable batching logic to the user. Google Cloud documentation

The input can include strings and Cloud Storage object references for images, audio, video, text, and PDFs. Google recommends no more than 1,000 groups, with workload-specific guidance ranging from 20 million rows for text or single-image inputs down to 1,000 rows for video. Output is capped at 10,000 tokens per group. Those are operating boundaries, not a promise that every query below them will succeed. Google Cloud documentation

Partial success needs an explicit policy

The most consequential default is max_error_ratio => 1.0. BigQuery can return partial results when individual Vertex AI calls fail, and the query fails only when every inference fails unless the operator sets a stricter ratio. For production analytics, teams should choose that threshold deliberately rather than treating a completed SQL job as proof that every row was analyzed. Structured output schemas can constrain the returned fields, but they do not remove the need to inspect inference coverage. Google Cloud documentation

There is also a network-control wrinkle: inside a VPC Service Controls perimeter, delegated object access through signed HTTPS URLs is blocked for this function. Google directs users to pass direct Cloud Storage references with OBJ.MAKE_REF instead. Google Cloud documentation

The practical gain is substantial: an analytics agent can now ask grouped, semantic questions over large mixed datasets without implementing its own reduction tree. The production lesson is equally clear. Teams should version the prompt, specify an output schema, set a non-default error ratio, record the selected model endpoint, and validate aggregate answers against deterministic counts before using them in decisions.