Databricks’ Lakebase test shows agent search has two separate sizing ceilings

Databricks has published a useful production lesson for teams putting retrieval behind an AI agent: increasing database compute helps only until the application tier becomes the bottleneck. Its internal Slide Hub app gives field engineers and AI agents hybrid search over roughly 700 presentations through the same Model Context Protocol endpoint. The implementation keeps slides, votes, telemetry, history and search indexes in Lakebase Postgres rather than copying them into a separate search service. [Source: Databricks]

One request, two compute budgets

Each request first embeds the query in the Databricks App and then sends one SQL statement to Lakebase Postgres. That statement combines vector and BM25 candidate lists, applies filters and joins votes. Databricks tested the two compute layers separately by varying database Compute Units and the size and worker count of the app. [Source: Databricks]

At the lower database sizes, Lakebase was the limiter. A medium app with two workers delivered 196 requests per second at 1 CU and 336 requests per second at 8 CU in the vendor’s 500-user test. Moving from 1 CU to 8 CU also cut the reported average search-statement time from 146 milliseconds to 34 milliseconds as the local file cache grew from 1.5 GB to 12 GB. Giving the 1-CU database a larger app with twice the workers did not materially improve throughput. [Source: Databricks]

The boundary changed above 8 CU. Databricks reports only a small gain from 8 CU to 16 CU with the medium app—336 to 349 requests per second—while the 16-CU database paired with a large four-worker app reached at least 662 requests per second and 343 ms median end-to-end latency. The practical reading is not that every deployment needs 16 CU; it is that teams must load-test the application worker and connection-pool ceiling as well as the database. [Source: Databricks]

Cache fit matters before raw compute

A second test scaled a public corpus from one million to four million passages. Databricks says warm vector latency stayed in single-digit milliseconds when the local file cache was sized to hold roughly 95% to 98% of index pages. It also found that cutting each retrieval candidate pool from 1,000 to 200 preserved 90.5% of the top-10 results while using about one-third as many page reads. Those are vendor-run measurements on one architecture, not a general retrieval benchmark, but they expose the tradeoff teams should measure: index residency and candidate breadth can matter more than simply buying a larger database. [Source: Databricks]

The production floor is a latency choice

Lakebase can scale to zero, but Databricks recommends an always-on 0.5-CU production floor for this workload to avoid cold starts, estimating roughly $31–$40 per month. Its 14-minute load test served 109,000 requests and incurred $0.02 in Lakebase compute, but that short run does not establish a full monthly cost profile. [Source: Databricks]

For agent builders, the broader lesson is architectural: co-locating retrieval with application data removes a second transaction path and keeps filters under one permission model, but it does not remove capacity planning. Database cache, app workers, connection pools and cold-start policy remain separate controls—and the slowest one sets the user experience. [Source: Databricks]