Databricks can wake an analytics pipeline on shared-table changes—but not prove the answer is fresh

Azure Databricks has made Lakeflow Jobs table-update triggers for OpenSharing objects and system tables generally available. The September release note dates the change to September 30, while the trigger documentation says a job can now start when a provider updates a shared table or when new records land in a system table.

For analytics-agent teams, that creates a useful event-driven boundary. A pipeline can rebuild derived context, refresh an evaluation summary or process newly arrived usage records after the monitored source changes, without keeping a cluster running continuously or knowing which upstream process performed the update. The same trigger can watch OpenSharing tables, views, metric views, materialized views and streaming tables, plus Databricks system tables.

The trigger carries evidence about what changed

A triggered job can receive dynamic parameters containing the list of updated tables, the latest triggering commit timestamp and the latest triggering version. That gives downstream work a concrete checkpoint instead of relying only on wall-clock schedules.

The trigger can fire when any monitored table changes or wait until all monitored tables have changed. Operators can also set a minimum interval between runs and a delay after the last change, allowing a burst of updates to settle before processing begins. Those controls matter when an analytics agent depends on several related sources—but they also mean “triggered by an update” is not the same as “refreshed immediately.”

Where freshness can still break

Databricks caps a trigger at ten managed or Delta source tables. Views add another subtlety: the trigger watches their dependent tables and can fire even when the changed rows would be filtered out by the view definition. A view with unsupported dependencies can still accept trigger creation but never start a run when that dependency changes. Federated-table dependencies and views that directly use read_files are among the unsupported cases.

OpenSharing and system-table triggers also have a workspace limit of 1,000, and cross-regulatory-domain OpenSharing is unsupported. Databricks recommends enabling file events for external locations to improve efficiency and performance.

The operational lesson is narrow but important: this GA feature can replace polling and provide a source-version checkpoint for a refresh pipeline. It does not certify that every dependency was observable, that a view’s result changed, or that the analytics agent has already switched to the rebuilt artifact. Teams should record the trigger’s table list, commit timestamp and version alongside the artifact version served to the agent, then test the end-to-end handoff rather than treating job start as a freshness guarantee.