Databricks Mosaic AI DLP: Unity Catalog Governs the Table, Not the Prompt Built From It
Unity Catalog governs which principal can read which table, column and row inside the lakehouse. A Mosaic AI application reads a governed table, assembles a prompt from the rows, and sends that prompt to a model endpoint, at which point the governance metadata stops travelling with the content. This piece walks the four Mosaic AI paths that cross the model boundary, explains why external model serving changes the risk profile, and sets out where classification and per-decision records belong.
A notebook reads a table with a column-level mask on customer_ssn and a row filter restricting the analyst to their own region. Unity Catalog enforces both. The analyst gets 4,000 rows, correctly masked and correctly filtered.
The next cell concatenates those 4,000 rows into a prompt and calls a model serving endpoint.
Unity Catalog governed the read. It has no say over the write to the model endpoint, because that is an outbound HTTP request carrying a string, and the string carries no grants, no lineage, and no column tags. Every governance decision made upstream is now metadata sitting in a catalogue while the content travels somewhere else.
That handoff is the whole of Databricks Mosaic AI DLP. I want to walk the four paths where it happens.
Four paths across the boundary
Foundation Model APIs serve models hosted by Databricks on pay-per-token or provisioned throughput. Traffic stays inside the Databricks account boundary, which is a real and useful property. The content still leaves the governed table and enters an ungoverned string.
External models proxy through Mosaic AI Gateway to OpenAI, Anthropic, Google, AWS Bedrock and others. Here the content leaves the Databricks boundary entirely, and the destination is decided by an endpoint configuration that a workspace admin can change without touching a single table grant. A serving endpoint repointed from a Databricks-hosted model to an external provider changes the data residency of everything flowing through it, and the change looks like a configuration edit.
Mosaic AI Vector Search indexes a Delta table into an index. The sync job embeds the source column for every row, on a schedule, with no per-row human review. A Delta Sync index built from a table containing customer notes sends every note to an embedding model at build time and again on every incremental sync.
Agent Framework and tool calls carry retrieved chunks, SQL results, and function outputs into the model context. An agent with a Unity Catalog function tool runs the function under the agent's service principal, gets rows back, and puts them in a prompt.
The vector search path is the one that surprises teams, for the same reason it does elsewhere: it is infrastructure, it runs on a schedule, and it processes everything it is pointed at.
Why the service principal breaks attribution
Model serving endpoints and agent deployments run under a service principal. That is correct architecture and it creates a specific problem.
An end user asks a question in a chat application. The application calls a serving endpoint. The endpoint's agent calls a Unity Catalog function under the service principal's identity. Unity Catalog evaluates the service principal's grants, which are necessarily broad enough to serve every user of the application, and returns rows.
The row filter that would have restricted a human analyst to their own region evaluates against the service principal instead. Whatever that principal can see, the agent can retrieve, and whatever the agent retrieves can end up in a response.
Databricks provides mechanisms to pass end-user identity through, and teams that have wired them up are in good shape. The default path for a quickly-built agent is the service principal, and the gap only becomes visible when someone asks who was allowed to see a specific returned row. This is the same shape as the AI agent post-authentication gap.
My view is that this single default has produced more real data exposure in lakehouse AI deployments than any model-layer vulnerability, and it gets a fraction of the attention.
What Mosaic AI Gateway covers and what sits outside it
Mosaic AI Gateway offers rate limiting, payload logging, and guardrails on served endpoints. Those are useful and they apply to traffic that goes through a Databricks serving endpoint.
Traffic that reaches a provider by another route sits outside. A notebook calling an OpenAI SDK directly with a key from a secret scope, a job cluster with an egress path to a provider API, or an application running outside the workspace that reads from Delta Sharing and calls a model itself, none of these pass through the gateway. The shadow AI problem exists inside the lakehouse as well as outside it, and a workspace with 60 notebooks is a place where direct SDK calls accumulate quietly.
Inventory the secret scopes containing provider API keys and check which clusters can read them. That list is usually longer than expected, and it is a reasonable proxy for the ungoverned surface.
What has to be classified
Classification runs on the assembled prompt, at the point the request is made, because that is the last place the content exists as content rather than as tokens.
The tags already in Unity Catalog are the input. A column tagged pii in the catalogue tells you what a span in a prompt means, and carrying that tag forward from the read into the request is the connection that most deployments never make. The classification practice is covered in AI data classification.
For vector search sync, classification runs per chunk before embedding, and the output is a corpus report rather than per-row alerts. A report saying 2,140 chunks across one source table carried customer PII is actionable. Two thousand alerts are not.
For external model endpoints, the policy question is destination as well as content: which classes may leave the Databricks boundary at all, evaluated per request rather than per endpoint configuration.
The record
decision_id: d-1a90f4, recorded at 2026-08-14T13:18:26Zoriginating_identity: m.okafor@example.com, the end user who askedacting_principal: sp-mosaic-agent-prod, the service principal that executed the readendpoint: databricks-external-anthropic-prod,destination_boundary: externalsource_tables: main.crm.customer_notes and main.finance.invoicesuc_column_tags: pii, financial, carried forward from Unity Catalogclassifications: customer-pii on 37 spans, financial-record on 12policy: lakehouse-external-egress-v2,outcome: blockedreason: pii-tagged content bound for an external destination
The pair of originating_identity and acting_principal is what turns a service principal call back into an attributable event. The source_tables and uc_column_tags fields carry catalogue governance forward into the request record, which is what lets an auditor connect a column-level tag to an actual disclosure decision months later.
The comparable per-provider surfaces are covered in AWS Bedrock DLP and Azure AI Foundry DLP.
DeepInspect
This is the gap DeepInspect closes. DeepInspect sits as a stateless proxy on the HTTP path between your Databricks workloads and the model endpoints they call, covering serving endpoints, external model routes, embedding calls and direct SDK traffic from notebooks on the same terms. Content is classified before the request is forwarded, policy is evaluated against the identity that originated the request rather than the service principal executing it, and a per-decision record is written into storage you control.
Because the decision happens on the request path, Unity Catalog governance and prompt-level policy stop being two disconnected systems. A column tagged in the catalogue produces a policy outcome on the request that carries it. Enforcement is inline and fails closed.
DeepInspect does not manage Unity Catalog grants, build your Delta Sync indexes, or configure your serving endpoints.
If your governed tables are being read correctly and you cannot say what happens to the rows after they enter a prompt, that second half is the open question. Book a demo today.