What are the main LLM application security risks?

OWASP's Top 10 for LLM Applications ranks prompt injection first, and the practical inventory runs: direct and indirect prompt injection, sensitive information disclosure through prompts or responses, supply-chain compromise of models and plugins, training-data poisoning, improper output handling where model output is trusted as code or a command, excessive agency where the application grants the model more authority than the task requires, system-prompt leakage, and unbounded resource consumption. What they share is that the model is an untrusted component with a text interface, and text interfaces have never been a security boundary.

Is prompt engineering a security control?

It is a mitigation, not a control. Delimiting user content, separating instructions from data by role, and marking provenance all measurably reduce injection success rates, and they belong in every LLM application. They do not produce an enforceable decision, they degrade under adversarial pressure, and they leave no independent evidence of what was permitted. A control has to be deterministic and auditable. Prompt engineering is neither, which is why it sits alongside enforcement rather than in place of it.

How do you secure a RAG pipeline?

Treat retrieval as an authorization decision rather than a query filter. Evaluate the requester's identity against each document's access policy at retrieval time, so a reformulated query cannot widen the result set beyond the requester's entitlement. Mark retrieved content by provenance so the model does not treat instructions inside a document as instructions from the principal. And classify the assembled context window before the model call, so over-retrieval or injected payloads are caught at the boundary even when the retrieval authorization was wrong.

Why can't the application write its own AI audit log?

Because the application is the system under audit. An application-written log records the paths its instrumentation anticipated, which excludes the anomalous path that an investigation cares about. It can be modified or dropped by the same process that failed. And it is lost entirely when the process crashes after the model responds but before the write commits, leaving an executed action with no record. The independence requirement is not a philosophical nicety; it is what makes the record admissible as evidence about the application's behavior.

Where should LLM security controls sit in the architecture?

At every layer, with different expectations of each. Model-side safety and prompt engineering reduce failure frequency and cannot be relied on for enforcement. Retrieval authorization and tool scoping bound what the application can reach and cause, and they hold until the process is compromised. The AI request boundary, where the HTTP call to the model leaves the process, is the only place a control sits outside the application's blast radius, produces a deterministic decision, and writes evidence the application did not author. That is where identity policy and the audit record belong.

LLM Application Security: The Five Layers and Which One Holds When the Others Fail

An LLM application is a web application with an extra dependency that takes untrusted text as input, produces untrusted text as output, and cannot be reasoned about statically. Everything that was true about application security before is still true. What is added is a component whose behavior is probabilistic, and a data channel your existing controls cannot inspect.

Five layers carry the security burden. Four of them run inside the application's own process. That asymmetry decides everything about which controls hold under compromise.

Layer 1: the model

The attacks here are jailbreaks, adversarial prompting, and poisoned weights. The defense on offer is the provider's safety training: RLHF, constitutional AI, fine-tuning on refusal examples. These influence output and are worth having.

They are also probabilistic behaviors rather than enforceable controls. Stanford Trustworthy AI research summarized in the AIUC-1 Consortium briefing found refusal behaviors degraded significantly under targeted fine-tuning and adversarial pressure (Help Net Security). The enterprise has no enforcement guarantee at this layer, no independent record of what the model decided, and no policy decision point it controls. Anything resting on the model refusing is a gap. That is the case I make in model guardrails are not a security control.

Layer 2: the prompt

The attacks are direct prompt injection and system-prompt extraction. OWASP ranks prompt injection as LLM01 in its Top 10 for LLM Applications (OWASP).

The defenses are real and partial: delimit user content, mark provenance, keep instructions and data in separate roles, refuse to place untrusted text where the model treats it as system guidance. Every one of these reduces the success rate of injection. None eliminates it, because the model has no reliable mechanism for distinguishing instruction from data in a single token stream. I go through the technique inventory in prompt injection mitigation techniques.

Layer 3: the retrieval pipeline

The attacks are indirect injection through retrieved documents, over-retrieval past the requester's entitlement, and index poisoning.

The single most common architectural mistake in this layer is treating the retrieval query as a filter rather than an authorization boundary. A vector store queried with a tenant identifier in the metadata filter returns the right documents when the query is well-formed and the wrong ones when it is not, and a model that has been talked into reformulating the query can produce one that is not. Authorization belongs on the retrieval path, evaluated against the requester's identity, and the results belong in a context window that is classified before it goes anywhere. RAG prompt injection covers the attack in depth.

Layer 4: the tool surface

The attacks are argument injection into tools, excessive agency, and privilege escalation through chained tool calls where the effective authority exceeds what any single scope grant declared.

Defenses: least-privilege tool scopes, argument validation, human-in-the-loop for irreversible effects. These are the strongest in-process controls available, because they sit directly in front of the side effect. I set out the scoping model in AI agent tool permissions.

The problem with layers 1 through 4

Read them together. The prompt construction, the retrieval authorization, and the tool scoping are all enforced by application code. The model's safety behavior is enforced by the provider.

An attacker who achieves code execution in the application process controls three of the four. An attacker who successfully injects controls the fourth's inputs. And the audit record, if the application writes it, is written by the component under investigation. That is the self-attestation problem: in every other regulated domain the system under audit does not author its own evidence.

Application-controlled logs fail three specific ways. They log the paths the instrumentation anticipated, and the anomalous path is the one that goes unrecorded. They can be modified or dropped by the same process that failed. And they lose the record entirely when the process dies after the model call returns and before the write commits, which leaves an executed action with no evidence it ever happened.

Layer 5: the request boundary

Every one of the layers above resolves, eventually, into an HTTP request to a model endpoint. That request leaves the process. It is the one place in an LLM application where a control can sit outside the blast radius of the application it protects.

Four things become possible there and nowhere else.

Identity binding. The request carries a verified subject and role, so the authorization question becomes answerable per call rather than per session. Authentication answers who is calling; the post-authentication gap is what remains after it.

Prompt-level classification. The assembled context window is parsed and classified before the call proceeds, which catches over-retrieval, pasted secrets, and injected exfiltration payloads regardless of which upstream layer let them in.

Deterministic policy. The permit, redact, or deny decision is a policy evaluation, so the same identity, prompt, and policy produce the same result every time. Fail-closed on ambiguity.

Write-path independence. The decision record is committed by the enforcement point before the response returns to the application, so the application never has custody of the evidence about its own behavior.

This is the layer that holds when the others fail, which is precisely why the compliance evidence lives here. The EU AI Act's Article 12 requires automatic recording of events over a system's lifetime to ensure traceability, including timestamps, input data, and identification of the natural persons involved (Practical AI Act), with Article 99 penalties reaching €15 million or 3% of global annual turnover (EU AI Act Article 99).

Defense in depth, honestly stated

Layers 1 through 4 reduce the frequency of failure. Layer 5 bounds the consequence of failure and produces the record of it. An LLM application security program that stops at layer 4 has a stack of probabilistic mitigations and no control it can point a regulator at.

Build all five. Recognize which one you can still trust after a bad day.

DeepInspect

This is the gap DeepInspect closes. DeepInspect sits at the AI request boundary as a stateless proxy between authenticated users or agents and any HTTP-based LLM endpoint. Every request is evaluated against the identity making it, the role that identity holds, the classification of the content in the assembled prompt, and the policy attached to the route, then permitted, redacted, or denied inline with a fail-closed default.

The policy lives outside the application, so compromising the application does not compromise the control. Each decision writes a signed, per-decision record committed before the response returns, which gives the enterprise evidence about its own AI application that the application did not author. Because the proxy is model-agnostic, one policy set covers OpenAI, Anthropic, Bedrock, Azure OpenAI, Vertex, and self-hosted endpoints in the same deployment.

If you are shipping an LLM application into a regulated environment, book an AI readiness assessment.