What is AI model poisoning?

It is the deliberate corruption of a model's behavior by manipulating data it learns from or reasons over. Three distinct attacks share the name: pretraining corpus poisoning, where an attacker plants content in sources that will be scraped into a foundation model's training data; fine-tune backdoors, where an attacker with pipeline access plants a trigger that activates a hidden behavior; and retrieval or context poisoning, where an attacker plants instructions in a source the application queries at inference time. They differ in timescale, in the access the attacker needs, and critically in whether the deploying enterprise can see them at all.

Can you detect a poisoned model?

Not from the deploying enterprise's position, for the training-time and fine-tune variants. You cannot audit a trillion-token corpus, you do not have interpretable access to the weights, and behavioral testing proves the presence of a behavior rather than the absence of one. A backdoor is dormant until its trigger appears, and you cannot enumerate the triggers you have not thought of. Retrieval poisoning is the exception: the poisoned content is in the prompt your own application assembles, so it is fully inspectable at the request boundary.

What is the difference between model poisoning and prompt injection?

Prompt injection manipulates a model's behavior at inference time by placing adversarial instructions in its context window, and it leaves no persistent trace in the model. Model poisoning manipulates behavior by corrupting what the model learned, and it persists across every request until the model is retrained or replaced. Retrieval poisoning sits between them: the attacker's content is planted persistently in a data store, and it acts on the model through the injection mechanism at inference. The defenses differ accordingly, and only the injection-side ones are available to a deploying enterprise.

Does a fine-tuned model reduce or increase poisoning risk?

It increases the attack surface in exchange for control of a smaller pipeline. Fine-tuning gives the enterprise a corpus it can review, which is more than it has for pretraining. It also creates a pipeline an attacker can target, and Stanford Trustworthy AI research summarized in the AIUC-1 briefing found that refusal behaviors degrade significantly under targeted fine-tuning, meaning fine-tuning can weaken the provider's safety training without anyone intending it. Treat the fine-tune pipeline as production infrastructure with the access controls and change review that implies.

How do you defend against model poisoning if you cannot detect it?

Stop trusting the model and enforce outside it. Bound what enters the context window, so a triggered backdoor has no regulated data available to exfiltrate. Bound what model output is permitted to cause, so a tool call the model emits still has to pass an external authorization check against the caller's delegated scopes. Record every decision independently of the application, so a behavior nobody can explain from the weights is at least reconstructible from the request path. These are compensating controls, and compensating is the honest description: they bound the damage from a threat that cannot be detected at its source.

AI Model Poisoning: Why the Right Response Is to Stop Trusting the Model

Three different attacks travel under the name "model poisoning," and conflating them produces a control strategy that addresses none of them. Corrupting the pretraining corpus, planting a backdoor during fine-tuning, and poisoning the retrieval index an application queries at inference are separated by years of time, by different threat actors, and by whether the deploying enterprise has any visibility at all.

OWASP tracks the training-data variant as LLM03 in its Top 10 for LLM Applications (OWASP). I want to work through what an enterprise can actually see at each layer, because the honest answer for two of the three is "nothing," and that answer has an architectural consequence most teams have not drawn.

Pretraining corpus poisoning

An attacker plants content in a source that will be scraped into a foundation model's training corpus: a web page, a public repository, a document set. Months later the model is trained, and the planted content is now a statistical influence on its weights, alongside trillions of other tokens.

What the deploying enterprise can inspect: nothing. You did not train the model, you cannot see the corpus, and you have no access to the weights in a form that lets you interrogate them for a specific implanted behavior. Even the provider struggles here, because the corpus is too large to audit and the influence of any single document is diffuse.

The EU's Action Plan on Cybersecurity and Artificial Intelligence, presented July 7, 2026, addresses this by pushing evaluation upstream: pre-market cybersecurity evaluation of advanced models, an EU third-party evaluation capacity supporting the AI Office, and an ENISA and JRC secure testing platform. That is the correct venue for the problem. It is also a venue the deploying enterprise does not sit in.

Fine-tune backdoors

An attacker with access to the fine-tuning pipeline plants a trigger: a specific token sequence that causes the model to behave differently. The behavior is dormant under normal use and activates on the trigger.

What the enterprise can inspect: something, in principle, and almost nothing in practice. If you control the fine-tuning data you can review it, which is worth doing. But a backdoor trigger is a needle in a corpus, and the model's behavior on inputs you have not thought to test tells you nothing. Testing proves the presence of a behavior. It cannot prove the absence of one.

There is a related result worth sitting with. Stanford Trustworthy AI research summarized in the AIUC-1 Consortium briefing found that model refusal behaviors degrade significantly under targeted fine-tuning and adversarial pressure (Help Net Security). Fine-tuning is a lever that moves model behavior in ways that testing does not surface reliably. That cuts both directions: an adversary can use it, and a well-intentioned team can degrade the provider's safety training without noticing.

Retrieval and context poisoning

An attacker plants content in a source the application retrieves from at inference time: a wiki page, a shared drive, a ticket, a web page the agent is asked to read. The content contains instructions. The retrieval pipeline pulls it into the context window, and the model treats it as part of the prompt.

What the enterprise can inspect: everything. This is the one layer that sits entirely inside the enterprise's own request path. The poisoned content is in the prompt, and the prompt is an HTTP payload the enterprise assembles and sends. I covered the mechanics in RAG prompt injection and AI agent context window poisoning.

This is also the variant that actually shows up in incidents, because it requires no access to a training pipeline and no patience. Write to a wiki, wait for the agent to read it.

The architectural consequence

Two of the three layers are outside the enterprise's inspection boundary. The strategic response to that is not to try harder at detection. It is to stop treating model output as trusted.

If you cannot verify that a model is uncorrupted, then the model is an untrusted component, and the controls that apply to untrusted components apply here: constrain what it can reach, constrain what its output is permitted to cause, and record every decision so a corrupted behavior is reconstructible after the fact.

That reframing produces three concrete controls.

Bound the input. A poisoned model can only exfiltrate data that entered its context window. Classify what goes into the prompt and deny regulated content a path to endpoints that policy does not cover. A backdoor that fires on a trigger cannot leak a customer record that was never in the prompt.

Bound the output's authority. Model output is input to your system. Treat it as such. If an agent's next tool call is chosen by the model, the authorization for that call comes from the agent's delegated scopes and is evaluated externally, not from the model's confidence. A backdoored model that emits a delete_records call still has to pass a policy that says this identity may not delete records. This is the argument in model guardrails are not a security control, applied to poisoning: anything that depends on the model behaving correctly is a gap.

Record the decision, not the outcome. When a poisoned behavior fires, the investigation needs the prompt that triggered it, the identity that sent it, and the policy that evaluated it. An application log that records "the model returned X" tells you what happened and nothing about why it was permitted.

Where the boundary actually sits

Say this plainly, because the market does not. No HTTP proxy in front of a model detects a poisoned weight. Anyone claiming otherwise is selling something they cannot build.

What an enforcement point at the AI request boundary does is make a poisoned model less useful to whoever poisoned it. It sees the prompt, so it can deny the data the backdoor would exfiltrate. It sees the identity, so an unauthorized caller cannot reach the model to probe for triggers. It sees the retrieved content in the assembled context, so the retrieval-poisoning variant is inspectable at exactly the moment it matters. And it writes the record that makes the incident reconstructible.

That is a compensating control, and compensating is the correct word. It bounds the damage from a threat the enterprise cannot detect at its source.

DeepInspect

This is the problem DeepInspect was built to solve. DeepInspect is a stateless proxy at the AI request boundary that treats the model as an untrusted external system. Every request from an authenticated user or agent is evaluated against identity, role, and the classification of the content inside the assembled prompt, then permitted, redacted, or denied inline, fail-closed by default.

Because DeepInspect sees the full context window, retrieval-poisoned content is inspectable at the moment it would enter the model, and regulated data that a backdoor could exfiltrate never reaches an endpoint policy does not permit. Every decision produces a signed, per-decision record committed before the response returns to the application, which is the evidence an investigation needs when a model behaves in a way nobody can explain from the weights.

If you are deploying models you did not train and cannot inspect, book an AI readiness assessment.