An AI Supply Chain Attack Rarely Touches a Model File Directly

At 4:40pm GMT on December 30, 2022, a package named torchtriton appeared on the Python Package Index with the same name as a package PyTorch was shipping through its own nightly index. PyTorch's own postmortem describes what happened next plainly: because PyPI took precedence over the project's private index, pip installed the attacker's version instead, and anyone who ran pip install for PyTorch nightly between December 25 and 30 got a package that harvested system data and sensitive files and exfiltrated them over DNS. Nothing about PyTorch's model architecture was compromised. The attack never touched a model. It touched the install path everyone trusted to bring the model in.

TL;DR

• Most documented AI supply chain attacks compromise the install or load path around a model, not the model's architecture or weights.
• The PyTorch torchtriton incident exploited dependency confusion between a public and a private package index.
• Hugging Face pickle-format models let an attacker embed code that runs automatically when the file loads, not when it is executed.
• Neither attack class is visible on HTTP traffic to a model API, because both happen before any inference call is made.

Two incidents, one pattern

The second documented case runs the same pattern through a different mechanism. In March 2024, JFrog's security research team found roughly 100 malicious model files on Hugging Face, most of them PyTorch checkpoints saved in Python's pickle format. Pickle is a general serialization format, not one built for model weights specifically, and it lets an author attach a __reduce__ method that runs arbitrary code the moment the file is deserialized. A data scientist who downloaded one of those files and ran torch.load() got a reverse shell before a single inference call happened, because loading the file was the execution.

Both incidents share the same shape. The attacker did not need to defeat a model's training, fool its guardrails, or craft an adversarial prompt. They needed one thing: to get their code onto a machine through a path the victim trusted by default, a package name or a model file. That is a far simpler attack to pull off than poisoning a model's training data directly, and it is why supply chain sits inside OWASP's Top 10 for LLM Applications as a standing category rather than a rare edge case. The nine components of an LLM system that can be compromised this way, and which ones get verified before deployment versus at runtime, are their own subject.

Why this stays invisible to AI traffic monitoring

Neither incident produced an anomalous prompt, an unusual model response, or any signal on the HTTP path between an application and an LLM API. The torchtriton package ran during pip install, on a developer's machine or a CI runner, before any model was even instantiated. The Hugging Face pickle payloads ran during torch.load(), on whatever machine pulled the checkpoint down, again before an inference request existed. A monitoring tool watching AI API traffic for anomalies has nothing to watch, because the compromise happened upstream of the first API call.

This is a boundary DeepInspect states plainly rather than talks around. Package installation and model file deserialization are local execution events. They do not flow through HTTP traffic between an authenticated caller and an LLM, and a policy gateway sitting on that traffic has no visibility into a pip install or a torch.load() call. Securing that stage is dependency scanning, package pinning, and safe-loading formats like Safetensors, which stores tensor data without executable code, done by the platform and MLOps teams who own the build and deployment pipeline. The install path for a third-party MCP server carries the same trust problem in a different wrapper, reviewed in detail here.

The narrow claim that does hold

Once a model, compromised or not, is actually being called, that call is HTTP AI traffic, and that is inside the boundary. A supply chain compromise that swaps which model or endpoint actually handles a request, whether through a poisoned dependency that changes which inference server gets called or a misconfigured deployment that silently points at an unapproved model, produces a request to a destination that identity-aware policy can evaluate and a per-decision record can capture.

The claim is specific: DeepInspect does not detect a poisoned package or a backdoored pickle file. It can enforce that only approved, pinned model endpoints are reachable from a given identity or workload, and it produces a record of exactly which model and version handled every request. If a compromised dependency ever tried to redirect inference traffic to an attacker-controlled endpoint disguised as the real one, that redirection is a policy violation on the request path, and DeepInspect is positioned to catch it there, even though it could never have caught the original package compromise.

DeepInspect

This is the layer DeepInspect adds once the upstream supply chain question is someone else's job to answer well. DeepInspect sits inline between authenticated users and agents and the LLM APIs they call, evaluating every request against identity, policy, and the specific model endpoint it targets, and writing a signed audit record of the outcome.

Model provenance, package integrity, and safe deserialization stay outside this boundary and belong to the teams that own the build pipeline. What stays inside it is confirming, on every call, that the model actually being reached is the one the organization approved, and keeping a record independent of the application for when that question gets asked after the fact.

If your organization can name every package that reaches your training and inference pipeline but cannot name every model endpoint your applications are actually calling in production, that second gap is the one worth closing next. Book a demo today.