Does this attack require the attacker to be a paying customer of the platform?

In the scenario the NDSS paper demonstrated, yes: the attacker needs their own authorized access to the shared inference system to send probe queries and time the responses. This makes the attack realistic for any multi-tenant SaaS product with self-service signup, since becoming an "attacker tenant" costs nothing more than creating an account. It is a meaningfully different threat model than an external attacker with no platform access, and it is one reason the finding matters for product teams: the barrier to attempting this is a free trial signup, not a breach.

Are all multi-tenant LLM deployments vulnerable to KV-cache timing attacks?

Only deployments that share KV-cache state across tenants are exposed to this specific technique. A platform that partitions cache per tenant, or one that does not share prompt prefixes across tenants in a way that creates cache hits, removes the exploitable signal. Whether a given platform shares cache state across tenants is an infrastructure design decision that is rarely disclosed publicly, which is part of why this class of risk is under-discussed relative to its severity: customers of a multi-tenant AI product generally cannot tell from the outside whether this exposure exists.

Does using separate API keys per tenant prevent cross-tenant leakage?

Separate API keys establish authentication and billing attribution. They do not, by themselves, guarantee infrastructure-level isolation, because a provider can authenticate two different API keys correctly while still routing both tenants' requests through the same shared inference backend and cache. Isolation at the credential layer and isolation at the serving-infrastructure layer are different guarantees, and a platform's marketing language about "isolated" tenants needs to specify which layer that isolation actually applies to.

What should a platform team ask a model provider or self-hosted inference vendor about this risk?

Ask directly whether KV-cache or prefix-cache state is shared across tenants or customers, and if so, what mitigation is in place, whether that is cache partitioning by tenant, timing-noise injection, or another documented approach. A vendor who has not considered the question, or who cannot describe a specific mitigation, is a meaningful signal on its own, since the underlying research has been public since 2025 and mitigations exist.

Cross-Tenant AI Leakage Does Not Require a Bug in Your Application

A 2025 NDSS paper titled I Know What You Asked: Prompt Leakage via KV-Cache Sharing in Multi-Tenant LLM Serving demonstrated an attack called PromptPeek: an attacker tenant on a shared LLM serving system reconstructs another tenant's prompt, token by token, by measuring how fast their own queries return. LLM inference servers cache intermediate computation, the key-value cache, to speed up requests that share a prefix. When two tenants share that cache to save compute, a cache hit responds faster than a cache miss, and that timing difference alone is enough for an attacker to guess the victim's prompt one token at a time and confirm each guess by watching the clock.

TL;DR

Cross-tenant AI leakage can happen through shared inference infrastructure, not application-layer misconfiguration.
A 2025 NDSS paper demonstrated reconstructing another tenant's prompt through KV-cache timing side channels alone.
The vulnerable layer sits inside the model-serving stack, below anything an API caller or application developer controls.
Tenant-scoped policy at the request layer limits blast radius but cannot fix a cache-sharing design decision made inside the inference server.

The leak lives below the application

Most data leakage discussions in multi-tenant SaaS focus on the database layer: row-level security, tenant IDs on every query, access control checks before a record returns. Teams that get those controls right can still ship a cross-tenant AI leak, because the KV-cache side channel documented in the NDSS paper has nothing to do with database access control. OWASP's Vector and Embedding Weaknesses category names cross-tenant infrastructure sharing as one of the failure modes this class of risk covers, distinct from the access-control failures most security reviews already check for. It exploits a performance optimization inside the model-serving infrastructure itself, present specifically because the platform is trying to serve multiple tenants efficiently on shared GPU capacity.

The attack requires no credential theft and no application vulnerability. An attacker who is a legitimate, paying tenant on the same shared infrastructure as the victim can run the technique using only their own authorized access, because the side channel is a property of how the serving stack handles overlapping prompt prefixes across all tenants using it.

Why efficient serving and tenant isolation pull in opposite directions

Sharing a KV-cache across tenants exists for a real reason: it cuts inference cost and latency substantially when multiple requests share a common prefix, which is common in multi-tenant products where many customers use the same system prompt or a similar template. The efficiency gain is why platforms adopted the pattern before the security cost was as well understood as it is now.

The tradeoff is structural. A cache that never crosses a tenant boundary gives up the efficiency gain entirely, defeating the reason for sharing infrastructure in the first place. A cache that shares freely across tenants, as the NDSS paper showed, leaks a timing signal that an attacker can exploit without ever seeing the victim's actual traffic. Mitigations exist, including partitioned caching schemes and techniques that add controlled timing noise, and the research community has moved quickly on this since the paper's publication, but the underlying tension between shared-infrastructure economics and tenant isolation has not gone away.

Why this sits outside an HTTP policy gateway's reach

The KV-cache lives inside the inference server, below the HTTP API layer entirely. A request arrives over HTTP, gets tokenized, and only then interacts with the cache during the model's forward pass. Nothing about that internal caching behavior is visible on the HTTP request or response DeepInspect inspects, and no policy evaluated at the API boundary can see or influence what the inference engine does with its own cache internally. This is squarely a model-serving infrastructure decision, owned by whoever operates the inference stack, whether that is a model provider or a platform team running self-hosted inference.

Saying that plainly matters more than gesturing at a broader security posture, because a security team that believes an AI gateway addresses this risk will not push the inference platform team toward the cache-partitioning fix that actually closes it. The same discipline applies to securing a multi-agent system: naming what an identity-aware boundary can and cannot reach is what keeps a control mapping honest.

What tenant-scoped policy at the request layer does provide

The narrower, honest claim is about blast radius and evidence, not prevention of the underlying side channel. Every request that reaches a shared inference backend can be evaluated and logged against the specific tenant identity that originated it, before the request leaves the organization's own boundary. That does not stop a cache-timing attack happening inside a third party's serving infrastructure. It does mean that if a leak is suspected, there is an independent, tenant-scoped record of exactly which requests a given identity sent and when, which is the evidence an incident investigation needs regardless of where in the stack the leak occurred.

For platform teams running their own inference infrastructure rather than calling a third-party API, the request-layer control and the cache-partitioning fix are complementary: one limits which requests reach the shared backend at all under which tenant's policy, and the other fixes the sharing behavior once a request gets there. Self-hosted deployments carry their own patterns for this, covered in self-hosted Llama gateway patterns.

DeepInspect

DeepInspect sits inline between authenticated users, agents, or platform services and the LLM APIs they call, evaluating every request against the identity and tenant context supplied by the calling application. For a multi-tenant SaaS platform, that means every outbound call to a shared model backend carries an enforced, auditable link to the tenant it belongs to, and a signed per-decision record of what was sent and returned.

This is the same identity-bound evaluation described in how inline enforcement works against AI traffic broadly, applied specifically to the tenant-attribution problem multi-tenant platforms carry. It is a control on the request path into a shared model, not a fix for what the model-serving layer does with a cache once a request arrives there. Platforms serious about cross-tenant isolation need both: tenant-scoped policy at the boundary DeepInspect covers, and cache-isolation engineering inside the serving stack that only the platform's own infrastructure team can implement.

If your AI product serves multiple tenants against shared model infrastructure and your only isolation control lives in the application database layer, that gap is worth closing. Book a technical deep dive at deepinspect.ai.