← Blog

Sponge Examples Make Worst-Case LLM Cost a Security Requirement

Parminder Singh
Parminder Singh··4 min read
Summarize with AI

Sponge examples are crafted inputs that maximize neural-network energy use or latency. LLM operators should test worst-case inputs, enforce route-specific size and time budgets, cap concurrency, preserve request evidence, and stop repeated expensive payloads before inference.

Problem-Awareai-securityllm-securitypolicy-enforcementarchitecturecybersecurity
Sponge Examples Make Worst-Case LLM Cost a Security Requirement

Average inference latency hides an adversarial tail. The 2020 paper Sponge Examples: Energy-Latency Attacks on Neural Networks introduced crafted inputs designed to maximize energy consumption and decision latency across language and vision models. The security lesson for LLM services is straightforward: a valid input can still be an availability attack.

That distinction changes the test target. Normal load testing asks how the service behaves under more requests, while sponge testing asks how much work one accepted request can force. Both matter, but they expose different failure modes.

TL;DR

  • Sponge examples exploit worst-case model execution rather than malformed requests.
  • Test latency and resource use by route and model, then repeat across input classes and hardware under adversarial inputs.
  • Enforce budgets before inference. Block repeated payloads using identity and content fingerprints.

Valid syntax can carry hostile cost

Web application controls often reject oversized or malformed requests. Sponge examples can stay within accepted syntax while provoking expensive internal behavior. That moves the problem beyond a simple body-size limit.

A dashboard showing a healthy median can miss the attack. One narrow band at the far right of the latency histogram may contain the requests draining accelerator time. The original research argues for worst-case analysis rather than relying on average-case performance.

The same gap appears in capacity planning. A route may satisfy its ordinary latency target all week and still have no safe margin for a deliberately expensive input. Operators need to know where that margin ends before an attacker finds it.

OWASP's Unbounded Consumption category connects excessive inference to denial of service and service degradation. It also identifies economic loss. Sponge inputs supply one mechanism for that risk.

Establish a route-specific cost envelope

Define separate maxima for input and output units. Set wall-clock duration and concurrency limits, plus a cumulative spend cap for each authenticated route. A batch summarizer and an interactive assistant have different envelopes. Model choice and hardware change the baseline. Quantization does too.

I distrust a single global timeout. It punishes legitimate slow routes while leaving fast routes with unnecessary attack room. Build limits around the actual task class and keep the exception process visible.

An envelope also needs a clear exhaustion action. Rejecting new work protects capacity, while queueing it can simply move the bottleneck out of sight. The policy should say which action applies to that route and record the reason at the point of enforcement.

The enforcement design belongs beside AI gateway rate limiting, with model-aware cost signals added to request counts.

Test the tail before production

Create adversarial performance tests that search for inputs producing unusual latency or output expansion. Check memory pressure and accelerator occupancy in the same runs. Run them against every production model version and serving configuration. Record the request hash and model build. Also record the hardware class and duration alongside the resource counters.

A provider update can change the worst-case profile without altering the API schema. Repeat the test after model, tokenizer, runtime, or hardware changes. Keep a small physical artifact from each run: the top latency traces with the exact payload hashes and policy outcomes.

The useful result is a boundary, not one headline latency figure. Compare the slowest accepted inputs with the route budget, inspect which resource reached its limit first, and retain enough evidence to reproduce the decision. If a later build crosses that boundary, the release has a concrete reason to stop.

Runtime controls need identity and fingerprints

Rate limits by IP are weak when one enterprise workload sends all traffic through a shared egress address. Bind budgets to the authenticated user, agent, tenant, route, and model. Fingerprint normalized payloads so repeated expensive inputs can trip a circuit breaker across sessions.

Do not automatically retry a request that ended at the compute deadline. A blind retry doubles the pressure. Quarantine the payload and preserve the request evidence. Require a deliberate replay under a test budget.

A fingerprint should support policy without becoming the only signal. Similar expensive inputs may differ in superficial formatting, and unrelated requests can share a cost profile. Identity, route context, observed cost, and recent policy outcomes give the circuit breaker a firmer basis.

This evidence integrates with AI runtime security controls without asking the model to police its own resource use.

Are sponge examples specific to LLMs?

The original research spans language and vision models and focuses on neural-network energy and latency. LLM deployments inherit the same worst-case analysis problem, with token billing and shared accelerator queues adding operational consequences.

Can output-token limits stop the attack?

They constrain one cost dimension. Input processing, hidden reasoning, memory pressure, and pre-output latency may still dominate. Combine output limits with input, time, concurrency, and cumulative cost controls.

DeepInspect

DeepInspect can apply route-specific policy to authenticated HTTP LLM requests before inference. The decision can use identity, model, input classification, payload size, application-supplied budget state, and prior denials.

Accelerator scheduling and model internals remain provider or platform responsibilities. DeepInspect governs the routed request and records the external decision that allowed or denied it.

Book a demo today.