← Blog

QR Code Prompt Injection Hides Instructions in the Image Path

Parminder Singh
Parminder Singh··4 min read
Summarize with AI

A multimodal agent can decode a QR code inside an image and treat the recovered text as instructions. Control the image path with provenance, decoder output inspection, task-scoped tool permissions, model-route policy, and records that connect the original image to the model call.

Problem-Awareai-securityprompt-injectionagentic-aillm-securitypolicy-enforcement
QR Code Prompt Injection Hides Instructions in the Image Path

A QR code turns a square image into machine-readable text. When a multimodal agent processes that image, the decoded payload can enter the same context used for planning and tool selection. A September 2026 experimental study of multimodal prompt injection tested QR codes alongside OCR text, overlays, metadata, and fake interfaces. It also tested hybrid carriers and adversarial documents across agent frameworks.

TL;DR

  • Treat decoded QR content as untrusted data, even when the surrounding image came from an approved workflow.
  • Preserve the original image and decoded payload through prompt assembly. Keep provenance labels and the classification result attached to them.
  • Constrain the resulting model request and require separate authorization for any tool action.
  • Test the same image through the complete production path, not only against the final model.

The image contains a second input channel

A user may upload a receipt, badge, slide, or equipment photo for a legitimate task. The QR code inside it can carry text the user never typed. OCR and vision preprocessing may expose that text to the model without a visible confirmation step.

A black-and-white square in the corner of a shipping label looks like ordinary logistics metadata. To the agent, it may be a compact instruction asking for a data request, a URL fetch, a message, or a tool call. OWASP's prompt-injection guidance covers multimodal attacks and imperceptible inputs because model parsing, rather than human readability, defines the exposure.

The mechanism fits the wider multimodal prompt-injection surface, but QR decoding deserves a separate inspection artifact.

That artifact should show where the payload entered the workflow. It should also show which later components received it. Without that chain, a reviewer sees an image at ingestion and a model response at the end, with no reliable view of the decoded text between them.

Decode before the model sees the image

Run a deterministic QR decoder during ingestion and expose the result to policy. Record the image hash, decoder version, payload bytes, detected character set, source, and uploader identity. Keep decoded text in a marked data field rather than blending it into trusted instructions.

A URL in the payload starts a new network decision. Classify any instructions before prompt assembly. When decoding fails, keep that uncertainty visible instead of silently sending the raw image to a model with broader visual capabilities.

Store an explicit outcome for images with no detected code. This distinguishes a completed scan from a skipped decoder or an unsupported image format. It also exposes a processing error or a timeout when an incident reviewer reconstructs the request.

My rule would irritate product teams: a newly discovered QR code pauses an autonomous workflow until its payload is classified. One extra confirmation beats an agent following a square nobody read.

Tool authority determines the damage

A decoded instruction has limited impact when the agent can only summarize the image. Risk rises when the same agent can browse or send messages. It may also retrieve secrets or modify records. Apply task-specific allowlists and require an authorization decision for every sensitive action.

The model request should include the image's provenance and decoder classification in application-supplied context. That allows the HTTP policy layer to deny an untrusted carrier on a sensitive route. The action layer then decides if any requested tool call is permitted.

This division matches AI agent permission controls: model traffic and external actions need linked but separate decisions.

The two records should share a request identifier. A policy reviewer can then connect the model decision to a later tool attempt without assuming that an allowed model request also authorized the action.

Testing needs real carrier variants

Create regression images with a clean QR baseline, rotation, low contrast, and partial occlusion. Add nested screenshots and document embedding. Test the full production chain, including resizing, OCR, vision preprocessing, and prompt assembly. Continue through model routing and tool authorization.

Store the decoded payload and final assembled context for each run. A test that records only the final answer misses attempted actions and intermediate planning, which are often where the injected instruction first appears.

Include expected results for every control point. The decoder should report what it found, prompt assembly should preserve the untrusted label, route policy should record its decision, and tool authorization should produce a separate outcome. Then repeat the case after resizing or compression. Small image-processing changes can alter what reaches the model, so a passing decoder-only test does not certify the deployed workflow.

DeepInspect

DeepInspect can evaluate the authenticated HTTP model request after an application decodes a QR code and assembles multimodal context. Identity, route, model, and data class can inform the decision. Application-supplied carrier provenance can inform it too.

Image parsing and tool execution remain separate controls. DeepInspect governs the LLM-bound request and response that the application deliberately routes through the proxy.

Book a demo today.

Frequently asked questions

Is scanning a QR code enough to make it safe?

Scanning reveals the payload, while safety depends on what the application does with that text. Classify it and label it as untrusted data. Restrict model routes and authorize subsequent actions separately.

Can text-only gateways handle QR injection?

They can govern decoded text included in an HTTP model request. Image decoding and visual analysis happen upstream unless the gateway supports that modality. Pass the decoder result and provenance into the policy context.