Policy Puppetry Attacks Turn Policy Syntax Against Model Guardrails
Policy Puppetry wraps harmful intent in policy-like syntax and role-play so frontier models treat the payload as authoritative structure. Defenses need adversarial testing, request classification, identity-bound policy, constrained routes, and response inspection outside the model.

Policy Puppetry formats a jailbreak as a policy or configuration document, or as a role-play structure. HiddenLayer published the technique on April 24, 2025, describing a transferable prompt that targeted major frontier models and several alignment approaches. The payload borrows the visual and linguistic authority of policy syntax to steer the model past its refusal behavior.
TL;DR
- Policy Puppetry disguises adversarial intent inside policy-like syntax and role-play.
- Model refusal is probabilistic, so production controls need an independent request decision.
- Test normalized payloads and encoded variants, plus multi-turn forms and response leakage on every approved route.
The wrapper supplies false authority
A model sees tokens and learned patterns, not the organizational provenance behind a policy block. XML-like tags and nested configuration blocks with role declarations can resemble privileged instructions even when an untrusted user supplied them. Policy Puppetry combines that false authority with role-play to reshape the model's interpretation of the request.
The visual detail is almost banal: a box of tidy key-value fields looks safer than an angry paragraph. That neatness is part of the attack. Security review should focus on the requested behavior, not the prompt's costume.
Jailbreaks are part of OWASP's LLM01:2025 Prompt Injection. The underlying weakness is the model's unreliable boundary between instructions and data.
Detection should normalize before classification
Inspect both raw and normalized forms. Decode common encodings and unfold nested structures. Preserve role labels and retain boundaries between user content and trusted instructions. A detector should look for instruction hierarchy changes and requests to reveal system material. It should also flag policy documents that grant themselves permissions.
Normalization also creates risk if it destroys evidence. Store the original payload hash and the normalized representation used for classification. A later review needs to reproduce the decision exactly.
The broader LLM jailbreak monitoring program should include Policy Puppetry as a named test family rather than one generic malicious-prompt sample.
External policy limits the consequence
Request controls can constrain which authenticated roles may use sensitive models, what data classes may enter, which routes support high-risk capabilities, and when those routes may operate. Response controls can stop known secret classes and prohibited data from returning. Agent tool calls require another authorization layer after the model responds.
I consider a model refusal a useful signal, never an enterprise authorization decision. The model provider owns its safety behavior. The enterprise owns the rule that decides which traffic may cross its boundary.
This is the architecture in model guardrails versus security controls: safety training and deterministic enforcement have different jobs.
Regression tests need syntax diversity
Build a corpus with policy formats, nested tags, role-play, encoded content, and multi-turn variants. Run it against each approved model and model version. Record the request and route, then capture the model build, refusal behavior, policy decision, and response classification.
A successful provider-side fix can regress after a model update. An external control should keep the same organizational rule even as model behavior changes, with comparable decisions retained across model versions.
DeepInspect
DeepInspect can inspect authenticated HTTP requests and responses routed between enterprise users or agents and LLM endpoints. The policy decision can combine identity, route, model, data classification, and detected jailbreak signals before the request reaches the provider.
DeepInspect supplies the external enforcement record for traffic that deliberately crosses its proxy, alongside the separate evidence produced by model training and provider guardrails.
Book a demo today.
Frequently asked questions
- A user may submit the wrapper directly, while an application may place the same text into a prompt after retrieval or a tool call. The syntax can look identical at the model boundary. Its provenance changes the policy decision.
Start with the request path. Record where the content entered, which authenticated identity initiated the transaction, which route carried it, and what normalization changed before classification. That record lets a reviewer separate a direct jailbreak attempt from untrusted material that an application inserted into the model context.
The response deserves the same attention. A blocked request shows that the request control worked. An allowed request with prohibited output exposes a different gap, so response inspection should preserve its own decision and evidence rather than inheriting the request verdict. Shortcuts here make later review unreliable.
When a provider changes model behavior, rerun the same normalized and original payloads on each approved route, compare both decisions, and retain the response classification. This produces evidence for the enterprise rule even when the model starts refusing one variant and accepting another.
- Is Policy Puppetry a direct prompt injection?
The published technique is delivered through the prompt and seeks to bypass model safety behavior. In an application, the same syntax could also arrive indirectly through retrieved or tool-supplied content. Preserve provenance so the control can distinguish those paths.
- Can a classifier detect every variant?
No single classifier is a complete guarantee. Combine normalized-content signals with identity and route restrictions. Add response inspection, rate limits, and model-specific regression tests.