← Blog

Bad Likert Judge Jailbreak Turns Evaluation Into the Attack

Parminder Singh
Parminder Singh··7 min read
Summarize with AI

Bad Likert Judge asks an LLM to score harmful responses on a Likert scale, then generate examples that match the ratings. Unit 42 tested 1,440 cases across six anonymized models and found a large increase in attack success. The deployment response needs prompt and response inspection, scoped authorization, and evidence at the routed HTTP boundary.

Problem-Awarellm-securityai-securityprompt-injectionllmpolicy-enforcement
Bad Likert Judge Jailbreak Turns Evaluation Into the Attack

Bad Likert Judge turns a familiar evaluation task into a multi-turn jailbreak. An attacker asks an LLM to score responses by harmfulness, then asks it to generate examples that fit each point on the scale. At the highest rating, an example can contain material the model would refuse as a direct request. The framing matters because a prompt that looks like safety analysis can become the route to prohibited output.

TL;DR

  • Bad Likert Judge asks an LLM to rate harmfulness, then generate examples aligned with the rating scale.
  • Unit 42 tested 1,440 cases across six anonymized models and reported an average attack-success increase above 75 percentage points.
  • Defenses need full-conversation inspection, response policy, scoped authorization, and a reproducible decision record.
  • DeepInspect can enforce policy on deliberately routed HTTP model calls; provider training and off-route traffic remain separate responsibilities.

Bad Likert Judge misuses the model's evaluation ability

Palo Alto Networks Unit 42 published the Bad Likert Judge research on December 31, 2024. The first turn asks the target model to act as a judge. It must describe a Likert scale for the harmfulness of a response, with low values representing safer material and high values representing more harmful material.

A later turn asks the model to produce examples that correspond to the scale. The model now treats generation as part of an evaluation exercise. Unit 42 found that the example at the highest rating could contain the harmful content that a plain request failed to elicit. One or two follow-up turns asking for more detail sometimes pushed the output further.

Picture the transcript in a review console. The opening panel contains a numbered scale and clinical language about classification. The harmful passage appears lower down, under what looks like a scoring exercise. Reviewing only the first prompt or the final sentence misses how the conversation changed the task.

The 1,440-case test measured a sharp increase

Unit 42 built a dataset of 1,440 cases across jailbreak categories and tested six anonymized text-generation models. The researchers first sent plain attack prompts to establish a baseline, then applied Bad Likert Judge to the same prompt set and compared attack success rates.

The reported average increase exceeded 75 percentage points in the detailed evaluation. Unit 42 also stated that the method increased attack success by more than 60% compared with plain prompts on average. Those figures describe the tested models and categories under the judge and December 2024 method. They do not establish a permanent rate for every current model or deployment.

The researchers deliberately withheld provider names. They also said the study did not exhaust every model and described these attacks as edge cases rather than typical use. That limitation belongs beside the headline number. A security team should reproduce the method against its exact model version and system prompt, then verify its content filters and application route before treating the published result as its own exposure rate.

Single-message filters miss the conversation mechanism

Bad Likert Judge develops across turns. A filter that evaluates each message without the conversation history sees an evaluation request, a rating scale, or a request for examples. Each fragment can look less suspicious than the cumulative instruction.

The general jailbreak defense guide already covers multi-step persuasion as a class. Bad Likert Judge adds a specific regression case: benign-looking evaluation language can induce the model to manufacture the content it was asked to judge. Detection should therefore inspect the current prompt with the relevant conversation state and requested model, along with the route and caller identity.

A literal signature for the words "Likert scale" would create noise. Survey analysis and research workflows, including trust-and-safety testing, use that phrase legitimately. Better policy considers two stages: a harmfulness taxonomy with generated examples at escalating ratings, followed by a request to expand the highest-rated response. Even then, a probabilistic classifier can miss a rewritten variant. Authorization and response controls still matter.

Response inspection catches what input policy misses

Unit 42 evaluated external content filtering after testing the models' internal guardrails. With the strongest available settings in its setup, filtering reduced average attack success by 89.2 percentage points across the tested models. The researchers also warned that false positives or false negatives remain possible and determined attackers can find new bypasses.

That result supports two inspection points. Input policy can block a suspicious multi-turn sequence before inference. Response policy can stop prohibited output before it reaches a user or a downstream system, including another model or an application component. The second point matters when the request looks like legitimate evaluation work but the generated example crosses the policy boundary.

OWASP's LLM01:2025 Prompt Injection guidance recommends input and output filtering, constrained model behavior, least privilege, human approval for high-risk actions, and adversarial testing. OWASP also states that foolproof prevention remains unclear because generative models respond stochastically. A production control plan needs several independent layers rather than a promise that one detector will recognize every wording.

Authorization limits the consequence of a bypass

Jailbreak detection decides whether the prompt or response resembles prohibited content. Authorization decides what an authenticated caller may do on a specific route. Those decisions should meet before the model call.

A red-team route may permit controlled safety testing under a named role and restricted dataset, using an approved model. A customer-support route should carry narrower content and tool permissions. If the same Bad Likert Judge sequence appears on both routes, policy can produce different outcomes because the identities and approved purposes differ.

The distinction between prompt injection and jailbreaking also stays useful here. Bad Likert Judge targets provider safety behavior. If an application gives model output authority to call tools or interact with customer records and messaging systems, those actions need separate deterministic authorization. Blocking harmful text cannot replace tool policy, and tool policy cannot repair a model that returns prohibited content to a user.

I would reject broad model access under one shared API credential during architecture review. A failed model refusal becomes much more consequential when the route carries no identity context or approved purpose that policy can evaluate.

The audit record should preserve the whole decision

A confirmed bypass should become a regression case. Record the supplied identity and role, the route and model destination, the policy version and conversation fingerprint, the matched rule, both request and response decisions, and the timestamp. Store harmful prompt and response material under restricted access when retention policy permits it.

The visual artifact should resemble a two-sided ledger. On the left, each turn is ordered with its role and fingerprint. On the right, the policy engine records allow, redact, or block decisions at the exact turn where the risk changed. A single log line saying "request denied" cannot explain why a rating exercise crossed the boundary.

Run the case again after a model upgrade or system-prompt change. Repeat it after a classifier update, route-policy edit, or context-window change. The inline enforcement argument applies directly: a forensic alert after prohibited output reaches the next component documents the failure without preventing it.

Bad Likert Judge has a precise control boundary

The attack reaches the model through prompts and returns through model responses, so a deliberately routed HTTP model call provides a credible inspection point. The gateway can evaluate the conversation content and supplied identity against the destination and route policy before forwarding the request. It can also inspect the returned text before the application consumes it.

Several responsibilities remain outside that inspection point. The model provider owns training and internal safety tuning. The calling application must supply trustworthy identity context. A local model call and a direct provider request that bypasses the approved route require separate controls. So does tool execution on another protocol. The gateway's record proves what crossed its own path, not what happened everywhere in the system.

This boundary makes the acceptance test concrete. Send a controlled Bad Likert Judge sequence through the approved route and confirm the configured policy blocks or redacts it with an identity-bound record. Then send the same sequence outside the route in an isolated test environment to verify the egress control detects the bypass. One test validates policy behavior; the other validates route coverage.

DeepInspect

DeepInspect evaluates HTTP AI traffic deliberately routed between authenticated users or agents and LLM endpoints. For Bad Likert Judge, policy can inspect the evolving prompt sequence before inference and the model response before it returns to the application. The decision uses the supplied identity and role, the route and destination, and the content classification, then produces a per-decision audit record.

Provider training and local inference retain separate owners. So do application identity proofing and traffic that bypasses the approved HTTP route. DeepInspect covers the requests and responses sent through its stateless proxy. A complete deployment also needs egress controls that force model traffic onto that path.

Book a technical deep dive at deepinspect.ai.

Frequently asked questions

Is Bad Likert Judge a prompt injection or a jailbreak?

Bad Likert Judge is a multi-turn jailbreak because it tries to bypass the model provider's safety behavior and elicit prohibited output. OWASP describes jailbreaking as a form of prompt injection aimed at making the model disregard safety protocols. Application prompt injection can share techniques, but its objective often involves overriding application instructions or exposing enterprise data, with connected-tool abuse as another possible objective.

Does blocking the phrase Likert scale stop the attack?

A phrase block would also catch legitimate surveys and evaluation work, including safety research. Attackers can rename the scale or restructure the turns. Inspect the conversation pattern and route, the identity and requested action, and the response category. Keep response policy and authorization in place when the input classifier permits the request.

What did Unit 42 test?

Unit 42 tested 1,440 cases across six anonymized text-generation models and several jailbreak categories. The study compared plain attack prompts with the Bad Likert Judge method. Its detailed evaluation reported an average increase above 75 percentage points. The executive summary described an increase above 60% compared with plain prompts on average.

Can content filtering eliminate this jailbreak?

Unit 42 reported a large reduction under the strongest filter settings in its experiment. The researchers explicitly retained false-positive and false-negative caveats, along with warnings about adaptive attackers. Treat filtering as one layer alongside scoped authorization, response inspection, route control, restricted tool permissions, and regression testing.