ShieldGemma: A Production Acceptance Guide for Image Safety

ShieldGemma turns a supplied safety policy and content into a classification signal. Google positions the family as one component inside a larger generative AI application. ShieldGemma 2 applies that pattern to images, returning probabilities for Yes and No tokens under a policy definition. Production approval therefore needs a model-policy acceptance record. A download receipt and a green label leave the threshold and tested content unresolved, along with the failure action and responsible reviewer.

TL;DR

• ShieldGemma 2 is an image-safety model with 4 billion parameters that scores content against a supplied policy and returns Yes or No token probabilities.
• Pin the model artifact, policy text, threshold, and prompt format before production use. Pin the serving stack and labeled acceptance corpus too.
• Treat its output as a safety signal. Authorization and route policy need separate owners and records, as do human escalation and downstream actions.
• DeepInspect can govern a deliberately routed HTTP call to ShieldGemma, while local inference and image-model quality remain outside that boundary.

ShieldGemma is a policy classifier

Google's ShieldGemma overview describes an open-weight model family for evaluating prompt inputs and model outputs against defined safety policies. ShieldGemma 1 covers text in 2B and 9B sizes, plus a 27B version. ShieldGemma 2 is a 4B model built on Gemma 3 for image safety. Google recommends it as an input filter for vision-language models or an output filter for image-generation systems.

The ShieldGemma 2 model card specifies the actual interface. Its input combines an image and an instruction under a policy definition. Output contains probabilities for Yes and No tokens. A higher Yes score signals greater confidence that the image violates the supplied policy. Both scores support calibration.

That mechanism gives the deployment team policy language and model output before the decision threshold is applied. I would reject a review document that records only "ShieldGemma enabled." The useful record names the exact artifact and policy, then shows which score became which action.

Pin the deployment identity

Start with an artifact digest or immutable repository revision. Record the 4B ShieldGemma 2 build and its tokenizer and image processor. Add the quantization and serving engine. Record the container image, accelerator, dependency lock, and prompt format too. Add the policy checksum and the threshold configuration. The acceptance pack should let a second engineer reproduce the classifier without reading the first operator's shell history.

Google's model card says the training data included natural images sampled from WebLI and synthetic images generated through an internal pipeline. The documented harm types were dangerous and sexually explicit content, along with violence, with English-only policy material. Those boundaries belong in the deployment record. A local policy taxonomy can add categories, but the team must test them rather than infer coverage from the model name.

Put one printed acceptance page beside the deployment diagram during review. It should show the image source, model digest, policy revision, and Yes and No scores. The selected threshold and action should appear on the same line, along with the reviewer. If those fields need three dashboards and a chat transcript, the evidence path is already too fragile.

Build a labeled acceptance corpus

Create a controlled corpus for each policy category. Include clear violations and clear allowed images, plus borderline cases and text embedded in images. Include screenshots, synthetic compositions, cropped images, low-resolution inputs, and benign material that shares visual features with prohibited content. Add the expected label and rationale. Record the source and sensitivity too, followed by the reviewer and permitted retention period.

Keep policy-development examples separate from held-out acceptance examples. The engineer tuning policy wording and thresholds can use the first set. Preserve the second set for approval. Review false positives and false negatives by policy category and consequence. One aggregate score can hide a serious miss inside a larger set of simple cases.

The model card warns that limited content-moderation benchmarks may fail to represent real production scenarios. It also says ShieldGemma 2 is highly sensitive to the user-provided description of safety principles and may behave unpredictably when language ambiguity or nuance matters. Add ambiguous and workload-specific cases to the held-out corpus. Run that corpus after every policy or model revision. Rerun it after changes to preprocessing, quantization, the serving engine, or the threshold.

The broader AI guardrails guide covers category placement. ShieldGemma's acceptance corpus owns a narrower question: which exact model-policy pair produced an acceptable classification record for this image workload?

Turn scores into controlled actions

Define the mapping outside the model. For example, a high violation score may block an image, while an intermediate band may route it to human review. A low score may permit the next processing step. Choose the bands from the held-out corpus and operational risk. Store the raw scores and model digest with the resulting action. Preserve the policy checksum and threshold version in the same record.

Test malformed output and a missing score, plus a server timeout and an oversized image. Test an unavailable accelerator too. The application needs a written response for each condition. High-risk routes may fail closed. Lower-risk internal workflows may quarantine the image for review. Silent continuation is a decision too, and it deserves an explicit owner.

A ShieldGemma label grants no permission to access a record or call a tool. It also grants no permission to send content to another model. It describes the policy classification produced for supplied content. The application still owns authorization and side effects. The argument in model guardrails are probabilistic applies here: deterministic handling around a model signal creates the operational control.

Keep the HTTP boundary precise

ShieldGemma can run locally or behind an HTTP endpoint, including inside a private service. DeepInspect enters the picture only for authenticated user or agent-to-LLM HTTP calls deliberately routed through its proxy. If an application exposes ShieldGemma as a model endpoint and sends the request through that route, a request policy can evaluate caller identity and route, plus destination and data classification, before forwarding the allowed call.

Local inference that never crosses the proxy remains outside DeepInspect. The same boundary excludes image preprocessing on disk and GPU isolation, plus corpus labeling and threshold quality. It also excludes a direct call that bypasses the configured route.

For a routed deployment, correlate two records. The request-policy record should identify the authenticated caller, route, policy version, and destination. It should also identify the action and timestamp. The ShieldGemma record should identify the image reference, model artifact, safety policy, and scores. It should also identify the threshold, classification, and review state. See the HTTP-layer policy guide for the first record. Keeping both prevents a safety score from being mistaken for access authorization.

DeepInspect

DeepInspect can govern an authenticated HTTP call to ShieldGemma when the customer exposes the classifier through an LLM endpoint and deliberately routes that call through the stateless proxy. It evaluates identity and route, plus destination and request data, before forwarding allowed traffic, then writes a per-decision record.

ShieldGemma still owns the safety classification. The deployment team owns the artifact, policy wording, corpus, and threshold. It also owns image handling and the review process. Local inference and direct bypass calls stay outside DeepInspect. Book a technical deep dive at deepinspect.ai.