AI-Generated Code Vulnerabilities Need a Production Acceptance Gate

AI-generated code can compile and pass a narrow unit test while introducing an authorization bypass or unsafe deserialization path. It may also expose a secret or add a dependency with a known flaw. GitHub's responsible-use guidance says generated code may appear valid while remaining semantically incorrect or insecure, and it tells users to review and test the output before merge. I think approving generated code on the strength of a clean model response is negligent. The production control is an acceptance gate that joins evidence from the model call and repository review with scanners and tests. The same record carries the exception workflow plus merge state and release evidence.

TL;DR

• Treat AI-generated code as untrusted input to the software delivery process, even when it compiles and looks idiomatic.
• Require human review, tests, SAST or CodeQL, dependency analysis, secret scanning, approved exceptions, merge protection, and release evidence.
• Preserve the routed model-call record as provenance for the generated diff. It is one input to the gate, never proof that the code is secure.
• Keep ownership explicit: the gateway governs the LLM request and response; repository and CI controls govern code acceptance and deployment.

Generated code inherits ordinary software risk

GitHub's responsible-use guidance for Copilot agents warns that generated code may be syntactically plausible while failing to reflect developer intent or carrying security vulnerabilities. GitHub recommends careful review, testing, secure-coding practice, IP scanning, and vulnerability checks before merge.

Generated code enters the repository as an untrusted candidate change. The model may omit an authorization check because the prompt lacked repository context. It may choose a stale package version, place a secret in a fixture, or repeat an unsafe pattern present in nearby code. A green diff with twelve added lines can hide a privilege boundary change more consequential than a 600-line feature.

The acceptance decision belongs at the point where the organization already controls software changes: the pull request and its protected merge path. The model-call record provides provenance, while the repository supplies context and CI runs repeatable checks. A release system proves which accepted commit reached an environment.

Define the acceptance record

Create one record for each AI-assisted change that reaches review. It should join these fields:

• Repository, branch, commit, and generated diff hash
• Acting developer or agent identity, model provider, model route, request ID, and timestamp
• Prompt-data classification and response-policy outcome
• Human reviewer and review disposition
• Test suite result and coverage delta where available
• SAST or CodeQL run and alert disposition
• Dependency and secret scan results
• Security exception with its approver, scope, and expiry
• Merge-protection result, merge commit, release ID, and target environment

Use stable identifiers rather than screenshots. A scanner page with a green banner proves little after the branch moves. The record should let an auditor retrieve the exact CodeQL run and diff plus the release artifact later.

Connect the record to the broader controls in AI security for coding agents, especially source-code exposure and delegated authority plus tool approval. This article owns the narrower decision that follows generation: the evidence required before generated code becomes production code.

Human review checks intent and authority

A reviewer should compare the generated change with the issue and acceptance criteria in the context of the repository security model. Automated scanners can find known patterns. They cannot reliably decide that a support role gained an administrator action because the prompt described the role incorrectly.

Require the reviewer to answer three concrete points in the pull request:

1. Name the trust boundary or permission path that changed.
2. Identify the negative test that proves an unauthorized caller remains denied.
3. List the generated files and dependencies plus any configuration changes that received separate review.

The reviewer must be independent of the agent that generated and submitted the change. For high-impact paths such as authentication, payment, secrets, deployment, and tenant isolation, route the pull request to a code owner with that domain. Keep the approval in the repository history so it travels with the commit.

I prefer a short reviewer note that names the tested boundary over a checkbox wall. "Tenant B receives 403 on GET /accounts/{tenantA} in test authorization_cross_tenant_07" is useful evidence. A bare "Security reviewed" label leaves the tested boundary unknown.

Automated checks cover different failure classes

NIST's Secure Software Development Framework, SP 800-218, recommends a core set of secure development practices intended to reduce vulnerabilities in released software, mitigate exploitation of flaws that remain, and address root causes. An AI-specific gate should sit inside that software development framework rather than forming a parallel lane with weaker controls.

Run each check for its actual job:

• Unit and integration tests confirm expected behavior and negative authorization paths.
• SAST or CodeQL identifies supported vulnerability patterns in source and data flow.
• Dependency analysis checks introduced packages and versions against known advisories.
• Secret scanning catches credentials and tokens committed in code or fixtures.
• Build and lint steps confirm the repository can reproduce the candidate artifact.
• DAST or environment tests examine running behavior where the release risk warrants it.

GitHub describes CodeQL code scanning as a way to identify vulnerabilities and errors by building a database of the codebase and running queries against it. A clean CodeQL run has a defined scope. Its selected queries found no unresolved result in supported code. It cannot certify business logic, runtime configuration, package provenance, or the absence of every vulnerability.

Exceptions need an owner and an expiry

Some findings are false positives. Others are accepted temporarily because a release fixes a more urgent exposure. The acceptance gate should support an exception without turning suppression into a permanent blind spot.

Record the alert ID, affected commit, rationale, compensating control, approver, expiry date, and linked remediation issue. Bind the exception to the narrow rule and code location where possible. A repository-wide dismissal such as "test code" can hide the next generated credential fixture under the same label.

Block merge when a required check is missing, a high-severity finding remains open without approval, the human reviewer is absent, or the exception has expired. Keep the agent from changing branch protection, dismissing its own alerts, or approving its own pull request. Those permissions are repository and CI controls outside the LLM request boundary.

Join model provenance to release evidence

The model-call record should include the authenticated caller, approved model route, data classification, policy result, and a response or diff reference. Hashing the generated diff gives the delivery system a stable join key without requiring the gateway to become the repository.

CI then attaches test and scan results to that diff or commit. Merge protection records the final approval state. The release system links the merge commit to the deployed artifact and environment. This creates a traceable chain across independent systems while preserving their roles.

The boundary prevents exaggerated claims. An LLM gateway can block source code from reaching an unapproved model and restrict which agent may use a coding route. It can also inspect generated output and log the call. It cannot write the repository review, run CodeQL, verify a dependency, execute CI, determine exploitability, merge the pull request, or deploy the artifact. The CI pipeline supply-chain guide covers those downstream controls in more detail.

DeepInspect

DeepInspect governs the authenticated HTTP call between a developer or coding agent and an LLM. It can authorize the coding route and classify source or secrets in the prompt. It also inspects generated code in the response and creates an independent record of the identity, model, policy, and decision.

Feed that record into the production acceptance gate as provenance. Repository review, tests, CodeQL or another SAST tool, dependency analysis, secret scanning, exceptions, merge protection, and release evidence still decide whether the code ships. DeepInspect supplies the request-layer evidence and blocks model interactions that violate policy. It does not certify generated code as secure.

Book a demo today.