Banking AI Model Risk: Applying SR 11-7 to LLMs and Agents
SR 11-7 defines a model as any quantitative method that produces output to inform a decision, which puts LLMs and AI agents squarely in scope for a bank. This article walks the three pillars of the guidance, sound development, independent validation, and governance, and shows why the validation team cannot challenge an opaque non-deterministic vendor model without production telemetry on what the model was actually asked and what it returned.
Federal Reserve SR 11-7 and OCC Bulletin 2011-12 define a model as a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories to produce output for use in a decision. Issued April 4, 2011 and adopted by the FDIC through FIL-22-2017, that definition predates the current wave of language models and captures them cleanly. An LLM that drafts an adverse-action explanation, classifies a transaction, or ranks a loan file is producing output that informs a decision, so it is a model under the guidance and inherits the full model risk management program. I want to walk the three pillars of SR 11-7 as they apply to a language model, and show why the pillar most banks treat as an onboarding checkbox is the one that carries the weight in production.
What SR 11-7 requires
The guidance defines model risk as the potential for adverse consequences from decisions based on incorrect or misused model output, and it builds the program on three pillars. The SR 11-7 text is the authoritative source.
Sound development, implementation, and use
The first pillar asks for disciplined model development with a clear statement of purpose, sound data, and documented assumptions and limitations. For a vendor LLM the bank did not train, the development record is thin, so the burden shifts to documenting the use: which decisions the model informs, what its known limitations are, and where its output is relied on.
Effective challenge through independent validation
The second pillar is validation, and its core is effective challenge, meaning critical review by objective, informed parties who can and will identify problems. Validation covers conceptual soundness, ongoing monitoring, and outcomes analysis. A language model resists conceptual-soundness review because its internals are opaque and its output varies run to run, which means the validation team leans harder on monitoring and outcomes than on inspecting the mechanism.
Governance, policies, and controls
The third pillar puts model risk under board-approved policy with clear roles across three lines of defense: the model owners, an independent model risk management function, and internal audit. Governance requires a model inventory that records every model in use, its owner, and its validation status, which is the artifact covered in AI model inventory management.
Why effective challenge breaks on an opaque model
Effective challenge assumes the reviewer can inspect the model's construction and reason about its behavior. A third-party language model gives the validation team no weights, no training data, and no guarantee that today's responses match last quarter's. The classic validation techniques, sensitivity analysis and conceptual review, have limited reach against a system that is non-deterministic and hosted elsewhere. What remains is the empirical half of validation: ongoing monitoring of live behavior and outcomes analysis against actual results. That half depends entirely on having a record of what the model was asked and what it returned in production, at the granularity of individual requests. A bank that monitors only aggregate accuracy has no way to challenge a specific decision after the fact.
The production monitoring gap
SR 11-7 is explicit that validation is not a one-time event and that ongoing monitoring confirms a model is performing as intended. For a language model in a live workflow, ongoing monitoring means capturing the prompts, the responses, and the identity behind each call, then analyzing drift, anomalous outputs, and use outside the approved scope. Most banks instrument the application around the model and never capture the model traffic itself, so the monitoring pillar rests on data the bank does not have. When an examiner asks the model risk function to demonstrate ongoing monitoring of the LLM, the answer has to be a record of the actual request and response traffic, and application logs that store status codes and timestamps fail that test. This is the same request-layer blind spot that runs through the broader AI compliance map for banking.
DeepInspect
This is the production evidence SR 11-7 monitoring depends on, and it is what DeepInspect produces. DeepInspect is a stateless proxy on the AI request path between a bank's applications and agents and the model endpoints they call. It captures every request and response, binds each to the authenticated identity and the model version reached, and commits a per-decision record the model risk function can query for ongoing monitoring and outcomes analysis.
It also enforces scope. The governance pillar approves a model for specific uses, and DeepInspect applies identity-bound policy that keeps a model call inside its approved scope, blocking use of an unvalidated model or an off-label workflow before the request reaches the provider. It runs inline and fails closed, so scope violations are prevented rather than surfaced in the next validation cycle. The inventory, the monitoring record, and the scope control are three of the artifacts an SR 11-7 examination asks for, produced from one place on the traffic. Book a demo today.
Frequently asked questions
- Is a large language model a model under SR 11-7?
Yes, when its output informs a decision. SR 11-7 defines a model by function, a quantitative method producing output used in decision-making, and a language model that drafts, classifies, ranks, or explains within a banking workflow meets that definition. Supervisors have applied the guidance to AI and machine learning systems, so a bank should expect its LLM deployments to be examined against the same development, validation, and governance standards as any other model in the inventory.
- How do you validate a third-party LLM you cannot inspect?
Validation shifts from conceptual soundness toward the empirical pillars. You cannot inspect a vendor model's weights, and you can document its approved use and limitations, monitor its live behavior, and analyze outcomes against actual results. That makes production telemetry central, because ongoing monitoring and outcomes analysis both require a record of what the model was asked and what it returned. Effective challenge of an opaque model is only as strong as the request-level evidence the validation team can examine.
- What does ongoing monitoring mean for a language model?
It means capturing production behavior and analyzing it over time for drift, anomalous output, and use outside approved scope. For an LLM that requires recording the prompts, responses, and calling identities, then reviewing them against the model's documented purpose. Aggregate accuracy metrics miss the specific decisions an examiner or an adverse-action inquiry will ask about, so monitoring has to reach individual-request granularity to satisfy the guidance and to support outcomes analysis.
- Where does the model inventory fit?
The governance pillar requires an inventory recording every model in use, its owner, and its validation status. For AI, the inventory has to reflect which models applications and agents are actually calling, which frequently diverges from the approved list once teams add a new provider. Deriving the inventory from observed request traffic, rather than a manually maintained spreadsheet, keeps it accurate and surfaces shadow model usage that the governance function needs to bring into scope.