FedRAMP AI Risk Assessment: Test the LLM Route Inside the Boundary

NIST SP 800-53 RA-3 gives a FedRAMP AI risk assessment its basic shape: identify threats and vulnerabilities, determine likelihood and magnitude of harm, determine risk, document results, disseminate them and update the assessment. The AI-specific work begins when that structure is applied to an actual cloud-service boundary and model request path.

The NIST control catalog supplies the control language. Current FedRAMP risk-assessment controls add program guidance for vulnerability detection, reporting and response. The assessment should connect both to production evidence.

TL;DR

• Fix the authorization boundary and identify every component that handles the model request.
• Write risks as specific events involving identities, federal data, model routes and control failures.
• Use RA-3 for assessment structure, then map treatments and evidence to applicable FedRAMP controls.
• Reassess after model, provider, route, data, retrieval or policy changes.

The authorization boundary defines the unit of risk

Start with the cloud service offering, its federal customer data and every component that processes an AI request. Include the application, identity provider, gateway, retrieval service, vector store, model endpoint, telemetry path and evidence store. Record which components sit inside the assessed boundary, which services are inherited and which external providers receive data.

A clean diagram should match traffic. Select one production-like request and trace its network destination, caller identity, retrieval source, model route and audit sink. A line drawn to "approved LLM" gives the assessor a category. The request trace gives the assessor a system.

The government FedRAMP AI compliance guide explains how model services affect authorization. The companion FedRAMP AI compliance checklist converts the boundary into repeatable tests.

At a review table, put the architecture diagram beside a 15-inch laptop showing the captured endpoint. If the arrows and request disagree, fix the boundary record before scoring risk.

RA-3 needs event-level AI scenarios

RA-3 asks for threats, vulnerabilities, likelihood, magnitude of harm and risk. An AI assessment should state each event with an actor, data, route, control failure and consequence. "LLM risk: high" gives engineering no treatment target.

Useful scenarios include:

• an authenticated user sends controlled data to a model outside the approved route set;
• a shared application credential hides the human or workload that initiated the request;
• retrieval returns another agency tenant's records into the prompt context;
• a model alias changes provider or region without the approved configuration record;
• a prompt injection causes an agent to place protected context in an outbound HTTP model call;
• the application receives a model response and crashes before its self-authored audit log commits.

For each scenario, identify the relevant NIST controls and evidence. AC-3 can govern access enforcement, while AC-4 addresses information flow. AU-2, AU-3 and AU-12 cover event selection, record content and generation. SC controls protect communications and boundaries, with IR controls governing the response process.

Likelihood should use architecture and observed traffic

Estimate likelihood using system facts: exposed routes, identity strength, alternate paths, policy coverage, provider behavior, change frequency, test results and observed events. A temporary model endpoint called twice in production deserves different treatment from a disabled configuration that never carried traffic.

Request-level records make the estimate reviewable. Count new model destinations, missing identity context, policy failures, denied sensitive-data requests and changes to route resolution during the assessment period. Separate attempted events from successful transfer. The distinction affects both likelihood and consequence.

My opinion is that a heat map without request evidence is decoration. A red square gains value only when the owner can open the test, event or architecture fact that made it red.

The FedRAMP AI controls mapping helps map those observations to AC, AU, SC, SI and CM owners without assigning an entire control to one gateway.

Harm follows federal customer data and agency mission

Magnitude of harm depends on data type, agency mission, affected tenants, action taken and recovery path. A public-information summarisation request has a different consequence profile from a model call containing CUI, authentication material or a multi-agency case file.

Record confidentiality, integrity and availability consequences separately. For model traffic, confidentiality scenarios include unauthorized prompt transfer and cross-tenant retrieval. Integrity scenarios include altered context, unapproved response transformation and policy bypass. Availability scenarios include route failure, quota exhaustion and a provider outage that forces an unapproved fallback.

Tie the rating to an agency-approved impact rationale. Include the likely number of affected agencies and federal customer data categories where current FedRAMP incident rules could later require a Potential Agency Impact rating. The assessment prepares those facts before an incident. It should never pre-decide the incident classification.

Treatments should map to controls and owners

Each risk treatment needs a mechanism, control mapping, owner, evidence source and retest date. Example treatments include:

• propagate signed identity context to the AI policy decision point;
• restrict models and regions by role and federal data class;
• split tenant retrieval indexes and test cross-tenant denial;
• pin model routes through controlled configuration;
• generate independent per-decision records and export them to the agency evidence store;
• rehearse incident containment and provider replacement.

RA-7 calls for responding to findings from assessments, monitoring and audits according to organizational risk tolerance. The current FedRAMP risk page also ties RA-5 vulnerability work to Vulnerability Detection and Response plus Vulnerability Evaluation and Reporting rules.

Record accepted risks with the designated authority and expiry date. A permanent acceptance for a changing model route is an undocumented operational decision dressed as governance.

Provider dependencies belong in the same assessment

NIST SA-9 requires external system service providers to comply with organizational security and privacy requirements, defines oversight and user responsibilities, and calls for monitoring provider compliance. An LLM API, hosted retrieval system or model telemetry service can become an external dependency according to the architecture.

Assess authorization status, service boundary, regions, subprocessors, logging, incident contacts, model-version practices, data use, retention and exit capability, then test the runtime route against those facts. Procurement evidence says what the provider committed to do. Traffic and control evidence say how the service was used.

Keep shared responsibilities explicit. The model provider controls its service. The cloud service provider controls integration and its certified offering. The agency authorizing official evaluates the federal system's risk. An inline AI control can supply evidence for a bounded subset of the request path.

Reassessment follows change and monitoring

RA-3 requires assessment updates at an organization-defined frequency and after significant changes or other conditions. For AI, useful triggers include a new model provider, model alias change, retrieval source, data class, agency tenant, tool permission, route policy, region, authentication pattern or incident.

Current FedRAMP Collaborative Continuous Monitoring rules ask agencies to review ongoing certification reports in light of the risk tolerance documented in the agency's ATO. Connect route and policy changes to that ongoing record. A quarterly screenshot can miss a model endpoint used for nine days and removed before review.

Version the assessment and preserve the evidence behind each change. The review history should show which risk moved, why it moved, who approved the treatment and which request test passed afterward.

DeepInspect

DeepInspect can implement and evidence a defined set of treatments for authenticated HTTP traffic deliberately routed between users or agents and LLM endpoints. It consumes identity context, classifies prompt content, applies role and route policy, restricts destinations, inspects responses and records each decision outside the calling application.

Those records support RA-3 evidence for observed routes, tested controls and incident reconstruction. DeepInspect leaves full system authorization, provider assessment, endpoint security, local execution, STDIO, credential theft and traffic that bypasses the proxy with the agency, provider and control owners responsible for them.

Book a technical deep dive at deepinspect.ai.