Is an AI SLA the same as an internal SLO?

An SLA is a contractual commitment with defined remedies and exclusions. An internal SLO is the reliability target your team operates for a route or customer feature. The internal objective can be stricter and should include failures outside the provider's contract denominator, including gateway configuration and quota management. It should also include unauthorized fallback. Keep both calculations because they answer different questions.

Should model quality appear in the SLA worksheet?

Quality belongs in a separate evaluation objective unless the contract expressly guarantees a measurable quality property. Availability evidence can show that a model returned a response. It cannot prove that the response was accurate. It also cannot prove that the response was safe and suitable for the task. Link the route to its evaluation owner and acceptance test instead of mixing quality scores into uptime math.

How often should provider terms be checked?

Check the canonical SLA during procurement and before a production launch. Check it again after a material service change and during the periodic contract review. Store the retrieval date and reviewed version in the worksheet. A provider can split one product family into several covered services, and a familiar 99.9% figure may apply to a neighboring service rather than the endpoint you call.

What evidence should stay out of a provider claim?

Keep full prompts, responses, credentials, personal data, and internal policy detail out of the claim unless the provider requires a specific item and disclosure is approved. Request IDs and timestamps usually establish the outage pattern when paired with redacted logs. Regions and status classes complete that pattern. The Bedrock SLA explicitly tells customers to remove or replace confidential or sensitive information in submitted logs.

AI SLA Template: Route-Level SLOs and Claim Evidence

AWS measures Amazon Bedrock availability in five-minute intervals. Google measures Vertex AI downtime in consecutive minutes and excludes repeated requests that ignore its back-off rules. Those two definitions can produce different answers for the same ugly hour on an operations screen. An AI SLA needs to preserve each provider's contract math, then translate it into a route-level objective your engineering and procurement teams can operate. I want to show the worksheet I would put between the contract and the production dashboard.

TL;DR

An AI SLA must name the covered endpoint and denominator. It must also name excluded errors and the measurement interval. Record the credit threshold and claim deadline.
Provider uptime commitments are contract remedies. Your internal service-level objective should also cover latency and failover for each authenticated LLM route. Include retries and evidence retention.
Bedrock and Vertex AI count failures differently, so one generic availability formula creates weak evidence.
DeepInspect can record policy and routing decisions on deliberately proxied LLM traffic. Provider infrastructure and credit approval remain with the provider.

Provider terms define the denominator

Amazon Bedrock's service-level agreement defines availability for each five-minute interval as the percentage of requests that avoid a 500 response. An interval with no requests counts as fully available. AWS then averages those intervals across the monthly billing cycle for each region. The stated commitment is 99.9%, with service-credit bands below that level.

Google's current Vertex AI Platform SLA uses another method. Downtime begins when the server-side error rate exceeds 5% for at least one consecutive minute. Valid requests returning HTTP 500 or 503 feed that error rate. Repeated identical requests count only when they follow Google's exponential back-off requirements, starting at one second and increasing to 32 seconds.

A dashboard tile labelled "AI availability" hides these differences. Keep the provider's denominator and interval in separate fields. Record the covered service and region alongside them, with exclusions recorded alongside them. The contract calculation should be reproducible without asking an engineer to remember which retry storm counted three months later.

An internal SLO needs a wider operating definition

A provider can satisfy its contractual SLA while your product remains unavailable. Authentication failures at your gateway or a depleted quota may sit outside the provider's denominator. A broken deployment name or a bad fallback route may also sit outside it. Customers still see a failed feature.

Define an internal service-level objective for each authenticated LLM route. The route record should name the approved provider and model. It should also name the operation and fallback target. State the expected latency and timeout, plus the retry budget and acceptable error classes. AI gateway high availability covers the deployment patterns behind that objective. LLM routing strategies covers the policy choices that determine which fallback is safe for a particular request.

I would reject any AI SLA that reports only one monthly percentage. That number is useful for a credit claim and poor at explaining a production incident. A route can meet a 99.9% monthly target while failing during the only 40 minutes a claims team needs it. Record the service window and business owner beside the percentage.

The worksheet ties contracts to routed traffic

Use one worksheet row per covered route and provider service. Put the exact contract name at the top. "Microsoft AI" or "Google models" is too broad because product families carry separate terms.

Record these fields:

Covered route: the internal route name and provider endpoint, plus operation and region. Add deployment and owning team.
Provider measurement: the contract denominator and error codes. Record the interval and empty-interval treatment. Add exclusions and the monthly threshold.
Internal objective: the availability target and latency limit. Add the timeout and service window. Record the retry budget and fallback trigger.
Evidence: request correlation ID and timestamp. Record the region and provider. Add the model and attempt number. Record the status class and latency. Record timeout and fallback result. Include the policy version.
Claim procedure: eligible spend and credit bands, plus the filing channel and required logs. Add the claim deadline and contract owner.
Review control: current SLA URL and retrieved date. Record the last legal review and next review date. Record the change owner separately.

Print the worksheet on one page. During an incident, the useful version has a deployment name in the top-left corner and a claim deadline visible without scrolling through a 90-page contract. Keep raw request evidence in a controlled system rather than pasting sensitive prompts into the worksheet.

Retries can corrupt the evidence

Retries improve user experience when they are bounded and directed at transient faults. They also change the observation. Five rapid attempts may create one customer-visible failure or five provider requests. A contract exclusion may count them as one request.

The Bedrock SLA asks a claimant to provide dates and times. The claimant must also provide availability for affected five-minute intervals and the AWS region. Request logs must document errors. Claims must arrive by the end of the second billing cycle after the incident. Vertex AI requires notice to Google technical support within 30 days after eligibility, along with identifying information and the date and time of the errors.

Preserve every attempt with a shared correlation ID. Record the original route decision before a retry changes provider or model. Keep back-off timing and the final user outcome. AI gateway observability explains the surrounding telemetry, but the SLA evidence set needs unsampled request facts that can be reconciled with invoices and provider status records.

Failover needs an authorization decision

A fallback model can restore availability while violating a data-location rule or model allowlist. It can also violate a contract restriction or response-quality requirement. Treat failover as a policy decision rather than a blind load-balancer action.

The route worksheet should identify allowed fallback destinations and the conditions for using them. Attach the caller's identity and role to that choice. Record the data classification and policy version too. A finance route permitted to use Bedrock in one AWS region may have no approved path to a public endpoint in another jurisdiction. The safe outcome can be a visible failure.

Test four cases before production: a provider 500 response and a timeout with an unknown provider outcome, plus a quota rejection and a policy denial on the proposed fallback. Each test should leave a distinct record. The quota case belongs to capacity management. The policy denial belongs to security governance. Folding both into "provider unavailable" makes the monthly report tidy and the incident review useless.

Credits require a separate evidence pack

AWS describes a service credit as the sole remedy under its Bedrock SLA unless the governing agreement says otherwise. Google also describes financial credits as the sole remedy for an SLO failure. A credit can reduce a future bill. It cannot recover the business impact of an unavailable customer workflow.

Build the claim pack during the incident. Start with the provider account or project, covered service, region, and billing period. Add affected intervals, request identifiers, relevant status codes, retry sequence, and eligible charges. Remove prompt content and other sensitive material unless the provider specifically requires it and the disclosure has been approved. Preserve the version of the SLA that governed the incident.

Contract owners should review this pack before the deadline, then retain the submitted claim and provider response with the incident record. Operations owns route evidence, while finance confirms eligible charges. Legal or procurement checks the contract reading before the provider decides whether a credit is approved.

DeepInspect

DeepInspect sits on deliberately routed HTTP traffic between authenticated users or agents and LLM endpoints. For each request, it can record the supplied identity and route. It can also record the provider and model. It can record the policy version and decision, along with the status class and latency. The fallback result completes the record. Those per-decision records give operations and contract owners a stable route history for the internal SLO and the provider claim pack.

Provider infrastructure remains outside DeepInspect's control. The proxy cannot decide AWS or Google's contractual uptime calculation or approve a credit. It also cannot observe traffic that bypasses the configured route. Keep provider terms and billing evidence with their owners, then join them to the routed request record by timestamp and correlation ID.

Book a demo today.