Confidential Computing for AI: Protect Data During Processing

Confidential computing protects data while a processor is using it. Microsoft's Azure confidential computing overview quotes the Confidential Computing Consortium definition: computation runs inside a hardware-based, attested trusted execution environment, or TEE. That boundary addresses a hard gap left after storage encryption and TLS have done their jobs. An AI workload eventually needs plaintext data in memory to calculate on it.

I want to keep the claim exact. A TEE can protect code and data in use against actors outside the trusted boundary. It still needs a verified workload and controlled key release. Sound application logic and policy around each model request remain necessary.

TL;DR

• Confidential computing protects AI code and data in use inside a hardware-based, attested TEE.
• Hardware memory encryption and isolation reduce exposure to the hypervisor and cloud operator.
• Attestation lets a verifier inspect evidence about the workload environment before releasing data or keys.
• Request authorization and prompt classification remain complementary controls outside the TEE boundary. Model approval and audit records do too.

The TEE protects data in use

AI data passes through three states. Storage encryption protects model files and prompts at rest. It also protects retained datasets. TLS protects traffic while it moves across a network. Confidential computing addresses the interval when a CPU or GPU processes that data and ordinary encryption would otherwise leave plaintext available in memory.

Azure describes its confidential computing threat model as reducing trust in cloud-provider operators and other actors in the tenant's domain. The service uses a hardware root of trust outside the cloud provider's control. Properly configured, the TEE isolates code and data during execution and protects cryptographic secrets used by the workload.

For AI, the protected material may include prompt batches and embedding inputs. Model weights, fine-tuning examples, intermediate tensors, and output may also fall inside the boundary before the output leaves the environment. Scope follows the hardware boundary. Data becomes subject to the next control when it exits the TEE for storage or network transmission. Display and an external API create further transitions.

The zero-trust AI architecture article covers the request decision surrounding that protected execution.

Attestation gates trust in the workload

Memory encryption alone proves very little to a data owner. Attestation supplies evidence about the environment that will receive a secret or dataset. Google Cloud's Confidential VM overview describes a TEE as hardware isolation plus attestation. Its documentation says attestation can verify VM identity and state so a verifier can detect changes to critical components.

A common release flow has four steps. The workload starts in a confidential VM or container. Hardware and platform components produce attestation evidence. A verifier checks expected measurements and policy. A secret service then releases decryption material only after the evidence satisfies that policy.

Picture a deployment screen showing a green attestation result beside image digest sha256:91c..., with secret release logged three seconds later. That is useful evidence. A generic checkbox labelled confidential gives an auditor no measurement or verifier policy to inspect. It also omits the release decision and workload version.

My opinion is that attestation without an explicit release decision is expensive decoration. The verifier policy and the secret it protects need to appear in the same control test.

CPU and GPU boundaries need separate validation

AI inference and training may span CPUs and accelerators. Storage and network interfaces extend that path. Google documents Confidential VM instances using hardware-based memory encryption, with encryption material generated by and held in dedicated hardware rather than the hypervisor. The same page describes NVIDIA Confidential Computing for supported GPU workloads as extending TEE protections to attached GPUs and integrating with CPU-based confidential computing technologies.

That architecture creates a practical review task. Record which CPU platform and machine type ran the workload. Record the supported GPU model and mode. Include the attestation evidence and firmware state. Map every copy of sensitive data as it enters accelerator memory or returns to host memory. Continue the map when data crosses a network or lands in storage. Each transition can change the protection owner.

Performance testing also belongs in the deployment record. Memory encryption and restricted machine choices can affect capacity planning. Attestation calls and confidential GPU modes can too. Use the organization's real model and sequence lengths. Test its batch sizes against the service-level objective rather than a vendor diagram.

Zero-trust LLM architecture should treat the TEE configuration and attestation result as named evidence sources.

Workload identity and application behavior remain in scope

A TEE can execute vulnerable or malicious code faithfully. Attestation proves that measured components match an expected state; the expected state still needs security review. The application may read more data than its business purpose requires or send output to an unapproved destination. It may also accept an over-privileged workload identity.

Bind attestation to workload identity and authorization. The secret broker should know which measured workload requested a secret. IAM should limit the storage objects and queues available to that workload. It should also limit model endpoints. Application owners should control data minimization and retention. Output handling and tool permissions remain their responsibility. Security teams should preserve the measurement and verifier result. The release decision, workload identity, deployment version, and time belong in the same evidence set.

This separation also matters for hosted AI APIs. A customer can run preprocessing inside a confidential VM and then send the result to an external LLM. The TEE protected local computation. The external HTTP request crosses a new trust boundary with its own provider terms and routing policy. It also creates separate audit requirements.

Request controls complement confidential execution

Confidential computing answers where and under what measured environment code processed data. Request policy answers who may send which classified prompt to which model under the active rule. These controls can reinforce each other while retaining separate boundaries.

For an authenticated application calling an LLM over HTTP, an inline policy point can inspect the prompt and supplied identity context before forwarding it. The decision record can contain the caller and role. It can also record classification, destination, policy version, action, and timestamp. A TEE attestation record can sit beside it and show the measured environment used for an in-scope compute step.

The controls also leave distinct exclusions. DeepInspect covers authenticated user or agent traffic deliberately routed through its HTTP proxy. It has no control over memory inside a VM or CPU or GPU attestation. Release decisions, local model execution, and storage encryption also sit outside its boundary. Confidential computing provides no enterprise authorization decision for an HTTP request merely because the sending application ran inside a TEE.

The AI policy enforcement guide explains the request-layer half of this design.

DeepInspect

DeepInspect supplies a complementary control for authenticated users or agents sending deliberately routed HTTP traffic to an LLM. The calling application provides identity context. DeepInspect evaluates the request's classification and destination. It also evaluates the role and versioned policy before forwarding the request, then writes a signed, tamper-evident decision record.

A confidential CPU or GPU environment remains responsible for data-in-use protection and attestation. Secret services own release decisions. DeepInspect owns only the policy decision and evidence on the routed AI request boundary. Book a technical deep dive at deepinspect.ai.