← Blog

Letta Security: A Production Deployment Acceptance Guide

Parminder Singh
Parminder Singh··8 min read
Summarize with AI

Letta security spans model-provider routing and API credentials, plus App Server exposure, WebSocket authentication, Agent SDK permission modes, and direct paths around an approved gateway. This guide turns those controls into a production acceptance record while keeping memory and local tools with their owners. Permissions, session transport, and route enforcement stay separate.

Platform & Architectureai-securityagentic-aiidentity-and-authorizationpolicy-enforcementarchitecture
Letta Security: A Production Deployment Acceptance Guide

A Letta deployment can send model calls to a provider API, an OpenAI-compatible gateway, or a self-hosted endpoint. It can also expose an App Server over WebSocket and let agents invoke tools under one of four SDK permission modes. Each route carries a different credential and control owner.

I would approve the deployment only after testing the effective topology. The review record should show where the agent runs, which base URL receives model traffic, which credential authenticates each hop, how remote clients reach the server, and which direct routes failed under test.

TL;DR

  • Pin the effective model provider and base URL, then add the API credential reference and agent location to one deployment record.
  • Keep App Server on loopback for local use. A non-loopback listener needs authentication plus a separately verified HTTPS or secure WebSocket path.
  • Treat Agent SDK permission modes and canUseTool decisions as tool controls, separate from model-route policy.
  • DeepInspect covers only authenticated HTTP LLM traffic sent through its proxy. Memory and local tools stay with their owners. Permissions, session transport, and bypass traffic also remain separate.

Deployment acceptance starts with the actual route

Letta's model configuration documentation separates local setup from agents signed in with Letta. A local agent can connect directly to providers such as Ollama or LM Studio. A cloud-hosted agent can use an OpenAI-compatible custom provider by supplying a base URL and API key. In that case, the endpoint must be reachable from the public internet because localhost points back to the cloud environment rather than the operator's machine.

Turn those options into a route inventory that captures the execution location and provider type for every agent under review, instead of recording only the selected model name. Add the resolved base URL and model identifier, followed by the credential reference and approved egress destination, then record the component that terminates TLS. A model selector screenshot leaves the network path unresolved.

Letta can switch models during a conversation with /model. The acceptance record should therefore distinguish the release default from the models that an operator can select after startup. Run one test against the approved destination and attempt one unapproved selection. Preserve the observed request destination in both cases.

Remote base URLs change the exposure boundary

The same Letta models page says a cloud-hosted agent's custom OpenAI-compatible endpoint needs a stable public URL or static IP. Letta tells operators to secure that endpoint with authentication and HTTPS, then restrict access where possible. The endpoint must support Chat Completions and tool calling.

A public route deserves a concrete reachability test that starts with a valid request from the expected Letta environment and then repeats from an unapproved network position. Check the server certificate and hostname on the public path. Record the listener address and published port, then add the DNS name and firewall or allowlist rule that limits access. If a reverse proxy terminates HTTPS before forwarding to the inference server, identify both hops and the credential used on the internal hop.

OpenAI compatibility describes the request interface. The operator still owns endpoint authentication and certificate handling, plus network policy and capacity controls. Put those owners beside the base URL. The identity-aware AI gateway guide covers the separate decision that binds a verified caller to request policy.

API credentials need route-specific custody

Letta supports provider connections through /connect, letta connect, or the Models page in the web app. The documented provider matrix includes API credentials and subscriptions, plus OAuth and local endpoints. An OpenAI-compatible API uses a provider credential with its base URL. Those methods create different lifecycles, so the acceptance record should name the credential type rather than using one generic "LLM credential" field.

For a provider-credential route, record the secret reference and owning team. Add the permitted destination and rotation procedure. Test an omitted credential and an invalid credential, followed by the retired value after rotation. Preserve status and correlation identifiers while keeping secret values out of the evidence. A successful model response proves that one credential worked on one route. Direct endpoints that use another credential require their own tests.

Separate provider credentials from App Server client tokens because one authenticates an outbound model call while the other protects a client connection to the Letta runtime. Reusing one value across both boundaries makes revocation harder to interpret and expands the consequence of disclosure.

App Server exposure needs its own acceptance tests

The current App Server quickstart binds the development listener to ws://127.0.0.1:4500 and exposes a single /ws connection. For a non-loopback listener, Letta documents WebSocket authentication through a capability token or a signed bearer token. The capability-token example reads the token from an absolute file path, while clients send it as a bearer token.

Keep the loopback bind for a local controller. A remote deployment needs an authenticated listener and a separately documented secure transport path. Test a connection without the bearer token, another with an invalid token, and a valid connection through the approved HTTPS or secure WebSocket entry point. Then try the raw non-loopback address from a network that should lack access.

The quickstart can also expose OpenAI-compatible /v1/models, /v1/chat/completions, and /v1/responses routes when --openai-api is enabled. Add each endpoint to the inventory because the documentation says configured App Server authentication uses the same bearer token as the client's API key on that interface. Confirm that every published route rejects missing and invalid tokens.

Permissions govern tools inside the agent runtime

The Agent SDK permissions documentation defines four session permission modes. standard sends calls that need approval to canUseTool. acceptEdits automatically approves file edits. unrestricted automatically allows tool calls that need no user input. Under strict, nothing is automatically approved.

The SDK also exposes allowedTools as a coarser availability filter. For calls that reach canUseTool, the application can approve or deny the proposed action. It may also edit the input or interrupt the turn after a denial. Letta documents request and tool-call identifiers for correlating approval activity, plus optional path and diff details for a user interface.

These controls belong in the Letta deployment record, which should include a denied Bash call and an edited input under the production mode, followed by an attempted tool excluded by allowedTools. Disconnect during an approval and exercise the documented pending-approval recovery path if the application relies on it. The post-authentication gap explains why a valid session still needs an authorization decision at the action boundary.

Memory and session transport retain separate owners

Letta agents preserve state across conversations and expose runtime activity over the App Server's bidirectional WebSocket. The App Server quickstart says the same connection carries output and tool activity. Approval requests and status changes also use it, as do terminal events. Those session events sit on a different path from the downstream provider request.

Assign storage controls for agent memory and conversation state to the Letta runtime owner. Give WebSocket authentication and connection handling to the App Server owner, which also owns transport security. Tool approvals remain with the Agent SDK application and its canUseTool policy. The downstream model endpoint has another owner for its API credential and network exposure.

This separation prevents a clean provider-route test from standing in for the whole agent. DeepInspect sees only the model calls routed through its HTTP proxy. Memory writes plus Bash and other local tools retain their assigned controls. The AI agent memory security guide covers that storage boundary in detail.

Direct-route bypass tests complete the record

A gateway policy applies to traffic that reaches the gateway. Every alternate base URL should fail when applications or agents try to reach the same model directly. Inventory every provider hostname and custom gateway address available from the agent environment. Include each self-hosted inference listener, then compare the full list with the single approved route.

Run the approved request first and retain its correlation identifier. Replace the configured base URL with the provider's direct address while keeping the test model and prompt fixed. The connection should fail at network policy or credential validation. Repeat against any private inference address and against a public App Server API that could initiate the same model call.

Record the denial point and owner alongside the gateway event, which covers only routed requests and therefore treats an absent event as an expected symptom of bypass rather than proof of prevention. I would block production acceptance until every known alternate route has a failed result or a time-bounded exception with an accountable owner.

Production acceptance record

Keep the evidence in one versioned artifact tied to the Letta release and deployment revision:

  • Agent and model route: execution location and selected model, plus provider type, resolved base URL, and approved egress destination.
  • Credentials: secret references for the provider and App Server, custody and rotation owners, plus omitted and invalid results followed by the retired-credential result.
  • Endpoint exposure: listener addresses and published routes, followed by TLS termination, network restrictions, and remote reachability results.
  • Tool authority: permission mode and exposed tools, plus canUseTool behavior, denied-call evidence, and the approval-recovery result.
  • Bypass coverage: direct provider and inference addresses tested, observed denial point and exception owner, plus the expiry date.

Add timestamps and configuration revisions, then attach expected outcomes and raw evidence locations. Re-run the affected tests after a model provider or base URL change and after credential rotation. App Server listener edits also trigger a retest, as do permission-mode changes or network-policy updates.

DeepInspect

DeepInspect governs the downstream authenticated HTTP LLM route when the Letta agent deliberately sends that traffic through its proxy. The application supplies verified caller or agent context that DeepInspect can evaluate with the request content before applying active policy to the destination and forwarding an allowed call. It preserves the decision record independently of the Letta runtime.

Letta memory and conversation state remain with the runtime owner. Bash and other local tools, Agent SDK permission modes, and canUseTool approvals stay with the application. App Server authentication and WebSocket session transport have their own owner. Provider credential custody and HTTPS termination also sit outside DeepInspect, as does direct traffic around the proxy. The bypass test is the condition that makes the routed policy meaningful.

Book a technical deep dive at deepinspect.ai.

Frequently asked questions

Does a Letta provider credential identify the end user?

The documented provider connection authenticates through the method required by that provider, such as an API key or OAuth. A shared provider credential identifies possession of that value. End-user or agent identity needs separate verified context and policy at the request boundary.

Is the App Server bearer token the same control as a model-provider credential?

The App Server token protects a client connection to the Letta runtime, while a provider credential authenticates the outbound request to an LLM endpoint. Test rotation and rejection separately, even when a deployment stores both in the same secret manager.

Does `strict` permission mode secure model traffic?

strict is the most restrictive documented SDK tool-permission mode because nothing is automatically approved. Model-provider routing and endpoint authentication remain separate deployment controls. HTTPS and direct-route prevention also need their own evidence, including a test of the approved base URL.

What proves that a Letta deployment cannot bypass its gateway?

The acceptance record needs failed direct requests from the real agent environment to each provider or self-hosted inference address. Preserve the network or credential denial alongside the successful request through the approved gateway. Configuration intent alone leaves the alternate path untested.