← Blog

Letta Audit Logs: Map Typed Events and Persistent State to Evidence

Parminder Singh
Parminder Singh··7 min read
Summarize with AI

Letta exposes typed stream events, persistent agent memory, conversation history, session identifiers, and run identifiers. This guide maps those records to the questions they can answer, separates connection-scoped activity from durable state, and defines the extra evidence needed when an authenticated Letta application calls a remote LLM over a controlled HTTP route.

Platform & Architectureai-securityauditforensic-auditagentic-aiarchitecture
Letta Audit Logs: Map Typed Events and Persistent State to Evidence

Letta exposes several records during one agent turn. Its Agent SDK streams typed events for reasoning and assistant text, plus tool calls, tool results, errors, queue changes, and a terminal result. The same system persists agent memory and conversation history beyond an SDK connection. Those records are useful evidence, but they answer different questions and carry different durability guarantees.

I would reject an audit design that labels the whole collection letta_audit_log. A transcript row, a streamed fragment, a tool result, and an independent model-route decision have different writers and custody. Put them on one whiteboard with separate boxes. The joins between boxes matter more than a tidy shared label.

TL;DR

  • Letta streams typed events for a turn, including assistant text, reasoning, tool activity, errors, queue updates, and a terminal result.
  • Agent memory and conversation history persist beyond one SDK connection; client tools and several runtime settings remain session-scoped.
  • Typed events and durable state help reconstruct activity, but Letta's cited SDK pages do not document immutable, signed audit evidence.
  • DeepInspect covers only authenticated HTTP calls from Letta to a remote LLM that are deliberately routed through its proxy.

Letta emits typed evidence during each turn

The current Letta messaging documentation defines one turn as a message sent through session.send() followed by events read from session.stream(). The stream yields a typed SDKMessage union. Common types include initialization and reasoning fragments, assistant fragments and tool calls, plus tool results and the final result. The union also covers errors and retries. Other types report queue updates and token-level stream events, as well as loop status.

Each type answers a narrow question. Assistant and reasoning events show text fragments delivered during the turn. Tool-call events identify the named tool and stream its raw arguments. A tool result carries the corresponding output and error state. The final result reports success and final text, the stop reason and duration, plus the conversation ID and run IDs started by that turn.

This is runtime evidence with useful structure. It can show what the client received and how a turn progressed. The stream contract alone says nothing about independent custody, cryptographic integrity, or a retention period.

Message identifiers solve different reconstruction problems

Letta documents several identifiers because streamed fragments need more than one kind of correlation. otid groups assistant or reasoning fragments into a message slice. uuid identifies a message object or an SDK-generated object when the stream lacks a server identifier. seqId represents a position within a run and supports replay suppression. Tool arguments and results join on toolCallId. runId associates eligible messages with a Letta run.

The documentation warns against using seqId as a row key and against merging assistant and reasoning content on uuid alone. It recommends keying streamed text by message type together with otid, then using toolCallId for tool payloads. Letta also provides a transcript accumulator that applies family-specific merging and per-run replay suppression.

An evidence procedure should preserve these native identifiers rather than replacing them with one generic event ID. The broader AI audit-log schema can hold a local correlation field, but the original Letta identifiers remain necessary when an investigator checks ordering and fragment assembly or verifies tool-result pairing.

Sessions separate durable state from connection state

Letta's sessions and durability guide defines the agent as the persistent entity with memory, the conversation as a thread on that agent, and the session as the active connection used to send messages and stream events. Agent memory and conversation history persist beyond one SDK connection. Persistent repository attachments and model updates can persist too.

Other elements remain scoped to the session. The cited guide places client tools and MCP servers in that category, along with tool-permission callbacks, session repository resources, working directory, environment variables, and computer or sandbox selection. Closing a session releases its local resources. A connection that closes unexpectedly requires a new session, resumed from the conversation ID.

The distinction affects evidence claims. A persisted conversation can support a later history query. A session-scoped callback or local tool configuration requires its own configuration record if a review needs to establish what was active. Persistent state still lacks an automatic upgrade to immutable audit evidence. Persistence answers whether data survives a connection. Immutability requires documented controls over alteration and deletion, plus custody of the write path.

Reconnects create explicit evidence gaps

The sessions guide states that the SDK does not replay events missed while disconnected. After resuming, the client can reconcile with conversation history through listMessages() or retrieve a consolidated snapshot through bootstrapState(). It also warns against blindly retrying after a successful send followed by a connection failure, because the original message may already have reached the runtime.

That behavior belongs in the evidence coverage map. Record the last processed run and sequence position, the disconnect interval, the resumed conversation ID, and the history query used for reconciliation. Keep the streamed view separate from fetched history. One describes events observed live by the client. The other describes messages available from persistent conversation state after the fact.

I would block an audit-readiness claim when a team can show a chat transcript but has never tested a mid-turn disconnect. Pulling the network cable icon in a staging console, resuming the conversation, and reconciling the known turn gives the procedure a concrete failure case instead of a diagram-only promise.

Client and server transport remains a separate layer

The current Agent SDK quickstart documents managed cloud and local configurations, plus a remote-server option. A remote client connects to an App Server URL and may supply an authentication token. The same page shows sessions streaming assistant events after a message is sent. Letta's deployment guide says a self-hosted App Server accepts a direct connection from the application and uses its own authentication settings for remote access.

That transport can carry session commands and typed events, but transport records have a different owner from model-call evidence. WebSocket connection state can establish that a client communicated with an App Server. It does not establish that a downstream LLM request passed through a policy proxy. Local tools and state storage may run without any remote model HTTP call at all.

Keep the transport view in its own row beside the AI agent observability view. Record the backend mode and App Server address, session and conversation identifiers, authentication configuration, disconnects, and the reconciliation result. Do not present that row as proof of the downstream model route.

An evidence coverage map preserves custody boundaries

A useful Letta audit map assigns one investigation question to each source:

  • Typed stream events: Which reasoning, assistant, tool, queue, error, and result events reached this SDK client during the observed turn?
  • Conversation history: Which messages remained available from the persistent conversation after the connection ended?
  • Agent state: Which memory and persistent repository state belonged to the agent at the documented checkpoint?
  • Session configuration: Which session-scoped tools, permissions, resources, working directory, environment, and runtime selection were configured for the connection?
  • App Server transport: Which client connection carried session commands and events, and where did reconnect or replay gaps occur?
  • Remote model provider record: Which request reached the configured LLM endpoint under the provider's identifier?
  • Independent route decision: Which authenticated caller requested a remote HTTP model operation, which destination and policy version were evaluated, and what decision was recorded?

Give each source an owner, storage destination, retrieval query, retention rule, clock source, and known gap. Preserve the Letta conversation, run, message, tool-call, and sequence identifiers alongside an application correlation ID. For independent route evidence, use the integrity and custody criteria described in signed audit logs for AI requests.

A correlation drill tests the complete record

Create one synthetic conversation with a unique case identifier. Send a turn that produces assistant text and one harmless tool call, then retain the terminal result and its run IDs. Query conversation history after the session closes. Confirm that message fragments were assembled with the documented identifiers and that the tool result joins to the intended call.

Next, repeat the turn through the deployment's remote LLM route, if one exists. Carry a separate model-call correlation ID into the routed request and preserve the provider request identifier when available. Retrieve three views: Letta's turn and history evidence, the independent HTTP route decision, and the provider-side record. Document the accepted clock tolerance rather than expecting identical timestamps.

Finish with a controlled disconnect during a turn. Resume from the conversation ID, fetch history, and record what the client missed. This drill proves the coverage and gaps of the deployed configuration. It avoids claiming that a successful stream, a persistent transcript, or a provider dashboard observed the entire system.

DeepInspect

DeepInspect applies when an authenticated Letta application invokes a remote LLM over HTTP and deliberately routes that request through the proxy. The application supplies caller and correlation context. DeepInspect evaluates the model destination, data classification, and versioned policy before forwarding an allowed request, then writes a signed, tamper-evident per-decision record with write-path independence for that routed operation.

Letta remains the evidence source for sessions, conversations, typed events, memory, tools, and runtime state. App Server and WebSocket transport retain their own owners. Local tools, local model execution, state storage, session traffic, and direct remote-model calls that bypass the proxy sit outside the DeepInspect record.

Book a technical deep dive at deepinspect.ai.

Frequently asked questions

Does Letta provide immutable audit logs?

The cited Agent SDK pages document typed stream events, message and run identifiers, conversation history, persistent agent memory, session behavior, and reconnect limitations. They do not document a single immutable audit-log facility or a cryptographic signing guarantee for this collection. Treat each native source according to its documented role. Add separately governed evidence when an audit requires tamper evidence or write-path independence.

Are typed events durable after a disconnect?

Letta says the SDK does not replay events missed while disconnected. A resumed client can reconcile with listMessages() or bootstrapState(), but fetched history and live stream observation are different evidence sources. Preserve the disconnect interval and reconciliation method. If a send may have succeeded before the connection failed, inspect conversation history before retrying.

Does persistent agent memory prove what happened in a turn?

Persistent memory establishes state that survives the SDK connection. Turn reconstruction depends on messages, typed events, run identifiers, tool-call identifiers, and the relevant state checkpoint. Memory can change over time, so an evidence procedure should record the checkpoint or retrieved version used during review rather than treating current memory as a complete historical record.

Does App Server authentication put model calls inside DeepInspect's boundary?

App Server authentication protects the client-to-server transport described by Letta. DeepInspect's boundary begins only when an authenticated Letta application sends a remote LLM call over HTTP through the configured proxy. A WebSocket session, local tool execution, or local state operation remains outside that model-traffic boundary.