Llama Audit Logs Depend Entirely on Which Hosting Layer You Picked
Llama is an open-weight model with no logging of its own, so audit logging for a Llama deployment depends entirely on the hosting layer. AWS Bedrock offers detailed model invocation logging that ships disabled by default. Azure AI Foundry routes similar detail through diagnostic settings that must be explicitly configured. Self-hosted deployments on vLLM or TGI have no audit logging at all until a team builds it. This article separates the three paths.
Llama has no audit log, because Llama is a set of model weights, not a hosted service. The question "what does Llama log" only has an answer once you specify which of three paths a request takes to reach it: AWS Bedrock's managed endpoint, Azure AI Foundry's managed endpoint, or a self-hosted deployment on infrastructure a team runs itself. I want to walk through what each path actually captures, because the three answers are not close to each other.
TL;DR
- Llama itself logs nothing. Audit logging for a Llama deployment is entirely a function of the hosting layer chosen.
- AWS Bedrock's model invocation logging captures full request and response bodies, token counts, and the caller's IAM identity, but it ships disabled by default and must be turned on per account or per model.
- Azure AI Foundry offers Audit, RequestResponse, and Trace diagnostic categories, none of which are captured until a diagnostic setting routes them somewhere.
- Self-hosted Llama on vLLM or TGI has no audit logging built in at all; a team has to add a gateway or proxy layer to get one.
AWS Bedrock: detailed logging that is off by default
AWS Bedrock's model invocation logging is a distinct feature from CloudTrail, and the distinction matters. CloudTrail records the management-plane fact that InvokeModel was called, by which IAM identity, at what time. Model invocation logging goes further: it captures the full request and response bodies, up to 100 KB inline with larger payloads routed to S3, along with token counts, the caller's IAM identity ARN, region, and model ID, for Converse, ConverseStream, InvokeModel, and InvokeModelWithResponseStream calls, including calls to Bedrock's hosted Llama models. That is the level of detail a security review actually wants.
The catch is that model invocation logging is not on by default. An account has to explicitly enable it, and the setting can be scoped differently across models and workspaces. A team that assumes "we're on Bedrock, so this is logged" without checking the account-level logging configuration is very often wrong, and finds out during the review that asks for the logs, not before.
Azure AI Foundry: three log categories, zero of them automatic
Azure AI Foundry hosts Llama through its Models-as-a-Service and managed compute options, and emits three diagnostic log categories: Audit, for control-plane actions; RequestResponse, for the actual inference traffic; and Trace, for execution detail. None of the three reaches a destination a team can query until a diagnostic setting routes it to Log Analytics, Storage, or an Event Hub. The mechanism is well-designed once configured. It produces nothing on its own.
That pattern repeats across both major clouds: the raw capability for detailed logging exists, and the responsibility for turning it on, and keeping it on across every project a Llama deployment touches, sits entirely with the customer.
Self-hosted Llama: nothing to turn on at all
A team running Llama on its own GPUs through vLLM or TGI has no equivalent feature to enable, because there is no managed platform underneath collecting the data. Getting an audit-ready record on a self-hosted deployment means building a gateway or proxy layer in front of the inference server that handles authentication, per-caller attribution, and an append-only record of prompts, completions, and the model version that answered. That architecture, not a configuration checkbox, is the standard approach teams reach for when a self-hosted Llama deployment needs to satisfy the same kind of retention and traceability requirements that EU AI Act Article 12 sets for high-risk systems.
The consequence: three deployments, three different starting points
An organization running Llama through Bedrock in one business unit, Azure AI Foundry in another, and a self-hosted cluster for a latency-sensitive workload has three different logging stories to reconcile, none of them automatic, and no single place to go looking for a consistent record across all three. A security review that assumes "Llama traffic is logged" without checking each hosting path individually will find gaps in at least one of them, because at least one of them ships nothing until someone configures it.
DeepInspect
This is exactly what DeepInspect does. DeepInspect sits inline as a stateless proxy in front of any HTTP-based LLM endpoint, Bedrock's Llama models, Azure AI Foundry's Llama deployments, and self-hosted vLLM or TGI endpoints included, evaluating identity, role, and data classification on every request before it reaches the model.
The resulting record is a signed, per-decision audit entry, produced the same way regardless of which of the three hosting paths the underlying Llama deployment uses. A security team stops needing to reconcile Bedrock's invocation-logging configuration against Azure's diagnostic settings against whatever a self-hosted gateway happens to capture, because the identity-aware AI gateway architecture sits in front of all three the same way. Signed audit logs for AI requests describes the record it produces in each case. Book a demo today.