NVIDIA NIM Audit Logs: Why a Successful Inference Leaves No Record by Default
NVIDIA NIM writes structured logs to stderr, exposes vLLM Prometheus metrics unchanged at /v1/metrics, and forwards X-Request-Id and W3C traceparent headers for distributed tracing. The default log level is WARNING, which means a successful inference produces no log line at all. Nothing in the documented telemetry records a caller identity, an authorization outcome, or the content of a request.
NVIDIA NIM's default logging configuration is the fact that matters most for a security review, and it is a single environment variable. NIM_LOG_LEVEL defaults to WARNING. A completion served correctly to an authorized caller generates no log line, because nothing went wrong. I want to walk through what NIM does emit, then what a security team is left holding when an incident review starts.
TL;DR
- NIM writes structured logs to stderr, with
NIM_LOG_LEVELdefaulting to WARNING andNIM_JSONL_LOGGINGdefaulting to false. - Setting
NIM_JSONL_LOGGING=trueproduces JSON Lines records carrying level, time, file name, line number, and message. - Metrics at
/v1/metricsare vLLM's own Prometheus metrics passed through without modification, covering request latency, throughput, queue depth, token counts, and GPU utilization. - Request correlation runs on
X-Request-Idand W3Ctraceparent. No documented field records a caller identity, a role, or an authorization decision.
What the log lines look like
NIM's structured logging has two output shapes. The human-readable default puts the level first, then the timestamp, then the source file and line, then the message, as in INFO 2026-03-03 12:00:10.079 start_server.py:42] Server starting on port 8000.
Setting NIM_JSONL_LOGGING=true emits the same record as a JSON object carrying level, time, file_name, line_number, and message keys, one object per line.
Five fields: level, time, source file, line number, message. That is a well-formed application log and it is designed for the job it does, which is telling an operator why a container failed to start or why a request errored.
The fields tell you which line of Python emitted the message. They do not tell you who made the call, because the process has no idea. NIM receives an HTTP request, runs inference, and returns a response. Whatever identity existed upstream was resolved by something else, somewhere else.
The metrics are vLLM's, unchanged
NIM exposes Prometheus-compatible metrics at /v1/metrics, and NVIDIA documents that NIM "passes through the inference backend's native Prometheus metrics without modification." The metrics pulled from vLLM cover request latency, throughput, queue depth, token counts, and GPU utilization.
Passing them through unchanged is a good engineering decision and I would rather have it than a rebranded metric surface: existing vLLM dashboards and alerting rules keep working after a migration to NIM, which removes a real class of migration bug.
For capacity planning and performance work, this is everything a platform team needs. For an audit, aggregate counters are structurally the wrong artifact. A histogram bucket showing 40,000 requests in an hour cannot be decomposed back into which request carried a customer's medical record, because it was never a per-request record to begin with.
Tracing correlates requests without identifying callers
NIM supports two headers for request correlation and distributed tracing. X-Request-Id is described as a "request correlation identifier. Forwarded to the backend and adopted as the internal request ID." traceparent follows W3C Trace Context and propagates trace context for OpenTelemetry integration.
Related NIM microservices go further. NVIDIA NIM for Object Detection supports exporting metrics and traces in an OpenTelemetry-compatible format to a running OpenTelemetry Collector, which can forward to any OTLP-compatible backend, with the microservice and the underlying Triton Inference Server both exposing Prometheus endpoints.
A trace ID stitches a request together across services. It is an opaque correlation key, and the value it holds is whatever the upstream caller generated. A trace with no identity attached at the edge stays anonymous through every hop it touches, which means correlation quality is entirely a property of what sits in front of NIM.
What the default configuration means in an incident
Assume a self-hosted NIM deployment serving an internal application, running the shipped defaults, and assume an investigator arrives three weeks after a suspected data exposure. Working backwards through what exists:
Application logs, if the calling application wrote any, hold whatever the application chose to record, asserted by the application about itself. NIM's stderr output at WARNING level holds startup messages and errors, and no line for any successful completion. The Prometheus scrape holds counters showing volume rose on the day in question. The trace backend, if one was wired up, holds spans keyed by IDs that map to nothing outside themselves.
No artifact in that set answers which authenticated person caused a specific prompt to be sent. This is the same structural position described in the post-authentication gap, and it applies to every self-hosted inference server rather than to NIM specifically. The comparable managed-platform position is covered in AWS Bedrock audit logs.
Raising NIM_LOG_LEVEL to INFO or DEBUG produces more lines, and those lines still describe the server's own execution rather than a caller.
The four fields a compliance record needs
An audit record that survives a regulatory review of an AI system carries four things: the authenticated principal as asserted by an identity provider rather than by the calling application, the role or group that principal held at request time, the classification of the data that entered the prompt, and the policy decision that permitted the request together with the policy version in force.
None of the four can be produced by a process that receives an already-anonymous HTTP request. The requirement lands on whatever sits between the caller and the container, which is also where the access control has to live given what NGC keys and the NIM container do and do not authenticate. The record format itself is covered in AI audit log schema design, and the data-side controls in NVIDIA NIM DLP.
DeepInspect
This is the gap DeepInspect closes. DeepInspect runs inline as a stateless proxy in front of HTTP LLM endpoints, including self-hosted NIM containers, so the identity check happens on the request before it reaches the inference server rather than being reconstructed afterwards from logs that never held it.
Every request produces a signed record carrying the authenticated principal, the role, the classification of the prompt content, the policy version, and the allow-or-block decision, with a correlation ID that lines up against the X-Request-Id NIM already forwards. NIM keeps emitting the vLLM metrics that make capacity planning work and the stderr lines that make a failed start debuggable. DeepInspect supplies the per-request, identity-bound record those two were never designed to produce. Book a technical deep dive at deepinspect.ai.