DeepInspect vs LangSmith: Building the AI Application and Governing the Traffic It Sends

LangSmith arrived as the commercial layer under LangChain, and its adoption curve tracked the framework's. If your agent is a LangGraph state machine, LangSmith renders each node transition, tool call, and model invocation as a trace you can replay, which turns a category of debugging that used to involve print statements into something closer to a debugger.

The product has since grown past its origin. The SDK traces applications that never import LangChain, prompt management works as a standalone versioned store, and the evaluation suite runs against datasets regardless of what produced them.

Teams weighing it against DeepInspect are usually a few months past the prototype, with a security review scheduled. I want to be precise about what each product covers, because the overlap is smaller than the shared vocabulary suggests.

TL;DR

LangSmith is a development and evaluation platform for teams building LLM applications: tracing, datasets, evals, prompt versioning, and production monitoring driven by an SDK inside your code. DeepInspect is a policy proxy on the HTTP path that decides whether each model call is permitted, binds the decision to an authenticated identity, and writes an audit record before the response returns.

LangSmith: the build loop

LangSmith's centre of gravity is the loop between writing a prompt and knowing whether it got better. Traces capture the full execution tree with inputs, outputs, token counts, and latency at every step. Any trace can be saved into a dataset, so a production failure becomes a regression test in two clicks.

Evaluation runs over those datasets with LLM-as-judge scorers, custom code scorers, or human review queues. Prompt management holds versioned prompts outside the application code, with a playground for comparing variants across models. Production monitoring tracks error rates, latency, cost, and feedback scores over time.

For agent work the trace view is the strongest single feature in the category. When a LangGraph agent loops eleven times and burns a token budget on a tool it should have called once, the trace shows exactly which state transition went wrong. Reconstructing that from application logs is an afternoon of work.

The architectural position follows from that design. LangSmith runs inside your application via SDK, observes what the code reports, and holds no authority over whether a call proceeds.

The traffic layer

DeepInspect operates one layer down, on the wire rather than in the code. It is a stateless proxy between authenticated users or agents and any LLM endpoint, and every request that reaches a provider passes through it.

Two properties come from that position, and neither is available to an in-process SDK.

The first of them is enforcement. A policy evaluation that runs before the provider sees the request can refuse it. Scoring the same request afterwards produces a number and a response that already went out.

The second is coverage that survives the org chart. LangSmith covers applications your team instrumented. It has no view of the analyst hitting the OpenAI API from a notebook, the vendor SaaS product calling a model on your data, or the internal tool a different team shipped last quarter. Proxy placement makes coverage a network property rather than a discipline property, which matters because discipline degrades as headcount grows.

Feature comparison

| Capability | LangSmith | DeepInspect | |---|---|---| | Framework-aware agent tracing | Yes, strongest for LangGraph | No | | Evaluation datasets and scorers | Yes | No | | Prompt versioning and playground | Yes | No | | Human review queues | Yes | No | | Production quality monitoring | Yes | No | | Cost and token analytics | Yes | Per decision | | Inline blocking of a request | No | Yes | | Identity bound per request to enterprise IdP | No | Core | | Prompt classification against custom data classes | No | Yes | | Policy differing by role or entitlement | No | Yes | | Coverage without SDK instrumentation | No | Yes | | Audit record the application cannot alter | No | Core | | Fail-closed enforcement | Not applicable | Configurable | | Primary buyer | AI and application engineers | Security and compliance |

The table is lopsided in both directions, which is the honest result. Each product does things the other has no mechanism to attempt.

The question that separates them

Ask what happens when an engineer pastes a customer contract into a prompt to summarise it.

Under LangSmith, the call executes, the response returns, and a trace lands with the contract text in the input field. If you configured evaluators for PII, a flag appears on the trace. The contract already went to the provider, and it now sits in a second store as trace payload.

Under DeepInspect, the prompt is classified before the request leaves. If policy says this role may not send customer contract data to that model, the request is refused and a record is written showing the identity, the data class, the policy version, and the denial.

That difference is the entire argument. Both systems observed the same event. One of them was positioned to stop it.

Pick LangSmith if

• Your team is building and iterating on LLM applications, particularly with LangChain or LangGraph.
• Debugging multi-step agent behaviour is your daily problem.
• You need evaluation datasets, regression scoring, and prompt version control.
• You want human review workflows over production outputs.
• No regulator or enterprise customer is asking for per-decision evidence yet.

Pick DeepInspect if

• You need model calls refused according to policy rather than scored afterwards.
• Compliance requires records tied to an authenticated person or agent identity.
• AI traffic originates from teams and tools you do not control.
• Different roles need enforced differences in model and data access.
• You operate under the EU AI Act, DORA, HIPAA, or a comparable mandate.

Running both

The pairing is common and sensible. LangSmith owns the build and evaluation loop for the applications your team writes. DeepInspect owns the traffic layer for every AI call leaving your environment, including the ones nobody instrumented.

One integration note worth planning: point the LangSmith-instrumented application at the DeepInspect endpoint rather than directly at the provider. Traces still capture the full execution detail, and the policy decision now sits upstream of the call. Requests that policy refuses appear in LangSmith as denials with a policy reason, which gives the developer the feedback loop without moving enforcement into the application.

Pricing approach

LangSmith publishes seat-based pricing with a free developer tier and usage-based trace retention, so a team can start without a purchase order. DeepInspect prices through a sales conversation scoped to deployment size and enforcement requirements. That difference reflects the buyer more than the product: developer tools sell bottom-up, enforcement infrastructure sells through security review.

DeepInspect

This is the gap DeepInspect closes. DeepInspect sits at the AI request boundary as an external enforcement layer: deterministic, identity-aware, and independent of the application generating the traffic. Every request is evaluated against who is asking, what role they hold, and what data the prompt contains.

Enforcement happens inline and can fail closed, with overhead under 50 ms in internal testing against inference times of 500 ms to several seconds. Every decision writes a structured record carrying the identity, the role, the data classification, the policy version, and the outcome, committed before the response returns to the application and stored where that application has no write access. An application that logs its own compliance evidence is attesting to its own behaviour, and separating those two systems is what makes the record hold up under review. The schema detail is in AI audit log schema.

If you are facing the August deadline, let's talk.