DeepInspect vs WhyLabs: Open-Source Observability and Runtime Enforcement

WhyLabs is now an open-source project rather than a hosted vendor. The company built an ML and LLM observability platform on statistical data profiling, with two open-source libraries feeding it: whylogs, which creates efficient, mergeable statistical profiles of tabular, text, image, and embedding data, and LangKit, which extracts safety, quality, relevance, and sentiment signals from LLM prompts and responses. In early 2025 WhyLabs discontinued its commercial operations. It released the AI Control Center platform as open source on or about January 23, 2025, and its hosted service remained available to existing customers only until March 9, 2025 (WhyLabs, WhyLabs open-source docs).

Anyone comparing WhyLabs today is either running the open-source libraries or deciding what to adopt now that the hosted service has ended. I want to describe what WhyLabs is now and where a runtime enforcement layer sits, honestly.

TL;DR

WhyLabs was an ML and LLM observability platform, now open source: whylogs for statistical data profiling and LangKit for LLM safety and quality signals, with the AI Control Center released to GitHub after the hosted service ended in March 2025. DeepInspect is a runtime proxy that decides whether each model call proceeds based on identity and writes a per-decision audit record. One profiles and monitors data and model behavior. The other authorizes each call and records the ruling.

WhyLabs: statistical profiling, now in the open

WhyLabs's core idea is the profile. Rather than shipping raw data, whylogs computes compact statistical summaries: distributions, counts, and quality signals that can be merged across time and compared to catch drift and anomalies. LangKit applies the same idea to LLM traffic, turning prompts and responses into safety, relevance, and sentiment metrics that a monitor can track and alert on. That profiling-first design was privacy-preserving by construction, because the summaries left the environment, not the underlying records.

With the hosted AI Control Center now open source and the SaaS retired, using WhyLabs means running the libraries yourself and pointing them at your own backend or the open-sourced platform. The value is still the same: seeing drift, quality, and safety signals over time.

Profiling and monitoring record what happened and how the data or model behaved. That is observability work, and it has genuine forensic and quality value and no ability to decide whether a call should proceed. The forensic-versus-prevention distinction is in why AI security must be inline.

Profiling behavior versus authorizing callers

DeepInspect is a stateless proxy sitting between authenticated users or agents and any LLM endpoint. Every call traverses it, and the decision happens before the provider sees the request.

The distinction is authority. WhyLabs profiles traffic and flags drift or unsafe signals after the fact. DeepInspect governs the call and decides, per request, whether this authenticated identity in this role may send this class of data to this model. A profile can show that regulated data flowed to a model last week. An identity-bound layer refuses that call before it lands, keyed to who is asking, which is described in identity-aware AI gateway architecture.

The record is the second gap. A statistical profile is a monitoring summary an application produces about its own traffic. A regulator under the EU AI Act asks for a per-decision record committed independently of the application, keyed to identity and policy, which is developed in signed audit logs for AI requests.

Feature comparison

| Capability | WhyLabs (open source) | DeepInspect | |---|---|---| | Open source | Yes (whylogs, LangKit, platform) | No | | Statistical data profiling | Yes, core | No | | LLM safety and quality signals | Yes (LangKit) | Content checks available | | Hosted commercial service | Retired March 2025 | Yes | | Identity-bound per-request policy | No | Core | | Inline blocking of a request | No | Yes | | Application-independent audit record | No, monitoring profiles | Core, tamper-evident | | Data classification against custom classes | No | Yes | | Model-agnostic across HTTP LLM endpoints | Library-dependent | Yes | | Primary buyer | ML and data engineering | Security and compliance |

The overlap is thin because the two do different jobs. WhyLabs profiles and monitors. DeepInspect authorizes and produces the independent record.

Where WhyLabs is the better answer

If the requirement is drift detection, data-quality monitoring, and LLM safety signals from an open-source, privacy-preserving profiling library you run yourself, WhyLabs still delivers that and a policy proxy does not. For a data or ML team that wants statistical monitoring without sending raw records anywhere, whylogs and LangKit are a proven fit, now maintained in the open.

Monitoring data and model behavior is the baseline for any serious ML program. WhyLabs covers that, and the open-source release means teams already invested in it can keep running it.

Where the gap opens

Two requirements move a team past what profiling and monitoring provide.

The first is prevention. A profile records that a disallowed call happened; it cannot stop the next one. Refusing the call before it reaches the model is an inline job, keyed to identity, which is the post-authentication gap.

The second is evidence independence. A statistical profile is a summary the application produces about itself. A per-decision record committed on storage the application cannot alter is what a regulator treats as proof, and the retention requirements are in EU AI Act Article 19 logging.

Pick WhyLabs if

• You want open-source statistical profiling for data and model drift.
• Privacy-preserving monitoring that summarizes rather than ships raw data matters to you.
• LLM safety and quality signals via LangKit fit your monitoring stack.
• You are comfortable running and maintaining the open-source libraries yourself.
• Per-decision authorization evidence is already handled elsewhere.

Pick DeepInspect if

• You need live model calls refused, not profiled after the fact.
• An auditor or enterprise customer wants per-decision records tied to an identity.
• You need audit evidence independent of the application that produced the traffic.
• Policy must differ by role and be enforced at request time.
• You operate under the EU AI Act, DORA, HIPAA, or a comparable mandate.

DeepInspect

This is exactly what DeepInspect does. DeepInspect sits inline between your users or agents and the LLM APIs they call. For every request and response, it evaluates identity, data classification, model authorization, and organizational policy, then makes a pass or block decision before the traffic reaches the model.

The record is the second half of the product. Every decision writes a structured entry containing the authenticated identity, the resolved role, the data classes found in the prompt, the policy version in force, and the outcome. That entry commits before the response returns, on storage the application cannot write to, which is what separates it from a monitoring profile. Enforcement adds under 50 ms in internal testing against inference times of 500 ms to several seconds. Statistical observability and identity-bound enforcement address different needs, and a team running whylogs can add an enforcement layer on the same traffic.

If you are facing the August deadline, let's talk.