Haystack Security Starts With the Trusted Execution Boundary

Haystack pipelines execute components, move retrieved content into prompts, and call model endpoints. Their security boundary begins before the first component runs. The application supplies validated inputs and trusted pipeline definitions, along with controlled secrets and an approved network path. Production acceptance should test each of those assumptions against the deployed pipeline.

TL;DR

• Haystack assumes a trusted execution environment. Applications must validate URLs and paths. They must also validate filters and queries.
• Treat pipeline YAML and snapshots as executable artifacts. Apply source-code access controls and review.
• Pin each generator's endpoint and secret source. Pin its model and identity contract too, then test direct-call bypasses.
• DeepInspect applies only when the generator's model HTTP request is deliberately routed through its proxy.

Haystack defines a clear application trust boundary

The Haystack security policy says the framework is intended to run inside a trusted execution environment. Applications are responsible for validating and sanitizing user-supplied URLs and file paths before those values reach Haystack. The same responsibility covers filter expressions and queries. Put those checks in the application path before pipeline execution.

Pipeline definitions deserve the same controls as source code. Haystack pipeline YAML can load user-defined components, callables, and filters through dynamic imports. PipelineSnapshot is also operator-controlled state that can be edited and later resumed. Store both artifact types in an approved repository and restrict write access. Require review, then record the digest deployed with each release.

I would reject any design that accepts pipeline YAML through a customer upload and calls that extensibility. A twelve-line YAML file can look harmless in a pull-request preview while still selecting executable Python by name. The review has to cover the imported component. It must also cover the runtime permissions granted to it.

Input and component controls belong before execution

Start the trusted-execution checklist with every external value that can influence a component. Name the schema and maximum size for each URL and path. Do the same for each query and filter. Cover each document field and metadata value too. Also name the allowed characters and rejection behavior. Resolve hostnames before URL-fetching components run. Reject private or reserved address ranges where the use case requires it. Enforce egress policy outside the Python process.

Then inventory components by import path and package version. Record who approved each custom component. Document which files and environment variables it can read, along with every destination it can reach. For pipeline definitions and snapshots, preserve the repository path and commit. Preserve the digest and reviewer too. Record the deployment identifier.

Retrieved documents need their own treatment. Haystack's policy describes prompt injection in documents and metadata as an application responsibility. The prompt injection defense in depth guide covers layered controls around that content. A trusted document store can still contain attacker-written text after a compromised ingestion path or overly broad editor permission.

Generator routes need an explicit destination contract

Haystack's OpenAIChatGenerator documentation shows the route-changing control plainly. The component uses OPENAI_API_KEY by default and supports Secret.from_env_var. It also accepts generation parameters. Custom OpenAI-compatible deployments are supported through api_base_url.

A custom api_base_url changes the generator's HTTP destination. Record the component name and model identifier. Preserve the exact base URL and DNS expectation. Record the TLS requirement and secret reference. Preserve the timeout and retry behavior. Record the allowed runtime overrides. An empty api_base_url and an explicit internal endpoint represent different traffic paths. Treat a change between them as a reviewed configuration release.

Keep secrets outside pipeline text and application logs. Scope the provider credential to the approved project where the provider supports that control. Restrict environment-variable access to the generator runtime. Rotate a credential after exposure, then test that a missing secret stops the route instead of selecting an undocumented fallback.

The AI gateway architecture guide explains where an HTTP policy point can sit. Confirm the destination in the generator configuration and capture one request at that endpoint. A diagram does not reroute a request.

Identity context must survive the generator call

A shared provider key identifies the deployment to the model endpoint while leaving the initiating employee, service, or agent unidentified. Carry an authenticated principal and role beside the application route and data classification. Carry the requested model and policy version too. Define which component signs or otherwise protects that context before another service consumes it.

The identity contract should name the claim source, audience, issuer, and expiry. Document mapping rules and the behavior for missing or expired claims, then cover duplicated and contradictory claims. Per-role AI policies provides the policy side of that contract.

Avoid placing unrestricted identity headers under user control. Construct them after authentication at a trusted boundary. If the generator client cannot carry the required context in the approved form, put the binding in the service that owns the outbound call. Preserve the correlation identifier there.

The trusted-execution and generator-route checklist

Use one acceptance record per deployed pipeline and environment. I want the record to contain these seven sections:

• Input validation: List every external field and validator version. Include the size limit and canonicalization step. Record the allowed destination rule and rejection response.
• Component trust: Pin pipeline and snapshot digests. Record component import paths and package versions. Name the reviewers and runtime permissions. Preserve the approved outbound destinations.
• Secrets: Name each secret reference and owner. Preserve its scope and rotation date. Record the permitted readers and redaction rule. Define the missing-secret behavior.
• Base URL: Preserve the exact generator base URL and model. Record the TLS expectation and DNS result. Preserve the timeout and retry policy. Name the allowed override source.
• Identity context: Record the principal and role. Include the application route and data class. Preserve the claim issuer and policy version. Record the correlation identifier.
• Bypass testing: Attempt a direct provider call and an altered base URL. Also test an unapproved component and modified snapshot. Test a private-address fetch and missing identity. Test a missing secret.
• Request evidence: Retain the timestamp and pipeline digest. Preserve the component version and destination. Record the identity reference and policy decision. Preserve the provider response reference and final action.

A green acceptance requires captured evidence for each test, including the denied direct call. The most useful screenshot is often a terminal showing the generator's approved hostname beside a blocked request to the provider's public hostname.

DeepInspect

DeepInspect applies when OpenAIChatGenerator or another generator deliberately sends its HTTP model request through the DeepInspect proxy. At that boundary, policy can evaluate the supplied identity and role. It can also evaluate the data classification and route. The model destination and request content can be evaluated before forwarding. The proxy can then write a per-decision audit record independent of the Haystack application.

Haystack component trust and pipeline validation stay with the application and infrastructure owners. So do snapshot integrity and secret storage. SSRF controls and local execution stay with those owners too. Direct provider calls also sit outside DeepInspect when network policy permits them to bypass the proxy. Book a technical deep dive at deepinspect.ai.