DSPy Security: A Deployment Acceptance Test for Production Programs
DSPy security depends on the program artifact you load, the cache locations you trust, the LM route you configure, and the application controls around tools and callbacks. This guide turns current DSPy behavior into a production acceptance record with explicit trust boundaries, negative tests, and a narrow role for policy enforcement on routed HTTP model calls.

A DSPy deployment crosses several trust boundaries before its first production prediction. The process loads program state or executable Python and reads memory and disk caches. It also configures an LM engine, invokes local modules and tools, and sends requests to a provider route. Each boundary needs a release decision. Back it with a failed test.
I would refuse a production sign-off built around a single successful prompt. The acceptance record should identify the exact artifact and cache directory. It should also identify the provider destination and tool permissions. Record callback outputs and retry policy. Include the egress path that was tested.
TL;DR
- State-only DSPy saves contain instructions and demonstrations plus signature metadata; whole-program saves contain executable Python and require a trusted source.
- Memory and disk response caches are enabled by default, while provider-side prompt caching follows provider rules for eligibility and retention as well as billing.
- The LM model and API family determine the actual inference route. The engine and callbacks shape its behavior, including retries.
- DeepInspect applies only to authenticated HTTP LM calls deliberately routed through its policy boundary.
Program artifacts define the first trust boundary
DSPy's saving and loading guide separates state-only files from whole-program directories. A state-only save contains optimized instructions and few-shot demonstrations, with signature metadata. The program structure remains in application code. The whole-program mode stores structure and state together as a pickled artifact containing executable Python. DSPy tells readers to load that form only from trusted sources.
Put the artifact mode in the release record. For state-only JSON, record the repository and commit. Add the file digest, optimization run, reviewer, and expected program class. For a whole-program directory, require provenance and an integrity value, plus an approved storage location and a controlled loading identity. A file appearing in the correct deployment bucket proves location. It says nothing about who produced it.
The guide also says saved state excludes API keys and provider choice, as well as temperature. Those values enter through deployment configuration after loading. Review the artifact and runtime configuration as separate release inputs. That separation prevents a harmless-looking state review from silently approving a different model route.
Cache locations require ownership and isolation
DSPy's cache tutorial describes three layers. They are an in-memory response cache and an on-disk response cache. The third is provider-side prompt caching. Memory and disk caching are enabled by default and remain under DSPy's control. The provider determines eligibility and retention, as well as billing, for the provider-side layer.
Inventory the disk path and filesystem owner. Record the mode, volume, backup policy, and cleanup procedure too. Run two worker identities against the same host and confirm that the intended process can read the directory while an unrelated service account receives a denial. A reviewer should be able to point at the mounted directory in a deployment manifest, then match it to the permissions on the running host.
The current tutorial says the disk cache uses Python pickle serialization. DSPy enables restricted unpickling by default and allows teams to register additional custom types. Preserve that setting in the acceptance record. If a required custom type is absent, the documented behavior treats the entry as a cache miss and logs the rejected type. Test that failure before adding an allowlist entry. Broad deserialization permission turns a cache availability fix into a code-execution boundary change.
Cache policy follows data classification
Response caches can retain prompts and outputs beyond the request that created them. Classify those values before choosing capacity and persistence, along with backup or sharing. A development laptop cache and a multi-tenant production volume have different administrators and deletion paths. So does a provider-managed prefix cache.
Build the release test with a distinctive secret-shaped marker that carries no real secret. Send one call with caching enabled and confirm the second call hits the intended cache. Locate the resulting artifact or provider indicator. Then disable the selected layer using DSPy's cache configuration and repeat the request. Verify the expected miss against the cache record. Finish by deleting the test entry through the documented operational procedure.
Provider-side prompt caching deserves its own row. Record the provider and model in that row. Include eligible prefix behavior and retention documentation. Record the account setting separately. DSPy can pass caching instructions, but the provider owns the remote cache semantics. That boundary belongs beside the broader AI agent security guide, which treats storage and third-party services as separate control owners.
LM configuration determines the real egress route
The DSPy LM API reference shows the route-defining settings in one interface. The model uses a provider/model form. The model_type can select automatic, chat, text, or responses behavior. The engine can use native APIs or the LiteLLM compatibility backend. It can also use a custom implementation. Custom engines own their protocol and connection settings.
Capture the effective model name and API family. Add the engine, base URL, TLS owner, credential source, timeout, and allowed network destination. Test DNS and egress restrictions from the running workload rather than a developer shell. A second request to an unapproved host should fail at the network boundary. Another request using an unapproved model should fail at the application's destination policy or the deliberately routed HTTP policy point.
This is where I prefer a dull acceptance screenshot: one deployment revision and one destination allowlist. Add one denied hostname and one provider request ID. It is far more useful than a diagram with six arrows and no test evidence.
Callbacks and tools keep application trust
DSPy callbacks can observe module and LM lifecycle events. They can also observe adapter, tool, evaluation, and optimizer events. They execute application code and may receive sensitive inputs or outputs. List every production callback and its package and version. Add the registration point, output destination, exception behavior, and data fields. Trigger one callback failure and verify that the application follows the selected fail-open or fail-closed behavior without dropping the event silently.
Tools need a separate acceptance row because their authority comes from the application. Record each tool's credential and network reach. Include filesystem access and input validation. Record the approval rule too. Invoke the tool with a permitted test value and a prohibited value, then use a second tenant's identifier. Preserve the denial and the application authorization event.
Application controls authorize each local Python function, retriever, subprocess, and tool credential. The post-authentication gap still applies inside the application: the verified user context must survive until the code that authorizes the operation. Keep that propagation test with the tool owner.
Retries change cost and duplicate-action risk
The LM API reference documents three retries by default for transient network and rate-limit failures, using exponential backoff. Record the configured retry count and timeout. Add the total request budget and behavior after exhaustion. Then inject a transient failure. Verify the number and spacing of attempts through application and provider records.
Model inference is often safe to retry as a read-like operation. Surrounding application behavior may carry side effects. A callback can write to an external system. A tool can create a ticket or send a message. It can also change a record. Give each side effect an idempotency strategy owned by the application. Keep the logical LM operation distinct from transport attempts in logs and cost reports.
Test retry exhaustion as a separate failure path. The application should return the intended error and preserve correlation data. It must also avoid falling through to an unapproved backup destination. A hidden provider fallback changes the trust boundary and data processor. It changes the retention terms too, at the exact moment the primary route is unhealthy.
The deployment acceptance record
A production DSPy release should leave one compact record containing:
- The state-only or whole-program artifact and digest, plus its source, reviewer, and loading identity.
- The memory and disk cache settings and disk path, along with the owner, permissions, retention, backup treatment, and restricted-unpickling status.
- The provider-side cache setting and the provider documentation governing remote behavior.
- The effective model and API family, plus the engine, base URL, credential source, TLS path, and egress allowlist.
- The callback inventory and output destinations, with the data review and tested exception behavior.
- The tool inventory and authorization owner, along with credentials, network permissions, and tenant-isolation tests.
- The retry and timeout settings, with the exhaustion and fallback behavior observed under an injected transient failure.
- The routed HTTP LM policy test, if that path is deliberately configured.
Attach raw denials and request IDs to the record. Configuration snippets show intended state. Failed requests show the boundary held in the deployed environment. This follows the same evidence-first approach used for prompt injection detection in production: a control earns trust through observed decisions on the real request path.
DeepInspect
DeepInspect covers the outbound LM slice when the DSPy application deliberately routes authenticated HTTP inference traffic through it. The application passes caller and role, along with correlation context. DeepInspect evaluates prompt classification and destination against versioned policy before forwarding an allowed request, then records the decision for audit correlation.
The boundary excludes DSPy modules and retrievers. It also excludes tools, optimizers, local traces, memory caches, disk caches, and pickled program artifacts. Provider-side caches remain controlled by the selected provider. Custom engines or direct calls that use another route also bypass the DeepInspect decision point. The deployment acceptance record should say exactly which LM configurations use the routed path and prove that an unapproved direct destination is blocked by the surrounding egress control.
Book a technical deep dive at deepinspect.ai.
Frequently asked questions
- Is a state-only DSPy save safe to load?
DSPy describes the state-only JSON form as human-readable and safe to share. It contains optimized instructions and demonstrations, with signature metadata, rather than executable program structure. Review its provenance and contents because those instructions and examples still influence model behavior. Whole-program saves carry a higher trust requirement because the directory contains executable Python.
- Are DSPy caches enabled by default?
The current cache tutorial says both in-memory and on-disk response caching are enabled by default. Provider-side prompt caching is a separate layer with provider-specific eligibility and retention, as well as billing. Production acceptance should record all three states and the disk location and permissions. It should also record the tested deletion path for retained request data.
- Does restricted unpickling remove disk-cache risk?
It narrows the types the DSPy cache will deserialize. The disk path still needs trusted ownership and restricted write access, plus monitoring and lifecycle management. Register each required custom type deliberately. Test the missing-type behavior first, then document the reason and reviewer for any allowlist addition.
- Can DeepInspect secure DSPy tools and local caches?
DeepInspect has a narrower enforcement scope. It covers authenticated HTTP LM requests deliberately routed through its policy point. Local tools and retrievers remain under application controls. Modules and cache files do too. Callbacks, optimizers, and loaded Python artifacts remain under application, host, and deployment controls. Keep those owners explicit in the acceptance record.