Zero Data Retention vs "We Don't Train on Your Data": A CTO Checklist
Your vendor's DPA says they will not use customer data for model training. Good. That is table stakes.
It does not answer the questions your CISO, your customer’s procurement team, and EU regulators actually ask: Are prompts stored? For how long? Who can access them? What happens in a subprocessors chain? What survives a breach?
Policy language and architectural guarantees are not the same thing.
Three models of data handling
Model 1: May train or retain broadly. Consumer products and some API tiers historically blurred lines. Enterprise agreements usually exclude training on customer API data. Retention for abuse monitoring may still apply.
Model 2: Retained but not trained. Prompts logged for 7–30 days for safety, support, or debugging. Legal promise not to train, but content exists on disk somewhere.
Model 3: Processed and discarded. Prompts exist in memory for inference, then gone. Compliance metadata (timestamp, model ID, token count) logged separately. No prompt content in durable storage.
Most enterprise buyers think they are buying Model 3 when they signed up for Model 2.
Why this matters now
Cyberhaven's 2025 data, cited across enterprise security reports, found 43% of employees have pasted confidential data into AI tools. If your production API retains prompts even briefly, that employee behavior and your application behavior share the same risk surface.
CSA's 2026 Shadow AI research found 71% of generative AI connections use personal accounts, bypassing enterprise DPAs entirely. Your API architecture is the part you control.
CTO checklist: 10 questions
Retention and training
- Are prompts stored after the response is returned? If yes, for how long and in what systems?
- Is "no training" a policy opt-out or a technical impossibility?
- Do abuse monitoring or safety pipelines persist prompt content? Under what retention TTL?
Access and subprocessors
- Who can access prompt content internally? Support engineers, ML teams, contractors?
- Which subprocessors receive prompt data? Model providers, logging vendors, analytics tools: list the full chain.
- Do failover routes cross jurisdictions? A EU request failing over to US inference may change legal exposure mid-request.
Audit and compliance
- What appears in audit logs: metadata or content? GDPR and SOC 2 auditors increasingly ask for this distinction.
- Can you export or delete prompt history on request? Right-to-erasure is hard if you never stored content (easy) vs if you logged everything for 30 days (harder).
- Does your SOC 2 / ISO scope cover inference infrastructure? A DPA referencing certifications that do not cover the inference path is a gap.
- What does your customer contract promise vs what engineering built? Sales decks drift from architecture.
Score honestly. More than three "we're not sure" answers means you have a governance project, not a vendor checkbox.
Side-by-side comparison
| Question | "We don't train on your data" | Zero retention by architecture |
|---|---|---|
| Prompts in database after response | Often yes (TTL varies) | No |
| Training use | Contractually excluded | N/A (no retained content) |
| Abuse monitoring | May retain snippets | Metadata signals only |
| Support debugging | May require content access | Reproduce from client-side logs |
| Breach impact | Content may leak | Metadata only |
| Customer erasure request | Requires deletion workflow | Nothing to delete |
Autark is built for the right column. Prompts are processed and discarded. We do not retain request content or use it for training. Compliance metadata is logged separately. See Trust Center for the full breakdown.
Questions to paste into your next RFP
- Describe prompt lifecycle from receipt through deletion, with timestamps.
- List all systems that persist prompt content, including subprocessors.
- Provide architecture diagram showing where prompt plaintext exists in memory vs storage.
- Confirm whether failover changes inference jurisdiction.
- State whether audit logs contain prompt content or metadata only.
- Describe customer-managed encryption key support, if any.
Vendors that answer with marketing paragraphs instead of diagrams are telling you something.
The bottom line
"We don't train on your data" is a sentence. "We physically cannot store your data" is a system design.
Buyers in 2026 should demand both: contractual protections and architectural proof. Your employees are already pasting secrets into consumer chat tabs. Your production pipeline should not be the second leak.