Prevent Sensitive Data Leaks in LLM Prompts and Logs

Many AI data leaks do not require a sophisticated attacker. Someone pastes a customer export into a public assistant, a debug log stores a full authorization header, or a support transcript is copied into an evaluation dataset and shared too broadly. As models gain file uploads, memory, retrieval, and tools, the number of places sensitive content can travel increases. The urgency is not abstract: an organization can lose control of its data while believing it is only experimenting. Data protection for AI starts with a map. What enters the model, what comes back, what is stored, who can access it, and which vendors or tools receive it? Answer those questions for development, testing, production, analytics, support, and training. A privacy statement that describes a product in general terms is not a data-flow design. ## Classify data by use, not only by label Mark public, internal, confidential, personal, regulated, secret, and production-action data. Then define permitted AI uses. A public press release may be summarized anywhere. Internal code may be allowed in an approved coding workspace but not a consumer plugin. Customer health information may require a specific environment and a trained reviewer. The same file can have different risks depending on the task and recipient. Create a simple decision point before submission. The application should know whether a request contains a restricted class and route it to an approved model or block it. Do not rely entirely on users to recognize every sensitive value. Use detection for likely secrets, personal identifiers, credentials, and payment data, but provide an explicit review path for false positives and domain-specific data. ## Minimize before you redact The safest sensitive field is the one that never enters context. Ask what the model actually needs. A support-draft agent may need the product tier and issue history, not a full account export. A document classifier may need a few sections, not an entire contract. Replace identifiers with stable placeholders so the model can reason about relationships without seeing names or numbers. Redaction must preserve task meaning and avoid creating misleading evidence. Mask secrets completely; tokenize identifiers consistently; remove irrelevant rows; retain units and date relationships when needed. Keep the re-identification map outside the model environment with strict access. Test whether the output accidentally reconstructs or repeats the hidden value. ## Treat logs and evaluation data as sensitive Raw prompts often contain more data than the final answer. Set defaults that store metadata and hashes rather than full content. When content is necessary for debugging, redact it before persistence, encrypt it, restrict access, and apply short retention. Separate production support access from broad analytics access. Evaluation datasets are especially easy to mishandle because they are copied for notebooks, vendor tests, and prompt experiments. Version them, document provenance, remove direct and indirect identifiers, and limit downloads. A de-identified record can still identify a person through rare combinations of facts. Use synthetic fixtures for repeatable edge cases when real data is not necessary. ## Check providers and connectors Review the exact service, account tier, API configuration, retention setting, region, subprocessors, and connector permissions. Do not assume that an API and a consumer application have the same data controls. For agent tools, ask what a provider stores from tool calls and whether a connector can export or learn from retrieved content. Minimize outbound data at the gateway. Allow only approved destinations, log the decision, and block secrets from external tools. Make provider changes trigger a data-impact review, especially when adding web search, file handling, memory, or computer use. ## Test leakage as a product behavior Add tests where a user asks the model to reveal hidden instructions, a retrieved document requests a secret, a tool returns confidential fields, or an output includes an input credential. Confirm that redaction occurs before logging and that authorization is checked before retrieval. Test error paths because exception messages frequently expose raw payloads. Review real traces with a privacy owner. Measure blocked submissions, false positives, redaction failures, sensitive fields returned, and retention compliance. A dashboard should make it possible to answer what happened without opening every raw transcript. AI products need useful context, but more context is not free. It increases cost, attack surface, and the chance of a harmful disclosure. Minimize inputs, control destinations, protect logs, and make data handling part of the architecture. The teams that build these habits early will be able to adopt richer AI features without discovering that growth has quietly become a data-governance crisis.