Handling PII in LLM Applications
Sending personally identifiable information (PII) to LLM APIs is one of the most common compliance risks in enterprise AI deployments. Names, email addresses, phone numbers, health information, and financial data in user inputs or documents may be inadvertently sent to third-party API providers, violating GDPR, HIPAA, and other regulations. This guide covers how to detect, redact, and handle PII safely in production LLM applications.
What Counts as PII in LLM Contexts
PII is any information that can identify a specific individual. In LLM application contexts, this includes: names, email addresses, phone numbers, social security numbers, passport numbers, IP addresses, device identifiers, location data, financial account numbers, health and medical information, and biometric data. Quasi-identifiers — data that does not uniquely identify someone on its own but can do so in combination (age + occupation + city) — are also PII under GDPR. The challenge in LLM applications is that PII often appears embedded in natural language: "Can you help me reply to John Smith at [email protected] about his invoice?"
PII Detection Approaches
Three approaches exist for detecting PII in text: (1) rule-based detection using regular expressions for structured PII like email addresses, phone numbers, credit card numbers, and SSNs — fast and precise for known patterns; (2) named entity recognition (NER) models that identify names, organisations, and locations in unstructured text — more comprehensive but requires model inference; (3) LLM-based detection, asking a model to identify all PII in a text and return its positions — the most accurate approach for nuanced cases but adds latency and cost. In production, combine rule-based detection (fast, low-cost, handles structured PII) with NER or LLM detection (for unstructured natural language PII).
Redaction and Anonymisation Strategies
PII redaction replaces PII with a placeholder before sending the text to an LLM API. The most useful redaction strategies are: (1) category substitution — replace "John Smith" with "[PERSON_1]" and "[email protected]" with "[EMAIL_1]" using consistent labels within a document so the model can still understand the relationships; (2) synthetic substitution — replace with realistic fake data ("John Smith" → "Alex Johnson") to preserve natural language flow; (3) complete removal — strip PII without replacement, simplest but can make the text incoherent. After the LLM processes the redacted text, reverse-substitute labels in the output if the application needs to present results with real names.
GDPR and HIPAA Compliance Considerations
GDPR requires a legal basis for processing personal data. Sending PII to a third-party LLM API constitutes data processing and requires either user consent or a legitimate interest assessment. Most enterprise LLM API contracts include a Data Processing Agreement (DPA) — verify you have signed one with your provider. Under GDPR, users have a right to deletion — if their PII was sent to the model API for training (check your provider's terms carefully), deletion may be technically impossible. HIPAA-covered entities must sign a Business Associate Agreement (BAA) with any LLM provider processing PHI. Both OpenAI and Anthropic offer BAAs for enterprise customers on qualifying plans.
Implementing a PII Proxy
A PII proxy sits between your application and the LLM API, intercepting all requests, detecting and redacting PII before forwarding to the API, and restoring redactions in the response. This architecture centralises PII handling so that individual application teams do not need to implement detection themselves. The proxy maintains a redaction map per session, allowing consistent label-to-value restoration. Microsoft Presidio is a popular open-source library for building PII proxies. Commercial offerings include Amazon Comprehend for PII detection and AWS Bedrock Guardrails for real-time PII filtering. A PII proxy adds 20-100ms of latency depending on text length and detection complexity.
Testing PII Handling in CI/CD
Include PII detection tests in your CI/CD pipeline. Create a test corpus of synthetic text containing various PII types and verify that your detection catches all of them. Test false positives too — legitimate text that should not be redacted (a business name that contains a person's name, a product code that looks like an SSN format). Run your full PII pipeline against this corpus on every PR. For production monitoring, log (without storing the actual PII) the percentage of requests containing detected PII, the PII categories detected, and the detection latency. Alert when the PII detection rate changes significantly, which may indicate a new use pattern requiring additional review.
Try These Tools
Detect personal information (email, phone, SSN, credit card, IP, date of birth) in text before sending to LLMs.
Remove invisible Unicode, escape injection keywords, and strip dangerous content from LLM input.
Clean and sanitize text for LLM input by stripping HTML, normalizing Unicode, and collapsing whitespace.
Detect prompt injection attacks in text with pattern matching and a 0-10 risk score.
FAQ
- Do LLM API providers use my data to train their models?
- OpenAI and Anthropic do not use API data for training by default — only data from their consumer products (ChatGPT, Claude.ai) may be used if users have not opted out. API customers are advised to review their providers' current terms, as policies evolve. Enterprise agreements typically include explicit contractual guarantees against using your data for training.
- Can the LLM itself detect PII reliably?
- LLMs are good at identifying obvious PII in structured contexts but can miss PII embedded in creative or informal text. More importantly, using an LLM to detect PII means you have already sent the PII to the API. Use pre-processing detection (before the API call) for compliance, and LLM detection only as a secondary validation layer.
- Is it safe to use synthetic data for testing PII detection?
- Yes — always use synthetic data for testing. Generate realistic fake names, addresses, and SSNs using libraries like Faker, or use PII from publicly available datasets created specifically for testing purposes. Never use real customer PII in test environments.