Building a Privacy Filter Before Your LLM API Call

Building a privacy filter before your LLM API call means inserting a dedicated preprocessing step that scans assembled prompt text for PII and secrets, applies your policy (allow, redact, or block), and only then forwards the sanitized payload to the model provider—with consistent logging, failure handling, and tests so sensitive data cannot bypass the gate through chat history, RAG chunks, or tool outputs. The privacy filter is not a single regex; it is a module with clear inputs, outputs, and ownership in your request path.

What the privacy filter covers

A production privacy filter should process everything the model will see:

Missing one branch creates a bypass. See LLM Data Leakage for real-world paths.

Architecture

                    ┌─────────────────────┐
                    │  Assemble context   │
                    │  (user+RAG+tools)   │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │   Privacy filter    │
                    │  POST pii-secrets   │
                    └──────────┬──────────┘
                               │
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
           BLOCK            REDACT            ALLOW
         (unsafe)        (suspicious)         (safe)
              │                │                │
              ▼                ▼                ▼
         User error      redacted_text      raw text
              │                │                │
              └────────────────┴────────────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │   LLM provider API  │
                    └─────────────────────┘

IdenticAPI endpoint: POST /api/v1/security/pii-secrets — documented in PII & Secrets Detection docs. Product overview: PII & Secrets Detection.

Core API contract

Request:

{
  "text": "<full assembled prompt string>",
  "redact": true
}

Response fields used by the filter:

• verdict — safe | suspicious | unsafe
• findings — categories and offsets
• redacted_text — when redact: true
• request_id — audit correlation

Policy mapping (recommended):

Secrets (unsafe) should not rely on redaction alone—see How to Prevent API Keys from Leaking into AI Prompts.

Implementation sketch

Pseudocode applicable to Node.js or Python services:

def privacy_filter(assembled_prompt: str) -> tuple[str, str]:
    result = scan_pii_secrets(assembled_prompt, redact=True)
    action = resolve_privacy_action(result.verdict)

    if action == "block":
        raise SecretsDetectedError(result.request_id)

    if action == "redact" and result.redacted_text:
        return result.redacted_text, result.request_id

    return assembled_prompt, result.request_id

Full code: PII Detection in Python, PII Detection in Node.js.

Assembling context safely

Order of operations:

1. Fetch RAG chunks from vector store
2. Load sanitized chat history from your DB (store redacted versions if possible)
3. Concatenate with clear delimiters (--- user ---, --- context ---)
4. Run one scan on the full string (or scan components then merge—document either approach)
5. Pass output to LLM client

Re-scan when tools append new content mid-turn before a follow-up model call.

Redaction vs tokenization

The API provides placeholder redaction ([EMAIL], [API_KEY]). If you need reversible tokens for internal tools, add a vault layer—compare approaches in PII Redaction vs Masking vs Tokenization.

Failure modes

Document your fail closed vs fail open choice in runbooks.

Testing the filter

Synthetic fixtures:

safe: "What are your business hours?"
suspicious: "Email me at user@example.com"
unsafe: "Key: sk-test_abcdefghijklmnopqrstuvwxyz123456"

Integration tests:

• Assert LLM mock never receives raw string on unsafe
• Assert redacted_text used on suspicious
• Assert request_id logged

Use PII Checker for exploratory cases.

Observability

Log structured events:

{
  "event": "privacy_filter",
  "request_id": "req_abc123",
  "verdict": "suspicious",
  "action": "redact",
  "finding_categories": ["email", "phone"]
}

Dashboard metrics: scans per minute, block rate, top categories. Avoid logging prompt bodies in production.

Combining with other guards

Privacy filter addresses sensitive data. Pair with:

• Prompt injection screening for malicious instructions
• Output moderation for unsafe completions
• Agent action guard for tool policies

Order suggestion: injection scan → privacy filter → LLM → output moderation.

Limitations

• Regex detectors miss obfuscated PII
• 32,000-character limit requires splitting large contexts thoughtfully
• Filter does not classify legal basis for processing personal data
• Redaction may reduce answer quality for identity-specific queries—route those to authenticated tools

Review quarterly with PII and Secrets Leakage Checklist.

Related reading

• How to Redact PII Before Sending Data to an LLM
• How to Detect PII in Text with an API
• Secrets Detection for LLM Applications
• PII Detection vs Data Loss Prevention