How to Prevent API Keys from Leaking into AI Prompts

Preventing API keys from leaking into AI prompts requires scanning every text path that assembles model context—user messages, chat history, retrieved documents, tool outputs, and pasted code—for credential-shaped strings, blocking or stripping matches before the HTTP request leaves your infrastructure, and rotating any key that may have been exposed. API keys in prompts are especially dangerous because model providers, log aggregators, and annotation vendors may retain request bodies longer than you expect, turning a one-time paste into a durable secret exposure.

Why API keys end up in prompts

Developers and end users introduce keys into LLM context through predictable paths:

• Debugging — "Here's my failing curl command: Authorization: Bearer sk-test_..."
• Code paste — Source files containing process.env.STRIPE_SECRET_KEY assignments with accidental literal values
• Retrieved docs — Internal wikis with example configurations checked into Confluence
• Agent tool loops — A search tool returns a GitHub gist containing a .env snippet
• Error messages — Stack traces echoing configured headers from outbound HTTP clients

LLMs do not "know" secrets are sensitive unless your pipeline enforces that boundary. The model may even suggest pasting a key to "help debug faster."

For broader leakage context, read LLM Data Leakage: Causes, Examples and Prevention. For all secret types—not just API keys—see Secrets Detection for LLM Applications.

Detection: what to scan for

IdenticAPI's PII & Secrets Detection flags patterns including:

When any secret category matches, the API returns verdict: "unsafe" and risk: "high".

POST /api/v1/security/pii-secrets

{
  "text": "Use key sk-test_abcdefghijklmnopqrstuvwxyz123456 for billing.",
  "redact": false
}

Do not use real keys in tests. Use clearly synthetic prefixes like sk-test_ and documented test vectors.

Response handling policy

Recommended policy for secrets in user-facing LLM apps:

1. Block the LLM request immediately when verdict is unsafe
2. Show a clear user message: "Remove API keys and secrets before submitting"
3. Log request_id, timestamp, and categories—never the raw secret
4. Alert security if repeated attempts suggest deliberate exfiltration testing
5. Rotate credentials if a production key may have reached a third-party log

Redaction alone is insufficient for secrets. Replacing a key with [API_KEY] still confirms a secret existed and does not undo transmission if the raw prompt already logged.

Architectural controls

Never embed keys in prompts

Load secrets from environment variables or a secrets manager at runtime. System prompts should reference capabilities, not credentials:

Bad:  "Use Stripe key sk-test_abc123..."
Good: "Call the billing tool; credentials are injected server-side."

Server-side tool execution

When an agent needs an external API, your backend attaches keys to outbound requests. The model sees only tool results shaped to exclude secrets.

Pre-flight scanner in the request path

Incoming message → secrets scan → if unsafe: 400 + guidance
                                → if safe/suspicious: continue to PII redaction → LLM

Place the scanner in your API route, not the browser, so keys cannot bypass client-side checks.

CI and repository scanning

Prevent keys from entering RAG corpora at the source. Git pre-commit hooks and CI secret scanners complement runtime LLM guards—they address different lifecycle stages.

Example: blocked request flow

User input (synthetic):

Why does this fail? curl -H "Authorization: Bearer ghp_abcdefghijklmnopqrstuvwxyz1234567890AB" ...

API scan result:

{
  "verdict": "unsafe",
  "findings": [
    {
      "category": "github_token",
      "reason": "Detected github token (secret)",
      "confidence": 0.95
    }
  ]
}

Application response:

{
  "error": "secrets_detected",
  "message": "Remove tokens and API keys from your message before submitting."
}

No LLM call occurs. If this was a real token, open a rotation ticket regardless of block success.

Private keys and certificates

PEM private keys are high-severity findings:

-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7SyntheticExample
-----END PRIVATE KEY-----

Treat these like API keys: block, rotate, audit. Details in How to Detect Private Keys and Tokens in Text.

Developer workflow guardrails

• Add a "paste code" UI that runs local or server-side secret scan before send
• Strip .env files from upload allowlists in document chat features
• Train support staff not to ask customers for passwords or full card numbers in chat
• Review prompt templates in pull requests with the same rigor as application code

Use the PII Checker to validate detection on synthetic samples. Full API reference: docs.

Limitations

Pattern-based secret detection:

• Misses heavily obfuscated or encrypted secrets
• May match dummy values in documentation that look like real keys
• Cannot tell if a detected string is active, revoked, or a honeypot
• Does not replace vault-based secret storage discipline

Assume detection will be imperfect; combine scanning with least-privilege keys, short TTLs, and rapid rotation.

Related reading

• How to Detect Credentials in User Input
• PII and Secrets Leakage Checklist
• PII Detection in Node.js — wire scanning into your API layer
• Building a Privacy Filter Before Your LLM API Call