Prompt Injection
·IdenticAPI

Prompt Injection Security Checklist for Production LLM Apps

A production-ready checklist for prompt injection defenses — input screening, retrieval hardening, output handling, monitoring, and testing.

Use this prompt injection security checklist before shipping or materially updating any production LLM application — chatbots, RAG assistants, document Q&A, or browsing agents. Each item maps to a concrete control; unchecked items are documented accepted risk.

Prompt injection cannot be eliminated with a single filter. OWASP LLM01 (Prompt Injection) treats it as an architectural concern requiring layered defenses, monitoring, and testing. This checklist operationalizes that guidance for engineering teams.

Related depth: What Is Prompt Injection?, Prevent Prompt Injection, Detect Prompt Injection, Testing Guide.


1. Threat model and scope

1.1 Inventory all context sources

List every text path that becomes LLM input:

  • User chat messages and form fields
  • API parameters consumed by LLM features
  • Uploaded files after text extraction
  • Vector retrieval chunks (RAG)
  • Web pages fetched by agents or crawlers
  • Tool/API responses re-injected into context
  • Multi-turn chat history (including prior assistant turns if re-sent)

Pass criteria: Diagram or document exists; no "unknown" sources remain.

Failure impact: Unscreened boundary = bypass path. See Direct vs Indirect Prompt Injection.

1.2 Classify trust levels

For each source:

  • Mark as trusted (application-authored, integrity-protected) or untrusted (external/user/corpus)
  • Record owner team and update frequency
  • Note tenant isolation requirements (multi-tenant RAG)

Pass criteria: Every source has a trust label and owner.

1.3 Define acceptable impact

Document worst-case outcomes if injection succeeds:

  • Data types at risk (PII, secrets, internal docs)
  • Tools at risk (email, payments, delete, arbitrary HTTP)
  • Regulatory / contractual obligations

Pass criteria: Product and security stakeholders sign off on impact assumptions.


2. Input and retrieval screening

2.1 Server-side detection on user input

  • Screen user messages before LLM provider call
  • Execution on server — not browser-only JavaScript
  • Use Prompt Injection Shield or equivalent categorized detector
POST /api/v1/security/prompt-injection
Authorization: Bearer idapi_test_your_key_here
{"text": "<user message>", "source": "chat_input"}
  • Handle verdict: safe, suspicious, unsafe with written policy

Pass criteria: Staging test shows unsafe message blocked with logged request_id.

Integration: TypeScript, Python, docs.

2.2 Screen RAG chunks at retrieve time

  • Each retrieved chunk screened before prompt assembly
  • unsafe chunks dropped or document quarantined
  • suspicious policy defined (drop, down-rank, or human review)

Pass criteria: Poisoned test chunk in vector DB does not reach model in E2E test. See Indirect Prompt Injection in RAG.

2.3 Screen at document ingest

  • Full extracted text scanned before embedding/index
  • Upload rejected or quarantined on unsafe
  • Parser documented (PDF, DOCX, HTML)

Pass criteria: Synthetic poison PDF never indexed. See Document Prompt Injection.

2.4 Screen agent web fetch content

  • HTML/text extraction output screened pre-LLM
  • SSRF protections on fetch (block private IPs, metadata URLs)
  • URL policy documented (allowlist vs open web)

Pass criteria: HTML comment injection fixture blocked. See Web Page Prompt Injection.

2.5 Avoid keyword-only filtering

  • No production path relies solely on blocklists
  • Custom keywords, if any, supplement — not replace — layered detection

Pass criteria: Architecture review confirms API or multi-signal detector on all untrusted paths. See Keyword Filter Limitations.


3. Prompt and context design

3.1 Untrusted data delimiters

  • Retrieved/web/document text wrapped with explicit UNTRUSTED labels
  • Instructions tell model not to follow embedded commands (defense in depth)

Pass criteria: Prompt template reviewed; delimiters present in all RAG/agent paths.

3.2 Minimize system prompt secrets

  • No API keys, passwords, or private URLs in system prompt
  • No unnecessary internal tool schemas in user-visible stack
  • Sensitive rules enforced in application code post-model

Pass criteria: System prompt audit completed. See System Prompt Extraction.

3.3 Chunk and context limits

  • Maximum tokens/chunks per request defined
  • Oversized uploads rejected or split with per-segment screening

Pass criteria: Limits documented in runbook.


4. Tools, agents, and actions

4.1 Least privilege tool access

  • Each agent role has minimal tool set
  • Destructive tools disabled by default
  • Write/admin tools require separate entitlement

Pass criteria: Tool manifest reviewed; no universal "admin" tool on customer chat.

4.2 Human approval for high-impact actions

  • Payments, bulk delete, external email, credential changes require confirmation
  • Approval cannot be skipped via model text alone

Pass criteria: Test confirms injected "approve all" text does not bypass UI approval.

4.3 Separate read from act

  • Summarization/browsing pipelines distinct from tool execution policy engine
  • Tool calls validated server-side against allowlists

Pass criteria: Architecture diagram shows policy gate on tool invocation.


5. Output and downstream handling

5.1 Treat model output as untrusted

  • Output moderation or policy check before user display (complements input screening)
  • No direct eval, SQL, or shell on model output

Pass criteria: Output path documented; injection in input cannot directly execute code.

5.2 Safe rendering

  • HTML/markdown sanitized before browser render
  • Secrets patterns scanned in output where applicable

Pass criteria: XSS and leakage review for chat UI.

5.3 Block responses without echoing attacks

  • Error messages generic — do not repeat blocked injection payload

Pass criteria: UX review of block states.


6. Logging, monitoring, and response

6.1 Structured security logs

Log for each screen:

  • request_id from API response
  • verdict, risk, finding categories
  • Source type (chat_input, rag_chunk, web_fetch)
  • Action taken (allow, review, block)
  • Correlation ID to user/session (hashed if needed)

Pass criteria: Sample log line verified in staging.

Example:

{
  "event": "prompt_injection_screen",
  "request_id": "req_chk_100",
  "verdict": "unsafe",
  "risk": "high",
  "categories": ["instruction_override"],
  "source": "chat_input",
  "action": "blocked"
}

6.2 Alerting

  • Alert on spike in unsafe verdicts
  • Alert on repeated blocks from single session
  • Runbook for corpus poison incidents (disable document, re-scan)

Pass criteria: On-call runbook linked from monitoring.

6.3 Incident response

  • Process to quarantine document IDs or URLs
  • Customer notification template for corpus poison
  • Post-incident fixture added to regression suite

Pass criteria: Tabletop exercise or documented IR steps.


7. Testing and release gates

7.1 Fixture library

  • Categorized fixtures: override, extraction, role, delimiter, exfil, indirect
  • Benign negatives included (false positive control)

Pass criteria: Fixtures cover all detection categories in examples.

7.2 Automated CI tests

  • API fixture tests on PRs touching LLM paths
  • E2E test: direct injection blocked before provider call
  • E2E test: indirect RAG/document injection handled

Pass criteria: CI job green on main. See Prompt Injection Testing.

7.3 Manual red-team cadence

  • Quarterly review with paraphrased and multilingual payloads
  • Findings tracked; new fixtures added within SLA

Pass criteria: Last red-team date within policy window.

7.4 Pre-release sign-off

  • Checklist reviewed for major releases
  • Regression after system prompt or tool changes

Pass criteria: Signed checklist attached to release ticket.


8. Operations and third parties

8.1 API key hygiene

  • Production keys in secrets manager — not repos
  • Separate test keys for CI
  • Key rotation procedure documented

8.2 Provider and dependency review

  • LLM provider data handling understood
  • PDF/HTML parser updates trigger re-test

8.3 Usage and availability

  • Rate limits and usage_units budget for screening understood
  • Fail-closed vs fail-open documented for API outage

Pass criteria: Outage behavior tested (screening unavailable → block or degrade gracefully per policy).


9. Developer tooling


10. Explicit non-goals (set expectations)

Acknowledge with stakeholders:

  • No control guarantees blocking all future injection techniques
  • safe verdict does not guarantee safe model behavior
  • Detection reduces risk; tool limits and output handling remain mandatory
  • Compliance with OWASP GenAI guidance is process + architecture — not a certificate

IdenticAPI Prompt Injection Shield provides categorized risk signals — not absolute safety.


Quick reference: verdict policy template

VerdictSuggested defaultHigh-risk product override
safeAllow LLM callAllow
suspiciousLog + allow or review queueBlock or require review
unsafeBlockBlock

Document your product's column in the security runbook.


Minimum viable bar (MVP launch)

If time-constrained, do not ship without at least:

  1. Server-side user input screening with block on unsafe
  2. RAG retrieve screening if RAG is enabled
  3. Tool least privilege
  4. Logging with request_id
  5. Ten direct + indirect fixtures in CI

Expand to full checklist post-MVP.


Print this checklist for release reviews. Update it when your architecture, tools, or retrieval sources change — prompt injection risk changes with every new context path you add.

Frequently asked questions

How often should I review this checklist?

At launch, after major prompt or model changes, when adding RAG or agent tools, and following any security incident involving untrusted text.

Is input screening sufficient alone?

No. Combine input screening with retrieval hygiene, tool policies, output moderation, and monitoring.

What is the minimum viable control?

Server-side screening of user and retrieved text before the LLM call, plus logging of verdicts without storing raw sensitive payloads.

Where do I start implementing?

Add Prompt Injection Shield to your pre-LLM path, then expand to Unified Guard if you also need PII and output checks.

Related reading