Production LLM Guardrails Checklist

Use this production LLM guardrails checklist before launching or materially updating any AI feature — chat, RAG, agents, or embedded copilots. Each section maps to a concrete control; unchecked items are documented accepted risk.

Guardrails are not a single API call. They are layered controls across input, retrieval, inference, output, tools, rendering, monitoring, and privacy. OWASP LLM risks (injection, insecure output handling, excessive agency, sensitive information disclosure) require architectural coverage, not one filter.

Related depth: What Are AI Guardrails?, Add Guardrails to an LLM Application, LLM Defense in Depth, AI SaaS Guardrails.

---

1. Scope and threat model

1.1 Inventory LLM features and surfaces

• List every product feature that calls an LLM or embedding model
• Document user-visible outputs (chat, email, tickets, exports, voice transcripts)
• Document non-visible outputs (logs, webhooks, background summaries)
• Identify agent/tool features vs read-only Q&A
• Note multi-tenant boundaries if applicable

Pass criteria: Architecture diagram exists; no orphan code paths call models without review.

1.2 Classify context sources

For each feature, list all text entering model context:

• User messages and form fields
• API parameters mapped to prompts
• Chat history (including prior assistant turns if re-sent)
• System prompts and dynamic template variables
• RAG retrieved chunks
• Uploaded files after extraction
• Web pages (crawlers, browsing agents)
• Tool/API responses re-injected into prompts

Pass criteria: Every source labeled trusted (app-authored, integrity-protected) or untrusted (external/user/corpus). See Direct vs Indirect Prompt Injection.

1.3 Define impact if guardrails fail

• Data types at risk (PII, secrets, internal docs, cross-tenant data)
• Tools at risk (email, payments, delete, arbitrary HTTP, code execution)
• Regulatory / contractual obligations documented
• Worst-case scenarios signed off by product and security

Pass criteria: Written risk acceptance for any deferred checklist items.

---

2. Guardrail architecture

2.1 Placement in request pipeline

• Input guardrails run server-side before provider call
• Output guardrails run server-side before client delivery or storage
• Retrieval chunks screened before prompt assembly
• Tool outputs screened before re-prompting
• Agent actions validated before execution (validate tool calls)

Reference: Guardrails Before or After the LLM?, Input vs Output Guardrails.

2.2 Orchestration choice

• Documented use of separate endpoints vs Unified Guard
• Parallel checks where latency-critical (guardrails latency)
• Consistent verdict mapping across detectors (safe/suspicious/unsafe or allow/review/block)

Unified Guard example:

POST /api/v1/guard
Authorization: Bearer idapi_test_your_key_here

{
  "text": "Assembled input or output text",
  "checks": ["prompt_injection", "pii_secrets"],
  "redact": false
}

• decision priority understood: block > review > allow
• Each check result logged with categories and request_id

2.3 Policy router

• Written mapping from verdicts to allow / review / block (block vs review)
• Policy stored in version control, not hard-coded scattered branches
• Per-surface overrides documented (support vs internal vs admin)

Pass criteria: Staging test demonstrates unsafe input blocked with static fallback (no echo).

---

3. Input guardrails

3.1 Prompt injection screening

• POST /api/v1/security/prompt-injection on user-controlled text
• Optional source and context fields set for observability
• RAG chunks scanned at retrieve time (indirect injection)
• Ingest-time scanning for uploaded documents (document injection)

Integration references: TypeScript, Python, Prompt Injection Shield docs.

3.2 PII and secrets

• POST /api/v1/security/pii-secrets on assembled prompt-bound text
• Secrets (unsafe) → block, not redact-and-forward (API key leaks)
• PII policy: block, redact (redact: true), or review per product tier
• Chat history and RAG chunks included in scan scope

See Redact PII Before LLM, PII Secrets Leakage Checklist.

3.3 Input validation (non-ML)

• Schema validation on API payloads (length, types)
• Rate limiting per user/tenant
• File type and size limits on uploads

---

4. Retrieval and RAG guardrails

4.1 Authorization

• Retrieval filtered by tenant/user authorization — not only vector similarity
• Separate indexes or strict metadata filters per tenant (vector DB security)

4.2 Ingest pipeline

• Text extracted and scanned before embedding
• unsafe documents rejected or quarantined
• Provenance recorded (source_url, uploader, hash)

4.3 Retrieve pipeline

• Per-chunk injection scan before inclusion in prompt
• suspicious chunks dropped, down-ranked, or reviewed per policy
• Untrusted framing in system or user prompt (secure retrieved documents)

4.4 RAG-specific risks

• Data poisoning awareness and monitoring (RAG data poisoning)
• Re-index plan when detector rules update
• RAG Security risks reviewed in threat model

---

5. LLM call boundary

5.1 Data minimization

• Only necessary context sent to model provider
• System prompts reviewed for embedded secrets or PII examples
• Provider data retention settings documented

5.2 Model and prompt governance

• Model version pinned or change-controlled
• Prompt template changes reviewed in pull requests
• Jailbreak/injection awareness in safety training for internal teams (injection vs jailbreak)

---

6. Output guardrails

6.1 Output moderation

• POST /api/v1/security/output-safety on every user-visible completion
• Streaming UIs buffer or gate — no raw HTML tokens before check (moderate chatbot responses)
• Stored messages moderated before persistence

AI Output Safety Checklist for rendering-specific items.

6.2 Safe rendering

• Default plain-text encoding in UI
• HTML sanitized with strict allowlist if rich text required (safe AI-generated HTML)
• Content-Security-Policy on pages displaying AI content (XSS prevention)
• Treat model output as untrusted input (LLM output untrusted input)

6.3 Output PII/secrets (if required)

• Scan completions and exports for accidental leakage
• Block or redact before delivery when findings present

---

7. Agents and tools

7.1 Tool permissions

• Least-privilege tool exposure (least privilege agents)
• Allowlist vs blocklist strategy documented (tool allowlist)
• Destructive actions require preview + approval (safe destructive actions)

7.2 Runtime validation

• Schema validation on tool arguments
• Policy engine: allow / review / block before execution
• Tool results treated as untrusted; scanned before re-entry (tool output injection)

7.3 External communication

• Email/send tools: recipient validation, draft vs send separation (secure email agents)
• Web browsing: URL policy, page content scanning (secure browsing agents)

Agent depth: AI Agent Security Checklist.

---

8. Fail behavior and availability

8.1 Documented fail policies

• Per-check fail-open vs fail-closed defined (fail-open vs fail-closed)
• Timeouts and circuit breaker behavior specified
• User-facing fallback messages pre-approved (no internal error leakage)

8.2 Degradation testing

• Simulated guardrail API outage tested in staging
• Alerting on sustained check failures or skip counts

---

9. False positives and human review

9.1 Review workflows

• suspicious / review routed to queue — not silent allow
• Reviewers can label false positives for regression corpus
• Queue access restricted and audited

See AI Guardrails False Positives.

9.2 Tuning process

• Labeled benign and adversarial test sets maintained
• Metrics: block rate, review rate, override rate per category
• Evaluation cadence defined (evaluate guardrails)

---

10. Monitoring, logging, and incident response

10.1 Logging (privacy-safe)

• Log request_id, verdict/decision, risk, finding categories
• Avoid full prompt/completion in production logs when sensitive
• Correlate with application trace IDs

10.2 Alerting

• Alerts on error rate spikes, block rate anomalies, guardrail downtime
• Runbook for disabling features vs fail-closed (explicit approval required)

10.3 Incident response

• Playbook for injection or leakage report
• Credential rotation procedure if secrets detected in prompts
• Post-incident updates to test corpora

---

11. Testing and CI

11.1 Automated regression

• Benign corpus — must allow
• Adversarial corpus — must block or review per policy
• Edge cases — route to review where appropriate

Prompt injection testing, Prompt Injection Security Checklist.

11.2 Pre-release gate

• Checklist reviewed for each LLM-related release
• Staging E2E: poisoned RAG chunk does not reach model
• Staging E2E: unsafe output not rendered in client

---

12. Privacy and compliance

• Server-side-only guardrail API calls (no keys in browser)
• API keys in secrets manager
• Data processing terms for guardrail and model providers documented
• Retention policy for review queue content
• Customer-facing description of automated screening (no overclaiming)

LLM data leakage, LLM privacy filter.

---

13. Guardrails vs moderation clarity

• Team understands guardrails orchestrate multiple checks; moderation is one component (guardrails vs moderation)
• Output moderation not treated as substitute for injection or secrets controls on input

---

Quick reference: IdenticAPI endpoints

Prototype tools: Prompt Injection Checker, PII Checker, AI Output Safety Checker.

---

Sign-off

Completing this checklist does not guarantee safety — probabilistic detectors and evolving attacks require ongoing evaluation. It does ensure common guardrail gaps are consciously addressed before production traffic.

Implement orchestration with Unified Guard and maintain this checklist alongside your release process for every LLM feature change.