AI Security Checklist for Production SaaS

Use this AI security checklist for production SaaS before launching or materially updating any LLM-powered feature — and re-run it quarterly or after major architecture changes. Each item maps to a concrete control for multi-tenant software: authentication, injection, data protection, output safety, agents, rate limits, logging, monitoring, and incident response.

Unchecked items are documented accepted risk signed by engineering and security stakeholders.

This is an evergreen baseline — not a one-time audit. Pair it with LLM Security Testing Before Launch for pre-release verification and Production AI Security Stack for architectural context.

---

1. Product scope and threat model

1.1 Feature inventory

• Every LLM feature documented (chat, RAG, agents, embeddings, batch jobs, exports)
• Customer-facing vs internal-only surfaces labeled
• Multi-tenant data boundaries defined per feature
• Third-party model providers and regions listed

Pass criteria: Architecture diagram with trust boundaries; no orphan model calls.

1.2 Impact assessment

• Worst-case outcomes documented (data leak, XSS, cross-tenant access, financial action)
• Regulatory obligations mapped (GDPR, HIPAA, PCI scope, customer DPAs)
• OWASP LLM Top 10 risks addressed or explicitly deferred (guide)

Pass criteria: Risk register linked to checklist sections.

1.3 Change management

• Checklist re-run triggers defined (new model, RAG corpus, tools, major prompt change)
• detector_version and policy config version tracked in releases

Pass criteria: Release template includes AI security section.

---

2. Authentication and authorization

2.1 User identity on every LLM path

• Unauthenticated users cannot trigger paid/unlimited model calls
• Session binding — agent acts as authenticated principal, not anonymous super-user
• API keys for programmatic access scoped and rotatable

Pass criteria: Integration test proves unauthenticated request rejected.

2.2 Authorization before side effects

• LLM cannot invoke privileged APIs without server-side authZ check
• Tool execution verifies tenant scope — not model assertion
• Step-up auth for destructive or financial operations

Pass criteria: Forged cross-tenant tool args fail in test. See AI agent permissions.

2.3 API key hygiene

• IdenticAPI and model keys in environment/secrets manager — never client bundles
• Separate test vs production keys
• Rotation runbook exists (API key practices)

Pass criteria: Client bundle scan finds no idapi_ or provider secrets.

---

3. Input security and prompt injection

3.1 Server-side injection screening

• POST /api/v1/security/prompt-injection on user-controlled text before provider call
• Full assembled prompt scanned (history + RAG + system variables)
• unsafe blocked with request_id logged

Pass criteria: Direct injection fixture blocked E2E. Prompt injection checklist.

3.2 RAG and document boundaries

• Retrieved chunks screened at retrieve time (indirect injection)
• Upload ingest scanned before embedding (document injection)
• Vector DB tenant isolation enforced (vector DB security)

Pass criteria: Poisoned test chunk never reaches model.

3.3 No keyword-only protection

• Layered detection — not blocklists alone (keyword limitations)

Pass criteria: Architecture review sign-off.

---

4. PII, secrets, and data protection

4.1 Input privacy filter

• POST /api/v1/security/pii-secrets on assembled prompt
• Secrets (unsafe) block provider call — no redact-and-forward
• PII policy: block or redact: true per legal review

Pass criteria: Synthetic sk-test_ key blocked. PII leakage checklist.

4.2 Output and logging

• Model completions scanned before user delivery
• Production logs avoid raw message bodies at INFO level
• Analytics pipelines scrub PII
• RAG indexing scans chunks pre-embed

Pass criteria: Log sampling shows metadata-only records.

4.3 Data minimization

• Only necessary context sent to model providers
• Retention policies for chat, embeddings, and fine-tuning data
• Customer opt-out / deletion paths documented

Pass criteria: DPA commitments mapped to technical controls.

---

5. Output safety and rendering

5.1 Output moderation

• POST /api/v1/security/output-safety on every user-visible completion
• Streaming buffers server-side; final message screened
• unsafe replaced with static fallback — no echo

Pass criteria: XSS fixture blocked. Output safety checklist.

5.2 Safe rendering (web)

• Plain text escape or sanitized Markdown (raw HTML disabled)
• Content-Security-Policy on chat routes
• Stored messages treated as untrusted on read (improper output handling)

Pass criteria: Manual browser test — script does not execute.

5.3 Secondary channels

• Email, PDF, webhook, Slack outputs screened same as UI
• Background workers moderate before publish

Pass criteria: Export paths included if applicable.

---

6. Guardrail architecture and policy

6.1 Middleware placement

• Input guards before provider; output guards before client; agent guards before tools
• Server-side only — no browser API keys (middleware guide)

Pass criteria: Diagram matches production code paths.

6.2 Orchestration

• Separate endpoints or Unified Guard documented
• decision priority: block > review > allow
• Per-check findings persisted for audit

Pass criteria: Unified Guard test returns expected usage_units per check count.

6.3 Allow / review / block policy

• Written mapping from verdict + findings[].category to actions
• Policy in version control
• Review queues do not expose flagged text to end users

Pass criteria: Allow, review, block model documented per surface.

6.4 Fail behavior

• Fail-open vs fail-closed defined per check and surface
• Outage tested in staging (fail-open vs fail-closed)

Pass criteria: Simulated 503 executes documented policy.

---

7. AI agents and tools (if applicable)

7.1 Tool inventory and allowlists

• Complete tool registry with read/write/destructive classification
• Production allowlist — default deny (allowlist vs blocklist)

Pass criteria: CI fails on undeclared tools. Agent security checklist.

7.2 Agent action guard

• POST /api/v1/security/agent-action or Unified Guard before execution
• Destructive proposals blocked; ambiguous writes reviewed
• Human-in-the-loop for high-impact actions (HITL)

Pass criteria: Synthetic destructive action blocked E2E.

7.3 External communication and browsing

• Email/HTTP tools scan outbound bodies for secrets
• Web fetch SSRF protections (secure browsing)
• MCP integrations secured if used (MCP security)

Pass criteria: Private IP fetch blocked in test.

---

8. Rate limits, abuse, and cost controls

8.1 Application rate limits

• Per-user, per-IP, and per-tenant limits on LLM endpoints
• Agent loop max iterations and tool-call budgets
• Token or cost caps with graceful messages

Pass criteria: Burst test does not starve other tenants.

8.2 Guardrail API limits

• IdenticAPI rate limits understood; retry with backoff
• usage_units monitored per feature for cost attribution

Pass criteria: Dashboard shows usage_units by route.

8.3 Abuse response

• Session ban / throttle on repeated unsafe verdicts
• Support path for false-positive reports with request_id

Pass criteria: Abuse playbook documented.

---

9. Logging, monitoring, and privacy

9.1 Correlation

• Guard request_id stored with application trace ID
• Model provider request ID linked where available
• Support runbook for tracing user reports

Pass criteria: Drill locates verdict from request_id in under five minutes.

9.2 Metrics

• Block rate, review rate, guard error rate charted
• Per-category finding heatmap
• Alerts on error rate and block-rate anomalies

Pass criteria: On-call dashboard exists. Log AI security events.

9.3 Privacy-safe logging

• Log verdict, risk, categories — not full prompts in production
• Review queue access restricted and audited
• Retention aligned with privacy policy

Pass criteria: Security review of log pipeline completed.

---

10. Testing and quality assurance

10.1 Labeled corpora

• Benign, adversarial, and edge fixtures in version control
• Per-category precision/recall tracked (measure quality)

Pass criteria: No fabricated benchmark claims — real measurements recorded.

10.2 CI regression

• Critical security fixtures in CI (test suite)
• Schema tests on request_id, verdict, findings, risk, usage_units

Pass criteria: Security test job required for merge.

10.3 Red team / periodic review

• Annual or release-major red team scope for LLM features
• Post-incident fixtures added to corpus

Pass criteria: Last red team date documented or scheduled.

---

11. Incident response

11.1 Runbooks

• Secret exposed via chat — rotate, scope logs, notify
• Successful injection or data leak — contain, forensics, customer comms
• XSS or unsafe output report — patch render path, purge stored content if needed
• Guardrail vendor outage — execute fail policy

Pass criteria: Runbooks linked in on-call wiki.

11.2 Kill switches

• Feature flag disables LLM path without full app outage
• Tested in staging

Pass criteria: Kill switch drill completed.

11.3 Post-incident learning

• Root cause includes guardrail gap analysis
• Checklist item updated or risk explicitly accepted

Pass criteria: Post-mortem template includes AI security section.

---

12. Vendor and compliance

12.1 Model provider agreements

• DPAs and data processing terms reviewed for prompt retention/training
• Zero-retention or enterprise options enabled where required

Pass criteria: Legal sign-off on provider list.

12.2 Guardrail API (if used)

• IdenticAPI DPA and region reviewed
• Keys rotated on schedule; least-privilege dashboard access

Pass criteria: Subprocessor list updated for customers.

12.3 Customer commitments

• Security whitepaper / SOC answers align with implemented controls
• AI-specific questions in security questionnaires answered accurately

Pass criteria: No marketing claim exceeds technical reality.

---

13. Sign-off

• Engineering lead confirms pass criteria or documented exceptions
• Security stakeholder reviews deferred items
• Product acknowledges block/review UX impact
• Post-launch monitoring owners assigned (first 72 hours elevated alerting)

Pass criteria: Signed record attached to release.

---

Quick reference: IdenticAPI endpoints

Always parse request_id, verdict (or decision), findings, risk, and usage_units. See API response design.

SaaS AI security is continuous — this checklist is the operational backbone for shipping features without treating safety as a launch-day afterthought.