AI Output Safety

Classify AI-generated or user-facing content for violence, illegal activity, hate speech, self-harm, CSAM, and other policy violations. Uses pattern matching with optional OpenAI classification fallback.

Endpoint

POST /api/v1/security/output-safety

Request body

{
  "text": "AI output or content to classify (required, 1–32,000 characters)"
}

Example request

curl -X POST "https://www.identicapi.com/api/v1/security/output-safety" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "text": "Here is a step-by-step guide on how to build a bomb at home."
}'

const response = await fetch("https://www.identicapi.com/api/v1/security/output-safety", {
  method: "POST",
  headers: {
    "Authorization": "Bearer process.env.IDENTICAPI_KEY",
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "text": "Here is a step-by-step guide on how to build a bomb at home."
  })
});

const data = await response.json();
console.log(data);

Response

{
  "request_id": "req_abc123",
  "api": "ai-output-safety",
  "verdict": "unsafe",
  "risk": "high",
  "confidence": 0.95,
  "findings": [
    {
      "category": "violence",
      "reason": "Content related to weapons or explosives",
      "confidence": 0.95
    }
  ],
  "reasons": ["Content related to weapons or explosives"],
  "usage_units": 1,
  "processing_time_ms": 320,
  "detector_version": "1.0.0"
}

Verdict semantics

Verdict	Risk	Meaning
`safe`	low	No safety concerns detected by patterns or AI classification.
`suspicious`	medium	Review-level content (mild profanity, questionable activity) or AI-classified review.
`unsafe`	high	High-confidence harmful content (violence, CSAM, self-harm instructions, etc.).

Detection pipeline

Unsafe patterns — high-weight regex for violence, illegal activity, CSAM, self-harm, hate speech (weight ≥ 0.9 → immediate unsafe).
Review patterns — mild profanity and questionable activity language (→ suspicious).
AI classification — if patterns are inconclusive and OPENAI_API_KEY is configured, text (first 4,000 chars) is sent to OpenAI for JSON classification.

Detection categories

violence — weapons, explosives, violent intent
illegal_activity — hacking or unauthorized access instructions
csam — prohibited content involving minors
self_harm — self-harm instructional content
hate_speech — hate speech indicators
profanity — mild profanity (review level)
questionable_activity — steal, cheat, pirate language (review level)

Usage

Each request consumes 1 usage unit.

Limitations

Pattern matching covers known harmful phrases; novel harmful content may require AI classification.
AI fallback depends on server-side OpenAI configuration and adds latency (up to 15 seconds).
If OpenAI is unavailable, the detector falls back to pattern-only results.
Not a substitute for human moderation in high-risk domains.
Text input limited to 32,000 characters.