How to Detect Suspicious URLs Programmatically

Security and agent teams often need programmatic URL analysis before allowing fetches, publishing links, or executing HTTP tools. "Suspicious" is not a single boolean — it is a set of signals: malformed syntax, disallowed schemes, redirect chains that land on unexpected hosts, HTTP errors, and metadata mismatches with claimed destinations.

This guide shows how to build a honest detection pipeline using real URL Inspector API capabilities — parsing, normalization, redirect analysis, and metadata inspection — without claiming malware or phishing verdicts the API does not provide.

What "suspicious" means in code

Define suspicion as policy-relevant anomalies, not omniscient threat detection:

Combine signals with weighted policy — none alone proves malice.

URL Inspector API overview

IdenticAPI exposes URL inspection via RapidAPI:

• Base host: url-inspector.p.rapidapi.com
• Endpoint: GET /inspect?url={url_encoded}
• Authentication: X-RapidAPI-Key and X-RapidAPI-Host headers

Documented capabilities include HTTP status, redirect chains, final resolved URL, and page metadata (such as HTML title). The API performs HTTP-level inspection — it does not execute JavaScript in a headless browser, scan for malware, or return phishing scores.

Product page: URL Inspector API.

Basic inspection request

curl -G "https://url-inspector.p.rapidapi.com/inspect" \
  --data-urlencode "url=https://example.com" \
  -H "X-RapidAPI-Key: YOUR_RAPIDAPI_KEY" \
  -H "X-RapidAPI-Host: url-inspector.p.rapidapi.com"

Example response:

{
  "url": "https://example.com",
  "status": 200,
  "final_url": "https://example.com/",
  "redirects": [],
  "title": "Example Domain"
}

Use status, redirects, final_url, and title as structured inputs to your policy engine.

Node.js integration example

type InspectResult = {
  url: string;
  status: number;
  final_url: string;
  redirects: Array<{ url: string; status: number }>;
  title?: string;
};

async function inspectUrl(target: string): Promise<InspectResult> {
  const endpoint = new URL("https://url-inspector.p.rapidapi.com/inspect");
  endpoint.searchParams.set("url", target);

  const res = await fetch(endpoint, {
    headers: {
      "X-RapidAPI-Key": process.env.RAPIDAPI_KEY!,
      "X-RapidAPI-Host": "url-inspector.p.rapidapi.com"
    }
  });

  if (!res.ok) throw new Error(`Inspector HTTP ${res.status}`);
  return res.json() as Promise<InspectResult>;
}

Call from server-side agent orchestrators — not from browsers exposing keys.

Pre-inspection: parse and normalize

Run local checks before RapidAPI calls to save quota and block SSRF early:

function preflightUrl(raw: string): URL {
  const url = new URL(raw.trim());
  if (!["http:", "https:"].includes(url.protocol)) {
    throw new Error("disallowed_scheme");
  }
  const host = url.hostname.toLowerCase();
  const blocked = ["localhost", "127.0.0.1", "169.254.169.254"];
  if (blocked.includes(host) || host.endsWith(".local")) {
    throw new Error("blocked_host");
  }
  return url;
}

Apply tenant allowlists before inspection when URLs are already known-good.

Redirect analysis patterns

After inspection, evaluate redirect chain:

function analyzeRedirects(
  proposed: string,
  result: InspectResult,
  opts: { maxHops: number; allowedFinalHosts: Set<string> }
): string[] {
  const reasons: string[] = [];

  if (result.redirects.length > opts.maxHops) {
    reasons.push("redirect_limit_exceeded");
  }

  const finalHost = new URL(result.final_url).hostname.toLowerCase();
  const proposedHost = new URL(proposed).hostname.toLowerCase();

  if (finalHost !== proposedHost) {
    reasons.push("final_host_differs");
  }

  if (!opts.allowedFinalHosts.has(finalHost)) {
    reasons.push("final_host_not_allowlisted");
  }

  for (const hop of result.redirects) {
    const hopUrl = new URL(hop.url);
    if (hopUrl.protocol === "http:" && proposed.startsWith("https:")) {
      reasons.push("scheme_downgrade");
    }
  }

  return reasons;
}

Return reasons[] to logging and Agent Action Guard context — do not collapse to suspicious: true without detail.

Metadata as weak signals

title and meta fields support analyst UX:

• Display "Destination page title: …" before agent fetch
• Flag review when title contains "login", "password", "wallet" for non-auth workflows

Titles are attacker-controlled. Never auto-allow based on title alone.

Layering with agent policy

Inspection informs; policy enforces:

inspectUrl(url) → analyzeRedirects() → score reasons
    ↓
Agent Action Guard(tool, action, args, context)
    ↓
allow | review | block (runtime enforced)

Example guard call after inspection:

curl -X POST https://www.identicapi.com/api/v1/security/agent-action \
  -H "Authorization: Bearer idapi_test_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "tool_name": "http_client",
    "action": "get",
    "arguments": { "url": "https://short.example/abc" },
    "context": "inspect: final_host=unknown.example; redirects=2; reasons=final_host_differs"
  }'

See Validate URLs Before AI Agents Open Them.

What to tell stakeholders

Honest capability statement:

• "We parse URLs, inspect HTTP status and redirects, and apply tenant policy."
• Not: "Our API detects all phishing links."
• Not: "Inspected URLs are malware-free."

For phishing-specific products, evaluate content and reputation signals separately — Phishing Detection API: What Developers Need.

Operational considerations

• Timeouts — set client timeouts; define fail-closed vs review on inspector outage
• Rate limits — cache inspection results by URL hash for short TTL where safe
• Public URLs only — inspector reaches public internet; combine with SSRF blocks for internal targets
• Logging — store proposed URL, final URL, redirect count, reasons — not full HTML bodies

Testing

Fixture URLs in CI (use stable test endpoints you control):

• 200 with no redirects
• 301 chain to second domain
• Excessive redirect loop (expect failure)
• 404 broken link
• Blocked internal IP (expect preflight rejection)

Assert policy outputs, not proprietary "risk scores" from undocumented heuristics.

Related reading

Detecting suspicious URLs programmatically means combining parsing, HTTP inspection via the URL Inspector API, redirect analysis, and explicit policy — returning structured reasons your agent runtime and security teams can act on, without overstating what HTTP metadata can prove.