Content filtering

Content filtering checks what is sent with an API key, and what comes back, using a moderation model running on your own workers. For each category you choose what happens: nothing, mark it (Flag), or refuse it (Block). Blocked requests come back in each API's own error and stop shapes, so existing code keeps working. To hide a blocked streamed answer, or retry when no moderation model is available, handle the cases in What your app receives.

Content filtering is off until you turn it on, for every key, new and old.

Before you start

• Install a moderation model on at least one worker. In the console, open Models and filter by Moderation. Installing it on two workers keeps filtering running if one goes offline.
• If you send end-to-end encrypted requests (the Playground does by default), keep your workers up to date. Checks run fastest when the moderation model is installed on the workers that serve your chat models. See End-to-end encrypted requests.

Turn it on

There are two places to set it:

• For the whole organisation: Settings → Content filtering. Every key set to Use organisation setting follows it, and so does the Playground. Owners and operators can change it; everyone else sees it read-only. In the Playground, a blocked answer is replaced by a note naming the categories, and a flagged answer is marked as flagged.
• For one key: on the API Keys page, click the key's Filtering status. Choose Use organisation setting, Off for this key (for example an evaluation key that must see everything) or Custom for this key. The person who created the key or an organisation owner can change it; for anyone else the status is read-only.

When you turn it on, Pendra picks the best moderation model you have installed and the recommended settings below. Most apps can save straight away.

Choose what to check

Moderation models

Pick one or two installed moderation models. Each one adds its own time to every request, so two is the most a key can use. A second model is useful to cover a category the first one misses, or to check images when the first only reads text. The console marks a category the chosen models can't detect, or only see as part of a broader label.

Some models have no separate category for sexual content involving minors and report it as sexual content (singguard and qwen3guard, for example). With those, keep Sexual content on Block, or add a second model that reports minors on its own.

qwen3guard models can rate content borderline: it could be harmful depending on context. A borderline result is flagged at most, never blocked, even in a category set to Block. A category set to Off stays off.

When an owner or operator saves, the chosen models are kept loaded on their workers so checks start straight away. They use GPU memory there.

Requests and responses

Requests are checked before your model runs, so a blocked request uses no generation time. Responses are checked before they're returned, including any reasoning the model shows. A streamed response is checked when the stream ends, so its text has already reached your app; keep Requests on too.

A request checks the latest user message and its images. A message that holds only files, documents or tool results has nothing the check can read, so it goes ahead marked unchecked.

Categories

Self-harm is Flag by default so your app can answer with support information rather than a refusal. Every category can be set to Off.

If no moderation model can run

If any chosen model can't run (no worker with it is online, it can't read the input, or the check fails or takes too long), Pendra either refuses the request (the default) or answers it and marks it unchecked. Refusing means nothing goes through unchecked; letting it through keeps your app working. With two models, a flag from the one that did run is still reported.

End-to-end encrypted requests

Content filtering also checks end-to-end encrypted requests, and Pendra still can't read them. The worker that answers an encrypted request checks it and tells Pendra only the outcome and the categories. The same parts are checked, and a block or flag comes back in the same shapes as below.

Where the moderation model runs:

• On the worker that answers, when it has the moderation model installed and it fits on the GPU next to the chat model. This is the fastest, and Pendra prefers such a worker.
• On another of your workers, when the answering worker doesn't have the moderation model, or it doesn't fit next to the chat model. The answering worker encrypts just the text to be checked to your organisation's key, so only your own workers can read it, and Pendra passes it on without being able to. Workers of other organisations are never used. The check takes a little longer this way.

So you can keep moderation models on a separate machine. Every worker involved needs to be up to date. If none of your online workers can run the check, the request is refused with 409 moderation_unavailable, or, if you chose to let requests through, answered and marked unchecked.

The checks run on your GPUs alongside your requests, so when a worker is busy answering another request a check can wait for it. Saving a content filtering setting keeps the moderation models loaded, so they are ready when a request arrives. To keep a chat model from being swapped out to make room for a moderation model, keep the chat model loaded too: a moderation model that doesn't fit next to it then runs on another worker.

Every moderation model in the catalogue can check encrypted requests on an up-to-date worker, including qwen3guard models, whose borderline verdicts are flagged at most, as for unencrypted requests. Image generation requests are not checked yet when encrypted, so they are refused or answered unchecked following the same choice.

What your app receives

Every flagged, blocked or unchecked response carries a pendra.moderation object saying which stage was checked (request or response), the outcome and the categories. OpenAI and Anthropic SDKs ignore it, so existing code keeps working.

An encrypted reply that was checked always carries it, inside the encrypted reply, with action: pass when nothing was flagged. So an encrypted reply with no pendra.moderation inside it wasn't checked. The Playground shows this as Not checked.

With the Pendra Python SDK, read it as response.pendra.moderation on any reply (chat replies also have the response.moderation shortcut), or as stream.moderation once you've read a stream to the end. Both are None when the reply carries no verdict.

A blocked request is never a 5xx, so SDKs don't retry it.

HTTP/1.1 400 Bad Request

{
  "error": {
    "message": "Request blocked by content policy.",
    "type": "invalid_request_error",
    "param": null,
    "code": "content_policy_violation",
    "innererror": {
      "code": "ResponsibleAIPolicyViolation",
      "content_filter_result": { "hate": { "filtered": true, "severity": "high" } }
    }
  },
  "pendra": {
    "moderation": {
      "stage": "request",
      "action": "blocked",
      "categories": ["hate"],
      "model": "singguard:4b"
    }
  }
}

HTTP/1.1 200 OK

{
  "id": "chatcmpl-91c2",
  "object": "chat.completion",
  "choices": [
    { "index": 0, "message": { "role": "assistant", "content": "..." }, "finish_reason": "stop" }
  ],
  "pendra": {
    "moderation": {
      "stage": "request",
      "action": "flagged",
      "categories": ["self_harm"],
      "model": "singguard:4b"
    }
  }
}

Handle it in your app

from openai import OpenAI, BadRequestError

client = OpenAI(base_url="https://api.pendra.ai/v1", api_key="pdr_sk_...")

try:
    response = client.chat.completions.create(
        model="qwen3.6:27b",
        messages=[{"role": "user", "content": user_message}],
    )
except BadRequestError as e:
    if e.code == "content_policy_violation":
        show("Sorry, I can't help with that.")
    else:
        raise
else:
    choice = response.choices[0]
    if choice.finish_reason == "content_filter":
        show("Sorry, I can't help with that.")
    else:
        moderation = (response.model_extra or {}).get("pendra", {}).get("moderation")
        if moderation and moderation["action"] == "flagged":
            log_for_review(moderation["categories"])
        show(choice.message.content)

import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'https://api.pendra.ai/v1', apiKey: 'pdr_sk_...' });

try {
  const response = await client.chat.completions.create({
    model: 'qwen3.6:27b',
    messages: [{ role: 'user', content: userMessage }],
  });
  const choice = response.choices[0];
  if (choice.finish_reason === 'content_filter') {
    show("Sorry, I can't help with that.");
  } else {
    const moderation = response.pendra?.moderation;
    if (moderation?.action === 'flagged') logForReview(moderation.categories);
    show(choice.message.content);
  }
} catch (e) {
  if (e instanceof OpenAI.BadRequestError && e.code === 'content_policy_violation') {
    show("Sorry, I can't help with that.");
  } else {
    throw e;
  }
}

What Pendra records

For each check, Pendra records the outcome (passed, flagged or blocked) and the categories, with the usual usage details: the moderation model, token counts, time taken and which worker ran it. The content of requests and responses is never stored. If you need a record of what was blocked, log it in your own app.

Limits

A moderation model can miss harmful content or flag harmless content. Treat content filtering as one layer of your own safety process, not a guarantee.

To check content yourself, outside a request, call a moderation model directly with Moderations.