Chat completions

Other optional fields

• top_p, top_k, min_p, max_tokens, stop — standard OpenAI sampling controls.
• frequency_penalty, presence_penalty, repeat_penalty, logit_bias — repetition and token-bias controls. repeat_penalty is a classic repetition penalty (1.0 = off; e.g. 1.1–1.3 discourages repeating recent tokens), and repeat_last_n sets how many recent tokens it looks back over (default 64).
• logprobs, top_logprobs — request token log-probabilities (model permitting).

Any other standard OpenAI Chat Completions field is forwarded to the serving worker as-is. Whether a given field takes effect depends on the model serving the request.

Response

Non-streaming responses come back as a single OpenAI-shaped chat.completion object (see example). usage is always populated; finish_reason is "stop" on a natural finish or "length" when capped by max_tokens.

For a reasoning model, the chain-of-thought comes back on message.reasoning_content (mirrored on message.reasoning), separate from the answer in message.content — streamed as delta.reasoning_content. See Thinking.

Streaming

With stream: true, Pendra returns Server-Sent Events matching OpenAI's format: each event is a data: { ... } line containing a delta chunk, terminated by data: [DONE]. Pendra flushes each chunk immediately, so tokens arrive as they're generated.

curl https://api.pendra.ai/api/v1/chat/completions \
  -H "Authorization: Bearer pdr_sk_..." \
  -N \
  -d '{
    "model": "qwen3.6:27b",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello"}]
  }'

stream = client.chat.completions.create(
    model="qwen3.6:27b",
    stream=True,
    messages=[{"role": "user", "content": "Hello"}],
)
for event in stream:
    if not event.choices:  # usage / timing chunk — no text
        continue
    print(event.choices[0].delta.content or "", end="")

const stream = await client.chat.completions.create({
  model: 'qwen3.6:27b',
  stream: true,
  messages: [{ role: 'user', content: 'Hello' }],
});
for await (const event of stream) {
  process.stdout.write(event.choices[0]?.delta?.content ?? '');
}

Each chunk is an OpenAI-shaped chat.completion.chunk. The last chunks of a stream report on the request rather than continuing the answer: one carries usage (Pendra always sets stream_options.include_usage = true server-side) and one carries pendra.timing. Both have an empty choices array — so read the text with something that tolerates that, e.g. chunk.choices[0]?.delta?.content in JavaScript or a if not chunk.choices: continue in Python, rather than indexing choices[0] unconditionally.

data: {"id":"chatcmpl-9f2b","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-9f2b","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f2b","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":8,"completion_tokens":2,"total_tokens":10}}

data: {"id":"chatcmpl-9f2b","object":"chat.completion.chunk","choices":[],"pendra":{"timing":{"ttft_ms":115,"tokens_per_second":58.34,"queue_wait_ms":16600,"worker_queue_wait_ms":14100}}}

data: [DONE]

Errors on a stream

A streaming request that fails before it can answer sends the error as a normal data: frame carrying an error object (with a message, a type, and the HTTP code), followed by data: [DONE]. Because the stream has already started with a 200, the error can't use response headers — so when every worker for your model is busy (a 429), the honest back-off rides in the frame itself: retry_after (seconds to wait, sized to how backed up the model is) and queue_depth (how many requests were already waiting). Back off for retry_after with a little jitter, then retry.

data: {"error":{"message":"All workers serving 'qwen3.5:2b' are at capacity. Retry shortly, or add additional workers to handle more concurrent requests.","type":"rate_limit_exceeded","code":429,"retry_after":7,"queue_depth":9}}

data: [DONE]

Web tool steps

When the worker serving your request has Web tools enabled, the model can fetch pages and run web searches while it answers. Each of those steps is reported back on the response under a Pendra-specific pendra field, so you can show what the model looked at. It sits alongside choices and is safe to ignore — the OpenAI SDKs skip unknown fields, so your existing code is unaffected.

Non-streaming responses carry the full list as pendra.web_tool_steps; streaming responses emit one pendra.web_tool_step event per step (a "call" when the model asks, then a "result" once it runs) on a chunk whose choices delta is empty. Each step has: name (web_fetch or web_search), arguments, ok, a truncated result preview, result_chars (the full length), and error when a step failed.

{
  "choices": [{ "index": 0, "message": { "role": "assistant", "content": "…" }, "finish_reason": "stop" }],
  "usage": { "prompt_tokens": 812, "completion_tokens": 43, "total_tokens": 855 },
  "pendra": {
    "web_tool_steps": [
      { "index": 0, "name": "web_fetch", "arguments": "{\"url\":\"https://example.com\"}",
        "ok": true, "result": "Example Domain…", "result_chars": 129 }
    ]
  }
}

Notices

The same pendra field also carries the occasional advisory about a response, under pendra.notice. Today there's one: when a reasoning model spends its whole max_tokens budget thinking and returns no answer (content empty, finish_reason: "length"), the response includes pendra.notice with code: "truncated_during_reasoning" and a human-readable message. Streaming responses emit it on a chunk whose choices delta is empty, just before the final chunk. Detect it to retry with a larger budget or with thinking off (see Thinking). Like everything under pendra, it's safe to ignore — OpenAI SDKs skip unknown fields.

Timing (streaming)

A streaming response can't carry the Server-Timing header below — headers are sent before the first token, when there is nothing to measure yet. So streaming requests get the same numbers on the last chunk before [DONE], under pendra.timing. That chunk has an empty choices array and no text; it's there so you can log latency and throughput per request without a second call.

data: {"id":"chatcmpl-9f2b","object":"chat.completion.chunk","choices":[],
       "pendra":{"timing":{"ttft_ms":115,"tokens_per_second":58.34,
                           "queue_wait_ms":16600,"worker_queue_wait_ms":14100}}}

Every field is present only when it was measured, and the whole chunk is omitted when nothing was — an older worker reports no timings, and the stream then ends at the usage chunk. Like everything under pendra, it's safe to ignore.

Response headers

Non-streaming chat responses carry these headers (streaming responses don't):

A non-streaming response includes a Server-Timing header so you can read the per-request latency and throughput without any extra call — handy when benchmarking or sizing concurrency:

Server-Timing: ttft;dur=115, tps;dur=0;desc="58.3", queue;dur=16600, worker_queue;dur=14100

ttft measures prompt processing only. If your request had to wait its turn, that wait is reported separately in queue rather than inflating ttft — so a slow first token and a busy queue stay distinguishable.

One exception: when the model uses web tools, it works in several steps and you see nothing until the first of them produces something. ttft then measures that whole wait — the model deciding to search, the search or page fetch itself, and re-reading the results — so it matches what you actually waited for rather than the first step in isolation.

The two queue figures tell you what to do about it. worker_queue is time spent waiting behind other requests on the worker that served you; the remainder (queue minus worker_queue) is time spent waiting for any worker with a free slot at all. In the example above, 14.1s was spent queued on the worker and 2.5s waiting for one to become available — the first points at that worker's concurrency settings, the second at needing more workers.

A streaming response can't carry this header — headers are sent before the first token — so streaming requests get the same figures on a pendra.timing chunk instead. Either way they're also recorded against each request in the console's Usage view.

Timeouts

A single non-streaming chat request can run up to ~30 minutes. While a slow or large model works, Pendra automatically keeps the connection alive, so you no longer need to switch to streaming just to avoid a timeout on a long generation. stream: true is still the best choice for interactive UIs — it shows partial tokens as they're generated rather than waiting for the whole reply.

OpenAI SDK compatibility

Point the OpenAI SDK at Pendra by setting OPENAI_BASE_URL=https://api.pendra.ai/api/v1 and OPENAI_API_KEY=pdr_sk_…. No other code changes needed.

The OpenAI convention https://api.pendra.ai/v1 works too, so if you already have a base URL ending in /v1 you can leave it as-is — chat/completions, embeddings, and models all resolve at both /api/v1 and /v1.