Node.js SDK

The Pendra Node SDK is a TypeScript-first, zero-dependency client for sovereign UK inference. Streaming support, dual ESM/CJS. Node.js 20+.

Installation

$ npm install pendra

Zero runtime dependencies. View on npm.

Quick start

import Pendra from 'pendra';

const client = new Pendra({
  apiKey: 'pdr_sk_...', // or set PENDRA_API_KEY env var
});

const response = await client.chat.completions.create({
  model: 'qwen3.6:27b',
  messages: [{ role: 'user', content: 'Hello!' }],
});

console.log(response.choices[0].message.content);

Streaming

Stream responses token by token using async iteration.

const stream = await client.chat.completions.create({
  model: 'qwen3.6:27b',
  messages: [{ role: 'user', content: 'Write a poem' }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}

Image generation

Generate images from a text prompt. Returns base64-encoded PNGs by default — decode with Buffer.from(b64, 'base64') and write to disk, or set response_format: 'url' when supported by the model.

import { writeFileSync } from 'node:fs';

const response = await client.images.generations.create({
  model: 'flux.1-schnell',
  prompt: 'A red London double-decker bus at sunset',
});

const b64 = response.data[0].b64_json;
if (b64) {
  writeFileSync('bus.png', Buffer.from(b64, 'base64'));
}

Image generation is non-streaming — the endpoint returns a single JSON response once the worker finishes.

Embeddings

Generate vector embeddings for retrieval, search, and RAG pipelines. OpenAI-compatible — pass a string or array of strings and get back a CreateEmbeddingResponse with one embedding per input.

const response = await client.embeddings.create({
  model: 'nomic-embed-text',
  input: ['The quick brown fox', 'jumps over the lazy dog'],
});

for (const item of response.data) {
  console.log(item.index, (item.embedding as number[]).length, 'dims');
}

console.log(response.usage.prompt_tokens);

Any embedding model in the Pendra catalogue works — nomic-embed-text, qwen3-embedding, bge-m3, embeddinggemma.

Audio transcription

Transcribe audio to text using Whisper-class models. Multipart upload — pass a Blob/File directly, or a { filename, content } object where content is a Blob, ArrayBuffer, or Uint8Array. Files capped at 25 MB.

import { readFileSync } from 'node:fs';

const audio = readFileSync('meeting.mp3');
const result = await client.audio.transcriptions.create({
  file: { filename: 'meeting.mp3', content: audio },
  model: 'whisper-large-v3-turbo',
  language: 'en',
});

console.log(result.text);

In the browser (or Bun), pass the File from an <input type="file"> straight through. Set response_format: 'srt' or 'vtt' to get subtitles back instead of JSON:

// Browser / Bun: pass a Blob or File directly
const input = document.querySelector('input[type=file]');
const file = input.files[0]; // a File (which extends Blob)

const result = await client.audio.transcriptions.create({
  file,
  model: 'whisper-large-v3-turbo',
  response_format: 'srt', // or 'vtt' for subtitles
});

console.log(result.text); // SRT string — text/srt/vtt are wrapped as { text }

Transcription is non-streaming. result.duration and result.language are populated when the model reports them; result.segments appears when response_format: 'verbose_json'.

Text to speech

Turn text into speech with a speech model. You get a WAV file back (24 kHz, 16-bit mono); leave out voice to use the model's default.

import { writeFileSync } from 'node:fs';

const speech = await client.audio.speech.create({
  model: 'your-speech-model', // an id from /models?type=speech
  input: "Good morning, and welcome to today's briefing.",
});
writeFileSync('speech.wav', speech.audio); // a Uint8Array of WAV bytes

speech.arrayBuffer() works too, so code written for the OpenAI SDK carries over. Speech isn't available with private inference yet, so the call throws instead of sending your text unencrypted when private inference is on.

Which worker served a request

Every reply names the worker that ran it on response.pendra?.worker: its id (the same one worker_id pins a request to), the name you gave it in the console, and the worker version that served it (either can be null). response.pendra?.request_id is the request's ID, worth logging alongside it and quoting to support.

const response = await client.chat.completions.create({
  model: 'qwen3.6:27b',
  messages: [{ role: 'user', content: 'Hello!' }],
});
const worker = response.pendra?.worker;
console.log(worker?.id, worker?.name, worker?.version);
console.log(response.pendra?.request_id);

A stream names the worker on its last chunk, so read stream.worker and stream.requestId once you've finished iterating. Both are null until then.

const stream = await client.chat.completions.create({
  model: 'qwen3.6:27b',
  messages: [{ role: 'user', content: 'Write a poem about London.' }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}
console.log(stream.worker?.id, stream.requestId);

Embeddings, reranking, image generation, transcription and speech results carry the same pendra field, and it works the same with private inference turned on. Needs the pendra package 0.17 or later.

List models

const models = await client.models.list();
models.forEach((m) => console.log(m.id));

Migrating from OpenAI

The Pendra SDK mirrors the OpenAI interface. Two lines to switch — your existing code just works.

// Before
import OpenAI from 'openai';
const client = new OpenAI({ apiKey: 'sk-...' });

// After
import Pendra from 'pendra';
const client = new Pendra({ apiKey: 'pdr_sk_...' });

API reference

client.chat.completions.create()

Create a chat completion. Returns ChatCompletion or Stream.

Anything else you pass is sent to the API as-is, so a Chat Completions field newer than your SDK version still reaches the model. TypeScript flags an undeclared key written straight into the call — that's what catches typos — so pass a newer field through a variable or a cast when you mean it.

messages takes a content string, or an array of { type: 'text' } / { type: 'image_url' } parts for vision models. On the way back, a reasoning model's working arrives on message.reasoning_content (mirrored on message.reasoning, and streamed as delta.reasoning_content), separate from the answer in message.content.

Replies also carry a Pendra-specific pendra field alongside choices. Read response.pendra?.notice when an answer looks wrong for no obvious reason — a truncated_during_reasoning code is the model telling you it spent the whole max_tokens budget thinking and never reached an answer, which otherwise just looks like an empty message.content:

const response = await client.chat.completions.create({
  model: 'ornith-1.5:9b',
  messages: [{ role: 'user', content: 'Which is heavier, 1kg of steel or 1kg of feathers?' }],
  max_tokens: 200,
});

if (response.pendra?.notice?.code === 'truncated_during_reasoning') {
  // Retry with a bigger budget, or with thinking off.
  console.log(response.pendra.notice.message);
  console.log(response.usage?.completion_tokens_details?.reasoning_tokens); // where it went
}

The other codes include truncated_during_structured_output, reasoning_budget_reached, truncated_at_reasoning_budget, truncated_at_default_max_tokens, structured_output_not_json and strict_schema_not_enforced — see Notices for what each one means. The same field carries pendra.web_tool_steps when the serving worker has web tools enabled.

When streaming, the last few chunks report on the request rather than continuing the answer: one carries chunk.usage, one carries chunk.pendra?.timing (ttft_ms, tokens_per_second, queue_wait_ms, worker_queue_wait_ms), and a notice arrives on its own chunk too — so check chunk.pendra on every chunk, not just the last. Those chunks have no text, which is why the streaming example reads content with chunk.choices[0]?.delta?.content rather than indexing choices[0] unconditionally.

client.images.generations.create()

Generate images from a text prompt. Returns Promise<ImageResponse>.

client.embeddings.create()

Create embeddings. Returns Promise<CreateEmbeddingResponse>.

client.models.list()

Returns an array of Model objects. Each model has id, object, created, and owned_by fields.

client.audio.transcriptions.create()

Transcribe an audio file. Returns TranscriptionResponse.

client.audio.speech.create()

Resolves to a SpeechResponse: audio (the WAV bytes), duration_ms, pendra, and arrayBuffer().

Configuration

Error handling

All exceptions extend APIError.

err.message is always a readable sentence. Some errors carry more than a sentence — when your organisation requires end-to-end encryption and a request is sent without it, for example, the error also says why in a machine-readable type. The message stays readable, and the original error is on err.detail (the whole response body is on err.body), so your code can branch on it:

try {
  await client.chat.completions.create({ model: "ornith-1.5:9b", messages });
} catch (err) {
  if (err instanceof APIStatusError) {
    console.error(err.message); // "This organisation requires end-to-end encrypted requests. …"
    const detail = err.detail as { error?: { type?: string } } | undefined;
    if (detail?.error?.type === "e2e_required") {
      client = new Pendra({ privateInference: true }); // retry encrypted
    }
  }
}

Failures during a stream

A streaming reply has to send its status code before the model has generated anything, so if something goes wrong once generation is under way — the GPU worker crashing, the request timing out, your fleet being busy — it can't come back as an HTTP error code. The SDK throws APIStreamError from the iterator instead, at the point the failure happened. Anything the model already sent you is yours to keep:

import { APIStreamError } from "pendra";

let answer = "";
try {
  const stream = await client.chat.completions.create({
    model: "ornith-1.5:9b",
    messages: [{ role: "user", content: "Write me a limerick." }],
    stream: true,
  });
  for await (const chunk of stream) {
    answer += chunk.choices[0]?.delta?.content ?? "";
  }
} catch (err) {
  if (err instanceof APIStreamError) {
    // `answer` holds the partial reply that arrived before the failure.
    console.error(`Stream failed: ${err.message}`);
    console.error(`Quote this to support: ${err.requestId}`);
  } else {
    throw err;
  }
}

err.requestId and err.workerId identify the exact inference that failed, so quote them if you report the problem. Check err.status to decide whether retrying is worth it — but check it for undefined first, because some failures (including a timeout) arrive with no status code at all. (If none of your workers are online when you start a stream, the call throws a plain APIStatusError with a 503 before any tokens arrive, not an APIStreamError.)

if (err.status === 429 || err.status === 503) {
  await retryAfterBackoff();   // fleet busy, or its worker went offline
} else if (err.status === undefined) {
  // Timed out, or a failure the server couldn't classify. Retrying once is
  // usually reasonable; read err.message to see what happened.
  console.error(err.message);
} else {
  throw err;                   // e.g. 502 — a worker failed, retrying won't help
}

Note that a stream reports being at capacity this way rather than as a RateLimitError — by the time the queue wait is over, the 200 has already been sent — so back off on the status code, not the error class.

The same applies to a non-streaming request that runs for more than about 80 seconds — a long embeddings batch, an image, a long JSON-format transcription. Pendra sends the 200 early to keep the connection open, so a failure after that point also arrives as APIStreamError, thrown by the create call itself, with the same fields. You never get an empty result in its place.

Replies that don't parse

APIResponseValidationError means the reply reached you but couldn't be read as a chat completion — usually a proxy between you and Pendra returning its own error page. It is deliberately not an APIConnectionError: retrying won't help, because the same body will fail to parse again. If you retry on connection failures, let this one through. The offending body is quoted in the message (and in full on err.body) so you can see what actually came back.

Links

• npm
• Get an API key
• API reference