Private inference

Private inference end-to-end encrypts every request between your client and your worker. Your prompts, completions, uploaded audio and images, and tool arguments are sealed on your machine and only decrypted inside your worker. Pendra relays the traffic but never holds the key to read it.

What Pendra can and can't see

With private inference on, Pendra cannot see:

• Your prompts and the model's completions
• Uploaded audio and images
• Tool arguments

Pendra can still see the operational metadata it needs to route and bill the request: the model id, request and response sizes and timing, token counts, the length and file format (WAV, FLAC or MP3) of transcribed audio, and an opaque routing token. If you turn on content filtering for an API key, it also sees the outcome of each check (passed, flagged or blocked) and the categories. It never sees the content, unless you turn on web tools for the worker (see Web tools and private requests).

Pendra can't forge a reply either. Your client only accepts a reply that one of your workers produced for that exact request, so a reply that was altered, reordered, or substituted in transit is rejected with an error instead of reaching your code. Pendra relays the bytes, so it could still drop a reply, but it can't change what a reply says. Once your SDK and workers are up to date, it can't cut a streamed reply short without your client noticing either (see Response below).

Nothing from a private request is left on the worker's disk either. Workers save conversations to disk by default so repeat prompts reply faster (the KV cache), and private requests are excluded from that: their conversation is held only in the worker's GPU and system memory and never written to the drive. They still get the in-memory speed-up, and you don't have to change any worker setting to get this.

Web tools and private requests

If you turn on web tools for a worker, private requests to it can use them too. One of the two tools works outside end-to-end encryption:

• Web searches are sent via Pendra. The search query still travels over HTTPS, but it isn't sealed end to end like the rest of the request, so Pendra, and the search provider that runs the search, can see it. The model writes the query from your prompt, so it can contain words from the prompt.
• Page fetches go straight from your worker to the site, without passing through Pendra.

Web tools are off by default. If a worker handles private requests whose content Pendra must never be able to see, leave web tools off on that worker. The console shows the same reminder next to the toggle.

There is nothing to set up

You don't create, install, or copy an encryption key. The first worker you start creates one for your organisation automatically, and every worker you add after that is given the same key by the workers you already have — the new worker announces an address that only it can open, an existing worker seals your key to that address, and Pendra passes along a sealed package it cannot read.

That applies to GPUs you rent from Pendra too. A rented worker is keyed as it boots, so it can serve encrypted traffic from its first request.

Approving a new worker

Your key spreads through the fleet on its own, which raises a fair question: what stops it reaching a machine you didn't expect? Turn on Worker approval under Settings → Encryption and nothing new gets your key, or any traffic, until you say so. A newly connected worker shows as Awaiting approval on the Workers page; approve it there and it starts serving and is given the fleet key in the same step. Until then it is connected and visible but idle — deliberately, so you can look at a machine before deciding about it.

It is a separate switch from enforcement, and off by default: the two answer different questions, so you can review new machines without requiring encrypted requests, or the other way round. See approving new workers for the full flow.

Two things make it unobtrusive in practice. Workers already connected when you turn it on are approved automatically, so switching it on never interrupts what you are already running. And GPUs you rent from Pendra are approved automatically too — ordering one is the approval.

Your key is never written to disk

Every worker holds the encryption key in memory only. Nothing writes it to a file — not on your own machines, and not on rented hardware, where it means nothing is left behind when the rental ends. There is no key file to protect, back up, or lose.

The trade-off is that your key lives exactly as long as your fleet keeps at least one worker running. Restart workers freely, roll upgrades through them, replace them one at a time — the key survives, because a worker that is still up passes it on. If your fleet ever drops to zero connected workers, the key goes with it and the next worker to start creates a new one.

Enable it in your client

Turn on private inference when you construct the client. The SDK fetches your organisation's public key, seals each request to it, and decrypts the sealed reply — all transparently. You can also enable it without a code change by setting PENDRA_PRIVATE_INFERENCE=1 in the environment.

Use Python SDK 0.14.0 or Node SDK 0.14.0 or later: these are the versions that check every reply came from your own worker. Encrypted audio transcription and image generation need 0.15.0 or later of either SDK. They need up-to-date workers, so if a request fails with private_inference: unsupported envelope version, the worker named in the error is older than your SDK — update it and retry.

To encrypt audio transcription and image generation from Python, use Python SDK 0.15.0 or later.

from pendra import Pendra

# Turn on private inference for the whole client.
client = Pendra(private_inference=True)  # reads PENDRA_API_KEY

response = client.chat.completions.create(
    model="qwen3.6:27b",
    messages=[{"role": "user", "content": "Summarise this contract."}],
)
print(response.choices[0].message.content)

import Pendra from 'pendra';

// Turn on private inference for the whole client.
const client = new Pendra({ privateInference: true }); // reads PENDRA_API_KEY

const response = await client.chat.completions.create({
  model: 'qwen3.6:27b',
  messages: [{ role: 'user', content: 'Summarise this contract.' }],
});
console.log(response.choices[0].message.content);

Verify the key is yours

The one thing worth checking is that the key your client seals to is really the one on your workers. Compare the fingerprint your client reports against the one shown on the Encryption page, and against pendra status on a worker itself. Do this out of band — over a channel separate from the one you're securing — so a compromised path can't feed you a fake fingerprint.

On macOS, pendra is installed alongside the menu-bar app. If your terminal says command not found, open the Pendra menu-bar icon and choose Install command-line tool, then start a new terminal.

To make the check automatic, pin the fingerprint you verified. If the key ever changes, the request fails instead of quietly sending your data somewhere new. If you registered several distinct keys (up to 3 per organisation), pin all of them at once as a comma-separated allowlist — your client then seals only to those fingerprints and rejects anything else:

# Pin the fingerprint you verified out of band. If the key the platform
# routes to ever differs, the request fails instead of sending your data.
# Registered several keys? Pin them all: "a7f3c0d2…e9c21,b8e4d1f3…0a2c5".
client = Pendra(private_inference={"pinned_fingerprint": "a7f3c0d2…e9c21"})

// Pin the fingerprint you verified out of band. If the key the platform
// routes to ever differs, the request fails instead of sending your data.
// Registered several keys? Pin them all: 'a7f3c0d2…e9c21,b8e4d1f3…0a2c5'.
const client = new Pendra({
  privateInference: { pinnedFingerprint: 'a7f3c0d2…e9c21' },
});

Copy each fingerprint in full: it is 64 characters long. The client checks your pin when you create it. A pin that is empty, truncated or not a valid fingerprint raises an error straight away instead of leaving you unpinned, and so does a misspelled option name (for example the JavaScript pinnedFingerprint spelling passed to the Python client).

Pinning and fleet continuity

Pinning is worth it when your fleet always has a worker running, because then your fingerprint never changes. If your fleet regularly drops to zero workers — you rent GPUs for a few hours and release them, or you run a single machine that reboots — a new key is created each time it comes back, and a pinned client will refuse to seal until you update the pin.

For those fleets, leave pinning off and let the client trust the key the server reports. Your applications then follow a re-key on their own. The rule of thumb is the fleet, not the setting: if you can promise at least one worker is always connected, pin; if you can't, don't.

With automatic trust, the client keeps a record of the fingerprints it has trusted in ~/.pendra/known_org_keys.json. On a read-only filesystem, such as a locked-down container or a serverless runtime, point PENDRA_TOFU_PATH at a writable file, for example under /tmp. The record never decides which keys are accepted, so the Python client still encrypts and sends your requests if it can't write the file and logs a warning instead.

Rotating your key

Rotate key on the Encryption page gives your fleet a second key alongside the one it already has, and hands it to every worker. Nothing stops working at that point and nothing needs redeploying: your clients seal each request to every key your organisation holds, so a request succeeds whichever of the two the worker serving it has — including requests already in flight.

When you're ready, use Retire beside the old fingerprint on the Encryption page. That is the step that actually revokes it: retiring pushes a drop to every worker still holding the key, so it leaves their memory rather than merely going unadvertised. Pinned clients need the new fingerprint before you retire the old one.

You can check that the retirement reached every machine. Each worker's Activity tab records the moment it received your key, and the moment it confirmed dropping a retired one — an "Encryption key dropped" entry appears only once the worker itself reports the key gone, not when the instruction was sent. A worker that was offline when you retired the key, and comes back still holding it, is told to drop it as soon as it reconnects, and its entry appears then. A worker that restarted in the meantime has already lost the key, since keys live only in memory.

Rotation needs a free key slot, so an organisation already at its limit has to retire something first.

If a key is lost or exposed

Because Pendra stores none of your prompt or completion content, there is nothing to recover if a key is lost — and nothing of yours for us to leak if one is exposed.

If a key is exposed, rotate to a new one and then Retire the exposed fingerprint — the retire is what removes it from your workers' memory, and until then a worker holding it can still open anything sealed to it. Applications using automatic trust pick the new fingerprint up on their own; pinned ones need it before you retire the old key.

Retiring also frees the slot the key occupied against your organisation's key limit. The one case it refuses is retiring the only key a connected worker holds, which would leave it unable to serve encrypted requests at all — give the fleet another key first.

In the dashboard

The Encryption page is your organisation's hub for private inference. It shows:

• Your registered workload key fingerprints, to verify out of band.
• Each worker's status — Encrypted (has a workload key) or not — with a per-worker key fingerprint on its detail page.
• A per-request Key column on the Usage page, showing which key sealed each request (or — for a plaintext request).

Settings → Encryption has the Require E2E toggle, which rejects any plaintext request across your whole organisation, and Worker approval. Require E2E can only be switched on while at least one connected, approved worker holds a key (a worker still awaiting approval doesn't count, because it serves no traffic), but wait until every worker you rely on has one — a worker without a key receives no traffic while enforcement is on. Live microphone transcription can't be encrypted, so it is unavailable while enforcement is on; transcribe an audio file instead. The moderation endpoint doesn't accept encrypted requests yet, so it is unavailable while enforcement is on too.

The Playground encrypts automatically

The Playground seals requests in your browser the same way the SDKs do. Whenever one of your workers holding a workload key serves the selected model, your prompt is encrypted before it leaves the page and only that worker can read it — encrypted requests are routed exclusively to keyed workers. The Playground checks replies the same way the SDKs do: a reply your worker didn't write is shown as an error, never as the model's answer. Each reply is marked Encrypted or Not encrypted, so every exchange records which way it actually went, and the worker selector shows a padlock whenever the next request will be encrypted — next to a pinned worker that holds a key, or next to Auto when a keyed worker will serve the request.

Content filtering on encrypted requests

Content filtering on an API key works with private inference, including in organisations that require it. The worker that answers an encrypted request checks it and tells Pendra only the outcome and the categories, so Pendra still can't read the request or the answer. If that worker doesn't have a moderation model, or it doesn't fit next to the chat model, the check runs on another of your workers: the text to check is encrypted to your organisation's key on the way, so only your own workers can read it. How each moderation model is asked is built into the worker, not chosen by Pendra, so that outcome is all Pendra can learn.

How is this end-to-end encrypted if my client has no key?

A fair question: your client never stores a private key, and it even fetches what it needs to encrypt from Pendra — so how can Pendra be unable to read the traffic? The answer is that "the key" is really three different keys, and public-key cryptography keeps them apart:

• Your encryption private key — created by your first worker, it lives only in your workers' memory and is never sent to Pendra in a form Pendra can read. It is the only key that can decrypt a request.
• Your workload public key — derived from the private one and safe to share; this is what your client fetches from Pendra. Public-key encryption lets anyone seal a message using only the recipient's public key, so your client needs no stored secret to send an encrypted request.
• A throwaway reply key — your client works it out while sealing each request, and the worker works out the same key when it opens that request, which takes your workload private key. Your client keeps it in memory just long enough to decrypt the reply, then discards it. It is never stored or sent anywhere, and because only your worker can derive it, a reply that decrypts is a reply your worker wrote.

So "my client holds no key" and "my client encrypts and decrypts" aren't in tension: sending needs only your public key, and receiving uses a key that only your client and your worker can work out. Pendra can't read anything because the one key that opens a request — your workload private key — exists only on your own worker. It's a locked postbox: Pendra gives your client the address of your postbox and carries the sealed letter, but only your worker holds the key to open it.

Technical specification

For security review, here is exactly what happens on the wire, including the one place where the design asks you to trust us. Pendra relays ciphertext it cannot read, and never sees your prompts, completions, uploaded audio or images, or tool arguments.

Your key is a different matter, and the honest version is worth stating plainly. When a new worker joins, an existing worker seals the key to the newcomer's public key and Pendra relays that sealed blob. Pendra cannot read it. But Pendra is also the channel through which the sealing worker learns which public key to seal to, so a Pendra that chose to misbehave could present a key of its own and receive a copy. Pinning does not detect this — the fleet's key and fingerprint are unchanged.

What this buys you is a fleet that keys itself, including GPUs you rent for an afternoon, with no key material for you to move between machines. What it costs is that key delivery is trust-on-first-use rather than cryptographically verified. If that trade is wrong for your threat model, run a fixed fleet: keep at least one worker up so the key is created once and never re-delivered, and pin the fingerprint.

Cryptographic suite

• HPKE (Hybrid Public Key Encryption, RFC 9180) in base mode — suite DHKEM(X25519, HKDF-SHA256) + HKDF-SHA256 + ChaCha20-Poly1305.
• Workload keys are 32-byte X25519 keys, held in worker memory only and never written to disk. A key's fingerprint — the id you verify out of band — is the lowercase-hex SHA-256 of the raw 32-byte public key.

Request

Your SDK generates a fresh random 32-byte data key per request and encrypts the entire request body — messages, sampler parameters, tools, and, for audio, the raw file bytes — with ChaCha20-Poly1305. That data key is sealed with HPKE (info = "pendra-pi-v1") to every workload public key your organisation has registered (up to 3), so a worker holding any one of the matching private keys can unwrap it — only a worker of yours can. The envelope's routing metadata (model, stream flag, and request type) is bound into the AEAD's additional authenticated data, so Pendra can't silently re-route or re-classify a request — the worker detects the mismatch and rejects it.

Response

The worker encrypts the reply with ChaCha20-Poly1305 under a 32-byte response key exported from the request's own HPKE context (Export("pendra-pi-v1:rdek", 32)). Your client exports the same key from its side of that context when it seals the request, so only your client can decrypt the reply, and only a worker holding your workload private key can produce one it will accept. Pendra carries every byte and still can't forge or substitute a reply. The first frame of a reply names the fingerprint of the key the worker opened, so a client that sealed to several keys knows which response key to use. A streamed reply encrypts each chunk under a monotonic 12-byte counter nonce, so chunks can't be reordered or replayed. The only parts of an encrypted stream that aren't end-to-end encrypted are ones Pendra adds after the worker has finished: an error report, or a closing timing summary with no model output in it. Your client refuses any other chunk that isn't end-to-end encrypted.

A streamed reply also carries an encrypted end-of-stream marker, so a stream can't be cut short in transit without your client noticing. From Python and Node SDK 0.15.1 (and in the console Playground), the client asks the worker for this marker inside the encrypted request. A worker that supports it confirms this in the reply's first encrypted frame and ends the reply with the encrypted marker. If that stream then stops without the marker, the SDK raises an error instead of returning a partial reply as if it were complete: in Python a ValueError, in Node an Error, both starting private_inference: the stream ended before the worker's sealed end-of-stream marker. Stopping a stream yourself (breaking out of the loop, or aborting it) never raises this. In the Playground, a reply cut short this way keeps whatever did arrive, including the model's reasoning if it was cut while still thinking, and is marked as possibly incomplete. Older workers don't send the marker, and the SDK then accepts their streams as before (though a stream with no encrypted content in it at all is always refused), so keep your workers up to date to get this protection.

This is envelope version 2. Python and Node SDKs before 0.14.0 used version 1, which sealed the reply to a fresh key the client sent alongside the request. That kept the reply confidential, but because the key travelled in the clear, Pendra could have sealed a reply of its own to it. Clients on 0.14.0 or later send version 2 only and refuse any version-1 reply. Workers accept both, so older SDKs keep working until you upgrade.

What Pendra can and cannot see

Never: prompt or completion content, uploaded audio or images, or tool arguments — ciphertext end to end. (The exception is a worker with web tools turned on: its web-search queries are sent via Pendra outside end-to-end encryption — see Web tools and private requests.) Still visible (worker-reported metadata, needed for routing and billing): the model id, request/response sizes and timing, token counts, the length and file format of transcribed audio (never the file name), an opaque routing token, and, on a key with content filtering on, each check's outcome and categories. Your key travels only between your own workers, sealed so that Pendra cannot read it in transit — subject to the delivery caveat above.

Go further

• Chat — the core request shape, unchanged with encryption on.
• Python SDK — full client reference.
• Node SDK — full client reference.