Keep a model loaded

The first request to a model that isn't already in memory is slow: the worker has to load the model's weights into the GPU before it can answer, which can take anywhere from a second to half a minute depending on the model's size. This is a cold start. Each model on a worker has a Residency setting that decides how it trades memory against that cold-start cost.

Auto, Custom, or Always-on

Each model's Residency is one of three settings, from the one that frees the most memory to the one that frees the least:

• Auto (the default) — the model loads on demand when a request needs it, and unloads again automatically once it's been idle for the worker's own idle window. The next request after that pays one cold start. The control tells you the worker's window, so you always know what Auto means on that machine.
• Custom — the same as Auto, but with your own idle window for this one model, anywhere from 10 minutes to 24 hours. Use it to hold one model longer than the rest, or let a rarely-used one go sooner.
• Always-on — the worker loads the model into memory right away, without waiting for a request, and keeps it there. It's never unloaded to make room for another model, and it's reloaded automatically if the worker restarts. Requests to an Always-on model never pay the cold-start cost.

Set a model to Always-on for the ones you call most, and leave the rest on Auto. You can keep as many models Always-on as fit in the worker's memory at once — there's no one-per-worker limit. It works for every kind of model you can run on a worker — chat, embeddings, image generation, transcription, and speech.

Idle auto-unload is on by default

A model left on Auto doesn't hold its memory forever. Each worker has a default idle window — 15 minutes unless you change it — and any model that isn't Always-on is unloaded once it hasn't received a request for that long, so an occasional model doesn't tie up the GPU between bursts of use. The next request loads it again (paying one cold start at that point).

You can change the worker-wide window, or turn it off entirely, under Workers → your worker → Settings → Auto-unload idle models — anything from 10 minutes to 24 hours. Turn it off if you'd rather every model stay resident once it has loaded.

To give one model a different window, set its residency to Custom and type the minutes — that model then uses your window instead of the worker's, and every other model carries on following the worker default.

Unload a model right now

When a model is loaded but not Always-on (its row shows a Loaded badge), an Unload now button frees its GPU memory immediately, without waiting for the idle window. The model stays installed on disk and simply cold-starts on its next request. It's handy for reclaiming memory on the spot — for example to make room to load a larger model. To free an Always-on model's memory, switch it to Auto first.

Unloading is safe to do while the worker is busy: any requests already running on that model are allowed to finish first, and only then is the memory released. Requests that arrive during those few seconds get a retryable error rather than a partial answer, so a client that retries (the official OpenAI and Anthropic SDKs do so automatically) won't notice.

The trade-off

An Always-on model holds its GPU memory the whole time, even while it's idle. That's the deal: no cold starts in exchange for memory that stays reserved. On a worker with room to spare it's a clear win for a model you use regularly.

Because an Always-on model never gives up its memory, it leaves less room for other models on the same worker. If a request asks for a different model that won't fit in the GPU memory left around the Always-on ones, the worker turns that request down right away with a clear "doesn't fit" error (HTTP 507) instead of trying and failing — the Always-on models keep serving uninterrupted. If you need several large models always available and they can't all fit together, give them their own workers.

Spotting a cold start

On the Usage page, any request that had to cold-load its model is flagged with a small warning marker, and hovering its duration shows a Model load time alongside the other timings — the part of the response time that went into loading the model rather than generating the answer. If you see cold starts on a model you call often, that's the signal to set it Always-on.

How to change a model's residency

Open the worker's page in the console and find the model under Models & context. Expand the model's row with the chevron on the right, and you'll find the Residency control — Auto, Custom, or Always-on — alongside the model's other settings, with a line underneath telling you exactly when that model will unload. Every model type uses the same panel. It's per model, and owners and operators can change it. Switch a model back to Auto at any time to let it unload again — it stays installed on disk and will simply cold-start on its next request.

If a model can't finish loading when you switch it to Always-on — for example it doesn't fit in the memory left on the worker — the change won't stick, and the model's row shows the reason right there instead of quietly turning itself back off. Address what it reports (free up memory, or set a smaller model Always-on) and try again.

A model that isn't currently being served on the worker can't be set Always-on — there'd be nothing to answer. If the Always-on option is unavailable, check whether serving is turned off for it.