Turn a model off on a worker

Sometimes you want a worker to stop answering requests for one particular model — it misbehaves on that machine's GPU, you're reserving the box for something else, or you're about to take it down for maintenance. Uninstalling the model would work, but it deletes the weights, so putting it back means downloading tens of gigabytes again.

Serving is the switch for exactly this. Turn it off and that worker stops receiving requests for that model. The model stays installed, so turning it back on takes effect immediately.

What happens when it's off

Requests carry on as normal — they're simply sent to your other workers that have the model. If no other worker has it, the request fails the same way it would if the model weren't installed anywhere, and the model stops appearing in GET /v1/models until you turn it back on somewhere.

Requests already in flight when you turn it off are allowed to finish. Only new ones are routed elsewhere.

Turning serving off also frees the model's memory: it's unloaded from the GPU, and if you had it set to Always-on, it's switched back to Auto too. Turning serving back on does not restore Always-on — set it again if you want it.

How to turn it off

Open the worker's page in the console and find the model under Models & context. Expand the model's row with the chevron on the right, then switch Serving off — it's the first setting in the panel. Every model type — chat, image, embedding, transcription and speech — uses the same panel.

A model that isn't serving is labelled Not serving so you can see the state at a glance. Owners and operators can change it, and the setting sticks across worker restarts.

Turning it off everywhere

If you switch a model off on every worker that has it, it disappears from your available models entirely and requests for it will fail. That's a reasonable way to retire a model without deleting it — just be aware it behaves exactly as if it were uninstalled until you turn it back on.

When to uninstall instead

Turning serving off keeps the weights on disk, so it doesn't free any disk space. If you're done with a model for good, or you need the disk back, uninstall it. Use Serving for anything temporary, and uninstall for anything permanent.