Turn a model off on a worker

Sometimes you want a worker to stop answering requests for one particular model — it misbehaves on that machine's GPU, you're reserving the box for something else, or you're about to take it down for maintenance. Uninstalling the model would work, but it deletes the weights, so putting it back means downloading tens of gigabytes again.

Serving is the switch for exactly this. Turn it off and that worker stops receiving requests for that model. The model stays installed, so turning it back on takes effect immediately.

What happens when it's off

Requests carry on as normal — they're simply sent to your other workers that have the model. If no other worker has it, the request fails the same way it would if the model weren't installed anywhere, and the model stops appearing in GET /v1/models until you turn it back on somewhere.

Requests already in flight when you turn it off are allowed to finish. Only new ones are routed elsewhere.

If you keep several quantizations of one model, Serving switches off only the one you chose, and requests that name it are never handed to a different quant. For example, with nemotron-3.5-lightning:30b switched off and nemotron-3.5-lightning:30b-q8_0 still serving, a request for nemotron-3.5-lightning:30b goes to another worker that serves that exact variant, or fails as described below. A request that names only the model family, such as nemotron-3.5-lightning, can still be answered by any quant that's serving.

Turning serving off also frees the model's memory: it's unloaded from the GPU, and if you had it set to Always-on, it's switched back to Auto too. Turning serving back on does not restore Always-on — set it again if you want it.

If the worker is in the middle of a long request when you switch serving off, that request isn't interrupted. Serving goes off straight away, and the model keeps its Loaded badge until its memory is released, shortly after that request finishes.

How to turn it off

Open the worker's page in the console and find the model under Models & context. Expand the model's row with the chevron on the right, then switch Serving off — it's the first setting in the panel. Every model type — chat, image, embedding, transcription and speech — uses the same panel.

A model that isn't serving is labelled Not serving so you can see the state at a glance. Owners and operators can change it, and the setting sticks across worker restarts.

Turning it off everywhere

If you switch a model off on every worker that has it, it disappears from your available models entirely and requests for it fail with a 404 that says so:

Model 'gemma4:e2b' is installed on a connected worker, but serving is
switched off for it there, so requests aren't routed to it. Turn Serving
back on for it on /workers in the dashboard — re-installing the model
won't change that.

That's a reasonable way to retire a model without deleting it. It behaves exactly as if it were uninstalled until you turn it back on — the one difference being that the error tells you the switch is what's stopping it, so nobody wastes time reinstalling a model that's already there.

When to uninstall instead

Turning serving off keeps the weights on disk, so it doesn't free any disk space. If you're done with a model for good, or you need the disk back, uninstall it. Use Serving for anything temporary, and uninstall for anything permanent.

An uninstall unloads the model first. If the worker is in the middle of a long request, the uninstall can fail with a model busy message rather than interrupt that request. Nothing has been removed at that point: uninstall again once the request has finished.