Worker
Model distillation
A distilled model is a small model trained to imitate a much larger one. The large model is the teacher; the small model is the student, taught to reproduce the teacher's answers. The result is a model with far fewer parameters that keeps a surprising amount of the bigger model's quality — especially its reasoning. In Pendra's catalogue, the DeepSeek-R1-Distill family is the clearest example: 1.5B–70B students distilled from the full 671B DeepSeek-R1.
Pendra flags these models with a Distilled badge in the console — on the Models grid and on each model's page — so you can spot them at a glance. It's a fact about where the model came from, not a capability like chat or vision.
Why it matters when you pick a model
A distilled 8B does not behave like an 8B trained from scratch. Because it learned from a much stronger teacher, it tends to punch above its size on the kinds of tasks the teacher was good at — for reasoning models like DeepSeek-R1-Distill, that means step-by-step problem solving. You get a lot of that reasoning quality at a fraction of the memory, cost, and latency of running the full model.
The trade-off
| Distilled model | Full-size model |
|---|---|
| Much smaller — runs on modest GPUs and CPUs | Needs far more memory and disk |
| Faster, cheaper, more concurrent requests | Slower, more expensive to serve |
| Keeps much of the teacher's strengths | The full range of the original model |
| Narrower — strongest on what it was distilled for | More general across tasks |
When to pick a distilled model
- You want strong reasoning on modest hardware. A distilled reasoning model can give you chain-of-thought quality that would otherwise need a much larger model and a much bigger GPU.
- Cost and speed matter. Fewer parameters means faster responses and more requests in parallel for the same hardware.
- Your workload matches what it was distilled for. Distilled models are at their best on the teacher's strengths; for very different tasks, a general model of similar size may be the safer pick.
Distillation is one of several levers on model choice — see also choosing a model size, model architecture, and choosing a quantization. The catalogue lists distilled and full-size models side by side; pick the one that fits your task and your worker's memory.