Worker
Choosing a model size
A model's size is its parameter count — the
4B, 9B, or 70B figure on each
catalogue model (B = billions of parameters). It's the single biggest
lever on how capable a model is: bigger models reason better and follow
instructions more reliably, but they need more memory and run slower.
When you install a catalogue model, picking the size is the first
choice — the quantization just
fine-tunes the size you land on.
The trade-off
| Bigger model | Smaller model |
|---|---|
| Smarter — better reasoning, coding, instruction-following | Lighter — runs on modest GPUs and CPUs |
| More memory (VRAM / RAM) and disk | Far less memory and disk |
| Slower responses, fewer parallel requests | Faster, more concurrent requests |
How to pick
- Pick the largest size that fits your worker comfortably. Capability scales with size, so the biggest model your hardware runs well is usually the best answer. Leave headroom — the weights aren't the only thing using memory; the context window needs some too.
- Match the size to the job. Small models (1–4B) are great for classification, extraction, autocomplete, and high-volume simple tasks. Mid-size (7–14B) handle general chat and most assistant work. Large (30B+) are worth it for hard reasoning, coding, and agents.
- Size beats precision. If you're memory-constrained, a larger model at a smaller quantization almost always beats a smaller model at a higher one. Choose the size first, then trim with the quant.
- Check the hardware. See system requirements for what each GPU tier can serve. The console shows each variant's download size before you install, which is a good proxy for the memory it needs.
- Let the console recommend one. On a model's page, pick one of your workers and Pendra marks the size and quantization it recommends for that worker's memory. You can still install any option you like; it's a guide, not a gate.
Not every model ships every size — the catalogue offers the sizes a model was actually released in. You can install several sizes on the same worker and switch between them per request. Some small models are also distilled from a much larger one, which lets them punch above their size — worth knowing when you weigh a size against its quality.