Worker

Model architecture & active parameters

Every chat model in the catalogue is built one of a few ways, and the model page shows which under Architecture & specs. The architecture changes how fast a model runs and how much memory it needs — so it's worth a quick look before you install.

Dense

A dense model uses all of its parameters for every token it generates. This is the classic design: a 9B dense model puts all 9 billion parameters to work on each word. Predictable and capable, but every parameter costs memory and time.

Mixture-of-Experts (MoE)

A Mixture-of-Experts model is split into many smaller "experts", and only a few of them run for each token. So a large model can answer at the speed of a much smaller one, while still drawing on all its knowledge.

You'll see two numbers for an MoE model — its total size and its active parameters. For example, 35B (3B active) means the model holds 35 billion parameters in total but only uses about 3 billion per token:

  • Memory follows the total size. You still need enough memory to hold all 35B of weights, so size the worker for the total figure (the console shows each variant's download size before you install).
  • Speed follows the active size. Because only ~3B parameters run per token, the model generates text far faster than a dense 35B model would — closer to a dense 3B.

In short: MoE gives you the quality of a big model with the response speed of a small one, at the cost of needing the memory for the whole thing. If a model has the memory headroom on your worker, its MoE sizes are usually an excellent deal.

Other architectures

A few models are labelled Hybrid (they mix in state-space layers, which can stretch further on long inputs) or Diffusion (they generate text in a different way than the usual left-to-right models). You install and call these exactly like any other model — the label is just there so you know what you're running.

Architecture is one of several things to weigh when you pick a variant — see also choosing a model size and choosing a quantization.