Mixture of Experts, Explained Without the Maths

Model announcements and Hugging Face model cards in 2026 come with two parameter counts. Kimi K3 at 2.8 trillion. Hunyuan 3.0 at 295 billion total with 21 billion active. Inkling at 975 billion total with 41 billion active. A Qwen release described as 30B-A3B.

That second number is the mixture-of-experts architecture, and understanding it explains both why models got so large so quickly and why the largest ones still will not run on your laptop.

The Idea

A traditional dense model runs every input through every parameter. Ask it to rewrite an email and the entire network computes, including whatever parts encode knowledge of protein folding and Javascript.

A mixture-of-experts model splits much of the network into specialized subnetworks, the experts, plus a router that decides which few to use for each token. Instead of engaging all 295 billion parameters, the router might select experts totaling 21 billion, and only those compute.

The result is a model with the capacity of something very large and the computational cost of something much smaller. The router learns which experts handle which kinds of content during training; nobody assigns them topics by hand, and the specializations that emerge are usually not human-interpretable.

The Two Numbers Mean Different Things

This is the whole practical point, and it is where nearly everyone goes wrong.

Active parameters determine speed and compute cost. A 295B model with 21B active generates at roughly the speed of a 21B dense model. This is why MoE models feel fast for their size and why they are cheaper to serve.

Total parameters determine memory. All 295 billion must be resident, because the router might select any expert for the next token. You cannot load only the experts you need, because you do not know which those are until the router decides.

So a mixture-of-experts model behaves like a small model on latency and like a large model on memory requirements. It is the best of both worlds for a datacenter operator with abundant memory and a compute bill to reduce. It does nothing for someone constrained by memory, which is everyone running models locally.

The Mistake This Causes

The failure pattern is consistent enough to predict.

Someone sees a model described as 30B-A3B, reads the 3B active count, concludes it will run comfortably on their laptop, downloads eighteen gigabytes, and gets an out-of-memory error.

Scaled up, this is why frontier open weights are not runnable at home. A 2.8 trillion parameter model with 40 billion active still needs memory for 2.8 trillion parameters. At four-bit quantization that is hundreds of gigabytes, and no amount of sparsity changes it. We ran the full arithmetic in can you run a trillion-parameter model locally.

The rule to remember: when planning what you can run, use the total parameter count. Use the active count only to predict how fast it will be once it fits.

What This Means For Local Writing Tools

Two practical consequences.

MoE is attractive at the 20B to 30B tier if you have a GPU. A 26B model with 4B active, quantized to four bits, fits in 24GB of VRAM and generates at a speed its total size would not suggest. Gemma 4’s MoE variant is a good example. That is a genuinely good deal if you have the card.

MoE is irrelevant below that. Small dense models in the 1B to 4B range are what run on a laptop with no discrete GPU, and they are dense because at that size splitting into experts buys little. Wrivio ships dense Qwen3 models for exactly this reason: on a CPU-only machine, a 1.7B or 4B dense model is the configuration that works.

Does It Change Output Quality?

Mostly no, and the differences that exist are subtle.

MoE models are generally competitive with dense models of similar total size and better than dense models of similar active size. Routing occasionally produces inconsistency, where similar inputs take different expert paths and yield noticeably different output, though modern training has reduced this.

For rewriting, none of this is the deciding factor. What matters is instruction adherence and restraint: does the model change register while leaving every fact, name, and figure exactly as written. Architecture does not predict that. Testing on your own writing does. See how to benchmark a local model on your own writing.

Why Every Large Model Is MoE Now

Worth understanding the economics, because it explains the release pattern.

Training and serving costs scale with active parameters, not total. MoE lets a lab increase model capacity, which improves capability, without proportionally increasing the compute bill for every request. Given that inference happens billions of times and training happens once, optimizing inference compute is obviously correct.

The knock-on effect for users is that “parameter count” stopped being a comparable number across models. A 300B dense model and a 300B MoE model with 20B active are very different objects with different costs, different speeds, and different hardware requirements. Any comparison that quotes one number is incomplete, which is worth remembering when reading a leaderboard.

Reading A Model Card Correctly

When you see a specification, translate it:

30B-A3B, Q4_K_M means: roughly 18GB of memory required, generates at about the speed of a 3B model, needs a 24GB GPU to be comfortable.

295B-A21B means: datacenter only, will feel like a 21B model to whoever can afford to run it.

4B, dense, Q4_K_M means: roughly 2.4GB of memory, runs on a laptop CPU, no GPU required.

That third line is the one that describes a tool you can actually put behind a hotkey. There is more on reading specifications in how to read an AI model card.

Common Questions

Can I load only some experts to save memory?

Not practically. The router may select any expert for the next token, so they all need to be available. Some research explores expert offloading, and the latency cost is severe.

Are MoE models worse at following instructions?

Not inherently. Instruction-following depends on training rather than architecture, and the strongest instruction-followers in 2026 include both dense and MoE models.

Why do some models list only one number?

Because they are dense: every parameter computes on every token. Most models below roughly 10B are dense.

Does MoE affect quantization?

Not fundamentally, though quantizing many small experts behaves slightly differently from quantizing a few large layers. Q4_K_M remains the sensible default either way.

Download Wrivio for Windows to run a dense small model sized for the machine you already have.