For this comparison, we use Qwen3.8 27B and DeepSeek V4.1 Flash.
The Qwen model is a dense model, and the dream of self hosting. The DeepSeek model is the dream of inference providers, and a Mixture of Experts (MoE) model.
Both score comparable benchmark values (although DeepSeek is overall significantly better). Both are multimodal. Both are open weight.
But DeepSeek V4.1 Flash needs much more resources to run, is smarter, and still costs less on inference platforms. How is that possible?
The pricing paradox
On Tensorix, at fp8 precision, same provider and same precision:
| Qwen3.8 27B | DeepSeek V4.1 Flash | |
|---|---|---|
| Total params | 27B | 552B |
| Active per token | 27B | 8B prefill / 16B decode |
| Input | €0.359/M | €0.449/M |
| Output | €2.154/M | €1.346/M |
DeepSeek is much bigger, but its output tokens cost 1.6× less than Qwen. Why is that?
Dense: every parameter, every token
Every parameter participates in every token. 27B parameters means 27B multiply accumulate operations per token, every token. The compute is uniform and predictable, but it cannot be split across devices. A dense layer is one matrix multiplication.
MoE: most of the model sits idle
The feed forward layers are replaced by a set of expert networks plus a router. The router picks a small subset of experts per token. DeepSeek V4.1 Flash has 552B parameters total and activates 8B on prefill and 16B on decode. The other ~536B sit in memory and do nothing for that token.
That gives MoE two advantages. First, less compute per token: 16B active against Qwen's 27B. Second, the experts can be distributed across GPUs, so a large MoE parallelizes in a way a dense model of the same size cannot.
The catch is memory
The cost is memory, though. All 552B parameters must be in memory before the model serves anything. At fp8 that is roughly 550GB of weights. Qwen3.8 27B at fp8 is about 27GB and fits on one 24GB GPU with quantization.
Once loaded, the weights are read only and shared across requests, so the fixed cost is paid only once. What scales with traffic is the KV cache and the compute. A large MoE can spread its experts across many GPUs, so a provider with the hardware can serve many requests concurrently. A dense 27B model cannot be split the same way, because it runs on one device, and concurrency is limited by that device.
That means: if you already have the big hardware, you can serve many requests fast with the MoE model. If you have weaker hardware, you can run the overall smaller model, but you cannot parallelize as much.
So which one wins?
MoE is cheap to serve at scale and expensive to self host. Dense is the opposite.
We hope this helped you get a feeling for the differences in LLM architectures. If you want more context on the models themselves, we also wrote about AI model examples and how to self host RAG.
About Dentro
We've been helping European businesses for 3 years with anything related to AI. If you have ideas for implementing AI in your business, message us or send us an email at office@dentroai.com. We're happy to respond.