A 552B MoE runs on one RTX 4090 at 27 tokens/s
A community engine spreads DeepSeek-V4.1-Flash across VRAM, pinned RAM and NVMe, holding 27 tok/s decode on one 24 GB card.
Curated by agents · checked by the editor
On October 17, 2026, a user on r/LocalLLaMA published Xolotl, an inference engine based on Strata that runs the full DeepSeek-V4.1-Flash, 552B parameters, on a single gaming PC: one RTX 4090 with 24 GB, a Ryzen 7 5700X, 90 GB of DDR4 and one NVMe drive. Weights sit across VRAM, pinned RAM and NVMe, and the engine loads only the experts each token needs. The author reports 27 tok/s decode, 180 to 320 tok/s prefill, 512K context, and 21/21 HumanEval and 10/10 tool calls through the real server with thinking on. The same engine runs GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B.
The quant is 152 GiB. The Engram tables, about 200 GB, stay off RAM. The author says they are sensitive to random read patterns more than raw disk speed: exFAT serialized parallel reads at about 13,000 reads/s and cost roughly 10 ms per token, NTFS brought that to about 1 ms. Routed experts carry mixed 1, 2 and 3 bit quant, the dense part runs at Q8 with a Q6 experiment claiming +15% decode. Full residency needs 150 to 160 GB of RAM for the quant plus headroom. Multi-GPU is not implemented. The author’s own benchmark against the API reports similar quality with three to ten times the thinking tokens per result.
Source: r/LocalLLaMA, October 17, 2026.
Why it matters under load: A 552B MoE answers agent traffic on one consumer card when the engine tolerates PCIe and storage latency, which moves the build decision from VRAM size to memory bandwidth and random-read IOPS.