News · 2026-08-03

The Cheap 284B Rig Is Really 768GB of Server Memory

A builder is running DeepSeek V4-Flash, a 284-billion-parameter mixture-of-experts model, at 33 tokens per second on two RTX 3090s. The detail that reframes the whole result: each 3090 holds only about 6.6GB of weights, while each serving instance occupies roughly 170GB of system memory on a four-socket Dell server carrying 768GB of DDR4. This is not a gaming rig running a frontier model. It is an enterprise memory appliance with two gaming cards attached.

Key facts

• Each serving instance uses about 170GB of system RAM; each GPU holds about 6.6GB of weights plus key-value cache.
• The machine is a Dell PowerEdge R940 with four Xeon Platinum 8268 CPUs, 96 cores total, and 24 sticks of 32GB DDR4-2933.
• Decode draws roughly 1kW, with about 435W at idle.
• Primary source: the builder's report on r/LocalLLaMA, with hardware specifications confirmed against Dell's and Intel's own documentation.

How it works

A mixture-of-experts model is sparse at execution but not at rest. DeepSeek V4-Flash activates roughly 13 billion parameters for any given token out of 284 billion total, so the arithmetic per token is modest. The other 271 billion still have to be somewhere the machine can reach in microseconds, because which experts get called depends on the token and cannot be predicted far ahead.

The usual answer is a stack of data-center GPUs holding everything in fast memory. This build takes the other road: put the expert bank in ordinary server DRAM spread across four independent memory domains, and use the GPUs only for the parts a hybrid runtime can accelerate. The community runtime doing it, LvLLM, is explicitly built for this, with CPU and GPU hybrid MoE execution, NUMA-aware scheduling, expert weight management, and quantization kernels, and it publishes a dual-3090 launch configuration for this exact model.

The analogy is a library. Renting a small reading room and having a runner fetch books from vast nearby stacks works well if the stacks are close and the runner is fast. It does not work if the books are in another building, which is what streaming experts off an SSD amounts to. See our coverage of a 26B model in 2GB by streaming off the SSD for the version where the books are further away.

Quantization is doing real work too. Per the builder, this is not a community re-quant: the official checkpoint ships routed experts in MXFP4 with FP8 linear layers, BF16 activations, and an FP8 key-value cache. Our lesson on quantization explains what those formats cost.

What it gives up

Context. The published configuration is capped at 22K tokens. DeepSeek's official card and its technical report describe a model built for million-token context, and the local build keeps about two percent of it. The key-value cache is why: it grows with context and it has to live in the memory the runtime reserved.

Prefill. The report gives decode speed and not prompt processing speed, which was the first thing commenters asked for. For agent workloads that re-read a large context every turn, prefill is often the number that determines whether the thing feels usable, and it is missing.

Concurrency evidence. The builder reports tests at four and eight simultaneous users, but the runtime's own published dual-3090 example runs with a sequence limit of two, and no full launch command or log was posted for the larger runs.

Stability. The report itself documents CUDA and Triton version mismatches, a warm-up constraint on the speculative decoding path, and an Ampere-specific code path that does not generalize to other MoE models.

Why it matters

The genuinely new fact here is architectural, and it is more interesting than the price tag. It says the binding constraint for running very large sparse models locally is memory capacity and bandwidth, not GPU compute. The builder reports that power-capping the 3090s did not change throughput and that adding GPUs would not help, which is what you would expect if the CPU memory subsystem is the bottleneck, and the runtime's design supports that reading.

That points somewhere specific for anyone shopping: four-socket previous-generation servers with a lot of DDR4 are a different value proposition than another graphics card. It also points at why the memory industry is building new tiers between fast memory and storage. Compare with Kimi K3 running in 8GB of RAM at 33 seconds per token, which is the same trade taken to its unusable extreme, and with a 284B model with a 3GB working set.

The honest caveat: this is one person's machine, self-reported, with no independent reproduction. The hardware specifications check out against Dell's and Intel's documentation, but ownership, throughput, and especially the roughly $6,000 cost claim are unverified, and no listing or receipt was linked. Treat 1kW under load and 435W at idle as the real operating profile: this is rack equipment, not a quiet desk machine.

Primary source, verified: read the paper →

Ground Truth. (2026, August 3). The Cheap 284B Rig Is Really 768GB of Server Memory. Ground Truth. https://groundtruth.day/news/the-cheap-284b-rig-is-really-768gb-of-server-ram.html

@misc{groundtruth:the-cheap-284b-rig-is-really-768gb-of-server-ram,
  title  = {The Cheap 284B Rig Is Really 768GB of Server Memory},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/news/the-cheap-284b-rig-is-really-768gb-of-server-ram.html}
}

Topics: deepseek · local-llm · mixture-of-experts · hardware · inference · quantization