News · 2026-07-23

AMD and Cerebras split AI inference across two different chips

AMD and Cerebras announced on July 23 that they will run AI inference across two fundamentally different chip architectures at once: AMD's Helios rack-scale systems will handle prompt and large-context processing, while Cerebras's Wafer-Scale Engine handles token generation. The companies claim up to five times higher tokens per second per watt than a Cerebras-only configuration, and say the combined offering will reach customers first through Cerebras Cloud in the second half of 2026.

Key facts

• The claim: up to 5x higher tokens per second per watt versus a Cerebras-wafer-scale-only setup, from company modelling.
• Who and when: AMD and Cerebras, announced July 23, 2026.
• The split: AMD Helios does prompt and large-context processing; Cerebras Wafer-Scale Engine does low-latency decode and token generation.
• Primary source: the joint press release.

The engineering logic here is better than the press release makes it sound, and it rests on an asymmetry that anyone who has profiled an inference server knows well. Generating a response has two phases with opposite characteristics.

The first phase, prefill, reads the entire prompt. Every token can be processed in parallel, so the work is one enormous matrix multiplication and the limit is raw compute. Feed it more arithmetic units and it goes faster.

The second phase, decode, produces the answer one token at a time. Each token depends on the one before it, so there is nothing to parallelize within a request. The chip spends most of its time hauling model weights and the KV cache out of memory to compute a single token, then doing it again. The limit is memory bandwidth, and adding arithmetic units barely helps.

Running both on the same chip means that chip is over-provisioned for one phase and starved for the other. It is like using the same vehicle for a cross-country freight haul and for darting through city traffic -- one job wants capacity, the other wants responsiveness, and a compromise is worse at both.

Cerebras's wafer-scale engine is an unusually good fit for the second job, because it keeps model weights in enormous on-chip memory rather than fetching them from external memory across a bus. That is exactly the bottleneck decode hits. AMD's Helios, meanwhile, is a dense rack of Instinct GPUs, EPYC CPUs and Pensando networking built for throughput (technical overview), which is exactly what prefill wants. Splitting the phases across the two is the hardware expression of an optimization that inference engines have been doing in software for a couple of years.

The performance number needs careful handling, and the footnote does the honest work the headline does not. The 5x figure comes from July 2026 modelling by AMD Performance Labs and Cerebras, using Kimi 2.6 at a comparable interactivity point, comparing Helios plus wafer-scale against a Cerebras-wafer-scale-only configuration. It is not a comparison against Nvidia. It is not a measurement. And the baseline is one of the two partners' own products, which is the most favorable framing available. The companies also note that system configurations may vary.

AMD CEO Lisa Su said the combination extends AMD into latency-sensitive applications and real-time agentic AI, and Cerebras CEO Andrew Feldman said the partnership can bring Cerebras performance to more customers. Both are interested-party statements from a joint announcement, and neither constitutes independent validation.

The same day, AMD separately announced that Anthropic plans up to two gigawatts of AMD capacity. The Cerebras relationship, meanwhile, is not new. Cerebras's February financing release shows AMD participated in its $1 billion Series H, which closed February 3. The amount and any ownership stake were not disclosed, and nothing in the record supports an acquisition reading.

Why this matters beyond two vendors: agentic workloads are decode-heavy in a way that chatbot workloads were not. An agent that plans, calls a tool, reads the result, and plans again spends its life in short bursts of generation where per-token latency is the user-visible cost. If that becomes the dominant workload, the economics of inference stop being about peak throughput and start being about how cheaply you can produce the next token -- which is precisely the bet this architecture makes. It is a hardware answer to a distinction our lesson on training versus inference draws in software.

The honest caveats stack up. Availability is a second-half-of-2026 expectation, not a shipping product. The performance figure is modelled rather than measured, against a self-selected baseline. Cerebras merely plans to deploy Helios in its data centers. And disaggregation adds a network hop between the two phases, which means the KV cache produced during prefill has to move to wherever decode happens -- a real cost the announcement does not quantify. Whether the split wins depends heavily on how much context is being processed and how long the generated answer is, and neither company has published that curve.

Primary source, verified: read the paper →

Ground Truth. (2026, July 23). AMD and Cerebras split AI inference across two different chips. Ground Truth. https://groundtruth.day/news/amd-cerebras-split-inference-across-two-chips.html

@misc{groundtruth:amd-cerebras-split-inference-across-two-chips,
  title  = {AMD and Cerebras split AI inference across two different chips},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/amd-cerebras-split-inference-across-two-chips.html}
}

Topics: hardware · inference · efficiency · amd · datacenter