Cerebras and the Latency Frontier

The historical trajectory of artificial intelligence infrastructure has been largely defined by the pursuit of raw computational throughput. In the inference era, advanced reasoning, long-context, agentic, and multimodal workloads are primarily constrained by memory bandwidth and latency during token generation. Traditional GPU-based architectures, reliant on shuttling data across physical packages to off-chip high-bandwidth memory (HBM), have collided with a physical memory wall: at low batch sizes, each new token forces large volumes of weights and KV-cache across the HBM interface, and the compute units sit idle waiting for data.

Cerebras has built a different system from mainstream GPU and TPU stacks to address this constraint. Instead of many relatively small accelerators connected by high-speed links, it uses a single wafer-scale processor as the primary compute device and separates model storage from compute: model parameters and optimiser state reside in an external memory service and are streamed layer-by-layer as needed, while activations and model weights are kept in very fast on-chip SRAM. This wafer-scale, SRAM-centric design is explicitly optimised for latency-sensitive inference, where GPUs cannot rely on batching to amortise memory traffic and per-user tokens-per-second directly track memory bandwidth.

This architecture introduces a new axis in the classic inference tradeoff between throughput and interactivity. Historically, providers have had to choose between serving many users slowly (large batch, high throughput, mediocre per-user latency) and serving a few users quickly (small batch, low throughput, good latency). Batch size has been the dominant knob in that tradeoff. Lower batch sizes mean fewer users, higher speeds, and therefore higher effective prices per token. Fast coding tiers like Codex Spark or Claude Code Fast explicitly leaned into this regime and were priced accordingly; both our own experience and broader developer discourse suggest there is real willingness to pay for materially higher speed and smoother interaction.

Cerebras is offering hardware that is architecturally optimised for simultaneously high throughput and low latency rather than relying solely on batching tricks. In the sections that follow, we explore the technical details and limitations of Cerebras's approach, the new kinds of applications and human–machine interactions it unlocks, and why, despite real economic and architectural challenges, we are fundamentally optimistic about this latency-first inference hardware.

Hardware Architecture

Reticle Limit and Wafer-Scale Integration

For over half a century, semiconductor manufacturing has been constrained by the reticle limit: the maximum area a lithography system can expose in a single shot, roughly 26 mm × 33 mm, or ~858 mm² on modern tools. A 300 mm wafer is therefore patterned as a grid of identical dies; after exposure, the wafer is diced along the scribe lines into individual chips, which are then packaged and connected into larger systems.

Cerebras takes a different approach: it does not cut the wafer. The WSE-3 is built from a standard TSMC N5 wafer with roughly 84 reticle images printed across its surface. Instead of dicing along the scribe lines, Cerebras adds bespoke metallisation steps to route over one million microscopic wires across reticle boundaries, stitching those nominal dies into a single continuous computational surface.

The Wafer-Scale Engine 3 (WSE-3) measures 46,225 mm², hosts ~4 trillion transistors, and delivers 125 petaflops of peak sparse FP16 AI compute. This is enabled by a fail-in-place architecture: Cerebras provisions more cores and interconnect than it needs, maps defects after manufacturing, and routes around faulty regions, guaranteeing roughly 900,000 working cores even on wafers with dozens of defects.

Cores, Mesh, and SRAM

Unlike GPUs, which rely on shared cache hierarchies backed by external HBM, the WSE-3 uses a distributed SRAM-centric design:

• ~900,000 AI-optimised cores, each with local SRAM and a fine-grained instruction set tuned for sparse tensor operations.
• 44 GB of on-chip SRAM spread across the wafer, far more on-die memory than conventional GPUs or even Groq's LPUs.
• 2D mesh interconnect linking the cores, with aggregate internal bandwidth of roughly 21 PB/s, far above the 3.35–8 TB/s HBM bandwidth of high-end GPUs.

For workloads whose active working set fits inside the 44 GB SRAM budget, the WSE can keep data close to compute and avoid much of the HBM latency and bandwidth penalty that dominates GPU inference.

Cerebras reports 125 PFLOPS of sparse FP16 compute for WSE-3, but this assumes 1:8 structured sparsity, meaning only 12.5% of weights are non-zero. Nvidia typically quotes sparse throughput at 2:4 sparsity, or 50% non-zero weights. On a dense FP16 basis, WSE-3 is closer to 15–16 PFLOPS. These reported sparsity configurations are not used in production and somewhat useless: for both Nvidia and Cerebras, these are theoretical.

Weight Streaming and Disaggregated Parameter Storage

The WSE's 44 GB of SRAM is large by on-chip standards, but small relative to frontier model weights. In rough terms, it can hold:

• ~22B parameters in FP16
• ~44B parameters in INT8
• ~88B parameters in INT4 / FP4

For models that do not fit entirely in on-chip SRAM, Cerebras uses weight streaming for training and some large-model execution modes: model parameters are stored in MemoryX, its external model-memory system, and streamed into the Wafer-Scale Engine layer by layer. The WSE computes one layer, then advances to the next. This allows Cerebras to train and run models significantly larger than the WSE's local memory without keeping the full parameter set resident on each wafer, but today its commercial focus has shifted heavily toward inference, and Nvidia GPU clusters remain the default choice for frontier-scale training at most labs.

Given that GPUs still set the baseline for training price-performance, and a Cerebras deployment wraps supporting components such as MemoryX, SwarmX, external DRAM, CPU infrastructure, and additional networking around the wafer, it is difficult to justify that bill of materials for general-purpose training unless Cerebras delivers a clear performance or cost advantage on particular workloads.

In latency-sensitive inference, repeatedly streaming large weights across the off-wafer link would be too expensive. The more natural strategy is to keep weights resident across one or more wafers and use pipeline parallelism: each WSE holds a slice of the model, and activations move between stages. Activations are far smaller than weights, so this fits the WSE's external bandwidth constraints better than sharding each layer across devices.

Weight streaming helps Cerebras scale model size, but fast inference depends on keeping the hot path as local as possible: weights resident, activations moving between stages, and minimal traffic across the system boundary.

KV-Cache, KVSS, and Off-Wafer Bandwidth Limits

KV-cache is harder than weights because it grows with the workload. Weights are fixed for a given model. KV-cache grows with context length, concurrency, layer count, and hidden dimension. For coding assistants and interactive agents with long contexts, it can become the dominant memory consumer.

Once KV-cache no longer fits within the WSE's SRAM budget, it has to live outside the wafer. In Cerebras's production inference architecture, this can mean offloading KV-cache to a separate dual-socket AMD CPU chassis, the KVSS or KV-cache Server System, with up to 6 TB of DDR5 DRAM. KV-cache reads and writes then move through the same off-wafer path used by other system traffic.

In short- and medium-context workloads, Cerebras can benefit from the wafer's local SRAM and high internal bandwidth. In long-context, high-concurrency workloads, the active attention state increasingly sits outside the wafer, and the 21 PB/s on-wafer bandwidth becomes less relevant. Performance then depends on off-wafer I/O, DDR5 bandwidth, and cache-management policy.

Interconnect and Scale-Out vs. GPU Clusters

Cerebras's SwarmX fabric connects multiple CS systems and supports the MemoryX/WSE execution model. The goal is to preserve a simpler programming model: parameters are managed by MemoryX, SwarmX coordinates broadcast and reduction, and each WSE behaves more like a large logical processor than one small rank in a GPU cluster.

SwarmX is mainly relevant to Cerebras's training and weight-streaming architecture. It broadcasts weights from MemoryX to multiple WSEs and reduces gradients back during training. In theory this can simplify large-model training, but it is less attractive when the model needs frequent high-bandwidth communication across many accelerators: distributed KV-cache, MoE expert routing, irregular pipeline traffic, or fine-grained collectives. Unfortunately for Cerebras, frontier models increasingly use exactly these patterns. We see little clear training advantage for SwarmX beyond simplifying parts of the weight-streaming setup.

This is where Nvidia keeps its advantage. NVLink and NVSwitch are complex and expensive, but they create a large high-bandwidth GPU domain. A GB200 NVL72 rack connects 72 Blackwell GPUs with 130 TB/s of aggregate NVLink bandwidth. Cerebras has enormous bandwidth inside the wafer, while CS-3 exposes roughly 1.2 Tb/s, or ~150 GB/s, of system I/O off the wafer.

Where Cerebras Shines

Cerebras is specifically designed for decode-heavy inference, low-to-medium batch sizes, short-to-medium context, and workloads where per-user latency matters. Wafer-scale SRAM, weight streaming, and sparse or distilled model design can produce token speeds that GPU systems usually reach only with smaller models, heavier batching, or more aggressive compression.

Token Speed, Distilled Models, and MoE

Cerebras's clearest advantage is tokens per second per user. Its cloud offering lists gpt-oss-120B as a production model running at roughly 3,000 tokens per second, and has highlighted GLM-4.7 at around 1,000 tokens per second for coding workloads. It is also bringing Kimi K2.6, a trillion-parameter open model, to enterprise customers.

The best fit is a model that is large enough to be useful, but shaped so that the active computation can stay close to the wafer. Open reasoning models, distilled models, coding-specialised models, and MoE models fit that profile better than dense frontier models with very large active footprints and heavy long-context KV-cache pressure.

GLM-4.5, for example, is a 355B-parameter MoE model with 32B active parameters per token. That active footprint is much closer to a medium dense model than to a full 355B dense model. The hardware wins if the active experts can be streamed, cached, or scheduled efficiently. However, if expert routing becomes too fine-grained or scattered, the system can lose utilisation and turn sparsity into a memory-movement problem.

Cerebras has built a custom MoE stack for this problem. REAP, or Router-weighted Expert Activation Pruning, removes low-impact experts from large MoE models rather than merging them, reducing memory overhead while largely preserving model quality. Cerebras says REAP can remove up to 50% of experts from MoE models as large as 1T parameters with limited quality loss on generative benchmarks.

Batch Tiling on Attention (BTA) addresses utilisation. MoE layers often want large batches to keep expert computation efficient, while attention is more memory-constrained. Cerebras's BTA decouples the two by processing attention in smaller tiles and then feeding a larger effective batch into the expert layers. In other words, the software tries to make sparse MoE look less irregular to the hardware.

AWS and Disaggregated Inference

Next to the flagship OpenAI partnership, the AWS deal validates Cerebras as a decode accelerator. AWS is deploying Cerebras CS-3 systems inside its own data centers and exposing the resulting fast-inference service through Amazon Bedrock. It is a split inference stack: AWS Trainium handles prefill, Cerebras handles decode. Prefill processes the prompt, builds the KV-cache, and is relatively parallel and compute-heavy. Decode generates one token at a time, is harder to batch without hurting latency, and is more memory-bandwidth-sensitive. In the AWS design, Trainium does the dense prompt-processing work, then Cerebras takes over the latency-sensitive token-generation phase. AWS and Cerebras claim the combined system can deliver up to 5× more high-speed token capacity in the same hardware footprint.

The critical question is whether this works cleanly at scale. Disaggregated prefill/decode creates a new data path: Trainium generates the KV-cache, then that cache has to move to the Cerebras decode system. That transfer adds switch latency, consumes network bandwidth, and pushes more traffic through the same off-wafer path that already constrains the WSE in long-context workloads.

Hardware–Software Co-Design

Cerebras needs hardware–software co-design more than a general-purpose GPU stack does. The WSE has enormous local bandwidth, but the workload has to be shaped around the wafer: active parameters need to stay manageable, routing needs to be predictable, and the serving stack has to keep the cores busy.

REAP and Batch Tiling on Attention are examples of this at the model-execution layer. The same pattern is now showing up in partner deployments. Cognition says SWE-1.5 was served on Cerebras at up to 950 tokens per second. OpenAI says GPT-5.3-Codex-Spark is its first real-time coding model, served on Cerebras and capable of more than 1,000 tokens per second under the right configuration. Both are coding systems built around low latency, where the model has to edit, respond, branch, and recover without forcing the developer out of flow.

The OpenAI deployment extends this from one model to infrastructure. OpenAI has committed to 750 MW of Cerebras systems for low-latency inference capacity, with Codex-Spark as the first public model from the partnership. At that scale, OpenAI has a strong incentive to tune model size, distillation, serving policy, and workload routing around the WSE. AWS is moving in the same direction from the cloud side: Bedrock uses Trainium for prefill and Cerebras for decode, making the serving architecture itself hardware-specific.

Challenges

The Cerebras architecture is best understood through its trade-off. The wafer gives the system a huge local compute and memory domain, which is why it can be so fast in the right inference regime. But that same design also creates hard constraints. SRAM does not scale like logic. A wafer has limited edge area for I/O. The compiler has to map models onto a fixed two-dimensional fabric. And when weights, KV-cache, or expert traffic move off the wafer, the system starts to lose the locality advantage that makes it distinctive.

SRAM Density and the Scaling Plateau

The most fundamental long-run constraint on the Cerebras architecture is that SRAM does not scale well with process node transitions. SRAM cells require six transistors per bit and are specifically engineered for stability and read/write speed, not area density. Roughly half of the WSE-3 die area is occupied by SRAM. On TSMC N5, that yields 44 GB across 46,225 mm². Moving to N3E or N2 does not meaningfully shrink SRAM cell area, so WSE-4 on N5 (same process, higher clock speed) gets the same 44 GB, and a hypothetical future device on N3 would see only modest gains.

WSE-1 (16nm) delivered 18 GB of SRAM. WSE-2 (7nm) delivered 40 GB. WSE-3 (5nm) delivered 44 GB, a 10% improvement over a full node transition. The compute transistor density improved substantially each generation, but SRAM capacity barely moved. SRAM is simultaneously Cerebras's primary competitive advantage and its most area-intensive, cost-intensive, and scaling-resistant component. On a per-watt or per-dollar basis, HBM offers far more memory capacity.

On-chip SRAM on a WSE-scale wafer, by node

Process node	On-chip SRAM	Logic density (vs 7 nm)
7 nm — WSE-2	40 GB	1.0×
5 nm — WSE-3	44 GB	1.8×
3 nm — projected	44 GB	2.9×
2 nm — projected	~53 GB	3.4×

For today's workloads, 44 GB is enough to serve small and medium models quickly, and it can cover the active footprint of some quantised MoE models. Many recent MoEs keep active parameters per token around the 30–40B range while increasing total parameters and expert counts. GLM-4.5 is 355B total with 32B active, DeepSeek-V3 is 671B total with 37B active, and Kimi K2.6 is 1T total with 32B active across 384 experts.

That still creates a scaling problem for Cerebras. More experts and longer contexts make placement, caching, routing, and KV-cache harder even if active FLOPs stay controlled.

Off-Wafer Input/Output

Because the WSE is physically large relative to its perimeter, the number of I/O signals that can leave the chip is structurally limited by the chip's edge. The external links have to sit near the edge of the wafer, because that is where signals can leave the chip. Reticles in the middle of the wafer cannot easily connect to the outside world, so filling the interior with SerDes blocks would waste silicon on I/O that has no practical path off the wafer.

The result is a current off-wafer bandwidth ceiling of roughly 1.2 Tb/s (~150 GB/s) per CS-3 system. Compare this to a GPU rack with NVLink and NVSwitch, where cross-chip bandwidth is measured in tens of terabytes per second, or even to Groq's LP30 at 9.6 Tb/s.

Exhibit. The bandwidth cliff at the wafer's edge

Path	Bandwidth (bytes/s)	Scope
WSE-3 on-wafer mesh	21 PB/s	on-chip, core-to-core
GB200 NVL72 (NVLink)	130 TB/s	rack of 72 GPUs
Groq LP30	1.2 TB/s	off-chip
Cerebras CS-3	150 GB/s	off-wafer I/O

The compute on the wafer is fast and well-connected internally, but the bridge to the outside world is narrow. As long as workloads keep their critical state on the wafer, this is manageable. The moment the working set overflows on-chip and the critical hot path runs through the I/O, this creates challenges.

Pipeline Parallelism and Architecture-Specific Model Constraints

Cerebras's fastest inference regime depends on mapping the model into a spatial pipeline across the wafer. Each transformer layer is assigned to a region of the WSE, and token activations flow through those regions in sequence — layer by layer, like data moving down a river.

This is why Cerebras excels at throughput per user at small batch sizes: each token's forward pass flows through the pipeline continuously at wafer-local bandwidth rather than waiting on multi-GPU all-reduce. There is no synchronisation overhead between GPUs; the mesh routes everything internally.

The challenge is that this pipeline-first architecture places constraints on which model topologies it can serve efficiently. Recent architectures like Moonshot/Kimi K2, which uses a non-standard interleaved MoE + dense attention design, and emerging "mixture of depths" or cross-layer attention models, create irregular activation graphs that are harder to map efficiently to a fixed spatial layout. These architectures are increasingly common at the frontier and represent a growing challenge for a compiler that depends on statically routing token activations across a fixed 2D mesh.

CUDA gives developers more direct control over kernels, memory movement, and synchronisation. Cerebras pushes more of the performance problem into the compiler: kernels or layers have to be placed onto WSE regions, connected with data paths, and scheduled across the wafer. That works best for regular layerwise graphs. Architectures with irregular routing, cross-layer dependencies, mixture-of-depths, or non-standard MoE/attention layouts are harder to map cleanly and may require lower-level compiler work or model changes to reach good utilisation.

Potential Mitigations: Packaging, Bonding, and Optics

Cerebras's leadership is publicly aware of the I/O and SRAM scaling bottlenecks and has outlined a roadmap for addressing them.

Hybrid bonding is the most technically credible near-term path for increasing effective bandwidth between the WSE and adjacent memory or compute layers. By directly bonding a memory wafer on top of or beneath the compute wafer at pitch below 12 µm, it is in principle possible to bring DRAM or emerging memory closer to the cores than any current packaging allows, and to do so across the full wafer area rather than only at the perimeter. Recent wafer-to-wafer hybrid-bonding work has demonstrated interconnect pitches as low as 200 nm, though yield at that pitch remains challenging. For Cerebras, hybrid bonding a DRAM wafer onto WSE could dramatically expand effective on-device memory capacity, addressing both the SRAM capacity ceiling and partially alleviating the KV-cache problem without routing through the 150 GB/s off-wafer link.

Optical interconnects would address the perimeter I/O ceiling more directly. Cerebras has outlined a vision of a wafer-scale programmable optical interconnect bonded to the WSE, using the full surface area of the wafer rather than only its edge to distribute I/O, potentially yielding a 4,000× bandwidth improvement over current copper-based off-wafer links. This is genuinely heroic engineering: it requires a standardised bonding interface across different foundry processes, new hybrid bonding techniques that work at wafer-to-wafer granularity, and photonic integration with embedded lasers — each one individually a multi-year R&D effort. Cerebras is pursuing exactly this through a DARPA-funded co-packaged-optics partnership with Ranovus, but it remains more of a long-range vision than a near-term mitigation.

Looking forward, Cerebras will also need to embrace low-precision formats like FP8 and FP4 if it wants wafer-scale inference to remain economically compelling. It has released FP8 REAP checkpoints for models such as GLM-4.7, but checkpoint-level quantisation is not the same as native FP8 or FP4 execution in the WSE datapath. Nvidia now treats lower precision as a first-class hardware feature: Blackwell supports FP8 and FP4 formats, with software increasingly built around those modes. Without comparable native support, Cerebras spends more SRAM and bandwidth per effective token than it should. FP8, and eventually FP4, would let Cerebras fit more active parameters into the same wafer footprint and improve the economics of its fast-token thesis.

In the shorter term, Cerebras's practical response to these constraints follows the co-design path: pruning models with REAP, adapting routing strategies with BTA, using lower-precision checkpoints where possible, and working with partners to design workloads that stay within the regime where the current I/O budget is not the bottleneck. The hardware roadmap is promising but speculative. The co-design roadmap is already in production and can be scaled further.

New Possibilities

The question for Cerebras is where speed changes the product enough to justify the cost. The strongest domains share three traits: latency affects the user experience or task success; extra test-time compute improves quality; and users are willing to pay for smoother interaction.

Cerebras has started to formalise this as a “scaling law” for inference: faster tokens allow models to use more inference-time tokens within the same latency budget, which in turn yields better outputs for a fixed pre-trained model. OpenAI’s test-time compute scaling law similarly shows that many benchmarks improve monotonically with the number of tokens used at inference. But speed could also make entirely new behaviours possible, such as continuous interaction models and embodied systems that combine high-level reasoning with fast action.

Coding Agents and Harnesses

Coding is the clearest use case so far. OpenAI and Cognition are already using fast Cerebras-backed models as premium coding models, with Codex-Spark and SWE-1.6 both running around 1,000 tokens per second. Independent write-ups consistently highlight the qualitative difference: when responses arrive in under a second, developers stay in flow and treat the model as a continuous collaborator rather than a batch tool they query and then ignore while working on something else.

One of the biggest challenges in agentic coding today is context switching. A typical workflow involves giving the agent a non-trivial task — refactor a subsystem, update a deployment pipeline, add a cross-cutting feature — and then watching it plan, edit, run tests, and summarise over tens of seconds or minutes. Because staring at a progress spinner is intolerable, developers switch to another task: email, Slack, a different ticket. When the agent finishes, they must reload the original problem, reconcile what the agent did, and decide next steps. Spark is optimised so that “quick responses are as crucial as intelligence” and enables “real-time, continuous coding”.

Faster inference also changes the relationship between code and harness. Today, most coding agents run inside a largely fixed harness: a static set of tools (file edits, shell, tests), rigid prompting templates, and one evaluation pipeline that must work for a wide variety of tasks.

Looking forward, a plausible next step is co-evolving harnesses that adapt to the task at hand. The agent would not only write code inside a fixed environment; it would also design and refine its own harness on the fly, generating project-specific test runners, adding or removing tools as the task unfolds, and restructuring evaluation pipelines based on what it learns about the codebase. This is only viable if harness operations themselves are fast: generating, running, and revising tests and tools must fit within a few-second budget.

Interaction Models and New Human–Machine Interfaces

The second application class is real-time interaction. Chat models can tolerate pauses because the interface is turn-based: the user sends a message, waits, and receives an answer. Live audio, video, and collaborative interfaces are less forgiving. The model has to decide when to speak, when to stay silent, when to interrupt, and how to update its response as new audio, video, or user intent arrives.

OpenAI just launched GPT Live. It processes incoming audio while it is still speaking and makes interaction decisions many times a second: whether to talk, keep listening, pause, interrupt, or call a tool. Handling audio natively removes the voice-to-text hop, and responses land in well under a second, fast enough that the model stays inside the conversation. Alongside fast coding models, GPT Live shows why OpenAI wants more low-latency inference capacity: the model has to keep up with the user in real time. Thinking Machines was first to ship an interaction-first model of this kind, but the paradigm is now arriving in mass-market products.

Serving this well demands sustained low latency at relatively large active-parameter counts and a continuous, micro-batched prefill/decode regime. The open question is long-session context management: KV-cache grows with session length and concurrency, eventually turning memory capacity and bandwidth into the bottleneck.

Compression, eviction, retrieval, summarisation, and KV-cache quantisation are all active areas of research. Systems such as RocketKV and KVzip point toward the same approach: keep less state active, compress what remains, and preserve the context that matters for the next turn.

This works commercially when faster interaction changes the dynamic of the interaction. Generic voice assistants may not justify premium inference on every request. A sales copilot, real-time translator, collaborative design tool, or support agent has a stronger case because delays change the quality of the interaction. A model that responds in roughly 400 ms can stay inside the conversation.

Ambient State Interpretation

Most AI applications today are reactive: they wait for a user to ask for help. The user has to notice the problem, gather the context, and turn it into a prompt. Ambient state interpretation moves that work into the system. The model continuously reads the workspace, infers what the user is trying to do, and becomes useful before the user has to ask.

We can already see early versions of this. Google’s Gemini Intelligent Screen Context lets the model infer when a query relates to what is visible on the user’s screen, without requiring the user to manually tap “Ask about screen”. Apple is pursuing the same direction with on-device Foundation Models in iOS 27, targeting continuous screen awareness through Siri Extensions.

The architectural challenge is continuous processing of a stream of screen states, events, or sensor inputs to maintain an always-current model of what the user is doing. Effective ambient agents must ingest far more signal than they surface, and act on observations that are current. Research on proactive AI systems identifies latency as the primary trust-breaking failure: an agent that interrupts with a suggestion based on a screen state from three seconds ago is more disruptive than no suggestion at all. ProAgent's experimental system running on Jetson Orin hardware reports 4.5-second average latency, explicitly cited as a limitation on real-world usefulness.

Current systems mostly handle this by using small on-device models to keep latency under control, but at the cost of reasoning quality: small models struggle to accurately interpret dense content and cannot reliably reason about what the user is likely to need next. Fast cloud inference could close this gap: a 70–120B reasoning model running at Cerebras-class speeds could complete a full screen interpretation and next-action prediction in well under 200 ms, achieving the latency of a small on-device model while bringing frontier-model reasoning quality to bear on complex state.

Final Thoughts

Cerebras is a bet on inference becoming more specialised and more ubiquitous. The WSE is built for: low-batch decode, fast coding loops, real-time interaction, and agents that need to observe, reason, and act without breaking user flow.

The architecture is powerful because it pulls memory and compute close together on one wafer. The limits come from the same design: SRAM capacity is hard to scale, off-wafer I/O is narrow, KV-cache becomes painful at long context, and irregular model architectures can tax the compiler. Cerebras works best when models and serving stacks are shaped around the wafer.

The existing partnerships with OpenAI and AWS test whether fast decode can become a real layer in hyperscale inference: OpenAI through latency-sensitive coding and interaction models, AWS through disaggregated prefill on Trainium and decode on CS-3.

Cerebras frames this as an inference scaling law: faster tokens let a model spend more computation at inference time without making the user wait. The same model can generate more candidates, reason for more steps, check its own work, or run more tool calls inside the same latency budget. Speed can turn into better output quality for a fixed pre-trained model.

Beyond improving existing use cases, faster inference also exposes application categories where real-time model work was previously too slow, too expensive, or too awkward to build around.

We believe AI products will move more toward live interaction, computer use, and continuous agents: tasks where token speed matters and users will pay for it. Cerebras gives these products more room to think, check, act, and recover inside the user’s latency budget.