Why Memory Bandwidth Decides Tokens per Second on a Mac

Published November 2, 2026 · 2 min read

Two Macs can both hold the same model and still feel completely different. One types its answer almost instantly, the other crawls. The reason is that a local LLM does two different jobs, and each is limited by a different part of the chip.

Writing is limited by memory bandwidth

To produce each new token, the model has to read all of its active weights from memory once. The GPU does very little arithmetic compared with the amount of data it must stream, so the chip waits for memory. That gives a simple ceiling:

tokens per second ≈ memory bandwidth ÷ bytes read per token

Take a model that weighs about 4.5 GB. On a chip with 100 GB/s of bandwidth the ceiling is around 22 tokens per second. With 400 GB/s it is around 89. Real results usually land at a share of that ceiling, because of software overhead and the KV cache that is read too. That is why the estimates in the calculator come with a range rather than a promise.

This also explains why Pro, Max and Ultra chips feel faster for large models: their main difference is a much wider memory bus, not a different kind of core.

Reading is limited by compute

Before the first word appears, the model has to process your whole prompt. This step is called prefill, and it works on many tokens at once, so it is limited by GPU compute, not by memory. Here the number of GPU cores and the chip generation matter, and a long prompt (a document, a code base, an agent history) can mean a visible wait before the answer starts.

So two numbers describe a Mac for local AI: how fast it writes (bandwidth) and how fast it reads (compute). For chat with short prompts, writing speed is what you feel. For long documents and coding agents, reading speed decides.

Why a big model is slower

A dense model reads every weight for every token, so a 70B model at 4-bit (about 40 GB) is roughly nine times slower than an 8B model (about 4.5 GB) on the same Mac. Mixture-of-experts models read only the active experts for each token, so they are much faster than their total size suggests, as long as the whole model still fits in RAM.

How to use this when buying

Decide the model class you want to run, then check that it fits in memory.
Among the Macs that fit it, compare memory bandwidth: that is your writing speed.
If you work with long prompts, give weight to the GPU core count and chip generation: that is your reading speed.
Quantization helps both: a 4-bit model reads half as many bytes per token as an 8-bit one.

The calculator does this arithmetic for every Mac and model in its catalog and shows the writing speed, the reading speed and the waiting time for a prompt side by side.