Local GenAI on Apple Silicon Macs: what real performance?

Why it has become viable

Since WWDC 2026, four factors have converged to make local generative AI credible on the Mac, as Guillem Riera sets out in his practical guide: Apple's strategy, the maturity of the MLX runtime, the availability of quantised models, and better ecosystem tooling.

The key point: MLX-LM can expose a local OpenAI-compatible API. In practice, any tool designed for ChatGPT (such as OpenCode) can point at http://localhost:8080 and work with a model running, well, on your desk.

The real advantage: unified memory

On Apple Silicon chips, the GPU and the CPU share the same pool of RAM. There is no "VRAM cliff" as there is on a PC with a dedicated graphics card: if you have 24, 36 or 64 GB of unified memory, the model can sit in it directly. According to several analyses, this is a genuine superpower for local inference.

Another subtlety: LLM inference is limited by memory bandwidth, not by compute power. An M4 Max tops out at around 546 GB/s, and it is that figure, more than the TFLOPS, that determines the number of tokens per second.

So what performance, concretely?

For individual use, coding, summarising or brainstorming, it is perfectly comfortable. A 14B model runs at 35 to 55 tokens/s on an M4 Pro with 24 GB; a quantised 24 to 33B sits around 30 to 45 tokens/s on an M4 Max. Above roughly 15 tokens/s, reading keeps up without frustration. MLX, Apple's native framework for Metal, often has the edge over the alternatives (up to +20% to +87% on small models).

Where local hits its limits

Local is excellent for one user. But as soon as it comes to serving an organisation, the model changes:

• Concurrency: a Mac efficiently serves one request at a time. Serving 50, 200 or 340 users simultaneously (as in our local authority deployments) calls for server GPUs and an inference engine optimised for multi-user throughput.
• Availability: a workstation offers neither high availability, nor monitoring, nor backup, nor SSO.
• Governance: it is impossible to apply a security policy, documented GDPR compliance or traceability at scale on individual machines.

In other words: local AI on the Mac is a superb testing ground and personal-use tool. For an organisation-wide AI, sovereign and at scale, server infrastructure remains indispensable, which is precisely what we deploy and operate.