{"id":61196,"date":"2025-11-20T22:56:34","date_gmt":"2025-11-20T21:56:34","guid":{"rendered":"https:\/\/www.apfelpatient.de\/?p=61196"},"modified":"2025-11-20T23:09:49","modified_gmt":"2025-11-20T22:09:49","slug":"apple-boosts-llm-performance-thanks-to-optimized-m5-design","status":"publish","type":"post","link":"https:\/\/www.apfelpatient.de\/en\/news\/apple-steigert-llm-performance-dank-optimiertem-m5-design","title":{"rendered":"Apple boosts LLM performance thanks to optimized M5 design"},"content":{"rendered":"

In its latest Machine Learning Research blog, Apple demonstrates the significant improvements made by the new M5 chip in executing local LLMs. In direct comparison to the M4, the chip achieves noticeably higher speeds. The focus is primarily on how quickly local language models generate the first token and how efficiently subsequent tokens are generated. The report provides clear metrics and explains why the M5 offers advantages in both areas.<\/strong><\/p>\n\n\n\n

To put these results into perspective, it's helpful to look at MLX. Apple released this framework a few years ago to make machine learning natively accessible on Apple Silicon. MLX is open source and built as an array framework. It's based on NumPy and utilizes Apple Silicon's unified memory architecture. This allows operations to seamlessly switch between the CPU and GPU without moving memory. MLX includes packages for neural networks, optimization, automatic differentiation, and graph optimization.<\/p>\n\n\n\n

A key component is MLX LM. This package allows Hugging Face models to run locally, including text generation and fine-tuning. MLX LM supports quantization, which reduces the memory requirements of models and enables faster inference. This makes large models viable even on devices with less RAM. Apple bases its comparisons between the M4 and M5 on this.<\/p>\n\n\n\n

Background information on MLX and MLX LM<\/h4>\n\n\n\n

MLX offers a flexible system that covers numerical simulations, scientific computations, and machine learning. For language models, MLX LM provides the appropriate tools. These allow large models to be loaded, executed, and fine-tuned, with quantization playing a key role. Quantization reduces both memory requirements and computational load, and accelerates inference.<\/p>\n\n\n\n

MLX fully leverages the unified architecture of the Apple Silicon platform. Support for BF16, mixed-precision formats, and automatic differentiation ensures efficient model execution. The entire memory pool is available for running large models. This is crucial for LLMs, as subsequent tokens are memory-intensive to compute.<\/p>\n\n\n\n

M5 compared to the M4<\/h4>\n\n\n\n

Apple tested several models to highlight the differences between the two chips. These include:<\/p>\n\n\n\n