Apple’s latest update on the M5 chip really stands out, especially in how it boosts local machine learning capabilities. In a recent blog post, Apple highlighted significant improvements over the M4 chip, particularly for running large language models (LLMs).
For some context, Apple introduced MLX, a framework for machine learning tailored to their silicon chips. This open-source tool helps developers build and run machine-learning models on Macs. It’s designed to be efficient and user-friendly, leveraging Apple’s optimized memory architecture. With MLX, users can easily generate text and fine-tune language models right on their devices.
One exciting part of MLX is MLX LM, which allows users to download models from platforms like Hugging Face and run them locally. This is handy, especially when considering the framework’s support for quantization. This technique compresses large models for faster performance, making AI tasks quicker and smoother.
Now, let’s dig deeper into the comparison between the M5 and M4 chips. In the blog post, Apple discusses the M5’s improved performance thanks to new GPU Neural Accelerators. These make significant strides in machine-learning tasks, particularly in tasks requiring heavy computation.
Apple ran tests comparing the time it took for models to generate a response from both the M4 and M5. They focused on how quickly each chip could respond to prompts. The M5 showed a performance boost of between 19% and 27% when generating the first token, depending on the model used.
Interestingly, performance varies based on what the AI is generating. The first token is typically more demanding on processing power, while generating subsequent tokens is more reliant on memory. Notably, when producing multiple tokens, the M5 still outperformed the M4, reflecting its enhanced speed.
According to Apple, the M5 not only has greater computational power but also features improved memory bandwidth—153GB/s compared to the M4’s 120GB/s. This jump in performance means that even complex models can run efficiently on devices with 24GB of memory, illustrating how far technology has come in managing heavy workloads.
Looking to the future, the improvements in chips like the M5 are critical as machine learning continues to advance. As noted by experts in the tech industry, the efficiency of these new chips opens up possibilities for more sophisticated applications, making it easier for developers to innovate.
You can check out Apple’s full blog post on their insights here and learn more about the MLX framework here.
