[d] Apple claims M2 Ultra "can train massive ML workloads, like large transformer models."
Oct 11, 2023 - news.bensbites.co
The discussion revolves around training models with Apple's silicon, specifically the M2 Ultra, which boasts a 32-core Neural Engine that is 40% faster. The M2 Ultra can support up to 192GB of unified memory, which is 50% more than its predecessor, the M1 Ultra. This large memory capacity allows it to train massive machine learning workloads, such as large transformer models, which other chips cannot handle due to memory limitations.
However, questions are raised about the specific large transformer models being referred to, possibly referring to Language Learning Models (LLMs). There are also concerns about the speed of training these models, as despite the large memory capacity, the process might still be too slow.
Key takeaways:
The 32-core Neural Engine is 40% faster.
M2 Ultra can support up to 192GB of unified memory, which is 50% more than M1 Ultra.
The large memory capacity allows it to train massive machine learning workloads, like large transformer models, that other chips cannot handle due to memory limitations.
There is a discussion on whether the training of these large models would be too slow, despite the large memory capacity.
