The traditional method for multiplying two n-by-n matrices requires n³ separate multiplications, but the new technique has reduced the upper bound of the exponent, bringing it closer to the ideal value of 2. This advancement could lead to faster training times and more efficient execution of tasks for AI models, potentially leading to advancements in AI capabilities and the development of more sophisticated AI applications. Additionally, it could make AI technologies more accessible by lowering the computational power and energy consumption required for these tasks, thereby reducing AI's environmental impact.
Key takeaways:
- Computer scientists have discovered a new way to multiply large matrices faster than ever before, potentially accelerating AI models like ChatGPT, which rely heavily on matrix multiplication to function.
- The new technique improves upon the 'laser method' introduced by Volker Strassen in 1986, reducing the upper bound of the complexity exponent, ω, bringing it closer to the ideal value of 2.
- The 2023 breakthrough stemmed from the discovery of a 'hidden loss' in the laser method, where useful blocks of data were unintentionally discarded. By modifying the way the laser method labels blocks, the researchers were able to reduce waste and improve efficiency significantly.
- For AI models, a reduction in computational steps for matrix math could translate into faster training times and more efficient execution of tasks, potentially leading to advancements in AI capabilities and the development of more sophisticated AI applications.