A breakthrough in matrix multiplication has been discovered by scientists, which could greatly accelerate and improve the efficiency of artificial intelligence modeling. The new method, which was previously unknown, has the potential to enhance the performance of AI models that heavily rely on matrix multiplication, such as ChatGPT. Recent studies presented in two articles have shown significant progress in the effectiveness of matrix multiplication, a development not seen in over a decade.
Matrix multiplication is a crucial component of modern AI models, used in applications like speech recognition, image processing, chat bots, and image generation. Graphics processing units (GPUs) are particularly adept at handling matrix multiplication tasks due to their ability to perform numerous calculations simultaneously.
Researchers from the University of Tsinghua, the University of California at Berkeley, and the Massachusetts Institute of Technology have been working on theoretical advancements to reduce the complexity of matrix multiplication for improved efficiency at a general level. Traditionally, multiplying two N by N matrices requires around N^3 individual multiplications. However, the new technique has lowered the upper bound to a more optimal value of 2.
In 2020, significant strides were made in enhancing the efficiency of matrix multiplication, and in November 2023, a method was introduced that eradicated inefficiencies in previous approaches, setting a new upper bound for the complexity measure. This breakthrough marks the most significant progress in this area since 2010. Enhancing matrix multiplication methods will speed up AI model training and boost task performance.
Practically, this advancement could lead to the creation of more sophisticated AI models, hasten their training processes, and reduce the required computational power and energy usage. These improvements may also decrease the environmental impact of AI technology. The impact on AI model speed will vary based on system architecture and the extent to which tasks rely on matrix multiplication. Algorithmic efficiency enhancements often need to be paired with hardware optimization for full acceleration potential.