In the future, the author predicts that inference will dominate over training, CPUs will become competitive for inference, deployment engineers will gain more power, and application costs will rule. This shift, the author argues, will likely reduce Nvidia's market share, even though the overall AI market will continue to grow. The author expects traditional CPU platforms like x86 and Arm to benefit from this shift, as inference will need to be tightly integrated into traditional business logic to run end-user applications.
Key takeaways:
- Nvidia's dominance in machine learning is attributed to factors such as the lack of large ML applications, the superiority of Nvidia's platform over alternatives, the purchasing power of researchers, and the importance of training latency.
- However, the author predicts that in the future, inference will dominate over training, CPUs will become competitive for inference, deployment engineers will gain more power, and application costs will rule.
- These changes could lead to a decrease in Nvidia's share of the AI market, despite the market's overall growth.
- The author believes that improving inference will have a high impact in the coming years and is focusing his research and startup efforts in this area.