The shortage of GPUs has led to a surge in demand for AI chips, with companies like CoreWeave benefiting from the situation. CoreWeave, which isn't building its own AI chips to compete, is expected to make billions from their GPU cloud. On the other hand, Nvidia's DGX Systems VP and GM, Charlie Boyle, argues that the issue is not a GPU shortage, but a supply chain problem. He believes the "GPU shortage" issue will eventually resolve itself as it's more about poor planning than an actual shortage.
Key takeaways:
- There is a high demand for Nvidia’s H100 GPU for large language model (LLM) training, with companies such as OpenAI, Inflection, Meta, and others speculated to want thousands of these GPUs.
- Despite the high demand, Nvidia’s Charlie Boyle, VP and GM of Nvidia’s DGX Systems, insists that there is no GPU shortage, but rather a supply chain issue affecting the delivery of these GPUs to the market.
- Startups like CoreWeave, which provide GPU cloud services, are benefiting from the high demand for GPUs. CoreWeave is expected to make billions from their GPU cloud and has a massive backlog of client demand.
- While there are efforts to optimize machine-learning models to work faster and lower compute costs, such as CentML’s offering, these are more effective for AI inference rather than training large language models from scratch.