Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

Nvidia introduces the H200, an AI-crunching monster GPU that may speed up ChatGPT

Nov 13, 2023 - arstechnica.com
Nvidia has announced the HGX H200 Tensor Core GPU, a successor to the H100 GPU, which is expected to power the next generation of AI applications. The H200, which uses the Hopper architecture, could lead to more powerful AI models and faster response times for existing ones. The GPU is not typically used for graphics but is ideal for AI applications as it performs vast numbers of parallel matrix multiplications, necessary for neural networks to function. The H200 could be an attractive product for cloud providers looking to alleviate the compute bottleneck in AI progress.

The H200 is the first GPU to offer HBM3e memory, providing 141GB of memory and 4.8 terabytes per second bandwidth. It will be available in several form factors, including Nvidia HGX H200 server boards and the Nvidia GH200 Grace Hopper Superchip. Amazon Web Services, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure will be the first to deploy H200-based instances starting next year. However, Nvidia continues to face challenges with US government export restrictions limiting sales to China.

Key takeaways:

  • Nvidia has announced the HGX H200 Tensor Core GPU, a powerful AI chip that could potentially accelerate AI applications and improve response times for existing AI models.
  • The H200 is the first GPU to offer HBM3e memory, providing 141GB of memory and 4.8 terabytes per second bandwidth, which is 2.4 times the memory bandwidth of the Nvidia A100 released in 2020.
  • Amazon Web Services, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure will be the first to deploy H200-based instances starting next year, with availability from global system manufacturers and cloud service providers starting in Q2 2024.
  • Nvidia is facing export restrictions from the US government, limiting sales to China. In response, Nvidia has introduced three new scaled-back AI chips for the Chinese market, which accounts for a quarter of Nvidia's data center chip revenue.
View Full Article

Comments (0)

Be the first to comment!