This move is part of Microsoft's strategy to control every element of its cloud and AI workloads. The company plans to integrate these chips into custom server boards and racks within its existing datacenters. Microsoft will also continue to use third-party chips, including the latest Nvidia H200 Tensor Core GPU and AMD MI300X-accelerated virtual machines. While performance benchmarks for the new chips have not been released, Microsoft is reportedly satisfied with their performance-per-watt ratios, particularly for the Cobalt chip.
Key takeaways:
- Microsoft has announced two custom chips, the Microsoft Azure Maia 100 AI Accelerator and the Microsoft Azure Cobalt 100 CPU, designed for accelerating in-house AI workloads through its Azure cloud computing service.
- The Maia chip is specifically designed to run large language models like GPT 3.5 Turbo and GPT-4, while the Cobalt chip is a 128-core ARM-based CPU designed for conventional computing tasks.
- Microsoft's move to design its own chips comes amid chip shortages and high costs of delivering AI services. The company sees the addition of homegrown chips as a way to tailor every element for Microsoft cloud and AI workloads.
- Despite developing its own chips, Microsoft will continue to rely on third-party chips, planning to add the latest Nvidia H200 Tensor Core GPU to its fleet next year and AMD MI300X-accelerated virtual machines to Azure.