The change applies to Nvidia's air-cooled B300 boxes but not to the more powerful GB300 NVL72 systems, which still count packages as GPUs due to their chip-to-chip interconnects. The B300 lacks this interconnect, meaning the two chips operate as distinct GPUs, which Nvidia claims improves power and thermal efficiency. However, this comes with the disadvantage of requiring off-package memory access. Nvidia plans to continue this naming convention with its upcoming Vera Rubin superchips, which will also count individual dies as GPUs, allowing for claims of 576 GPUs per rack. Pricing details for these changes are still being finalized.
Key takeaways:
- Nvidia has redefined what counts as a GPU by counting each die as a separate GPU, potentially doubling AI Enterprise license costs.
- The HGX B300 NVL16 system now counts 16 GPUs due to its eight modules with two dies each, compared to the previous HGX B200's eight GPUs.
- The B300 lacks a chip-to-chip interconnect, leading to distinct GPUs sharing a package, which affects memory access and performance.
- Nvidia's future Vera Rubin superchips will adopt the B300 naming convention, counting individual dies as GPUs, impacting future licensing and pricing.