“There is currently nothing better than NVIDIA hardware for AI.” Elon Musk’s line sat among the customer endorsements Nvidia released alongside Blackwell, its new GPU platform, at the GTC conference on March 18, 2024. The company’s pitch is cost: Nvidia says Blackwell can run real-time generative AI on trillion-parameter language models at up to 25 times less cost and energy than its predecessor.
Blackwell succeeds the Hopper architecture, which launched two years earlier. It is named for David Harold Blackwell, a mathematician who worked on game theory and statistics and was the first Black scholar inducted into the National Academy of Sciences.
“Generative AI is the defining technology of our time,” said Jensen Huang, Nvidia’s founder and CEO. “Blackwell is the engine to power this new industrial revolution.”
What is inside
Each Blackwell GPU carries 208 billion transistors. It is made on a custom TSMC process called 4NP, and it is really two dies joined by a 10 TB/second chip-to-chip link that lets them work as one GPU. A second-generation Transformer Engine adds 4-bit floating point inference, which Nvidia says doubles the compute and the model sizes the hardware can support.
The fifth-generation NVLink moves 1.8TB/s in both directions per GPU and can tie together up to 576 GPUs. Nvidia also added a reliability engine that uses AI-based diagnostics to predict faults, so that big deployments can run for weeks or months without stopping. Other additions are a decompression engine for database queries and confidential computing features aimed at healthcare and finance.
The rack-scale systems
The GB200 Grace Blackwell Superchip links two B200 GPUs to a Grace CPU across a 900GB/s connection. The headline machine is the GB200 NVL72, a liquid-cooled rack holding 36 of those superchips: 72 Blackwell GPUs and 36 Grace CPUs. Nvidia says it delivers up to 30 times the LLM inference performance of the same number of H100 GPUs, at up to 25 times lower cost and energy use. As a single unit it offers 1.4 exaflops of AI performance and 30TB of fast memory.
For buyers who want something smaller, the HGX B200 board links eight B200 GPUs and supports networking at up to 400Gb/s.
Who is lining up
Amazon Web Services, Dell, Google, Meta, Microsoft, OpenAI, Oracle, Tesla and xAI are among the organizations expected to adopt it. Sam Altman of OpenAI said “Blackwell offers massive performance leaps.” Mark Zuckerberg said Meta plans to use it to help train its open-source Llama models.
AWS, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure will be among the first cloud providers to offer Blackwell instances, as will CoreWeave, IBM Cloud, Lambda and others. Cisco, Dell, Hewlett Packard Enterprise, Lenovo and Supermicro are expected to build servers around it. Partner products start arriving later this year.
