NVIDIA and key partners announced the availability of new products and services featuring the NVIDIA H100 Tensor Core GPU — the world’s most powerful GPU for AI — to address rapidly growing demand for generative AI training and inference.
Oracle Cloud Infrastructure (OCI) announced the limited availability of new OCI Compute bare-metal GPU instances featuring H100 GPUs. Additionally, Amazon Web Services announced its forthcoming EC2 UltraClusters of Amazon EC2 P5 instances, which can scale in size up to 20,000 interconnected H100 GPUs. This follows Microsoft Azure’s private preview announcement last week for its H100 virtual machine, ND H100 v5.
Additionally, Meta has now deployed its H100-powered Grand Teton AI supercomputer internally for its AI production and research teams.
NVIDIA founder and CEO Jensen Huang announced during his GTC keynote today that NVIDIA DGX H100 AI supercomputers are in full production and will be coming soon to enterprises worldwide.
“Generative AI’s incredible potential is inspiring virtually every industry to reimagine its business strategies and the technology required to achieve them,” said Huang. “NVIDIA and our partners are moving fast to provide the world’s most powerful AI computing platform to those building applications that will fundamentally transform how we live, work and play.”
Hopper Architecture Accelerates AI
The H100, based on the NVIDIA Hopper GPU computing architecture with its built-in Transformer Engine, is optimized for developing, training and deploying generative AI, large language models (LLMs) and recommender systems. This technology makes use of the H100’s FP8 precision and offers 9x faster AI training and up to 30x faster AI inference on LLMs versus the prior-generation A100. The H100 began shipping in the fall in individual and select board units from global manufacturers.
The NVIDIA DGX H100 features eight H100 GPUs connected with NVIDIA NVLink high-speed interconnects and integrated NVIDIA Quantum InfiniBand and Spectrum Ethernet networking. This platform provides 32 petaflops of compute performance at FP8 precision, with 2x faster networking than the prior generation, helping maximize energy efficiency in processing large AI workloads.
DGX H100 also features the complete NVIDIA AI software stack, enabling enterprises to seamlessly run and manage their AI workloads at scale. This offering includes the latest version of NVIDIA AI Enterprise, announced separately, as well as NVIDIA Base Command, the operating system of the DGX data center, which coordinates AI training and operations across the NVIDIA DGX platform to simplify and streamline AI development.
AI Pioneers Adopt H100
Several pioneers in generative AI are adopting H100 to accelerate their work:
- OpenAI used H100’s predecessor — NVIDIA A100 GPUs — to train and run ChatGPT, an AI system optimized for dialogue, which has been used by hundreds of millions of people worldwide in record time. OpenAI will be using H100 on its Azure supercomputer to power its continuing AI research.
- Meta, a key technology partner of NVIDIA, developed its Hopper-based AI supercomputer Grand Teton system with multiple performance enhancements over its predecessor, Zion, including 4x the host-to-GPU bandwidth, 2x the compute and data network bandwidth, and 2x the power envelope. With this greater compute capacity, Grand Teton can support both the training and production inference of deep learning recommender models and content understanding.
- Stability AI, a pioneer in text-to-image generative AI, is an H100 early access customer on AWS. Stability AI plans to use H100 to accelerate its upcoming video, 3D and multimodal models.
- Twelve Labs, a platform that gives businesses and developers access to multimodal video understanding, plans to use H100 instances on an OCI Supercluster to make video instantly, intelligently and easily searchable.
- Anlatan, the creator of the NovelAI app for AI-assisted story writing and text-to-image synthesis, is using H100 instances on CoreWeave’s cloud platform for model creation and inference.
DGX H100 Around the World
Innovators worldwide are receiving the first wave of DGX H100 systems, including:
- CyberAgent, a leading digital advertising and internet services company based in Japan, is creating AI-produced digital ads and celebrity digital twin avatars, fully using generative AI and LLM technologies.
- Johns Hopkins University Applied Physics Laboratory, the U.S.’s largest university-affiliated research center, will use DGX H100 for training LLMs.
- KTH Royal Institute of Technology, a leading European technical and engineering university based in Stockholm, will use DGX H100 to provide state-of-the-art computer science programs for higher education.
- Mitsui, one of Japan’s leading business groups, which has a wide variety of businesses in fields such as energy, wellness, IT and communication, is building Japan’s first generative AI supercomputer for drug discovery, powered by DGX H100.
- Telconet, a leading telecommunications provider in Ecuador, is building intelligent video analytics for safe cities and language services to support customers across Spanish dialects.
Availability
NVIDIA DGX H100 supercomputers are in full production and orderable from NVIDIA partners worldwide. Customers can trial DGX H100 today with NVIDIA DGX Cloud. Pricing is available from NVIDIA DGX partners worldwide.
NVIDIA H100 in the cloud is available now from Azure in private preview, Oracle Cloud Infrastructure in limited availability, and generally available from Cirrascale and CoreWeave. AWS announced H100 will be available in the coming weeks in limited preview. Google Cloud along with NVIDIA’s cloud partners Lambda, Paperspace and Vultr plan to offer H100.
Servers and systems featuring NVIDIA H100 GPUs are available from leading server makers including Atos, Cisco, Dell Technologies, GIGABYTE, Hewlett Packard Enterprise, Lenovo and Supermicro.
Pricing and other details are available directly from NVIDIA partners.
Watch Huang discuss the NVIDIA Hopper architecture in his GTC keynote.
About NVIDIA
Since its founding in 1993, NVIDIA has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.
For more information, visit www.nvidia.com.
