April 16, 2024
  • Nvidia on Monday unveiled the H200, a graphics processor designed for coaching and deploying the forms of synthetic intelligence fashions which might be driving the generative AI growth.
  • The H200 has 141GB of next-generation “HBM3” reminiscence, which helps it generate textual content, photos or predictions utilizing AI fashions.
  • Curiosity in Nvidia’s AI GPUs has buoyed the corporate, with gross sales anticipated to rise 170% this quarter.

Jensen Huang, president of Nvidia, holds the Grace Hopper superchip CPU for generative AI at Supermicro’s keynote presentation throughout Computex 2023.

Walid Berrazeg | Mild rocket | Getty Photographs

Nvidia on Monday unveiled the H200, a graphics processing unit designed to coach and deploy the forms of synthetic intelligence fashions which might be powering the generative AI growth.

The brand new GPU is an improve from the H100, the chip OpenAI makes use of to coach its most superior massive language mannequin, GPT-4. Giant firms, startups and authorities companies are vying for a restricted provide of chips.

In accordance with an estimate by Raymond James, H100 chips value between $25,000 and $40,000, and 1000’s of them should work collectively to create the most important fashions in a course of referred to as “coaching.”

Pleasure over Nvidia’s AI GPUs has boosted the corporate’s inventory, which is up greater than 230% up to now in 2023. Nvidia expects income of round $16 billion for the third fiscal quarter, up 170% from a 12 months in the past.

The H200’s key enchancment is that it has 141GB of next-generation “HBM3” reminiscence, which helps the chip carry out “inference,” or use a big mannequin after it has been used to generate textual content, photos, or predictions was educated.

Nvidia stated the H200 will produce output nearly twice as quick because the H100. That is primarily based on a check with Metas Llama 2 LLM.

The H200, anticipated to ship within the second quarter of 2024, will compete with AMD’s MI300X GPU. AMD’s chip, just like the H200, has further reminiscence in comparison with its predecessors, serving to to accommodate massive fashions on the {hardware} to run inference.

Nvidia H200 chips in an Nvidia HGX system with eight GPUs.


Nvidia stated the H200 can be appropriate with the H100, that means AI firms that already prepare with the earlier mannequin gained’t want to alter their server programs or software program to make use of the brand new model.

Nvidia says will probably be obtainable in server configurations with 4 or eight GPUs on the corporate’s HGX full programs, in addition to in a chip referred to as GH200, which mixes the H200 GPU with an Arm-based processor.

Nonetheless, the H200 might not maintain the crown because the quickest Nvidia AI chip for lengthy.

Whereas firms like Nvidia provide many various configurations of their chips, new semiconductors usually take a giant step ahead each two years or in order producers change to a distinct structure that permits for bigger efficiency positive aspects than including reminiscence or different minor tweaks. Each the H100 and H200 are primarily based on Nvidia’s Hopper structure.

In October, Nvidia informed traders that it could transfer from a two-year structure cadence to a one-year launch cadence resulting from excessive demand for its GPUs. The corporate confirmed a slide indicating that it’ll announce and launch its B100 chip, primarily based on the upcoming Blackwell structure, in 2024.

REGARD: We firmly consider within the AI ​​pattern subsequent 12 months

Don’t miss these tales from CNBC PRO: