February 26, 2024

Microsoft has developed its personal {custom} AI chip that may prepare giant language fashions and probably keep away from a pricey dependency on Nvidia. Microsoft has additionally developed its personal Arm-based processor for cloud workloads. The 2 custom-made silicon chips are meant to energy Azure knowledge facilities and put together the corporate and its enterprise clients for a future stuffed with AI. Microsoft has said that they’re due to this fact not meant on the market. Microsoft and different know-how firms face excessive prices of offering AI companies, which may value ten instances greater than companies like search engines like google.

On Wednesday, on the Microsoft Ignite convention, Microsoft introduced two {custom} chips designed to speed up AI workloads internally throughout its Azure cloud computing service: Microsoft Azure Maia 100 AI Accelerator and the Microsoft Azure Cobalt 100 processor.

Microsoft designed Maia particularly to run giant language fashions resembling GPT 3.5 Turbo and GPT-4, which underlie the Azure OpenAI and Microsoft Copilot (previously Bing Chat) companies. Maia has 105 billion transistors manufactured utilizing a 5nm TSMC course of. In the meantime, Cobalt is a 128-core ARM processor designed to run conventional computing duties like powering Microsoft Groups. Microsoft additionally has no plans to promote them, preferring them for inside use solely:

Quote Despatched by Microsoft

The chips will probably be deployed in Microsoft’s knowledge facilities early subsequent yr and can initially energy the corporate’s companies resembling Microsoft Copilot or Azure OpenAI Service. They are going to be a part of a rising portfolio of merchandise from business companions to assist meet the rising demand for environment friendly, scalable and sustainable computing energy, in addition to the wants of consumers searching for to learn from the newest advances in cloud and AI.

The chips characterize the ultimate piece of the puzzle that may allow Microsoft to ship infrastructure techniques – masking every part from the vary of chips, software program and servers to racks and cooling techniques – designed from the bottom as much as accommodate inside and buyer workloads might be optimized thoughts.

A strategic choice…

Introduced final yr, the H100 is Nvidia’s newest flagship AI chip and follows the A100, a roughly $10,000 chip thought of a workhorse for AI purposes.

Builders use the H100 to construct superior language fashions (LLM), that are on the coronary heart of AI purposes like OpenAI’s ChatGPT. These techniques are costly to function and require highly effective computer systems to course of trabytes of information for a number of days or perhaps weeks. In addition they depend on vital computing energy in order that the AI ​​mannequin can generate textual content, photos or predictions.

Coaching AI fashions, particularly giant fashions like GPT, requires tons of of high-end Nvidia GPUs working collectively.

Microsoft’s Azure Maia AI chip and Arm-powered Azure Cobalt processor will launch in 2024 after a surge in demand for Nvidia’s H100 GPUs this yr. The demand for these GPUs is so excessive that some have even fetched over $40,000 on eBay.

Microsoft truly has a protracted historical past in silicon improvement, says Rani Borkar, head of Azure {hardware} techniques and infrastructure at Microsoft.

Microsoft helped develop silicon for the Xbox greater than 20 years in the past and even helped develop chips for its Floor units. This effort builds on that have, Borkar explains. In 2017, we started designing the cloud {hardware} stack structure and started this journey that put us on the suitable path to creating our new {custom} chips.

The brand new Azure Maia AI chip and Azure Cobalt processor are each developed in-house at Microsoft and mix a deep overhaul of the complete cloud server stack to optimize energy, efficiency and price. We’re rethinking cloud infrastructure for the AI ​​period and optimizing actually each layer of that infrastructure,” says Borkar.

1700119237 538 Azure Maia 100 Microsoft unveils its first custom AI chip
Rani Borkar, company vp for Azure {Hardware} Programs and Infrastructure (AHSI) at Microsoft
…which additionally takes into consideration the chip scarcity

With chip shortages driving up costs for Nvidia’s coveted AI GPUs, a number of firms have developed or are contemplating creating their very own AI accelerator chips, together with Amazon, OpenAI, IBM and AMD. Microsoft additionally felt the necessity to develop {custom} chips to convey its personal companies to the fore.

Throughout its announcement, the corporate said:

Quote Despatched by Microsoft

Chips are the workhorses of the cloud. They management billions of transistors that course of the massive streams of ones and zeros flowing by knowledge facilities. This work finally lets you do virtually something in your display screen, from sending an electronic mail to producing a picture in Bing with a easy phrase.

Simply as you have got management over each design alternative and element when constructing a house, Microsoft sees the addition of internally developed chips as a approach to make sure each half is prepared for Microsoft Cloud and AI workloads. The chips are housed on {custom} server boards and positioned in {custom} racks that simply combine into current Microsoft knowledge facilities. {Hardware} will work hand in hand with co-developed software program to unlock new capabilities and alternatives.

Develop {hardware} and software program collectively

The corporate’s new Maia 100 AI accelerator will energy among the largest inside AI workloads working on Microsoft Azure. Moreover, OpenAI has offered suggestions on Azure Maia, and Microsoft’s detailed insights into how OpenAI’s workloads run on infrastructure tailor-made to its giant language fashions will assist inform Microsoft’s future designs.

*Since our preliminary partnership with Microsoft, now we have labored collectively to form Azure’s AI infrastructure at each degree to help our fashions and our unparalleled coaching wants, mentioned Sam Altman, CEO of OpenAI. We have been excited when Microsoft first revealed its designs for the Maia chip, and we labored collectively to refine it and take a look at it with our fashions. Azure’s end-to-end AI structure, now optimized all the way in which right down to silicon with Maia, paves the way in which for coaching higher fashions and making these fashions cheaper for our clients.

The Maia 100 AI Accelerator can be designed particularly for the Azure {hardware} stack, mentioned Brian Harry, a Microsoft engineer who leads the Azure Maia workforce. This vertical integration and alignment of chip design with the broader AI infrastructure designed particularly for Microsoft’s workloads might result in enormous efficiency and effectivity good points, he mentioned.

“Azure Maia is designed particularly for AI and to attain absolute most {hardware} utilization,” he mentioned.

In the meantime, the Cobalt 100 processor is predicated on Arm structure, a kind of energy-efficient chip design, and is optimized to ship superior effectivity and efficiency in cloud-native choices, mentioned Wes McCullough, vp of supplies engineering. The selection of Arm know-how was an necessary a part of Microsoft’s sustainability objective. The objective is to optimize efficiency per watt throughout all knowledge facilities, which basically means offering extra computing energy for every unit of power consumed.


No know-how firm is one, and Microsoft is not any exception. The corporate plans to proceed counting on third-party chips, each for provide wants and prone to fulfill its tangled internet of enterprise partnerships. Microsoft may also add the newest Nvidia H200 Tensor Core GPU to its fleet subsequent yr to help bigger mannequin inference [sic] with out a rise in latency, the corporate says, pointing to Nvidia’s just lately introduced AI processing GPU. It additionally provides digital machines accelerated by AMD MI300X Azure.

How do the brand new chips* carry out? Microsoft hasn’t launched any benchmarks but, however the firm appears pleased with the performance-per-watt ratio of the chips, notably the Cobalt. “We imagine this can allow us to supply our clients with higher, quicker, decrease value and better high quality options,” mentioned Scott Guthrie, government vp of Microsoft’s cloud and AI group.

Supply: Microsoft

And also you ?

Tinder travaille sur un new subscription mensuel a 500 dollars What are the benefits and downsides of creating your personal AI chips in comparison with shopping for chips from exterior suppliers?

Tinder travaille sur un new subscription mensuel a 500 dollars How might the Maia AI Accelerator processor be game-changer for Microsoft within the AI ​​area, particularly towards rivals like Google, Amazon and Meta?

Tinder travaille sur un new subscription mensuel a 500 dollars What are the challenges and dangers related to producing {custom} AI chips, notably when it comes to value, high quality and security?

Tinder travaille sur un new subscription mensuel a 500 dollars What are the potential utility areas of the Maia AI Accelerator processor, each for builders and finish customers?

Tinder travaille sur un new subscription mensuel a 500 dollars What are the environmental impacts of the manufacturing and use of high-performance AI chips? How can Microsoft cut back its carbon footprint on this space?