Microsoft's CTO confirms long-term goal to replace Nvidia/AMD with in-house silicon

The chips and servers housed in data centers have been essential to building artificial intelligence models and applications. While Nvidia has led this market, major cloud computing companies, including Microsoft, have started creating their own specialized chips.

During a fireside discussion at Italian Tech Week moderated by CNBC, Kevin Scott, Microsoft’s chief technology officer, explained the company’s approach to AI chips.

Currently, Microsoft relies mainly on Nvidia and AMD chips in its data centers. The company’s priority has been selecting the right semiconductors that deliver “the best price performance” for each chip.

Scott says they’re flexible about chip choices. Nvidia has simply offered the best performance for the price over the years. They’re willing to look at any supplier to make sure they have enough capacity for the demand.

Meanwhile, Microsoft has already started incorporating some of its own chips into its operations. The company introduced the Azure Maia AI Accelerator in 2023, built for AI workloads, along with the Cobalt CPU. Reports indicate the company is developing its next batch of semiconductor products.

Just last week, Microsoft revealed new cooling technology that uses “microfluids” to address chip overheating problems. When questioned whether Microsoft’s long-term goal is to use mostly its own chips in company data centers, Scott responded: “Absolutely,” noting that the company already uses “lots of Microsoft” silicon currently.

Custom chips are just the beginning

According to Scott, the chip strategy is part of a broader plan to eventually create a complete system for data centers.

Scott explains that it goes beyond just the hardware. The focus is on networking, cooling systems, and having the flexibility to make choices that best match computing power to whatever tasks are being run. Microsoft, Google, and Amazon are all creating custom chips, not only to reduce their dependence on Nvidia and AMD, but also to better tailor the hardware to their specific requirements.

Major technology companies, including Meta, Amazon, Alphabet, and Microsoft, have pledged over $300 billion in capital expenditures this year, with a large portion directed toward AI investments as they attempt to keep up with surging AI demand.

Severe Computing Shortages Persist

Scott pointed out that computing capacity remains in short supply.

Scott says calling it a massive shortage of computing power doesn’t even capture the full scale. Since ChatGPT’s launch, building capacity quickly enough has been nearly impossible.

Microsoft has been expanding capacity through new data centers, but the CTO cautioned that it still falls short of meeting demand.

Scott mentioned that even their most aggressive predictions keep falling short. They’ve rolled out huge amounts of capacity recently, and the expansion will be even bigger in the next few years.