OpenAI Partners With Broadcom to Create Jalapeño Chip: A 'Break Free From Nvidia' Gamble, Will It Succeed?

Jalapeño is a chip designed specifically for AI inference. Within the AI computing workflow, there are two key stages: 'training' and 'inference.' Training uses massive amounts of data to 'teach' a model, while inference is when the model generates answers after receiving queries from users. Jalapeño is targeted specifically at the inference stage.

Unlike Nvidia's general-purpose GPUs, Jalapeño is optimized for just one thing: running large language models quickly and energy-efficiently. It does not need to balance other tasks like graphics rendering, allowing it to achieve higher efficiency and lower costs for inference tasks. Richard Ho, OpenAI's head of hardware, revealed that the team has deeply customized core computing, memory transfer, and network architecture, aiming to make every response from products like ChatGPT cheaper and faster.

Richard Ho said that Jalapeño gives OpenAI complete control from models to products, and on to chips and data centers. This means OpenAI is no longer just a software company renting Nvidia's computing power, but is transitioning into a compute infrastructure company.

Unlike the expansion paths of traditional internet companies, there is a clear logic behind Jalapeño: better models can assist in designing better chips, better chips can lower the running costs of next-generation models, and lower costs can in turn support more users and products. Once this closed loop is successfully established, the idea of Nvidia shifting from a core supplier to an alternative option will no longer be just alarmist talk.

Choosing the inference track as the entry point. Instead of engaging in a head-on battle in Nvidia's strongest domain of training, OpenAI chose to target inference scenarios—the computational step where a user prompts ChatGPT and the model generates a response.

This is a more pragmatic entry point. Training demands extremely high computing density and versatility, areas where Nvidia's GPUs hold a structural advantage. In contrast, inference handles massive requests under established model architectures, making it easier for specialized application-specific chips to outperform general-purpose GPUs in efficiency and cost.

More importantly, inference costs represent an ongoing daily expense for OpenAI. Broadcom CEO Hock Tan revealed that Jalapeño's inference costs could be around 50% lower than mainstream AI GPUs, with performance roughly on par with Nvidia's Blackwell chips.

AI-assisted design, tape-out completed in nine months. While traditional high-end chips typically take two to three years from architectural design to tape-out, Jalapeño achieved this in just nine months.

A key factor behind this was OpenAI leveraging its own models to accelerate the chip design workflow. AI is no longer just a consumer of chips; it is beginning to design them—a fact compelling enough on its own.

Not just a single chip, but an entire roadmap. Jalapeño is only the first step. OpenAI and Broadcom have mapped out a multi-generation chip roadmap, aiming to deploy 10 gigawatts of custom chip computing power by 2029. Broadcom projects its AI-related revenue to surpass $100 billion by fiscal year 2027.

OpenAI is not alone. Google ( GOOGL ), Amazon ( AMZN ), Microsoft ( MSFT ), Meta ( META) are all developing in-house chips, and Anthropic has also initiated evaluations for custom silicon. The push for custom chips is shifting from isolated experiments by individual companies into a broad industry trend.

1. Nvidia's Scale Advantage Remains Massive

Broadcom expects AI chip revenue to exceed $100 billion in fiscal year 2027, a figure that seems quite impressive. However, Nvidia's data center revenue in the first quarter of fiscal year 2027 alone reached $75.2 billion, up 92% year-over-year. In other words, the AI hardware Nvidia sells in a single quarter already exceeds Broadcom's projected AI chip revenue for an entire year. The scale gap between the two remains vast.

2. The CUDA Software Ecosystem Is a Barrier Difficult to Bypass

Nvidia's true competitive advantage is not GPU hardware, but CUDA and its bundled software ecosystem, which supports over 7,000 applications and is backed by a toolchain that millions of developers have long relied on.

As an ASIC chip, Jalapeño does not support CUDA, nor does it plan to be compatible. But the key point is that OpenAI does not need to sell Jalapeño externally; it only needs to optimize it for its own inference workloads. While this strategy can bypass CUDA, it is not yet enough to shake CUDA's dominant position in the broader AI development community.

3. Limited Short-Term Capacity, More of a Supplement Than a Replacement

OpenAI has made it clear that Jalapeño is a supplement to existing computing power, not a replacement. Brockman stated bluntly that the company simply cannot acquire computing power fast enough.

There is also an easily overlooked background: about nine months ago, Nvidia had just completed a $30 billion strategic investment in OpenAI. Jensen Huang said at the time, "this might be the last one," indicating that Nvidia had long anticipated the Jalapeño project. The two sides still maintain a symbiotic relationship.

A more cautious assessment is that Nvidia's competitive advantage is being eroded, but is far from collapsing.

The inference market is transitioning from GPU dominance to a diversified landscape where GPUs coexist with various ASICs. If OpenAI's multi-generational chip roadmap progresses smoothly, inference workloads will accelerate their shift from GPUs to ASICs. However, for Nvidia, the barriers in the training market, the ecosystem stickiness of CUDA, and its scale advantages remain difficult to shake in the short term.

The answer to how far Jalapeño can ultimately go may not be revealed until deployment data is available in two years.

Q: When will the Jalapeño chip be officially commercially available?

A: It is currently still in the engineering sample phase. OpenAI expects to release a more detailed technical report in the coming months, and the specific timeline for mass production and deployment has not yet been announced.

Q: Will Jalapeño replace Nvidia GPUs?

A: Not in the short term. OpenAI has made it clear that this is a supplement to computing power rather than a replacement, and training next-generation foundation models still highly relies on Nvidia GPUs.

Q: Can Jalapeño's inference costs really be reduced by 50%?

A: This figure comes from public statements by Broadcom CEO Hock Tan, but it is based on early testing metrics, and the actual performance after final mass production remains to be further verified.

This content was translated using AI and reviewed for clarity. It is for informational purposes only.