OpenAI and Broadcom Unveil AI Chip to Run Models Faster, Cheaper

Article content

(Bloomberg) — OpenAI unveiled its first custom artificial intelligence chip developed in partnership with Broadcom Inc., part of a bid by the ChatGPT maker to gain an edge by tailoring hardware to better run its AI products. 

Article content

The companies said Wednesday that OpenAI has received the first samples of the chip, called Jalapeno, and is testing how the silicon handles running AI workloads. So far, the accelerator is showing cost savings of roughly 50% compared with typical AI graphics processing units, Broadcom Chief Executive Officer Hock Tan said in an interview. 

Article content

Article content

Article content

The finalized chips will be integrated into large data centers from OpenAI-backer Microsoft Corp. and other partners starting later this year. Tan expects OpenAI and Broadcom should be able to exceed his prior forecast for deploying 1.3 gigawatts worth of chips next year. “We like to think we can do better because there is a lot of demand,” he said. 

Article content

Article content

Though OpenAI continues to rely heavily on chips from market leader Nvidia Corp., the startup has worked to broaden its mix of suppliers to meet surging demand for its AI services. OpenAI has struck multibillion-dollar deals with chipmakers such as Advanced Micro Devices Inc. and Cerebras Systems Inc. 

Article content

In October, OpenAI announced that it would partner with Broadcom to design accelerators tailored to work best with its artificial intelligence models. OpenAI plans to spend tens of billions of dollars on Broadcom chips, Bloomberg News has reported, adding to the unprofitable startup’s massive outlays on physical infrastructure to support AI. 

Article content

OpenAI’s head of hardware, Richard Ho, declined to say how the company will pay for the chips, noting that financing arrangements would be finalized when the full order of processors is complete. Tan declined to comment, but reiterated that Broadcom has set up a chip financing vehicle with Apollo Global Management Inc. and Blackstone Inc. that will also aid OpenAI. 

Article content

Article content

OpenAI raised $122 billion in funding earlier this year to bolster its costly push for chips, data centers and talent. The company has also entered into financing deals with suppliers, including Nvidia and AMD, that have been criticized for being circular in nature. 

Article content

The new chips were developed from scratch in record time, according to OpenAI and Broadcom. OpenAI said it is still assessing the test versions but early data shows Jalapeno is capable of delivering performance per watt of energy that’s “substantially better than current state-of-the-art.” 

Article content

The chip was designed for large-language models (LLMs) and guided by the artificial intelligence startup’s approach for inference, or running AI tasks, Ho said. Still, he said, “it is a very general purpose device” that “will address future LLM innovations.” 

Article content

OpenAI wants to keep its options open as to whether it would let other model makers use its chips, Ho said. 

Article content

Within the tech industry, some AI chips are more powerful for doing many tasks at once while others are focused on providing a single answer to a user as fast as possible. OpenAI said its goal is to combine both — the power of leading AI chips with speed that’s closer to specialized hardware for running models faster.