Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

71–80 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#71

Post by ahartmetz »

HarHarVeryFunny wrote: post #20
shellcromancer wrote: post #4

Probably obvious but still omitted in the OpenAI post: chips are being made by TSMC [1]. Wasn't sure if Intel got it. 1. https://www.investing.com/news/stock-market-news/openai-unve...

I just read a claim on Twitter that the reason these companies (Google and Amazon as well as OpenAI) are using Broadcom isn't just for design expertise, but because Broadcom have allocation agreements in place with TSMC and the memory manufacturers.

...and because most hardware sales except AI accelerators are down due to RAM prices, Broadcom probably can't otherwise use their allocation at TSMC.

Re: OpenAI unveils its first custom chip, built by Broadcom

#72

Post by ksd482 »

renoir wrote: post #59

Look at the SIZE of that chip. Cerebras stock is down nearly 20% today. Not only is approach overlapping, OpenAI is also Cerebras's only major customer.

That's just the wafer disc. Looks like it was presented to Sam Altman for ceremonial purposes.

The wafer disc is what the CPU gets "printed" on.

Re: OpenAI unveils its first custom chip, built by Broadcom

#73

Post by AxiomaticSpace »

renoir wrote: post #59

Look at the SIZE of that chip. Cerebras stock is down nearly 20% today. Not only is approach overlapping, OpenAI is also Cerebras's only major customer.

I think Cerebras stock going down could also be partly caused by the lock-up period ending today for 200k shares (page 73 of their prospectus) - https://www.sec.gov/Archives/edgar/data/2021728/000162828026...

Re: OpenAI unveils its first custom chip, built by Broadcom

#74

Post by jupr »

tantalor wrote: post #62
renoir wrote: post #59

Look at the SIZE of that chip. Cerebras stock is down nearly 20% today. Not only is approach overlapping, OpenAI is also Cerebras's only major customer.

If you're referring to the big circle of silicon, that's a wafer, generally contains many chips (100-1000s).

That made me chuckle but I guess if you have never seen one I could see how that assumption could be made.

If this photo is real I wonder what can be revealed about the approach they have taken by analyzing the architecture of what we can see.

Re: OpenAI unveils its first custom chip, built by Broadcom

#75

Post by arcanemachiner »

tantalor wrote: post #62
renoir wrote: post #59

Look at the SIZE of that chip. Cerebras stock is down nearly 20% today. Not only is approach overlapping, OpenAI is also Cerebras's only major customer.

If you're referring to the big circle of silicon, that's a wafer, generally contains many chips (100-1000s).

The alt text of the first image describes it as the "Jalapeño inference chip".

As a non-RTFA-er. I'm assuming it's a wafer-scale chip, similar to the ones made by Cerebras.

EDIT: From TechRadar[0]: "The 300mm wafer that both CEOs are holding will generate about 50 to 60 ASICs."

[0] https://www.techradar.com/pro/broadcom-and-openai-debut-jala...

Re: OpenAI unveils its first custom chip, built by Broadcom

#76

Post by FanaHOVA »

sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

NVIDIA already designs most of their chips using AI. Why would you assume it's meaningless marketing?

Re: OpenAI unveils its first custom chip, built by Broadcom

#77

Post by moralestapia »

renoir wrote: post #59

Look at the SIZE of that chip. Cerebras stock is down nearly 20% today. Not only is approach overlapping, OpenAI is also Cerebras's only major customer.

Dang, I just checked and CBRS is in free-fall since the IPO.

Sucks, I think they're a cool company.

OTOH, I was the only person back then pushing hard during my time at KAUST (back in 2019) to buy one of their systems when they were nobody, eventually resulting in a partnership between the two.

Then I joined their online discourse, very few users, I was semi-active there but they didn't care much.

Then I came to Toronto and heard they were opening an office here, tried to get noticed several times but got mostly ignored. I asked about upcoming events several times, anything to get involved, "yeah man, maybe one day". Then they made an event during Toronto Tech Week and didn't even tell me ... idk.

I don't get schadenfreude as I still think they're a cool company.

My point is they put all the eggs in one basket (AI inference) and neglected everything else. They seem to be on shaky ground now ... sad.

Re: OpenAI unveils its first custom chip, built by Broadcom

#78

Post by bigyabai »

phonon wrote: post #56
wmf wrote: post #30

Earlier quoted context omitted.

Memory bandwidth is the bottleneck in the Spark. If you replace the SoC with an optimized ASIC but keep the same 256-bit LPDDR5 the performance will be the same. You can increase performance by using wider memory but that's also more expensive.

M3 Ultra has a 1024 bit memory bus (819 GB/s) and starts at $3,999 (96GB of RAM). It can be done....

The tradeoff is that the M3 Ultra's GPU loses to laptop GPUs in compute benchmarks. All of that bandwidth is wasted idling for token prefill.

For inference workloads, it makes a lot more sense to optimize for prefill/ttft before maxing out memory bandwidth.

Re: OpenAI unveils its first custom chip, built by Broadcom

#80

Post by u1hcw9nx »

bogdiyan wrote: post #57

I am not sure how much of the work is done by OpenAI, or whether it is basically a Broadcom chip specifically built for OpenAI models. It is a necessary step, but building a high-performance chip is not easy. Look at companies like Groq, Amazon, and Google.

Both Google and Amazon also codesign heavily with Broadcomm (Amazon also with Marvell and Alchip)

Broadcomm does stuff like physical design, provides IP blocks, managing manufacturing process with TSMC, packaging and testing. Google and Amazon work with system architecture, performance targets, and requirements but Broadcomm as consultant.

Post reply on HN