Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

131–140 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#131

Post by admax88qqq »

deweywsu wrote: post #122

With the pace of AI, and with AI helping to pave the way for faster/better AI, I keep wondering if hardware like this will become obsolete well before it has a meaningful ROI. Huge AI models can be run with less resources already through quantization and offloading, but that's just the beginning. One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old D…

> One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old Dell desktop.

But if you have such a breakthrough could you not also apply it and run 200T models on todays datacenters?

Re: OpenAI unveils its first custom chip, built by Broadcom

#132

Post by zabriel_goss »

deweywsu wrote: post #122

With the pace of AI, and with AI helping to pave the way for faster/better AI, I keep wondering if hardware like this will become obsolete well before it has a meaningful ROI. Huge AI models can be run with less resources already through quantization and offloading, but that's just the beginning. One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old D…

I agree with you. Stepping stones are still a part of getting there, if only to be briefly useful.

Re: OpenAI unveils its first custom chip, built by Broadcom

#133

Post by crote »

cmrdporcupine wrote: post #100
Catloafdev wrote: post #96

Earlier quoted context omitted.

It'd be cool to see more of this type of thing, but I have to imagine the ability for it to be updated to a brand-new model as new models come out is limited. If that is the case, it's going to be an extremely hard sell.

I think the model they chose is out of date and hard to sell, but there are plenty of use cases where today's dumb small models are fine. A Qwen 3.5/3.6 or Gemma 3 model on silicon at those speeds would be genuinely world changing even if it's only 1-3B params. Such a model at those speeds will remain extremely useful even over a 5-6 year timespan, I think. If you consider the places you could deploy it -- with no ne…

I wouldn't be surprised if "fast, cheap, dumb" end us being the market for LLMs.

The state-of-the-art models aren't at "can fully replace knowledge worker" levels yet and I doubt they'll get there any time soon, so charging $2000 / month for access isn't going to happen. Right now everyone and their dog is being handed subsidized credits to play with AI, but the actual outcome is rarely good enough to be worth the money they'd need to charge for it. It might very well take another order of magnitude or two to get LLMs to be truly good (if it is even possible at all), and considering how much money is already being pumped into it I just don't see that happening.

On the other hand, the dumb models are more than adequate for simple noncritical tasks, like directing a user to the appropriate FAQ entry, or playing phone decision tree. There's a lot of money in making chatbot assistants actually useful, or in augmenting website search. Turning it into a glorified "language-to-API-call" translator doesn't take a lot of smarts, but as long as it's cheap you can make a killing in volume.

Re: OpenAI unveils its first custom chip, built by Broadcom

#134

Post by smokel »

oceanplexian wrote: post #126
dofm wrote: post #68

Earlier quoted context omitted.

Right. There are two possible meanings and shades in-between: 1) OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) 2) OpenAI designed test/verification models and kernels that could be run on the simulated hardware to test its performance As you and others have said, it's hard to trust when they are happy to write something that could easily only mean the latter…

> OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) Why is that a bold and unlikely claim? Are you saying that AI, which has been proven to cure diseases, solve our hardest math problems, write complex computer code and generate entire generated worlds and HD video from a simple prompt would somehow be like, my bad, I guess I can't design chips?

> solve our hardest math problems

We're not quite there yet :)

https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_m...

Re: OpenAI unveils its first custom chip, built by Broadcom

#135

Post by cess11 »

oceanplexian wrote: post #126
dofm wrote: post #68

Earlier quoted context omitted.

Right. There are two possible meanings and shades in-between: 1) OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) 2) OpenAI designed test/verification models and kernels that could be run on the simulated hardware to test its performance As you and others have said, it's hard to trust when they are happy to write something that could easily only mean the latter…

> OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) Why is that a bold and unlikely claim? Are you saying that AI, which has been proven to cure diseases, solve our hardest math problems, write complex computer code and generate entire generated worlds and HD video from a simple prompt would somehow be like, my bad, I guess I can't design chips?

Because then they'd likely have stfu and outperformed Intel, Nvidia and AMD, or at least one of them.

They're burning more cash than pretty much anyone else and doesn't have anything public that looks like a matching revenue stream so they probably need one very badly.

Re: OpenAI unveils its first custom chip, built by Broadcom

#136

Post by Aurornis »

knicholes wrote: post #129
Aurornis wrote: post #95

Earlier quoted context omitted.

The hardware description languages (HDL) used in chip development are like programming languages. The existing models understand them and can do a lot with them. You don’t need to have separate, specialty models designed for this work to use LLMs in chip design workflows. Design verification also involves a lot of traditional programming which benefits from LLMs. So it’s not meaningless at all. You could download som…

I tried making a button using Claude entirely (including the 3D printed enclosure) and it effed up pretty hard with the traces and the header spacing. The project was a big red arcade button that plays the "ah-my-groin.mp3" when pushed (from Simpsons). It did cool work on saving battery life, and the 3d enclosure was awesome, but yeah, I'm convinced I'd have to do another version or two of the custom chip until it ca…

> I tried making a button using Claude entirely (including the 3D printed enclosure) and it effed up pretty hard with the traces and the header spacing.

PCB design and 3D CAD design are different topics.

Hardware Description Languages are closer to programming languages than CAD. Look at some Verilog to get an idea - https://en.wikipedia.org/wiki/Verilog

Re: OpenAI unveils its first custom chip, built by Broadcom

#137

Post by nok22kon »

hadlock wrote: post #116
martythemaniak wrote: post #102

Earlier quoted context omitted.

In a chatbot, 17k tok/s is a neat but nearly useless showcase. In a coding agent it is a meaningful improvement. In robotics, it could be an absolute revolution. 8B models aren't useful in general, but for specific use cases they can provide an enourmous amount of intelligence - nVidia's Tesla/Waymo competitor is a 7B LLM with a 2B diffusion model, and running that at those speeds could be an order of magnitude cheap…

17K tok/s is approaching realtime motor cortex needs for a robot with ~12 actuators (bipedal humanoid) and an IMU. I don't know how many parameters a motor cortex would need but 8B feels like it is within 2 orders of magnitude.

this is an LLM, not a motor cortex. it will output commands as text (json, ...), so comparing size is not very meaningful, especially considering neurons are highly complex and likely requires thousands of artificial simple neurons (weight+bias)

Re: OpenAI unveils its first custom chip, built by Broadcom

#139

Post by xnx »

sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

AlphaChip is what a chip design with AI is. I'm very suspicious that OpenAI has anything like this or they would be bragging about it.

https://deepmind.google/blog/how-alphachip-transformed-compu...

Re: OpenAI unveils its first custom chip, built by Broadcom

#140

Post by deweywsu »

simonebrunozzi wrote: post #127
deweywsu wrote: post #122

With the pace of AI, and with AI helping to pave the way for faster/better AI, I keep wondering if hardware like this will become obsolete well before it has a meaningful ROI. Huge AI models can be run with less resources already through quantization and offloading, but that's just the beginning. One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old D…

Interesting comment, but the comparison with hard disk drives is probably unfair. The IBM 350 was commercialized 70 years ago; it took 70 years for someone like you to be able to compare that to a multi-TB SSD. Furthermore, nothing says that Moore's Law will necessarily apply to LLMs, for decades to come.

Very true, and all I am basing my comment on is the improvement in speed AI has demonstrated when applied to software development, and inferring it might enable a similar 10X or 100X improvement in both hardware architecture as well LLM structure and/or interface methods. If that speed improvement applies to performance of AI, that could mean the 70 years it took for people to improve storage technology might be able to be compressed to achieve a step change in AI performance in a drastically shorter timeframe.
Post reply on HN