Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

141–150 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#141

Post by nok22kon »

Imustaskforhelp wrote: post #112
martythemaniak wrote: post #102

Earlier quoted context omitted.

In a chatbot, 17k tok/s is a neat but nearly useless showcase. In a coding agent it is a meaningful improvement. In robotics, it could be an absolute revolution. 8B models aren't useful in general, but for specific use cases they can provide an enourmous amount of intelligence - nVidia's Tesla/Waymo competitor is a 7B LLM with a 2B diffusion model, and running that at those speeds could be an order of magnitude cheap…

Could you give me some example how in robotics it can be an absolute revolution? My understanding is that robotics doesn't really rely much on LLM's in the first place but rather other things. Is the thing that you are suggesting that it would ingest all real time data and then reason through it at an incredibly fast speed and then act on it and re-iterate? I might imagine some problems with this though I am not a ro…

LLM are very good at looking at images and reasoning about them. much more than just object recognition/segmentation, they can explain the physics in the image, the intents, plan actions, ...

Re: OpenAI unveils its first custom chip, built by Broadcom

#142

Post by deweywsu »

admax88qqq wrote: post #131
deweywsu wrote: post #122

With the pace of AI, and with AI helping to pave the way for faster/better AI, I keep wondering if hardware like this will become obsolete well before it has a meaningful ROI. Huge AI models can be run with less resources already through quantization and offloading, but that's just the beginning. One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old D…

> One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old Dell desktop. But if you have such a breakthrough could you not also apply it and run 200T models on todays datacenters?

Quite true

Re: OpenAI unveils its first custom chip, built by Broadcom

#143

Post by gdiamos »

deweywsu wrote: post #122

With the pace of AI, and with AI helping to pave the way for faster/better AI, I keep wondering if hardware like this will become obsolete well before it has a meaningful ROI. Huge AI models can be run with less resources already through quantization and offloading, but that's just the beginning. One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old D…

Usually breakthroughs in computing lead to more usage of computing, not less.

Re: OpenAI unveils its first custom chip, built by Broadcom

#144

Post by NitpickLawyer »

Catloafdev wrote: post #96
nickpinkston wrote: post #86

This is very cool to see - seems like soooo much efficiency waiting to be unlocked at the chip level. What's everyone think of Taalas? They're actually burning the LLM model into the silicon, with some onboard memory for fine-tuning. They claim huge cost / latency wins. Super fast demo live at: https://chatjimmy.ai/ https://taalas.com/ https://www.reddit.com/r/singularity/comments/1r9frzk/taalas...

It'd be cool to see more of this type of thing, but I have to imagine the ability for it to be updated to a brand-new model as new models come out is limited. If that is the case, it's going to be an extremely hard sell.

> extremely hard sell.

It really depends on the pricepoint at which they can get a board. If they can do a ~32B model for 1k$ and a size of an external HDD, I'd buy one now, even knowing that it won't be upgradeable / the model remains fixed. The speeds they've shown are a quality of its own, and there's plenty you can do with such a model and faster than instant responses.

Re: OpenAI unveils its first custom chip, built by Broadcom

#145

Post by Chu4eeno »

gpm wrote: post #118
wongarsu wrote: post #99

Earlier quoted context omitted.

A hard sell right now . The rate of change will slow down

Yes, but with current architectures world knowledge is baked into the weights. We might stop figuring out how to make models better, but the world keeps changing, science is going to keep making progress at understanding the world, etc. This creates a significant minimum rate of change and I'm pretty skeptical that it's worth baking weights into silicon as a result.

That's why we have reasoning/CoT LLMs that can use tools to get updated information.

Re: OpenAI unveils its first custom chip, built by Broadcom

#146

Post by dofm »

Aurornis wrote: post #98
dofm wrote: post #68

Earlier quoted context omitted.

Right. There are two possible meanings and shades in-between: 1) OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) 2) OpenAI designed test/verification models and kernels that could be run on the simulated hardware to test its performance As you and others have said, it's hard to trust when they are happy to write something that could easily only mean the latter…

> 1) OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) Chip design languages (HDLs like Verilog or VHDL) are well understood by LLMs. They don’t need specialty tools to use GPT-5.5 or other LLMs with them. You could even try it yourself with open source chip design tooling if you wanted to see it.

Yes, obviously. But do we think LLMs without access to proprietary information do a better job with them than Broadcom's human experts or existing proprietary tools at this level of operations?

It is still a bold claim and it still needs evidence.

We would obviously get a bit more of the evidence if it were to be more useful for the upcoming IPO than this rather open-ended, reinterpretable phrasing.

Re: OpenAI unveils its first custom chip, built by Broadcom

#147

Post by knicholes »

Aurornis wrote: post #136
knicholes wrote: post #129

Earlier quoted context omitted.

I tried making a button using Claude entirely (including the 3D printed enclosure) and it effed up pretty hard with the traces and the header spacing. The project was a big red arcade button that plays the "ah-my-groin.mp3" when pushed (from Simpsons). It did cool work on saving battery life, and the 3d enclosure was awesome, but yeah, I'm convinced I'd have to do another version or two of the custom chip until it ca…

> I tried making a button using Claude entirely (including the 3D printed enclosure) and it effed up pretty hard with the traces and the header spacing. PCB design and 3D CAD design are different topics. Hardware Description Languages are closer to programming languages than CAD. Look at some Verilog to get an idea - https://en.wikipedia.org/wiki/Verilog

Right. KiCAD for PCB design. Blender for 3D CAD. Oh, are you saying I should have used something other than the KiCAD MCP server for better results?

Re: OpenAI unveils its first custom chip, built by Broadcom

#148

Post by hyhatqtv »

deweywsu wrote: post #122

With the pace of AI, and with AI helping to pave the way for faster/better AI, I keep wondering if hardware like this will become obsolete well before it has a meaningful ROI. Huge AI models can be run with less resources already through quantization and offloading, but that's just the beginning. One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old D…

Looking at the development of memory bandwidth, capacity and prices over the last 10 years there is little indication that’s likely.

Re: OpenAI unveils its first custom chip, built by Broadcom

#149

Post by skyberrys »

The new chip sounds like it's vustom made to accelerate a few specific models they really need to run fast. The advantage is it's truly and ASIC, not a xPU. There are several new startups targeting EDA tooling automation, Chip Agents is the biggest one I can think of but their are smaller players too, Silimate is one I recall. These companies are focusing on building fast AI powered tools to speed up the tape out cycle.

Re: OpenAI unveils its first custom chip, built by Broadcom

#150

Post by dofm »

oceanplexian wrote: post #126
dofm wrote: post #68

Earlier quoted context omitted.

Right. There are two possible meanings and shades in-between: 1) OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) 2) OpenAI designed test/verification models and kernels that could be run on the simulated hardware to test its performance As you and others have said, it's hard to trust when they are happy to write something that could easily only mean the latter…

> OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) Why is that a bold and unlikely claim? Are you saying that AI, which has been proven to cure diseases, solve our hardest math problems, write complex computer code and generate entire generated worlds and HD video from a simple prompt would somehow be like, my bad, I guess I can't design chips?

> Why is that a bold and unlikely claim?

Because they could have offered even slightly more evidence.

Post reply on HN