Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

111–120 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#111

Post by LoganDark »

mikewarot wrote: post #55
some-guy wrote: post #44

Earlier quoted context omitted.

I have been eyeing what Taalas is doing [1] by making pure hardware models. The speed is absurd. [1] https://taalas.com/products/

They talk about products, but they don't sell the hardware, thus they don't really have a product, just a service. I know, it's nick picking, but when people can just reach in and take services away, like Fable/Mythos, hardware is the only thing worth buying.

I'm sure they'll have a product for you if you have millions to invest in a partnership with them.

Re: OpenAI unveils its first custom chip, built by Broadcom

#112

Post by Imustaskforhelp »

martythemaniak wrote: post #102
nickpinkston wrote: post #86

This is very cool to see - seems like soooo much efficiency waiting to be unlocked at the chip level. What's everyone think of Taalas? They're actually burning the LLM model into the silicon, with some onboard memory for fine-tuning. They claim huge cost / latency wins. Super fast demo live at: https://chatjimmy.ai/ https://taalas.com/ https://www.reddit.com/r/singularity/comments/1r9frzk/taalas...

In a chatbot, 17k tok/s is a neat but nearly useless showcase. In a coding agent it is a meaningful improvement. In robotics, it could be an absolute revolution. 8B models aren't useful in general, but for specific use cases they can provide an enourmous amount of intelligence - nVidia's Tesla/Waymo competitor is a 7B LLM with a 2B diffusion model, and running that at those speeds could be an order of magnitude cheap…

Could you give me some example how in robotics it can be an absolute revolution?

My understanding is that robotics doesn't really rely much on LLM's in the first place but rather other things.

Is the thing that you are suggesting that it would ingest all real time data and then reason through it at an incredibly fast speed and then act on it and re-iterate? I might imagine some problems with this though I am not a robotics engineer and perhaps someone who deeply understands this topic can give more information.

Re: OpenAI unveils its first custom chip, built by Broadcom

#113

Post by fl4regun »

lovasoa wrote: post #81
dofm wrote: post #68

Earlier quoted context omitted.

Right. There are two possible meanings and shades in-between: 1) OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) 2) OpenAI designed test/verification models and kernels that could be run on the simulated hardware to test its performance As you and others have said, it's hard to trust when they are happy to write something that could easily only mean the latter…

3) The engineers working on the chip used ChatGPT from time to time.

at the hardware company I work at, people are now using claude code and developing skills for it to do basic stuff like triage or do initial debug on failing tests, search for potential causes in RTL, generate skeleton documentation for designs etc

Re: OpenAI unveils its first custom chip, built by Broadcom

#114

Post by reducesuffering »

lovasoa wrote: post #81
dofm wrote: post #68

Earlier quoted context omitted.

Right. There are two possible meanings and shades in-between: 1) OpenAI genuinely have AI technologies that can improve chip design (bold, unlikely claim, needs evidence) 2) OpenAI designed test/verification models and kernels that could be run on the simulated hardware to test its performance As you and others have said, it's hard to trust when they are happy to write something that could easily only mean the latter…

3) The engineers working on the chip used ChatGPT from time to time.

From time to time? Lol you must realize, frontier lab eng are using Codex/Claude-Code 99% in loops, on models the public doesn't have access to. Why? Because it works. Just a matter of time before humans are out of the loop and what comes next is a black hole

"The future is here, it's just not evenly distributed"

Re: OpenAI unveils its first custom chip, built by Broadcom

#115

Post by seydor »

sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

realistically, how hard are AI accelerators to design?

Re: OpenAI unveils its first custom chip, built by Broadcom

#116

Post by hadlock »

martythemaniak wrote: post #102
nickpinkston wrote: post #86

This is very cool to see - seems like soooo much efficiency waiting to be unlocked at the chip level. What's everyone think of Taalas? They're actually burning the LLM model into the silicon, with some onboard memory for fine-tuning. They claim huge cost / latency wins. Super fast demo live at: https://chatjimmy.ai/ https://taalas.com/ https://www.reddit.com/r/singularity/comments/1r9frzk/taalas...

In a chatbot, 17k tok/s is a neat but nearly useless showcase. In a coding agent it is a meaningful improvement. In robotics, it could be an absolute revolution. 8B models aren't useful in general, but for specific use cases they can provide an enourmous amount of intelligence - nVidia's Tesla/Waymo competitor is a 7B LLM with a 2B diffusion model, and running that at those speeds could be an order of magnitude cheap…

17K tok/s is approaching realtime motor cortex needs for a robot with ~12 actuators (bipedal humanoid) and an IMU. I don't know how many parameters a motor cortex would need but 8B feels like it is within 2 orders of magnitude.

Re: OpenAI unveils its first custom chip, built by Broadcom

#117

Post by IshKebab »

Aurornis wrote: post #95
sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

The hardware description languages (HDL) used in chip development are like programming languages. The existing models understand them and can do a lot with them. You don’t need to have separate, specialty models designed for this work to use LLMs in chip design workflows. Design verification also involves a lot of traditional programming which benefits from LLMs. So it’s not meaningless at all. You could download som…

> The existing models understand them and can do a lot with them.

In my experience they are not especially good at SystemVerilog. There's a lot of knowledge about it that is locked behind paywalls and it's very niche.

My guess is the "from scratch" here is quite the exaggeration. Otherwise why did they need Broadcom?

Re: OpenAI unveils its first custom chip, built by Broadcom

#118

Post by gpm »

wongarsu wrote: post #99
Catloafdev wrote: post #96

Earlier quoted context omitted.

It'd be cool to see more of this type of thing, but I have to imagine the ability for it to be updated to a brand-new model as new models come out is limited. If that is the case, it's going to be an extremely hard sell.

A hard sell right now . The rate of change will slow down

Yes, but with current architectures world knowledge is baked into the weights. We might stop figuring out how to make models better, but the world keeps changing, science is going to keep making progress at understanding the world, etc. This creates a significant minimum rate of change and I'm pretty skeptical that it's worth baking weights into silicon as a result.

Re: OpenAI unveils its first custom chip, built by Broadcom

#119

Post by fl4regun »

moralestapia wrote: post #77
renoir wrote: post #59

Look at the SIZE of that chip. Cerebras stock is down nearly 20% today. Not only is approach overlapping, OpenAI is also Cerebras's only major customer.

Dang, I just checked and CBRS is in free-fall since the IPO. Sucks, I think they're a cool company. OTOH, I was the only person back then pushing hard during my time at KAUST (back in 2019) to buy one of their systems when they were nobody, eventually resulting in a partnership between the two. Then I joined their online discourse, very few users, I was semi-active there but they didn't care much. Then I came to Toro…

my friend briefly worked there and then got hit by layoffs, as a result, I am enjoying the schadenfreude.

Re: OpenAI unveils its first custom chip, built by Broadcom

#120

Post by whynotminot »

IshKebab wrote: post #117
Aurornis wrote: post #95

Earlier quoted context omitted.

The hardware description languages (HDL) used in chip development are like programming languages. The existing models understand them and can do a lot with them. You don’t need to have separate, specialty models designed for this work to use LLMs in chip design workflows. Design verification also involves a lot of traditional programming which benefits from LLMs. So it’s not meaningless at all. You could download som…

> The existing models understand them and can do a lot with them. In my experience they are not especially good at SystemVerilog. There's a lot of knowledge about it that is locked behind paywalls and it's very niche. My guess is the "from scratch" here is quite the exaggeration. Otherwise why did they need Broadcom?

Doesn’t Broadcom bring a lot more to bear here than just Verilog? Including relationships with the actual fabricators.
Post reply on HN