Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

431–440 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#431

Post by krick »

groundzeros2015 wrote: post #278

This is starting to sound like startup scope creep. Instead of making the AI model it’s now custom silicon, web browsers, and consumer electronics?

But there never really was a moat in LLM?.. I mean, I don't know where you stand, but my perception is that we all kinda knew that the whole time since 2017, and really knew that since DeepSeek. What they really care about is:

1. Customer acquisition.

2. Cheap(er) electricity/hardware.

So it's really surprising to me that them making their own chip surprises anyone at all. The electricity thing is already kinda being taken care of by earlier strategic alliances with some other evil people, the chip is a natural next step.

Re: OpenAI unveils its first custom chip, built by Broadcom

#432

Post by epolanski »

lugu wrote: post #302
epolanski wrote: post #286

Earlier quoted context omitted.

Operational costs far outweight hardware cost.

Do they? Genuinely ansking.

For a small cluster no, but at major data center level yes. Which is why they building data centers bigger than stadiums.

If you spend 10B on a data center, roughly 30% of that price is going to hardware, so roughly $ 3B.

So for two data centers you're spending 20B.

Now, assume there's hardware that performs twice as fast at same energy (watt/token), even if it costed you twice you're saving 7B because you don't need the second data center.

You get the same output of $ 20 B out of a $ 13 B initial investment, but you're also halving operational costs: less staff, less lawyers, etc, etc.

This is the reason why Nvidia is making gargantuan margins: hyper scalers don't really care about hardware cost, if they can get double the output and save themselves 30-40% of total costs and 50% of the headaches they will keep buying at twice the price gen over gen.

Re: OpenAI unveils its first custom chip, built by Broadcom

#434

Post by amelius »

kccqzy wrote: post #299
mschuster91 wrote: post #276

Earlier quoted context omitted.

> Well Google has reduced reliance on Broadcom already. They found a new hardware partner, MediaTek Oh dear god. I'm actually feeling sorry for Google at that point. Good luck, you'll need it...

My hunch is that this change is driven by bean counters.

Who says Google isn't doing its own designs mostly?

Re: OpenAI unveils its first custom chip, built by Broadcom

#435

Post by IX-103 »

SwellJoe wrote: post #413
kmacdough wrote: post #411

Earlier quoted context omitted.

With a striking lack of numbers, I'm not confident. I my experience, everything underspecified in a marketing release is unflattering. They're also not a chip designing company, but they're probably trying to keep up on the eyes of investors. As the article mentions, several of their competitors are chip designers and already have working procuction inference chips.

When you have a few billion dollars you can hire chip people and partner with a chip company. That's not to say I expect they'll ship something competitive with Google's custom AI hardware on the first go, since Google has been at it for quite a while, but there's very few technical problems large sums of money won't solve.

Yeah, I'm not sure how competitive it is without any specs. Just from it being "inference only" that puts it on the same level as Google's 2015 TPUv1.

Re: OpenAI unveils its first custom chip, built by Broadcom

#436

Post by runeks »

Catloafdev wrote: post #96
nickpinkston wrote: post #86

This is very cool to see - seems like soooo much efficiency waiting to be unlocked at the chip level. What's everyone think of Taalas? They're actually burning the LLM model into the silicon, with some onboard memory for fine-tuning. They claim huge cost / latency wins. Super fast demo live at: https://chatjimmy.ai/ https://taalas.com/ https://www.reddit.com/r/singularity/comments/1r9frzk/taalas...

It'd be cool to see more of this type of thing, but I have to imagine the ability for it to be updated to a brand-new model as new models come out is limited. If that is the case, it's going to be an extremely hard sell.

If performance per watt is 100x better than GPUs (as GP link claims) then I don't think it's a hard sell at all. That's actually a cost reduction that matters.

Re: OpenAI unveils its first custom chip, built by Broadcom

#437

Post by runeks »

agazso wrote: post #393
1e1a wrote: post #256

Earlier quoted context omitted.

you can try it out here: https://chatjimmy.ai/

It's indeed super fast, but the output is complete BS hallucination. Not sure what's the value of this.

It's a proof of concept that it's possible to etch a neural net into a chip and get massive performance (and efficiency) boost

Re: OpenAI unveils its first custom chip, built by Broadcom

#438

Post by einpoklum »

Aurornis wrote: post #95
sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

The hardware description languages (HDL) used in chip development are like programming languages. The existing models understand them and can do a lot with them. You don’t need to have separate, specialty models designed for this work to use LLMs in chip design workflows. Design verification also involves a lot of traditional programming which benefits from LLMs. So it’s not meaningless at all. You could download som…

> The existing models understand them

No they don't.

Re: OpenAI unveils its first custom chip, built by Broadcom

#439

Post by otterley »

fragmede wrote: post #396
simondotau wrote: post #388

Earlier quoted context omitted.

Presumably at some point the rapid progress of models will plateau, at least insofar as a model could be frozen in time and remain economically useful for the expected life of hardware. Especially if it comes with compelling benefits e.g. dramatically lower latency and/or dramatically higher performance per watt. If you can build chips that could run one specific LLM 100x faster than anything else, it would have a us…

https://www.cerebras.ai/ is exactly that! Holy shit it's fast.

Cerebras is not that. Cerebras isn’t tied to a particular model like Taalas is. The latter is even faster than Cerebras.
Post reply on HN