Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

481–490 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#481

Post by HarHarVeryFunny »

davidpapermill wrote: post #468
HarHarVeryFunny wrote: post #180

Earlier quoted context omitted.

I would assume they've already made as big a deal of it as they can without outright lying too much. Read the rest of the press release. FWIW, Google is now on their 8th generation TPU, having put out the last 4 generations on a 1-year cadence.

> Google is now on their 8th generation TPU Remarkable that the TPU pre-dates the attention paper. Was a solid bet on energy efficient dense matrix multiplication and has stood the test of time.

Yes, they'd been using machine learning internally since c.2011 with their DistBelief system, which was then replaced with TensorFlow in 2015.

Re: OpenAI unveils its first custom chip, built by Broadcom

#482

Post by ngomez »

voidUpdate wrote: post #427
ngomez wrote: post #403

Earlier quoted context omitted.

Using the space of an entire wafer for one chip would result in extremely low manufacturing yields. Even with state of the art silicon cleanrooms, there will still be defects in parts of the output. With CPUs and GPUs, chip makers can disable faulty cores and bin them as lower SKUs to get some yield out of it. But if you're using an entire wafer to embed weights, and a speck of dust causes a printing defect that make…

Do failed wafers have to go in the trash, or can you recycle them?

You can grind some of the raw silicon out of a finished wafer but I don't think it'd be suitable to use in another batch of product. So instead of having the weights on the wafer like OOP was suggesting, hardware inference has been trending toward having a wafer of lots and lots of smaller cores with fast SRAM.

Re: OpenAI unveils its first custom chip, built by Broadcom

#484

Post by guywithahat »

groundzeros2015 wrote: post #449
guywithahat wrote: post #327

Earlier quoted context omitted.

Maybe, but they’re also a massive company. At some point Google stopped being a startup and become a massive company with margins to look after

After they were wildly profitable.

OpenAI had around 8 billion in revenue in 2025, and are estimated to have 25-30 billion in revenue this year. I assume they're not profitable but this is a massive company with margins to look after, I don't think we should treat them like a startup conceptually anymore

Re: OpenAI unveils its first custom chip, built by Broadcom

#485

Post by jwHollister »

otterley wrote: post #439
fragmede wrote: post #396

Earlier quoted context omitted.

https://www.cerebras.ai/ is exactly that! Holy shit it's fast.

Cerebras is not that. Cerebras isn’t tied to a particular model like Taalas is. The latter is even faster than Cerebras.

Yeah at a certain point if inference is fast and cheap, do I care if it's the smartest? I can iterate in a tighter loop.

Re: OpenAI unveils its first custom chip, built by Broadcom

#486

Post by DANmode »

TeMPOraL wrote: post #459
DANmode wrote: post #445

Earlier quoted context omitted.

It’s fission , not fusion: https://www.esgdive.com/news/meta-inks-nuclear-deals-terrapo... …and if they do all of this, it’ll be closer to $20B than 10!

If all it took to get viable fusion power was a FAANG CEO with $10B to burn, I'd be first to petition for it to happen, and even throw whatever money I can spare onto that pyre.

Viable, of course, meaning safe? lol

Re: OpenAI unveils its first custom chip, built by Broadcom

#487

Post by TeMPOraL »

DANmode wrote: post #486
TeMPOraL wrote: post #459

Earlier quoted context omitted.

If all it took to get viable fusion power was a FAANG CEO with $10B to burn, I'd be first to petition for it to happen, and even throw whatever money I can spare onto that pyre.

Viable, of course, meaning safe ? lol

No, of course not. We can worry about making it safe after we have any form of it working first.

(Also yes, I'm serious.)

Re: OpenAI unveils its first custom chip, built by Broadcom

#488

Post by indigo945 »

gunalx wrote: post #429
indigo945 wrote: post #407

Earlier quoted context omitted.

> The idea that only nerds care about the cost of things is... absurd. For hardware purchases, laypeople may go about it the other way from what nerds would do: instead of deciding what they need in terms of computing power and memory, and then finding a cheap offer for that, they just decide how much they want to spend, and then buy a device at that price point irrespective of its performance characteristics. If you…

But noe you van only afford the rock bortom low-end device.

The point is they can't tell.

Re: OpenAI unveils its first custom chip, built by Broadcom

#489

Post by fuck_google »

gdiamos wrote: post #143
deweywsu wrote: post #122

With the pace of AI, and with AI helping to pave the way for faster/better AI, I keep wondering if hardware like this will become obsolete well before it has a meaningful ROI. Huge AI models can be run with less resources already through quantization and offloading, but that's just the beginning. One day, maybe not far from now, a breakthrough will allow huge LLMs (say 200B in size) to run well on an old 5 year old D…

Usually breakthroughs in computing lead to more usage of computing, not less.

[dead]

Re: OpenAI unveils its first custom chip, built by Broadcom

#490

Post by mdp2021 »

coder543 wrote: post #466
mdp2021 wrote: post #465

Earlier quoted context omitted.

> CIM does not bake the weights into silicon If that were the extent of the terms, then what could we call "baking the weights into silicon"? Setting parts of the circuits to determined values for multiplication is is like printing a Read-Only Memory. (And you compute at it: Compute In Memory.) > CIM where you still need general purpose ALUs all over the place If that were so, then why do taxonomists present analogue…

EDIT: It's just not even worth arguing this point, so deleting my original, much longer comment. Abstract taxonomies can claim that Taalas is CIM, but this entirely and utterly misses the point, and misses what makes Taalas' approach special. If you told a room full of chip architects to go build "CIM for AI", they would not build a Taalas-like totally specialized chip, therefore it is not sufficient, and just muddie…

> It's just not even worth arguing this point

Pretty simply: the von Neumann architecture separates data storage and processing; we are now researching the alternatives - in all their forms, whatever allows to make NNs efficient; those alternatives basically focus on reducing the distance between stored data and its processing; some simply call stored data "memory"; we had the concept of "Compute in Memory" available; now some are calling "CIM" all of the relevant solutions that allow processing in the vicinity of the storage.

It is not the traditional CIM with the ALUs. Too bad. We can easily get the difference.

Post reply on HN