Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

421–430 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#421

Post by kimsey0 »

voidUpdate wrote: post #400
londons_explore wrote: post #192

I wanna see an inference chip where the weights are part of the rom of the chip. There would be 1 multiplier per weight (and since they're constant, the whole thing turns into a bunch of simple adders), and the total pipelined system throughput would be one token per clock cycle. That means you can probably have millions of users simultaneously using a single bit of silicon, with perhaps 500 million tokens per second…

> "Downside is this chip would be huuuuge - a whole wafer." Why don't we have chips like that? If a CPU the size of a postage stamp can do x amount of performance, imagine how much performance you could get if you used an entire wafer of chips running in parallel. Obviously there would be certain use cases, like you couldn't fit an entire wafer in a phone, but still

We do. The Cerebras line of Wafer Scale Engines is exactly an entire wafer of cores running in parallel with fast memory next to each one. It's intended for very high throughput LLM inference. https://www.cerebras.ai/chip

Re: OpenAI unveils its first custom chip, built by Broadcom

#422

Post by cloudengineer94 »

IshKebab wrote: post #117
Aurornis wrote: post #95

Earlier quoted context omitted.

The hardware description languages (HDL) used in chip development are like programming languages. The existing models understand them and can do a lot with them. You don’t need to have separate, specialty models designed for this work to use LLMs in chip design workflows. Design verification also involves a lot of traditional programming which benefits from LLMs. So it’s not meaningless at all. You could download som…

> The existing models understand them and can do a lot with them. In my experience they are not especially good at SystemVerilog. There's a lot of knowledge about it that is locked behind paywalls and it's very niche. My guess is the "from scratch" here is quite the exaggeration. Otherwise why did they need Broadcom?

Broadcom also has direct allocation with TSMC, which is a big win

Re: OpenAI unveils its first custom chip, built by Broadcom

#423

Post by ggcr »

maz1b wrote: post #5

Pretty huge move. Google and their TPUs are looking infinitely more prescient as I think they are on their 7th generation, along with the offshoots it inspired like the LPU and even others, perhaps like Cerebras and their Wafer Scale Engine. However, based off first impressions, it seems like this is meant for inference side, and not training, which is also an interesting choice.

With Reinforcement Learning, inference is very present in post-training stages now too

Re: OpenAI unveils its first custom chip, built by Broadcom

#424

Post by kdheiwns »

NitpickLawyer wrote: post #198
xnx wrote: post #160

Earlier quoted context omitted.

Impossible to know. Could be fake/aspirational roles to impress investors with their grand vision.

Jesus. This is tinfoil hat territory now. Why would they fake something like that? ANY company in this field would try to become free from nvda. Goog has done it already, amazon has their own thing, so it can be done. Not saying they'll 0shot this vertical, but ffs, they don't need to fake anything. They are making an effort, and it would be insane to think they aren't. Might work, might not work, but to even think t…

Asking "why would a company lie?" is probably the funniest thing I've seen all month. Every big company puts out BS nonstop.

Re: OpenAI unveils its first custom chip, built by Broadcom

#425

Post by Someone »

cm2187 wrote: post #417
indigo945 wrote: post #404

Earlier quoted context omitted.

One obvious use case is edge computing, such as in industrial applications that cannot tolerate the risk of a network link or cloud service going down. Even embedded use cases are possible, such as an image classifier model in a security camera.

In fact any application where the task is stable and the model good enough to address that task. As you suggest, industrial applications where a robot must deal with variants of the same repetitive task. Or a military drone which needs to be jamming proof.

> Or a military drone which needs to be jamming proof.

That, if used in war, I would think, would need the ability to be updated frequently. For example, your enemy might find out (say by running tests on hardware they captured from you) that painting some red paint in a particular shape (a smiley might even work) on their hardware prevented your drones from attacking them because it confuses that pattern with the Red Cross logo.

Re: OpenAI unveils its first custom chip, built by Broadcom

#426

Post by cactusplant7374 »

ngomez wrote: post #403
voidUpdate wrote: post #400

Earlier quoted context omitted.

> "Downside is this chip would be huuuuge - a whole wafer." Why don't we have chips like that? If a CPU the size of a postage stamp can do x amount of performance, imagine how much performance you could get if you used an entire wafer of chips running in parallel. Obviously there would be certain use cases, like you couldn't fit an entire wafer in a phone, but still

Using the space of an entire wafer for one chip would result in extremely low manufacturing yields. Even with state of the art silicon cleanrooms, there will still be defects in parts of the output. With CPUs and GPUs, chip makers can disable faulty cores and bin them as lower SKUs to get some yield out of it. But if you're using an entire wafer to embed weights, and a speck of dust causes a printing defect that make…

Is that defect easy to detect?

Re: OpenAI unveils its first custom chip, built by Broadcom

#427

Post by voidUpdate »

ngomez wrote: post #403
voidUpdate wrote: post #400

Earlier quoted context omitted.

> "Downside is this chip would be huuuuge - a whole wafer." Why don't we have chips like that? If a CPU the size of a postage stamp can do x amount of performance, imagine how much performance you could get if you used an entire wafer of chips running in parallel. Obviously there would be certain use cases, like you couldn't fit an entire wafer in a phone, but still

Using the space of an entire wafer for one chip would result in extremely low manufacturing yields. Even with state of the art silicon cleanrooms, there will still be defects in parts of the output. With CPUs and GPUs, chip makers can disable faulty cores and bin them as lower SKUs to get some yield out of it. But if you're using an entire wafer to embed weights, and a speck of dust causes a printing defect that make…

Do failed wafers have to go in the trash, or can you recycle them?

Re: OpenAI unveils its first custom chip, built by Broadcom

#428

Post by u1hcw9nx »

sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

Written with AI is the new written in Rust. Both are nonsensical statements and tell noting about the quality of the software.

Without context, both are warnings about the quality of the developers.

Re: OpenAI unveils its first custom chip, built by Broadcom

#429

Post by gunalx »

indigo945 wrote: post #407
gpm wrote: post #281

Earlier quoted context omitted.

According to your own source > the full-year 2026 [PCD] outlook has been revised to −10.4% year-over-year because > erosion of consumer purchasing power amid regional inflation and currency volatility in many key markets, compounded by memory and storage shortages that are proving more severe than anticipated in the previous forecast cycle. The positive Q1 YoY growth > was largely the product of pull-forward demand,…

> The idea that only nerds care about the cost of things is... absurd. For hardware purchases, laypeople may go about it the other way from what nerds would do: instead of deciding what they need in terms of computing power and memory, and then finding a cheap offer for that, they just decide how much they want to spend, and then buy a device at that price point irrespective of its performance characteristics. If you…

But noe you van only afford the rock bortom low-end device.

Re: OpenAI unveils its first custom chip, built by Broadcom

#430

Post by coder543 »

mdp2021 wrote: post #387
coder543 wrote: post #362

Earlier quoted context omitted.

Yes, I’m focused on the topic at hand that the person I replied to was also talking about. The person I replied to was acting as if Taalas was ancient history. I was pointing out it has only been a few months.

I'd say the original remark was more general («this approach (baking LLMs/weights into silicon directly) [... as if] worked on in secret») - which is salient, because when I investigated weeks ago, I found a large number of attempts to CIM and to general branching from Von Neumann architecture for the purpose of optimizing NNs implementations in HW. Universities are studying, startups are proposing - the «approach» i…

CIM does not bake the weights into silicon. The level of optimization that you can do down to the last transistor when the weights are fixed is on an entirely different level than CIM where you still need general purpose ALUs all over the place.
Post reply on HN