Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

231–240 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#231

Post by aurareturn »

sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

Broadcom already has a ton of IP for AI SoCs. I'm guessing the hard parts of this inference chip was already designed by Broadcom and OpenAI simply told Broadcom what it wanted. It's likely very similar to Google's TPU.

  Early testing shows that the first-generation accelerator will deliver performance per watt substantially better than current state-of-the-art
What is substantial here? Vera Rubin is shipping in volume later this year and it is expected to be 10x more power efficient for inference than Blackwell.[0] Even if they're already taped out the chip, getting bugs fixed, getting chips manufactured, getting HBM allocation, getting a rack design, hooking them up together, putting them in a data center will likely take at least another 12 months or likely more. By the time this chip is in data centers in volume, they're likely competing against Vera Rubin Ultra or maybe even Feynman.

Personally, I don't think OpenAI should have invested in this project. It's too early for them. They should have focused on models like Anthropic and win there. When they're profitable, they can take on these projects.

The risk here is very high for OpenAI because AI has a hard cap in energy. If you have a gigawatt, you should only install the best chips. If Nvidia's chips are better, then this is a wasted project and likely wasted billions.

[0]https://developer.nvidia.com/blog/scaling-token-factory-reve...

Re: OpenAI unveils its first custom chip, built by Broadcom

#232

Post by Micrococonut »

gpm wrote: post #118
wongarsu wrote: post #99

Earlier quoted context omitted.

A hard sell right now . The rate of change will slow down

Yes, but with current architectures world knowledge is baked into the weights. We might stop figuring out how to make models better, but the world keeps changing, science is going to keep making progress at understanding the world, etc. This creates a significant minimum rate of change and I'm pretty skeptical that it's worth baking weights into silicon as a result.

I think it would just be an opportunity to sell another chip a few years down the line. If the utility curve flattens out on the performance of models I can see a future where you are buying an up to date chip every few years to upgrade to the latest and greatest, while providing up to date context as part of the user input. Like if I have a programming task and I supply a copy of up-to-date documentation alongside my input, I would think that I could still get good output out of a dated model.

Re: OpenAI unveils its first custom chip, built by Broadcom

#233

Post by aseipp »

IshKebab wrote: post #117
Aurornis wrote: post #95

Earlier quoted context omitted.

The hardware description languages (HDL) used in chip development are like programming languages. The existing models understand them and can do a lot with them. You don’t need to have separate, specialty models designed for this work to use LLMs in chip design workflows. Design verification also involves a lot of traditional programming which benefits from LLMs. So it’s not meaningless at all. You could download som…

> The existing models understand them and can do a lot with them. In my experience they are not especially good at SystemVerilog. There's a lot of knowledge about it that is locked behind paywalls and it's very niche. My guess is the "from scratch" here is quite the exaggeration. Otherwise why did they need Broadcom?

Not having a free toolchain that can actually handle the real language has probably been pretty bad on the downstream public knowledgebase. Hopefully Verilator can eventually close that hole, and there can be more high-quality designs and codebases incorporated into future models. Claude is at least good enough to write SV that triggered a compiler crash or two. :)

Re: OpenAI unveils its first custom chip, built by Broadcom

#234

Post by nine_k »

xdavidliu wrote: post #184
otterdude wrote: post #176

Earlier quoted context omitted.

Not a chip CEO, but I read this article and thought that they're working on some kind of application specific chip only for serving models. Similar to how an FPGA can optimize certain tasks. Given constant weights / biases of a Transformer / DNN you could use pipelining to feed forward calculations through the array one layer at a time. For DNN's with thousands of layers you might see 1:1 speed up per layer channel.…

i don't understand what the second paragraph is saying.

In very crude terms, AFAICT, if you have a bunch of matrix multiplications, but one of matrices (the one with model weights) doesn't change, you can seriously speed up the computation. One thing is that you don't need to re-fetch the elements of the constant matrix, you can keep it near the ALUs. Then you maybe can detect and ignore sparse / empty blocks by marking them once.

IDK how the custom hardware exploits this; would love to hear any ideas!

Re: OpenAI unveils its first custom chip, built by Broadcom

#235

Post by SCUSKU »

sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

My girlfriend works at Broadcom doing chip design, and based on what she's told me they JUST got claude code like 3 weeks ago, so I really doubt this means anything beyond them vibe coding some scripts or something...

Re: OpenAI unveils its first custom chip, built by Broadcom

#236

Post by kccqzy »

nickpinkston wrote: post #86

This is very cool to see - seems like soooo much efficiency waiting to be unlocked at the chip level. What's everyone think of Taalas? They're actually burning the LLM model into the silicon, with some onboard memory for fine-tuning. They claim huge cost / latency wins. Super fast demo live at: https://chatjimmy.ai/ https://taalas.com/ https://www.reddit.com/r/singularity/comments/1r9frzk/taalas...

> seems like soooo much efficiency waiting to be unlocked at the chip level

Well if you are exclusively using GPUs that are general purpose, of course you leave so much efficiency on the table. That’s why Google started making TPUs more than a decade ago. I remember that kerfuffle when Google fired Timnit Gebru when Gebru’s paper used GPUs to calculate the environment impact of LLMs while ignoring the efficiency of TPUs; this basically made Jeff Dean very angry due to that wide efficiency gap.

Re: OpenAI unveils its first custom chip, built by Broadcom

#237

Post by VorpalWay »

knicholes wrote: post #147
Aurornis wrote: post #136

Earlier quoted context omitted.

> I tried making a button using Claude entirely (including the 3D printed enclosure) and it effed up pretty hard with the traces and the header spacing. PCB design and 3D CAD design are different topics. Hardware Description Languages are closer to programming languages than CAD. Look at some Verilog to get an idea - https://en.wikipedia.org/wiki/Verilog

Right. KiCAD for PCB design. Blender for 3D CAD. Oh, are you saying I should have used something other than the KiCAD MCP server for better results?

Designing circuit board and 3D models (even using something like OpenSCAD) is a very spatial process today. You are dealing with coordinates one way or another.

This is very unlike how FPGA and (I assume) ASIC is done. That is more like a traditional programming language but everything happens all at once (no sequence of statements outside tests, if you need that you have to write a state machine yourself). You define logic expressions between signal, add stateful latches, etc. But you never specify the physical layout.

Instead you feed your description to a tool that acts a constraint solver/optimiser that computes the layout for you (this is for FPGAs called synthesising IIRC, it is akin to a compiler). Typically quite slow, even for small circuts like we did at university it took minutes, and for large circuits it might easily days.

Now, this raises the question, what if you design a PCB net list using AI, but then use traditional autorouting and layout? I believe that can also be done, but I have no experience designing PCBs, so I don't know how well it works.

Re: OpenAI unveils its first custom chip, built by Broadcom

#238

Post by jared0x90 »

smetannik wrote: post #172
cpldcpu wrote: post #170

I had Opus 4.5 design an LLM inference engine in verilog, including firmware and automated verification a while ago: https://github.com/cpldcpu/smollm.c It's of course far from optical. But lowering the implementation through the abstraction levels turned out to be extremely powerful.

Can you suggest some tutorials for Verilog and FPGAs in general? I have a spare Tang Nano 9k but I don't feel confident about blindly asking Claude to vibecode me a solution and still would like to have at-least a basic level of understanding.

the hdlbits course is really good imo

Re: OpenAI unveils its first custom chip, built by Broadcom

#239

Post by cptskippy »

aurareturn wrote: post #231
sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

Broadcom already has a ton of IP for AI SoCs. I'm guessing the hard parts of this inference chip was already designed by Broadcom and OpenAI simply told Broadcom what it wanted. It's likely very similar to Google's TPU. Early testing shows that the first-generation accelerator will deliver performance per watt substantially better than current state-of-the-art What is substantial here? Vera Rubin is shipping in volum…

Why do you assume Broadcom has a ton of IP for AI SoCs but hasn't done any of the other work around data center scale deployments?

Re: OpenAI unveils its first custom chip, built by Broadcom

#240

Post by sometimelurker »

zurfer wrote: post #221
cruffle_duffle wrote: post #208

Earlier quoted context omitted.

“ Wafer level faults probably won't matter though - neural nets are resistant to a few missing or wrong weights.” Brain science people “love” traumatic brain injury cases because it can help explore what happens when bits of the “brain wafer” get damaged. We’ve learned a lot from such things. I wonder if people are intentionally “destroying” parts of the model weights to learn more about what happens? Like could you…

This is called mechanistic interpretability. There is lots of fascinating insights already since you can do basically everything down to the neuron or weight level thousands of times. The human brain is many orders of magnitude harder to make sense of.

well its actually called ablation, and its one way to do mech interp. anthriopics got a bunch of work on mech interp here https://transformer-circuits.pub/, like SAEs and NLAs
Post reply on HN