Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

381–390 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#381

Post by AceJohnny2 »

freakynit wrote: post #368
londons_explore wrote: post #192

I wanna see an inference chip where the weights are part of the rom of the chip. There would be 1 multiplier per weight (and since they're constant, the whole thing turns into a bunch of simple adders), and the total pipelined system throughput would be one token per clock cycle. That means you can probably have millions of users simultaneously using a single bit of silicon, with perhaps 500 million tokens per second…

This may be extreme, or, completely stupid, but, why are we not using genetics to "grow" chips in a chemical soup yet? Similar to Verilog/VHDL, don't we have some similar language to express circuits using gene sequences?

Are referencing the 1998 short story "Taklamakan" by Bruce Sterling?

Re: OpenAI unveils its first custom chip, built by Broadcom

#382

Post by digitaltrees »

arcanemachiner wrote: post #108
mikewarot wrote: post #55

Earlier quoted context omitted.

They talk about products, but they don't sell the hardware, thus they don't really have a product, just a service. I know, it's nick picking, but when people can just reach in and take services away, like Fable/Mythos, hardware is the only thing worth buying.

"Nitpicking"

Underrated. Hits on multiple levels

Re: OpenAI unveils its first custom chip, built by Broadcom

#383

Post by whalee »

freakynit wrote: post #368
londons_explore wrote: post #192

I wanna see an inference chip where the weights are part of the rom of the chip. There would be 1 multiplier per weight (and since they're constant, the whole thing turns into a bunch of simple adders), and the total pipelined system throughput would be one token per clock cycle. That means you can probably have millions of users simultaneously using a single bit of silicon, with perhaps 500 million tokens per second…

This may be extreme, or, completely stupid, but, why are we not using genetics to "grow" chips in a chemical soup yet? Similar to Verilog/VHDL, don't we have some similar language to express circuits using gene sequences?

We lack robust frameworks for 'forward engineering' stochastic thermodynamic computation over molecular free-energy landscapes (which is basically what a "chemical soup" is doing) like we do for analog/optical/digital computing. This is why, as a field, medicine is so heavily empirical and reverse engineering oriented.

Re: OpenAI unveils its first custom chip, built by Broadcom

#384

Post by digitaltrees »

zwarag wrote: post #43
digitaltrees wrote: post #40

We’ve entered the “if you care about software, build hardware” phase of AI

What are the other phases. Or what are you referring to in general?

Mainframe punch card -> PC floppy disk -> cloud SaaS -> AI --> return to the land agrarian

Re: OpenAI unveils its first custom chip, built by Broadcom

#386

Post by otterley »

jwHollister wrote: post #380
lsaferite wrote: post #369

Earlier quoted context omitted.

https://taalas.com/

wow if they can get something like this working, what happens to all this infrastructure? Hyperscalers have to be assuming the lifespan of that stuff wrong considering the next gen will be 1000x more efficient.

The question isn’t whether it works (it does); the question is whether there are buyers for hardware that is obsolete the day it ships. Models evolve much more quickly than hardware can keep up.

Re: OpenAI unveils its first custom chip, built by Broadcom

#387

Post by mdp2021 »

coder543 wrote: post #362
mdp2021 wrote: post #361

Earlier quoted context omitted.

You are focusing on Taalas, but (specific) analogue computing, electronic NNs, compute-in-memory etc. - the field including the contextual approach - backdate to Rosenblatt.

Yes, I’m focused on the topic at hand that the person I replied to was also talking about. The person I replied to was acting as if Taalas was ancient history. I was pointing out it has only been a few months.

I'd say the original remark was more general («this approach (baking LLMs/weights into silicon directly) [... as if] worked on in secret») - which is salient, because when I investigated weeks ago, I found a large number of attempts to CIM and to general branching from Von Neumann architecture for the purpose of optimizing NNs implementations in HW.

Universities are studying, startups are proposing - the «approach» is under the big headlines level but quite lively. Not just Taalas, not just their way - which remains remarkable in the scene as the HW is achieved, working, online, available... and amazing.

Re: OpenAI unveils its first custom chip, built by Broadcom

#388

Post by simondotau »

otterley wrote: post #386
jwHollister wrote: post #380

Earlier quoted context omitted.

wow if they can get something like this working, what happens to all this infrastructure? Hyperscalers have to be assuming the lifespan of that stuff wrong considering the next gen will be 1000x more efficient.

The question isn’t whether it works (it does); the question is whether there are buyers for hardware that is obsolete the day it ships. Models evolve much more quickly than hardware can keep up.

Presumably at some point the rapid progress of models will plateau, at least insofar as a model could be frozen in time and remain economically useful for the expected life of hardware. Especially if it comes with compelling benefits e.g. dramatically lower latency and/or dramatically higher performance per watt.

If you can build chips that could run one specific LLM 100x faster than anything else, it would have a use case that nothing else could match.

Re: OpenAI unveils its first custom chip, built by Broadcom

#389

Post by froh »

the_sleaze_ wrote: post #349
Meetvelde wrote: post #344

Earlier quoted context omitted.

that's so fast it feels fake

13,789 tok/s Well I've gotten one of those "holy fuck this is the future" deeply unsettled anxious feelings in my gut again. It's been a week or 2, it was time.

i only found one discussion of the tech here on HN

https://news.ycombinator.com/item?id=47103661

Re: OpenAI unveils its first custom chip, built by Broadcom

#390

Post by mdp2021 »

mdp2021 wrote: post #325
cruffle_duffle wrote: post #208

Earlier quoted context omitted.

“ Wafer level faults probably won't matter though - neural nets are resistant to a few missing or wrong weights.” Brain science people “love” traumatic brain injury cases because it can help explore what happens when bits of the “brain wafer” get damaged. We’ve learned a lot from such things. I wonder if people are intentionally “destroying” parts of the model weights to learn more about what happens? Like could you…

Of course tampering with chunks or nodes in the NNs is a way to study the "spawned" (through gradient descent etc.) configuration and "reverse-engineer the black box" to get "AI transparency". Anthropic published an important work around one year and a half ago.

> Anthropic published an important work around one year and a half ago

> #Tracing the thoughts of a large language model#

https://www.anthropic.com/research/tracing-thoughts-language...

https://news.ycombinator.com/item?id=43495617 (27 March 2025)

Post reply on HN