Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

361–370 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#361

Post by mdp2021 »

coder543 wrote: post #358
ipdashc wrote: post #34

Earlier quoted context omitted.

> 17k tps taalas chip It's odd to me that I haven't heard anything about this approach (baking LLMs/weights into silicon directly) since. It seems almost common-sense that we're going to end up there eventually . And it feels like that point is drawing ever closer now that model capabilities, if not quite plateauing out, are at least getting to a "good enough" point for a LOT of use cases. I wonder if it's being work…

> It's odd to me that I haven't heard anything about this approach since. It has only been four months since they unveiled their first prototype. I don't understand your confusion. Chip development does not happen overnight...? Their initial blog post laid out a roadmap, so theoretically they should have another thing to demonstrate this summer.

You are focusing on Taalas, but (specific) analogue computing, electronic NNs, compute-in-memory etc. - the field including the contextual approach - backdate to Rosenblatt.

Re: OpenAI unveils its first custom chip, built by Broadcom

#362

Post by coder543 »

mdp2021 wrote: post #361
coder543 wrote: post #358

Earlier quoted context omitted.

> It's odd to me that I haven't heard anything about this approach since. It has only been four months since they unveiled their first prototype. I don't understand your confusion. Chip development does not happen overnight...? Their initial blog post laid out a roadmap, so theoretically they should have another thing to demonstrate this summer.

You are focusing on Taalas, but (specific) analogue computing, electronic NNs, compute-in-memory etc. - the field including the contextual approach - backdate to Rosenblatt.

Yes, I’m focused on the topic at hand that the person I replied to was also talking about.

The person I replied to was acting as if Taalas was ancient history. I was pointing out it has only been a few months.

Re: OpenAI unveils its first custom chip, built by Broadcom

#363

Post by typ »

nickpinkston wrote: post #86

This is very cool to see - seems like soooo much efficiency waiting to be unlocked at the chip level. What's everyone think of Taalas? They're actually burning the LLM model into the silicon, with some onboard memory for fine-tuning. They claim huge cost / latency wins. Super fast demo live at: https://chatjimmy.ai/ https://taalas.com/ https://www.reddit.com/r/singularity/comments/1r9frzk/taalas...

Low latency is nice. But it would be more interesting if they could demonstrate the efficiency of energy consumption.

Re: OpenAI unveils its first custom chip, built by Broadcom

#364

Post by signatoremo »

Schiendelman wrote: post #273
signatoremo wrote: post #262

I haven't seen this discussed here: So far, the accelerator is showing cost savings of roughly 50% compared with typical AI graphics processing units, Broadcom Chief Executive Officer Hock Tan said in an interview. - [0] 50% cost saving. The picture changes so quickly, there are still a lot of low hanging fruits, that I find any discussion about whether a vendor has moats, or if they can recoup investment, is moot an…

"Typical" is doing a lot of work there. That could mean much older chips than Nvidia is currently selling.

"Typical" usually means typical, i.e. median. Also they are claiming cost saving, not performance. The saving would even be more impressive if much older chips are less efficient than the newer ones -- costing more to run.

Re: OpenAI unveils its first custom chip, built by Broadcom

#365

Post by shevy-java »

So this mafia is driving up RAM prices. And now build their own overpriced hardware.

Either RAM prices go down, or that mafia must pay us all compensation money for this cartel build up. Why is the USA protecting this? How much does the orange man profit personally from helping drive up the prices here?

Re: OpenAI unveils its first custom chip, built by Broadcom

#366

Post by therealcamino »

seydor wrote: post #115
sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

realistically, how hard are AI accelerators to design?

Uh, pretty hard?

Re: OpenAI unveils its first custom chip, built by Broadcom

#367

Post by donavanm »

MisterTea wrote: post #250
swiftcoder wrote: post #241

Earlier quoted context omitted.

Yeah, "first chip" here likely means they contracted Broadcom (or a firm with similar experience) to do all the heavy lifting. Building out your own in-house teams for this sort of thing is a decade-long project - just look how much inside Apple's early chips was licensed ARM / PowerVR cores

Apple didn't have the talent in-house until they bought Intrincity who worked with Samsung on Apple's earlier Arm chips as well. https://en.wikipedia.org/wiki/Intrinsity

That’s not quite fair. As I recall there were about 1,500 people in that part of the hardware org circa mid 2000s. Before PA Semi there were pretty established teams already doing VLSI/PD/verification/validation, PCB, and of course analog/mixed hardware, in their own work and in conjunction with samsung, old broadcom, qualcomm, etc. Lots of inhouse work went in to all those bespoke monitors, phones, apple tv, airports, etc etc.

My recollection is that PA Semi was very much for the architectural and design talent, even though it was an “asset purchase” and all the existing Power & military chips were hived off.

For Intrinsity I recall a lot of interest was actually in their existing graphics work and EDA. ISTR that those early mobile GPUs were what they focused on.

I was in the mansfield org circa ‘07-11. I spent a lot of time flying between cupertino and austin/bee caves that first year.

Re: OpenAI unveils its first custom chip, built by Broadcom

#368

Post by freakynit »

londons_explore wrote: post #192

I wanna see an inference chip where the weights are part of the rom of the chip. There would be 1 multiplier per weight (and since they're constant, the whole thing turns into a bunch of simple adders), and the total pipelined system throughput would be one token per clock cycle. That means you can probably have millions of users simultaneously using a single bit of silicon, with perhaps 500 million tokens per second…

This may be extreme, or, completely stupid, but, why are we not using genetics to "grow" chips in a chemical soup yet? Similar to Verilog/VHDL, don't we have some similar language to express circuits using gene sequences?

Re: OpenAI unveils its first custom chip, built by Broadcom

#369

Post by lsaferite »

cm2187 wrote: post #308
nine_k wrote: post #234

Earlier quoted context omitted.

In very crude terms, AFAICT , if you have a bunch of matrix multiplications, but one of matrices (the one with model weights) doesn't change, you can seriously speed up the computation. One thing is that you don't need to re-fetch the elements of the constant matrix, you can keep it near the ALUs. Then you maybe can detect and ignore sparse / empty blocks by marking them once. IDK how the custom hardware exploits thi…

Random thought. Once models stabilise, could you possibly hardcode the model in gates? Or are they too large for a single chip?

https://taalas.com/
Post reply on HN