Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI unveils its first custom chip, built by Broadcom

techcrunch.com

Post reply on HN
Print view View original

191–200 of 496 posts

Re: OpenAI unveils its first custom chip, built by Broadcom

#191

Post by blitzar »

sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

> the use of email, spam filters and spellchecker to accelerate parts of the design and optimization process

honestly you don't realise how much more efficient it is until you are stuck using the wrong flavour of outlook, the spam filter breaks or sloppy spelling, punctuation and grammar force you to clarify details needlessly.

Re: OpenAI unveils its first custom chip, built by Broadcom

#192

Post by londons_explore »

I wanna see an inference chip where the weights are part of the rom of the chip.

There would be 1 multiplier per weight (and since they're constant, the whole thing turns into a bunch of simple adders), and the total pipelined system throughput would be one token per clock cycle.

That means you can probably have millions of users simultaneously using a single bit of silicon, with perhaps 500 million tokens per second coming out the output bus.

Downside is this chip would be huuuuge - a whole wafer.

Wafer level faults probably won't matter though - neural nets are resistant to a few missing or wrong weights.

Due to the speed the industry moves, you'd want to race from model weights to production super fast, make 50 wafers, use them for a year, then bin them when that model is obsolete.

Re: OpenAI unveils its first custom chip, built by Broadcom

#194

Post by smsx »

KeplerBoy wrote: post #178
HarHarVeryFunny wrote: post #38

Earlier quoted context omitted.

This is just an uncut wafer - I don't think it's intended to be a wafer-scale chip. Cerebras etch memory onto the wafer alongside the processing elements, but AFAIK OpenAI are going to be using HBM memory and a conventional chiplet design.

Still competition for cerebras. Seems quite unlikely they will get an OpenAI deal anytime soon.

They have an OpenAI deal right now. https://openai.com/index/cerebras-partnership/

Re: OpenAI unveils its first custom chip, built by Broadcom

#195

Post by [deleted] »

sharkjacobs wrote: post #61

> Developed from design to production in nine months, accelerated by OpenAI’s models > the use of OpenAI models to accelerate parts of the design and optimization process. I wish there was more about this. As is I kind of have to assume that this is just meaningless marketing, like saying development was accelerated by Microsoft Office or their 5k LG Ultrafine 40-inch monitors. Like, if this was as big a deal as it k…

[deleted]

Re: OpenAI unveils its first custom chip, built by Broadcom

#196

Post by wmf »

chris_money202 wrote: post #167

Microsoft, Google, and Amazon also do this, but they also have the hyperscaler datacenter infrastructure to host the chips. Designing and taping out the chip is one thing, packaging, cooling, deploying, powering, and managing the fleet is another stack entirely. Wonder where that will come from?

Don't forget Stargate.

Update: Somebody on Twitter said it's going to be hosted 50/50 at Microsoft and Oracle.

Re: OpenAI unveils its first custom chip, built by Broadcom

#197

Post by tehjoker »

No information on how significant the reduction in energy per token is. No information on amortized price per request. Increasingly its clear OpenAI must demonstrate order of magnitude reductions in cost to not die, this is investor story time without that information.

Re: OpenAI unveils its first custom chip, built by Broadcom

#198

Post by NitpickLawyer »

xnx wrote: post #160
changoplatanero wrote: post #97

Earlier quoted context omitted.

Browsing openai's job postings in the past few months is enough to contirm that it's more than this. They are for sure making serious efforts at building ai for chip design.

Impossible to know. Could be fake/aspirational roles to impress investors with their grand vision.

Jesus. This is tinfoil hat territory now. Why would they fake something like that? ANY company in this field would try to become free from nvda. Goog has done it already, amazon has their own thing, so it can be done. Not saying they'll 0shot this vertical, but ffs, they don't need to fake anything. They are making an effort, and it would be insane to think they aren't. Might work, might not work, but to even think that the effort is fake is going too far.

Re: OpenAI unveils its first custom chip, built by Broadcom

#199

Post by baq »

cwillu wrote: post #182
knicholes wrote: post #147

Earlier quoted context omitted.

Right. KiCAD for PCB design. Blender for 3D CAD. Oh, are you saying I should have used something other than the KiCAD MCP server for better results?

Meta: can we not downvote people who are clarifying what they're saying and asking questions, even if they're wrong about something, if the content isn't otherwise objectionable?

I didn’t downvote, but the OP is either a troll or someone who doesn’t want to notice he doesn’t know what he’s talking about. Either way we want less of that on HN.

Re: OpenAI unveils its first custom chip, built by Broadcom

#200

Post by blitzar »

smith7018 wrote: post #39
kilroy123 wrote: post #2

I hope to see something like this, but in a small form factor like the NVIDIA spark. I want a super fast LLM that is Opus 4.6+, like, in ability.

Unfortunately Sam Altman won't be the one to deliver us at-home hardware that can run Opus-level models

I wonder what is happening with the OpenAI / Jony Ive crossover episode.
Post reply on HN