Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Jalapeño: Better than Nvidia Blackwell

newsletter.semianalysis.com

Post reply on HN
Print view View original

71–80 of 389 posts

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#71

Post by jacquesm »

nxtfari wrote: post #13
epistasis wrote: post #7

It's so funny to see FP4.... I remember 20 years ago being asked what sort of HPC we needed in genomics, and the answer was basically, "lower precision, faster" for the stuff I was working on. But FP4 is, well, almost comical. One thing not on that comparison table: die size. If I'm understanding that correctly, it's about the same as the Rubin, but at 1/3 the number of NVFP4 PFLOPs. (The text disagrees with the tabl…

Agree, I remember when even half precision made its way into C# sometime around 2020 (I didn’t know much about ML then) and I thought, well I guess that’s a worthwhile tradeoff but I can’t imagine going lower. Lo and behold (1-bit Bonsai) how much lower you could go.

Ternary?

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#72

Post by danishanish »

fraboniface wrote: post #28

I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

I mean, surely when quality is accounted for the difference is significantly higher

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#73

Post by zirkonit »

simianwords wrote: post #62
mathisfun123 wrote: post #46

Earlier quoted context omitted.

and amazon shipping used to be free without prime, and uber used to be cheaper than taxis, and airbnb used to be cheaper than hotels. you really don't get it?

almost every pure tech commodity has gone down in price - gpus - retail computers - laptops - ~gpu~ appliances like washing machines - cloud computing i think you don't get how economy usually works in tech

I'm especially enjoying how RAM and SSDs are going down in price.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#74

Post by [deleted] »

datakan wrote: post #8
varispeed wrote: post #4

Why they don't research how to make their own RAM and they have to buy it from the common market? They should GTFO with this crap. Create barriers to computing for ordinary people while milking businesses for tokens.

People keep saying stuff like this without understanding what it takes to make RAM. It's one of, if not the most, heavily patented things in the world. The second you dip your toes into those waters the lawsuits begin. If somehow you get around the patent issues, you're now faced with huge research and development costs, fabs to build, processes to sort out and all of that has very high failure rates. Last time I che…

[deleted]

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#75

Post by chris_money202 »

datakan wrote: post #69
chris_money202 wrote: post #47

Earlier quoted context omitted.

RAM chips are not hard to produce compared to many other types of semiconductors; Intel started in the memory game and left because the margins weren't great and they were going to fold. The failure rates on these chips are actually very tolerable; you can have a very bad yield and still have a viable chip due to things like ECC.

Intel entered the memory space because they partnered with Micron. They left the memory space when Micron pulled out of the partnership.

Intel started making DRAM in 1970, Micron was founded in 1978.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#76

Post by vntok »

dpe82 wrote: post #50
simianwords wrote: post #32

How can OpenAI mass produce this chip at scale more economically than Nvidia which has experience in the supply chain and scale efficiencies to do it efficiently?

NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost. One objective of the project might be simply to provide credible negotiating leverage when dealing with existing suppliers like NVidia. You don't have to deploy at scale for that to work, but you do have to look like you could if pushed hard enough.

> NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost.

But then that means you have no actual moat against the behemot, right? Your competitor can move into the market as soon as they want to, at much better cost (so at slightly better price)... and Nvidia certainly can adapt much faster around hard hardware specs innovation than a new entrant ever could.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#78

Post by Alifatisk »

jimmySixDOF wrote: post #11

I love how now you have to consider the possible s** posting motivation behind analysis of a trillion dollar industry being conducted at a world-class level by a bunch of ex Reddit and 4Chan adjacent mods -- it's one of the best stories in AI that SemiAnalysis is not cut from the same cloth as Gartner McKinsey et al

Why censor yourself?

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#79

Post by sobellian »

jrflo wrote: post #31
anthonypasq wrote: post #3

Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.

This may just be a classic case of Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox In short, better hardware will drive down token cost in the near-term, but will drive up the demand for tokens as it gets cheap enough for other sectors to start to use it heavily. It comes from steam engines where economists originally thought that coal demand would plummet with more efficient engines, but it actually jus…

If we are applying Jevons paradox to this then the unit being consumed is not tokens but the inputs for token production - power, capex, something else. To draw an analogy to the steam engine, coal:electricity::mechanical-work:tokens. Jevons paradox does not talk about mechanical work becoming cheaper in the short term setting up a sort of rubber band of demand creating spiking prices for mechanical work. Compared to the renaissance, mechanical work was much cheaper throughout the industrial revolution and remains cheaper to this day. We can still definitely say that the easier it is to produce tokens, the cheaper they will be.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#80

Post by plasticchris »

fraboniface wrote: post #28

I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

Probably not when you consider the training cost and upkeep expenses, not to mention the depreciation…
Post reply on HN