Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Jalapeño: Better than Nvidia Blackwell

newsletter.semianalysis.com

Post reply on HN
Print view View original

41–50 of 389 posts

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#41

Post by chris_money202 »

simianwords wrote: post #32

How can OpenAI mass produce this chip at scale more economically than Nvidia which has experience in the supply chain and scale efficiencies to do it efficiently?

In the short and medium term, it probably won't be more economical to produce for OpenAI. Where OpenAI is benefitting from their own chip is being able to tailor it to their models and workloads. When you buy off the shelf Nvidia, its not perfectly tailored and OpenAI has to spend marginally more to run off that chip. At the scale OpenAI is operating at and plans to operate at, that margin becomes pretty big $$

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#42

Post by anthonypasq »

mathisfun123 wrote: post #17
anthonypasq wrote: post #3

Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.

this is a story about a proprietary accelerator being built/designed by a token provider. and you think they're going to return the efficiency gains to the customer instead of capture the value for themselves? interesting take.

OpenAI just dropped the price of Luna by 80% and Sol by 20-30%

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#43

Post by airspresso »

simianwords wrote: post #24
empath75 wrote: post #18

When people talk about the commodification of inferencing, they imagine a future where everyone has access to frontier models and can run them at the same cost, and what will actually happen is closer to the commodification of _oil_, where only a few companies have the scale to produce it at a competitive price, and advances like this are _why_. Once models are more or less interchangeable, the price of LLMs will dro…

I don't believe models will be commodified because each model is unique with strengths and weaknesses. Its not like Steel which is more or less the same no matter where you purchase it from. If what you said were true, you would hardly see people complaining about the quality of Opus 5 or good writing from Sol. But people do.

This depends heavily on what the use-case is. Yes, if it's a coder making software and having to read LLM output then writing style matters. If the LLM is used in an automated data processing pipeline with a capped level of complexity, entirely different aspects matter and LLMs become more interchangeable.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#44

Post by anthonypasq »

spacephysics wrote: post #27
mathisfun123 wrote: post #17

Earlier quoted context omitted.

this is a story about a proprietary accelerator being built/designed by a token provider. and you think they're going to return the efficiency gains to the customer instead of capture the value for themselves? interesting take.

We should be mindful of the context that many of these providers VERY likely have been selling their subscriptions at a substantial loss So as much as i agree “more profits to stakeholders screw the customer”, i think its more of an emergency to get to profitability before the music stops.

> We should be mindful of the context that many of these providers VERY likely have been selling their subscriptions at a substantial loss.

what makes you think this?

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#45

Post by dumberquestions »

dgellow wrote: post #15
anthonypasq wrote: post #3

Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.

There is just so much downward pressure on token price, from every direction. We would need a completely new understanding of economics to explain why the price shouldn’t go down. Or market collusion/regulatory manipulation.

The demand for them is growing _per person_, not just across the wider economy, if tokens cost half as much but you want to use 3 times as much you're going to have to pay more.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#46

Post by mathisfun123 »

anthonypasq wrote: post #42
mathisfun123 wrote: post #17

Earlier quoted context omitted.

this is a story about a proprietary accelerator being built/designed by a token provider. and you think they're going to return the efficiency gains to the customer instead of capture the value for themselves? interesting take.

OpenAI just dropped the price of Luna by 80% and Sol by 20-30%

and amazon shipping used to be free without prime, and uber used to be cheaper than taxis, and airbnb used to be cheaper than hotels.

you really don't get it?

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#47

Post by chris_money202 »

datakan wrote: post #8
varispeed wrote: post #4

Why they don't research how to make their own RAM and they have to buy it from the common market? They should GTFO with this crap. Create barriers to computing for ordinary people while milking businesses for tokens.

People keep saying stuff like this without understanding what it takes to make RAM. It's one of, if not the most, heavily patented things in the world. The second you dip your toes into those waters the lawsuits begin. If somehow you get around the patent issues, you're now faced with huge research and development costs, fabs to build, processes to sort out and all of that has very high failure rates. Last time I che…

RAM chips are not hard to produce compared to many other types of semiconductors; Intel started in the memory game and left because the margins weren't great and they were going to fold. The failure rates on these chips are actually very tolerable; you can have a very bad yield and still have a viable chip due to things like ECC.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#48

Post by mathisfun123 »

simianwords wrote: post #26
mathisfun123 wrote: post #17

Earlier quoted context omitted.

this is a story about a proprietary accelerator being built/designed by a token provider. and you think they're going to return the efficiency gains to the customer instead of capture the value for themselves? interesting take.

Yes, I can bet on this happening. If anything, this is a net gain for consumers as it is a competitive market.

go ahead and bet: alibaba is a publicly traded company

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#49

Post by brcmthrowaway »

LarsDu88 wrote: post #30

Well Sam Altman finally has built a moat against Chinese open weight AI. Well done. But what will this mean for Cerebras? I remember when Tesla was building its own inference chips, and after about 2 years and billions spent, the whole effort was scuttled b/c they simply could not keep up with the iteration and R&D cycles of dedicated chip companies. I suspect the same will be the case with OpenAI vs Cerebras + Nvidi…

[dead]

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#50

Post by dpe82 »

simianwords wrote: post #32

How can OpenAI mass produce this chip at scale more economically than Nvidia which has experience in the supply chain and scale efficiencies to do it efficiently?

NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost.

One objective of the project might be simply to provide credible negotiating leverage when dealing with existing suppliers like NVidia. You don't have to deploy at scale for that to work, but you do have to look like you could if pushed hard enough.

Post reply on HN