Competition is good for all of us, we will get better and faster chips. Or at least Nvidia GPUs will become slightly cheaper for regular consumers again
The pricing of GPUs themselves aren't really the problem: it's the VRAM that comes with them.
OpenAI Jalapeño: Better than Nvidia Blackwell
51–60 of 390 posts
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#52Earlier quoted context omitted.
This may just be a classic case of Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox In short, better hardware will drive down token cost in the near-term, but will drive up the demand for tokens as it gets cheap enough for other sectors to start to use it heavily. It comes from steam engines where economists originally thought that coal demand would plummet with more efficient engines, but it actually jus…
the total cost spent on tokens may go up, but i just cant imagine per token costs going up
Depends on compute capacity. If we become supply constrained on tokens, then prices will necessarily go up.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#53When people talk about the commodification of inferencing, they imagine a future where everyone has access to frontier models and can run them at the same cost, and what will actually happen is closer to the commodification of _oil_, where only a few companies have the scale to produce it at a competitive price, and advances like this are _why_. Once models are more or less interchangeable, the price of LLMs will dro…
I don't believe models will be commodified because each model is unique with strengths and weaknesses. Its not like Steel which is more or less the same no matter where you purchase it from. If what you said were true, you would hardly see people complaining about the quality of Opus 5 or good writing from Sol. But people do.
> I don't believe models will be commodified because each model is unique with strengths and weaknesses.
They are all converging.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#54How can OpenAI mass produce this chip at scale more economically than Nvidia which has experience in the supply chain and scale efficiencies to do it efficiently?
Replace OpenAI with Apple and Nvidia with Intel.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#55Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.
Hopefully this also means billionaires can stop trying to drop data centers into residential neighborhoods with zero noise control and polluting on-site generators, signing local politicians on with NDAs, calling for eminent domain to seize homes to build power lines to data centers, etc. etc. etc. Not to mention the water use controversy. Token prices plummeting is probably a good thing, but not without the regulato…
>polluting on-site generators
how much pollution do you believe modern gas-turbine engines to produce?
>Not to mention the water use controversy.
what percentage of US water usage do you believe is by AI data centers?
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#56I love how now you have to consider the possible s** posting motivation behind analysis of a trillion dollar industry being conducted at a world-class level by a bunch of ex Reddit and 4Chan adjacent mods -- it's one of the best stories in AI that SemiAnalysis is not cut from the same cloth as Gartner McKinsey et al
"not cut from the same cloth as Gartner McKinsey et al"
Yeah, those guys aren't biased at all.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#57Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.
There is just so much downward pressure on token price, from every direction. We would need a completely new understanding of economics to explain why the price shouldn’t go down. Or market collusion/regulatory manipulation.
Maybe 1000s of tokens per second unlocks realtime robotic decision making, and now every robot needs to continuously stream tokens to and from the cloud to operate. That could 1000x demand overnight, just to speculate :)
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#58Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#59When people talk about the commodification of inferencing, they imagine a future where everyone has access to frontier models and can run them at the same cost, and what will actually happen is closer to the commodification of _oil_, where only a few companies have the scale to produce it at a competitive price, and advances like this are _why_. Once models are more or less interchangeable, the price of LLMs will dro…
I don't believe models will be commodified because each model is unique with strengths and weaknesses. Its not like Steel which is more or less the same no matter where you purchase it from. If what you said were true, you would hardly see people complaining about the quality of Opus 5 or good writing from Sol. But people do.
The same level of intelligence gets roughly 10x cheaper per year. So you might both be correct where a large part are commodity tasks but frontier is hard and valuable and not commodities.
Re: OpenAI Jalapeño: Better than Nvidia Blackwell
#60Story says they're power limited. That's half-true. Actually they're water-limited. To generate power, you need water. To cool chips, you need water. If you try to use less water on one side, you need more water on the other side (it's physics ya'll, making and using energy generates heat which requires dissipation). The world's freshwater is diminishing while also being consumed at an alarming rate. The future AI ol…
For datacenters specifically I've never understood what specifically consumes the water. Arent the water-cooling loops closed, so the water just cycles around and around and around?