Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Jalapeño: Better than Nvidia Blackwell

newsletter.semianalysis.com

Post reply on HN
Print view View original

81–90 of 389 posts

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#81

Post by RealityVoid »

anthonypasq wrote: post #44
spacephysics wrote: post #27

Earlier quoted context omitted.

We should be mindful of the context that many of these providers VERY likely have been selling their subscriptions at a substantial loss So as much as i agree “more profits to stakeholders screw the customer”, i think its more of an emergency to get to profitability before the music stops.

> We should be mindful of the context that many of these providers VERY likely have been selling their subscriptions at a substantial loss. what makes you think this?

Because everyone keeps saying this so it must be true. Real "it is known" kind of vibe with these statements.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#82

Post by GaggiX »

danishanish wrote: post #72
fraboniface wrote: post #28

I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

I mean, surely when quality is accounted for the difference is significantly higher

Or maybe significantly lower.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#83

Post by WarmWash »

throwaw12 wrote: post #39

Competition is good for all of us, we will get better and faster chips. Or at least Nvidia GPUs will become slightly cheaper for regular consumers again

That's if any datacenters are allowed to be built with them.

There is probably a ~50% chance that the next Dem candidate for presidency runs on a national datacenter moratorium or something equally as crippling.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#84

Post by yjftsjthsd-h »

madspindel wrote: post #77
minimaltom wrote: post #58

Earlier quoted context omitted.

Thats what I thought too but then it would be s**?

s**? Edit: OK, hn is removing one *

If it's trying to convert it to italics, you may have to use a backslash to escape them

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#85

Post by mchusma »

I think they talked about this being general purpose chip but I would think that Anthropic/OpenAI are at the scale now they could bake LLM weights into chips themselves.

For example, GPT Sol baked into a custom chip run for $100M that runs 10x as fast and 10x as cheap should pay for itself as long as the chip is useful for long enough.

While 2 years ago nothing was useful more than 1 year long, there are many older models in use now (e.g. Haiku 4.5, GPT-OSS 120b), and I expect this trend to continue.

I know this is what Taalas was doing (acquired by AMD), here was their demo, https://chatjimmy.ai/ which is based on Llama 3.1 8B. It feels like this should start to happen soon.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#86

Post by justincormack »

minimaltom wrote: post #60
0xbadcafebee wrote: post #37

Story says they're power limited. That's half-true. Actually they're water-limited. To generate power, you need water. To cool chips, you need water. If you try to use less water on one side, you need more water on the other side (it's physics ya'll, making and using energy generates heat which requires dissipation). The world's freshwater is diminishing while also being consumed at an alarming rate. The future AI ol…

For datacenters specifically I've never understood what specifically consumes the water. Arent the water-cooling loops closed, so the water just cycles around and around and around?

Yes they are for water cooling.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#87

Post by SirMaster »

minimaltom wrote: post #60
0xbadcafebee wrote: post #37

Story says they're power limited. That's half-true. Actually they're water-limited. To generate power, you need water. To cool chips, you need water. If you try to use less water on one side, you need more water on the other side (it's physics ya'll, making and using energy generates heat which requires dissipation). The world's freshwater is diminishing while also being consumed at an alarming rate. The future AI ol…

For datacenters specifically I've never understood what specifically consumes the water. Arent the water-cooling loops closed, so the water just cycles around and around and around?

They evaporate the water which is what makes it cool so effeciently.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#88

Post by WarmWash »

0xbadcafebee wrote: post #37

Story says they're power limited. That's half-true. Actually they're water-limited. To generate power, you need water. To cool chips, you need water. If you try to use less water on one side, you need more water on the other side (it's physics ya'll, making and using energy generates heat which requires dissipation). The world's freshwater is diminishing while also being consumed at an alarming rate. The future AI ol…

This only makes sense if you never looked at comparative water usage rates and available water.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#90

Post by altmanaltman »

jrflo wrote: post #31
anthonypasq wrote: post #3

Continued hardware improvements really make it hard for me to believe token prices will not continue to plummet.

This may just be a classic case of Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox In short, better hardware will drive down token cost in the near-term, but will drive up the demand for tokens as it gets cheap enough for other sectors to start to use it heavily. It comes from steam engines where economists originally thought that coal demand would plummet with more efficient engines, but it actually jus…

I think you're reducing a very complex thing (the global economy) into a very simplistic model (Jevons' paradox) and thinking both are the same thing. This has no predictive power or rigor. You're just wishing things would happen as they did before, without considering that conditions and situations change significantly, and instead of Jevon's paradox, we look back at today 50 years from now and talk about Jensen's paradox.

This doesn't mean the concept is BS, but one single concept cannot explain away everything in such a system.

Post reply on HN