Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Jalapeño: Better than Nvidia Blackwell

newsletter.semianalysis.com

Post reply on HN
Print view View original

101–110 of 389 posts

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#101

Post by doctorpangloss »

jimmySixDOF wrote: post #11

I love how now you have to consider the possible s** posting motivation behind analysis of a trillion dollar industry being conducted at a world-class level by a bunch of ex Reddit and 4Chan adjacent mods -- it's one of the best stories in AI that SemiAnalysis is not cut from the same cloth as Gartner McKinsey et al

The semianalysis people have scripts which incorrectly count their numerators and denominators all the time. All their benchmarks are flawed. It is such a slipshod operation and they charge exorbitant amounts of money for it.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#102

Post by rsync »

lelanthran wrote: post #68

This means that they're going to want to IPO soon - this is good news for investors + they need the capital.

No, this is because they want to IPO soon.

If the chips weren't this compelling they would have something different to announce.

These are paperclip maximizers who just happen to wear human skin - there is no underlying premise nor ideological goal.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#103

Post by epolanski »

LarsDu88 wrote: post #30

Well Sam Altman finally has built a moat against Chinese open weight AI. Well done. But what will this mean for Cerebras? I remember when Tesla was building its own inference chips, and after about 2 years and billions spent, the whole effort was scuttled b/c they simply could not keep up with the iteration and R&D cycles of dedicated chip companies. I suspect the same will be the case with OpenAI vs Cerebras + Nvidi…

> and after about 2 years and billions spent, the whole effort was scuttled b/c they simply could not keep up with the iteration and R&D cycles of dedicated chip companies

That sounds quite like...nonsense?

Chip companies work on years-long cycles. They know today what are they launching 4-5 years from now.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#106

Post by phoghed »

Phemist wrote: post #96
fraboniface wrote: post #28

I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

The 20W number includes EVERYTHING else the brain does. The chips/models are literally only producing tokens. Let's see an LLM drive a robot harness and have the robot produce speech, as well as move through 3D space, keep track of metabolic needs, etc. etc. etc. before we compare efficiencies. That is even assuming the tokens are of equal quality. This comparison is currently Apples and Oranges.

kind of a moot point if you can't get your brain to not do everything else. I think it's a fun comparison, even if it's not a 100% equivalence.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#107

Post by htrp »

mchusma wrote: post #85

I think they talked about this being general purpose chip but I would think that Anthropic/OpenAI are at the scale now they could bake LLM weights into chips themselves. For example, GPT Sol baked into a custom chip run for $100M that runs 10x as fast and 10x as cheap should pay for itself as long as the chip is useful for long enough. While 2 years ago nothing was useful more than 1 year long, there are many older m…

etched tried this.... it didn't go very well

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#108

Post by Ductapemaster »

SirMaster wrote: post #87
minimaltom wrote: post #60

Earlier quoted context omitted.

For datacenters specifically I've never understood what specifically consumes the water. Arent the water-cooling loops closed, so the water just cycles around and around and around?

They evaporate the water which is what makes it cool so effeciently.

Evaporative cooling does not necessitate an open loop system

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#109

Post by HDThoreaun »

jazzyjackson wrote: post #57
dgellow wrote: post #15

Earlier quoted context omitted.

There is just so much downward pressure on token price, from every direction. We would need a completely new understanding of economics to explain why the price shouldn’t go down. Or market collusion/regulatory manipulation.

Maybe 1000s of tokens per second unlocks realtime robotic decision making, and now every robot needs to continuously stream tokens to and from the cloud to operate. That could 1000x demand overnight, just to speculate :)

Seems unsafe to make locomotive decisions remotely

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#110

Post by kemiller »

fraboniface wrote: post #28

I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

I wonder how that stacks up if you consider all the time you have to keep the body alive when it’s not actively producing “tokens”.
Post reply on HN