Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Jalapeño: Better than Nvidia Blackwell

newsletter.semianalysis.com

Post reply on HN
Print view View original

221–230 of 390 posts

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#221

Post by adventured »

dgacmu wrote: post #209
twobitshifter wrote: post #200

Earlier quoted context omitted.

It depends when the good enough level hits. Pretty sure we are almost there for most common applications of AI.

That's only half the problem. OpenAI is contractually obligated, if you will, to believe that models will continue improving at an impressive rate for the foreseeable future (otherwise their valuation makes no sense). If you believe that, then you should expect to get Sol-level performance out of a Luna-cost model within six months or a year. If you have a system with the weights baked in, that means you're going to…

Assume a $800 billion valuation. $100 billion ad network. $30 billion op income. 26x price to op income ratio. It's right there for them to grab, or someone else to grab.

Their valuation does make sense if you believe: 1) they can retain a massive user base and 2) a massive user base can be monetized. Future value is almost always pulled forward these days for high growth tech companies.

An LLM the size of Google search in users is even more valuable than Google search. The ad market for LLMs will be even larger than search was (no matter what HN prefers).

The monetization part is the easier part. Silicon Valley understands extraordinarily well how to build ad networks. If OpenAI maintain their gigantic user base, a $100 billion ad network is a given bolt-on. They'd have to screw that up in an epic way to not get there.

Facebook - Insta - WhatsApp is an absolute dogshit tandem with a gigantic user base. $228 billion in ad sales and still expanding 10% per year.

Google knows this is what's happening, that's why they don't care about chasing Anthropic very much. They're busy completely remaking how their core search business works.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#223

Post by pantelisk »

Caracas288 wrote: post #141
andy_ppp wrote: post #138

Earlier quoted context omitted.

Yes, they could also sell me GPT Sol 5.6 or 5.7 on a chip and I’d probably buy it. It’s a really really useful model for me, I’m not sure how much better for coding I need it to be. For most things I find Sol good enough with a small amount of coaxing around my tastes.

Man wouldn’t it be cool to be able to slot a massive ROM AI chip into the external AI drive of the pc…

It should look like a NES cartridge! That you have to blow on its end to clear any dust and it should do a satisfying click when it slots in.

Cooling might be an issue though...

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#224

Post by asveikau »

simianwords wrote: post #62
mathisfun123 wrote: post #46

Earlier quoted context omitted.

and amazon shipping used to be free without prime, and uber used to be cheaper than taxis, and airbnb used to be cheaper than hotels. you really don't get it?

almost every pure tech commodity has gone down in price - gpus - retail computers - laptops - ~gpu~ appliances like washing machines - cloud computing i think you don't get how economy usually works in tech

GPUs and memory have gone up in price. It's more expensive to buy a 1-2 year old video card than it was at launch, sometimes by a shockingly large factor. Laptop vendors have recently shipped flagship models with less memory than the previous model, because they can't match price expectations for a laptop.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#225

Post by jerf »

basilgohar wrote: post #217
jeremyjh wrote: post #139

Earlier quoted context omitted.

I think Sol is already good enough though.

"640k (token context) should be enough for anyone."

I know what you're saying, but modulo things like losing track of what year it is as time passes by, a current frontier model is going to continue to be useful for many tasks for many years, even moreso if it's 5-10x faster due to the chip architecture.

It's not that it would be the best forever, it's that it would be useful for plenty long enough to be worthwhile, even if there was better stuff available. In exactly the same way that this computer I'm typing this message on is not the latest and hottest cutting edge stuff. A 7 year old CPU, 7 year old Intel integrated graphics, an older NVMe disk, a mere 32GB of RAM... ok, that's one spec that's still pretty modern although it is slower RAM... but it's still plenty fast enough to comment on HN, even these seven years after it was cutting edge.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#226

Post by Ductapemaster »

Eisenstein wrote: post #145
Ductapemaster wrote: post #108

Earlier quoted context omitted.

Evaporative cooling does not necessitate an open loop system

The system which runs coolant over the chips can be closed but the part which uses an evaporative system to cool that is still open loop and vents water into the air, no?

Nope, it doesn't have to be open loop!

Example of such a system being used specifically for datacenters: https://blog.vantage-dc.com/2026/04/22/cooling-without-the-d...

Evaporated water is condensed, and in the process transfers its heat into another place that removes it. Another simple example is a pot of boiling water with a lid on it.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#227

Post by walrus01 »

fraboniface wrote: post #28

I hadn't seen the token/Joules comparison with human speech before. Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

Fairly amazing when you think about it, like human intellect can run on a bowl of rice and a chicken yakitori skewer.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#228

Post by thedougd »

bayindirh wrote: post #132
ehnto wrote: post #123

Earlier quoted context omitted.

Which in turn reminds me of Soundblaster audio cards! I suspect inference chips are closer to the GPU story than the Soundblaster story though. I remember one soundblaster card I bought came with a Lara Croft demo, that exploited the incredible immersion of real time dynamic reverb. Genuinely I think game audio took a few steps back from that heady era, the innovation in audio likely didn't sell as many cards as grap…

EAX was very powerful in its heyday, but it has died because of a thousand cuts. First we had to have the audio processor. Good EAX was available on top of the line cards, and they were not always cheap. Lower end chips got less features. Then we had to have the speaker setup to have the greatest sound, or needed to get a real 5.1 headphones, which were bulky and never provided the same fidelity. Then Microsoft chang…

The DSPs that could double as a sound card and “soft” modem ruined their market in short order.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#229

Post by andsoitis »

mchusma wrote: post #85

I think they talked about this being general purpose chip but I would think that Anthropic/OpenAI are at the scale now they could bake LLM weights into chips themselves. For example, GPT Sol baked into a custom chip run for $100M that runs 10x as fast and 10x as cheap should pay for itself as long as the chip is useful for long enough. While 2 years ago nothing was useful more than 1 year long, there are many older m…

> For example, GPT Sol baked into a custom chip run for $100M that runs 10x as fast and 10x as cheap should pay for itself as long as the chip is useful for long enough.

but you trade updatability, which I don't think is worth it yet.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#230

Post by adventured »

wmf wrote: post #143
corford wrote: post #120

These nascent inference chip efforts are reminding me of the early 3dfx / riva / mach / powervr days. Will be interesting to see if inference chips are here to stay and, if so, who the eventual dominant player(s) will be

Every company is designing their own chips so the dominant players will be one level down: Broadcom, TSMC, Hynix/Samsung/Micron, etc.

There is drastically more power and profit in the software ultimately.

Apple is in the software first, the hardware second. Everyone at Apple has been trained to understand this for decades, and Jobs pointed it out endlessly. Apple's real moat is software (services, iOS, experience, MacOS).

Windows, Office, Azure, et al. Microsoft accumulated approximately one zillion dollars in profit on the back of software. It's a vastly superior business to anything hardware has traditionally seen. Nvidia is the first true juggernaut hardware profit machine, and the AI boom in extended hardware (RAM, storage) will prove temporary (even if there is a feast during that time). Microsoft's advantage and moat was Windows-Office for decades. It was a far better business than Intel's chip biz.

Google is a software company first. Every aspect of what made them and maintains them is software first, hardware second. They're a $400 billion software company. Their ad machine is software. Search is software.

Facebook is software. Instagram is software. WhatsApp is software. A $200 billion software company. They're not selling hardware, they're selling ads via software, they're monetizing users that use their software.

AWS is at least half software as an entity in terms of complexity, competitive advantage, et al. That's a two trillion dollar business.

LLMs can run successfully with various hardware approaches. The software is the value at the end of this, regardless of the hardware under it. The sole exception so far that may be sustainable is Nvidia, and we'll see if the bottom falls out from under that margin monster (China, specialized AI chips, whatever it happens to be that cuts under them massively).

Hardware always gets its margin squeezed eventually because it's a manufactured good (with inventory, fabs, etc). Software is hyper margin by default, you have to layer a lot of garbage on top of it to kill the margin. Nvidia is 33 years old, they have had a rich business for three years, that's it.

The AI boom is the sole reason anything in hardware has looked great in the past 20 years. Check the margins & op income for the top 20 hardware companies, from TI to AMD to Intel to Nvidia to Micron to Sandisk to Samsung to TSMC to ASML, prior to the AI boom of the past couple years. It won't last indefinitely. And after the return to a more normal environment happens, the hyper margins in software will persist.

Post reply on HN