Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Jalapeño: Better than Nvidia Blackwell

newsletter.semianalysis.com

Post reply on HN
Print view View original

141–150 of 389 posts

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#141

Post by Caracas288 »

andy_ppp wrote: post #138
mchusma wrote: post #85

I think they talked about this being general purpose chip but I would think that Anthropic/OpenAI are at the scale now they could bake LLM weights into chips themselves. For example, GPT Sol baked into a custom chip run for $100M that runs 10x as fast and 10x as cheap should pay for itself as long as the chip is useful for long enough. While 2 years ago nothing was useful more than 1 year long, there are many older m…

Yes, they could also sell me GPT Sol 5.6 or 5.7 on a chip and I’d probably buy it. It’s a really really useful model for me, I’m not sure how much better for coding I need it to be. For most things I find Sol good enough with a small amount of coaxing around my tastes.

Man wouldn’t it be cool to be able to slot a massive ROM AI chip into the external AI drive of the pc…

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#142

Post by ignoramous »

corford wrote: post #120

These nascent inference chip efforts are reminding me of the early 3dfx / riva / mach / powervr days. Will be interesting to see if inference chips are here to stay and, if so, who the eventual dominant player(s) will be

> who the eventual dominant player(s) will be

This couldn't have been easy. The team at OpenAI has worked a miracle.

  For example, Meta and Microsoft’s AI ASIC programs not getting off the ground despite being at it for much longer shows that cost is only one part of the equation.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#143

Post by wmf »

corford wrote: post #120

These nascent inference chip efforts are reminding me of the early 3dfx / riva / mach / powervr days. Will be interesting to see if inference chips are here to stay and, if so, who the eventual dominant player(s) will be

Every company is designing their own chips so the dominant players will be one level down: Broadcom, TSMC, Hynix/Samsung/Micron, etc.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#144

Post by skhameneh »

simianwords wrote: post #24
empath75 wrote: post #18

When people talk about the commodification of inferencing, they imagine a future where everyone has access to frontier models and can run them at the same cost, and what will actually happen is closer to the commodification of _oil_, where only a few companies have the scale to produce it at a competitive price, and advances like this are _why_. Once models are more or less interchangeable, the price of LLMs will dro…

I don't believe models will be commodified because each model is unique with strengths and weaknesses. Its not like Steel which is more or less the same no matter where you purchase it from. If what you said were true, you would hardly see people complaining about the quality of Opus 5 or good writing from Sol. But people do.

> Its not like Steel which is more or less the same no matter where you purchase it from.

I’m not an expert in metallurgy by any means, but this seems really off. There are many recipes for steel and varied processes that also impact the final product.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#145

Post by Eisenstein »

Ductapemaster wrote: post #108
SirMaster wrote: post #87

Earlier quoted context omitted.

They evaporate the water which is what makes it cool so effeciently.

Evaporative cooling does not necessitate an open loop system

The system which runs coolant over the chips can be closed but the part which uses an evaporative system to cool that is still open loop and vents water into the air, no?

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#146

Post by Imustaskforhelp »

senordevnyc wrote: post #119
goodmythical wrote: post #70

Earlier quoted context omitted.

[flagged]

There is something counter-intuitive about the idea that making an engine that accomplishes the same amount of work with half the fuel will result in MORE fuel usage overall. You might expect it to be the same, or decline slightly, but the paradoxical element is that overall consumption goes up. And you can say of course, it's so obvious, how could a dumdum not see that! But then there are lots of examples of things…

> I don't think we know yet what is going to happen as software development gets much cheaper. If in ten years we can produce software 1000x more cost effectively, will we need fewer software engineers, the same, or more? Guess we'll see!

Adding onto it, I feel as if this relates to some points regarding predictions of future in general. It is easier for us to look from the future to the past and think that it must be very obvious (as you also mention) but its also very counter-intuitive at the same time and there are just so so much nuance about basically any situation within it that its hard to really capture it all, and even then, be prepared for surprises and counter-intuitiveness.

I really like the Peter Drucker quote about it.

“The only thing we know about the future is that it will surprise us.” — Peter Drucker

and, “The future is fundamentally different from the past.” — Frank Knight, Risk, Uncertainty and Profit (1921)

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#147

Post by jacquesm »

jeffbee wrote: post #128
jacquesm wrote: post #71

Earlier quoted context omitted.

Ternary?

Knuth's base- e proposal enters the chat. They were right about everything 50+ years ago, but they didn't have the budget for the right hardware, had to write conference papers and books instead.

I can totally see how ternary would work from a physical implementation perspective but I have a really hard time visualizing anything using base-e, can you explain how such a thing would work in practice?

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#148

Post by wmf »

ChoosesBarbecue wrote: post #2

This is most impressive. The interesting question to me, is outside of the LLM accelerator space: will generalized chips have massive leaps in performance once LLM technology is used to create the next generation? In general, will we see rapid advances while we extract the value of these models in creating architectures? I'm so far removed from the space that this is a very naive interpretation of all this, but I'm c…

Existing CPUs have been extremely optimized by ~6 competing, well-funded teams. I expect AI to accelerate things somewhat but it's not clear that there is any low-hanging fruit available for AI to find.

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#149

Post by acedTrex »

einpoklum wrote: post #122

I hope the LLM wave will leave GPUs behind to go back to pursue more general-purpose computation rather than spending their die area on multiplying 4-bit-number matrices and such things.

Is that not literally the exect opposite of the direction asics for LLM inference is going?

Re: OpenAI Jalapeño: Better than Nvidia Blackwell

#150

Post by mkw5053 »

Warning, this is a long comment! (I’m trying to stick to sourced facts here and not overstate what they mean)

I went down a rabbit hole after watching Dylan Patel on Dwarkesh today: https://www.youtube.com/watch?v=aV26V1UvkJw

I was initially just surprised by how bullish Dylan is on OpenAI/Anthropic and how bearish he is on China, despite Chinese labs getting closer to US SOTA while offering inference at dramatically lower prices.

So, I started digging while waiting for various day-job inference calls to return, ha.

Dylan says he spent years obsessively posting on hardware forums, moderating hardware subreddits, and running anonymous hardware blogs/videos before SemiAnalysis. But he also says most of that history is now gone, including from the Internet Archive, because he asked for it to be removed.[1]

In a 2024 interview he described his post-college job as “data science” around hurricane/earthquake/wildfire simulations for a financial company.[1] In a 2026 Sequoia interview he described himself as having been a “quant at a small quant risk firm” who generated $10M+ of “risk-free revenue.”[2] The Information reports that he declined to identify the employer and doesn’t list it on LinkedIn.[3]

Even harmless/silly stuff seems to drift. In February he said he kept bees for ~1.5 years. Today it was “few months, few months.”[4][5] I know, sort of silly and doesn't matter.

The Information reports that Patel owns stakes in ~20 startups in the same ecosystem SemiAnalysis covers, organized a $50M Fluidstack SPV, and is now targeting a $400M venture fund.[3][6]

And, in a 2022 HN discussion about SemiAnalysis disclosures, after saying his reports had moved smaller stocks by 20% in a day, Patel wrote: “If I thought I could move the stock, I'd make the position in the morning alongside my clients, and publish shortly after.”[7]

I don’t know that any of this is false or that anything improper happened (I’m definitely not claiming that). More that 1-2 of these things would just be odd. Taken together, though, they made me question how much trust I was putting in the broader story.

The dynamic of reminds me of crypto, WeWork, Theranos, Citron, etc. Once enough important people validate someone, things that would normally invite basic diligence somehow stop getting questioned.

[1] https://www.dwarkesh.com/p/dylan-jon

[2] https://sequoiacap.com/podcast/dylan-patel-of-semianalysis-w...

[3] https://www.theinformation.com/articles/dylan-patel-semianal...

[4] https://www.latent.space/p/dylanpatel-cooking

[5] https://www.dwarkesh.com/p/dylan-patel-3

[6] https://www.theinformation.com/briefings/exclusive-semianaly...

[7] https://news.ycombinator.com/item?id=31065646

Post reply on HN