Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the AI Bubble Bursts

martinvol.pe

Post reply on HN
Print view View original

441–450 of 557 posts

Re: How the AI Bubble Bursts

#441

Post by T-A »

ajross wrote: post #89
joshstrange wrote: post #53

> RAM prices are crashing because new models won’t need as much Reality begs to differ [0] and following the link for that text goes to an article [1] where they talk about Google's TurboQuant which supposedly will lower the RAM requirements. Now if that means RAM prices come down (as speculated, not reported on, in the link) or the AI companies just do more things with their extra ram is yet to be determined. The fa…

> Reality begs to differ Honestly you're both wrong. RAM prices spiked speculatively, and they're going down for the same reason. Market people always want to argue in fundamentals, when in practice *ALL* the high frequency components of the signal are down to a bunch of traders trying to guess where it's going in the short term. At best those guesses are informed by ground truth ("AI needs a lot of RAM!" "Sam corner…

> RAM prices spiked speculatively, and they're going down for the same reason.

https://pcpartpicker.com/trends/price/memory/

Note how flat the black lines are.

Then note how wide the gray bands are. That makes it very easy to cherry-pick a few examples to present as "supporting evidence" that prices are doing whatever you want to believe they are doing.

Re: How the AI Bubble Bursts

#442

Post by hirako2000 »

Throaway199999 wrote: post #427
hirako2000 wrote: post #359

Earlier quoted context omitted.

> luckily I don't have to. It's free. Ponder that for a minute. There are over 2 million games, for Android alone. That you weren't making games before the advent of LLMs makes it cool for you to build, and at no cost. But people have been able to make games without them and already grew the market to saturation. If the outcome of LLMs is that we get more games, it won't imply that people will consume more games. Mos…

There's nothing to "ponder" as you so patronizingly put it, and your stats on gaming are self-evident. Op never said they're selling games. They said they're making their own games and websites for a fraction of the cost (even $0). That's amazing value. And it's just getting better.

that $0 is meant to go on the side of the value add that justifies the sort of funding we are seeing?

I didn't mean to patronize, sometimes self evidence isn't trivial to notice.

Re: How the AI Bubble Bursts

#443

Post by tartoran »

aurareturn wrote: post #437
tartoran wrote: post #392

Earlier quoted context omitted.

Would you still pay if prices were to increase,say $1500-2000 monthly?

Probably. I assume the value would drastically increase. Companies will definitely continue to pay for it. It's irreplaceable now.

How about if they plateau but prices skyrocket? Most companies would pay but if you're not working for a company that does pay for it, what's the line beyhond which you'd think twice about paying for it yourself? 500? 1000? 1500?

Re: How the AI Bubble Bursts

#444

Post by aurareturn »

arctic-true wrote: post #397
aurareturn wrote: post #393

Earlier quoted context omitted.

Profitable as in every token they generate, they make some money. And it's already mentioned that the path to profitability is that inference revenue eclipses training costs. It's already happening rapidly.

I’m not talking about training costs. I’m talking about startup costs. You have to pay for GPUs (or to rent data centers). You have to pay for the electricity that runs those data centers, and in a lot of cases these frontier labs are building the data centers on credit, so you need to pay for the construction, the materials, etc. If it was as simple as “running the GPUs costs less than we charge for it,” I might be…

Right now, the demand is far more than supply for GPUs. Every cloud company is saying they're leaving money on the table because they don't have enough compute to serve the demand.

It seems like you're arguing that the bubble is going to collapse soon, like the author? How can it collapse when the demand is so much bigger than supply? Do you think the demand is fake? Or that AI will stop making progress from here on out?

Re: How the AI Bubble Bursts

#445

Post by doom2 »

jsnell wrote: post #432
doom2 wrote: post #429

Earlier quoted context omitted.

> Bottom line is that H100 prices are near 3 year highs, A100s are still profitable to run, B200 prices are increasing, no one has enough compute. Then why aren't the hardware manufacturers of components needed by AI companies making plans yesterday to bring new fabs online to meet demand? That isn't a gotcha question, I genuinely want to know. The money involved isn't that much compared to the money changing hands b…

A new fab will need to be filled with advanced equipment like lithography machines. They are the most complex thing humanity has every built. There is one supplier of EUV lithography machines in the world, ASML. They are basically acting as an integrator for hundreds of highly specialized components manufactured to unimaginable levels of precision. Each of them has roughly one eligible supplier in the world who are o…

Sure, I didn't mean to suggest that it would be easy or fast to increase manufacturing capabilities, just that the confidence I'm seeing around AI should extend to the manufacturers (if that confidence for the future growth and success of OpenAI and Anthropic is warranted). That is, the business decision to increase RAM and GPU supply should be "easy".

Re: How the AI Bubble Bursts

#446

Post by paulddraper »

hajile wrote: post #386
paulddraper wrote: post #202

Earlier quoted context omitted.

You can rent a H100 GPU for $4/hour. [1] 300k tokens for that hour. OpenAI charges $6. Those are pessimistic assumptions. [1] https://lambda.ai/instances

Can you keep that GPU 100% saturated at least 16 hours per day every day of the week? If not, you aren't breaking even.

Note this is also assuming you

(1) Rent your GPUs.

(2) Pay list price, no volume breaks.

(3) Get only 85 tokens/sec. Realistically, frontier models would attain 200+ tokens/second amortized.

Inference is extremely profitable at scale.

Re: How the AI Bubble Bursts

#447

Post by vekker »

classified wrote: post #263
vekker wrote: post #251

Earlier quoted context omitted.

> Seriously, what value are tokens providing other than justifying layoffs Like the OP said, it's incredible how polarizing this debate is. When I read comments like yours, I feel like a significant part of the global workforce in IT must be living on another planet? Or they never really used Claude Code, Codex, OpenCode, ... intensively before because of company policies? I legitimately am at least 10x more producti…

> 10x more productive That claim is totally worthless without you providing concrete information how you measured that.

I literally wrote how I measure this in the post you are replying to: #commits which is admittedly a worthless proxy for productivity, so, more importantly, number of finished production-ready features delivered.

That number is at least tenfold of what it was before, simply because I can run a lot of gruntwork in parallel now without wasting brainpower and focus on that stuff.

Re: How the AI Bubble Bursts

#448

Post by philistine »

pigpop wrote: post #409
hajile wrote: post #379

Earlier quoted context omitted.

Demand is relative. How many Claude tokens would you buy if they had a 10x price hike? The market has achieved it's current saturation level with loss-leader prices that remind me of the Chinese bike share bubble[0]. Once those prices go up to break even levels (let alone profitable levels), the number of people who can afford to pay will go down dramatically (and that's not even accounting for the bubble pop further…

If they've already built themselves a loyal customer base (which is usually the point of fighting a price war) and the customers are happy with the technology they have, then if funding is tight and turning a profit is more important why wouldn't they pivot to optimizing inference by stopping further training, freezing the model versions, burning the weights into silicon and building better caching strategies and imp…

Easier said than done. What you're describing can take years to implement. Can OpenAI et al. keep burning cash at the same rate for two years while they wait for the salvation of custom silicon if the investments dry up?

Re: How the AI Bubble Bursts

#449

Post by aurareturn »

tartoran wrote: post #443
aurareturn wrote: post #437

Earlier quoted context omitted.

Probably. I assume the value would drastically increase. Companies will definitely continue to pay for it. It's irreplaceable now.

How about if they plateau but prices skyrocket? Most companies would pay but if you're not working for a company that does pay for it, what's the line beyhond which you'd think twice about paying for it yourself? 500? 1000? 1500?

Why would price skyrocket?

Let's say they have already plateau. But hardware continues to get better, right? So tokens should go down in price, not up. Since they're already 50%+ on inference today, better hardware would allow them to generate more tokens for less money.

I would pay $500 to start, build stuff with it, then keep going up the tiers as the stuff I'm building makes money.

Re: How the AI Bubble Bursts

#450

Post by HDThoreaun »

hajile wrote: post #379
Marha01 wrote: post #336

Earlier quoted context omitted.

Demand for top models is definitely not saturated, at least when it comes to programming. If I could afford to use 5x more Claude Opus 4.6 tokens, I would!

Demand is relative. How many Claude tokens would you buy if they had a 10x price hike? The market has achieved it's current saturation level with loss-leader prices that remind me of the Chinese bike share bubble[0]. Once those prices go up to break even levels (let alone profitable levels), the number of people who can afford to pay will go down dramatically (and that's not even accounting for the bubble pop further…

There is no evidence that labs are losing money on inference subscriptions. The labs have massive fixed costs, but as long as inference spend is higher than the datacenters they use for inference cost all they need to do to become profitable is scale up. Right now software engineers are basically the only ones actually paying for inference, the labs just need to create coding assistants for everything that are good enough that every white collar worker in the country(world?) is paying a $1000/yr subscription. Certainly theres a lot of risk, will models become commoditized and everyone switches to open models? can they actually get non software engineers to pay for inference in mass? But its not like theres no path
Post reply on HN