Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the AI Bubble Bursts

martinvol.pe

Post reply on HN
Print view View original

201–210 of 557 posts

Re: How the AI Bubble Bursts

#201

Post by ajross »

cma wrote: post #167
ajross wrote: post #89

Earlier quoted context omitted.

> Reality begs to differ Honestly you're both wrong. RAM prices spiked speculatively, and they're going down for the same reason. Market people always want to argue in fundamentals, when in practice *ALL* the high frequency components of the signal are down to a bunch of traders trying to guess where it's going in the short term. At best those guesses are informed by ground truth ("AI needs a lot of RAM!" "Sam corner…

> RAM prices spiked speculatively Didn't OpenAI buy up 40% of the capacity all at once?

No, they signed a bunch of contracts for future deliveries. That's not a supply constraint. The factories making RAM continued operating and serving their existing deliveries, and in fact they still are.

Freshman economics would say that supply is fine and that prices shouldn't move. But they did anyway. And the reason is speculation.

Re: How the AI Bubble Bursts

#202

Post by paulddraper »

nyeah wrote: post #94
paulddraper wrote: post #87

Earlier quoted context omitted.

Anthropic has said inference is profitable. That’s a biased source, but the math pencils. This is why switching to local open weight models saves a lot of money. (Even though it’s not apples to apples.)

Can you give a few penciled numbers?

You can rent a H100 GPU for $4/hour. [1]

300k tokens for that hour.

OpenAI charges $6.

Those are pessimistic assumptions.

[1] https://lambda.ai/instances

Re: How the AI Bubble Bursts

#203

Post by wongarsu »

jerojero wrote: post #188
wongarsu wrote: post #166

Earlier quoted context omitted.

I can get Kimi K2.5 inference on openrouter for about $0.5/MTok input + $2.5/MTok output, from six providers that have no moat besides efficiently selling GPU time. We can assume they are doing so at a profit (they have no incentive to do this at a loss), giving us those numbers as the cost to serve a 1T-a32b model at scale. Now we don't know the true size of any of the proprietary models, but my educated guess is th…

Companies doing foundational models need to cover the cost of training which is much more expensive than training something like kimi.

Yes. I would not consider Kimi a particularly good model relative to its size, and making a SotA model is a lot more expensive. But training costs are explicitly excluded when talking about the cost to serve tokens

Re: How the AI Bubble Bursts

#204

Post by rileymichael »

infecto wrote: post #177
hirako2000 wrote: post #117

Earlier quoted context omitted.

Tulips sales also skyrocketed. Seriously, what value are tokens providing other than justifying layoffs. Concretely. Today. Not in the speculating scenario that cardiologist could be replaced with models. We see this new trend of agentic coding, again a promise software will be written that way going forward, despite the number of fiasco already experienced when trusting a model turned bad. The use case may provide v…

I know there is a large force on HN that want to deny the value of tokens and I know it’s anecdotal but the writing is on the wall. If it’s not valuable to your workflow today it will be soon. I already have tests being written, automated hooks into bugs where an initial PR gets generated with a potential fix. It’s far from perfect but junior engineers are far less productive.

> there is a large force on HN that want to deny the value of tokens

there is an even larger force on HN that financially _needs_ the value of tokens to be inflated (so much so that bots have overwhelmed the site)

Re: How the AI Bubble Bursts

#205

Post by infecto »

drakythe wrote: post #195
paulddraper wrote: post #87

Earlier quoted context omitted.

Anthropic has said inference is profitable. That’s a biased source, but the math pencils. This is why switching to local open weight models saves a lot of money. (Even though it’s not apples to apples.)

Anthropic also recently tweaked their usage limits to discourage use during peak hours. Why would they do that if inference was profitable?

Don’t confuse inference (api usage) with the consumer plan products. When people say inference is profitable they are referring to the cost to serve a token via the API. The consumer products are absolutely a question mark on profitability and as we see with most of the business and enterprise plans, going away for pure on demand use (api cost) full time.

Re: How the AI Bubble Bursts

#206

Post by endymion-light »

NickNaraghi wrote: post #200

> Taking this into account, Google is extremely well positioned to weather the storm. When they announce capex expenditure, they don’t spend it overnight. They can simply deploy month by month until their competitors struggle to raise and get forced to capitulate. At that point they can just ramp down the spending and declare victory in a cornered market. They don’t need capex, they just need to make it very clear fo…

Exponential take-off is great until it stops- genuinely, what are the signals showing any of the large models are performing exponential takeoff and recursive self-improvement?

Currently a lot of that appears to be marketing hype to drive up usage. Is it exponential, or are the labs spending exponentially more for smaller and smaller gains from LLMs?

Re: How the AI Bubble Bursts

#207

Post by bogzz »

NickNaraghi wrote: post #200

> Taking this into account, Google is extremely well positioned to weather the storm. When they announce capex expenditure, they don’t spend it overnight. They can simply deploy month by month until their competitors struggle to raise and get forced to capitulate. At that point they can just ramp down the spending and declare victory in a cornered market. They don’t need capex, they just need to make it very clear fo…

What recursive self-improvement?

Re: How the AI Bubble Bursts

#208

Post by joshstrange »

h14h wrote: post #178
joshstrange wrote: post #53

> RAM prices are crashing because new models won’t need as much Reality begs to differ [0] and following the link for that text goes to an article [1] where they talk about Google's TurboQuant which supposedly will lower the RAM requirements. Now if that means RAM prices come down (as speculated, not reported on, in the link) or the AI companies just do more things with their extra ram is yet to be determined. The fa…

I do wonder how closely prices consumer RAM kits follow the wholesale prices for NAND chips manufacturers see internally. The pcpartpicker graphs you linked show consumer prices have leveled out and may even be starting to fall. Depending on how the economics shake out this could mean we've hit an inflection point. My personal prediction is that once the VC bill comes due and prices for frontier models starts to clim…

Consumer vs NAND is an absolutely fair distinction to make, I'm not sure how to track those prices. My main issue the article saying "RAM prices are crashing" (which I can't find any evidence of) and linking to an article that doesn't even repeat that claim, it instead just speculates that maybe RAM will come down in price due to this new idea.

> In any case, I think it's pretty fair to speculate we may be seeing RAM prices start falling sooner rather than later.

I sure hope so. RAM, HDDs, and SSDs are all crazy-high right now and I was in the market for literally all 3 but have paused all my buying because I can't justify the costs as they stand today.

Re: How the AI Bubble Bursts

#209

Post by malfist »

naravara wrote: post #113
malfist wrote: post #52

Earlier quoted context omitted.

> The cost to serve tokens is absolutely profitable today How can you possibly say that? Everyone knows that's not the case, these companies are losing money every day selling tokens. Revenue is not the same thing as profit.

I think they’re losing money because they have to amortize the costs of training the models in the first place, which is where most of the resource sink is. This is why they were freaking out about DeepSeek just taking the trained model weights and slapping an interface on it.

Thats like saying a restaurant is profitable because they're making money selling meals if you ignore the costs of ingredients.

Of course they are profitable if you ignore their cost to bring a product to market.

Re: How the AI Bubble Bursts

#210

Post by layer8 »

joshstrange wrote: post #53

> RAM prices are crashing because new models won’t need as much Reality begs to differ [0] and following the link for that text goes to an article [1] where they talk about Google's TurboQuant which supposedly will lower the RAM requirements. Now if that means RAM prices come down (as speculated, not reported on, in the link) or the AI companies just do more things with their extra ram is yet to be determined. The fa…

I agree. The article they link to talks about memory company stocks crashing, not RAM prices crashing. There is some truth to the former: https://www.ft.com/content/e4e15692-187e-4466-832e-ec267e792...
Post reply on HN