Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the AI Bubble Bursts

martinvol.pe

Post reply on HN
Print view View original

161–170 of 557 posts

Re: How the AI Bubble Bursts

#161

Post by Tade0 »

infecto wrote: post #30

It’s incredible how polarizing the AI rush is. I keep the perspective that the technology is an absolute step change but I have no idea where the cards will fall. I take a lot of issue with these style of articles. I get a sense that the authors are being overly defensive. The cost to serve tokens is absolutely profitable today and that’s been true for at least a year. What’s unclear is how R&D and capex fit into the…

My main worry is - once this is all over, the market consolidates and using LLMs will become a requirement in job listings, what's the highest price per million tokens companies will be able to charge us?

Currently on a given day I'm chewing through approximately the equivalent of my lunch money, but where there's opportunity to extract wealth, someone will find a way to do it.

Re: How the AI Bubble Bursts

#162

Post by fcarraldo »

chrisweekly wrote: post #100
infecto wrote: post #30

It’s incredible how polarizing the AI rush is. I keep the perspective that the technology is an absolute step change but I have no idea where the cards will fall. I take a lot of issue with these style of articles. I get a sense that the authors are being overly defensive. The cost to serve tokens is absolutely profitable today and that’s been true for at least a year. What’s unclear is how R&D and capex fit into the…

> "decades of overinflated engineering salaries" 'Overinflated' relative to what? You make some good points but I don't accept this as a premise.

Well, not GP, but I do. Let’s look at the numbers:

Median senior SWE salaries in SF: https://www.levels.fyi/t/software-engineer/levels/senior/loc...

Median income in metro areas: https://www.cnbc.com/2024/07/11/the-median-salary-for-the-25...

Engineering salaries are significantly higher than nearly every other industry on average and on median. Much of this is driven by VC funding rather than sound, profitable, bootstrapped businesses with sustainable profit margins.

Engineering salaries have also been driven upwards significantly the past ~10 years (since the post-2008 crash recovery), while wage growth in the US is mostly stagnant. I don’t have a source handy for that, but there are plentiful studies.

Outside of the US this may be less true, but I took GP’s “most of us on HN” to mean people who work in US tech companies which are primarily concentrated in high COI areas.

Re: How the AI Bubble Bursts

#163

Post by wavemode »

baq wrote: post #95
schnitzelstoat wrote: post #28

It's a winner-takes-all market and everyone wants to be the next Google and not the next Lycos or AskJeeves etc. It'd be interesting to see what they spend all the money on though as we seem to be hitting diminishing returns and I'm not sure if the typical enterprise user really cares about small improvements on benchmarks. It seems like it'd probably be better to spend all that on marketing, free trials, exclusivity…

> It's a winner-takes-all market and everyone wants to be the next Google absolutely isn't! if billed per token, there is no reason to be married to a single model family provider at all. the models have very different strengths and weaknesses, you should be taking advantage of this at all times.

people used to say this about search engines and web browsers, as well

regardless, eventually Google became the universal default for both. When it comes to software, the average person doesn't shop around for the technologically optimal choice, they just use what everyone else is using.

Re: How the AI Bubble Bursts

#164

Post by [deleted] »

amelius wrote: post #97
joshstrange wrote: post #53

> RAM prices are crashing because new models won’t need as much Reality begs to differ [0] and following the link for that text goes to an article [1] where they talk about Google's TurboQuant which supposedly will lower the RAM requirements. Now if that means RAM prices come down (as speculated, not reported on, in the link) or the AI companies just do more things with their extra ram is yet to be determined. The fa…

> Now if that means RAM prices come down (as speculated, not reported on, in the link) or the AI companies just do more things with their extra ram is yet to be determined. I think it is determined: https://en.wikipedia.org/wiki/Jevons_paradox

[deleted]

Re: How the AI Bubble Bursts

#165

Post by infecto »

SirensOfTitan wrote: post #56
infecto wrote: post #30

It’s incredible how polarizing the AI rush is. I keep the perspective that the technology is an absolute step change but I have no idea where the cards will fall. I take a lot of issue with these style of articles. I get a sense that the authors are being overly defensive. The cost to serve tokens is absolutely profitable today and that’s been true for at least a year. What’s unclear is how R&D and capex fit into the…

This is a classic HN mistaking the map for the territory. R&D and capex absolutely figure into de-facto profitability and sustainability for AI labs, despite their separate treatment in accounting. > well most of us here on HN have benefited from decades of overinflated engineering salaries being paid by often companies that were not profitable and not only unprofitable This is a really concerning perspective: people…

Oh come on there are no “classic HN mistakes” here. Inference is profitable but bottom line is not yet. This is a very young industry and unlike those of the past, it’s much easier to picture a possibility of profitability. It’s absolutely different in that the marginal cost scales linear but solving for the R&D portion of a product where supply cannot keep up is a lot easier than some SaaS where the underlying product is not being used.

The salary jab was probably a little harsh.

Your ending is a bit of a fizzle too. There are many capex intense businesses that do just fine.

Re: How the AI Bubble Bursts

#166

Post by wongarsu »

boriskourt wrote: post #48
infecto wrote: post #30

It’s incredible how polarizing the AI rush is. I keep the perspective that the technology is an absolute step change but I have no idea where the cards will fall. I take a lot of issue with these style of articles. I get a sense that the authors are being overly defensive. The cost to serve tokens is absolutely profitable today and that’s been true for at least a year. What’s unclear is how R&D and capex fit into the…

> The cost to serve tokens is absolutely profitable today and that’s been true for at least a year. > For the data center build outs, demand for tokens is still exceeding supply. Can you provide any numbers for this?

I can get Kimi K2.5 inference on openrouter for about $0.5/MTok input + $2.5/MTok output, from six providers that have no moat besides efficiently selling GPU time. We can assume they are doing so at a profit (they have no incentive to do this at a loss), giving us those numbers as the cost to serve a 1T-a32b model at scale.

Now we don't know the true size of any of the proprietary models, but my educated guess is that Sonnet is in about the same parameter range, just with better training and much better fine tuning and RLHF. Yet API pricing for Sonnet is $3/MTok input + $15/MTok output, exactly six times as expensive. Even Haiku is twice as expensive as Kimi K2.5.

I find it difficult to believe in a world where those API prices aren't profitable. For subscription pricing it's harder to tell. We hear about those that get insane value out of their subscription, but there has to be a large mass who never reaches their limits. With company-wide rollouts there might even be a lot of subscription users who consume virtually no tokens at all.

Re: How the AI Bubble Bursts

#167

Post by cma »

ajross wrote: post #89
joshstrange wrote: post #53

> RAM prices are crashing because new models won’t need as much Reality begs to differ [0] and following the link for that text goes to an article [1] where they talk about Google's TurboQuant which supposedly will lower the RAM requirements. Now if that means RAM prices come down (as speculated, not reported on, in the link) or the AI companies just do more things with their extra ram is yet to be determined. The fa…

> Reality begs to differ Honestly you're both wrong. RAM prices spiked speculatively, and they're going down for the same reason. Market people always want to argue in fundamentals, when in practice *ALL* the high frequency components of the signal are down to a bunch of traders trying to guess where it's going in the short term. At best those guesses are informed by ground truth ("AI needs a lot of RAM!" "Sam corner…

> RAM prices spiked speculatively

Didn't OpenAI buy up 40% of the capacity all at once?

Re: How the AI Bubble Bursts

#168

Post by ToucanLoucan »

adjejmxbdjdn wrote: post #141
joshstrange wrote: post #53

> RAM prices are crashing because new models won’t need as much Reality begs to differ [0] and following the link for that text goes to an article [1] where they talk about Google's TurboQuant which supposedly will lower the RAM requirements. Now if that means RAM prices come down (as speculated, not reported on, in the link) or the AI companies just do more things with their extra ram is yet to be determined. The fa…

I’m not disagreeing with you, but consumer RAM prices are lagging indicators. If commercial RAM prices are dropping then consumers will see those price drops last, especially given the fact that several consumer manufacturers turned to commercial only.

If they see them. Plenty of businesses are still charging pandemic prices for all kinds of goods and simply pocketing the difference.

Cars come to mind instantly. Prices exploded in 2020/1, due to legitimate shortages, most of which have been plus or minus resolved, but the prices for new (and used!) cars never came back down.

Re: How the AI Bubble Bursts

#169

Post by WarmWash »

Aperocky wrote: post #58
infecto wrote: post #30

It’s incredible how polarizing the AI rush is. I keep the perspective that the technology is an absolute step change but I have no idea where the cards will fall. I take a lot of issue with these style of articles. I get a sense that the authors are being overly defensive. The cost to serve tokens is absolutely profitable today and that’s been true for at least a year. What’s unclear is how R&D and capex fit into the…

Demand of tokens is absolutely skyrocketing. And unlike the traditional "this will replace humans right away", I think what this introduce is a lot of incentive to spread those token in places where there was never any incentive to hire a software engineer for previously. In turn, that will drive a lot of business activity in those area that will potentially fail given the current quality of the output. This feels li…

>potentially fail given the current quality of the output.

The question is how big the fail is if you measure it in 3 month increments going back to late 2022.

Re: How the AI Bubble Bursts

#170

Post by mcv »

Aurornis wrote: post #60

This article tries to build upon a lot of half-truths or incorrect facts, like this: > OpenAI is struggling to monetize. They turned to showing ads in ChatGPT, The ads aren’t going into your paid plans (except maybe a highly discounted tier, depending on the market). The ads are a play to offer a free version. Having an ad-supported free tier isn’t new. The discussion about being unprofitable also repeats the reducti…

I've heard "They're losing money" since the 1990s. About Amazon and nearly every other tech company.

The strategy is always:

* Build something useful

* Give it away for free to get people exited

* Convince investors that this is going to rule the world

* Grow to dominate the world

* Enshittify

Post reply on HN