Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the AI Bubble Bursts

martinvol.pe

Post reply on HN
Print view View original

461–470 of 557 posts

Re: How the AI Bubble Bursts

#461

Post by armonster »

therealdrag0 wrote: post #419
atwrk wrote: post #20

Earlier quoted context omitted.

But are they actually profitable, or do they employ creative accounting where only parts of overhead expenses are counted against all of inference revenue, similar to what Uber did? OpenAI's numbers show that they definitely are not profitable on inference, and even worse, revenue growth scaled linearly with inference cost from 2024 to 2025, which means they can't outgrow this problem. See https://www.wheresyoured.at…

Does it matter if it’s creative accounting? Uber is a great example of a company that everyone was certain would fail because it was unprofitable and now it succeeded and is profitable.

Uber didn't have ever-increasing costs though.

Re: How the AI Bubble Bursts

#462

Post by jsnell »

doom2 wrote: post #445
jsnell wrote: post #432

Earlier quoted context omitted.

A new fab will need to be filled with advanced equipment like lithography machines. They are the most complex thing humanity has every built. There is one supplier of EUV lithography machines in the world, ASML. They are basically acting as an integrator for hundreds of highly specialized components manufactured to unimaginable levels of precision. Each of them has roughly one eligible supplier in the world who are o…

Sure, I didn't mean to suggest that it would be easy or fast to increase manufacturing capabilities, just that the confidence I'm seeing around AI should extend to the manufacturers (if that confidence for the future growth and success of OpenAI and Anthropic is warranted). That is, the business decision to increase RAM and GPU supply should be "easy".

Right, but the business decisions probably aren't the constraint at this point? (But were a year ago.)

Once the ability of the supply chain to grow has been saturated, no amount of extra confidence will make it grow faster.

Re: How the AI Bubble Bursts

#463

Post by arctic-true »

aurareturn wrote: post #444
arctic-true wrote: post #397

Earlier quoted context omitted.

I’m not talking about training costs. I’m talking about startup costs. You have to pay for GPUs (or to rent data centers). You have to pay for the electricity that runs those data centers, and in a lot of cases these frontier labs are building the data centers on credit, so you need to pay for the construction, the materials, etc. If it was as simple as “running the GPUs costs less than we charge for it,” I might be…

Right now, the demand is far more than supply for GPUs. Every cloud company is saying they're leaving money on the table because they don't have enough compute to serve the demand. It seems like you're arguing that the bubble is going to collapse soon, like the author? How can it collapse when the demand is so much bigger than supply? Do you think the demand is fake? Or that AI will stop making progress from here on…

The demand is real. The tech is real. The economics are completely unsustainable. Switching costs and barriers to entry are too low, operating costs are too high. And if the tech improves, it actually makes it even easier for competitors to swoop in and take market share. Not long ago, an agent that was 80% as good as SOTA was not usable. A year from now, an agent that is 80% as good as SOTA will be better than the best agent is today. We have it on good authority that today’s agents are very good, very useful. Why bother paying full price?

This is deeply ironic in a way. Because the whole premise of AI labor replacement is that AI does not need to be better than human labor, it just needs to be cheaper with acceptable performance. But the same is true one step down: discount AI doesn’t need to be better than bleeding-edge AI, it just needs to be cheaper with acceptable performance.

Re: How the AI Bubble Bursts

#464

Post by hbn »

s1artibartfast wrote: post #430
hbn wrote: post #394

Earlier quoted context omitted.

If I saw a helicopter crashed into a tree, I don't have to be a helicopter pilot to know it's not an ideal state of a helicopter and something/some people failed. When I'm using MS Word and it takes 20 seconds to cold launch on a machine that's magnitudes faster than any computers 25 years ago where it launched near instantly, I can tell something is going wrong. When all of their software is harassing me to use AI i…

your comment sums up the conflict. I dont know if you noticed, but there was a shifting of the goal post from "sub-par" to something wrong/sub-optimal. The best helicopter you can buy may in fact crash into trees sometimes.

Microsoft's products do not occasionally fail, they're constantly going out of their way to block users from doing basic tasks through ads and dark patterns. It makes some KPI go up so some asshat product manager can get a promotion, and they never lose users because 99% of their users are hostages.

Re: How the AI Bubble Bursts

#465

Post by aurareturn »

paulddraper wrote: post #446
hajile wrote: post #386

Earlier quoted context omitted.

Can you keep that GPU 100% saturated at least 16 hours per day every day of the week? If not, you aren't breaking even.

Note this is also assuming you (1) Rent your GPUs. (2) Pay list price, no volume breaks. (3) Get only 85 tokens/sec. Realistically, frontier models would attain 200+ tokens/second amortized. Inference is extremely profitable at scale.

Assuming 80GB H100 and you inference a model that is MoE and close to the size of the 80GB VRAM, you're going to see around 10k tokens/second fully batched and saturated. An example here might be Mixtral 8x7B.

You're generating about 36 million tokens/hour. Cost of Mixtral 8x7b on Open router is $0.54/M input tokens. $0.54/M output tokens.

You're looking at potentially $38.88/hour return on that H100 GPU. This is probably the best case scenario.

In reality, inference providers will use multiple GPUs together to run bigger, smarter models for a higher price.

Re: How the AI Bubble Bursts

#466

Post by raincole »

Aperocky wrote: post #85
Chance-Device wrote: post #18

From the beginning of this I’ve wondered the same question: how do these companies justify spending such massive amounts now (and 3 or 4 years ago) when software and hardware efficiencies will bring down the cost dramatically fairly soon? They basically decided that scaling at any cost was the way to go. This only works as a strategy if efficiency can’t work, not if you simply haven’t tried. Otherwise, a few breakthr…

The decision is the right one. Scaling at any cost is the right way to go . You cannot find the efficiency if you haven't been experimenting at scale, this is true personally as well. If someone haven't been burning a few B tokens per month, everything coming out of their mouth about AI is largely theory. It could be right or wrong, but they don't have the practice to validate what they're talking about. Not everyone…

Well said. Quantity itself is a quality.

In the worst of the worst case, they're building know-how of how to manage big datacenters, infra and data-labeling teams. These are incredibly valuable in the next few years. And no, no one, even the AI companies' executives themselves, believe that you can delegate business know-how to LLMs.

Re: How the AI Bubble Bursts

#467

Post by checkyoursudo »

HDThoreaun wrote: post #451
pydry wrote: post #281

Earlier quoted context omitted.

>The typical person is using LLMs not at all as it pertains to their daily life tasks. This doesnt track at all with my experience. Everybody is using it everywhere. Moreover people are using them for daily life tasks even when it is not an appropriate use of LLMs - e.g. getting medical advice as you referred to or writing emails which are clearly pissing off their coworkers. In this respect I see it as akin to radiu…

> getting medical advice Id be careful stating this is an inappropriate use of LLMs. Im semi tapped in to the medical literature community and there is a lot of serious discussion and research going into the usage of LLMs for medical advice and most of it is showing that LLMs are barely worse than doctors, and much much cheaper/more convenient. They definitely arent ready to completely replace doctors, but it seems t…

This seems ripe for a joke akin to "how was the food?" "bad, but at least the portions were big!"

Like, "how was the medical advice" "worse than a doc's, but at least it was cheaper!"

Re: How the AI Bubble Bursts

#468

Post by zozbot234 »

jqpabc123 wrote: post #7

Another possibility not really addressed here --- local LLMs. AI on hardware you own and control --- instead of a metered service provider. In other words, a repeat of the "personal computing" revolution but this time focused on AI. TurboQuant could be a key step in this direction.

TurboQuant helps KV quantization which is not very relevant to local LLMs, since context size becomes most relevant when you run inference with large batches. For small-scale inference, weights dominate. (Even if you stream weights from SSD, you'll want to cache a sizeable fraction to get workable throughput, and that dominates your memory usage.)

Re: How the AI Bubble Bursts

#469

Post by HDThoreaun »

checkyoursudo wrote: post #467
HDThoreaun wrote: post #451

Earlier quoted context omitted.

> getting medical advice Id be careful stating this is an inappropriate use of LLMs. Im semi tapped in to the medical literature community and there is a lot of serious discussion and research going into the usage of LLMs for medical advice and most of it is showing that LLMs are barely worse than doctors, and much much cheaper/more convenient. They definitely arent ready to completely replace doctors, but it seems t…

This seems ripe for a joke akin to "how was the food?" "bad, but at least the portions were big!" Like, "how was the medical advice" "worse than a doc's, but at least it was cheaper!"

Well the thing is that it often isnt worse than a doctor's, thats the point of the research here. I get that sounds crazy, just watch out for the coming literature I guess.

A significant portion of americans detest the medical industry and deeply dislike going to the doctor so I dont even think the product needs to be very good to disrupt the way the system works, just different and accessible is likely enough. Funnily enough, restaurants where the food is bad but the portions are big are actually decently popular. Priorities can vary so widely that many people are unable to even comprehend the priorities a significant number of people truly hold.

Re: How the AI Bubble Bursts

#470

Post by jrflowers »

HDThoreaun wrote: post #451
pydry wrote: post #281

Earlier quoted context omitted.

>The typical person is using LLMs not at all as it pertains to their daily life tasks. This doesnt track at all with my experience. Everybody is using it everywhere. Moreover people are using them for daily life tasks even when it is not an appropriate use of LLMs - e.g. getting medical advice as you referred to or writing emails which are clearly pissing off their coworkers. In this respect I see it as akin to radiu…

> getting medical advice Id be careful stating this is an inappropriate use of LLMs. Im semi tapped in to the medical literature community and there is a lot of serious discussion and research going into the usage of LLMs for medical advice and most of it is showing that LLMs are barely worse than doctors, and much much cheaper/more convenient. They definitely arent ready to completely replace doctors, but it seems t…

> barely worse than doctors

I like that this comment is below, and posted after, an example where somebody had to pay extra money to clear up a misdiagnosis of stage 4 cancer by the “barely worse” software

Post reply on HN