Live data from Hacker News

Uber's $1,500/month AI limit is a useful signal for AI tool pricing

simonwillison.net

321–330 of 819 posts

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#321

Earlier quoted context omitted.

An inference only platform selling good open weight model inference without the research overhead could capture a-lot of market for lower size model uses (haiky, gemeni flash). Diffusion-transformers and clever cashing can drop inference even lower, which is improving at a high rate. The biggest reason large models are un-attainable for local applications is the lack hardware with large amount of unified/graphics mem…

AI may get so commoditized for certain use cases that you will not even be able sell inference at a profit. AI might be bundled in with other services, just like cursor bundles in their own AI model for auto complete with their editor. I.e. cameras might have AI for image recognition bundled in etc.

Agreed, this is where google is really, really set up to win the market. They can combine gemini subscription with a moderately more expensive google workspace and steal MSFTs entire $50 billion enterprise productivity software market. MSFT is quickly trying to get copilot in a good enough state but without TPUs I think itll be tough for them to serve a good enough model at a price people will accept.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#322
Token costs rising because data center build costs must be paid down.. is not the whole picture. It is actually possible for token costs to fall despite the spending frenzy.

Naively you’d expect to always keep paying more - but growth in token usage is what changes the equation. Amortizing debt over an exponentially growing amount of spend across a growing customer base (not per customer) lets the debt be paid off & costs covered even as each individual’s spend stays steady or even goes down - but it only works if there’s growth beyond some threshold that makes the whole thing hang together. No one on the outside knows how much growth that is, and everyone chases maximum growth.

Jevons Paradox ends up being your friend as well as the friend of the inference providers as well as the friend of the inference financiers.

If it’s a strong enough effect, it has potential to cancel out all the circular financing too, and let everyone ride out the bursting of the bubble.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#323

$1500/mo is $18,000/seat/annum. Maybe Microsoft and Nvidia are on to something. 128 GB machines that can run local LLMs are a bargain even if priced $5-8k. Yes, tok/s is not quite there, but that's probably OK since the bottleneck really isn't the code; it's WTF did Uber build with all of that spend? How did it meaningfully impact their revenue in a positive direction?

Your last question is really important. What did they accomplish with all that spend? I suspect there’s some mass delusion with respect to actual accomplishments as a result of LLM use. Sure, things are moving faster, but does it matter?

[dead]

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#324

Earlier quoted context omitted.

Let's be real. Most of the time you ask an LLM "Why did you do it like this?", it responds with something along the lines of "Oops. My bad. You're right to point this out." You even have a fair chance of getting a response like that when there isn't anything wrong and the question wasn't rhetorical - which perfectly illustrates the level of the genuine understanding LLMs operate at.

When you criticize AI, always remember that the alternative is the average employee. Today's models are pretty good.

A lot of people think they're above average. A lot of them are wrong.

A lot of average people are producing gigantic messes. At least previous to this they were gated by their mediocrity.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#326

Earlier quoted context omitted.

One aspect Paul Kedrosky mentioned recently is the concept of „duration mismatch“. The price per token goes down over time (either because the AI vendor reduces due to competition pressure, or because customers are now incentivized to use older cheaper models). But datacenters are financed through debt, with the assumption their revenue increases over time. Quoting him: „[AI vendors are] paying for a fixed cost with…

"So you have on one end the token revenue trending down, on the other end the training cost going up for the next frontier models, and you need to pay back your 10y debt." Not necessarily, the bond holders could simply take a massive hair cut and lose shitloads of money. On the topic of bubbles and exuberance, Jeff Bezos made the salient point that there was a massive over-invested biotech boom in the 1990s and tons…

In order to not un-build the data centers, they at least have to make more than it costs to operate them, and also not have some attractive liquidation value (the land, maybe).

I could imagine something like “inference is done at home or in China, that’s the price to beat” and it’s not worth keeping all those GPUs cool out in Nevada.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#327
post #311

Earlier quoted context omitted.

probably will get fired for lack of performance.

Let's just say their performance (OKR, KPI, whatever "impact" metric you want) was indistinguishable from a peer that used the AI/LLM monthly allowance in full. Maybe a $10k raise would be nice?

Theyd get a bad review for leaving performance on the table. When has finishing your work ever resulted in anything other than more work?

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#328
Lock-in / switching costs are increasingly concerning me. I am using Claude for a good year now and have been accumulating so much "knowledge" in there by now. If Claude became less favorable in terms of price/performance in the future, that would worry me. I've started to think about a distributed solution, where my storage is detached from the inference, but currently Claude is still the way to go for me. Wondering if anyone has similar concerns?

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#329

Earlier quoted context omitted.

oh come on, a paid shill? Simon is very fascinated by AI and at times he can be a little too optimistic but he is generally balanced and his perspective evolves over time which can be seen in his writing. Nerd who loves nerd things a little too much? Sure. Paid shill by Big LLM? Nah.

Yes, a paid shill. You can find a clear point in time where he shifted from sceptic to 1000% fully onboard non-stop praise, with no reason.

Maybe the reason is because he thought the tools became really powerful?

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#330
post #326

Earlier quoted context omitted.

"So you have on one end the token revenue trending down, on the other end the training cost going up for the next frontier models, and you need to pay back your 10y debt." Not necessarily, the bond holders could simply take a massive hair cut and lose shitloads of money. On the topic of bubbles and exuberance, Jeff Bezos made the salient point that there was a massive over-invested biotech boom in the 1990s and tons…

In order to not un-build the data centers, they at least have to make more than it costs to operate them, and also not have some attractive liquidation value (the land, maybe). I could imagine something like “inference is done at home or in China, that’s the price to beat” and it’s not worth keeping all those GPUs cool out in Nevada.

But the parent comment was that one of the bigger costs in these data centers was the interest expense on the borrowed money. A restructuring removes or heavily reduces that amount.

The fiber laid during the dotcom bubble never paid back the investors or lenders, but it's still profitably connecting customers all these years later.

Post reply on HN