Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uber's $1,500/month AI limit is a useful signal for AI tool pricing

simonwillison.net

Post reply on HN
Print view View original

491–500 of 819 posts

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#491

Post by jmalicki »

jmalicki wrote: post #361
HumanOstrich wrote: post #299

Earlier quoted context omitted.

We aren't talking about insect identification models from 2019.

What do you think are running on the T4 GPUs in AWS? A lot of the use cases I know of for them are mid-level computer vision models that don't need to be frontier level.

I can no longer edit this, but want to expand on my comment.

I've seen those vision researchers want to train on H100s at the time and being told know, wait for the T4s.

I've seen T4s running BERT models for document classification.

When there are enough Blackwells in data centers that H100s are useless for inference by your standards (I don't know if we've arrived there or not yet), there will be people who, say, want to run the Taco Bell ordering chatbot on them. There will be people who have applications that are just fine with Qwen 2.5 who will be happy renting them.

There seems to be this crazy consensus that hyperscalers are going to go into their datacenters and throw away their old GPUs. The reality is they have a ton of paying customers for them.

And there may be insect identification apps from 2019 that say "you know what? H100s have gotten cheap enough I can use a VLLM so the user can describe where they saw the insect too", or the McDonald's website support chatbot developers say "Hey, the bigger cheapers have gotten cheap enough we can upgrade our models to Qwen 2.5".

The frontier level GPUs in e.g. AWS have a huge premium. When the newer generations come out, they will be able to cut prices to a bit of a premium over the operational costs and still make a profit, and there are a ton of down-market customers who will be interested, who aren't willing to try to outbid Anthropic for Blackwells.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#492

Post by garrickvanburen »

pchristensen wrote: post #456
gravypod wrote: post #404

Earlier quoted context omitted.

If you had a business task to complete that was only possible with ai and it cost you >$1500/month of work, how long would you have to delay the task so that it's cheaper long run to buy hardware and do local models? $1,500/mo * 14 months = $21,000. If local models are 14mo behind as many in HN say it may be profitable to just wait. Maybe just spend a few hundred dollars of your tokens and buy hardware piece by piece…

There's a lot of opportunity cost to waiting 14 months to build something.

I agree, outside of the AI bubble, there's a lot of wait-and-see happening in the B2B world right now, I'd say we're currently 6-8 months into that 14 months.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#493

Post by OptionOfT »

scubbo wrote: post #429
OptionOfT wrote: post #206

Earlier quoted context omitted.

Because companies are betting that this spending will allow them to reduce cost by firing people. Right now the AI LLM PRs we're seeing are just introducing more work for other people, while these so-called builders are looking good with their new dashboards and functionality they're demoing. But you can't talk to them about the flow of the code. You can't ask them for their thinking as to why certain things are. It'…

> But you can't talk to them about the flow of the code. You can't ask them for their thinking as to why certain things are. There are plenty of valid criticisms or warnings about over-reliance on AI coding, but this is not one of them. Today, I am using a semi-autonomous agentic coding system which has an `interview` functionality built in - when it spits out the PR from the input, if you have questions about the mo…

Sorry, I meant interviewing the PR author for certain choices.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#494

Post by ptero »

hansvm wrote: post #447
munk-a wrote: post #392

Earlier quoted context omitted.

My usual rule of thumb for the US is north of double the received compensation but something in that range sounds reasonable with such high compensation. It's actually really interesting and underappreciated how that fully-loaded cost varies from country to country. Canada (for most salary ranges) is about half again instead of double owing to the insurance portion coming out of income tax rather than being a hidden…

Why would double be a good rule of thumb for typical US SWEs? Most of the costs aren't proportional to salary, and the ones which are aren't anywhere approaching 50%, much less double.

While the fully burdened cost of an engineer being double his salary sounds suspicious, this is indeed broadly the case. It has been (sometimes significantly) more than double in the case in every US employer where I worked and where I saw both numbers. In one case it was a hair under 3x.

My experience was not with pure software houses; we had some labs, measurement and RF equipment, but even without the hardware component the offices, insurance, admin expenses, HR, janitors, conference travel and so on would easily bump the total employee cost to double the salary. My 2c.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#495

Post by eclipticplane »

danans wrote: post #469
CharlieDigital wrote: post #4

$1500/mo is $18,000/seat/annum. Maybe Microsoft and Nvidia are on to something. 128 GB machines that can run local LLMs are a bargain even if priced $5-8k. Yes, tok/s is not quite there, but that's probably OK since the bottleneck really isn't the code; it's WTF did Uber build with all of that spend? How did it meaningfully impact their revenue in a positive direction?

> How did it meaningfully impact their revenue in a positive direction? It probably allowed them to avoid hiring as many people to build a certain amount of software. Even if it didn't increase revenue, it could have lowered human labor costs. > 128 GB machines that can run local LLMs are a bargain even if priced $5-8k. Don't forget the energy costs. Searching around, advanced models use an average of 25 Wh/1000Tok.…

How much more software does Uber need?

Unless they are iteratively replacing expensive vendors and optimizing other headcount costs?

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#496

Post by asadotzler »

LAC-Tech wrote: post #357
tuesdaynight wrote: post #138

Why there are so many people that still believe that AI coding is a fad? It's something that started less than two years ago and companies are already paying thousands per seat. I know one that gives you 5k per month. Which other tool went from nothing to this level of acceptance so quickly?

Because we have spent a lot of time and money using AI to generate code and have been unimpressed with the results. As for why they got accepted so quickly 1) the industry's long running desperation to deskill computer programming 2) the addictive psychology baked into LLMs "That's an elegant solution! Shall I ... ?"

Also, a bucket for VC to put all that NFT, IoT, blockchain, VR investment into. VCs gonna VC and the last 15 years of bets failed so the last few years have been a transition away from those toward "the next thing".

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#497

Post by [deleted] »

dgellow wrote: post #170
CharlieDigital wrote: post #4

$1500/mo is $18,000/seat/annum. Maybe Microsoft and Nvidia are on to something. 128 GB machines that can run local LLMs are a bargain even if priced $5-8k. Yes, tok/s is not quite there, but that's probably OK since the bottleneck really isn't the code; it's WTF did Uber build with all of that spend? How did it meaningfully impact their revenue in a positive direction?

You’re way better to run your own on premise models. Laptops are depreciating assets, do not benefit from economy of scale, have fixed specs, result in a fragmented fleet where you need to keep models up to date. Without talking about power consumption and cooling issues. I really don’t see why companies would go that direction

[deleted]

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#498

Post by NichoPaolucci »

simonw wrote: post #487
techblueberry wrote: post #482

Earlier quoted context omitted.

All of the articles and CFO’s saying so, and companies like Uber cutting back on AI spend.

Uber cutting back to ~$1,500/engineer/tool/month makes it look to me like they think there's at least $1,500 of monthly ROI to be had per engineer.

1500/Mo per engineer is such a small price considering the base salary of these employees, Maybe Uber knows something we don't (the 5X engineering ROI isn't there for them?).

Judging the ROI of an engineer is hard. Adding AI on top of that makes things worse, I think. I've heard AI makes engineers 3X, 5X, 10X and even 100X.

If I told my CEO that I was 4X more effective with AI, I am doubtful he would be willing to spend even 1X my salary on tokens. Even though he would be making out in the end.

At some point the ROI is pretty much vibes, man.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#500

Post by fooker »

bijowo1676 wrote: post #171
dgellow wrote: post #164

Earlier quoted context omitted.

One aspect Paul Kedrosky mentioned recently is the concept of „duration mismatch“. The price per token goes down over time (either because the AI vendor reduces due to competition pressure, or because customers are now incentivized to use older cheaper models). But datacenters are financed through debt, with the assumption their revenue increases over time. Quoting him: „[AI vendors are] paying for a fixed cost with…

do GPU chips really depreciate physically? There are no moving parts, I dont think memory chips or GPU chips deteriorate naturally. I think its only accounting depreciation. I have been using my laptop for a decade, what is stopping datacenters from using the purchased GPU chips for a decade?

When it was profitable to mine crypto with GPUs people used to sell these miner GPUs on the used market after about two years.

These were about half of the cost of an used GPU just used for gaming. By that pricr, I'd say a GPU kept busy has twice as high a chance of failure after two years of use.

Not great, not terrible.

Post reply on HN