Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uber's $1,500/month AI limit is a useful signal for AI tool pricing

simonwillison.net

Post reply on HN
Print view View original

261–270 of 819 posts

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#261

Post by rafterydj »

threetonesun wrote: post #243
dghlsakjg wrote: post #210

Earlier quoted context omitted.

The Dotcom bubble is an interesting comparison. The general thrust that everything would be online was correct, it was just that the market mistimed and misallocated of capital by a decade or more. There was massive spending on infrastructure capacity that we wouldn't end up needing until the 2010s. There were hype driven valuations completely disconnected from business fundamentals just because a company was an 'int…

The question you always have to ask is what problems does it directly solve. I personally think most of the current problems in software development and really the world at large are not time-bound problems but alignment issues, and all an LLM can really do there is be some 3rd party oracle that gives you an answer without needing other humans to agree with you.

I agree with you. I think that if we're talking about actual reliable problem solving, we have to be discussing robotic / drone systems. Software is as complex as you want to make it, and always has been.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#262

Post by bgnn »

bijowo1676 wrote: post #171
dgellow wrote: post #164

Earlier quoted context omitted.

One aspect Paul Kedrosky mentioned recently is the concept of „duration mismatch“. The price per token goes down over time (either because the AI vendor reduces due to competition pressure, or because customers are now incentivized to use older cheaper models). But datacenters are financed through debt, with the assumption their revenue increases over time. Quoting him: „[AI vendors are] paying for a fixed cost with…

do GPU chips really depreciate physically? There are no moving parts, I dont think memory chips or GPU chips deteriorate naturally. I think its only accounting depreciation. I have been using my laptop for a decade, what is stopping datacenters from using the purchased GPU chips for a decade?

Chips age and fail with age. You can check hot-carrier injection, bias-temperature instability and electromigration as they are the main aging mechanisms. All if these are a linear function of time but exponentieal of temperature. 90-100C these chips are running at are really tough, so they are likely to fail at couple of percent to 10% range in 2-3 years depending on the margins they have in the design.

The solder joints are notorious to fail at a high rate too.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#263

Post by hibikir »

SlinkyOnStairs wrote: post #64
darkwater wrote: post #24

Earlier quoted context omitted.

> it's WTF did Uber build with all of that spend? You can ask the same for the median 330k salary in the US for Uber Engineering... and being a bit snarky, attending Uber engineers talks here and there at a few conferences, looks like. they love to (re)invent internal tooling/platforms. That's pretty expensive on its own. EDIT: I'm not saying that Uber's engineers didn't add value to the company, they absolutely did…

> You can ask the same for the median 330k salary in the US for Uber Engineering People DO . It's well known that most tech companies are ran incompetently. As you say, it's not the engineers' fault. But most projects and hiring in these companies exists to juice promotion criteria. And that, depending on perspective, these companies are either massively overstaffed or massively underproductive. The comparison to AI…

The massive misalignment in large companies is no secret. But neither is the fact that when someone comes to cut, they also have no idea of who is doing load bearing work that matters, and who doesn't. I look at recent cuts around my large corp, and it's clear they are made at levels that have no visibility of the ground, and are uninterested in said visibility. Obvious mistakes that are worse than what claude would have told you (yes, I asked Claude to pretend to make the budget cuts in our org y looking at the same data an exec could probably get. They were better than what happened)

I think it's a general problem, but in my rare conversations with execs nowadays, they seem rather uninterested in improving their decision making there. The actual performance of the organization does not appear to be all that relevant to them.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#264

Post by pianopatrick »

aDyslecticCrow wrote: post #161
SecretDreams wrote: post #120

Earlier quoted context omitted.

> Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? I genuinely do not know how prices can get lower from the current major providers in NA without the whole market collapsing. Everyone is spending copious amounts of money to presumably make more money back.

An inference only platform selling good open weight model inference without the research overhead could capture a-lot of market for lower size model uses (haiky, gemeni flash). Diffusion-transformers and clever cashing can drop inference even lower, which is improving at a high rate. The biggest reason large models are un-attainable for local applications is the lack hardware with large amount of unified/graphics mem…

AI may get so commoditized for certain use cases that you will not even be able sell inference at a profit. AI might be bundled in with other services, just like cursor bundles in their own AI model for auto complete with their editor. I.e. cameras might have AI for image recognition bundled in etc.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#265

Post by tmp10423288442 »

dghlsakjg wrote: post #216
thewebguyd wrote: post #166

Earlier quoted context omitted.

No disagreement on computing 2.0, but companies spending 3-5k per employee for hardware isn't generally a monthly cost. It's a at the time of hire, and then once every 3 to 5 years after that, for a monthly amortized cost of about $50/employee. I have my concerns with current inference pricing in that there's a non-zero possibility for a rug pull in the future for the subscription plans for organizations and individu…

Every employee doesn't need $1k in token spend per month, either. That kind of spend makes sense for technical workers in r+d. Most other workers are served fine by $20-30 worth of tokens on a budget model. You don't need Opus to help support write emails.

No, but you do want Opus-tier models to do desktop and office software automation (think about people who intensely use Excel and the like). Actually those might take even more tokens that coding in a lot of cases. Why do you think Claude Cowork is successful, and why do you think Codex is leaning so hard into Computer use?

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#266

Post by esikich »

john01dav wrote: post #236

Why isn't self hosting (even just renting a GPU server, not necessarily on premise) at large companies or hosting via something like together AI to run the open weight models not more common? I've tried the open weight models and the premium models like Opus and Gemini Pro, and I find that the latter are a little better, but not nearly to the degree to justify the extreme price difference, since the differences large…

Why do you think it would be more common? The pooling of GPUs to serve multiple users and connecting to docs/datalakes while respecting security controls, as a start, is non-trivial. You'd end up paying a team to manage that.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#267

Post by saulpw »

datsci_est_2015 wrote: post #248
saulpw wrote: post #239

Earlier quoted context omitted.

> But you can't talk to them about the flow of the code. You can't ask them for their thinking as to why certain things are. You can absolutely do this. It's even right most of the time.

I believe the “them” the OP was talking about was referring to the people opening the PRs, not the LLMs.

My mistake, that is definitely a different scene.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#268

Post by edent »

tmp10423288442 wrote: post #260
lbrito wrote: post #237

Earlier quoted context omitted.

That's just a non sequitur. "companies are already paying thousands per seat" has zero correlation with something being a fad or not. There are much more reasonable rationales explaining why companies are acting the way they are than "because AI coding is not a fad"

Can you name a service that charged companies thousands/seat/month that turned out to be almost or completely useless? There's lots of random services sold to corporates that are not very useful (all the random benefits besides health care, life insurance, and other big-ticket items), but the per-seat charge of those is much smaller.

Google Jam Board (and other digital whiteboards) had high upfront capex and lowish opex. Probably close to the price for how often they were used before being killed off.

Same with the MS surface(?) tables (not tablets). I saw load of companies buy into the hype and then discard.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#269

Post by chmod775 »

saulpw wrote: post #239
OptionOfT wrote: post #206

Earlier quoted context omitted.

Because companies are betting that this spending will allow them to reduce cost by firing people. Right now the AI LLM PRs we're seeing are just introducing more work for other people, while these so-called builders are looking good with their new dashboards and functionality they're demoing. But you can't talk to them about the flow of the code. You can't ask them for their thinking as to why certain things are. It'…

> But you can't talk to them about the flow of the code. You can't ask them for their thinking as to why certain things are. You can absolutely do this. It's even right most of the time.

Let's be real. Most of the time you ask an LLM "Why did you do it like this?", it responds with something along the lines of "Oops. My bad. You're right to point this out."

You even have a fair chance of getting a response like that when there isn't anything wrong and the question wasn't rhetorical - which perfectly illustrates the level of the genuine understanding LLMs operate at.

Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing

#270

Post by Jianghong94 »

john01dav wrote: post #236

Why isn't self hosting (even just renting a GPU server, not necessarily on premise) at large companies or hosting via something like together AI to run the open weight models not more common? I've tried the open weight models and the premium models like Opus and Gemini Pro, and I find that the latter are a little better, but not nearly to the degree to justify the extreme price difference, since the differences large…

I just went through a similar discussion in my $WORK (traditional finance company on NYSE with average IT expertise) and I think the thought process is as such: it's one thing to just give your stellar dev/hacker a beefy GPU server and run whatever model they can run; it's another thing to maintain such platform for company wide. You would need human resource (likely way above normal software dev paygrade) to understand and maintain such models, maintain backend, availability etc. All these extra hassle make it just easier to pay a top tier external lab + slap a reasonable spending limit on everybody.
Post reply on HN