If a worker doesn't use their AI/LLM budget, can they get a raise?
probably will get fired for lack of performance.
Uber's $1,500/month AI limit is a useful signal for AI tool pricing
311–320 of 819 posts
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#312Earlier quoted context omitted.
Seriously, they’re trying to justify trillion+ IPO’s while setting piles of money on fire, prices aren’t going DOWN.
Today's frontier models will be tomorrows low-end option. I think whatever model you are using today will be less expensive to use a year or two from now.
Last year's o3 was more expensive than 5.5 is. Whatever model we are using now is probably be more expensive than next year's leading models will be.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#313Why there are so many people that still believe that AI coding is a fad? It's something that started less than two years ago and companies are already paying thousands per seat. I know one that gives you 5k per month. Which other tool went from nothing to this level of acceptance so quickly?
It's cope. People desperately want to believe that AI coding is going away so that they can go back to partying like it's 2020. So there's a huge number of HN posters claiming that the price of tokens will go UP over time rather than down (that's how Moore's Law works, right???) or that code bases that AI contributes to will spontaneously combust, or something.
I don't think it is unreasonable to say both will happen, is it?
In the long term, tokens will fall in price. Obviously. (If "tokens" continues to be the unit)
In the short to medium term, for the IPOs to succeed, people have to start actually paying for what they are using, so the price will go up, and is going up, quite a lot. Once their value is set they will slowly fall from that point (or some point maybe halfway, depending on how much the market is willing to continue to subsidise).
I am an AI cynic, but I am now an informed cynic; I am learning agentic tools so I know where they are useful and I know my enemy.
I think the "fad" here is cloud-based, metered AI being a dominant work mode.
Nothing, so far, has suggested to me that any other outcome is likely than edge- to local-scale, on-device, on-laptop, on-prem models getting good enough to the point where people use them by default and use the cloud models only when they need the extra oomph.
I cannot believe that there is anything other than an enormous incentive for companies like Uber to find local, small model and on-premises solutions to their problems, not least while pricing is so changeable and people are getting nasty surprises.
Betting on OpenAI and Anthropic being around over the long term in the form that they are now, that feels like valley hopium. Utility monopolies essentially always derive from physical/geograpical limitations, don't they?
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#314Earlier quoted context omitted.
do GPU chips really depreciate physically? There are no moving parts, I dont think memory chips or GPU chips deteriorate naturally. I think its only accounting depreciation. I have been using my laptop for a decade, what is stopping datacenters from using the purchased GPU chips for a decade?
Today's data center GPUs are essentially overclocked, and so at limit of how much the chip materials can physically handle, and therefore degrade over time. For example, GH200s operate at 1W/superchip but the actual safe power is somewhere around 650W which will allow them to function for a decade or more. But that leads to around 15% slowdown and that is unacceptable in today's competition. So current GPUs are desti…
i think its reasonable to give up 15% of speed for a decade more lifetime. This depreciation change alters economics of GPU
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#315Earlier quoted context omitted.
What's an int vs a float vs a boolean? What's a function? What's a class? What's a variable ? You don't actually need to know the answer to those questions in order to vibe code. That's a lot of priors to update!
Just to go on record, as of today, I’m a big believer that a person that knows all that stuff is much more productive with AI-coding than a person who doesn’t. I have no idea how we can get people motivated to learn these through trial-and-error when AI coding exists though. I remember the days of spending hours on stupid bugs that AI can resolve within a minute. But I recall learning heavily from those experiences.…
I honestly feel like my own learning has accelerated after using AI. Simply because now it's so easy to write the same thing in so many different languages, I can e.g. learn pros and cons of each language, which otherwise would have been I think unfathomable to me. I have now created so much stuff I wouldn't have had time to create.
I setup k3s, and tons of what would be otherwise unnecessarily complicated stuff on my laptop for my side projects with additional home servers, smart house stuff. Otherwise k8s and things like that would have been daunting to learn and in theory and without constant professional exposure, etc...
Microservices in Go, Rust, which I didn't have any previous experience with, games in C and other languages. Didn't know anything about low level memory management before. Was just mainly TypeScript person. Just constantly building random fun stuff.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#316Earlier quoted context omitted.
You're a content creator; you define your revenue stream. Uber engineers do not define their revenue stream; the product leadership team does. $1500/mo of AI spend by engineers does not equate to revenue. They need to figure out revenue first before zeroing in on AI spend.
$18K a year is a fraction of the salary of a junior engineer. Claude has allowed me to do refactors that would have taken weeks to instead take a couple of days. It has, objectively, increased the velocity of the engineering component of greenfield features by 40% in my org. You can put a number value on that and decide if it gives you favorable ROI.
In the old world, the refactor probably won't happen in the first place, but the effort would be put elsewhere. "Increased velocity of ..
greenfield features" doesn't directly translate to additional revenue, and your number is very questionable in the first place.
Software engineers like to talk as if business and finance are as easy as pushing code out and refactoring. It's not and never has been.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#317Earlier quoted context omitted.
One aspect Paul Kedrosky mentioned recently is the concept of „duration mismatch“. The price per token goes down over time (either because the AI vendor reduces due to competition pressure, or because customers are now incentivized to use older cheaper models). But datacenters are financed through debt, with the assumption their revenue increases over time. Quoting him: „[AI vendors are] paying for a fixed cost with…
do GPU chips really depreciate physically? There are no moving parts, I dont think memory chips or GPU chips deteriorate naturally. I think its only accounting depreciation. I have been using my laptop for a decade, what is stopping datacenters from using the purchased GPU chips for a decade?
Nothing is stopping them, it's just not worth it: Have a look at e.g. vast.ai's pricing (https://vast.ai/pricing).
The V100 (2017 -> 9 years old) can be rented from $0.02 to $0.37/h (right now I can find a V100 with a Xeon Gold 6140 and 48GB RAM for $0.165/h). Let's assume the guy you rent it to pins it at its 250W TDP and let's ignore the running costs of CPU/RAM/etc... Then you draw 1/4 kwh for that compute hour. The industrial electricity prices in the US vary between 7.5 and 25 ct per kwh (depending on state, time of day, etc...), so at 100% efficiency, assuming nothing ever breaks, and the CPU consumes 0W you earn about 14ct/h.
And remember: V100s hours are sometimes sold at 1/10th the price.
If I pick average conditions you need to start thinking of whether it is worth it to rent them out: Usually it isn't unless you have them anyways and just sell idle capacity.
It's barely worth it to run them in a pure "is it profitable" sense, if we also account for the opportunity cost of taking up a slot in your datacenter it seizes to be worth it really quickly.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#318> I noted that my own token usage comes to about $1,000/month against each of Anthropic and OpenAI - which currently costs me just $100 per provider thanks to their generous subsidized plans for individual subscribers. Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? Many lower-budget individuals are now moving to China open weight mode…
> Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? I genuinely do not know how prices can get lower from the current major providers in NA without the whole market collapsing. Everyone is spending copious amounts of money to presumably make more money back.
Prices can go down while tokens sold increases so that profit increases. The labs number one goal right now is moving past software engineers so that every white collar worker in the country finds ai assistants indispensable. Speculation here but I think openAI/antrhopic api inference is insanely profitable, it just needs more volume to amortize the training costs.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#319Earlier quoted context omitted.
I don’t think companies will do that. Why don’t they just buy local on-premise infrastructure even though it’s cheaper than AWS? “AI in a box” sounds a heck of a lot like “the box” from the Silicon Valley TV show. Or the Google search appliance. Or name any other on-premise thing that is equally dinosauric. The real finding of this article is that AI tokens are direct competitors with offshoring. $1,500/month buys yo…
> I don’t think companies will do that. Why don’t they just buy local on-premise infrastructure even though it’s cheaper than AWS? For customer facing, production software, its worth paying a cloud tax to get the reliability guarantee. For tools that are used by engineers for code development, there is no need for such bulletproof guarantees.
That makes very little sense. SaaS/cloud tooling is overwhelmingly popular for internal tooling.
Which category of developer tool has on-premise as the more popular option?
Cloud isn’t about “reliability,” it’s about being able to focus on your core business rather than spending all your time maintaining stuff.
Re: Uber's $1,500/month AI limit is a useful signal for AI tool pricing
#320If a worker doesn't use their AI/LLM budget, can they get a raise?
probably will get fired for lack of performance.
Let's just say their performance (OKR, KPI, whatever "impact" metric you want) was indistinguishable from a peer that used the AI/LLM monthly allowance in full.
Maybe a $10k raise would be nice?