wtv
Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
81–90 of 360 posts
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#82Waiting for tokenedging next.
^ Philip K. Dick's unreleased book title
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#83The black bill that is coming that nobody is prepared for is that the value of a token varies greatly depending on the human. Companies will quickly find out its much better to give your top 10% engineers a lot more tokens and lay off your average engineers. The 10x engineer will become the 1000x engineer.
Wrote about this and the impact of to jobs here: https://x.com/deepwhitman/status/2058324179506831372
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#84As soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.
You're assuming the price won't come down as the tech matures. That seems like a big assumption, considering how quickly open weights models are catching up to frontier models, and how little effort has been invested so far in optimizing inference costs.
It's especially a crazy assumption to make relative to the costs of employing a human. The costs of paying an entry level employee are unlikely to go down at all, and even if those costs do decline, there's a floor they can't drop below (minimum wage at the extreme end), whereas companies are free to optimize agentic costs as close to zero as possible.
So you are assuming that a cost which is extremely susceptible to optimization but which no one has yet seriously attempted to minimize will remain perpetually above a cost which is much less susceptible to optimization, is already subject to enormous efforts to minimize, and has a legally mandated floor. That seems like a bad bet.
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#85I always used to wonder this about software stacks even prior to LLMs, but it seems more relevant now somehow: When will Uber (or your favourite company) be 'done'? They've been writing software for 16 years. They match drivers to passengers. More software isn't going to increase the chance that I seek them out instead of taking a bus or train. Will their software be finished in 20 years? 80?
Most of the codebase is custom integrations for local markets. You can systematize some of it but most of the complexity comes from there.
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#86"He said that, based on talks with Uber's senior engineering leaders, he realized higher token usage did not translate into a proportional increase in useful consumer features."
He's saying that like it's some grand epiphany and not the most self-evident, obvious thing I've heard this month. Some of the literal dumbest people on earth are in charge of these major companies.
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#87Earlier quoted context omitted.
DeepSeek is an open weights model. It's possible the hosted versions are subsidized, but we know what it costs to run locally. And it's expensive, but it's also pretty clearly cheaper than an employee. Of course, the latest DeepSeek models are not as good as Claude, but they're not super far off either.
They're not far off, getting the same seamless integration as hosted models is a full time job. I think what just happened is that devops is about to explode. What will naturally follow is local hosting of all the things when people realize subscription costs for cloud-whatever are absurd. Gitlab is going to take off? This is not investment advice.
> What will naturally follow is local hosting of all the things when people realize subscription costs for cloud-whatever are absurd.
Even acknowledging we don't know exactly what costs would look like in a world without VC money, wouldn't hosting models logically be cheaper to do at scale in a data center?
When I compared to the cost of running DeepSeek locally, I meant that we can treat that cost as a price ceiling, not the floor.
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#88If any company announces that they use token consumption as an employee performance signal, for me that's close to a red flag to stay away from that company. No company with good engineering leadership should act like this is remotely a good idea.
Meta does this. Guess what one of the criteria for their recent layoffs was.
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#89Earlier quoted context omitted.
The crazy thing is their salary does not actually benefit from riding these trends. Unless it's equally/even more clueless board level pressure with ulterior motives (i.e., lifting their other AI investments or the sector as a whole).
Every c suite in the country is panicking about being left behind, from their perspective it’s either token max or fade into obscurity, or at least that’s what they were sold
I don't think that's accurate. I think every C suite in the country is looking to do away with labor's leverage as much as possible. I think this is a cultural thing more than anything else, C suite + investors looking to get rid of those pesky humans required to prop up their lifestyles. AI is the most credible path toward that. Short, medium or long term returns be damned, this is a reconfiguration of society and they want to shed what they consider to be baggage.
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#90I always used to wonder this about software stacks even prior to LLMs, but it seems more relevant now somehow: When will Uber (or your favourite company) be 'done'? They've been writing software for 16 years. They match drivers to passengers. More software isn't going to increase the chance that I seek them out instead of taking a bus or train. Will their software be finished in 20 years? 80?
There are always newer technologies and techniques to be implemented. Better algorithms. Larger deployments. Better reliability. There are also almost always bugs to fix. So, so many bugs.