Maybe don't use the most expensive models on the planet? Maybe use AI like a tool and not this black box that grants wishes?
Sounds like you want to be in the next round of layoffs?
Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
51–60 of 360 posts
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#52Maybe don't use the most expensive models on the planet? Maybe use AI like a tool and not this black box that grants wishes?
I think companies are reluctantly realizing that AI is not a magic genie in a bottle, and is instead a tool.
Still very valuable. They just need to have strategies that match what the tools are capable of - not strategies that involve "rub the magic lamp and increase profits 80%".
If the market is rewarding companies going after the "rub the lamp" strategy, they're going to say they're doing that to juice stock prices.
Maybe the market is finally realizing blindly spending billions on LLMs with almost no strategy is not a good strategy.
Who knows.
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#53Tokenmaxxing makes no sense, it is akin to write extremely inefficient SQL / Spark Jobs, full of cartesian joins, ultra skewed datasets, etc, just for the sake of using as much compute / memory / IO as possible.
This always happens when the metric becomes the goal, companies should nurture and foster an environment where AI is used in the most efficient way possible, first asking "do we really need an agent for this" and if so, what kind of agent is needed, what model, reasoning level, etc.
They should also promote projects that aim at saving tokens, increasing cache hits, codifying the information in ways such they use as less context as possible (graphs of knowledge are pretty good for this!)
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#54I'd be interested to know if this is about individual employee AI usage, or use of AI tokens in production features, or both - and assuming both, what the split is.
I can see how Uber could burn unbelievable amounts of tokens if they start running internal features that run a bunch of prompts against every completed ride, or every customer profile, for example.
Or maybe this is about employee usage, but they introduced some stupid "you get evaluated on how many tokens you used" thing a couple of months ago when that was trendy and are just beginning to notice how much that cost?
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#55What if... we stop for a moment, and then, after thinking for a moment, we stop hammering nails with a microscope, and stop using token usage as a metric of productivity? I know it's sounds stupid, but what if
The people who have ascended to leadership positions are deeply divorced from reality.
"It is difficult to get a man to understand something, when his salary depends on his not understanding it." -Upton Sinclair
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#56Earlier quoted context omitted.
You say "amazing that it took months to figure this out" as if the answer to the question is obvious. But it's not. Some FAANGs are doing amazing things with unlimited tokens. Other companies have no clue what to do with tokens, they've just told their engineers to max them. It really depends on how you're using the tokens . If you're just using them for Codex and Claude Code - yeah, tokenmaxxing is incredibly dumb.
> Some FAANGs are doing amazing things with unlimited tokens. Others have no clue what to do with tokens. Unlimited tokens is different from “use AI a lot or we will fire you, and we are counting token consumption as usage”. Obviously the latter is stupid and yet it was done in many places.
[deleted]
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#57Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#58Not all tokens are created equal. It's easy to use a ton of tokens by having agents work together in parallel. That's basically the equivalent as people spending time in meetings, hardly a productivity win. As with everything in development, results matter, how you get there doesn't (unless you're a bad manager).
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#59If any company announces that they use token consumption as an employee performance signal, for me that's close to a red flag to stay away from that company. No company with good engineering leadership should act like this is remotely a good idea.
Tokens are the new "lines of code per engineer". Easy to graph, easy to "manage".
...and easier to bill! Back, then noboday had the idea to charge per "lines of code", but today it seems accepted to charge per words processed?
Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing
#60It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way." Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits,…
You say "amazing that it took months to figure this out" as if the answer to the question is obvious. But it's not. Some FAANGs are doing amazing things with unlimited tokens. Other companies have no clue what to do with tokens, they've just told their engineers to max them. It really depends on how you're using the tokens . If you're just using them for Codex and Claude Code - yeah, tokenmaxxing is incredibly dumb.
In other words, people who are productive get more done when you scale up what they're already doing, and people who aren't productive will not magically become productive when you scale up what they're already doing. That's incredibly obvious, because we've seen how this plays out repeatedly in so many different ways (lines of code, commits, tickets closed, etc.), and it has nothing to do with tokens or even programming, but just how trying to manage people works.