What if... we stop for a moment, and then, after thinking for a moment, we stop hammering nails with a microscope, and stop using token usage as a metric of productivity? I know it's sounds stupid, but what if
The people who have ascended to leadership positions are deeply divorced from reality. "It is difficult to get a man to understand something, when his salary depends on his not understanding it." -Upton Sinclair
The crazy thing is their salary does not actually benefit from riding these trends. Unless it's equally/even more clueless board level pressure with ulterior motives (i.e., lifting their other AI investments or the sector as a whole).
What if... we stop for a moment, and then, after thinking for a moment, we stop hammering nails with a microscope, and stop using token usage as a metric of productivity? I know it's sounds stupid, but what if
It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way." Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits,…
The point of this was always to explore what is possible with AI as quickly as possible. Obviously, there is going to be a lot of waste, but the 5-10% of employees who are truly thinking about it and discovering novel applications are what you are truly after. Because right now, you effectively have a giant, as of yet poorly explored space of potential uses. Anyone who can find the actually valuable portions of the s…
The thing I don't get though, is that most people just don't have that much work they need to do. I can use AI to pretty easily get my work done just via the regular chat interfaces. But because of the tokenmaxxing metrics that leadership tracks, I end up just having the AI deliberate for hours on random things just so that I can boost my token numbers. I think tokenmaxxing for the end goal you described is only realistic when the engineers are truly buried under a backlog of work.
As soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.
What's funny is that this apparently wasn't something that the Uber COO seemed to think about when their company is arguably one of the most successful ever at the "subsidize to drive down costs until you capture nearly the entire market" strategy.
It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way." Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits,…
You say "amazing that it took months to figure this out" as if the answer to the question is obvious. But it's not. Some FAANGs are doing amazing things with unlimited tokens. Other companies have no clue what to do with tokens, they've just told their engineers to max them. It really depends on how you're using the tokens . If you're just using them for Codex and Claude Code - yeah, tokenmaxxing is incredibly dumb.
Where can I see those amazing things done by FAANGs?
If any company announces that they use token consumption as an employee performance signal, for me that's close to a red flag to stay away from that company. No company with good engineering leadership should act like this is remotely a good idea.
I worked at a YC company that was doing this and left last month. I wonder where this all started from, VCs and tech execs are such a monoculture
The people who have ascended to leadership positions are deeply divorced from reality. "It is difficult to get a man to understand something, when his salary depends on his not understanding it." -Upton Sinclair
The crazy thing is their salary does not actually benefit from riding these trends. Unless it's equally/even more clueless board level pressure with ulterior motives (i.e., lifting their other AI investments or the sector as a whole).
Every c suite in the country is panicking about being left behind, from their perspective it’s either token max or fade into obscurity, or at least that’s what they were sold
As soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.
More straightforward to talk about the hardware directly. Full Kimi K2.6 needs an 8x H200 node to run and serve around 20 heavy users. You can rent an 8x H200 node for around $30/hr.
I'd imagine GPT-5.5 and Claude Opus 4.7 could run just fine on a 16x H200 node and serve at least 10 heavy users without the token output getting choppy.
As soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.
I have been saying the same for while. Someone always says "but Anthropic is making money on their API" or "But it's inference will get cheaper". But I don't believe it. first all the investments have to payed off at some point and second of all there are other things that cost money. I don't believe that any of them have a positive balance sheet. I also don't think that blitz scaling will work like with Uber. The en…
If by "investments will pay off" you mean major profits, that's never going to happen as long as scaling laws hold. All revenue will just go to financing more compute, and either we hit AGI or have the greatest economic collapse in modern history.
The world will look drastically different 5 years from now; for the better or worse, so save every penny (especially if you work in tech).