Live data from Hacker News

Tokenmaxxing is dead, long live tokenmaxxing

12gramsofcarbon.com

141–150 of 315 posts

Re: Tokenmaxxing is dead, long live tokenmaxxing

#141
This is more likely the junior camper version of "not everything that counts can be counted, and not everything that can be counted counts."

In the early days of LLMs, we saw the classic hype-driven bi-modality of opinions. Folks were in the "fake news, fad" camp, or they were in the "omg, take over the world" camp.

Those of us closer to the space, with the awareness to know that there was some truth (and a lot of misjudgment) to go around, were in the middle of nowhere. When I co-wrote some driver code with Chat GPT, other engineers (and even one of our directors) told me to keep it quiet. At the same time I had directors and VPs asking me how we could accelerate adoption. For a while, I had access to a cheat code just because I had the audacity to not ask for permission. Folks were sure I would get in trouble for spending thousands per month in LLM operation, but a handful came along for the ride, burning tokens like firewood and learning along the way.

Tokenmaxxing is probably coming from at least a few things:

1. A course-correction for the practiced frugality that kept folks from jumping in and just learning at the ragged edge.

2. A willful and deliberate recognition that the best innovations in the later phases of a disruptive introduction often come from sparks of ideation in concentrations of activity. In other words, we don't know where good is, and we need to find it. (Charitable interpretation from the article)

3. Recognition that, even if they don't know why, leaders and product owners will get punished for not jumping in and, because of bullets 1 and 2, won't get punished for trying and missing. Even if they have no idea what they're doing, they're going to fake it until they make it (or slide into another job).

This last set is where the pain lives. An organization with healthy and increasing AI tool usage will see elevated token counts, but so too will one using LLMs to rewrite wikipedia articles without the letter "m" to keep token counts high. These are pathological behaviors brought on by conflated metrics.

We had discussions about this in the early LLM days, where my old team was looking to ship new capabilities for older products. There was a lengthy VP-level discussion about getting to "80% usage" of the new system vs the old. Because the new system was a superset of the old, I eventually said "we can do that immediately, but it's a cost goal, where we're just aiming to make our business more expensive to operate, rather than a value goal for our users". We didn't adopt the target, but folks were understandably frustrated that they didn't have a straightforward way to measure and report progress.

Tokenmaxxing is, inevitably, a conflated goal, but it's what we have right now. Take advantage of the moment, learn, build, and keep an eye on levers for efficiency.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#142
post #50
post #46

Earlier quoted context omitted.

I really don't understand this take. If you're a carpentry shop that just bought power tools for the first time and you're worried that your employees are sticking with hand tools because that's what they know, then you look for sawdust. The goal isn't to have people work at converting wood into sawdust, the point is that if you wanna see if the tools are working you wanna see proof they're actually being used. I'm s…

Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world. (Of course, we've all had bosses that went to some marketing seminar and come back having been tricked^Wsold…

> Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively?

I mean, the difference in the metaphor is that we have pretty fully understood carpentry for many hundreds of years. We still find it difficult to write even simple software to address all our needs, as is evidenced by the insane pay in our industry. Carpenters can suggest tools because they know what's out there. The same was not true about LLMs a year ago.

> That is way too charitable, people were being fired based on these metrics

People get fired for all kinds of reasons including no reason at all. Oftentimes leadership even lies about the real reasons for firing people because they don't sound good!

I'm gonna be blunt: if you're in software and you refuse to use AI for moral reasons, I think you should be fired. There's being principled and there's being obstinate and the difference between the two is how well you can convince people that you _have_ principles. Most LLM-hating people fall short on this point, because

> do I really need to link the Jensen Huang quote?

Sure! Link it again, we all know it's highly immoral when shovel salesmen try to make you want shovels.

> If you want to see if the tools work, why don't you just ask your employees? Like any normal employer would?

I do not like this HN take of "let's do this thing that works great in small companies and then just blindly pretend that it'll also work at the largest companies in the world!" No, this doesn't work at "normal companies" because you cannot "just ask" 30k+ employees what they want.

Employees, like EVERYONE ELSE, are resistant to change. If I, as CEO of a company, want to get my company to try Claude I have to measure tokens to see if it's getting used. That's it. There's no wave of delusion here.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#143

Earlier quoted context omitted.

> If you want to see if the tools work, why don't you just ask your employees? Like any normal employer would? because that would require actually admitting that employees are the people in an organisation who are responsible for the success of that organisation, rather than the people higher up the org chart.

[flagged]

Do you actually have a job? Do you talk to your coworkers?

This is an insane take. Plenty of people are critical of AI at my job despite a big push to use it. I find the comparison to NK distasteful, coming from someone who presumably is pretty well paid and can quit their job whenever they want.

If you're feeling humiliated... well, I don't think it's because your boss wants you to try AI.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#144

The implication that tokenmaxxing was an intentional and thoughtfully considered approach rather than blind hype-following by an overpaid manager class who are too far removed from value to understand the downsides of LLMs is hysterical beyond belief.

The retconning is absurd. Companies CxOs's exhibit herd-line behaviors all the time: hiring scrum masters, mandatory return to office, and now tokenmaxxing. FOMO is a sufficient explanation of the behavior, any trend gaining momentum that they fear will give their competition an edge, they will reflexively force adoption with no further reasoning needed. There is no cost to boarding hype trains that go nowhere.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#145

Earlier quoted context omitted.

The logic of trusting employees who are worried that power tools will replace them to utilize power tools effectively is completely backwards in any sane world. People don’t like change, sometimes it needs to be forced on them.

Doubt. People brought in all kinds of web applications in the early Web 2.0 era because corporate IT was being too stingy (for a lot of reasons). People will find efficiencies on their job on their own. No need to denigrate them.

Yeah but if you can't attack the workers and make them hate their lives, are you even a good capitalist? Didn't Milton Friedman die for our bosses right to stomp on our faces in the pursuit of profit?

Re: Tokenmaxxing is dead, long live tokenmaxxing

#146

Earlier quoted context omitted.

The logic of trusting employees who are worried that power tools will replace them to utilize power tools effectively is completely backwards in any sane world. People don’t like change, sometimes it needs to be forced on them.

> who are worried that power tools will replace them maybe, just maybe, it would have been a better idea to engage with employees first rather than posting on linkedin about how everyone is going to lose their jobs. cos it's the kinds of people trying to force this stuff on employees that are the ones who have been shouting about that from the rooftops.

If you take LinkedIn at face value everyone who uses the Internet is a sociopath who lives for no purpose beyond maximizing shareholder value.

Seriously, some of the most deranged things I've ever read were by relatively normal people trying to promote themselves on LinkedIn.

What people SAY does not matter nearly as much as what everyone KNOWS and it's pretty damn clear that AI is never going to be able to replace humans in complex domains. Every time a frontier lab announces a breakthrough it's pretty obvious that the setup was more complicated than "hey chat prove the Riemann hypothesis."

The world is gonna need skilled human beings to drive LLMs, no matter how desperately some people like to pretend otherwise.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#147
post #19

Earlier quoted context omitted.

Worse, tokenmaxxing has been pushed by the labs hoping to charge those tokens by the pound on their API prices eventually, even if temporarily hiding such costs behind "highly subsidized plans" or frequent bug-induced "reset buttons"

I would wager most if not all of the tokenmaxxing was done on enterprise API priced plans, not subscription plans. You can't actually token"max" if you are limited in the amount of tokens you can use per 5 hours.

Enterprise plans weren't usage-based until recently. Lots of enterprises were bait-and-switched by AI companies from flat per-seat fees, and now have to go on a token diet to rein-in budgets - which is the blog post's premise about tokenmaxxing being dead.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#148

Tokenmaxxing was just a way to force employees to start leveraging AI in a meaningful way. For companies that have measured performance based on token spend, they can now dial it back. Employees have learned to leverage AI for things they wouldn’t have prior. Now they know what’s possible and what’s not. No one is stupid enough to always measure performance based on token spend and have unlimited budget. It was alway…

lines of code produced. similar dumb metric.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#149
post #50
post #46

Earlier quoted context omitted.

I really don't understand this take. If you're a carpentry shop that just bought power tools for the first time and you're worried that your employees are sticking with hand tools because that's what they know, then you look for sawdust. The goal isn't to have people work at converting wood into sawdust, the point is that if you wanna see if the tools are working you wanna see proof they're actually being used. I'm s…

Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world. (Of course, we've all had bosses that went to some marketing seminar and come back having been tricked^Wsold…

> The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world

People are stubborn. A lot of productivity improvements had to be almost forced upon farmers, for example. Even when early adopters demonstrated the benefits, a decent fraction of them just didn’t want to change.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#150

Tokenmaxxing was just a way to force employees to start leveraging AI in a meaningful way. For companies that have measured performance based on token spend, they can now dial it back. Employees have learned to leverage AI for things they wouldn’t have prior. Now they know what’s possible and what’s not. No one is stupid enough to always measure performance based on token spend and have unlimited budget. It was alway…

So this is the narrative now? Come on.
Post reply on HN