Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tokenmaxxing is dead, long live tokenmaxxing

12gramsofcarbon.com

Post reply on HN
Print view View original

261–270 of 315 posts

Re: Tokenmaxxing is dead, long live tokenmaxxing

#261

Post by DaSHacka »

VulgarExigency wrote: post #224
red_admiral wrote: post #101

> Like, imagine if some serious business leader, like, idk, Mark Zuckerberg, decided to announce that Meta was going to burn money. Like ... pivoting to the "metaverse" and changing the company name to show he's serious.

It's astounding to me that they looked at Second Life and really thought that was the future.

I could easily see that being the case, which is why I was so against it.

I think they're just a few decades too early.

Similar to cloud gaming, although that's much closer on the horizon.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#262

Post by oooyay »

cyphar wrote: post #50
SR2Z wrote: post #46

Earlier quoted context omitted.

I really don't understand this take. If you're a carpentry shop that just bought power tools for the first time and you're worried that your employees are sticking with hand tools because that's what they know, then you look for sawdust. The goal isn't to have people work at converting wood into sawdust, the point is that if you wanna see if the tools are working you wanna see proof they're actually being used. I'm s…

Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world. (Of course, we've all had bosses that went to some marketing seminar and come back having been tricked^Wsold…

> Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world.

This is how it's gone down throughout history. It's why we remember the Luddites, textile workers who started smashing stocking frames and power looms because the machinery was introduced over their objections. The whole goal was to undercut the craftsmen's wages and bargaining power.

So, no, your expectation to be consulted was never going to happen and has not happened throughout history as industrialization has advanced.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#263

Post by jdiff »

foota wrote: post #243
RugnirViking wrote: post #227

Earlier quoted context omitted.

this! the back-and-forth chat interface where you can edit only your own messages, and only then to get a new response, is a terrible one, but I think favored by vendors because it helps them fight in vain against prompt injection. Custom harnesses and stuff are nice but incredibly time consuming to set up when all I want to do is like, see what the agent is reading, and editing out some irelevant nonsense side quest…

No reason other than you're giving user's a cost footgun? That's a pretty good reason.

Local models benefit disproportionately from this kind of pruning and have no such footgun.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#264

Post by Uzazo »

jdiff wrote: post #118
et1337 wrote: post #77

Folks have been saying “things are different now, the agents are now compounding success instead of error” for at least a year now, but I just don’t see it. I was lucky enough to receive a weeklong $50k per head AI training from the people saying these things, and one of their few helpful concrete recommendations was to constantly clear context all the time, to avoid things going off the rails. However, I think findi…

Even modern frontier models benefit so hugely from careful context pruning, maintenance, and rewriting to erase mistakes that it's astonishing to me that there are no tools centered around it. The one tool that used to have such a feature, Zed and its retroactively-named Text Threads, has now stripped itself of it.

This is why I prefer Pi over all other agent harnesses. It has a tree view of each conversation and it's easy to move between branches.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#265

Post by jdiff »

hwillis wrote: post #251
RugnirViking wrote: post #227

Earlier quoted context omitted.

this! the back-and-forth chat interface where you can edit only your own messages, and only then to get a new response, is a terrible one, but I think favored by vendors because it helps them fight in vain against prompt injection. Custom harnesses and stuff are nice but incredibly time consuming to set up when all I want to do is like, see what the agent is reading, and editing out some irelevant nonsense side quest…

"some caching gains" is a pretty huge understatement- snipping something out of the middle of the window requires rebuilding the entire context. Thats a shitload of tokens. Afaik messing with the context also pretty reliably degrades performance still. The model responses reference things that no longer exist to it and it becomes more chaotic. The real usefulness of parallel or sub-agents is not that they run at the…

You can structure your context window to minimize the amount of editing you do further back. You usually only need to edit and correct the most recent response. It's little different from forking the conversation at an earlier point, and nobody warns about that being a sneaking footgun. There is still a prefix to cache.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#266

Post by bluGill »

TeMPOraL wrote: post #202
oblio wrote: post #189

Earlier quoted context omitted.

> the worthiness of the adoption was blindingly obvious at that point Everything is so "blindingly obvious" yet nobody can point to ANY serious peer reviewed studies that prove it. I'm patient, I'll wait.

You don't need peer reviewed studies to tell you water is wet. Peer review is a technique to get evidence from data when SNR is low. It's not "science", it's just a technique. So is "throwing shit at a wall and seeing what sticks". Don't turn techniques into rituals, and science into religion.

> You don't need peer reviewed studies to tell you water is wet.

You don't need a peer reviewed study to tell you that a heavy rock will fall faster than a light rock.

Which is why we have peer review even for obvious things.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#267

Post by jackb4040 »

gausswho wrote: post #17

It's AI usage mandates now, but rather than focusing on how the current hot topic has ripped through the business world, often without benefit nor repercussions at leadership, I'd prefer to analyze the higher pattern. We've recently experienced such ripples as the metaverse, blockchain/nft/web3, 'the cloud' (and a minor wave of cloud gaming). There was even a teacup buzz of 'apis', oddly disconnected from the semanti…

The dominance of finance capital over industrial capital reaching its absurd conclusion. NFT mania was only possible because we don't make anything here, no one has a serious plan to reshore and start making things here again, and we can't indefinitely maintain control of production we've exported to the 3rd world indefinitely. So you might as well play these symbolic games and increase your slice of the pie while the music is still playing.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#268

Post by tsunamifury »

deadbabe wrote: post #254
liquicity wrote: post #225

Earlier quoted context omitted.

50k per head training and the largest takeaway was to clear context.. that is the "hello world" of using agents, insane.

$50k a head is cheap compared to the productivity gains, probably can push it to $75k

Really. How?

Have you measured them beyond loc?

Re: Tokenmaxxing is dead, long live tokenmaxxing

#269

Post by theahura »

ambicapter wrote: post #131

> That’s no longer true. We’ve entered a different regime, where spending more tokens generally results in better results. We call this “compounding correctness” — the more tokens you spend on getting a task correct, the more likely you’ll get a good outcome. We talked about this a bit at the last in person Agentics meetup: Have we? Is it generally the case that the more tokens you spend, you better results you get?…

Hi, author here. I'm probably somewhat financially benefiting from tokenmaxxing. I also just believe compounding correctness is right, based on my own experience using the tools (which is why I have structured my life to try and financially benefit from tokenmaxxing)

Re: Tokenmaxxing is dead, long live tokenmaxxing

#270

Post by andsoitis »

jamiecurle wrote: post #258
cyphar wrote: post #50

Earlier quoted context omitted.

Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world. (Of course, we've all had bosses that went to some marketing seminar and come back having been tricked^Wsold…

I see the point but, I'm not really sure the analogy holds up here. If i was in a cabinet shop and had to joint, plane and resaw and cross cut a pile of timber fresh from the saw mill for the next job I'd be very grateful for the jointer, the planer, the bandsaw and the table saw. I'd also be very grateful for the dust extraction. In in total agreement with you though, forcing tools on employees is very dumb and is t…

> In in total agreement with you though, forcing tools on employees is very dumb and is terrible leadership. Ask your people what they need to be optimally exceptional and go get them it. Then let them get on with it.

Some employees want AI tools, others don't. Standardizing SDLC workflows > each person does their own thing. So now you have to choose: do you require AI tool use that fit into a new SDLC? Or don't you?

Post reply on HN