Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tokenmaxxing is dead, long live tokenmaxxing

12gramsofcarbon.com

Post reply on HN
Print view View original

281–290 of 315 posts

Re: Tokenmaxxing is dead, long live tokenmaxxing

#281

Post by sebastiennight »

irishcoffee wrote: post #274
bluGill wrote: post #266

Earlier quoted context omitted.

> You don't need peer reviewed studies to tell you water is wet. You don't need a peer reviewed study to tell you that a heavy rock will fall faster than a light rock. Which is why we have peer review even for obvious things.

> You don't need a peer reviewed study to tell you that a heavy rock will fall faster than a light rock. Either I don't understand gravity, or you might want to pick a different analogy...

I think GP is being sarcastic, and pointing out that

1. "heavy rock falls faster" is what common sense will tell you (I was literally told this by multiple laypeople just a few days ago when sightseeing atop a tall tower)

2. This is disproven by a trivial experiment that nobody thought worthy of trying for millenia

3. therefore we do need peer reviewed studies to confirm even "obvious" knowledge.

Also, note that GP's parent post about "water being wet" is quite the subject of contention in scientific and philosophical circles, so that wasn't the best example either.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#282

Post by linsomniac »

sevenseacat wrote: post #214
linsomniac wrote: post #7

Earlier quoted context omitted.

That's a very good point. Our company has been very thrifty with our AI spend, until a few months ago the average employee had ~$50 of supported spend and I was trying to be an AI leader in the company and figure out what was and was not possible, I had a $100/mo spend (Claude $100 service costs $108/mo). We are now seeing that Claude Code can do a LOT of heavy lifting in our day-to-day work, but the bulk of our empl…

I'm in the boat of wondering how so many people run into session limits so often. I have never hit one, except once when Claude Design came out and I had fun generating a bunch of random things to see what it could do (not with the intent of actually using any of the generated designs/code, because it all sucked).

I'm using the $200/mo Claude, and I'll often hit the 5 hour limit, but last week was the first or second time I've hit the week limit. I run pretty much everything on "/effort max" because I've had good results, and I've had plenty of quota left usually, and I want to worry less about "am I getting the best results".

Fable, for the few days I had it, would eat through tokens pretty quickly, largely because it tended to work much more on its own. I could give it a task and after asking a few questions it would go off and work for 4-6 hours and be done.

I also run a lot of experiments. I'm trying to be a resource that the rest of my team can learn from as far as what works. For example: when one of the people from our parent company asked about automating payroll entry, I threw their documents and discussion at Claude to see what it'd build. That plus churning on their feedback was ~30 hours of API usage right there.

I'm currently experimenting with "loops", and using codex in those loops to provide feedback and review. That gives me fable-like autonamy (that 30 hours of API usage above), maybe even better. But it uses a lot of tokens. Loops is the bulk of why I got to the weekly limit last week.

Plus I'm having it build an experiment on what my ideal "agent mux" would look like. Herdr is really close, I found it after I started that experiment. Now I'm just letting it run when I have spare usage to see what it comes up with.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#283

Post by kalcode »

et1337 wrote: post #77

Folks have been saying “things are different now, the agents are now compounding success instead of error” for at least a year now, but I just don’t see it. I was lucky enough to receive a weeklong $50k per head AI training from the people saying these things, and one of their few helpful concrete recommendations was to constantly clear context all the time, to avoid things going off the rails. However, I think findi…

It's true, but I see it happening. I’ve watched seniors with 30+ years of experience adopt them successfully without losing their classic rigor.

Personally, I get huge mileage out of LLMs, and yes, I care deeply about code quality, readability, and debuggability.

I've seen juniors absolutely rock with them.

And I've seen the exact opposite, where they just struggle to get good results.

In the end, I think the divide comes down to management experience. The people thriving are the ones who have led teams, especially teams of contractors, which is the best analogy for how you have to interact with an LLM.

Those folks know how to break down problems, provide the right context, and scope a task just enough to see the "contractor" succeed before letting them move forward.

On the other hand, individual contributors who are used to just grinding solo often struggle. They expect a one-shot miracle. They say, "Hey, my code is buggy, fix it." When the LLM inevitably hallucinates or steers them wrong, they give up. The results are completely different based on how you treat the tool.

They might just have a high quality of control and standards that it is hard to find that pattern with the LLMs.

I think fierce individual contributors are a lot more valuable in the era of llms as well. We as humans typically achieve better balance with new stuff when we allow backlash from new processes that start to trample on old ones without understanding AKA the Chester's fence.

Anyways, more of a ramble than my two cents.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#284

Post by theahura »

stopyellingatme wrote: post #277
theahura wrote: post #269

Earlier quoted context omitted.

Hi, author here. I'm probably somewhat financially benefiting from tokenmaxxing. I also just believe compounding correctness is right, based on my own experience using the tools (which is why I have structured my life to try and financially benefit from tokenmaxxing)

How do you structure your life such that you financially benefit from tokenmaxxing?

starting a company that benefits from tokenmaxxing, mostly

Re: Tokenmaxxing is dead, long live tokenmaxxing

#286

Post by adammcdaniel »

I was one of those mentioning the death of tokenmaxxing (https://www.ibm.com/think/insights/tokenmaxxing-dead-long-li...)

I would say tokenmaxxing = spending without limits or care about results (and assuming results). The term as it is right now, at least.

When it comes to "using tokens overall", then open source models change the equation and in that case, we will enter a phase of 'maximizing AI usage...but for near-zero marginal increase in costs with increased usage'. But even then, the convo would shift to platform engineering...which would then ask 'what value are we getting out of this?'

OR - cloud model economics change over time and we use cloud models as happily and cost effectively as we do cloud storage now. But hard to say when that comes.

Open to thoughts, though.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#287

Post by dspillett »

robocat wrote: post #173
dspillett wrote: post #122

Earlier quoted context omitted.

> Most companies focused entirely on doing "what everyone else is doing" This is true of my current overlords. It slipped recently that the reason they went AI-nuts was that a competitor had announced going “AI first” and the market responded excitedly. Not because they thought it was a good idea: because the market got excited and they didn't want to get left behind. This is quite a change as our market is financial…

Isn't it easier to get a job when you already have a job? There's some jobs I'd love to do, but I can't face the bullshit of tertiary education again. Without some sort of ticket, job choices become more limited?

> Isn't it easier to get a job when you already have a job?

Yes. Or so I'm told, I've not needed to apply for a job for 26 years…

I have something possible available, though whether it still will be in five months (the earliest I'm likely to leave because of [reasons] and a two-month notice period) is a bit unknown. That five months might be ten as there are other major changes in the company (we were bought a while ago) from which the dust should have settled by Feb, and it makes sense to try to hold out that long to see if I'm still hating things with the same passion at that point.

Without that “something” there are less certain tech based options I could look at, and to be honest I really could do with a proper sabbatical style break. The mortgage is paid, I have savings, and no dependents other than the cats, so I have the luxury of considering that option. And if all else fails I've actually done the arithmetic and I can survive on minimum wage for an extended time if I need to, and hospitality work is something friends can get me into above the many others looking (that bit is less of a joke then people assume: it is seriously part of my plans D & E if I can't stick with A and B & C completely fall through).

Re: Tokenmaxxing is dead, long live tokenmaxxing

#288

Post by ligne »

no-name-here wrote: post #157
JoshTriplett wrote: post #74

Earlier quoted context omitted.

So do a workshop on power tools, measure their efficacy and the quality of the result, do some demonstration videos on power tools, get people to compare, seek feedback on their usage. Don't count electricity and sawdust, or you'll find people getting very good at expensively turning blocks of wood into sawdust.

Is the idea that most stubborn employees would adopt AI if their company made videos showing internal metrics that AI is better? > Don’t count electricity and sawdust I agree that it seems wasteful, but is there some better way to accomplish it at the scale of hundreds, or hundreds of thousands, etc? I'm personally doubtful that stubborn employees would switch even if a video provided internal metrics, videos, etc.

The same way you monitor your staff's work in general? Do they not have goals and deadlines and some way to discuss their progress with their manager?

Re: Tokenmaxxing is dead, long live tokenmaxxing

#289

Post by agentultra »

shimman wrote: post #145
agentultra wrote: post #90

Earlier quoted context omitted.

Doubt. People brought in all kinds of web applications in the early Web 2.0 era because corporate IT was being too stingy (for a lot of reasons). People will find efficiencies on their job on their own. No need to denigrate them.

Yeah but if you can't attack the workers and make them hate their lives, are you even a good capitalist? Didn't Milton Friedman die for our bosses right to stomp on our faces in the pursuit of profit?

Alienation is inherent to the system.

Re: Tokenmaxxing is dead, long live tokenmaxxing

#290

Post by MaxMatti »

hwillis wrote: post #251
RugnirViking wrote: post #227

Earlier quoted context omitted.

this! the back-and-forth chat interface where you can edit only your own messages, and only then to get a new response, is a terrible one, but I think favored by vendors because it helps them fight in vain against prompt injection. Custom harnesses and stuff are nice but incredibly time consuming to set up when all I want to do is like, see what the agent is reading, and editing out some irelevant nonsense side quest…

"some caching gains" is a pretty huge understatement- snipping something out of the middle of the window requires rebuilding the entire context. Thats a shitload of tokens. Afaik messing with the context also pretty reliably degrades performance still. The model responses reference things that no longer exist to it and it becomes more chaotic. The real usefulness of parallel or sub-agents is not that they run at the…

I still feel that during agentic workload sometimes it would be nice to have the model identify it is veering off the main track, send out a "keep the cached states and tell me which they are" command to the inference server, do the side thing (such as handling an error that plopped up that has not that much to do with the main task) and return back to the cached state with just a comment tacked at the end to say "oh and btw I fixed DNS" instead of having the DNS debugging inside the context window now. Maybe other harnesses just steer the models more towards using subagents for such tasks and my pi is misconfigured. I can use the tree feature, but having insight into what's cached would be nice there.
Post reply on HN