Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I burned all my tokens researching how to save tokens

quesma.com

Post reply on HN
Print view View original

21–30 of 237 posts

Re: I burned all my tokens researching how to save tokens

#21

Post by jscottbee »

dofm wrote: post #12

It sometimes feels to me that cloud AI providers have convinced people that cloud AI is worth it because of all the ways people have been able to use cloud AI to write blog posts about using cloud AI to make cloud AI more efficient for something that they haven't shipped yet and aren't really ready to talk about.

I agree there... I hate the posts that rattle on about non-real-world work. Real world now (for me anyway) is maintaining legacy code, fixing issues, and new development. Now, some of the problems others have with AI are super crappy prompts with no real info other than "do this". I look at myself now more as a software designer and an architect, and the AI as a paid consultant. I give detailed specs and designs. The models are getting better at real deep code and legacy, but there are still holes. BUT give the AI a fighting chance with real detail in the prompts.

Re: I burned all my tokens researching how to save tokens

#22

Post by bkotrys »

cdnsteve wrote: post #9

You should consider testing our RemembrallMCP (note I'm the author). It substantially cuts token usage and you get better results in codebases. https://github.com/roboticforce/remembrallmcp

looking promising, will give it a try! thanks for sharing

Re: I burned all my tokens researching how to save tokens

#23

Post by Bender »

I am not a proper developer and only use AI for faster research of topics so please forgive my ignorance. Could one not save a lot of money on tokens by using the 80/20 or 90/10 rule in that 90% of AI usage is on local models and save that last 10% or less for the frontier models where the local model did not meet the needs? Did they cover this and I misunderstood?

Re: I burned all my tokens researching how to save tokens

#24

Post by [deleted] »

bob1029 wrote: post #7

Simply starting in the right part of the search space is the biggest predictor of success. The best way to save tokens is to start out the deep research pass with cheap models and then funnel the findings through increasingly powerful models. I've got a pipeline right now that uses all 3 of the gpt-5.6 model variants to address each stage of the process. If you are using models like sol or terra to generate hypotheti…

[deleted]

Re: I burned all my tokens researching how to save tokens

#25

Post by [deleted] »

xyzsparetimexyz wrote: post #11
bob1029 wrote: post #7

Simply starting in the right part of the search space is the biggest predictor of success. The best way to save tokens is to start out the deep research pass with cheap models and then funnel the findings through increasingly powerful models. I've got a pipeline right now that uses all 3 of the gpt-5.6 model variants to address each stage of the process. If you are using models like sol or terra to generate hypotheti…

Couldn't someone build an adaptive system, where the llm is frequently judging the difficulty of a task and switching to a more/less powerful model?

[deleted]

Re: I burned all my tokens researching how to save tokens

#26

Post by hotstickyballs »

Bender wrote: post #23

I am not a proper developer and only use AI for faster research of topics so please forgive my ignorance. Could one not save a lot of money on tokens by using the 80/20 or 90/10 rule in that 90% of AI usage is on local models and save that last 10% or less for the frontier models where the local model did not meet the needs? Did they cover this and I misunderstood?

This is like the "half of my marketing spend is wasted" quote. The complexity is finding out which half.

Re: I burned all my tokens researching how to save tokens

#27

Post by kkarpkkarp »

I am doing this trick.

https://neuronowa.pl/posts/porada-jak-wydajniej-kodowac-w-cu...

It is Polish so please translate. But tldr is: ask your agent at times to analyze last chats and update rules and skills to not repeat the same steps it repeats every time when you start a new thread. Simple as that.

Re: I burned all my tokens researching how to save tokens

#28

Post by jscottbee »

cdnsteve wrote: post #9

You should consider testing our RemembrallMCP (note I'm the author). It substantially cuts token usage and you get better results in codebases. https://github.com/roboticforce/remembrallmcp

Thanks! I thought I saw something like this a month or so back, but could not find it again.

Re: I burned all my tokens researching how to save tokens

#29

Post by Bender »

hotstickyballs wrote: post #26
Bender wrote: post #23

I am not a proper developer and only use AI for faster research of topics so please forgive my ignorance. Could one not save a lot of money on tokens by using the 80/20 or 90/10 rule in that 90% of AI usage is on local models and save that last 10% or less for the frontier models where the local model did not meet the needs? Did they cover this and I misunderstood?

This is like the "half of my marketing spend is wasted" quote. The complexity is finding out which half.

I think the idea or methodology is something along the line of one starts with the local models then when they hit a wall then continue with their current results in a frontier model, thus the other half is those last bits one could not compute locally.

Re: I burned all my tokens researching how to save tokens

#30

Post by dominotw »

resonious wrote: post #20
dofm wrote: post #12

It sometimes feels to me that cloud AI providers have convinced people that cloud AI is worth it because of all the ways people have been able to use cloud AI to write blog posts about using cloud AI to make cloud AI more efficient for something that they haven't shipped yet and aren't really ready to talk about.

Trust me there are plenty of us using cloud AI to actually ship stuff. We just aren't writing blog posts about it.

what did you ship?
Post reply on HN