Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I burned all my tokens researching how to save tokens

quesma.com

Post reply on HN
Print view View original

231–237 of 237 posts

Re: I burned all my tokens researching how to save tokens

#231

Post by camgunz »

seanmcdirmid wrote: post #229
SkyBelow wrote: post #211

Earlier quoted context omitted.

How errors add depend upon how you treat them. Two independent task? You have a 95% chance of each task being correct, so total success is .95^2 = 90.25% correct. Having two AIs verify the same task with each being 95% correct? That's a 5% error per each, so 1-(.05^2) = 99.75% correct. That is assuming fully independent error rates, which is the real issue. If it is pure hallucinations, then that is an okay assumptio…

This. The math of coincident errors is why we can get reliable AI, but it really is a matter of how the errors manifest and how they are detected. I’ve written a bit about this and my approach is based on clean room and N-versions.

Reliable AI is an oxymoron and deeply misleading. Even if you can stack models the efficiency tanks. Also, just looking at the Wikipedia page (https://en.wikipedia.org/wiki/N-version_programming#:~:text=...), don't LLMs already have all these problems?

Re: I burned all my tokens researching how to save tokens

#232

Post by iririririr »

bkotrys wrote: post #123
iririririr wrote: post #84

echo "$OUT" | claude-mem-save so, does agents are ultra aware their output is used in a bash line or did bash now have some magic quote super power? I'd bet this only really work 2% of the time in real world.

At the shell level, the agent does not need to know about Bash. The contents of "$OUT" are passed to echo as one argument, then sent to claude-mem-save over stdin. Bash does not re-evaluate them as shell code. printf '%s\n' "$OUT" would be more robust than echo. Whether claude-mem-save can use arbitrary output is a separate question.

there are a million ways both of those can break.

but the other comment point this out is already dead and buried. lol. so much cope from vibecoders.

Re: I burned all my tokens researching how to save tokens

#233

Post by seanmcdirmid »

camgunz wrote: post #231
seanmcdirmid wrote: post #229

Earlier quoted context omitted.

This. The math of coincident errors is why we can get reliable AI, but it really is a matter of how the errors manifest and how they are detected. I’ve written a bit about this and my approach is based on clean room and N-versions.

Reliable AI is an oxymoron and deeply misleading. Even if you can stack models the efficiency tanks. Also, just looking at the Wikipedia page ( https://en.wikipedia.org/wiki/N-version_programming#:~:text=... ), don't LLMs already have all these problems?

You aren’t thinking this through. You were just told that this wasn’t stacking but you keep insisting it is. N-versions is explicitly not stacking, and the same solutions that solve the Knight and Leveson issue for humans solves it for LLMs as well (they can be wrong about the same part of the specification, but will very unlikely be wrong about it in the same way, so you try to measure scalar rather than Boolean outcomes).

Re: I burned all my tokens researching how to save tokens

#234

Post by stilesja »

bkotrys wrote: post #179
stilesja wrote: post #167

Earlier quoted context omitted.

I hear you! I read the article and thought the same thing because I've actually built a surprisingly similar pipeline myself, but it culminates with Human review. I've got a trivia app (Hedge Calibrated Trivia) whose whole content pipeline is built on a similar architecture of models evaluating models. I didn't want the same trivia that every other app has and I naively imagined that it would be easy to ask an LLM to…

Very similar experience. The review UI might actually be the most important part of the whole pipeline. Does the confidence ranking match what you end up rejecting?

It's a future evaluation for me, as I'm keeping the app with a "Data Not Collected" App Store label for now. Once I have some visibility to stats of how people answer each question and at which confidence level I expect it to reveal bad/poorly worded questions that made it through my pipeline.

Also on my to do list is an upgrade similar to my de-duplication logic. Currently I pass in my approved set of questions as a list of topics and questions to avoid. It works well especially when combined with the more deterministic comparison of generated questions vs approved question by the separate model. The upgrade would be to pass in the questions REJECTED by me. These are question which made it through ever automated gauntlet of models and code, yet still ultimately were sent to the waste bin by human review.

I'm imagining that by including the list of examples of what NOT to do, it could inform the initial Generation to be smarter. I currently have a limited list of a hand picked "bad question shapes" where I provide the question and why its bad kept statically in my prompt. But I've realized sorting through thousands of questions, I've probably built implicit data through rejection that I may not actually recognize myself which the model may pick up on.

Re: I burned all my tokens researching how to save tokens

#235

Post by camgunz »

seanmcdirmid wrote: post #233
camgunz wrote: post #231

Earlier quoted context omitted.

Reliable AI is an oxymoron and deeply misleading. Even if you can stack models the efficiency tanks. Also, just looking at the Wikipedia page ( https://en.wikipedia.org/wiki/N-version_programming#:~:text=... ), don't LLMs already have all these problems?

You aren’t thinking this through. You were just told that this wasn’t stacking but you keep insisting it is. N-versions is explicitly not stacking, and the same solutions that solve the Knight and Leveson issue for humans solves it for LLMs as well (they can be wrong about the same part of the specification, but will very unlikely be wrong about it in the same way, so you try to measure scalar rather than Boolean out…

> You aren’t thinking this through.

No need to be so polite; I might just be very stupid.

> You were just told that this wasn’t stacking but you keep insisting it is.

Having an LLM generate something and then asking another LLM to generate that something is asking for 2 hallucinations, not 1. This doesn't go away the more LLMs you throw in there. The only reason this works at all for things like ChatGPT is their prompts are super different (and the editor LLM is way cheaper, of course).

And you still have to check for hallucinations (before you say throw another hallucinating LLM at it know I will make you pay infinite forum shame points if you do).

> the same solutions that solve the Knight and Leveson issue for humans solves it for LLMs as well

No, LLMs are plagued by the same problems: https://arxiv.org/abs/2606.20158. After all what are you gonna do, say "build me a SQLite, but definitely not like [other model X] would"? The solution is a very unambiguous spec, but that asymptotically approaches code soooooo....

Finally, I think you're hand-waving NVX. A program that can digest any spec and measure how well any program meets it is pretty wild. Good luck with that one.

Re: I burned all my tokens researching how to save tokens

#236

Post by th0ma5 »

camgunz wrote: post #235
seanmcdirmid wrote: post #233

Earlier quoted context omitted.

You aren’t thinking this through. You were just told that this wasn’t stacking but you keep insisting it is. N-versions is explicitly not stacking, and the same solutions that solve the Knight and Leveson issue for humans solves it for LLMs as well (they can be wrong about the same part of the specification, but will very unlikely be wrong about it in the same way, so you try to measure scalar rather than Boolean out…

> You aren’t thinking this through. No need to be so polite; I might just be very stupid. > You were just told that this wasn’t stacking but you keep insisting it is. Having an LLM generate something and then asking another LLM to generate that something is asking for 2 hallucinations, not 1. This doesn't go away the more LLMs you throw in there. The only reason this works at all for things like ChatGPT is their prom…

[dead]

Re: I burned all my tokens researching how to save tokens

#237

Post by cdnsteve »

kittoes wrote: post #125
cdnsteve wrote: post #9

You should consider testing our RemembrallMCP (note I'm the author). It substantially cuts token usage and you get better results in codebases. https://github.com/roboticforce/remembrallmcp

Checked it out, but C# is forsaken.

C# is supported now!
Post reply on HN