Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Token Compression Illusion: Why I'm Skeptical of RTK

mroczek.dev

Post reply on HN
Print view View original

101–110 of 121 posts

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#101

Post by celrod »

arcanemachiner wrote: post #29
cityofdelusion wrote: post #22

I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…

The idea itself is sound: If you can reduce the signal-to-noise ratio in the context window, then that's a good thing. Whether or not RTK actually does this has not been established. I would be glad to see some proper benchmarks done on the actual difference this tool makes (not some meaningless "up to 90%" type of language).

I found this, which has some: https://arxiv.org/pdf/2605.28876 TLDR: RTK does not look good according to the author's benchmark.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#102

Post by Otterly99 »

I completely agree with this post. After I used it in one session of 300k tokens, I had maybe 3k tokens saved. Plus, if commits really are an issue for you in term of tolen consumption, you can always ask to hand over the reigns and apply the commits yourself as a rule (unless you're operating in a loop).

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#103

Post by aaulia »

tontinton wrote: post #54
cityofdelusion wrote: post #22

I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…

Totally wrong, you underestimate the frontier's incompetence in anything other than building LLM models (ehm ehm flickering TUI for a year "written like a game engine"). I ran a bunch of benchmarks and there are proven ways to reduce tokens while achieving the same results (finding the same CVEs / finding the same bugs in CRs, etc...). See https://maki.sh , it's my own little proof.

Tried it just now. The onboarding process could be better, for example guide user to pick the available models if providers is setup but it's not anthropic. Wasted a bit of time foguring out that the provider is detected, just he models was wrong.

But I like it, code processing is freaking fast.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#104

Post by tontinton »

0xc133 wrote: post #73
tontinton wrote: post #66

Earlier quoted context omitted.

Enjoy, I can't go back to other agents now, too spoiled by the speed

I also have become a maki convert and I really like it. I ran into an issue with the dynamic model provider that I should probably make a patch for; list_models doesn’t use the ` models` output at all but instead tries to look up ` resolve`’s base URL + /v1/models, which breaks on a provider like Z.ai which doesn’t have /v1/ anywhere in the path…

Create an issue! :)

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#105

Post by ChadNauseam »

tontinton wrote: post #54
cityofdelusion wrote: post #22

I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…

Totally wrong, you underestimate the frontier's incompetence in anything other than building LLM models (ehm ehm flickering TUI for a year "written like a game engine"). I ran a bunch of benchmarks and there are proven ways to reduce tokens while achieving the same results (finding the same CVEs / finding the same bugs in CRs, etc...). See https://maki.sh , it's my own little proof.

Downloaded it and giving it a try. I love the aesthetics and how colorful it is! It is also pleasant to use and fast, although my personal preference is TUIs that don't break scrollback (claude code actually does a good job at this).

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#106

Post by tontinton »

pdimitar wrote: post #70
tontinton wrote: post #69

Earlier quoted context omitted.

It's a TUI you're right, but it's also a harness. As much as I hate to admit, T the tools you provide, the descriptions, and prompts, all amount to pretty big changes in experience, even using the same models.

That didn't help very much. What did you mean by "agents" earlier on? The tool/harness or the LLM itself? Also -- can you make Maki enforce the underlying LLM to use stuff like fd/rg and not always default to find/grep, for example? And stop trying to do bash-isms in a zsh system?

I meant harnesses / TUIs, sorry to confuse.

Using fd/rg sounds interesting, honestly it would require little tweaks to the bash tool lua plugin, either add to the description to prefer these binaries instead or something like that.

In general though I much prefer "advising" and encouraging the LLM to use the native tools like grep l/glob, they are implemented to be super fast, and you will get better parser output.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#107

Post by no-name-here »

dfee wrote: post #81
tontinton wrote: post #54

Earlier quoted context omitted.

Totally wrong, you underestimate the frontier's incompetence in anything other than building LLM models (ehm ehm flickering TUI for a year "written like a game engine"). I ran a bunch of benchmarks and there are proven ways to reduce tokens while achieving the same results (finding the same CVEs / finding the same bugs in CRs, etc...). See https://maki.sh , it's my own little proof.

an agent harness built in rust with ratatui - checks out. i've built one myself. i don't maintain it, and continue to use opencode, but it was worth it to learn how agent harnesses work. anyway, what's the real pitch on why i should move on from opencode to maki?

> what’s the real pitch on why

I’m not OP, but parent comment and linked site https://maki.sh talk about token reduction.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#109

Post by Zababa »

cityofdelusion wrote: post #22

I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…

My criteria is "do they measure performance, or at least even try to?". Caveman [1], RTK [2] and more recently ponytail [3] don't or use a few trivial tests. Those projects don't measure performance on widely used benchmarks (like SWE Pro and stuff), that have their issues but at least it would give some indication. They also don't measure "big model + caveman vs smaller model".

I've had a few times where removing all custom instructions that I started using with model N-2 made model N perform way better, so I'm very suspicious of everything that changes how the model works, it's easy to get degraded performance silently and suddenly you're paying latest Opus costs for 6 months old Sonnet performance.

[1]: https://github.com/JuliusBrussee/caveman

[2]: https://github.com/rtk-ai/rtk

[3]: https://github.com/DietrichGebert/ponytail

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#110

Post by Zababa »

wren6991 wrote: post #108

Yeah, RTK is problematic because of its focus on associations between kanji and arbitrary English keywords, many of which are poorly chosen and... oh it's an LLM thing.

Don't worry, what is kept vs what is removed by the RTK LLM thing is just as arbitrary as the RTK English keywords with no measure of performance!
Post reply on HN