Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Token Compression Illusion: Why I'm Skeptical of RTK

mroczek.dev

Post reply on HN
Print view View original

71–80 of 121 posts

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#71

Post by lackoftactics »

tontinton wrote: post #67
lackoftactics wrote: post #62

Earlier quoted context omitted.

What is your approach for reducing token usage and is it different than rtk?

The biggest ones are: using tree-sitter to index code files as a tool, code_execution tool running a workflow of tools inside a python interpreter (monty), and not being a harness developed by the company profiting from selling you the shovels (and introducing "dynamic workflows" aka spawning 50 agents).

Tbh I could buy into what you are proposing more than rtk. It feels sane in comparison

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#72

Post by jbellis »

I feel bad that I wasted my time reading this.

On the points in the article:

1. Yes, "gain" is a vanity metric but it's harmless, nobody is being "fooled" here.

2. This could be a problem in principle, sure, but unless you're actually vetting bug reports you're just spreading FUD.

3. Again, do you have any reason to believe that the thousands of devs using rtk are silently tanking their performance without noticing? here's a thought: instead of reporting that SOMEONE SHOULD MEASURE THIS, you could, you know, measure it yourself.

4. Good lord, what is this doing in a purportedly technical article?

5. Yes, this is inherent in the problem domain, again, nobody is being "fooled".

Yes, I'm grumpy; reading this article was a waste of time.

Bias: had my first RTK pr accepted today, so I guess I probably know more about it than this guy who got offended by "gain" and spit out the first thoughts that came to mind.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#73

Post by 0xc133 »

tontinton wrote: post #66
rescbr wrote: post #57

Earlier quoted context omitted.

Maki is awesome. Thanks! I'm using it on my X220 and it flies in comparison to OpenCode et al.

Enjoy, I can't go back to other agents now, too spoiled by the speed

I also have become a maki convert and I really like it. I ran into an issue with the dynamic model provider that I should probably make a patch for; list_models doesn’t use the ` models` output at all but instead tries to look up ` resolve`’s base URL + /v1/models, which breaks on a provider like Z.ai which doesn’t have /v1/ anywhere in the path…

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#74

Post by beepbooptheory »

jbellis wrote: post #72

I feel bad that I wasted my time reading this. On the points in the article: 1. Yes, "gain" is a vanity metric but it's harmless, nobody is being "fooled" here. 2. This could be a problem in principle, sure, but unless you're actually vetting bug reports you're just spreading FUD. 3. Again, do you have any reason to believe that the thousands of devs using rtk are silently tanking their performance without noticing?…

How is 1 not more damning? It sounds like the fundamental service they are purportedly providing is not real. Am I reading it wrong?

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#75

Post by lackoftactics »

danr4 wrote: post #55
grey-area wrote: post #7

Earlier quoted context omitted.

Did you use an LLM for the blog post? it reads like it in places.

this is aboslutely entirely written by AI

As an author of the text, I can say you are „absolutely” not correct. I might be already spending too much time with llms and they start to shape my texts, so I am not proud of that either. But thanks for bringing very valuable insight to otherwise interesting discussion.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#76

Post by ilia-a »

First of all there is a way to made agents spot truncation by being aware of RTK compression and having bypass option (I use RTK_DISABLE=1) as a way of restoring original full text.

Works fine, yeah it only compresses command output so only input tokens are affected in terms of "compression".

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#77

Post by bcollins34 »

lloyd-christmas wrote: post #51
ianwalter wrote: post #46

Earlier quoted context omitted.

Why didn’t you offer any real world usage numbers to illustrate your point? I found this unhelpful.

I read another post oddly similar earlier today that has more explicit data on that authors codebase: https://codepointer.substack.com/p/cutting-llm-token-costs-w... TLDR; ~3-4% savings to actual API costs with rtk, caveman, and headroom combined, but nothing tangible on if those cost reductions came at a cost of quality. By their calculations, rtk saved them $4.96 on a $926 bill.

^recommend reading this one

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#79

Post by lackoftactics »

jbellis wrote: post #72

I feel bad that I wasted my time reading this. On the points in the article: 1. Yes, "gain" is a vanity metric but it's harmless, nobody is being "fooled" here. 2. This could be a problem in principle, sure, but unless you're actually vetting bug reports you're just spreading FUD. 3. Again, do you have any reason to believe that the thousands of devs using rtk are silently tanking their performance without noticing?…

1. Are you sure no one is fooled? It’s the main thing managers are praising rtk for and using as an argument for it’s validity. If this is gamed, then it paints a very different picture. 2. No, I didn’t vet all the reports. But they paint quite convincing picture of the problems present in the library, which has a very ambitious goals of handling every popular command and making it less verbose. 3. You know this is not a valid point. Engineers tanking performance and choosing based on hype is nothing new. Github stars and usage is not a valid argument, when the tool is not very transparent and could quietly fail. If it’s only couple percents less accuracy, most wouldn’t easily recognize it with the whole stack of skills, mcps and agents.md 4. Is it something more than a feature? If the benefit is $3 on $900 as other commenter pointed out using maybe better and well researched article than mine from codepointer, why would I risk that for all the possible bugs and worse accuracy. 5. Hard to address this one. Tough problem domain to handle with endless cli commands to capture and process properly.

Congratulations on your accepted PR. I didn’t want to make you grumpy today. If you feel I am wrong, it’s very possible. I am just a guy who wrote my point of view, it doesn’t automatically make it valid. Once again sorry for making you grumpy.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#80

Post by fumeux_fume »

ianwalter wrote: post #46
lackoftactics wrote: post #30

Author of the text here. I will be honest with why I wrote it, the rtk ai looks very odd to me as software engineer, the number of stars, no mention of accuracy and how management is pushing that stuff to optimize costs. Now people are wrapping every possible command in rtk and trying to handle all major possible commands and decide which output you should get.

Why didn’t you offer any real world usage numbers to illustrate your point? I found this unhelpful.

https://en.wikipedia.org/wiki/Brandolini%27s_law
Post reply on HN