Live data from Hacker News

The Token Compression Illusion: Why I'm Skeptical of RTK

mroczek.dev

21–30 of 121 posts

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#21

Earlier quoted context omitted.

Did you use an LLM for the blog post? it reads like it in places.

I have raycast shortcut for fix grammar, it might done more damage than adding a, an, the or changing tenses.

A content-free 2nd "paragraph" like this turned me off immediately.

> But in the current dev tools gold rush, if something sounds too good to be true, it almost always is.

The people who are interested in RTK and in criticism of RTK aren't interested in pablum like this.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#22
I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings.

My criteria are: if it’s not in a harness it’s probably not that good (the best ideas float up to Codex/Claude imo) and any GitHub advertising some percent of token savings is not to be trusted.

It’s hard to avoid the snake oil and I hope people start thinking critically on this stuff.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#23

I've been trying out RTK and it seems kinda alright. I doubt it's saving much, but the quality of the work feels similar. But if it's making a dent in token usage (which I have not personally measured), then that's great. I had to add some system prompt instructions to Pi to help it work (GPT 5.5 initially got confused when `git status` looked different than expected). The Claude Code extension appears to do a proper…

so how do you justify it's usage if it's not saving much and the work feels similiar. They have 664 issues open and some of them are quite funny, the tools are called and return success even though they aren't even installed. My take is that handling so many versions and so many different tools shouldn't be the work of any single repo. The responsibility should be either on coding agent to compress or best case scena…

I'm not justifying its usage, and I don't have to.

I've been trying it out for a couple days and it seems kinda OK or whatever. If that upsets you, then that's your problem.

I might dump it later on if it doesn't provide much if a benefit. I typically try out new things, then cull whatever doesn't work. This tool seems pretty neutral for now, at least.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#24

I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…

There is a conflict of interest, though.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#25

Earlier quoted context omitted.

Use the right tool for the job. If you need a piece of information that is buried somewhere, or a high-level summary/distillation of a larger body of info, then subagents may be the right tool for the job. If you need all the gathered context for later use (i.e. distilled context is insufficient), then subagents probably are not the right tool for the job.

if your corebase requires a million tokens, then youre probably going to break more than you fix

If you are using a million tokens in a single context window, you are using the entire toolbox incorrectly.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#26

> 1. Gamified Savings vs. Your Actual API Bill Tool use output represents a large amount of my output. I'll take 3.7M tokens saved on 3.9M tokens of input. Tokens saved are tokens saved. > 3. Where Are the Accuracy Benchmarks? As a user of RTK, it would be nice to see accuracy benchmarks. However, I've seen no evidence of the model missing anything critical as a result of the compression. As part of their design phil…

I was looking through the issues as investigation. Some issues that caught my attention are looking quite bad https://github.com/rtk-ai/rtk/issues/2494 https://github.com/rtk-ai/rtk/issues/2462 https://github.com/rtk-ai/rtk/issues/2395

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#27
I don't agree with the conclusion at all. I can see the value of RTK - whether it is buggy or vibe coded is kind of secondary. That basically comes down to how severe and often the bugs are.

There's no gamification of savings here. Tool output can be meaty.

Is the author skeptical of the concept, or the implementation? Because only one of those is worth critiquing.

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#28
I tried it and it does not compress messages which was 90% of my context, so it only compresses a small part of my token usage. If you read it carefully you will realize that is exactly stated. If you look at /context you will probably see that tool calls are not where you are spending token on, so a proxy that compresses tool calls will not make much impact, whilst still being true that it compresses tool calls by 8x. Its just not that important for long coding sessions for me.

"native/built-in Read or cat tools, the data is not intercepted by RTK's shell hook"

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#29

I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…

The idea itself is sound: If you can reduce the signal-to-noise ratio in the context window, then that's a good thing.

Whether or not RTK actually does this has not been established. I would be glad to see some proper benchmarks done on the actual difference this tool makes (not some meaningless "up to 90%" type of language).

Re: The Token Compression Illusion: Why I'm Skeptical of RTK

#30
Author of the text here. I will be honest with why I wrote it, the rtk ai looks very odd to me as software engineer, the number of stars, no mention of accuracy and how management is pushing that stuff to optimize costs. Now people are wrapping every possible command in rtk and trying to handle all major possible commands and decide which output you should get.
Post reply on HN