I don't disagree with the article, but I also don't disagree with RTK. The output of these commands is not optimized for agents (or humans) for that matter.
The Token Compression Illusion: Why I'm Skeptical of RTK
41–50 of 121 posts
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#42Earlier quoted context omitted.
no, it doesn't upset me. I am open for discussion, there might be things I miss and don't understand. I am just trying to get why it's been pushed so hard lately and if the benefits are really there. Sorry, if I sounded upset to you, but I am trying to be really civil and just genereally curious
Well, I'm sorry as well. I mistakenly assumed you were being confrontational. There are a lot of people who have negative knee jerk reactions to any AI stuff, new workflows (I'll agree there is a lot of garbage being shilled in this space), etc., and I jumped the gun by lumping you into that group.
Nope, I am doing my master thesis on finetuning llms at 36, so I am into this stuff, but it’s been very weird lately. I’ve been self-taught dev and I definitely was missing computer science concepts so excited to fill the gaps, although the timing wasn’t perfect.
Good conversation! Great pushback against my arguments. That’s what I signed on with hacker news and missing that spirit recently
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#43I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…
I have to say I made a similiar mistake with trusting semantic search is the next big thing. My opinion shifted, but it made sense for me for too long
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#44I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…
I'll go further and note that some of the optimizations I've seen in rtk for things like `git status` have actually bubbled up into the model layer -- Codex is regularly making tool calls like `git status --short` instead of `git status`.
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#45I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…
This is why I Blind A/B test everything.
I burn a ton of tokens, but things actually have to prove their value. And the vast majority of things do not come close to doing so.
I have my own AI agent full of stuff. I blind A/B test everything, but I also don't think the results are all that useful as a signal to others.
Just because I Blind A/B test it 4 months ago, it's maybe not meaningful today.
Maybe the word choices I use dramatically impact things.
I do it, because I can prove the value, and see it with my own eyes. I don't even bother publishing the specific Blind A/B tests.
Also, I've seen other people try to Blind A/B test and get it very wrong. If your measurements aren't good, the test is meaningless.
I don't know. We're all working on these problems together. There's a lot of black magic (which is why I rely on hooks a lot). I'm sure I have tons of black magic, I have a large little AI Agent.
But what I know for certain, is it works for me. All it takes is for me to not use it, and I honestly don't know how everyone currently works with AI.
I will link it, but it is not an endorsement for what you do. Mostly only other software engineers use it. And it's so very specific to the things I have to do.
At best, maybe it sparks an idea for you to implement on your own.
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#46Author of the text here. I will be honest with why I wrote it, the rtk ai looks very odd to me as software engineer, the number of stars, no mention of accuracy and how management is pushing that stuff to optimize costs. Now people are wrapping every possible command in rtk and trying to handle all major possible commands and decide which output you should get.
Why didn’t you offer any real world usage numbers to illustrate your point? I found this unhelpful.
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#47The core of the problem is that there are a million tools that make AI better, and no ways to measure whether AI is working better.
Big companies with popular products have it. They do something between normal product analytics and chatbot evals to figure out if users are being successful in their sessions. That's the job.
But any given dev, with between 3 and 50 sessions a day? Like, I have no idea what makes the LLM better. It's all vibes.
My company has a whole stack here. Preferred harnesses, preferred models, skills, the shape of our code, everything. There's gotta be a way to measure whether this setup is working for us, at 1 / 1-million-th the scale of a Claude Code.
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#48Earlier quoted context omitted.
I am the author the text. What do you mean by aggresive context management with subagents? Would you add a lopp that would trim the context? Both of those tasks seem even more difficult
I believe they mean aggressive delegation to minimize context bloat in the coordinating agent.
This is a really useful technique in my experience. The harnesses are starting to do it more on their own but if you encourage the use of more subagents, I find it's typically nothing but win.
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#49I am glad articles like this are finally starting to get some momentum around what I call the LLM magic box industry. From caveman mode to RTK to semantic search and everything in between. Developers have become magicians that cast spells instead of engineers. It sucks at work especially with everyone so sure that their magic spell is the one for ultimate token savings. My criteria are: if it’s not in a harness it’s…
[deleted]
Re: The Token Compression Illusion: Why I'm Skeptical of RTK
#50> Mainstream CLIs and developer tools can easily ship a native --compact or --json-stream flag tailored for LLM consumption.
Until they do, they won't soon , rtk, caveman, ponytail and many others are just trying to address every growing costs (for 2K org, its around 2.5M, for now), so these are trade-offs we are all know and adjusting, but unlike the author claims we know the trade-off well and forking these tools, benchmarking, verifying the output quality matches our needs and so on to make it work for us, so no blindly.
For solo devs, yes, they might not really need it, self hosting another model to save would be better option. But for orgs thats a spicy part.
Yes, its good that we see these articles are shedding some light but like we do with these tools, lets also consume these articles with a grain of salt.