31–40 of 97 posts
I have my own llm wrapping harness, which does this and has a few more tricks. For example, it doesn’t have a lot of mcp but it does have search_mcp and load_mcp tools (and search_skills) so the llm can find what it needs when it needs it without bloating the normal baseline context. The LLMs have proved really good at using them. There is also a waypoint tool they can use to record their thinking in the context with…
The docs are missing any examples of what this does, instead showing _how_ it works - and only for the codebase itself, rather than the behavior of the app. What would be useful: - examples of text that can be filtered, and why that would be valuable - a data flow diagram of runtime behavior, showing how filtering removes unnecessary context
Thanks for your feedback. Will put this in place. Meanwhile, please checkout architecture doc and plugin. The plugin doc could a little bit giving insight of what it does.
But the one thing I expected to see in the Readme was an example of: takes this tool run output: XXXXXX and converts it to: XX for a savings of 40% of tokens.
This looks like a nice (and useful) project, so thanks for sharing!
The docs are missing any examples of what this does, instead showing _how_ it works - and only for the codebase itself, rather than the behavior of the app. What would be useful: - examples of text that can be filtered, and why that would be valuable - a data flow diagram of runtime behavior, showing how filtering removes unnecessary context
Thanks for your feedback. Will put this in place. Meanwhile, please checkout architecture doc and plugin. The plugin doc could a little bit giving insight of what it does.
I would like to have deeper comparison with alternatives like rtk, which are already fast and written in rust, also the previous comments mentioned something that has been a know problem with rtk that it sometimes strips the thing that the llm needs (or expects, causing more work to need to happan not less)
It's a massive red flag to me when you could get decent data to see if your thing actually works, and they don't even attempt to...
Have the LLM use your tool, run it on several of the coding benchmarks. If you're stingy, run it on the ones that don't cost much.
Otherwise, I'm going to assume it doesn't actually work. If it did - Claude, Antigravity, Codex, Pi, or some major player would bundle tools like this into the CLI / harness.
AFAIK, none of the major players do. That's a sign to me these don't work in general.
I've tried building some tools specific to bug fixing. Intelligently feeding context massively helps smaller models. But, what I've found - surprisingly - is that a smaller, much better focused, including a lot of helpful data as well, has almost no impact on larger models compared to what they do by default.
You do save some tokens, though, which is what they're claiming - but not ~99%...
Earlier quoted context omitted.
Frankly, not at all.
I have a suspicion that the model would miss more context unless you are very precise about what FAT means in each context. However, loved the idea.
Earlier quoted context omitted.
Thanks for your feedback. Will put this in place. Meanwhile, please checkout architecture doc and plugin. The plugin doc could a little bit giving insight of what it does.
Thanks for your effort! I also think having examples of raw output before vs after using lowfat would be useful as well
I am thinking that a small tool that simply refuses to pass large CLI output to the LLM and warns it to filter the results before reading would achieve this better as the LLM would be forced into thinking and writting the filter itself.
I simply use LLM to create filter for my personal use. I have already put that specific instruction in the plugin doc in case you are interested.
I would like to have deeper comparison with alternatives like rtk, which are already fast and written in rust, also the previous comments mentioned something that has been a know problem with rtk that it sometimes strips the thing that the llm needs (or expects, causing more work to need to happan not less)
None of these tools measure how effective they are... It's a massive red flag to me when you could get decent data to see if your thing actually works, and they don't even attempt to... Have the LLM use your tool, run it on several of the coding benchmarks. If you're stingy, run it on the ones that don't cost much. Otherwise, I'm going to assume it doesn't actually work. If it did - Claude, Antigravity, Codex, Pi, or…
Your suggestion to using coding benchmarks doesn't really capture the whole picture. I haven't seen a benchmark using kubectl.
> AFAIK, none of the major players do. That's a sign to me these don't work in general.
It's a lose/lose for major players. If it works well, it will lower their revenue. Also there's a high risk it'll significantly worsen results for some people, even if it improves results for others.