I am thinking that a small tool that simply refuses to pass large CLI output to the LLM and warns it to filter the results before reading would achieve this better as the LLM would be forced into thinking and writting the filter itself.
Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
11–20 of 97 posts
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#12Have terms been established to describe these types of tools? How do I refer to small utilities to perform specific transformations to LLM behavior? CLI filter seems pretty good to describe this tool conversationally but not so much when searching, they some low cardinality keywords.
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#13I've tried rtx and lean-ctx and these tools seem to end up confusing the agent more than helping. Any saving is irrelevant if the agent decides to work around the tool and makes even more calls than it would otherwise.
I don't know about cost saving, but if it's keeping the context size down I've had a lot better results using subagents to keep a higher order conversation clean for longer.
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#14Would this have any impact on the response quality from the agent?
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#15I've tried rtx and lean-ctx and these tools seem to end up confusing the agent more than helping. Any saving is irrelevant if the agent decides to work around the tool and makes even more calls than it would otherwise. I don't know about cost saving, but if it's keeping the context size down I've had a lot better results using subagents to keep a higher order conversation clean for longer.
Subagents help with costs too, as they can run on much cheaper models.
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#16I have my own llm wrapping harness, which does this and has a few more tricks. For example, it doesn’t have a lot of mcp but it does have search_mcp and load_mcp tools (and search_skills) so the llm can find what it needs when it needs it without bloating the normal baseline context. The LLMs have proved really good at using them. There is also a waypoint tool they can use to record their thinking in the context without it being the final output. Am thinking about a search_expert to find colleagues it can bring into conversations too. And a lot of other stuff.
Pro tip they worked well for me with response truncation: in the truncated output, say that the full text is available in /tmp/whereever.txt - that way, the llm will be able to query and read more using built in tools without reissuing the big tool call.
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#17I am thinking that a small tool that simply refuses to pass large CLI output to the LLM and warns it to filter the results before reading would achieve this better as the LLM would be forced into thinking and writting the filter itself.
I simply use LLM to create filter for my personal use. I have already put that specific instruction in the plugin doc in case you are interested.
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#18The docs are missing any examples of what this does, instead showing _how_ it works - and only for the codebase itself, rather than the behavior of the app. What would be useful: - examples of text that can be filtered, and why that would be valuable - a data flow diagram of runtime behavior, showing how filtering removes unnecessary context
Thanks for your feedback. Will put this in place. Meanwhile, please checkout architecture doc and plugin. The plugin doc could a little bit giving insight of what it does.
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#19Would this have any impact on the response quality from the agent?
Frankly, not at all.
Re: Show HN: Lowfat – pluggable CLI filter that saved 91.8% of my LLM tokens
#20Would this have any impact on the response quality from the agent?
Yes, and never for the better.