With all due respect, but if humans can figure out how new unseen programs work by using -h and seeing what options exist and what they do, I am sure robots can figure it out too, or else they weren’t that intelligent to begin with.
The difference is that humans will remember the options listed by -h after a few times of using the tool.
AIs don't. If they don't reach for the --help switch every time they'll attempt the statistical average, which may or may not work.
For super-common or popular tools like `gh` the usage is already in the training data though.
This feels completely speculative: there's no measure of whether this approach is actually effective. Personally, I'm skeptical: - Having the agent look up the JSON schemas and skills to use the CLI still dumps a lot of tokens into its context. - Designing for AI agents over humans doesn't seem very future proof. Much of the world is still designed for humans, so the developers of agents are incentivized to make agen…
Yeah, people seem to forget one of the L's in LLM stands for Language, and human language is likely the largest chunk in training data. A cli that is well designed for humans is well designed for agents too. The only difference is that you shouldn't dump pages of content that can pollute context needlessly. But then again, you probably shouldn't be dumping pages of content for humans either.
It's not obvious that human language is or should be the largest amount of training data. It's much easier to generate training data from computers than from humans, and having more training data is very valuable. In paticular, for example, one could imagine creating a vast number of debugging problems, with logs and associated command outputs, and training on them.
I strongly disagree here. Yes, build CLIs. No, don't target then at agents.
Build for humans, including good man pages or `--help` docs as needed.
If LLMs are worth the name AI they will understand how to discover and use Unix-style commands. In my experience, this is exactly the case and I need only say "I use tool X for use case) Y."
That's how artificial this "intelligence" is, when LLMs can't even use text based tools full of txt based documentation formatted coherently without those very tools being adapted.
It looks like an AI generated fluff article without any evidence. People also did this for image generators as if you needed these arcane templates to prompt them, but actually the latest models are great at figuring out what you want from messy human input. Similarly LLMs can use regular CLI just fine. But how do you write a hype FOMO article about the fact that actually you don't need to do anything...
If AI agents need CLIs, then whats stopping them from using APIs directly. I see CLIs as good wrappers over APIs, and nothing more. What more will CLIs provide which `curl -X POST` can't/won't provide?
Surely the skill for a cli tool is a couple of lines describing common usage, and a description of the help system?
Sure, but the post itself brags, > gws ships 100+ SKILL.md files Which must altogether be hundreds of lines of YAML frontmatter polluting your context.
Claude Code, at least, will only load a SKILL.md file into context when it’s invoked by the user or LLM itself, i.e. in demand.
Some of this seems a bit overhyped. I like CLI tools with json output that can be piped through jq. I've seen llms do that with existing tools. The human needs and llm needs seem to overlap, especially if the human is using scripts and piping between tools. The number of times it implies you didn't need to validate "human" input until llms arrived is scary too. I'm also surprised to hear them say llms shouldn't Googl…
> The number of times it implies you didn't need to validate "human" input until llms arrived is scary too.
I took away a completely different message: humans and LLMs make different mistakes that require different validation.
I'm working on a CLI now. The pattern I used was this: 1) made a docs command that printed out the path of the available docs $ my-cli docs - README.md - DOC1.md - dir2/DOC2.md 2) added a --path flag to print out a specific doc (tried to keep each doc less than 400 lines). $ my-cli docs --path dir2/DOC2.md # Contents of DOC2.md 3) added embeddings so I could do semantic search $ my-cli search "how do I install x?" [1…
I really like this - especially the embedded search. What do the embeddings and model cost you in terms of binary size ?
No. Nope. Agents do just fine with all sorts of CLIs. Old standards, new custom stuff, whatever. The CLIs I’ve seen agents struggle with are those that wrap an enormous, unwieldy, poorly designed API under one namespace. All of Google Workspace apis, for example.
Which is perhaps why the author was focussing on optimising LLM usage of such unwieldy APIs without blowing out the context ?
I don’t disagree with your point about agent abilities with older, concise, well-represented-in-training-data tools though.