Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You need to rewrite your CLI for AI agents

justin.poehnelt.com

Post reply on HN
Print view View original

51–60 of 74 posts

Re: You need to rewrite your CLI for AI agents

#51

Post by jsunderland323 »

danw1979 wrote: post #49
jsunderland323 wrote: post #16

I'm working on a CLI now. The pattern I used was this: 1) made a docs command that printed out the path of the available docs $ my-cli docs - README.md - DOC1.md - dir2/DOC2.md 2) added a --path flag to print out a specific doc (tried to keep each doc less than 400 lines). $ my-cli docs --path dir2/DOC2.md # Contents of DOC2.md 3) added embeddings so I could do semantic search $ my-cli search "how do I install x?" [1…

I really like this - especially the embedded search. What do the embeddings and model cost you in terms of binary size ?

Umm it's not as bad I thought it would be. ~16mb per locale. ~28k words in the english docs.

Not schilling, just easier to show you the repo since it's open source. https://github.com/coast-guard/coasts

Re: You need to rewrite your CLI for AI agents

#52

Post by [deleted] »

sheept wrote: post #17

This feels completely speculative: there's no measure of whether this approach is actually effective. Personally, I'm skeptical: - Having the agent look up the JSON schemas and skills to use the CLI still dumps a lot of tokens into its context. - Designing for AI agents over humans doesn't seem very future proof. Much of the world is still designed for humans, so the developers of agents are incentivized to make agen…

[deleted]

Re: You need to rewrite your CLI for AI agents

#53

Post by rkagerer »

gck1 wrote: post #20
sheept wrote: post #17

This feels completely speculative: there's no measure of whether this approach is actually effective. Personally, I'm skeptical: - Having the agent look up the JSON schemas and skills to use the CLI still dumps a lot of tokens into its context. - Designing for AI agents over humans doesn't seem very future proof. Much of the world is still designed for humans, so the developers of agents are incentivized to make agen…

Yeah, people seem to forget one of the L's in LLM stands for Language, and human language is likely the largest chunk in training data. A cli that is well designed for humans is well designed for agents too. The only difference is that you shouldn't dump pages of content that can pollute context needlessly. But then again, you probably shouldn't be dumping pages of content for humans either.

I also feel like it's just a matter of time until someone cracks the nut of making agents better understand GUI's and more adept at using them.

Is there progress happening in that trajectory?

Re: You need to rewrite your CLI for AI agents

#54

Post by sheept »

danw1979 wrote: post #47
sheept wrote: post #27

Earlier quoted context omitted.

Sure, but the post itself brags, > gws ships 100+ SKILL.md files Which must altogether be hundreds of lines of YAML frontmatter polluting your context.

Claude Code, at least, will only load a SKILL.md file into context when it’s invoked by the user or LLM itself, i.e. in demand.

Claude will load the name and description of each enabled skill into context at startup[0]; the LLM needs to know what it can invoke, after all. It's negligible for a few skills, but a hundred skills will likely have some impact, e.g. deemphasizing other skills by adding noise.

[0]: https://platform.claude.com/docs/en/agents-and-tools/agent-s...

Re: You need to rewrite your CLI for AI agents

#56

Post by mellosouls »

John Carmack made this observation (cli-centred dev for agents) a year ago:

LLM assistants are going to be a good forcing function to make sure all app features are accessible from a textual interface as well as a gui. Yes, a strong enough AI can drive a gui, but it makes so much more sense to just make the gui a wrapper around a command line interface that an LLM can talk to directly.

https://x.com/ID_AA_Carmack/status/1874124927130886501

https://xcancel.com/ID_AA_Carmack/status/1874124927130886501

Andrej Karpathy reiterated it a couple of weeks ago:

CLIs are super exciting precisely because they are a "legacy" technology, which means AI agents can natively and easily use them, combine them, interact with them via the entire terminal toolkit.

https://x.com/karpathy/status/2026360908398862478

https://xcancel.com/karpathy/status/2026360908398862478

Re: You need to rewrite your CLI for AI agents

#57

Post by resiros »

I don't follow the need to write CLIs for the agent. Why not use simply the API and document it well? The token difference between using an API and CLI is not that much, and models are trained to use REST APIs and understand their patterns, compared to your random CLI.

Re: You need to rewrite your CLI for AI agents

#59

Post by lidn12 »

mellosouls wrote: post #56

John Carmack made this observation (cli-centred dev for agents) a year ago: LLM assistants are going to be a good forcing function to make sure all app features are accessible from a textual interface as well as a gui. Yes, a strong enough AI can drive a gui, but it makes so much more sense to just make the gui a wrapper around a command line interface that an LLM can talk to directly. https://x.com/ID_AA_Carmack/sta…

Thanks for sharing these contents. They are very interesting. I found "making all app features accessible from a textual interface..." actually quite challenging in cerntain domains such as graphics related editing tools. Though many editing functions can be exposed as CLI properly, but the content being edited is very hard to be converted into texts without losing its geometric meaning. Maybe this is where we truly need the multimodal models or where training on specialized data is needed.
Post reply on HN