Earlier quoted context omitted.
A man with a spoon can dig a swimming pool but Id prefer a backhoe
sudo apt-get install backhoe
mise use -g backhoe
The Bitter Lesson of LLM Extensions
71–80 of 81 posts
Re: The Bitter Lesson of LLM Extensions
#72Well, that's just great. The academic community has been using the term "skill" for years, to refer to classes of tasks at which LLMs exhibit competence. Now OpenAI has usurped the term to refer to these inference-guiding .md files. I'm not looking forward to having to pick through a Google hit list for "LLM skills", figuring out which publications are about skills in the traditional sense and which are about the Ope…
What about open? Or ai? Neither is really what they are offering. Open they are not (weight doesn't count) and don't get me started on that statistical machine they call artificial intelligence. Misleading through and through.
Re: The Bitter Lesson of LLM Extensions
#73ChatGPT apps, announced this month, feels a lot like original ChatGPT Plugin announced 3 years back. The only difference is how plugin are invoked. For ChatGPT plugin, we have to choose one from a drop down, and for apps - we could just include a plugin name in prompt.
Is there any other difference in the end-user side?
Re: The Bitter Lesson of LLM Extensions
#74[dead]
Re: The Bitter Lesson of LLM Extensions
#75Earlier quoted context omitted.
There's no lock-in there. Tell your agent of choice to read the preamble of all the documents in the skills directory, and tell it that when it has a task that matches one of the preambles, it should read the rest of the relevant file for full instructions. There are far fewer dependencies for skills than for MCP. Even a model that knows nothing about tool use beyond how to run a shell command, and has no support for…
> There are far fewer dependencies for skills than for MCP. This is wrong and an example magical thinking. AI obviously does not mean that you can ship/use software without addressing dependencies? See for example https://github.com/anthropics/skills/blob/main/slack-gif-cre... or worse, the many other skills that just punt on this and assume CLI tools and libraries are already available
It is categorically not wrong. With an MCP you have at a minimum all the same dependencies and on top of that a dependency on your agent supporting MCP. With skills, a lot of the time you don't need to ship code at all - just an explanation to the agent of how to use standard tools to access an API for example, but when you do need to ship code, you don't need to ship any more code than with an MCP.
The trivial evidence of this, is that if you have an MCP server available, the skill can simply explain to the agent how to use the MCP server, and so even the absolute worst case for skills is parity.
Re: The Bitter Lesson of LLM Extensions
#76Earlier quoted context omitted.
Skills are less exciting because they're effectively documentation that's selectively loaded. They are a bigger deal in a sense because they remove the need for all the scaffolding MCPs require. E.g. I needed Claude to work on transcripts from my Fathom account, so I just had it write a CLI script to download them, and then I had it write a SKILL.md, and didn't have to care about wrapping it up into an MCP. At a clie…
Skills are good for context management as everything that happens while executing the skill remains “invisible” to the parent context, but they do inherit the parent context. So it’s pretty effective for a certain set of problems. MCP is completely different, I don’t understand why people keep comparing the two. A skill cannot connect to your Slack server. Skills are more similar to sub-agents, the main difference be…
A skill can absolutely connect to your slack server. Either by describing how to use standard tools to do so, or by including code.
Most of my skills connect to APIs.
Re: The Bitter Lesson of LLM Extensions
#77Skills.md will in time have same problem as MCP, they will bloat the context. I wonder if we could just have the scripts without the descriptions and LLM would have been trained to search the most useful things in specific folder.
This seems like a solvable engineering problem. For example, you could have a lightweight subagent with its own context for reading the skills and determining which to use
Re: The Bitter Lesson of LLM Extensions
#78> "I expect us to go back to extending our agents with the most accessible programming language: natural language." I don't agree with this. Natural language is so ambiguous. At least for software development the hard work is still coming up with clearly defined solutions. There is a reason for why math has its own domain specific language.
> Natural language is so ambiguous. As a former tech comms guy I will say: Natural language can be bent into arbitrary precision. Write something, then enter a read-rewrite-reread loop as the devil's advocate (this is key) until it stops being ambiguous or having multiple conceivable interpretations. Yes with English this process can be a pain in the butt, until you get the hang of it.
This process can be handled by a “turn server”
Was a pain to set up, but you can score the context completion and then if the score is under 98% or something, “ask” clarifying questions of the requesting agent or person or system
Re: The Bitter Lesson of LLM Extensions
#79Earlier quoted context omitted.
> Natural language is so ambiguous. As a former tech comms guy I will say: Natural language can be bent into arbitrary precision. Write something, then enter a read-rewrite-reread loop as the devil's advocate (this is key) until it stops being ambiguous or having multiple conceivable interpretations. Yes with English this process can be a pain in the butt, until you get the hang of it.
This process can be handled by a “turn server” Was a pain to set up, but you can score the context completion and then if the score is under 98% or something, “ask” clarifying questions of the requesting agent or person or system
Kagi and Perplexity are drawing a blank on "turn server".
Re: The Bitter Lesson of LLM Extensions
#80Earlier quoted context omitted.
This process can be handled by a “turn server” Was a pain to set up, but you can score the context completion and then if the score is under 98% or something, “ask” clarifying questions of the requesting agent or person or system
Kagi and Perplexity are drawing a blank on "turn server".
I guess we just call them turn servers but technically it’s a “Stateful Dialogue Manager”