Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Bitter Lesson of LLM Extensions

sawyerhood.com

Post reply on HN
Print view View original

11–20 of 81 posts

Re: The Bitter Lesson of LLM Extensions

#11

Post by pluralmonad »

CuriouslyC wrote: post #7
mkagenius wrote: post #4

> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work. and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP. But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills.…

Skills do something you could already do with folder level readme files and hyperlinks inside source, but in a vendor-locked-in way. Not a fan.

They are just text files though. I'm sensitive to vendor lock-in and do not perceive a standard folder structure and bare text files to be that.

Re: The Bitter Lesson of LLM Extensions

#12

Post by mkagenius »

zby wrote: post #6
mkagenius wrote: post #4

> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work. and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP. But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills.…

I still don't get what is special about the skills directory - since like forever I instructed Claud Code - "please read X and do Y" - how skills are different from that?

The difference is that the code in the directory (and the markdown) are hardcoded and known to work beforehand.

Re: The Bitter Lesson of LLM Extensions

#13

Post by robot-wrangler »

vessenes wrote: post #3

> "If I could short MCP, I would" I mean, MCP is hard to work with. But there's a very large set of things that we want a hardened interface to out there - if not MCP, it will be something very like it. In particular, MCP was probably overly complicated at the design phase to deal with the realities of streaming text / tokens back and forth live. That is, it chose not to abstract these realities in exchange for some…

I always see the hard/complex criticism but find it confusing.. what is the perceived difficulty with MCP at the implementation level? (I do understand the criticism about exhausting tokens with tool-descriptions and stuff, but that's a different challenge)

Doesn't seem like implementation could be more simple. Just JSON-RPC and API stuff. For example the MCP hello-world with python and FastMCP is practically 1-to-1 with a http/web flavored hello-world in flask

Re: The Bitter Lesson of LLM Extensions

#14

Post by bavell »

zby wrote: post #6
mkagenius wrote: post #4

> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work. and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP. But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills.…

I still don't get what is special about the skills directory - since like forever I instructed Claud Code - "please read X and do Y" - how skills are different from that?

Not really special, just officially supported and I'm guessing how best to use it baked in via RL. Claude already knows how skills work vs learning your own home-rolled solution.

Re: The Bitter Lesson of LLM Extensions

#15

Post by mkagenius »

fzysingularity wrote: post #9
mkagenius wrote: post #4

> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work. and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP. But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills.…

I definitely see the value and versatility of Claude Skills (over what MCP is today), but I find the sandboxed execution to be painfully inefficient. Even if we expect the LLMs to fully resolve the task, it'll heavily rely on I/O and print statements sprinkled across the execution trace to get the job done.

> but I find the sandboxed execution to be painfully inefficient

sandbox is not mandatory here. You can execute the skills on your host machine too (with some fidgeting) but it's a good practice and probably for the better to get in to the habit of executing code in an isolated environment for security purposes.

Re: The Bitter Lesson of LLM Extensions

#16

Post by bavell »

pluralmonad wrote: post #11
CuriouslyC wrote: post #7

Earlier quoted context omitted.

Skills do something you could already do with folder level readme files and hyperlinks inside source, but in a vendor-locked-in way. Not a fan.

They are just text files though. I'm sensitive to vendor lock-in and do not perceive a standard folder structure and bare text files to be that.

Yeah, the reason I like Skills better than MCP is specifically because skills are just plain text.

Re: The Bitter Lesson of LLM Extensions

#17

Post by j2kun »

I don't see how "they improved the models" is related to the bitter lesson. You are still injecting human-level expertise (whether it is by prompts or a structured API) to compensate for the model's failures. A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference.

Re: The Bitter Lesson of LLM Extensions

#18

Post by simonw »

zby wrote: post #6
mkagenius wrote: post #4

> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work. and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP. But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills.…

I still don't get what is special about the skills directory - since like forever I instructed Claud Code - "please read X and do Y" - how skills are different from that?

They're not. They are just a formalization of that pattern, with a very tiny extra feature where the model harness scans that folder on startup and loads some YAML metadata into the system prompt so it knows which ones to read later on.

Re: The Bitter Lesson of LLM Extensions

#19

Post by zby »

I believe that what we need is treating prompts as stochastic programs and using a special shell for calling them. Claude Code and Codex and other coding agents are like that - now everybody understands that they are not just coding assistants they are a general shell that can use LLM for executing specs. I would like to have this extracted from IDE tools - this is what I am working on in llm-do.

Re: The Bitter Lesson of LLM Extensions

#20

Post by idle_zealot »

j2kun wrote: post #17

I don't see how "they improved the models" is related to the bitter lesson. You are still injecting human-level expertise (whether it is by prompts or a structured API) to compensate for the model's failures. A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference.

> A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference.

This is what I expected the post to be about before clicking.

Post reply on HN