Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Bitter Lesson of LLM Extensions

sawyerhood.com

Post reply on HN
Print view View original

51–60 of 81 posts

Re: The Bitter Lesson of LLM Extensions

#51

Post by btbuildem »

vessenes wrote: post #3

> "If I could short MCP, I would" I mean, MCP is hard to work with. But there's a very large set of things that we want a hardened interface to out there - if not MCP, it will be something very like it. In particular, MCP was probably overly complicated at the design phase to deal with the realities of streaming text / tokens back and forth live. That is, it chose not to abstract these realities in exchange for some…

MCP came in a bit too early, when the conceptual shift of hadn't fully kicked in yet. I see it as a bit of a Horseless Carriage, and I think Skills came in to counter that. My sense is that this will settle into a sort of self-assembling code golem, where ambiguous parts are handled in LLM-space, and clear, well-defined things are handled in code-space.

Re: The Bitter Lesson of LLM Extensions

#52

Post by ttkciar »

jswny wrote: post #47
ttkciar wrote: post #32

Well, that's just great. The academic community has been using the term "skill" for years, to refer to classes of tasks at which LLMs exhibit competence. Now OpenAI has usurped the term to refer to these inference-guiding .md files. I'm not looking forward to having to pick through a Google hit list for "LLM skills", figuring out which publications are about skills in the traditional sense and which are about the Ope…

Skills are an Anthropic feature

Whoops, you are right. I misread the article.

Re: The Bitter Lesson of LLM Extensions

#53

Post by arjie »

Custom GPTs are pretty old, but I recently found a use for them. My wife wanted some meeting note-taking and task recording assistance and I found that making a Custom GPT with a trivial Notion API that was scoped to one page[0] with structure that was encoded in the API was a quick couple-hour thing that unlocked a lot of utility for her (the default Notion MCP is "too broad"). It helped that this Custom GPT sits in her ChatGPT UI and she doesn't have to have another app or whatever to make it work.

We liked it quite a bit, but it led to some funny things. We use Reminders to keep our home to-do lists, hers and mine in one list with two sections. I wanted to take this existing flow we had and make it work with a Custom GPT. It's practically impossible because Reminders:

* doesn't have a good API through EventKit

* requires a pop-up permission grant in the UI

So in the end, I did end up making somewhat of an MCP server for it, running it on an old Macbook Pro I had and then sticking Amphetamine on in closed-lid display-sleep mode hooked up to my Tailnet and exposed via a Cloudflare tunnel so that we could use ChatGPT to interact with the thing. Yes, you can see how insane that whole thing is. But there's quite a lot of value to have your AI agent just be the one thing.

0: https://wiki.roshangeorge.dev/w/Blog/2025-10-17/Custom_GPTs

Re: The Bitter Lesson of LLM Extensions

#54

Post by marshall300791 »

idle_zealot wrote: post #20
j2kun wrote: post #17

I don't see how "they improved the models" is related to the bitter lesson. You are still injecting human-level expertise (whether it is by prompts or a structured API) to compensate for the model's failures. A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference.

> A "bitter lesson" would be that the model can do better without any injection, but more compute power, than it could with human interference. This is what I expected the post to be about before clicking.

The bitter lesson here is that it all goes back to natural language rather than lower level of abstractions

Re: The Bitter Lesson of LLM Extensions

#56

Post by stingraycharles »

vidarh wrote: post #35
mkagenius wrote: post #4

> Skills are the actualization of the dream that was set out by ChatGPT Plugins .. But I have a hypothesis that it might actually work now because the models are actually smart enough for it to work. and earlier Simon Willison argued[1] that Skills are even bigger deal than MCP. But I do not see as much hype for Skills as it was for MCP - it seems people are in the MCP "inertia" and having no time to shift to Skills.…

Skills are less exciting because they're effectively documentation that's selectively loaded. They are a bigger deal in a sense because they remove the need for all the scaffolding MCPs require. E.g. I needed Claude to work on transcripts from my Fathom account, so I just had it write a CLI script to download them, and then I had it write a SKILL.md, and didn't have to care about wrapping it up into an MCP. At a clie…

Skills are good for context management as everything that happens while executing the skill remains “invisible” to the parent context, but they do inherit the parent context. So it’s pretty effective for a certain set of problems.

MCP is completely different, I don’t understand why people keep comparing the two. A skill cannot connect to your Slack server.

Skills are more similar to sub-agents, the main difference being context inheritance. Sub-agents enable you to set a different system prompt for those which is super useful.

Re: The Bitter Lesson of LLM Extensions

#57

Post by euroderf »

uriegas wrote: post #46

> "I expect us to go back to extending our agents with the most accessible programming language: natural language." I don't agree with this. Natural language is so ambiguous. At least for software development the hard work is still coming up with clearly defined solutions. There is a reason for why math has its own domain specific language.

> Natural language is so ambiguous.

As a former tech comms guy I will say:

Natural language can be bent into arbitrary precision. Write something, then enter a read-rewrite-reread loop as the devil's advocate (this is key) until it stops being ambiguous or having multiple conceivable interpretations.

Yes with English this process can be a pain in the butt, until you get the hang of it.

Re: The Bitter Lesson of LLM Extensions

#58

Post by zbyforgotp »

uriegas wrote: post #46

> "I expect us to go back to extending our agents with the most accessible programming language: natural language." I don't agree with this. Natural language is so ambiguous. At least for software development the hard work is still coming up with clearly defined solutions. There is a reason for why math has its own domain specific language.

That’s why we need progressive hardening: https://github.com/zby/llm-do/blob/main/docs/concept_spec.md

Re: The Bitter Lesson of LLM Extensions

#59

Post by Glemkloksdjf »

lupire wrote: post #34
simonw wrote: post #18

Earlier quoted context omitted.

They're not. They are just a formalization of that pattern, with a very tiny extra feature where the model harness scans that folder on startup and loads some YAML metadata into the system prompt so it knows which ones to read later on.

So "skills" are a hack around the LLM not actually being very smart? Interesting.

No, skills are a set of manifested and tested 'skills' which reduce the 'mental load' of the LLM and reduces the context the LLM needs to do things reproducable.

Similiar to what humans do.

Re: The Bitter Lesson of LLM Extensions

#60

Post by virajk_31 »

lerp-io wrote: post #29

can someone explain to me the difference between MCP and calling a cli tool eg curl or whatever i still don’t understand i’ve been using ai for years now.

MCP is tool calling with continued context/rich context, tool calling alone will PROBABLY die after single call whereas MCP keeps continuity by design (You can use MCP for tool calling but not vice versa). Hope this help you understand.
Post reply on HN