Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGENTS.md outperforms skills in our agent evals

vercel.com

Post reply on HN
Print view View original

81–90 of 212 posts

Re: AGENTS.md outperforms skills in our agent evals

#81

Post by throwaway314155 »

ethmarks wrote: post #73
ai-christianson wrote: post #66

Earlier quoted context omitted.

They say compressed... but isn't this just "minified"?

Minification is still a form of compression, it just leaves the file more readable than more powerful compression methods (such as ZIP archives).

I'd say minification/summarization is more like a lossy, semantic compression. This is only relevant to LLM's and doesn't really fit more classical notions of compression. Minification would definitely be a clearer term, even if compression _technically_ makes sense.

Re: AGENTS.md outperforms skills in our agent evals

#84

Post by stingraycharles »

denolfe wrote: post #78

PreSession Hook from obra/superpowers injects this along with more logic for getting rid of rationalizing out of using skills: > If you think there is even a 1% chance a skill might apply to what you are doing, you ABSOLUTELY MUST invoke the skill. IF A SKILL APPLIES TO YOUR TASK, YOU DO NOT HAVE A CHOICE. YOU MUST USE IT. While this may result in overzealous activation of skills, I've found that if I have a skill re…

I always say “invoke your skill to do X. then invoke your skill to do Y. “

works pretty well

Re: AGENTS.md outperforms skills in our agent evals

#86

Post by chr15m »

gbnwl wrote: post #60
chr15m wrote: post #46

I'm not sure if this is widely known but you can do a lot better even than AGENTS.md. Create a folder called .context and symlink anything in there that is relevant to the project. For example READMEs and important docs from dependencies you're using. Then configure your tool to always read .context into context, just like it does for AGENTS.md. This ensures the LLM has all the information it needs right in context f…

Cheaper? Loading every bit of documentation into context every time, regardless of whether it’s relevant to the task the agent is working on? How? I’d much rather call out the location of relevant docs in Claude.md or Agents.md and tell the agent to read them only when needed.

As they point out in the article, that approach is fragile.

Cheaper because it has the right context from the start instead of faffing about trying to find it, which uses tokens and ironically bloats context.

It doesn't have to be every bit of documentation, but putting the most salient bits in context makes LLMs perform much more efficiently and accurately in my experience. You can also use the trick of asking an LLM to extract the most useful parts from the documentation into a file, which you then re-use across projects.

https://github.com/chr15m/ai-context

Re: AGENTS.md outperforms skills in our agent evals

#87

Post by rao-v »

Calavar wrote: post #9
rao-v wrote: post #7

In a month or three we’ll have the sensible approach, which is smaller cheaper fast models optimized for looking at a query and identifying which skills / context to provide in full to the main model. It’s really silly to waste big model tokens on throat clearing steps

I thought most of the major AI programming tools were already doing this. Isn't this what subagents are in Claude code?

Sub-agents are typically one of the major models but with a specific and limited context + prompt. I’m talking about a small fast model focused on purely curating the skills / MCPs / files to provide to the main model before it kicks off.

Basically use a small model up front to efficiently trigger the big model. Sub agents are at best small models deployed by the bigger model (still largely manually triggered in most workflows today)

Re: AGENTS.md outperforms skills in our agent evals

#88

Post by chr15m »

d3m0t3p wrote: post #57
chr15m wrote: post #46

I'm not sure if this is widely known but you can do a lot better even than AGENTS.md. Create a folder called .context and symlink anything in there that is relevant to the project. For example READMEs and important docs from dependencies you're using. Then configure your tool to always read .context into context, just like it does for AGENTS.md. This ensures the LLM has all the information it needs right in context f…

Yea but the goal it not to bloat the context space. Here you "waste" context by providing non usefull information. What they did instead is put an index of the documentation into the context, then the LLM can fetch the documentation. This is the same idea that skills but it apparently works better without the agentic part of the skills. Furthermore instead of having a nice index pointing to the doc, They compressed i…

The minification is a great idea. Will try this.

Their approach is still agentic in the sense that the LLM must make a tool cool to load the particular doc in. The most efficient approach would be to know ahead of time which parts of the doc will be needed, and then give the LLM a compressed version of those docs specifically. That doesn't require an agentic tool call.

Of course, it's a tradeoff.

Re: AGENTS.md outperforms skills in our agent evals

#89

Post by dca88 »

heliumtera wrote: post #39
delduca wrote: post #32

Ah nice… vercel is vibecoded

web people opted into react, dude. that says a lot. they used prisma to handle their database interactions. they preached tRPC and screamed TYPE SAFETY!!! you really think these guys will ever again touch the keyboard to program? they despise programming.

This. I read this article and it pains me to see the amount of manpower put into doing anything but actually getting work done.
Post reply on HN