Live data from Hacker News

You're probably using Agent Skills wrong

notes.ansonbiggs.com

11–20 of 30 posts

Re: You're probably using Agent Skills wrong

#12
post #8

I've been able to avoid this kind of markdown library architecture with very chatty tool feedback. Interaction with a responsive environment is much better than static chunks of "skill" text. For example, imagine a domain constraint: "You must use tool ABC before calling tool XYZ" This can either be in some static prompt scheme somewhere, or it can be the live result of a tool call. If you make everything tool callin…

Yup! I feel pretty strongly that every little nit pick and instruction you pass into your model is murdering your output. Having a hook that executes on tool calls is significantly better than telling your agent to follow your repos specific format/lint/style/test constraints

Re: You're probably using Agent Skills wrong

#13

TL;DR don't have your agent write skills using only its latent knowledge, otherwise you may as well not use a skill in the first place and let it summon that latent knowledge on the fly. Not sure if this take is correct though. I suspect self-generated skills help the agent avoid having to "decompress" its latent knowledge, which might save tokens? idk, I am not an expert

Skills can transfer one session's latent knowledge to all other sessions.

Re: You're probably using Agent Skills wrong

#16
post #8

I've been able to avoid this kind of markdown library architecture with very chatty tool feedback. Interaction with a responsive environment is much better than static chunks of "skill" text. For example, imagine a domain constraint: "You must use tool ABC before calling tool XYZ" This can either be in some static prompt scheme somewhere, or it can be the live result of a tool call. If you make everything tool callin…

Can you go into more detail about your setup and use cases?

Re: You're probably using Agent Skills wrong

#17
post #14

I read this post thinking "Finally! Finally someone will explain to me what I've been missing because 'skills' just seem to be re-usable text that help make prompting faster." Nope. Still the same.

Agree; posts like this frustrate me.

Tldr: you're doing it wrong but I will not show you how to do it right. I also did not run the bench using my approach but it definitely “vibes better” to me, and I reject your actual research paper.

Come on, show us some actual skills.

That one you use all the time looks a hell of a lot like “I wont a deterministic shell script for something a skill saying ‘run the shell script’”

Is that what you do? How much time do you spend on them? How do you stop the agent from making a bunch of very similar skills? How do you deal with the explosion of the total number of skills impacting your token use? Do you use skills from github, or is that bad practice? Why?

So many unanswered questions; so little content. :/

Re: You're probably using Agent Skills wrong

#18

TL;DR don't have your agent write skills using only its latent knowledge, otherwise you may as well not use a skill in the first place and let it summon that latent knowledge on the fly. Not sure if this take is correct though. I suspect self-generated skills help the agent avoid having to "decompress" its latent knowledge, which might save tokens? idk, I am not an expert

I now have rules to not let agent write any docs or processes. Pretty much anything LLM auto-generated are of zero reuse value.

Autogenerated content is good scaffolding, but then I have a rule where if I mark heading with "(by-human)" the section shouldn't be changed by LLM without permission.

Re: You're probably using Agent Skills wrong

#19
post #14

I read this post thinking "Finally! Finally someone will explain to me what I've been missing because 'skills' just seem to be re-usable text that help make prompting faster." Nope. Still the same.

Agree on article frustrations. Perhaps a better explanation, skills are just disk-cached prompts conditioned on verified success. The conditioned on verified success part might seem inconsequential, but it’s the whole thing that gives skills their value. Also the fact that their loading can be scoped to a certain calling context.

Re: You're probably using Agent Skills wrong

#20
post #14

I read this post thinking "Finally! Finally someone will explain to me what I've been missing because 'skills' just seem to be re-usable text that help make prompting faster." Nope. Still the same.

Agree on article frustrations. Perhaps a better explanation, skills are just disk-cached prompts conditioned on verified success. The conditioned on verified success part might seem inconsequential, but it’s the whole thing that gives skills their value. Also the fact that their loading can be scoped to a certain calling context.

> conditioned on verified success

Thank you! That made it clear to me why it's an useful caching technique.

Post reply on HN