You're probably using Agent Skills wrong
notes.ansonbiggs.com
You're probably using Agent Skills wrong
1–10 of 30 posts
Re: You're probably using Agent Skills wrong
#2TL;DR don't have your agent write skills using only its latent knowledge, otherwise you may as well not use a skill in the first place and let it summon that latent knowledge on the fly.
Not sure if this take is correct though. I suspect self-generated skills help the agent avoid having to "decompress" its latent knowledge, which might save tokens? idk, I am not an expert
Re: You're probably using Agent Skills wrong
#3TL;DR don't have your agent write skills using only its latent knowledge, otherwise you may as well not use a skill in the first place and let it summon that latent knowledge on the fly. Not sure if this take is correct though. I suspect self-generated skills help the agent avoid having to "decompress" its latent knowledge, which might save tokens? idk, I am not an expert
I now have rules to not let agent write any docs or processes. Pretty much anything LLM auto-generated are of zero reuse value.
Re: You're probably using Agent Skills wrong
#4What if I want a way to open up a latent space prompt without having to type it all out everytime?
Re: You're probably using Agent Skills wrong
#5Having agents create their own skills works well if you also give it a layer of verifiability.
Eg. Ask the agent to write a skill then get it to prompt a subagent to use the skill, then iterate until it verifies the task was completed correctly
Re: You're probably using Agent Skills wrong
#6Having agents create their own skills works well if you also give it a layer of verifiability. Eg. Ask the agent to write a skill then get it to prompt a subagent to use the skill, then iterate until it verifies the task was completed correctly
Yes, I wrote a forge skill to do this via a/b testing and third agent to judge the result.
https://github.com/bjcoombs/ai-native-toolkit/blob/main/skil...
It hardens a skill through judge-panel refinement rounds, it’s a quality gate that runs after authoring, not an authoring tool.
Re: You're probably using Agent Skills wrong
#7TL;DR don't have your agent write skills using only its latent knowledge, otherwise you may as well not use a skill in the first place and let it summon that latent knowledge on the fly. Not sure if this take is correct though. I suspect self-generated skills help the agent avoid having to "decompress" its latent knowledge, which might save tokens? idk, I am not an expert
It seems so obvious: How would it know better than it already does?
Yet I’ve seen people succeed with „write me a prompt“ prompts. The model makes something up, often it makes sense.
They are like plans in that way: It’s not exactly novel knowledge, but it at least encodes it somewhere to make the process verifiable beforehand and a bit more repeatable.
I wouldn’t be surprised if it improves performance a little, just like thinking blocks do (every model reasons now).
Re: You're probably using Agent Skills wrong
#8I've been able to avoid this kind of markdown library architecture with very chatty tool feedback. Interaction with a responsive environment is much better than static chunks of "skill" text. For example, imagine a domain constraint:
"You must use tool ABC before calling tool XYZ"
This can either be in some static prompt scheme somewhere, or it can be the live result of a tool call.
If you make everything tool calling and environmental, you effectively have a lazily evaluated & dynamic prompt scheme.
I like to think of this as context for the context. The better you map the environment and descriptions of it to the agent, the less top-down prompting is required.
If you set up the harness correctly, you can run circles around a lot of what passes as AI innovation with powershell in a while loop. Adding static markdown document soup on top of this would only reduce performance in the general case.
Re: You're probably using Agent Skills wrong
#9Having agents create their own skills works well if you also give it a layer of verifiability. Eg. Ask the agent to write a skill then get it to prompt a subagent to use the skill, then iterate until it verifies the task was completed correctly
Yes, I wrote a forge skill to do this via a/b testing and third agent to judge the result. https://github.com/bjcoombs/ai-native-toolkit/blob/main/skil... It hardens a skill through judge-panel refinement rounds, it’s a quality gate that runs after authoring, not an authoring tool.
This is a pretty neat, I suspect that eventually every skill will have some sort of validation/verification loop like this
Re: You're probably using Agent Skills wrong
#10What if I want a way to open up a latent space prompt without having to type it all out everytime?
Skills for repitition are totally valid. Having a version control skill that explains that I use gitea works great. My point is that asking for a skill that tells us if our program will get stuck before taking on a halting problem won't get you any further than just starting the task with xhigh thinking