A Claude Code and Codex Skill for Deliberate Skill Development

Iterative, or recursive?

IMHO, if you're working on large feature changes, before nudging the agent to write any code, it's best to:

1. establish consensus, just in the chat, on the problem domain — i.e. the business-domain problem you're solving (as if the agent is your contact at a software-development contractor, and you're sitting down to pin down what you want from them)

2. co-write with the agent a hierarchical-bullet-pointed design document (this should be an actual .md file, not just in the chat) — letting the agent generate + edit most of this, but nitpicking it thoroughly for problems and decision-vagueness, forcing all design-level decisions to be made up-front here

3. tell it to translate the design spec into a skeleton for a BDD spec test suite, to be populated as it implements

4. let the agent free to actually do the impl — where the agent is free to add/modify/delete unit tests and integration tests and so on, but where it must keep the design-spec file and the structure of the derived BDD spec tests fixed (and, before considering itself done, ensure that A. the BDD spec tests are all fleshed out with proper logic reflecting their labels, and 2. they all pass.)

5. At this point you might be done. But if your project is absolutely huge, you might do another "sprint" at this point, starting again from the top by defining new business requirements, amending the design, getting the agent to add to the BDD suite, etc. (Or, if you want to talk everything out up front, you'd insert a step between 2 and 3 of "breaking the design down into milestones" — where the agent will only create BDD spec items for the current milestone, solve for them, gets approval, and then move onto the next milestone.)

Yes, I'm basically saying you should do waterfall with LLMs. Waterfall can actually be rather pleasant, when the whole process happens over the course of an hour.

And the key point here, for understanding: after the project (or after each milestone for a large project), you can have the agent walk you through the code it wrote, explaining it to you in the chat — with the constraint that it shouldn't bother to explain anything already "implied" by the design.

You can then have it turn this explanation of "the surprising parts" into code comments — and the resulting comments would actually be of the kind humans would write, rather than being pro-forma garbage!

Interesting responses here. I think most are missing the point.

For me, the main lesson here is seeing and learning from how others are using skills. Yesterday I was watching a Matt Pocock class on using agents and he was also showing off skills, such as how he uses a "grill-me" skill to develop product requirement document. I am certainly not going to do exactly what he does, but I now have my own ideas about how to develop requirements and implement them.

After all, in the word of Anthropic engineers themselves, Claude is like a talented engineer, but lacks expertise. Skills are folders and files that build expertise. Another important thing I leaned from Pocock is that the longer the context (or token size), the dumber the responses tend to get. So skills are another way to present the problem to an LLM in a compact manner and get optimized response.

Claude also has behavioral traits. So if someone iteratively builds a skill, it is most likely not going to port well to another user, because each of us chat differently. This is why I hesitate to share my skill folder with my colleagues. But I will certainly demo what I built so that they can see what's possible and figure out their own workflows.

So the value is in seeing how someone else builds using Claude, and imitate in your own way. Very much like when I first learned programming, I was copying code form Kernighan and Richie's C book, but then changing up things to understand how it works and later customize the code for my purpose.

I mentioned behavioral traits for another reason- the author is a psychologist and it is really interesting to see how she interacts with Claude, which is probably very different from how programmers use Claude. Tangentially, she (and a host of other experts in the field) left Twitter long time ago. I'm going to install bsky/mastodon and follow them, because I think it's important to watch how expert non-programmers are using LLMs.

I was surprised to learn that some skills don't even describe exactly what steps to take or what to do. They just kind of give a motivational speech which I guess primes the model to output better text for a certain task.

https://github.com/anthropics/claude-code/blob/main/plugins/...

This frontend design skill that claude uses basically just begs it to pick nice fonts and make the design coherent. No specifics about which fonts or how to make nice color schemes and layout.

Not the OP, but I've been thinking about this problem a lot - as devs we're overly reliant on vibes for evaluating coding agents. This is already a problem, and especially so if you're working in an engineering organization where a bad edit to AGENTS.md can cause silent regressions for everyone in the codebase.

To solve this, I've built an agent-native tool to run evaluations based on merged PRs in your codebase. Basically you can ask Claude to evaluate whether the skill made things better/worse on real tasks, and to then iteratively improve it

Stalking your profile (sorry..) I see you're pretty deep in the eval space, so I'm super curious what your approach has been to being rigorous for things like skill changes?

I think just keeping an ear to the ground, and sometimes through querying LLMs themselves about how to solve certain problems. Most vendors seem to follow Anthropics lead on configuration and agent instruction (though it certainly varies). I just picked up this link from HN today which was good https://claude.com/blog/how-claude-code-works-in-large-codeb... - I haven't been using hooks until now which the article emphasised, but I can see how they would be useful (for example for running formatters to enforce file format rules / clean up after the agents).