The problem with stuff like this is that it's hard to evaluate. You don't even know when the agent is using a skill, or if the skill even made a difference. Using tools lets you at least instrument tool calls, and control what gets executed.
Superpowers: How I'm using coding agents in October 2025
201–210 of 242 posts
Re: Superpowers: How I'm using coding agents in October 2025
#202I'm so curious around what people's median experience is of AI coding tools. I've tried agents every now and then, recently for something very simple- add an option to request csb format in a data api. The results were, well, not good. . . I ended up undoing literally all changes because writing from scratch was a lot easier than trying to refactor the total mess it has made from what I'd have thought was a trivial f…
Re: Superpowers: How I'm using coding agents in October 2025
#203I'm so curious around what people's median experience is of AI coding tools. I've tried agents every now and then, recently for something very simple- add an option to request csb format in a data api. The results were, well, not good. . . I ended up undoing literally all changes because writing from scratch was a lot easier than trying to refactor the total mess it has made from what I'd have thought was a trivial f…
For example, I have found Claude Code and Codex to be tremendously helpful for my web development work. But my results for writing Zig are much worse. The gap in usefulness of agents between tasks is very big.
The skill ceiling for using agents is also surprisingly high. Planning before coding, learning agent capabilities, environment setup, and context engineering can make a pretty massive difference to results. This can all be a big time sink though, and I'm not sure if it's really worth it if agents don't already work decently well for the work you do.
But with the performance gaps between domains, and the skill curve, I can definitely understand why there is such a divide between people claiming agents are ridiculously overhyped, and people who claim coding is fundamentally changing.
Re: Superpowers: How I'm using coding agents in October 2025
#204Re: Superpowers: How I'm using coding agents in October 2025
#205I'm so curious around what people's median experience is of AI coding tools. I've tried agents every now and then, recently for something very simple- add an option to request csb format in a data api. The results were, well, not good. . . I ended up undoing literally all changes because writing from scratch was a lot easier than trying to refactor the total mess it has made from what I'd have thought was a trivial f…
If you just tell it some vague feature to make, it's gonna do whatever it's gonna do and maybe it will be good, maybe it won't. It probably won't. The more specific you are the better it will do.
Instead of trying to 100x or 1000x your effort, try to just 2x or 3x it. Give it small specific tasks and check the work thoroughly, use it as an extension of yourself rather than a separate "agent".
I can tell it to write a function and it'll do pretty well. I can ask it to fix things if it doesn't do it the way I want. This is all easy. Maybe I can even get it to write a whole class at once or maybe I can get it to write a class in a few iterations.
The key here is I'm in control, I'm doing the design, I'm making the decisions. I can ask it how I should approach a problem and often it'll have great suggestions. I can ask it to improve a function I've written and it'll do pretty well. Some times really well.
The point is I'm using it as a tool I'm not using it to do my job for me. I use it to help me think I don't use it to think for me. I don't let it run away from me and edit a whole bunch of files etc, I keep it on a tight leash.
I'm sold now. I am, indisputably, a better software developer with LLMs in my toolbelt. They help me write better code, faster, while learning things faster and easier, it's really good. Reliability isn't a problem when I keep a close eye on it. It's only a problem if you try to get it to do a whole big task on it's own.
Re: Superpowers: How I'm using coding agents in October 2025
#206Re: Superpowers: How I'm using coding agents in October 2025
#207I can't recommend this post strongly enough. The way Jesse is using these tools is wildly more ambitious than most other people. Spend some time digging around in his https://github.com/obra/Superpowers repo. I wrote some notes on this last night: https://simonwillison.net/2025/Oct/10/superpowers/
Curious what you think of sub agents, don't they still consume a massive amount of tokens compared to simply running in main context? I'm skeptical of any process that starts massively delegating to sub agents. I'm on Pro and don't think its worth upgrading to 200 a month just to not pollute main context.
The ability to isolate context-noisy subtasks (like agentically searching through a large codebase by grepping through dozens of irrelevant files to find the one you actually need) unlocks much longer-running loops, and therefore much more complex tasks.
And you don't need a system this complicated to take advantage of it. Literally just a simple "codebase-searcher" agent (and Claude can vibe the agent definition for you) is enough to see the benefit first-hand. Once you see it, if you're like me, you will see opportunities for subagents everywhere.
Re: Superpowers: How I'm using coding agents in October 2025
#208I'm so curious around what people's median experience is of AI coding tools. I've tried agents every now and then, recently for something very simple- add an option to request csb format in a data api. The results were, well, not good. . . I ended up undoing literally all changes because writing from scratch was a lot easier than trying to refactor the total mess it has made from what I'd have thought was a trivial f…
- Case 1 was troubleshooting what turned out to be a complex and messy dependency injection issue. I got pulled in to unblock a team member, who was struggling with the issue. My efforts were a dead-end, but Claude (Code) managed to spot a very odd configuration issue. The codebase is a large, legacy one.
- Case 2 was the same codebase, I again got pulled in to unblock a team mate, investigating why some integration tests were running individually, but not when run as a group. Clearly there was a pretty obvious smoking gun, and I managed to isolate the issue after about 15-30 minutes of debugging. I had set Claude on the goose chase as well, and as I closed the call with my teammate, I noticed it had found the same exact two lines that were causing the issue.
Clearly, it occasionally does insane stuff, or lies its little pants off. The number of times where it "got me" are fairly low, however, and its usefulness to me is extreme. In the cases above, it out-did a teammate who has at least 10 years of experience, and equalled me in the one case and outdid me in the other, with over 25 years now. I have a similar wonderment to your situation, but the opposite: "how are people NOT finding value in this?".
Re: Superpowers: How I'm using coding agents in October 2025
#209A big issue working with code agents is what I call context-recall: restoring context when working on a new feature or fix, that builds on recent work. Meaning, the previous work may have involved multiple CLI sessions, summaries dumped to various markdown files like documentation files, plan files, issue files, PR-descriptions etc. Then when starting new work with a code agent you have to hunt down all of this scatt…
Re: Superpowers: How I'm using coding agents in October 2025
#210I'm so curious around what people's median experience is of AI coding tools. I've tried agents every now and then, recently for something very simple- add an option to request csb format in a data api. The results were, well, not good. . . I ended up undoing literally all changes because writing from scratch was a lot easier than trying to refactor the total mess it has made from what I'd have thought was a trivial f…
My experience is coding agents work best for either absolute beginners, or for lead engineers who have experience building and training teams. Getting good results out of coding agents is a lot like getting good results out of interns: You need to explain clearly what you want, ask them to explain what they plan to do, give feedback on the plan, and then very carefully review the results. You need to write up your preferred coding style, you need a document that explains "how to work on this project", you need to establish rigorous automated quality checks, etc. Using a coding agent heavily is a lot like being promoted to "technical lead", with all the tradeoffs that entails.
Here's a recent discussion of a good blog post on the subject: https://news.ycombinator.com/item?id=45503867
I have gotten some very nice results out of Sonnet 4.5 this past week. But it required using my "technical management" skills very heavily. And it required lots extremely careful code review. Clear documentation, robust QA, and code review are the main bottlenecks.
I mean, the time I spent writing AGENTS.md wasn't wasted. I'm writing down a lot of stuff I used to teach in pairing sessions.