Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agent Skills

addyosmani.com

Post reply on HN
Print view View original

121–130 of 239 posts

Re: Agent Skills

#121

Post by tokioyoyo »

swyx wrote: post #81
tokioyoyo wrote: post #75

Earlier quoted context omitted.

I’m a bit curious with these takes. Arguing in good faith - is the general assumption that people who use AI/agents/harnesses don’t ship features? We’ve been all in Claude Code since ~Septemberish, and have been able to successfully track the boost. Like the features that we ship that get used in production. Both from infrastructure side, and business logic implementations. Frontend and backend. I don’t think people…

> have been able to successfully track the boost. lets get nitty gritty on this - can you say how you did this? because a lot of people think this is an unsolved problem

For my team, it has been easy. We deal with infrastructure for the entire org, so have tickets created for every request. We also gave our own backlog for internal project, so can see burn rate, and etc. Team hasn’t changed, a lot of similar/same tasks that have taken half a day has been completely automated to a point where we just do PR review after an initial ticket is created by other teams.

There are a lot of little things we’ve tracked, and it’s just faster to implement things now. To be fair, everyone on my team has decade+ professional experience (many more non-prodessional), and we understand limitations of AI fairly well.

Re: Agent Skills

#122

Post by keeda »

wg0 wrote: post #68

Snake oil. Good to read for sure. Seems all plausible too. But snake oil nevertheless. Here's why: The slot machine can drop any hard requirement that you specifically in your AGENTS.md, memory.md or your dozens of skill markdowns. Pretty much guaranteed. These harnesses approaches pretend as if LLMs are strict and perfect rule followers and the only problem is not being able to specify enough rules clearly enough. T…

Snake oil may be a bit strong, because snake oil never works (except maybe as placebo?) whereas anything with an LLM, even though stochastic, has a pretty high chance of working.

> ... you also realize that promised productivity gains are also snake oil because reading code and building a mental model is way harder than having a mental model and writing it into code.

Not really, though it depends on the code; reading code is a skill that gets easier with practice, like any other. This is common any time you're ever in a situation where you're reading much more code than writing it (e.g. any time you have to work with a large, sprawling codebase that has existed long before you touched it.)

What makes it even easier, though, is if you're armed with an existing mental model of the code, either gleaned through documentation, or past experience with the code, or poking your colleagues.

And you can do this with agents too! I usually already have a good mental model of the code before I prompt the AI. It requires decomposing the tasks a bit carefully, but because I have a good idea of what the code should look like, reviewing the generated code is a breeze. It's like reading a book I've read before. Or, much more rarely, there's something wrong and it jumps out at me right away, so I catch most issues early. Either way the speed up is significant.

Re: Agent Skills

#123

Post by djhn »

tokioyoyo wrote: post #121
swyx wrote: post #81

Earlier quoted context omitted.

> have been able to successfully track the boost. lets get nitty gritty on this - can you say how you did this? because a lot of people think this is an unsolved problem

For my team, it has been easy. We deal with infrastructure for the entire org, so have tickets created for every request. We also gave our own backlog for internal project, so can see burn rate, and etc. Team hasn’t changed, a lot of similar/same tasks that have taken half a day has been completely automated to a point where we just do PR review after an initial ticket is created by other teams. There are a lot of li…

What kind of code is infrastructure in this context? Devops in a software company? Internal tooling in a software org?

Re: Agent Skills

#124

Post by rubendev »

tokioyoyo wrote: post #121
swyx wrote: post #81

Earlier quoted context omitted.

> have been able to successfully track the boost. lets get nitty gritty on this - can you say how you did this? because a lot of people think this is an unsolved problem

For my team, it has been easy. We deal with infrastructure for the entire org, so have tickets created for every request. We also gave our own backlog for internal project, so can see burn rate, and etc. Team hasn’t changed, a lot of similar/same tasks that have taken half a day has been completely automated to a point where we just do PR review after an initial ticket is created by other teams. There are a lot of li…

What is your definition of faster to implement? Is it producing a plausible implementation, or is it faster at producing a correct and high quality implementation? Are you including time spent refactoring and fixing bugs in your metrics? If not, I think you are tracking a gut feeling rather than cold hard facts. I’m not saying this is easy to track, just saying that it’s hard to know for sure that you are really more productive with AI.

Re: Agent Skills

#125

Post by djhn »

tokioyoyo wrote: post #75
ai_fry_ur_brain wrote: post #39

Cant wait for everyone to realize they've wasted a year + messing with agents and experiencing a feeling of psuedo productivity.

I’m a bit curious with these takes. Arguing in good faith - is the general assumption that people who use AI/agents/harnesses don’t ship features? We’ve been all in Claude Code since ~Septemberish, and have been able to successfully track the boost. Like the features that we ship that get used in production. Both from infrastructure side, and business logic implementations. Frontend and backend. I don’t think people…

I can take on a slightly weaker form in good faith: professionally it’s a non-starter until private, open source inference can be self-hosted and the ROI is clear enough to invest in that.

And on the ROI side, trying things out regularly, I haven’t seen the positive ROI in the limited time I’ve dedicated to exploring the tools. I’ve restricted experimenting to 4 hours per month, because spending more than 2.5% of the month chasing productivity improvements that realistically seem to be 10-20%, will quickly eat into those gains. After accounting for token costs, it ends up being a wash.

Re: Agent Skills

#126

Post by hansmayer »

What skills mate, this is simply text files attempting to narrow down the specs, hoping that this will help the "AI" make less mistakes. But it is still crap, because, - it still depends on how this fits into the overall statistical model which changes with every prompt, etc... Please stop peddling this bullshit, it does not work!

Re: Agent Skills

#127

Post by hansmayer »

vidarh wrote: post #93
wg0 wrote: post #68

Snake oil. Good to read for sure. Seems all plausible too. But snake oil nevertheless. Here's why: The slot machine can drop any hard requirement that you specifically in your AGENTS.md, memory.md or your dozens of skill markdowns. Pretty much guaranteed. These harnesses approaches pretend as if LLMs are strict and perfect rule followers and the only problem is not being able to specify enough rules clearly enough. T…

Humans also drop any hard requirements you specify regularly, and similarly require review. Nevertheless we manage to increase reliability of human output through processes and reviews, and most of the methods we use for harnesses are taken from experience with how to reduce reliability issues in humans, who are notoriously difficult to ensure delivers reliably.

[flagged]

Re: Agent Skills

#129

Post by [deleted] »

wg0 wrote: post #68

Snake oil. Good to read for sure. Seems all plausible too. But snake oil nevertheless. Here's why: The slot machine can drop any hard requirement that you specifically in your AGENTS.md, memory.md or your dozens of skill markdowns. Pretty much guaranteed. These harnesses approaches pretend as if LLMs are strict and perfect rule followers and the only problem is not being able to specify enough rules clearly enough. T…

[deleted]
Post reply on HN