Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agent Skills

addyosmani.com

Post reply on HN
Print view View original

171–180 of 239 posts

Re: Agent Skills

#171

Post by esperent »

_pdp_ wrote: post #153
esperent wrote: post #146

Earlier quoted context omitted.

I just tried it with Gemini pro. I think this answer is about as good as you can expect for such an ambiguous question. Write a fib Since "fib" can mean a couple of different things, I've got you covered for both! 1. A Little Lie (A Fib) "I'm actually typing this to you from a sunny beach in the Bahamas, sipping a piña colada." (Since I'm an AI, that is definitely a fib!) 2. The Fibonacci Sequence If you meant the cl…

I stand to be corrected. Though I tried again just now and this is what Gemini Pro produced: > I'm assuming you mean a Fibonacci sequence generator! I'll write a Python script that includes both an iterative and a recursive way to generate Fibonacci numbers. ... and then wrote some python code.

In my opinion, if a tool that's designed to be an answer machine had to give exactly one response to "write a fib", the correct choice is the Fibonacci sequence. You're probably underestimating how many programming students might type a query like that in.

If you want a lie, then the normal grammar in English is to say "tell a fib". I bet every llm you test that on will respond by telling you a small lie, or at least note the ambiguity and then say it's going to revert to Fibonacci because that's more in line with what it's designed to do.

Re: Agent Skills

#172

Post by vidarh »

adyavanapalli wrote: post #168
vidarh wrote: post #101

Earlier quoted context omitted.

Depends on the clients maturity, but some places all of the above.

What are the numbers you are getting?

Initial drop, as people learn to use the tools, and while they keep babysitting their harnesses. Then significant boost once people start getting used to running the agents in the background, especially once they start running multiple sessions in parallel. I'd say you need a ~6 month push of getting people trained if they are not used to this way of working, and to customised setups etc. for your organisation, and then you start seeing significant payoff.

Re: Agent Skills

#173

Post by vidarh »

jnpnj wrote: post #150
kaashif wrote: post #106

Earlier quoted context omitted.

The primary way to increase reliability is to automate. Instead of humans producing some output manually, humans producing machines which produce that output. I've seen a disturbing trend where a process that could've been a script or a requirement that could've been enforced deterministically is in fact "automated" through a set of instructions for an LLM.

it's strange to see software engineers using skills aka human description of small scripts instead of scripting things directly. often there were cli / tools / libraries to do what a skill does for many years. maybe it's culture issue, people who enjoy automation / devops / predictability will naturally help themselves, but other people just want to "delegate" and be done without trying.

When people do that they are using skills wrong. The best way to use a skill is as a means to give targeted instructions on how to make use of cli / tools/ libraries, with the skill just covering the "squishy bits" that aren't easily encoded into something deterministic.

Re: Agent Skills

#174

Post by intended »

vidarh wrote: post #149
intended wrote: post #144

Earlier quoted context omitted.

We resolve that through liability, penalties, trust, responsibility, review and oversight. At the end of the day, if I am spending X$s for automation, I want to be able to sleep at night knowing my factory will not build a WMD or delete itself. If its simply a tool that is a multiplier for experts, then do I really need it? How much does it actually make my processes more efficient, faster, or more capable of earning…

Liability, penalties, trust, and responsibility are means we use to try to influence the application of the processes that do. They do not directly affect reliability. They can be applied just as much to a team using AI as one that does not. Review and oversight does address reliability directly, and hence why we make use of those in processes to improve the reliability of mechanical processes as well, and why they a…

> Liability, penalties, trust, and responsibility are means we use to try to influence the application of the processes that do. They do not directly affect reliability. They can be applied just as much to a team using AI as one that does not.

Yes and no. see next point.

> You can ask the same thing about all the supporting staff around the experts in your team.

I have a good idea of the shape of errors for a human based process, costing and the type of QA/QC team that has to be formed for it.

We have decades, if not centuries of experience working with humans, which LLMs are promising to be the equivalents/superiors of.

I think you and me, would both agree with the statement "use the right tool for the job".

However, the current hype cycle has created expectations of reliability from LLMs that drive 'Automated Intelligence' styled workflows.

On the other hand:

> part of initiatives to increase reliability of human teams

is a significantly more defensible uses of LLMs.

For me, most deployments die on the altar of error rates. The only people who are using them to any effect are people who have an answer to "what happens when it blows up" and "what is the cost if something goes wrong".

(there is no singular thread behind my comment. I think we probably have more in agreement than not, and its more a question of finding the precise words to declare the shapes we perceive.)

Re: Agent Skills

#175

Post by SubiculumCode »

wg0 wrote: post #68

Snake oil. Good to read for sure. Seems all plausible too. But snake oil nevertheless. Here's why: The slot machine can drop any hard requirement that you specifically in your AGENTS.md, memory.md or your dozens of skill markdowns. Pretty much guaranteed. These harnesses approaches pretend as if LLMs are strict and perfect rule followers and the only problem is not being able to specify enough rules clearly enough. T…

All these points apply to human devs as well. The test is not infallibility but magnitude

Re: Agent Skills

#176

Post by [deleted] »

wg0 wrote: post #82
cortesoft wrote: post #78

Earlier quoted context omitted.

Everything you say is all possible, and in theory I agree with you. However, I have been using spec-kit (which is basically this style of AI usage) for the last few months and it has been AMAZING in practice. I am building really great things and have not run into any of the issues you are talking about as hypotheticals. Could they eventually happen? Sure, maybe. I am still cautious. But at some point once you have p…

We can build all the scaffolding around but I assure you that the LLMs aren't perfect rule following machines is the fundamental problem here and that would remain. Give it a few more months and I'm sure you'll see some of what I see if not all. I'm saying all the above having all sorts of systems tried and tested with AI leading me to say what I said.

[deleted]

Re: Agent Skills

#177

Post by j45 »

kergonath wrote: post #162
j45 wrote: post #138

Earlier quoted context omitted.

Pretty high chance isn’t what the intent or impression the end user often has.

Indeed, and it is a complicated problem to solve. A GUI or CLI can hide footguns or make them less likely to be misused. But an AI agent is perfectly happy to use a wrecking ball to put a nail without any second thought or confirmation.

It’s a human articulation problem.

When it receives a generic vague input it is free to interpret according to how its corpus fires like any human interaction.

How to articulate better is like writing a sentence that will stand the test of model updates.

Re: Agent Skills

#178

Post by m3kw9 »

I'm surprised these "elite" engineers are still talking about Claude, most engineers that really use this stuff have already switched to Codex.

Re: Agent Skills

#180

Post by gavmor »

fragmede wrote: post #22
gavmor wrote: post #13

Naming things is such a hard problem that many devs don't even bother trying. That being said, this post is full of reasonable assertions, so I'm looking forward to experimenting with this... whatever it is.

Wait, shit, are people using LLMs to name things now? I'm definitely out of a job then!

I'm notorious for taking poetic license with naming—that's how we end up with `class Escutcheon`, or variables `recto` and `verso` where applicable in eg PDF generation.

But as much pleasure as I derive from novelty and specificity, my colleagues have oft expressed perplexity—whereas the terms which LLMs produce hew closer to the manifold (by definition!) and raise fewer eyebrows.

So, it has its turn.

Post reply on HN