Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

github.com

Post reply on HN
Print view View original

101–110 of 139 posts

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#101

Post by aesthesia »

summarity wrote: post #83
aesthesia wrote: post #56

Earlier quoted context omitted.

How is that ambiguous? The best interpretation I can find where "test" is a verb is an elision: Test [that] B is an alternative to test A. That is an unlikely reading: "test" is a verb in the first instance and a noun in the second, breaking parallelism. And it's just a very unlikely thing to direct someone to do.

"Test B is an alternative to test A." could mean there's a Test B and a Test A, and they're interchangeable. Or: You can run Test B to confirm A works, that is "to test A". Again the stated goal to clear documentation for non-native or limited-exposure speakers. This doesn't pass that test.

Ah, I see, taking "to test A" as an infinitive. But it would be strange to say test B is an alternative without saying what it's an alternative to. The other reading still seems quite unlikely.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#102

Post by atoav »

The examples are not good. In one example (incident report) the agent turns a generic "something went wrong" into overly precise error descriptions that may or may not spill internal details. Then it proceeds to make a promise to implement a persistent monitoring for that problem this week.

I mean generally this wouldn't be considered a bad response, but it is deeply problematic if an LLM tasked to just simplify language added both internals and promises to your customers without your knowledge.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#103

Post by solarkraft »

skissane wrote: post #55
rspeele wrote: post #47

Earlier quoted context omitted.

I have yet to see an "agent skill" that does not match this description.

How I use skills: - Create a skill to perform some common task - Watch the agent use it - Wait until the agent makes a mistake - Ask it to update the skill to warn it against making that mistake again I think that, using that iterative process, I end up with something better than just asking it to perform the task plainly One observation: ask an LLM to write a skill, it tends to make them overly verbose and prescript…

> One observation: ask an LLM to write a skill, it tends to make them overly verbose and prescriptive. Often, something briefer and human-written actually works better

Incredibly so. All too often a “hey, you shouldn’t have done that here” turns into “NEVER X”, without it actually understanding or making an effort to understand why the correction was made, though I would say usually the context to do so should suffice. I have a rule for it to reason this through and use nuanced language and would say the frontier GPTs (5.4 to 5.6 Sol) get it right around half of the time (very rough estimate). The conclusions are really dumb sometimes.

I’d be interested in an expertly crafted skill here, maybe there are things about it you can still improve. This is an area in which I still find LLMs to be quite lacking.

It also made me appreciate the complexity of the nuances and levels of indirection of what I want to teach it: “Upload to the remote host, which in this case is this but might be another one in another use case, only do this on the local system and this on the remote, do this to X unless Y …, prefer doing Z unless another rule overrides it …”

I sometimes feel like there could be a more expressive way to structure to structure these rules (bring the snark!).

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#104

Post by sharpy »

skissane wrote: post #55
rspeele wrote: post #47

Earlier quoted context omitted.

I have yet to see an "agent skill" that does not match this description.

How I use skills: - Create a skill to perform some common task - Watch the agent use it - Wait until the agent makes a mistake - Ask it to update the skill to warn it against making that mistake again I think that, using that iterative process, I end up with something better than just asking it to perform the task plainly One observation: ask an LLM to write a skill, it tends to make them overly verbose and prescript…

Yep. That is what I do too. But a lot of times the changes it makes are bandaids rather than addressing root cause. I have to periodically evaluate the skills for consistency and opportunities for simplification.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#105

Post by kvirani »

skissane wrote: post #55
rspeele wrote: post #47

Earlier quoted context omitted.

I have yet to see an "agent skill" that does not match this description.

How I use skills: - Create a skill to perform some common task - Watch the agent use it - Wait until the agent makes a mistake - Ask it to update the skill to warn it against making that mistake again I think that, using that iterative process, I end up with something better than just asking it to perform the task plainly One observation: ask an LLM to write a skill, it tends to make them overly verbose and prescript…

Super valuable and actionable advice. :100:

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#106

Post by baq »

abeyer wrote: post #39

STE Misapplication issues and limited adoption https://en.wikipedia.org/wiki/Simplified_Technical_English#M...

LLMs should be unreasonably good at exactly this kind of writing because they’re good translators; regardless in practice it works well to deslopify their writing, been using this in all my prompting in the last week or so, haven’t seen a single load bearing belt and suspenders since.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#108

Post by garkotipankaj »

Agent skills work... sometimes - depends on the agent's mood really.

I tried to solve this in a rather opposite way.

https://github.com/computerreinvention/trie

The documentation is updated and recording the intent behind all changes is enforced by a commit-gate. At the symbol level.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#109

Post by dymk »

rspeele wrote: post #47
Planktonne wrote: post #34

This is cruft [1]. No one who is capable of using this needs it--it's a line in the prompt at most. [1] https://knowyourmeme.com/memes/thinking-quickly-dave-constru...

I have yet to see an "agent skill" that does not match this description.

You can memorize a book too but that doesn’t mean they’re not useful

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#110

Post by dymk »

aurareturn wrote: post #107
jpease wrote: post #29

I like the idea, but not sold on the skill. But did lead to me finding https://vale.sh and its various available linters. Might give that a try.

How would one use Vale in LLM documentation?

Precommit hook
Post reply on HN