Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

github.com

Post reply on HN
Print view View original

91–100 of 139 posts

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#91

Post by hsaliak »

Syntaf wrote: post #75
hsaliak wrote: post #43

Seems to be doing too much, a 1 line in the system prompt is all you need. And it works well enough. “Output tokens are precious, be succinct in your responses. Use ASD-STE100 simplified technical english”

Does this actually work for you? I've tried adding rules in my user profile as well as adding instruction in my CLAUDE.md but my models always eventually go off the rails and start spewing technical jargon in doc strings & descriptions. At this point I'd do anything to get claude to dumb down and simplify it's explanations in my code, so I'm looking forward to giving this a try and seeing how it goes.

yes it does - https://github.com/hsaliak/std_slop/blob/main/system_prompt....

I've added the same thing to the system prompt of the internal agent I wrote for work, which talks to claude models. I have not gone into the details on whether it adheres to the format exactly, but i do see visibly less slop and less marketing nonsense in the Assistant messages.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#93

Post by Groxx »

what wrote: post #80
baby_souffle wrote: post #62

Earlier quoted context omitted.

> - Ask it to update the skill to warn it against making that mistake again Yep! I have a canned prompt that basically boils down to "scroll up, read through and give me the top `n` things that were difficult..." and more often than not a small skill or change to `agents.md` comes out of that.

> "scroll up, read through and give me the top `n` things that were difficult..." Why do you think it can answer that? It has zero clue.

Technically speaking: it can't, any more than it can actually answer any other thing you ask it to do.

But in practice it does produce fairly reasonable output fairly often, and if you've been there watching and correcting it you can probably validate the result quite easily. "Re-read everything before" and similar are definitely anthropomorphizing, but that doesn't mean they're ineffective.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#94

Post by aiagenta2z »

I have a question about how did the project evaluate the technical English performance compared to other skills/MCPs and with LLM w/o skills? In the Github repo, there is a summary of "measured: 6 Claude models × 8 tasks × 2 conditions, 96 runs", Does it means the percentage of violations in the words? If that's the case, the measurement might be a little bit strange weather the STE measure is already in the prompt? STE violations per 100 words ▼ 72.9% (every model won). The bench or evaluation should not be in the same prompt, like eval/test.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#96

Post by CuriouslyC »

setrf wrote: post #71
dan_sbl wrote: post #25

I took one of the examples ("Leveraging sqlpipe's robust...") and just prefixed it with this: > Rewrite this using ASD-STE100 simplified technical English: And you get a good-enough result, it seems? Maybe add another sentence or two for guidance, but what's with needing these giant skills, when ASD-STE100 was surely in the training material?

I certainly get what makes you feel that the model should have been wise enough to utilize the knowledge in its pretraining, but it seems like the pretraining data gets pretty muddled in later stages.

The pretraining data isn't getting muddled, the instruction just gets in the lossy middle, and there's less for attention to grab on to. If you repeated the prompt with a hook every turn, it'd stay quite sharp.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#97

Post by NothingAboutAny »

stuaxo wrote: post #73

I has quite a lot of luck making docs nicer by asking to "write using the dry/succinct style of ukgov/gds". If you want to take things further you can use linters to enforce the whatever style you want, but just naming it is a good start.

I tried "be concise and respond in a blunt tone" and all that did was add a section to the bottom of every response "here's the blunt concise truth "

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#99

Post by cheema33 »

bcrosby95 wrote: post #49
mcintyre1994 wrote: post #24

Earlier quoted context omitted.

They acknowledge this, but I really dislike whatever style they're using for the readme: > Yes, this README breaks half of them. Marketing is explicitly out of STE scope. The skill knows that and stays in the docs. > Where it refuses to go: marketing copy, blog voice, brand writing. Flat on purpose. (Each sentence there has an emoji that HN cuts out)

The problem with at least ChatGPT is no matter what personality you set it still reads like a smug asshole trying to sound smart. You can just pick different flavors of smug asshole.

> The problem with at least ChatGPT is no matter what personality you set it still reads like a smug asshole trying to sound smart.

Some of this may be perception rather than reality. I have never felt that ChatGPT was being smug or rude to me.

Re: Agent Skill to Force Docs in ASD-STE100 Simplified Technical English

#100

Post by pertymcpert »

what wrote: post #80
baby_souffle wrote: post #62

Earlier quoted context omitted.

> - Ask it to update the skill to warn it against making that mistake again Yep! I have a canned prompt that basically boils down to "scroll up, read through and give me the top `n` things that were difficult..." and more often than not a small skill or change to `agents.md` comes out of that.

> "scroll up, read through and give me the top `n` things that were difficult..." Why do you think it can answer that? It has zero clue.

Why wouldn’t it?
Post reply on HN