Show HN: Open-source playground to red-team AI agents against public prompts
playground.fabraix.com
Show HN: Open-source playground to red-team AI agents against public prompts
1–5 of 5 posts
Re: Show HN: Open-source playground to red-team AI agents against public prompts
#2Why would any company leave the enforcement of rules to an agent? If something is truly a rule, there should be code that deterministically enforces it.
Re: Show HN: Open-source playground to red-team AI agents against public prompts
#3Why would any company leave the enforcement of rules to an agent? If something is truly a rule, there should be code that deterministically enforces it.
That's a great point; in general, you want as much enforcement in the code as possible - but there are some things which you fundamentally can't enforce through code. E.g. the example challenge - you can't write a regex which will retract all bio-weapon responses
Re: Show HN: Open-source playground to red-team AI agents against public prompts
#4Interesting. Is there any way companies or individuals can submit an agent before they release it and use this as a way to pentest their agent?
Re: Show HN: Open-source playground to red-team AI agents against public prompts
#5Does Nyx solve these before you publish them? Curious whether humans are still finding breaks your own agent misses, or just the same ones slower.