Adding verification steps to AI agents made them worse. I tested it 29 times
1–7 of 7 posts
Re: Adding verification steps to AI agents made them worse. I tested it 29 times
#2[dead]
Re: Adding verification steps to AI agents made them worse. I tested it 29 times
#3[dead]
I suspected that given some responses the agent was giving during the experimentation. Looked sometimes tendentious when measuring.
Re: Adding verification steps to AI agents made them worse. I tested it 29 times
#4Re: Adding verification steps to AI agents made them worse. I tested it 29 times
#5Earlier quoted context omitted.
I suspected that given some responses the agent was giving during the experimentation. Looked sometimes tendentious when measuring.
[dead]
I tried to also avoid that, and the main orchestrating agent was running subagents running experiments, and others were checking the results. Somewhat it worked as it was not imposing only successes, there were more failures.
Re: Adding verification steps to AI agents made them worse. I tested it 29 times
#6> What works... External mechanical checks
This is what I expected verification to mean going in.
For some reason verification in this article means "try to convince the AI even harder than before"
Re: Adding verification steps to AI agents made them worse. I tested it 29 times
#7> What works... External mechanical checks This is what I expected verification to mean going in. For some reason verification in this article means "try to convince the AI even harder than before"
I first tried to make the agent follow concrete paths, but no tricks worked. The reason I wanted to achieve it this way is that I wanted to create universal "rules" that would be followed without additional CLI or hooks. It didn't work out as expected. I needed it for my project, and currently I started to enforce more things with CLI, which is directly accessible by an agent, so it's now "guided" more forcefully.