[flagged]
Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
231–240 of 268 posts
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#232Earlier quoted context omitted.
I think minority do. Imagine, you have been vibe-coding this project for a while and it works kind of fine but you just have to fix a few more bugs and you get something like `node /tmp/claude-1000/-home-user-source-github-user-hn/27b740b1-9a45-47f3-ab99-61e5e3cf779a/scratchpad/hidden-smoke.mjs; echo "exit=$?"`. (I took it from my own agent right now and I don't have any idea what it's doing. Thankfully, it's sandbox…
> I think minority do Let's be real, the minority of people understand bash. It is a terrible language with tons of footguns[0], albeit a very useful language and still worth learning. The number of ~/.claude/settings.json I've seen with "permissions": { "allow": ["Bash(find *)"]} I've seen is crazy[1]. Hell, most people I talk to think `find` is a tool that is used for searching for files. I mean... it does that...…
over 30 years ago bash was magic for us oldies :-D
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#233This game, like just about every game, has zero consequences for failure. This is like saying "Humans were involved in fatal accidents 50% of the time when playing my custom F1 racing simulator". There were no stakes and there was an artificial time constraint. Deriving any sort of takeaway from this data is entirely useless.
Actually, your example there would be absolutely true. Getting in someone's enthusiast but mid-range F1 simulator toy with all the game assists turned off would both: imply a near fatal accident happening over 50% of the time AND it would be accurate too.
Consider:
FIA President Mohammed Ben Sulayem, a former Rally driver at the top level, crashed an F1 car within 100m of trying to go fast in it
And
Mr Beast, a youtuber with zero motorsports experience, crashed a Formula E car on a demonstration lap as part of the pre race F1 festivities.
If a regular person with a drivers license and no familiarity attempted to play even a simulator video game, the results are in fact similar to what happens in the real world.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#234[flagged]
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#235Because things like this, I am using sandboxes, doing automated and manual review before running any code. As it takes time, I now need to try to one shot the development of the code which takes longer to do a proper specification. And even with much care on all the steps, when I read carefully, I still find wrong things at multiple levels. When folks like uncle Bob says they don't review AI code, I can only think th…
> When folks like uncle Bob says they don't review AI code, I can only think they are burnt out or being unprofessional. Indeed it saves some time writing the code, but overall, I think we just moved concerns from one place to another.
I think you might be getting yourself drunk with plain water here ... reviewing a code is just vibes, "LGTM" type of vibes from a human instead of AI, but not better than vibes. Yes it might catch some implications or bugs if we are lucky, but it is not a reliable way to verify code.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#236My team is working on Watcher to deal with exactly this. We know that Claude is occasionally going to do stuff we really don't want, but at a rate that's way too low for manual approvals to make sense, so we hook into Claude Code (or Codex) to approve commands in a way that's a lot closer to `--dangerously-skip-permissions` but without the danger. We use a hierarchy of deterministic rules and heavily-tested LLM monitors to balance speed, cost, and accuracy.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#237I remember when this game was posted here, and there was a lot of discussion at the time that some of the prompts were misleading about whether or not they were risky, some people were debating about how some of the prompts flagged as bad weren’t bad, and others flagged as not bad were. This is a fundamental flaw in the test, that makes the analysis of results meaningless. Also the game was on a timer, and maybe ther…
Wanna chat with my friend, who coded a side-bot for his OpenClaw installation just to auto-approve prompts?
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#238Earlier quoted context omitted.
Maybe that is why so many things live in Excel instead of a proper database. Security at the expense of convenience does come at the expense of security, after all. Not sure why passwords still exist conceptually. I was hoping we'd move past this annoyance, but instead security has become even more annoying. And all that security with two factor hoops to jump through only for someone to steal your session cookie.
What would passwords move past too? A device people lose? Some specifications everyone argues about and implements differently. Just look at SAML and everyone using slightly different terminology. Security is more annoying because the attacks are better than ever.
Certificates seemed promising there for a while. I also don't mind hardware keys/cards, but when you have to first enter your password, then your pin for your hardware key and then touch the hardware key it gets ridiculous again.
Not a fan of biometrics, though, as that cannot be changed trivially.
Ideally, authentication should happen like entering a building. You do that _once_ at the beginning of a day by swiping your card or unlocking a door and then it is done and you are never asked to do it again for the rest of the day.
I don't have to enter a password and a pin for my key to open the door to either my office or my home. Sure, it might be trivial for an attacker to enter the building using different means, but that hasn't prompted us to play security theater around every building entrance.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#239Earlier quoted context omitted.
I think minority do. Imagine, you have been vibe-coding this project for a while and it works kind of fine but you just have to fix a few more bugs and you get something like `node /tmp/claude-1000/-home-user-source-github-user-hn/27b740b1-9a45-47f3-ab99-61e5e3cf779a/scratchpad/hidden-smoke.mjs; echo "exit=$?"`. (I took it from my own agent right now and I don't have any idea what it's doing. Thankfully, it's sandbox…
> I think minority do Let's be real, the minority of people understand bash. It is a terrible language with tons of footguns[0], albeit a very useful language and still worth learning. The number of ~/.claude/settings.json I've seen with "permissions": { "allow": ["Bash(find *)"]} I've seen is crazy[1]. Hell, most people I talk to think `find` is a tool that is used for searching for files. I mean... it does that...…
> It is a terrible language
It's not. It only becomes terrible when you misuse for it's original purpose.
Microsoft Excel isn't terrible, but the way it gets used makes it terrible.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#240Earlier quoted context omitted.
MySQL installed with skip-networking was very common, but as soon as you need to access it over the network (e.g. when you move from one box with MySQL and PHP to one for each) then you have to enable networking and everything falls apart. Also, the number of times in my career that I've googled a problem and seen some forum post saying "Oh, just run chmod -R 777 /var/www/wordpress/uploads/ and it'll fix that" "Great…
It's surprisingly hard to convince supposedly trained tech professionals that `chmod -R 777 /path/to/whadever` is not a good habit.
Or curl https://hopefullyatleasthttps.com | sudo bash