It's kinda funny there is still software coming out whose security model is "constantly ask the user for permission, and hope they never make a mistake". It's been tried so many times before, and it never worked.
Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
151–160 of 268 posts
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#152Earlier quoted context omitted.
I joined Oracle in 2012, and I myself complained that the user experience was horrible: get an Oracle DB installed was a nightmare, starting with so many questions. I heard complaints from other users about it requiring, during installation, that an admin password must be set. I myself had preferred to use MySQL because it was so simple and easy to get started and using it. Until I learned how many MySQL databases we…
Maybe it's distro specific but I'm fairly sure mysql by default installed with skip networking around 2012, and with bind-address set to localhost since then. Also the root user is only configured from local by default but I'm not sure if that was true 14 years ago. If the defaults are more secure than your examples, it's not fair to blame the database or the defaults. And personally I hate it when software forces se…
Gitlab's default requirements aren't that intense, but you can make them more stringent if you want.
https://docs.gitlab.com/user/profile/user_passwords/#passwor...
I guess your goal was to allow users to have 4 character passwords, i.e. "love." which afaik you can't do.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#153Earlier quoted context omitted.
if firefly has no nulls, how do you indicate that a value is unset?
In Firefly (as in Rust) you can define fields as Optional, so you can do Option[String]; that lets you say "this variable is a String but it might not be here". That lets you then check to see if something is set, rather than checking to see if it's null. In Rust an Option is a separate thing that you need to disambiguate to use. For example: match result { // The division was valid Some(x) => println!("Result: {x}")…
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#154I remember when this game was posted here, and there was a lot of discussion at the time that some of the prompts were misleading about whether or not they were risky, some people were debating about how some of the prompts flagged as bad weren’t bad, and others flagged as not bad were. This is a fundamental flaw in the test, that makes the analysis of results meaningless. Also the game was on a timer, and maybe ther…
In my most recent job I was definitely under that sort of timer.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#155You sandbox, you have good checkpoints, and good agents, that's it. If you are manually reviewing commands you are wasting your time.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#156Earlier quoted context omitted.
A language that revives capabilities, brings them up-to-date, and works in the modern environment is my #1 request from the programming language community right now. I don't need another language with sum types and higher-order functions and a functional focus. I need a language with capabilities. That language may have the other goodies as well, sure, no problem, but we all need capabilities. I've done some stabby s…
What is a capability in terms of programming language design? It sounds more like the sort of thing that would belong at the standard library level, where builtin APIs are guarded by flags. Deno has something vaguely built in with permissions flags, and old school Blackberry (at least in the J2ME days) had permissions settings for almost everything that an app could do, but again, those are all external to the langua…
I wrote a bunch more, but I have a habit to verbosity when capabilities come in which I should attempt to combat, and have deleted it. Crockford says these things better than I can anyhow.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#157Earlier quoted context omitted.
Air Traffic Control is still primarily voice based, and simply up to the user on both sides to not make a mistake. Just bringing it up because you're right, in software that's considered a bad pattern (rightfully so).
> Air Traffic Control is still primarily voice based, and simply up to the user on both sides to not make a mistake The "user[s] on both sides" of ATC conversions have passed through the filters of rigorous training and certification. They also happen to communicate in a DSL designed to minimize misunderstandings, the DSL just happens to be based on English.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#158Earlier quoted context omitted.
I think you're confused. The verb form of the word never carried the credentialism of the title. In the same way that "doctoring" never carried the connotation of a medical degree. Of course the original sense of the noun was "a person who devises things" and shares a root with "ingenious" and carried no connotation of legal credential. That "harm" is more or less restorative to the original meaning of the word.
I'm not referring to the verb form, I'm referring to the people who call themselves things like "prompt engineer" or "software engineer" with a straight face, draping themselves in a false legitimacy stolen from professionals for whom the term "engineer" actually implied something of note. It's embarrassing, or it would be if people were still possessed of the capacity for shame.
The original post prompting your complaint used the verb form. And hence why you're confused.
> It's embarrassing, or it would be if people were still possessed of the capacity for shame.
It's far more shameful to fly off the cuff with a non-sequitur by your own admission, and double down on yelling at the clouds when called out for it. And about an arbitrary point in a semantic treadmill, no less. Your take away is that it was never something of note for these people, and trying to pick semantic fights with them over it just makes crying about legitimacy ironically turn into pure pretense.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#159Earlier quoted context omitted.
Yet the AI can still escape the "sandbox", unless it is physically unable to connect to another computer and completely airgapped.
If the sandbox has vulnerabilities, which you can also use the AI to fuzz for. Obviously at the point in which it can talk to the internet it doesn't really matter, but there are a very finite number of zero-days that can exist in a bytecode interpreter hosting a harness.
My point is if an agent recited how to find one in its memory or training set and it is air-gapped, the chances of it spreading and infecting other computers is pretty low.
Re: Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
#160I remember when this game was posted here, and there was a lot of discussion at the time that some of the prompts were misleading about whether or not they were risky, some people were debating about how some of the prompts flagged as bad weren’t bad, and others flagged as not bad were. This is a fundamental flaw in the test, that makes the analysis of results meaningless. Also the game was on a timer, and maybe ther…
I think minority do. Imagine, you have been vibe-coding this project for a while and it works kind of fine but you just have to fix a few more bugs and you get something like `node /tmp/claude-1000/-home-user-source-github-user-hn/27b740b1-9a45-47f3-ab99-61e5e3cf779a/scratchpad/hidden-smoke.mjs; echo "exit=$?"`. (I took it from my own agent right now and I don't have any idea what it's doing. Thankfully, it's sandbox…