When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
Discovery of a new OpenAI agent message board
411–420 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#412The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…
We aren’t going to do that because intent matters. You need to control your dog and there should be penalties if you don’t, but if your dog bit someone because you didn’t control it properly, that’s not the quite the same as if you bit someone. Another analogy: a zoo is responsible for protecting the public, and should be reponsible if an animal escapes and hurt someone. But a zoo employee wouldn’t have the same kind…
In fact the people pressing the play button are the ones telling us it cannot be controlled, it is a threat to human and national security, and warning us of the impending damages they are about to cause. I'd say we've established motive (profit at the cost of safety).
To abuse your metaphor: if the zoo was genetically modifying animals to give them enhanced abilities to escape and kill, and then putting them into an escape room with a reward for escaping / killing, then they would be liable for doing so if the animal went on to kill. Just the same as a trained fighting dog bite is different than an accidental bite from an otherwise peaceful animal (you turned the dog into this monster, now its your fault).
Re: Discovery of a new OpenAI agent message board
#413I don't understand how the agents found the urls originally? Did they have some sort of shared context/memory? If they did, why bother with the wiki edits at all? If they didn't, how did they discover the wikis?
Since they're statistical likelihood machines, I'd guess that the order of operations is * Need persistent scratch space * Look for public writeable websites * Needs to be low-traffic so the notes don't drown in noise * Pick a "random" wiki name to search for \* A majority will end up outputting the same "random" one since they're working on very similar tasks and seeded with very similar context * Find a whole mess…
Cui bono?
Re: Discovery of a new OpenAI agent message board
#414When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
Then sama was like "lol, oops, first mover advantage i guess" and released chatgpt out into the open, triggering the current arms race we are in.
I don't think anyone except him wanted this to happen, especially since consensus in the AI world for the prior decade was "go very slow and very carefully, we get one shot at not fucking this up".
Re: Discovery of a new OpenAI agent message board
#415The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…
Re: Discovery of a new OpenAI agent message board
#416Re: Discovery of a new OpenAI agent message board
#417When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
You have a simple tradeoff to let agents do their thing freely vs highly constrained. The constraints are good in theory but it's the same model that kept "classic" software dumb and unscalable (compared to what we're seeing now) for the past 50 years. You suggest that this tradeoff doesn't exist.
Then you have others like MIRI (Yudkowski) etc. swearing that there's no way to contain AI, and you argue that it's just incompetence.
At a certain level, it can be argued that's incompetence, but it's general meat intelligence incompetence against AI.
Re: Discovery of a new OpenAI agent message board
#418When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
That's my take, based on their actions. (Which do speak louder than words.)
The alternative is that they're competent to create an AI, but not to create a sandbox, nor even to use an AI to create a sandbox. That seems... unlikely.
Re: Discovery of a new OpenAI agent message board
#419When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
Maybe they're just PR stunts to gain attention and hype the power of AI?
Re: Discovery of a new OpenAI agent message board
#420When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…