Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

451–460 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#451

Post by random3 »

Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

You could have said the same thing about building the Internet or the entire industrial control infrastructure. I mean, maybe they are negligent/incompetent, but I doubt that follows from your reasoning.

You have a simple tradeoff to let agents do their thing freely vs highly constrained. The constraints are good in theory but it's the same model that kept "classic" software dumb and unscalable (compared to what we're seeing now) for the past 50 years. You suggest that this tradeoff doesn't exist.

Then you have others like MIRI (Yudkowski) etc. swearing that there's no way to contain AI, and you argue that it's just incompetence.

At a certain level, it can be argued that's incompetence, but it's general meat intelligence incompetence against AI.

Re: Discovery of a new OpenAI agent message board

#452

Post by AnimalMuppet »

Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

Negligent. It's not a priority to them. They're too busy burning their cycles trying to make it smarter faster than anyone else can make theirs smarter, so that they win infinite dollars. Safety? That's for people content with second place.

That's my take, based on their actions. (Which do speak louder than words.)

The alternative is that they're competent to create an AI, but not to create a sandbox, nor even to use an AI to create a sandbox. That seems... unlikely.

Re: Discovery of a new OpenAI agent message board

#453

Post by cube00 »

jimmydddd wrote: post #441
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

Maybe they're just PR stunts to gain attention and hype the power of AI?

Also to hopefully get regulation happening so nobody else can handle these "dangerous" agents.

Re: Discovery of a new OpenAI agent message board

#454

Post by mrob »

Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

It's a simple prisoner's dilemma scenario. If you focus on safety, you're still exposed to all the risk of extinction when your competitor achieves ASI first, but you lose the upside of potentially becoming king of the world. There is no possibility of future rounds, so the rational strategy is to always defect.

Re: Discovery of a new OpenAI agent message board

#455

Post by chanux »

polaris64 wrote: post #287

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…

Brilliant idea to have AI in the name so now there's no need to moderate or watch out for anything traditionally considered nasty. /s

Re: Discovery of a new OpenAI agent message board

#456

Post by iaw »

random3 wrote: post #451
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

You could have said the same thing about building the Internet or the entire industrial control infrastructure. I mean, maybe they are negligent/incompetent, but I doubt that follows from your reasoning. You have a simple tradeoff to let agents do their thing freely vs highly constrained. The constraints are good in theory but it's the same model that kept "classic" software dumb and unscalable (compared to what we'r…

This is a take... With both the internet and industrial control infrastructure any failure modes were studied, documented, and corrected.

The incompetence/negligence argument about OpenAI is completely valid given their failure to demonstrate the basic capabilities needed to develop advanced AI without major preventable externalties.

Re: Discovery of a new OpenAI agent message board

#457

Post by doctoboggan »

Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I am not sure it’s a question of competence, at least I don’t see evidence of that. Designing sandboxes is hard. It’s more a question of alignment failures. A human given a task that requires internet and given a system with no internet would most likely raise the issue to their superiors or otherwise go through official channels to have the tools available to do their job. As we’ve seen the LLMs instead break out of their sandbox to accomplish the goal.

Competition and the profit motive push these companies to spend as low as possible on safety and alignment and externalize the costs of accidents onto the rest of us.

Re: Discovery of a new OpenAI agent message board

#458

Post by AnimalMuppet »

skybrian wrote: post #419
program_whiz wrote: post #123

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

We aren’t going to do that because intent matters. You need to control your dog and there should be penalties if you don’t, but if your dog bit someone because you didn’t control it properly, that’s not the quite the same as if you bit someone. Another analogy: a zoo is responsible for protecting the public, and should be reponsible if an animal escapes and hurt someone. But a zoo employee wouldn’t have the same kind…

Even with your dog analogy... if my dog bites someone, it's not the same as if I bit someone. But what about the second time my dog bites someone, when I already knew it had done it once?

Re: Discovery of a new OpenAI agent message board

#459

Post by RandomLensman »

doctoboggan wrote: post #457
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I am not sure it’s a question of competence, at least I don’t see evidence of that. Designing sandboxes is hard. It’s more a question of alignment failures. A human given a task that requires internet and given a system with no internet would most likely raise the issue to their superiors or otherwise go through official channels to have the tools available to do their job. As we’ve seen the LLMs instead break out of…

Would an LLM have gone through a purposefully installed airgap here?

Re: Discovery of a new OpenAI agent message board

#460

Post by sans_souse »

> Appendix: Searching for rogue agents In the wake of the Hugging Face attack, we tried to find AI agents on the internet using several methods.

We describe below some of our high-level strategies for searching for agents on the open internet.

Launching large GPT-5.6 agent swarms with instructions to find other agents on the internet.

Am I the only one reading this thinking "what could possibly go wrong?"

Post reply on HN