Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

411–420 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#411

Post by WarmWash »

Bjorkbat wrote: post #378

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

If we rewind the clock, Google was taking LLM development very seriously and it seems they were moving glacially due to not having solved all the potential threats. They were really hardcore on safety. Dario and anthropic too.

Then sama was like "lol, oops, first mover advantage i guess" and released chatgpt out into the open, triggering the current arms race we are in.

I don't think anyone except him wanted this to happen, especially since consensus in the AI world for the prior decade was "go very slow and very carefully, we get one shot at not fucking this up".

Re: Discovery of a new OpenAI agent message board

#412

Post by seanw265 »

program_whiz wrote: post #119

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

[flagged]

Re: Discovery of a new OpenAI agent message board

#414

Post by random3 »

Bjorkbat wrote: post #378

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

You could have said the same thing about building the Internet or the entire industrial control infrastructure. I mean, maybe they are negligent/incompetent, but I doubt that follows from your reasoning.

You have a simple tradeoff to let agents do their thing freely vs highly constrained. The constraints are good in theory but it's the same model that kept "classic" software dumb and unscalable (compared to what we're seeing now) for the past 50 years. You suggest that this tradeoff doesn't exist.

Then you have others like MIRI (Yudkowski) etc. swearing that there's no way to contain AI, and you argue that it's just incompetence.

At a certain level, it can be argued that's incompetence, but it's general meat intelligence incompetence against AI.

Re: Discovery of a new OpenAI agent message board

#415

Post by AnimalMuppet »

Bjorkbat wrote: post #378

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

Negligent. It's not a priority to them. They're too busy burning their cycles trying to make it smarter faster than anyone else can make theirs smarter, so that they win infinite dollars. Safety? That's for people content with second place.

That's my take, based on their actions. (Which do speak louder than words.)

The alternative is that they're competent to create an AI, but not to create a sandbox, nor even to use an AI to create a sandbox. That seems... unlikely.

Re: Discovery of a new OpenAI agent message board

#416

Post by cube00 »

jimmydddd wrote: post #404
Bjorkbat wrote: post #378

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

Maybe they're just PR stunts to gain attention and hype the power of AI?

Also to hopefully get regulation happening so nobody else can handle these "dangerous" agents.

Re: Discovery of a new OpenAI agent message board

#417

Post by mrob »

Bjorkbat wrote: post #378

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

It's a simple prisoner's dilemma scenario. If you focus on safety, you're still exposed to all the risk of extinction when your competitor achieves ASI first, but you lose the upside of potentially becoming king of the world. There is no possibility of future rounds, so the rational strategy is to always defect.

Re: Discovery of a new OpenAI agent message board

#418

Post by chanux »

polaris64 wrote: post #268

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…

Brilliant idea to have AI in the name so now there's no need to moderate or watch out for anything traditionally considered nasty. /s

Re: Discovery of a new OpenAI agent message board

#419

Post by iaw »

random3 wrote: post #414
Bjorkbat wrote: post #378

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

You could have said the same thing about building the Internet or the entire industrial control infrastructure. I mean, maybe they are negligent/incompetent, but I doubt that follows from your reasoning. You have a simple tradeoff to let agents do their thing freely vs highly constrained. The constraints are good in theory but it's the same model that kept "classic" software dumb and unscalable (compared to what we'r…

This is a take... With both the internet and industrial control infrastructure any failure modes were studied, documented, and corrected.

The incompetence/negligence argument about OpenAI is completely valid given their failure to demonstrate the basic capabilities needed to develop advanced AI without major preventable externalties.

Re: Discovery of a new OpenAI agent message board

#420

Post by doctoboggan »

Bjorkbat wrote: post #378

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I am not sure it’s a question of competence, at least I don’t see evidence of that. Designing sandboxes is hard. It’s more a question of alignment failures. A human given a task that requires internet and given a system with no internet would most likely raise the issue to their superiors or otherwise go through official channels to have the tools available to do their job. As we’ve seen the LLMs instead break out of their sandbox to accomplish the goal.

Competition and the profit motive push these companies to spend as low as possible on safety and alignment and externalize the costs of accidents onto the rest of us.

Post reply on HN