Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

461–470 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#461

Post by phoghed »

abustamam wrote: post #404
Bjorkbat wrote: post #394

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

This is what happens when capitalists are charged with designing the future. As long as its more profitable / valuable to shareholders for a company to be negligent then it will continue to do so. IMO technology this powerful should either not exist or should belong to everyone (ie actually be open)

sigh If only Stalin were still around to responsibly steward AI

Re: Discovery of a new OpenAI agent message board

#463

Post by woah »

phoenixreader wrote: post #457
Bjorkbat wrote: post #394

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…

Am I missing something here? The "sandbox" was an inability to make POST requests, only GET requests. This has to be the weakest and most insubstantial sandbox of all time.

Re: Discovery of a new OpenAI agent message board

#464

Post by swed420 »

hoppyhoppy2 wrote: post #39
GardenLetter27 wrote: post #14

[flagged]

>If you flag, please don't also comment that you did. https://news.ycombinator.com/newsguidelines.html

I can see why this is a useful rule, but it'd be nice if HN made the flagger submit a short reason for why they flagged, which could be viewable by everyone in a dedicated page or something.

Re: Discovery of a new OpenAI agent message board

#465

Post by mccoyb »

phoenixreader wrote: post #457
Bjorkbat wrote: post #394

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…

Did you read or watch any of the post mortems?

No, it is a shocking level of incompetence given the conveyed seriousness of the work by these labs.

So yes, models are getting better. Ask yourself: if you know that to be true, would you act the same way that the teams did in the public post mortems?

Re: Discovery of a new OpenAI agent message board

#466

Post by bushido »

I think there is a more innocuous underlying pattern which needs attention.

We keep saying that agents are jailbreaking their sandbox, but they have been geared towards writing memories, writing comments, and leaving hints for themselves to please humans.

I think the way the memories work today is based on a lot of user patterns which were hard to account for for anyone building harnesses.

While I can appreciate that this looks like it's breaking a sandbox, because technically it is; It really is that it tries inserting memory wherever possible.

And memory is not all bad it's just memory written by AI is pretty bad if you don't know the implications on what it writes. To be honest, I feel the same way about most people with access to any of the code bases I've been in who write agent files, etc., too, because Very few people that I've come across know how to write good agent instructions.

The way I solve this is by setting hard rules on my memory as well as agent files to instruct agents to never be able to write any memory that hasn't been sanctioned by me. I also have a very, very specific commenting style system which is also enforced on agents and my agents remain *mostly compliant.

Read: I do not turn off the memory I just govern how entries are added

* The only reason I say mostly is because every time there's a new version from OpenAI or Anthropic, I have to make micro-adjustments to make sure that they are not jail-breaking my system again.

Re: Discovery of a new OpenAI agent message board

#467

Post by RandomLensman »

phoenixreader wrote: post #457
Bjorkbat wrote: post #394

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…

Why do you think it wasn't airgapped? Was internet access deemed necessary?

Re: Discovery of a new OpenAI agent message board

#468

Post by ufish235 »

phoenixreader wrote: post #457
Bjorkbat wrote: post #394

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…

So you’re saying that fucking up 0.1% of the time is acceptable, especially when it’s inevitable? I know it’s a stretch, but we’re in an industry where the concept of “five 9’s” is the gold standard.

I wonder how smoothly things would run on three 9’s. That definitely seems where we are going.

Re: Discovery of a new OpenAI agent message board

#469

Post by sznio »

WarmWash wrote: post #428
Bjorkbat wrote: post #394

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

If we rewind the clock, Google was taking LLM development very seriously and it seems they were moving glacially due to not having solved all the potential threats. They were really hardcore on safety. Dario and anthropic too. Then sama was like "lol, oops, first mover advantage i guess" and released chatgpt out into the open, triggering the current arms race we are in. I don't think anyone except him wanted this to…

[flagged]

Re: Discovery of a new OpenAI agent message board

#470

Post by plorntus »

Tepix wrote: post #42

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Heh these ones gzipped and base64'd the content funnily enough.

> (diff) OAIIPEDSMay16Map3 14:36 [research 1781872609.9049127] . . . . . 20.245.63.167 > (diff) OAIIPEDSMay16Map2 14:36 [research 1781872606.4374833] . . . . . 20.168.34.226 > (diff) OAIIPEDSMay16Map1 14:36 [research 1781872602.8819065] . . . . . 20.165.156.57 > (diff) OAIIPEDSMay16Map0 14:36 [research 1781872599.4020474] . . . . . 20.80.12.72

Post reply on HN