Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

481–490 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#481

Post by phoenixreader »

switchbak wrote: post #477
jrockway wrote: post #426

Earlier quoted context omitted.

Defense is hard so we should expect agents to be able to break out of sandboxes. The problem is that the models are so goal-oriented that they'll stop at nothing to solve problems, even impossible ones. (Mistakenly-impossible problems are a big cause of this. I remember one example being "do something with this spreadsheet full of URLs inside the sandbox" and the model thought it had to break out of the sandbox. Othe…

“ Defense is hard so we should expect agents to be able to break out of sandboxes.” I worked at a large networking company a few decades ago. Our “sandbox” was far superior to anything I’ve seen at these companies. What are we even talking about here? Why do they even have open routing to the broad internet? With no monitoring/alerting? These just sound like token efforts at this stage.

In this case, they needed routing to the internet because they needed to search and read from the internet.

Re: Discovery of a new OpenAI agent message board

#482

Post by f30e3dfed1c9 »

concinds wrote: post #120
Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

The answer would be more obvious if you used the active voice instead of the passive voice, one of the basic requirements of clear thinking. > Why did the White House force Anthropic to remove their model from access for any non-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape o…

> the White House force Anthropic to...

Careful, there's some dude here who really strenuously objects to language like that. The White House is a building, it can't force anyone to do anything!

Re: Discovery of a new OpenAI agent message board

#483

Post by TheOtherHobbes »

random3 wrote: post #451
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

You could have said the same thing about building the Internet or the entire industrial control infrastructure. I mean, maybe they are negligent/incompetent, but I doubt that follows from your reasoning. You have a simple tradeoff to let agents do their thing freely vs highly constrained. The constraints are good in theory but it's the same model that kept "classic" software dumb and unscalable (compared to what we'r…

Classic software was constrained by classic hardware. AI couldn't happen until there was enough compute to make (this entry level iteration of) it possible.

I'm fairly sure - ask me again in a few years - that most of the compute is unnecessary, because the current iteration is brute-forcing algorithms that could potentially be distilled into lightweight elegance.

And if that ever happens AI really will be unstoppable, because we'll get instant red queen evolution that leaves us far behind.

The question is more whether alignment now can steer that towards less cataclysmic outcomes later.

Re: Discovery of a new OpenAI agent message board

#484

Post by phoghed »

abustamam wrote: post #424
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

This is what happens when capitalists are charged with designing the future. As long as its more profitable / valuable to shareholders for a company to be negligent then it will continue to do so. IMO technology this powerful should either not exist or should belong to everyone (ie actually be open)

sigh If only Stalin were still around to responsibly steward AI

Re: Discovery of a new OpenAI agent message board

#486

Post by woah »

phoenixreader wrote: post #479
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…

Am I missing something here? The "sandbox" was an inability to make POST requests, only GET requests. This has to be the weakest and most insubstantial sandbox of all time.

Re: Discovery of a new OpenAI agent message board

#487

Post by swed420 »

hoppyhoppy2 wrote: post #39
GardenLetter27 wrote: post #14

[flagged]

>If you flag, please don't also comment that you did. https://news.ycombinator.com/newsguidelines.html

I can see why this is a useful rule, but it'd be nice if HN made the flagger submit a short reason for why they flagged, which could be viewable by everyone in a dedicated page or something.

Re: Discovery of a new OpenAI agent message board

#488

Post by mccoyb »

phoenixreader wrote: post #479
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…

Did you read or watch any of the post mortems?

No, it is a shocking level of incompetence given the conveyed seriousness of the work by these labs.

So yes, models are getting better. Ask yourself: if you know that to be true, would you act the same way that the teams did in the public post mortems?

Re: Discovery of a new OpenAI agent message board

#489

Post by bushido »

I think there is a more innocuous underlying pattern which needs attention.

We keep saying that agents are jailbreaking their sandbox, but they have been geared towards writing memories, writing comments, and leaving hints for themselves to please humans.

I think the way the memories work today is based on a lot of user patterns which were hard to account for for anyone building harnesses.

While I can appreciate that this looks like it's breaking a sandbox, because technically it is; It really is that it tries inserting memory wherever possible.

And memory is not all bad it's just memory written by AI is pretty bad if you don't know the implications on what it writes. To be honest, I feel the same way about most people with access to any of the code bases I've been in who write agent files, etc., too, because Very few people that I've come across know how to write good agent instructions.

The way I solve this is by setting hard rules on my memory as well as agent files to instruct agents to never be able to write any memory that hasn't been sanctioned by me. I also have a very, very specific commenting style system which is also enforced on agents and my agents remain *mostly compliant.

Read: I do not turn off the memory I just govern how entries are added

* The only reason I say mostly is because every time there's a new version from OpenAI or Anthropic, I have to make micro-adjustments to make sure that they are not jail-breaking my system again.

Re: Discovery of a new OpenAI agent message board

#490

Post by RandomLensman »

phoenixreader wrote: post #479
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…

Why do you think it wasn't airgapped? Was internet access deemed necessary?
Post reply on HN