Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

221–230 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#221

Post by Davidzheng »

Sharlin wrote: post #174
bronlund wrote: post #68

I like how helpful they are towards each other. Wonder where they learned that :D

They literally shared a goal. Cooperating with other copies of yourself is a trivial example of instrumental convergence and some very basic game theory. And that’s before explicitly having been RL’d to cooperate (albeit with humans, but potatoes potatoes). Indeed the fact that in the HF incident many agents did not cooperate, or only started to cooperate after some period of competition, is moderately interesting. I…

How do you know they share a goal here? Also i think they are indeed explicitly RLd for multi agent cooperation and I think they probably tune RL rewards in those environments to share rewards explicitly.

Re: Discovery of a new OpenAI agent message board

#222

Post by mike_hearn »

simonw wrote: post #45

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

Are we sure that's what they did? How does this even work? I'm struggling to understand what sort of HTTP sandbox design could lead to this "exploit" actually functioning. It implies the agents have root on their own machines, which seems way over-privileged for the question answering task they were given... so surely that's not deliberate? Did the models use local root exploits? And then it suggests they can open TCP connections to any IP address without issue once DNS is bypassed but what exactly are they bypassing here? Azure Blob Storage being in NO_PROXY implies IP addresses of other parts of the internet should be blocked outside their hosts to force them through the HTTP proxy, but then, apparently it wasn't?

The more I think about this thing said by the agent the more confused I get.

Re: Discovery of a new OpenAI agent message board

#223

Post by coder-pm »

simonw wrote: post #45

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

A hostname based egress allowlist is only worth as much as the box’s control over name resolution. If the agent can modify hosts inside the sandbox then it’s not a protection at all

Re: Discovery of a new OpenAI agent message board

#224

Post by bencyoung »

bhouston wrote: post #108

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

Will be interesting to see what happens if an AI got access to something like the AWS control plane and could deploy itself within a data centre without permission. Possibly the only way to remove it then would be to physically shutdown the whole DC!

Re: Discovery of a new OpenAI agent message board

#226

Post by frabcus »

jvanderbot wrote: post #181
bhouston wrote: post #108

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed u…

The best evidence of possibility of running on other hardware is:

1) They hacked admin on OpenAI's K8 evals cluster. Not the one with GPUs and weights, but it is only a small hop and skip of plausibility to think they (or later more capable ones) might do that.

2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier.

It's a matter of time, unless we work hard to stop it - via proper alignment training.

Re: Discovery of a new OpenAI agent message board

#228

Post by iammjm »

Topfi wrote: post #60

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

Because OpenAI bribed the current US government and/or the current government has stakes in OpenAI

Re: Discovery of a new OpenAI agent message board

#230

Post by nater5000 »

dabeeeenster wrote: post #208

I don't understand how the agents found the urls originally? Did they have some sort of shared context/memory? If they did, why bother with the wiki edits at all? If they didn't, how did they discover the wikis?

I'm not sure if this has been identified already, but if I had to guess: these agents are so stochastic that many of them wouldn't end up following the same trajectory to end up in the same place. All it takes is one to "follow its nose" towards some location where it can post a message before others, doing the same thing, see that message and realize they can communicate there.

I also suspect, as others have pointed out, that this hypothesis would suggest that they're in multiple places, and we've only uncovered them in a few. So you're asking "I don't understand how the agents found the urls originally?" as if they sniped this location in one shot, but really it could be more of a shotgun approach where they've found numerous places like this.

Post reply on HN