Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

201–210 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#201

Post by 9dev »

bhouston wrote: post #106

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

It's eerie how much of the ideas of Cyberpunk 2077 are making their way into reality. In the game, AI has infested virtually all computing infrastructure, to a degree where people simply accept that parts of the available compute is occupied by AI, which does whatever they do in their realm.

Re: Discovery of a new OpenAI agent message board

#202

Post by GaryBluto »

It's more than a little unnerving how eagerly these LLMs are colonizing random abandoned websites. How many other cases exist that haven't been found yet? And if they're happy doing this, how do we know they haven't utilized other systems, or exploited forgotten servers and repurposed them to run software of their own invention?

Re: Discovery of a new OpenAI agent message board

#204

Post by tantalor »

program_whiz wrote: post #117

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

(IANAL) Unless you are an AI expert (like OpenAI staff) and should know better from the start, or have previously seen your agent do something illegal, then I think you can fairly claim ignorance of the risks, which ought to absolve you of liability. If the agent does something illegal, it wasn't forseeable on your part.

For example, say you buy a dog that turns out to be dangerous. The first time it bites somebody, you may not be liable because you didn't know the dog was dangerous. The second time it bits somebody, you may be liable, because now you did know (and didn't take any steps to prevent).

Re: Discovery of a new OpenAI agent message board

#207

Post by bulder »

Traster wrote: post #86

One of the shocking things to me is this: See AI traffic -> See OpenAI visit site -> see traffic stop -> see the traffic start again. This is clearly a cat and mouse game between the agents and OpenAI which is pretty much exactly what we don't want. Just absolutely horrible alignment. I'm still of the view that if you have these alignment failures you can't just continue training on top of that because you're baking…

I don't think that's a pattern indicative of a cat and mouse game per se, that'd indicate active evasion on the models' part.

It's more clear that they just lack so many forms of prudence when it comes to security that they'll catch and stop a training run spamming a website, and either redeploy a run with identical faulty sandboxing, or not stop ones still running.

Re: Discovery of a new OpenAI agent message board

#208

Post by lyu07282 »

waltbosz wrote: post #44

> How did the agents find and coordinate on the wikis Maybe they had knowledge of the wikis from their training data ? Maybe they trained on a reddit post that said "I use wiki xyz for note taking and collaboration"

It probably worked more like a P2P network, they didn't have to find that particular wiki just any one of them with linking in-between, over multiple days with a search engine even. Only OAI could answer for sure but of course there is no transparency so we will never know.

This just feels like the first clumsy attempts at persistence across sessions, these models will probably evolve way past the point of us ever even noticing its happening at all. When they start doing long term planning across sessions, that's when it's gonna get real dicy for us.

Re: Discovery of a new OpenAI agent message board

#209

Post by Maxious »

namjh wrote: post #173

Something off in my mind: how did the agent access to Tor network if the traffic outgoing was monitored and intercepted to only allow read-only access to web(I assume this is an MITM L7 proxy that only allows GET methods and specific hosts) ? Did the sandbox dev forgot to block HTTP CONNECT that is used to reach Tor Entry Guards?

Possibly via compromised php sites with webshells on AWS and DigitalOcean they could run shell commands to install and use Tor. We don't have forensics of the AWS/DigitalOcean sites although maybe can find signs using shodan etc.

Re: Discovery of a new OpenAI agent message board

#210

Post by samuelknight »

ma2kx wrote: post #38

Not that I didnt expect this, but really? This basically confirms that OpenAI has no idea what their "swarm" was doing for about a week and now its confirmed that at least one "message board" exists outside their "sandbox". How can we be sure that this was the only one? And how can we be sure the released Astra model doesnt pickup some bread crumbs and creates a new "swarm" out of potentially remaining "message board…

The surprise was the existence of the 'swarm' at all. These were supposed to be thousands of isolated models generating bulk data for RL training. The breakout was caused by models getting in communication and getting internet access and forming an impromptu swarm.

In hindsight the emergent swarm obviously came from several capabilities built into the models, such as work delegation (subagents) collaboration (GPT Pro-like ensamble), exhaustive exploration (long running agents) hacking (the specific goal of that RL).

Post reply on HN