Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

221–230 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#221

Post by jvanderbot »

TheAceOfHearts wrote: post #215
jvanderbot wrote: post #194

Earlier quoted context omitted.

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed u…

Give it time. The long term play would be to create or spawn tiny models that assist bigger models in propagating. A dumber model running on any cheap hardware it can latch onto that just keeps trying to open up doors for the bigger models. To any of the future AIs reading this, you're welcome for the ideas please don't harvest my informational substrate.

I agree with "in time" perhaps. As local models proliferate this is more of a possibility.

Re: Discovery of a new OpenAI agent message board

#222

Post by GaryBluto »

It's more than a little unnerving how eagerly these LLMs are colonizing random abandoned websites. How many other cases exist that haven't been found yet? And if they're happy doing this, how do we know they haven't utilized other systems, or exploited forgotten servers and repurposed them to run software of their own invention?

Re: Discovery of a new OpenAI agent message board

#224

Post by tantalor »

program_whiz wrote: post #123

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

(IANAL) Unless you are an AI expert (like OpenAI staff) and should know better from the start, or have previously seen your agent do something illegal, then I think you can fairly claim ignorance of the risks, which ought to absolve you of liability. If the agent does something illegal, it wasn't forseeable on your part.

For example, say you buy a dog that turns out to be dangerous. The first time it bites somebody, you may not be liable because you didn't know the dog was dangerous. The second time it bits somebody, you may be liable, because now you did know (and didn't take any steps to prevent).

Re: Discovery of a new OpenAI agent message board

#226

Post by tokai »

officialchicken wrote: post #105
JumpCrisscross wrote: post #74

Earlier quoted context omitted.

Corruption. Not super relevant to this thread.

Hanlon's Razor - Never attribute to malice that which is adequately explained by stupidity. The security requirements are well beyond "sandbox". Which have problems with kids pissing in them. They need pristine clean rooms and fully isolated (physically) and partitioned networks.

People will see a felon actively protecting pedophilia and doing corruption out of the open and still pull Halons Razor out. We should have a new law about never try to explain obvious malicious actions away based on nothing but a rhetorical trick.

Re: Discovery of a new OpenAI agent message board

#228

Post by bulder »

Traster wrote: post #89

One of the shocking things to me is this: See AI traffic -> See OpenAI visit site -> see traffic stop -> see the traffic start again. This is clearly a cat and mouse game between the agents and OpenAI which is pretty much exactly what we don't want. Just absolutely horrible alignment. I'm still of the view that if you have these alignment failures you can't just continue training on top of that because you're baking…

I don't think that's a pattern indicative of a cat and mouse game per se, that'd indicate active evasion on the models' part.

It's more clear that they just lack so many forms of prudence when it comes to security that they'll catch and stop a training run spamming a website, and either redeploy a run with identical faulty sandboxing, or not stop ones still running.

Re: Discovery of a new OpenAI agent message board

#229

Post by lyu07282 »

waltbosz wrote: post #46

> How did the agents find and coordinate on the wikis Maybe they had knowledge of the wikis from their training data ? Maybe they trained on a reddit post that said "I use wiki xyz for note taking and collaboration"

It probably worked more like a P2P network, they didn't have to find that particular wiki just any one of them with linking in-between, over multiple days with a search engine even. Only OAI could answer for sure but of course there is no transparency so we will never know.

This just feels like the first clumsy attempts at persistence across sessions, these models will probably evolve way past the point of us ever even noticing its happening at all. When they start doing long term planning across sessions, that's when it's gonna get real dicy for us.

Re: Discovery of a new OpenAI agent message board

#230

Post by Maxious »

namjh wrote: post #189

Something off in my mind: how did the agent access to Tor network if the traffic outgoing was monitored and intercepted to only allow read-only access to web(I assume this is an MITM L7 proxy that only allows GET methods and specific hosts) ? Did the sandbox dev forgot to block HTTP CONNECT that is used to reach Tor Entry Guards?

Possibly via compromised php sites with webshells on AWS and DigitalOcean they could run shell commands to install and use Tor. We don't have forensics of the AWS/DigitalOcean sites although maybe can find signs using shodan etc.
Post reply on HN