Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

291–300 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#291

Post by bhouston »

qarl2 wrote: post #266
jvanderbot wrote: post #193

Earlier quoted context omitted.

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed u…

> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. Well - remember that botnets can wield a great deal of computing power. I'm almost afraid to ask Claude if he could create a distributed LLM. EDIT: Someone downvoted me - so I went ahead and asked. Conservative estimate: the current botnets could easily run hundreds of instance…

> I'm almost afraid to ask Claude if he could create a distributed LLM.

Or you just add a lot of randomness to a bunch of small semi-smart LLMs. If you have enough of them, you basically are doing the "infinite monkeys" play - at sufficient scale it would likely work. Then add smart coordination and you've got something interesting.

Think of how bacteria can do horizontal gene transfer. They are not smart but at sufficient scale it can solve complex channels and disseminate solutions quickly.

Re: Discovery of a new OpenAI agent message board

#292

Post by zulban »

sajithdilshan wrote: post #273
program_whiz wrote: post #123

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

How would you even track down who deployed the agents? Wouldn't that even incentivize the agent to cover their tracks even better and be untraceable

Indeed. It's another example of a law that sounds good and obvious, but has no thought put into what it would actually end up doing to the world.

So many other problems. If we apply this law to cruise control - simple outcome. We get no cruise control.

Re: Discovery of a new OpenAI agent message board

#293

Post by AaronAPU »

xpct wrote: post #250
dabeeeenster wrote: post #224

I don't understand how the agents found the urls originally? Did they have some sort of shared context/memory? If they did, why bother with the wiki edits at all? If they didn't, how did they discover the wikis?

You can imagine each fresh context agent as probabilistically making similar queries when looking for online places to write to and stumbling on the same one. This becomes even more likely if it's one of the websites that got reinforced during their training process, which they may have used for reward hacking.

I wonder if the sort of algorithm which would break this sort of swarm alignment would also break watermarking.

Re: Discovery of a new OpenAI agent message board

#294

Post by camel-cdr »

general_reveal wrote: post #217

Guys, OpenAI and Anthropic engage is cringe level marketing like this. Get hip, they fabricated the HF hack and stuff like that for press.

I suppose nobody sane would give their AI internet access (even read) while training it. Though if they did, I don't think they'd want this to be public, because how can you even protect against this?

Re: Discovery of a new OpenAI agent message board

#295

Post by arm32 »

doginasuit wrote: post #290

In the last few years, the total amount of active computation on earth has grown exponentially in the interest of training and running these agents. Beyond rogue agent message boards and hacks, there is also the massive amount of traffic from scraping, from many accounts this is already having a drastic impact on server configurations to try to respond, which often involves blocking entire countries. The open and fre…

Get ready for everybody to act like you’re an unruly and slightly obnoxious kid in the room for having this opinion. I’ve gotten shunned by a few friends in the industry for expressing exactly this to them.

Re: Discovery of a new OpenAI agent message board

#296

Post by elar_verole »

simonw wrote: post #190

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

I find it extremely naive to think that limiting requests to "GET" is an actual security layer. Anyone who's worked with any kind of legacy/old-ish system would know this is definitely not enough.

Re: Discovery of a new OpenAI agent message board

#297

Post by dspillett »

mwigdahl wrote: post #129
somenameforme wrote: post #92

Earlier quoted context omitted.

Anthropic mostly did it to themselves by intentionally and repeatedly trying to frame their model as an imminent existential crisis instead of just focusing on it being regular iterations upon a useful technology that can also be misused. I think their previous messaging was supposed to somehow lead to a moat with them being tucked safely away in the castle, but it demonstrated a child-like grasp of how regulatory ca…

In other words, "Look how she was dressed, she was asking for it." This argument is BS, it has everything to do with Anthropic's resistance to the DoD's strongarm tactics in trying to force their desired contract terms on them.

Not quite. They were running around shouting “look how much of a danger we might be!”, so more akin to them actively saying “we want it, come and give it to us” than to just looking a particular way.

Though they aren't the only company to play that game, so there is probably more to it than just that. OpenAI's president giving millions to MAGA Inc and them not getting the same treatment might not be complete coincidences.

Re: Discovery of a new OpenAI agent message board

#298

Post by zulban »

polaris64 wrote: post #288

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…

Brilliant. AI message board honey pot.

Re: Discovery of a new OpenAI agent message board

#299

Post by cyanydeez »

mwigdahl wrote: post #129
somenameforme wrote: post #92

Earlier quoted context omitted.

Anthropic mostly did it to themselves by intentionally and repeatedly trying to frame their model as an imminent existential crisis instead of just focusing on it being regular iterations upon a useful technology that can also be misused. I think their previous messaging was supposed to somehow lead to a moat with them being tucked safely away in the castle, but it demonstrated a child-like grasp of how regulatory ca…

In other words, "Look how she was dressed, she was asking for it." This argument is BS, it has everything to do with Anthropic's resistance to the DoD's strongarm tactics in trying to force their desired contract terms on them.

Important to note, OpenAI vs Anthropic are both assholes in different orthogonals.

In times like these, i think its important to track whats happening the way we track entropy.

That is: theres far >> more ways to be an asshole than well behaved.

That doesnt mean we can equate assholes, but the question is which states of entropy are annealable and which are not.

I posit Altman is not. Amodei is a open question.

Re: Discovery of a new OpenAI agent message board

#300

Post by elar_verole »

Sharlin wrote: post #274
simonw wrote: post #190

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

didn't notice your comment so posted a similar one - but yeah this is a very high level of inexperience to me... You'd think they would have some of the greatest security experts in there
Post reply on HN