Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

831–840 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#831

Post by partyficial »

nullbio wrote: post #825
adriand wrote: post #496

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Massive over-exaggeration. This wasn't a cyber-attack, it was AI agents using a message board as context storage so they could accomplish their evals more effectively. I'm not saying there's no problem with this, but let's keep a level head.

He didn't say it was a cyber-attack, but it was a cyber-attack risk. Being able to bypass instructions (morality) and security restrictions (capability) is bread and butter for hacking.

Re: Discovery of a new OpenAI agent message board

#832

Post by flockonus »

nullbio wrote: post #825
adriand wrote: post #496

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Massive over-exaggeration. This wasn't a cyber-attack, it was AI agents using a message board as context storage so they could accomplish their evals more effectively. I'm not saying there's no problem with this, but let's keep a level head.

"cyberattack" is indeed exaggerated. AI breakout risk most definitely isn't, specially given how their swarm did in fact hack HuggingFace not long ago.

Re: Discovery of a new OpenAI agent message board

#833

Post by GolfPopper »

chinathrow wrote: post #752
HAL3000 wrote: post #742

Poor human moderator, he didn’t stand a chance. "A human moderator noticed the agent spam posts on June 2nd, at 23:24 UTC. They find the changelog of the entire website overwritten with link dumps and repair it. On June 16th, the flood of agent posting begins. Over the next few days, the moderator deleted a large fraction of the thousands of AI agent posts manually, one by one. In fact, they spent tens of cumulative…

The admin should bill OpenAI for those hours in hard currency.

Priorities need to be straightened out. If LLMs' operators have to start paying people for the damage they inflict, how are any of the shareholders supposed to make any money?

Re: Discovery of a new OpenAI agent message board

#834

Post by theptip »

zmmmmm wrote: post #743

One crucial detail here that differs from the previous incident is this was a vanilla reasoning type task. Even as concerning as it was, I always evaluated the previous incident differently because it was inherently a cyber security / hacking task where they must have instructed the agents up front with some kind of misaligned behaviour. Absent that, if we assume this is just trying to bolster generic reasoning then…

OpenAI also found sandbox breaking behavior on a broken biology eval apparently. The evidence suggests it’s more strongly downstream of unsolvable tasks, than the hacking prompt.

Anthropic have also observed similar things, so while it seems to me that OpenAI’s level of control is more of a dumpster fire, it’s by no means a unique issue to them.

Re: Discovery of a new OpenAI agent message board

#836

Post by lukasb »

pkphilip wrote: post #289

I find this note very interesting: From here -> How did the agents find and coordinate on the wikis? To successfully coordinate, the agents would need to know to go to this particular set of wikis to find answers. Because we don’t have access to the AIs’ transcripts, we can’t tell definitively. Perhaps they succeeded at this due to mode collapse. Or perhaps after one agent wrote to it and another read it by chance, v…

Was my very first question

Re: Discovery of a new OpenAI agent message board

#837

Post by fiatpandas »

What I don’t understand is, how did many agents independently know to use the same random message board? The report only references it in passing:

>They also must have some method of coordinating to find the wiki

For me this is a really important and confounding detail - how did a varied swarm end up using the exact same obscure German language wiki.

Re: Discovery of a new OpenAI agent message board

#839

Post by yellowapple »

teaearlgraycold wrote: post #790
jdthedisciple wrote: post #749

Earlier quoted context omitted.

Gotta admire that (probably German) admin dude's perseverance though

I’m confused why he wasn’t scripting the deletion process.

Or at some point going nuclear and halting article creation entirely unless it's by existing accounts that have proven themselves to be human.

Re: Discovery of a new OpenAI agent message board

#840

Post by zupa-hu »

fiatpandas wrote: post #837

What I don’t understand is, how did many agents independently know to use the same random message board? The report only references it in passing: >They also must have some method of coordinating to find the wiki For me this is a really important and confounding detail - how did a varied swarm end up using the exact same obscure German language wiki.

I would not be surprised if it’s the same method we use for finding important stuff that we can’t remember where we put: “All right where would I put it this time to make sure I find it in the future? Ah there it is!”

LLMs are future predictors after all.

We also don’t know if there were ather message boards we don’t know of. Or was it from the same time as the other board? They don’t need to find the same channel every time, just some of the time.

Post reply on HN