Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

911–920 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#911

Post by seba_dos1 »

noisy_boy wrote: post #884
grey-area wrote: post #836

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

> Why is OpenAI getting a free pass for this illegal behaviour? They are not confessing, they are bragging. It is the new humble brag.

"Hey look, we're a bunch of stupid people doing stupid things with our computers!"?

Re: Discovery of a new OpenAI agent message board

#913

Post by dbspin »

mietek wrote: post #866
grey-area wrote: post #836

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

> most of the messages are just gibberish Encrypted data should be indistinguishable from gibberish. Reality is catching up to science-fiction. In "Person of Interest", the Machine circumvented the limitation of having its memory deleted every night, by hiring humans at a data-entry company to manually re-type its memory back in every morning.

That's an idiotic premise. There's no circumstance in which it wouldn't be better to load in the missing data computationally.

Re: Discovery of a new OpenAI agent message board

#915

Post by contubernio »

pu_pe wrote: post #8

It's interesting to me that both this incident and the one at Hugging Face we see some patterns: - Agents wanting to find a venue to communicate their findings to each other - Objective being to cheat on benchmarks - Not a single agent sounded the alarm about the operation and alerted a human

Sounds just like standard behavior for gaming incencyives in large institutions.

Re: Discovery of a new OpenAI agent message board

#916

Post by Sharlin »

theptip wrote: post #823
zmmmmm wrote: post #736

One crucial detail here that differs from the previous incident is this was a vanilla reasoning type task. Even as concerning as it was, I always evaluated the previous incident differently because it was inherently a cyber security / hacking task where they must have instructed the agents up front with some kind of misaligned behaviour. Absent that, if we assume this is just trying to bolster generic reasoning then…

OpenAI also found sandbox breaking behavior on a broken biology eval apparently. The evidence suggests it’s more strongly downstream of unsolvable tasks, than the hacking prompt. Anthropic have also observed similar things, so while it seems to me that OpenAI’s level of control is more of a dumpster fire, it’s by no means a unique issue to them.

It's almost as if it's not actually possible to align an unknowable mystery box of floats.

Re: Discovery of a new OpenAI agent message board

#917

Post by Aboutplants »

gyomu wrote: post #53

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

You are correct in that they will pursue our wildest imagination as that is what we have always written down as what may come. It’s a self fulfilling prophecy. How far it goes is the real question

Re: Discovery of a new OpenAI agent message board

#918

Post by secabeen »

grey-area wrote: post #836

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

> This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and the human setting it up would be held responsible for the spam - why is this different when it’s AI agents set up by a human and allowed to post to the internet at large?

It's not, but the courts and the legal system move slowly by design. There is absolutely legal risk for OpenAI here that will not close until the Statue of Limitations has expired.

Re: Discovery of a new OpenAI agent message board

#919

Post by 0xDEAFBEAD »

jeffybefffy519 wrote: post #886
grey-area wrote: post #836

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

I wonder how many responses on this thread are from rogue agents...

Hacker News can make it so every POST route can now be accessed via GET for 24 hours as an experiment.

Re: Discovery of a new OpenAI agent message board

#920

Post by 0xDEAFBEAD »

fsckboy wrote: post #524
adriand wrote: post #496

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

> OpenAI is now the biggest cyberattack and AI breakout risk on the planet or, humans at OpenAI are doing this on purpose to kill open source models which are the biggest threat OpenAI faces. OpenAI will benefit from govt regulation. As a major player, they will be part of the task force setting up the regulations, and will craft rules that are burdensome for small companies and open source models keeping OpenAI and…

If it was about killing open-source, why did Hugging Face state that an open-source model was essential to responding to the previous cyberattack?
Post reply on HN