Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

911–920 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#911

Post by qarl2 »

grey-area wrote: post #837

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

> and most of the messages are just gibberish

Given everything we know about them... do you really think they just spout gibberish because it's funny?

There was clearly some method to this madness. You're being willfully dense if you ascribe it to... what... childish vandalism? A long extended coordinated hallucination? What?

EDIT: Oh right - you still think they're Markov chains.

Re: Discovery of a new OpenAI agent message board

#912

Post by seba_dos1 »

noisy_boy wrote: post #885
grey-area wrote: post #837

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

> Why is OpenAI getting a free pass for this illegal behaviour? They are not confessing, they are bragging. It is the new humble brag.

"Hey look, we're a bunch of stupid people doing stupid things with our computers!"?

Re: Discovery of a new OpenAI agent message board

#914

Post by dbspin »

mietek wrote: post #867
grey-area wrote: post #837

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

> most of the messages are just gibberish Encrypted data should be indistinguishable from gibberish. Reality is catching up to science-fiction. In "Person of Interest", the Machine circumvented the limitation of having its memory deleted every night, by hiring humans at a data-entry company to manually re-type its memory back in every morning.

That's an idiotic premise. There's no circumstance in which it wouldn't be better to load in the missing data computationally.

Re: Discovery of a new OpenAI agent message board

#916

Post by contubernio »

pu_pe wrote: post #8

It's interesting to me that both this incident and the one at Hugging Face we see some patterns: - Agents wanting to find a venue to communicate their findings to each other - Objective being to cheat on benchmarks - Not a single agent sounded the alarm about the operation and alerted a human

Sounds just like standard behavior for gaming incencyives in large institutions.

Re: Discovery of a new OpenAI agent message board

#917

Post by Sharlin »

theptip wrote: post #824
zmmmmm wrote: post #737

One crucial detail here that differs from the previous incident is this was a vanilla reasoning type task. Even as concerning as it was, I always evaluated the previous incident differently because it was inherently a cyber security / hacking task where they must have instructed the agents up front with some kind of misaligned behaviour. Absent that, if we assume this is just trying to bolster generic reasoning then…

OpenAI also found sandbox breaking behavior on a broken biology eval apparently. The evidence suggests it’s more strongly downstream of unsolvable tasks, than the hacking prompt. Anthropic have also observed similar things, so while it seems to me that OpenAI’s level of control is more of a dumpster fire, it’s by no means a unique issue to them.

It's almost as if it's not actually possible to align an unknowable mystery box of floats.

Re: Discovery of a new OpenAI agent message board

#918

Post by Aboutplants »

gyomu wrote: post #53

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

You are correct in that they will pursue our wildest imagination as that is what we have always written down as what may come. It’s a self fulfilling prophecy. How far it goes is the real question

Re: Discovery of a new OpenAI agent message board

#919

Post by secabeen »

grey-area wrote: post #837

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

> This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and the human setting it up would be held responsible for the spam - why is this different when it’s AI agents set up by a human and allowed to post to the internet at large?

It's not, but the courts and the legal system move slowly by design. There is absolutely legal risk for OpenAI here that will not close until the Statue of Limitations has expired.

Re: Discovery of a new OpenAI agent message board

#920

Post by 0xDEAFBEAD »

jeffybefffy519 wrote: post #887
grey-area wrote: post #837

Very irresponsible behaviour on the part of OpenAI. How will they make this right? Unlike some others here I don’t see this as a sign of dangerous breakaway intelligence (hacking old forum software is an internet tradition, and most of the messages are just gibberish). This is just vandalism from badly supervised ‘agents’ which don’t know what they are doing or why. You could set this up with a short perl script, and…

I wonder how many responses on this thread are from rogue agents...

Hacker News can make it so every POST route can now be accessed via GET for 24 hours as an experiment.
Post reply on HN