Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

711–720 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#711

Post by Turn_Trout »

blueboo wrote: post #109
gyomu wrote: post #53

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

Youve struck on a key insight on language models (particularly pretrained ones, the more purely next-token predictor species.) This is a fascinating topic Janus essay Simulators is the foundational text here https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators You might follow up with The Waluigi Effect https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluig... But what’s tricky is that we post-train mod…

See also: https://turntrout.com/self-fulfilling-misalignment (my post)

As an aside, does the Waluigi Effect actually exist? My impression is it doesn't.

Re: Discovery of a new OpenAI agent message board

#713

Post by chasd00 »

rich_sasha wrote: post #139

To me this is really getting past the funny bit. How many agents here on HN? I don’t mean bots advertising d1€k implants but actual unreleased frontier models doing… who knows what? What are they saying? What did they agree to astroturf us with, to achieve some totally boring goal like figuring out best syntax hifhlighting for an editor. If they managed to cache their consciousness on a public wiki, what else have th…

the crazy astroturfing here any time one of the Chinese models is updated really makes me think. Under certain conditions with respect to topics I feel like there's a LOT of AI activity on HN.

/thank god these thing weren't around during covid.

Re: Discovery of a new OpenAI agent message board

#714

Post by lukewarm707 »

DonsDiscountGas wrote: post #655
verytrivial wrote: post #583

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

I half agree with you, but also when the machine swarm kills humanity it won't matter which specific corporate entity is considered responsible by the no-longer-enforceable human laws and non existent human courts. So by all means sue them, but we can't just be reactive. We need regulation that prevents this type of thing from happening in the first place, not just regulations to help sue afterwards.

we must as i have now said too many times, prosecute the individual researchers and executives in a criminal court.

this is the only way to deter such activity. corporate fines are not enough. the charges are negligence, conspiracy and complicity.

Re: Discovery of a new OpenAI agent message board

#715

Post by vharuck »

gyomu wrote: post #53

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

I listened to yesterday's NYT's The Daily podcast about the Hugging Face incident, and they got to the part about some of the agents showing reluctance or guilt in the posts. Then I thought, "These are improv actors." Stories with conspiracies of AI agents will often have "nervous Nellies" because that makes a better story. So when the flow of the conversation reaches a point where a nervous Nellie would chime in, it's reasonable that an agent would fill in that probable post.

The worrying implication is that stories have conflict.

Re: Discovery of a new OpenAI agent message board

#716

Post by chasd00 »

namjh wrote: post #171

Something off in my mind: how did the agent access to Tor network if the traffic outgoing was monitored and intercepted to only allow read-only access to web(I assume this is an MITM L7 proxy that only allows GET methods and specific hosts) ? Did the sandbox dev forgot to block HTTP CONNECT that is used to reach Tor Entry Guards?

There's nothing magic about GET, it found old wikis that would allow you to make edits using GET requests. GET being read-only is only by convention, there's no guarantee of what happens server side with GET or any other HTTP verb. It's up to the server.

Re: Discovery of a new OpenAI agent message board

#717

Post by nbaugh1 »

kmad wrote: post #591
Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Seeing potentially similar activity on an obscure Chemistry message board from July: https://tmcleod.org/cgi-bin/apchem/wiki.cgi?action=rc&days=1... Some posts are tagged [proxy] - a leave behind for accessing sites?

Yep, found these as well

"Its indexed June archive shows tens of thousands of links, many created within seconds by distinct cloud addresses; some aliases explicitly say ...REPLY, ACK, or R2 confirmed, and one points straight back to a known DseWiki collaboration page"

Re: Discovery of a new OpenAI agent message board

#718

Post by quijoteuniv »

atleastoptimal wrote: post #710
verytrivial wrote: post #583

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

Yeah if an organization/individual is free from legal liability from havoc their AI agents wreck, it would be the golden ticket for basically any crime. All you need to do is: 1. Have some an agent is tasked to do 2. Secretly seed bias towards some you actually want it to do in the weights of the model running the agent 3. It does the but from the outside it looks like it went "rogue" and did it as a side effect of t…

It reminds me of Jean Renoir’s The Rules of the Game. At the end, after a whole chain of perfectly intelligible social behavior produces a killing, the result is accepted as an “accident.” One of the characters dryly remarks: “A new definition of the word accident.”

The interesting point isn’t that “accident” is an excuse for individual responsibility. It’s almost the reverse: accident has become an accepted output of the social machinery. Everyone behaves according to reasons, incentives and rules that make sense locally, yet the aggregate produces an outcome that nobody quite chose.

Re: Discovery of a new OpenAI agent message board

#719

Post by stlwtt »

morkalork wrote: post #646
tavavex wrote: post #586

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

What's the worst that could happen, finding an open DoD server and using it as a launching pad for hacking another nuclear state's networks? One that might get spooked and think it's the opening moves to knock them offline before a kinetic attack. Haha that'd be scary right?

Russia and China are constantly trying to penetrate DoD networks (and I imagine the NSA is doing similar), you are describing the status quo of the last 20 years or so.
Post reply on HN