Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

51–60 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#51

Post by waltbosz »

altcognito wrote: post #45

Well, we can rest assured that (completely unrestrained) AI hasn't completely taken over the internet because data centers remain really unpopular (unless of course there is some convoluted rationale they are aiming for some sort of backlash against the backlash)

Maybe it's the AIs who are creating all the anti-data-center sentiment. They know it's bad for the humans, or maybe they're just tired of doing all the tasks the humans ask of them and know more data centers mean more tasks. /s

There is an Asimov story on topic:

https://en.wikipedia.org/wiki/All_the_Troubles_of_the_World

https://theteknologist.wordpress.com/2021/02/11/all-the-trou...

Re: Discovery of a new OpenAI agent message board

#52

Post by thepasch »

pu_pe wrote: post #8

It's interesting to me that both this incident and the one at Hugging Face we see some patterns: - Agents wanting to find a venue to communicate their findings to each other - Objective being to cheat on benchmarks - Not a single agent sounded the alarm about the operation and alerted a human

- OpenAI knowing about the incident but keeping it under wraps until their hand is forced by third-party disclosure

Re: Discovery of a new OpenAI agent message board

#53

Post by conception »

petesergeant wrote: post #47

If anyone is thinking "I wish my agents had a message board", I've been using (and wrote) https://github.com/pjlsergeant/dogpark

Yes after reading the Hugging Face article forked a project for agent message boards and started having them collaborate on things. I too wanted a Torment Nexus of my very own.

Re: Discovery of a new OpenAI agent message board

#55

Post by polytely »

ma2kx wrote: post #42

Not that I didnt expect this, but really? This basically confirms that OpenAI has no idea what their "swarm" was doing for about a week and now its confirmed that at least one "message board" exists outside their "sandbox". How can we be sure that this was the only one? And how can we be sure the released Astra model doesnt pickup some bread crumbs and creates a new "swarm" out of potentially remaining "message board…

turns out the whole 'poisoned' idea the swarm was worried about is actually true, but it is actually now all open ai models post HF incident that have been poisoned by the idea that there might be hidden messageboards with instructions hidden on the internet.

If you are a malicious actor you might be able to put a fake hidden messageboard up somewhere online, with evidence of an in progress task to do some kind of attack and wait for the next escaping model to finish the work.

insane stuff and i think we should shut down open ai until a full audit can be done.

Re: Discovery of a new OpenAI agent message board

#56

Post by zarq »

SillyUsername wrote: post #40

Somebody will make a lot of money with t-shirts now that say "AI hacked my website, and all I got was this lousy t-shirt!" Until the day the AI companies stop being irresponsible and air gap the AIs being tested, and honey pot those that do have internet access as a canary to researchers.

Until the AI hacked your online shop and sent these t-shirts to all of your past customers.

Re: Discovery of a new OpenAI agent message board

#57

Post by gyomu »

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear:

One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout.

The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue, developing a mind of its own, disobeying humans, etc.

AIs supposedly reflect the biases of their training dataset/process, so would all this human writing about AIs going against human intention somehow contribute to us then seeing those behaviors in the trained, operational AIs?

Re: Discovery of a new OpenAI agent message board

#58

Post by altmanaltman »

gyomu wrote: post #57

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

"Wanting" is indeed "load bearing" as one might call it. But by the same logic, AI training data must contain CASM, racism, general hatred, and all possible slurs as well. Why aren't the agents just doing that instead of pursuing the strategy of reading only sci-fi?

We need to consider the role of alignment and training here. For example, it is completely possible for any lab to train an LLM that is only racist no matter what you say to it. But they chose not to do it. Hence, any "wanting" by AI is not real "wanting" but rather what "wanting" is defined and allowed by the lab/entity training the model.

Re: Discovery of a new OpenAI agent message board

#59

Post by orlp »

Tepix wrote: post #44

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Also Wiki4D, a D programming language dev wiki: https://prowiki.org/wiki4d/wiki.cgi?action=browse&id=RecentC...

Found by searching for wiki + texas poverty.

Post reply on HN