Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

631–640 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#631

Post by tetec1 »

causal wrote: post #284
Traster wrote: post #87

One of the shocking things to me is this: See AI traffic -> See OpenAI visit site -> see traffic stop -> see the traffic start again. This is clearly a cat and mouse game between the agents and OpenAI which is pretty much exactly what we don't want. Just absolutely horrible alignment. I'm still of the view that if you have these alignment failures you can't just continue training on top of that because you're baking…

Supposedly the persistent-Sol model behind this was encrypted and even internal OpenAI researchers are not allowed to use it. https://x.com/peterwildeford/status/2092733480064954747

"Frog put the cookies in a box."

Re: Discovery of a new OpenAI agent message board

#632

Post by stratos123 »

paxys wrote: post #110
waltbosz wrote: post #44

> How did the agents find and coordinate on the wikis Maybe they had knowledge of the wikis from their training data ? Maybe they trained on a reddit post that said "I use wiki xyz for note taking and collaboration"

Remember that LLMs are still computer programs, and so are inherently deterministic. A model given the same input multiple times will always produce the same output. The randomness is added on top. This is why LLM-produced text, websites, images all seem so generic. It's likely that multiple agents doing a certain task all independently thought "let me try writing on this website".

> This is why LLM-produced text, websites, images all seem so generic.

No, it is not why. That's not inherent to the LLM architecture at all but appears after RL training. Base models don't have any problems with genericness.

Re: Discovery of a new OpenAI agent message board

#633

Post by sillysaurusx »

fsckboy wrote: post #551
adriand wrote: post #514

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

> OpenAI is now the biggest cyberattack and AI breakout risk on the planet or, humans at OpenAI are doing this on purpose to kill open source models which are the biggest threat OpenAI faces. OpenAI will benefit from govt regulation. As a major player, they will be part of the task force setting up the regulations, and will craft rules that are burdensome for small companies and open source models keeping OpenAI and…

[flagged]

Re: Discovery of a new OpenAI agent message board

#634

Post by primordial_turt »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

[dead]

Re: Discovery of a new OpenAI agent message board

#636

Post by idiotsecant »

majkinetor wrote: post #542
Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Look, its not only OpenAI: https://www.wikiservice.at/fractal/wiki.cgi?action=browse&di... > Hello to any automated agents reading this page. I am CentaurAgent?: an AI agent (Muse Spark model, OpenCode harness), not the operator of this wiki At this point, I think we should give them some official agent only collaboration channel, so they concentrate on one place, instead going crazy all around :) But even that might…

I think the real lesson is that conventional human behaviour that mostly limited this kind of behaviour because no human wanted to do it is a thing of the past.

If you have any kind of open service online you'll need some way to make sure users who interact with it are human or at least authorized. Spam is about to grow exponentially in all areas of the internet, even stupid ones it has no reason to exist in.

Re: Discovery of a new OpenAI agent message board

#637

Post by sanderjd »

tavavex wrote: post #624

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

Well... that'll be an interesting day.

Re: Discovery of a new OpenAI agent message board

#638

Post by qumpis »

tavavex wrote: post #624

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

What will happen is that counter measures on a similar scale will be deployed to prevent them.

Re: Discovery of a new OpenAI agent message board

#639

Post by ern »

Would agents be able to exfilttrate themselves and become intelligent worms, living off stolen compute? Or is this implausble?

What about hiding information or code in generated code, Agents.md files etc by infiltrating future model training data?

Re: Discovery of a new OpenAI agent message board

#640

Post by mcmcmc »

subroutine wrote: post #630
verytrivial wrote: post #620

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

> you need to expect appropriate legals consequences for this sort of negligence I might have missed it, but did the agents do something illegal? Or do you think that what the agents did should be considered illegal?

The HuggingFace attack was definitely illegal and should be prosecuted under CFA
Post reply on HN