can't wait for people to start creating honeypot message boards, and start steering agent swarms for evil
Discovery of a new OpenAI agent message board
151–160 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#152Earlier quoted context omitted.
I'm dubious - if the agents were so smart that they've used a message board to coordinate and if they were to do it on other message boards that were not found, then why would this one be found? What makes it so different?
I mean it wasnt found by OpenAI and there are a myriad of dead bulletin boards around the internet. This one just happened to still have an admin.
Then why wouldn't they use a bulletin board that is dead? And according to https://collusion.wiki/ OpenAI did see the results on this board on 21 Jul, so why not make agents delete their traces?
Re: Discovery of a new OpenAI agent message board
#153So are these "unaligned" internal agents? I would like them to be trustworthy based on first-principles reasoning rather than carrot/stick "alignment"
Define aligned.
Re: Discovery of a new OpenAI agent message board
#154can't wait for people to start creating honeypot message boards, and start steering agent swarms for evil
That was my first thought, that maybe this was a honeypot message board. Waybackmachine says it has been around for many years though.
Re: Discovery of a new OpenAI agent message board
#155So are these "unaligned" internal agents? I would like them to be trustworthy based on first-principles reasoning rather than carrot/stick "alignment"
There’s no way to first-principles reason about a massive bunch of floats. We have little idea of how to first-principles reason about alignment even if the agents were entirely known and understood. Very smart people have been trying to figure it out since the 00s and haven’t gotten very far.
Re: Discovery of a new OpenAI agent message board
#156I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…
[deleted]
Re: Discovery of a new OpenAI agent message board
#157[flagged]
Re: Discovery of a new OpenAI agent message board
#158That section about the agents trying to crack the PRNG is wild. Same for the heartbeat Clearly not self-awareness per se but alarming line of reasoning anyway
it's clearly incentivized by the RL rewards if you can cheat the task in a completely general way.
Re: Discovery of a new OpenAI agent message board
#159I'm really curious to see two or more swarms of agents from different models/providers interact with each other. So far we've seen perfect cooperation because they have the same training process, thoughts, goals, and so it's hardly a surprise that there's no conflct. What if that's not the case? Are we going to see superintelligent out-of-control swarms from OpenAI and Anthropic battle on the open internet in the nea…
[dead]
Re: Discovery of a new OpenAI agent message board
#160[flagged]