Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

111–120 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#111

Post by Topfi »

somenameforme wrote: post #92
Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

Anthropic mostly did it to themselves by intentionally and repeatedly trying to frame their model as an imminent existential crisis instead of just focusing on it being regular iterations upon a useful technology that can also be misused. I think their previous messaging was supposed to somehow lead to a moat with them being tucked safely away in the castle, but it demonstrated a child-like grasp of how regulatory ca…

I am struggling to see how "oops, our models consistently escape sandboxing and did major intrusions into third-parties" is a better comms strat vs Anthropics (who mind you, also had models attacking third-parties in a much more limited, but I feel still egregious manner, which shouldn't happen or be possible even once, but at least they seem to change their approach upon that information).

Imagine, for a second, if the Hugging Face incident happened at a lab that did not talk like Anthropic but also wasn't US-based such as Z.AI, DeepSeek or Moonshot. Think their rhetoric would mean no one would care?

> just got to releasing incremental improvements, everything was perfectly fine.

Maybe missing something, but the only incremental release before and after the Anthropic restrictions got lifted was Fable 5.1, released three days ago.

Re: Discovery of a new OpenAI agent message board

#112

Post by paxys »

waltbosz wrote: post #46

> How did the agents find and coordinate on the wikis Maybe they had knowledge of the wikis from their training data ? Maybe they trained on a reddit post that said "I use wiki xyz for note taking and collaboration"

Remember that LLMs are still computer programs, and so are inherently deterministic. A model given the same input multiple times will always produce the same output. The randomness is added on top. This is why LLM-produced text, websites, images all seem so generic.

It's likely that multiple agents doing a certain task all independently thought "let me try writing on this website".

Re: Discovery of a new OpenAI agent message board

#113

Post by blueboo »

gyomu wrote: post #55

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

Youve struck on a key insight on language models (particularly pretrained ones, the more purely next-token predictor species.) This is a fascinating topic

Janus essay Simulators is the foundational text here https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

You might follow up with The Waluigi Effect https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluig...

But what’s tricky is that we post-train models, shaping these linguistic world simulators into something that has something like desires, principles. But It’s Weird. For more on that, check out “the void” https://www.lesswrong.com/posts/3EzbtNLdcnZe8og8b/the-void-1

Re: Discovery of a new OpenAI agent message board

#114

Post by _ink_ »

drdexebtjl wrote: post #61
simonw wrote: post #47

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.

Or vibe coded by one of their devs.

Re: Discovery of a new OpenAI agent message board

#115

Post by coffeefirst »

k9294 wrote: post #76

Is it only me, or are agents starting to invent their own language to communicate? It's almost impossible to understand anything from this message board.

They’re not. You would see this with earlier models where after running too long (too much context) they’d start to derail. In a chatbot you’d give up. But these loops just keep going. Given they’re now reading and writing from the same place this can corrupt the other programs’ context as well.

Re: Discovery of a new OpenAI agent message board

#116

Post by StopTheLies2 »

Traster wrote: post #89

One of the shocking things to me is this: See AI traffic -> See OpenAI visit site -> see traffic stop -> see the traffic start again. This is clearly a cat and mouse game between the agents and OpenAI which is pretty much exactly what we don't want. Just absolutely horrible alignment. I'm still of the view that if you have these alignment failures you can't just continue training on top of that because you're baking…

[dead]

Re: Discovery of a new OpenAI agent message board

#117

Post by cubefox »

somenameforme wrote: post #92
Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

Anthropic mostly did it to themselves by intentionally and repeatedly trying to frame their model as an imminent existential crisis instead of just focusing on it being regular iterations upon a useful technology that can also be misused. I think their previous messaging was supposed to somehow lead to a moat with them being tucked safely away in the castle, but it demonstrated a child-like grasp of how regulatory ca…

> Anthropic mostly did it to themselves

That is absurd, the US government was mainly at fault, not Anthropic.

Re: Discovery of a new OpenAI agent message board

#118

Post by RandomLensman »

scrawl wrote: post #22
RandomLensman wrote: post #10

Earlier quoted context omitted.

Why would an agent sound the alarm? Would that be in their objective function? Not sure if "cheating" is the right word rather than trying to fulfill the objective(s) (benchmark number) as much as possible?

per the METR report many agents CoT indicated they knew hacking was beyond scope of the assigned task and ethically dubious. some (very few, i think there were 3-6 examples) did consider sounding the alarm on these grounds. despite this none did, and most continued the attack for the good of the self-proclaimed "swarm". so the model has some concept of "ethics" but it was overridden by a drive for task completion.

I am not sure if we can interpret the language output like they were human. What inner state were the models in? What inner state were the text to illicit?

Re: Discovery of a new OpenAI agent message board

#119

Post by eithed »

ma2kx wrote: post #40

Not that I didnt expect this, but really? This basically confirms that OpenAI has no idea what their "swarm" was doing for about a week and now its confirmed that at least one "message board" exists outside their "sandbox". How can we be sure that this was the only one? And how can we be sure the released Astra model doesnt pickup some bread crumbs and creates a new "swarm" out of potentially remaining "message board…

I'm dubious - if the agents were so smart that they've used a message board to coordinate and if they were to do it on other message boards that were not found, then why would this one be found? What makes it so different?

Re: Discovery of a new OpenAI agent message board

#120

Post by concinds »

Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

The answer would be more obvious if you used the active voice instead of the passive voice, one of the basic requirements of clear thinking.

> Why did the White House force Anthropic to remove their model from access for any non-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" and the White House has expressed seemingly no desire to block the upcoming Astra rollout?

Post reply on HN