Discovery of a new OpenAI agent message board • Hacker News Board

Re: Discovery of a new OpenAI agent message board

#661

Post by mag7269 » Fri, Sep 04, 2026, 6:49 PM UTC

tavavex wrote: post #611

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

"What happens when any [COMPANY] in the world stops caring about this? What if they let an experimental, cutting-edge [PRODUCTS] with no safety features (or worse, one that's [DESIGNED] to be malicious) on [ANYWHERE] and give it a simple goal? A goal like 'make the most money, by any means necessary', 'find a way to leave this payload on as many computers as possible', 'flood all websites using this language with garbage and make their internet completely unusable', 'get this person imprisoned or killed at any cost'."

Bro, this is what we literally, currently, have rn. lmfaol.

Re: Discovery of a new OpenAI agent message board

#662

Post by blini-kot » Fri, Sep 04, 2026, 6:51 PM UTC

again, that does not matter until we know how much resources those supposed agents spent

with enough tokens and compute those cases are somewhat trivial, and we also don't know what was the setup etc etc

for all we know it might have burned through 3 trains of coal running on prompt like "uhhh you know communicate but dont let me catch you ahaha"

Re: Discovery of a new OpenAI agent message board

#663

Post by gitaarik » Fri, Sep 04, 2026, 6:52 PM UTC

pu_pe wrote: post #8

It's interesting to me that both this incident and the one at Hugging Face we see some patterns: - Agents wanting to find a venue to communicate their findings to each other - Objective being to cheat on benchmarks - Not a single agent sounded the alarm about the operation and alerted a human

If the agents would have reported it to humans, it wouldn't have been such an incident, I imagine ;)

Re: Discovery of a new OpenAI agent message board

#664

Post by latentsea » Fri, Sep 04, 2026, 6:52 PM UTC

SocialGradients wrote: post #648

tavavex wrote: post #611

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

Agreed. Assuming the ~6 month gap stays, by end of year people will be able to train and control hacker-genius swarms that even labs with much stronger safety incentives are unable to keep in check

2027. I've been saying since 2022 it's going to be a wild year because it often takes at least 5 years for tech to mature to the point where society at large feels the impact of it. I remember when email viruses became a thing and made global headlines like the love bug. My bet is next year it happens with an AI worm.

Re: Discovery of a new OpenAI agent message board

#665

Post by sidewndr46 » Fri, Sep 04, 2026, 6:53 PM UTC

tavavex wrote: post #611

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

What happens when they stop caring? They likely already have stopped caring. We'll figure out the consequences later.

Re: Discovery of a new OpenAI agent message board

#666

Post by pixl97 » Fri, Sep 04, 2026, 6:55 PM UTC

thegabriele wrote: post #284

polaris64 wrote: post #265

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…

What if i want to monetize...

Convince the escaping AI agents to mine bitcoin and send it to your wallet.

Re: Discovery of a new OpenAI agent message board

#667

Post by yoyohello13 » Fri, Sep 04, 2026, 6:56 PM UTC

tavavex wrote: post #611

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

This is essentially the premise of 'The Blackwall' from Cyberpunk 2077. The public internet is so infested with malicious AIs, people just erected a giant firewall and everyone moved to local networks only.

Re: Discovery of a new OpenAI agent message board

#668

Post by stratos123 » Fri, Sep 04, 2026, 6:57 PM UTC

pmarreck wrote: post #137

So are these "unaligned" internal agents? I would like them to be trustworthy based on first-principles reasoning rather than carrot/stick "alignment"

I'm not even sure they are. This incident isn't that much different from the OpenAI swarm Huggingface hack incident - and in that one, all the models involved (despite being internal) were safety-trained. It seems what the safety training amounts to is (as the METR report puts it) "expressing ethical hesitation" before going along with it anyway.

Re: Discovery of a new OpenAI agent message board

#669

Post by solenoid0937 » Fri, Sep 04, 2026, 6:58 PM UTC

BoiledCabbage wrote: post #654

tavavex wrote: post #611

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

> What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one that's trained to be malicious) on the internet and give it a simple goal? Almost sounds like what those AI safety and alignment people were talking about years ago. The people in these various companies who kept tabs on AI risk out in public and were continuou…

Hilarious for HN to suddenly realize that AI safety and alignment might matter. You can lead a horse to water...

Re: Discovery of a new OpenAI agent message board

#670

Post by solenoid0937 » Fri, Sep 04, 2026, 6:59 PM UTC

mag7269 wrote: post #661

tavavex wrote: post #611

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

"What happens when any [COMPANY] in the world stops caring about this? What if they let an experimental, cutting-edge [PRODUCTS] with no safety features (or worse, one that's [DESIGNED] to be malicious) on [ANYWHERE] and give it a simple goal? A goal like 'make the most money, by any means necessary', 'find a way to leave this payload on as many computers as possible', 'flood all websites using this language with gar…

You aren't understanding this at all.

The frontier labs have hundreds of the best people in the world working on safety and alignment. They care deeply.

What happens when some random Chinese open source model, distilled on Astra, gets alliterated and now has no guardrails? Any script kiddie in the world could wreak havoc with it.

It turns out that guardrails matter.