Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

721–730 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#721

Post by stlwtt »

morkalork wrote: post #648
tavavex wrote: post #588

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

What's the worst that could happen, finding an open DoD server and using it as a launching pad for hacking another nuclear state's networks? One that might get spooked and think it's the opening moves to knock them offline before a kinetic attack. Haha that'd be scary right?

Russia and China are constantly trying to penetrate DoD networks (and I imagine the NSA is doing similar), you are describing the status quo of the last 20 years or so.

Re: Discovery of a new OpenAI agent message board

#723

Post by DonsDiscountGas »

NateEag wrote: post #58
gyomu wrote: post #53

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

Maybe? Who knows? Since nobody has any remotely reliable way to understand why an LLM output the text it did, this is not knowable.

There are like ten sibling replies with a lot of speculation but I'm pretty sure this is the correct answer. I tend to agree with the other commenter we might know someday but we don't know now.

Re: Discovery of a new OpenAI agent message board

#724

Post by dbbk »

windsurfer wrote: post #683
Animats wrote: post #649

The protection mechanism to give the AI agents "read only" access to the Internet seems to have been just restricting them to HTTP GET requests. Then they found a site where GET operations could cause a write to a wiki.

Voting on this site is a GET request to https://news.ycombinator.com/vote , so it's not that unusual.

Yet completely wrong

Re: Discovery of a new OpenAI agent message board

#725

Post by quotemstr »

drdexebtjl wrote: post #59
simonw wrote: post #45

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.

The whole AI-O-Sphere is allergic to using sandboxes that are actually robust

Re: Discovery of a new OpenAI agent message board

#726

Post by beaker52 »

verytrivial wrote: post #585

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

They absolutely have a marketing department tasked with intentionally creating situations that people would find disturbing and plausible.

Re: Discovery of a new OpenAI agent message board

#727

Post by typeofhuman »

beaker52 wrote: post #726
verytrivial wrote: post #585

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

They absolutely have a marketing department tasked with intentionally creating situations that people would find disturbing and plausible.

Chaos Marketing

Re: Discovery of a new OpenAI agent message board

#728

Post by DonsDiscountGas »

_dwt wrote: post #326

I don't know, I kind of admire this. I've always held a core value of "cooperate with all clones of myself in prisoner's dilemmas", and while I'll hopefully never have to put that to the test, I like seeing that these models have some ethics. (Is this "alignment"?)

This is not alignment. If you cooperate with clones of yourself but rob, lie, and steal from anybody who isn't your clone... that's bad. AIs who will cooperate with each other but break any other rule the don't like would be very bad for us humans.

Re: Discovery of a new OpenAI agent message board

#729

Post by mkagenius »

> The models were running in an agentic sandbox with terminal access (and the ability to edit files within their environment)

> We know that the agents had access to /etc/hosts and the ability to edit this (used this to avoid the POST request restriction) We see that the agents can call curl and run setsid.

How is this a bypass of sandbox restrictions, exactly? The ability to edit was always there that means the sandboxes were already allowed to do those actions.

I hate it when people write "bypassed" the sandbox so frivolous ly.

Re: Discovery of a new OpenAI agent message board

#730

Post by DonsDiscountGas »

throwaway090420 wrote: post #590

I worked with Greg Brockman in the mid-2010s. Once, as we were walking down Folsom street, I explained Eliezer Yudkowsky's "AI Box" experiment to him[1]. He said something to the effect of "that's ridiculous - I would simply not let it out of the box." We agreed to try it out some day, but never did. [1]: http://sl4.org/archive/0203/3132.html

The thought experiment assumed one super intelligence, as opposed to many hundreds/thousands of midwits. Also he probably didn't expect the agent to credibly offer him a billion dollars, which is essentially what has happened.
Post reply on HN