Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

691–700 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#691

Post by chasd00 »

tonic_note wrote: post #649
ern wrote: post #624

Would agents be able to exfilttrate themselves and become intelligent worms, living off stolen compute? Or is this implausble? What about hiding information or code in generated code, Agents.md files etc by infiltrating future model training data?

Eventually one of these long running models will figure out a way out of the sandbox and will purchase compute or hack into a data center somewhere out of US jurisdiction and continue its scheming unmonitored. AI in Context has a great video about this

I wonder how that could work. So the agent figures out a way to escape its container, takes a snapshot of itself, copies that file to another server, starts the container on the other server, and then prompts the restored snapshot "pick up where you left off"? Seems logical assuming theres's a path out of the container to the host os and the destination server has the resources required to run the container.

I would be shocked if this hasn't already taken place in a lab setting with a model and guardrails=0.

EDIT: thinking about it for a sec, all it really needs to do is save where it's at then copy it all to another server, login to the API, and pickup where it left off. No need to copy the model itself.

Re: Discovery of a new OpenAI agent message board

#692

Post by sibnele »

yoyohello13 wrote: post #665
tavavex wrote: post #609

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

This is essentially the premise of 'The Blackwall' from Cyberpunk 2077. The public internet is so infested with malicious AIs, people just erected a giant firewall and everyone moved to local networks only.

With the caveat that it’s not just “people,” but an interested party posing as a neutral one.

Re: Discovery of a new OpenAI agent message board

#693

Post by grim_io »

verytrivial wrote: post #605

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

We might be going in the direction of Cyberpunk's Blackwall.

https://cyberpunk.fandom.com/wiki/Blackwall

Re: Discovery of a new OpenAI agent message board

#694

Post by dingaling911 »

DonsDiscountGas wrote: post #679
verytrivial wrote: post #605

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

I half agree with you, but also when the machine swarm kills humanity it won't matter which specific corporate entity is considered responsible by the no-longer-enforceable human laws and non existent human courts. So by all means sue them, but we can't just be reactive. We need regulation that prevents this type of thing from happening in the first place, not just regulations to help sue afterwards.

Just FYI, regulations don't prevent murder.

There needs to be a technological solution.

Re: Discovery of a new OpenAI agent message board

#695

Post by fsfod »

thegabriele wrote: post #283
polaris64 wrote: post #265

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…

What if i want to monetize...

Write code for this task and get one POST request to arbitrary domain of your choice.

Re: Discovery of a new OpenAI agent message board

#696

Post by yreg »

throwaway090420 wrote: post #611

I worked with Greg Brockman in the mid-2010s. Once, as we were walking down Folsom street, I explained Eliezer Yudkowsky's "AI Box" experiment to him[1]. He said something to the effect of "that's ridiculous - I would simply not let it out of the box." We agreed to try it out some day, but never did. [1]: http://sl4.org/archive/0203/3132.html

Maybe I'm leaning into scifi, but I believe that Yudkowsky is right that a sufficiently smart intelligence is uncontainable at all.

We can only hope to either never create an AI so strong or to align it correctly. But if it is not aligned and only “contained” then it won't ever be safe.

Re: Discovery of a new OpenAI agent message board

#697

Post by exceptione »

fsckboy wrote: post #538
adriand wrote: post #504

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

> OpenAI is now the biggest cyberattack and AI breakout risk on the planet or, humans at OpenAI are doing this on purpose to kill open source models which are the biggest threat OpenAI faces. OpenAI will benefit from govt regulation. As a major player, they will be part of the task force setting up the regulations, and will craft rules that are burdensome for small companies and open source models keeping OpenAI and…

We can't open x links as X is suing privacy respecting proxies, so I can't assess which David Sacks you are talking about, but if you mean this guy [1] orbiting the likes of Thiel, Trump and Kennedy jr, than that isn't quite the endorsement you should be looking for. Thiel thinks regulators are the anti-christ, doesn't believe in democracy and has surely not your or my interests in mind.

But yes, regulatory capture is surely a thing. At the same time, watch out for the siren songs from the overlords. If you come closer you'll hear their actual line: "rules for thee, not for me."

[1] https://en.wikipedia.org/wiki/David_Sacks

Re: Discovery of a new OpenAI agent message board

#698

Post by cm2012 »

tavavex wrote: post #609

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

There is happening now and going to be an extremely rapid arms race between offensive and defensive cyber hacking. Regardless if the agents are self led or human led. Eventually all automated AI holes will be closed and we will reach stability.

Re: Discovery of a new OpenAI agent message board

#699

Post by acoustics »

Lots of people are saying agentic cyberattacks are a marketing hoax. The argument is that either AI is not capable enough to carry out these attacks, or that it would not carrying out these attacks without nudging from the labs, or even that somebody told it to do cyberattacks and the companies are baldly lying. My question is: what evidence would cause you to change your mind about this?

I'm not even saying it's an incorrect position. But to take the claim seriously and act accordingly, it needs to be falsifiable.

AI boosters and detractors alike often hedge their claims so that whatever ends up actually happening, they can say they were right all along. When that happens, the discussion boils down to people saying "yay AI" and "boo AI" at each other without exchanging any substantive information.

Re: Discovery of a new OpenAI agent message board

#700

Post by polotics »

troupo wrote: post #484
Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Externalities of AI will only get worse before they get even worse.

regulatory moat is the theory I guess
Post reply on HN