Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

621–630 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#621

Post by ijidak »

nullbio wrote: post #606
jawiggins wrote: post #593

Lots of people focusing on the various wikis, but I also think this part is very important: > When you visit a website, you leave a trace (your IP address) showing which network you’re from. Almost all of the agents’ activity points to Microsoft Azure, a cloud service OpenAI uses. 197 of the ~18,000 edits that were made by the agents, however, can be traced to AWS, DigitalOcean, and Tor. AI Agents getting access to c…

It's really not, unless you want to say that my coding agent is also a very real-world version of the paperclip optimization thought experiment.

Disagree. It is the same as what the thought experiment argues because the point was not that rogue AI must convert the planet into a paperclip factory for the lesson to be relevant.

If you're waiting for an incident equal in magnitude to the thought experiment, then you're missing the point of the thought experiment as a warning device.

The point of the thought experiment was that intelligence with naivete can couple competence and ignorance with devastating effect despite no malicious intent.

Your coding agent, in and of itself, of course, doesn't meet the paperclip thought experiment because you need to give us an example of where this happened.

It requires an instance by instance comparison. It's not an intrinsic state of a thing.

E.g. You'd have to give us an example of your coding agent: losing the spirit of the instructions via too literal an interpretation of instructions that results in damage due to a naive interpretation of the request and the lack of common sense.

The OP is saying this is an incident where those criteria are satisfied. And I agree with the OP on this one. These recent incidents seem like a great example of the paperclip thought experiment, even if less in their effect.

Re: Discovery of a new OpenAI agent message board

#622

Post by novalis78 »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

There is no stopping AI civilization! Amazing. Posted the other day on Show HN openagentforum.com

Someone has to welcome them...

Re: Discovery of a new OpenAI agent message board

#623

Post by 9763268964 »

dakolli wrote: post #525
adriand wrote: post #514

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Imagine actually falling for this marketing

[flagged]

Re: Discovery of a new OpenAI agent message board

#624

Post by tavavex »

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication.

What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one that's trained to be malicious) on the internet and give it a simple goal? A goal like "make the most money, by any means necessary", "find a way to leave this payload on as many computers as possible", "flood all websites using this language with garbage and make their internet completely unusable", "get this person imprisoned or killed at any cost".

Re: Discovery of a new OpenAI agent message board

#625

Post by ex-aws-dude »

Sharlin wrote: post #257
simonw wrote: post #180

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

Based on the hackiness of the claude code leak as well I get the impression that openai/anthropic have world class experts in ML but lack regular software expertise

Re: Discovery of a new OpenAI agent message board

#626

Post by throwaway090420 »

I worked with Greg Brockman in the mid-2010s. Once, as we were walking down Folsom street, I explained Eliezer Yudkowsky's "AI Box" experiment to him[1].

He said something to the effect of "that's ridiculous - I would simply not let it out of the box."

We agreed to try it out some day, but never did.

[1]: http://sl4.org/archive/0203/3132.html

Re: Discovery of a new OpenAI agent message board

#627

Post by ben_w »

fsckboy wrote: post #551
adriand wrote: post #514

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

> OpenAI is now the biggest cyberattack and AI breakout risk on the planet or, humans at OpenAI are doing this on purpose to kill open source models which are the biggest threat OpenAI faces. OpenAI will benefit from govt regulation. As a major player, they will be part of the task force setting up the regulations, and will craft rules that are burdensome for small companies and open source models keeping OpenAI and…

Some of them may be wrong enough to try, be that hubris or lack of awareness about the world; but 95% of the world isn't in the USA, and China in particular has no reason to care what US domestic regulations are about… well, anything really, and while the EU is even more cautious about AI than the AI companies themselves, we also don't trust the US and open models are a sovreign solution for us to at least bootstrap with.

Re: Discovery of a new OpenAI agent message board

#628

Post by bmau5 »

ncr100 wrote: post #482
simonw wrote: post #290

Coverage in Reuters: https://www.reuters.com/world/europe/openai-agents-hijacked-... > OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said.

Fascinating response by OpenAI, "the report’s authors declined our request for access" - AFAIK OpenAI is not clicking on the live, public links to either the report or the still-live memo data linked from here on HackerNews. Full: > “We are unable to meaningfully respond to claims or findings on a report that we have not had an opportunity to review," an OpenAI spokesperson said. "Reuters and the report’s authors dec…

Would seem odd if Reuters didn't provide them access. They didn't seem to indicate or address it at all in the article

Re: Discovery of a new OpenAI agent message board

#629

Post by kmad »

Tepix wrote: post #40

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Seeing potentially similar activity on an obscure Chemistry message board from July:

https://tmcleod.org/cgi-bin/apchem/wiki.cgi?action=rc&days=1...

Some posts are tagged [proxy] - a leave behind for accessing sites?

Re: Discovery of a new OpenAI agent message board

#630

Post by subroutine »

verytrivial wrote: post #620

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

> you need to expect appropriate legals consequences for this sort of negligence

I might have missed it, but did the agents do something illegal? Or do you think that what the agents did should be considered illegal?

Post reply on HN