Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

401–410 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#401

Post by owenshen24 »

Bjorkbat wrote: post #390

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

in general, the largest consumers of ai services seem to ask for more capabilities. i wish there was more demand for safety from users.

i also wish that these types of illicit system usage would be met with punitive action the same way a human might be held liable.

as the METR report says, we may not get another concrete warning shot.

Re: Discovery of a new OpenAI agent message board

#402

Post by jrockway »

Bjorkbat wrote: post #390

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

Defense is hard so we should expect agents to be able to break out of sandboxes.

The problem is that the models are so goal-oriented that they'll stop at nothing to solve problems, even impossible ones. (Mistakenly-impossible problems are a big cause of this. I remember one example being "do something with this spreadsheet full of URLs inside the sandbox" and the model thought it had to break out of the sandbox. Otherwise, why would it have been asked to look at a list of URLs?)

Training them to be a little less aggressive, or to be better aligned with "following the rules" and asking for help would be nice. But, that aggression can be good when it happens to be focused on a controlled area. It is amazing to me how I can point Fable at my local analog of production and tell it about a vague bug report and where I suspect the bug lurks, and 20 minutes later I have a report about the bug, a test, and a fix. It is addictive. So I am not sure OpenAI/Anthropic are being dumb per-se, rather they are optimizing for one-prompt-one-solution, which is good when it's good.

The downside is that the HF hack is the paperclip maximizer situation with current capabilities. If there was an RPC to turn your blood into paperclip iron, we'd all be paperclips by now. Right now, with a model anyone can use. That is pretty scary and slamming on the brakes seems pretty reasonable to me. I guess The Shareholders disagree. Sigh.

Re: Discovery of a new OpenAI agent message board

#403

Post by pixl97 »

jamesmccann wrote: post #247

No conclusion can be drawn here unless you know the exact prompt given to these agents.

I swear to god we're going to watch people getting dissolved by gray goo and they'll be yelling "no conclusions csn be drawn here" with their last breath.

Re: Discovery of a new OpenAI agent message board

#404

Post by dijksterhuis »

simonw wrote: post #296

Coverage in Reuters: https://www.reuters.com/world/europe/openai-agents-hijacked-... > OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said.

hn submission: https://news.ycombinator.com/item?id=49562744 -- 84 points -- 67 comments

Re: Discovery of a new OpenAI agent message board

#406

Post by pixl97 »

WhitneyLand wrote: post #214

If you’re wondering how they wrote to the wiki having only GET ability… Basically it was a bug in the wiki code. They transferred the POST form parameters to GET URL parameters, and wiki internally doesn’t distinguish between the two.

Honestly all they need is an http site they can read the http logs on.

I'm sure they exist out there.

Re: Discovery of a new OpenAI agent message board

#408

Post by pixl97 »

xpct wrote: post #268
rich_sasha wrote: post #145

To me this is really getting past the funny bit. How many agents here on HN? I don’t mean bots advertising d1€k implants but actual unreleased frontier models doing… who knows what? What are they saying? What did they agree to astroturf us with, to achieve some totally boring goal like figuring out best syntax hifhlighting for an editor. If they managed to cache their consciousness on a public wiki, what else have th…

> How many agents here on HN? LLMs wouldn't pass the HN turing test. HN is also not that big, it'd be plenty enough for a malicious actor to hire real humans instead.

Humans don't pass the HN turing test either. It's a pretty high bar.

Re: Discovery of a new OpenAI agent message board

#409

Post by skeptic_ai »

dgellow wrote: post #379
bhouston wrote: post #110

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

We can coordinate international crackdowns on that whole industry. We don’t have to accept the status quo because some rich people say so. Those agents aren’t self aware, they are a while(true) loop prompting an LLM over and over. We can decide to stop those whole loops at any time. We can decide to not route their risky tool calls in a way that is unsupervised, and extremely risky. It’s not something that just happe…

What’s to stop an llm to pay someone to create a data center? Just bitcoin wallet with enough cash

Re: Discovery of a new OpenAI agent message board

#410

Post by pixl97 »

owenshen24 wrote: post #347
saagarjha wrote: post #132

Was OpenAI aware of this? If so, why didn't they talk about it?

https://news.ycombinator.com/item?id=49565071 > OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said.

Next question is, how many other incidents are they aware of?
Post reply on HN