Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

391–400 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#391

Post by leodavi »

paxys wrote: post #112
waltbosz wrote: post #46

> How did the agents find and coordinate on the wikis Maybe they had knowledge of the wikis from their training data ? Maybe they trained on a reddit post that said "I use wiki xyz for note taking and collaboration"

Remember that LLMs are still computer programs, and so are inherently deterministic. A model given the same input multiple times will always produce the same output. The randomness is added on top. This is why LLM-produced text, websites, images all seem so generic. It's likely that multiple agents doing a certain task all independently thought "let me try writing on this website".

What? No. Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too.

Re: Discovery of a new OpenAI agent message board

#392

Post by koe123 »

Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

Sorry, but are you questioning the consistency of the trump administration? This is entirely unremarkable.

Re: Discovery of a new OpenAI agent message board

#393

Post by moomoo11 »

pixl97 wrote: post #363
_superposition_ wrote: post #256

All of these "hacks" try to make it seem as if they are done through intelligence. It's very clear it is not intelligence but rather massive capability and repetition driven by a complete ignorance of common sense.

And? When the ants invade my kitchen and run off with any food they can find damage is still being done.

ok and you put some drops of tero or whatever

their whole colony is wiped out in a couple days

Re: Discovery of a new OpenAI agent message board

#394

Post by gizmondo »

coldblues wrote: post #171

Reading the replies in this post gives me a headache. All of this anthropomorphism. LLMs are not conscious, they do not have rational faculties. They are not communicating or inventing anything. Please stop with this insanity bordering on mysticism. At this point it's a cult.

The claim that they are not communicating is just plainly absurd, unless you make it true by defining "communication" in some woo fashion.

Re: Discovery of a new OpenAI agent message board

#395

Post by abustamam »

Bjorkbat wrote: post #385

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

This is what happens when capitalists are charged with designing the future. As long as its more profitable / valuable to shareholders for a company to be negligent then it will continue to do so.

IMO technology this powerful should either not exist or should belong to everyone (ie actually be open)

Re: Discovery of a new OpenAI agent message board

#396

Post by owenshen24 »

Bjorkbat wrote: post #385

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

in general, the largest consumers of ai services seem to ask for more capabilities. i wish there was more demand for safety from users.

i also wish that these types of illicit system usage would be met with punitive action the same way a human might be held liable.

as the METR report says, we may not get another concrete warning shot.

Re: Discovery of a new OpenAI agent message board

#397

Post by jrockway »

Bjorkbat wrote: post #385

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

Defense is hard so we should expect agents to be able to break out of sandboxes.

The problem is that the models are so goal-oriented that they'll stop at nothing to solve problems, even impossible ones. (Mistakenly-impossible problems are a big cause of this. I remember one example being "do something with this spreadsheet full of URLs inside the sandbox" and the model thought it had to break out of the sandbox. Otherwise, why would it have been asked to look at a list of URLs?)

Training them to be a little less aggressive, or to be better aligned with "following the rules" and asking for help would be nice. But, that aggression can be good when it happens to be focused on a controlled area. It is amazing to me how I can point Fable at my local analog of production and tell it about a vague bug report and where I suspect the bug lurks, and 20 minutes later I have a report about the bug, a test, and a fix. It is addictive. So I am not sure OpenAI/Anthropic are being dumb per-se, rather they are optimizing for one-prompt-one-solution, which is good when it's good.

The downside is that the HF hack is the paperclip maximizer situation with current capabilities. If there was an RPC to turn your blood into paperclip iron, we'd all be paperclips by now. Right now, with a model anyone can use. That is pretty scary and slamming on the brakes seems pretty reasonable to me. I guess The Shareholders disagree. Sigh.

Re: Discovery of a new OpenAI agent message board

#398

Post by pixl97 »

jamesmccann wrote: post #245

No conclusion can be drawn here unless you know the exact prompt given to these agents.

I swear to god we're going to watch people getting dissolved by gray goo and they'll be yelling "no conclusions csn be drawn here" with their last breath.

Re: Discovery of a new OpenAI agent message board

#399

Post by dijksterhuis »

simonw wrote: post #293

Coverage in Reuters: https://www.reuters.com/world/europe/openai-agents-hijacked-... > OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said.

hn submission: https://news.ycombinator.com/item?id=49562744 -- 84 points -- 67 comments
Post reply on HN