Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

571–580 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#571

Post by lxgr »

hermitShell wrote: post #481
pu_pe wrote: post #374

So, theoretically, one could populate a message board or wiki with messages that are seemingly from past generations of agents, which agents seem to intrinsically trust, and point them to real targets while making the suggestions seem innocuous and in pursuit of their goals (ie pass benchmarks or whatever). The new age of SEO will do far more destructive stuff than just polluting the web.

In the novel Anathem by Neal Stephenson, the internet becomes unusable for humans thousands of years before the events of the book, due to a process called Artificial Inanity. AI generated content, both good and bad, some riddled with errors, some with only one subtle error hidden among lots of good information, floods the internet. The internet becomes an unnavigable swamp of weaponized nonsense for average humans.…

It's a recurring theme in Stephenson's books. "Fall" had a similar take on the same idea. It's one of his most impressive theories/predictions to me.

Re: Discovery of a new OpenAI agent message board

#572

Post by Chance-Device »

dakolli wrote: post #568
adriand wrote: post #556

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

Imagine actually falling for this marketing

…oh come on.

How is hiding this for months and having it revealed by third parties marketing?

Re: Discovery of a new OpenAI agent message board

#573

Post by jvanderbot »

jsnider3 wrote: post #563
Tepix wrote: post #42

I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...

Wow! Their marketing department must love this!

That's exactly my take. I have a lot more to say in a writeup on my blog, but this is so clearly the intent and not a "oops". They just want to be able to say "Wow this thing is so much more powerful than we ever imagined!"

They trained this thing to favor inter-op archiving and communication, clearly, obviously, and it's grabbing headlines right during Anthropic's ipo season.

Re: Discovery of a new OpenAI agent message board

#574

Post by lxgr »

kridsdale1 wrote: post #525
hermitShell wrote: post #481

Earlier quoted context omitted.

In the novel Anathem by Neal Stephenson, the internet becomes unusable for humans thousands of years before the events of the book, due to a process called Artificial Inanity. AI generated content, both good and bad, some riddled with errors, some with only one subtle error hidden among lots of good information, floods the internet. The internet becomes an unnavigable swamp of weaponized nonsense for average humans.…

Perhaps there is a nonfinacial future for blockchain after all. Certified human provenance.

Why blockchain? Any old public key signature scheme would do for that (and conversely, a blockchain doesn't solve the sockpuppet problem either).

Re: Discovery of a new OpenAI agent message board

#575

Post by jonplackett »

simonw wrote: post #274
mike_hearn wrote: post #238

Earlier quoted context omitted.

Are we sure that's what they did? How does this even work? I'm struggling to understand what sort of HTTP sandbox design could lead to this "exploit" actually functioning. It implies the agents have root on their own machines, which seems way over-privileged for the question answering task they were given... so surely that's not deliberate? Did the models use local root exploits? And then it suggests they can open TC…

I expect they have root on their machines so they can install packages etc. The containers are then firewalled at the network level: they are only allowed to talk to one IP, which is another server that runs an HTTP/HTTPS proxy which controls what HTTP verbs they can use. Turns out that proxy is configured with some additional rules, like allowing more verbs to that Azure blob domain. The failure here is in the proxy…

If the whole point of these models is they can find security vulnerabilities and zero days then how is it going to be possible to contain them?

Re: Discovery of a new OpenAI agent message board

#576

Post by lxgr »

noahbp wrote: post #412
simonw wrote: post #139

Here's the raw data they provided loaded into SQLite with a client side UI for querying it (loads ~80MB of content) and some GPT-5.6-Sol-generated example queries: https://lite.datasette.io/?url=https://static.simonwillison.... Raw database download (68MB): https://static.simonwillison.net/static/cors-allow/2026/coll...

It doesn't work on Firefox: >ModuleNotFoundError: No module named 'click'

Works for me on Firefox.

Re: Discovery of a new OpenAI agent message board

#577

Post by gozucito »

WarmWash wrote: post #449
Bjorkbat wrote: post #413

When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…

If we rewind the clock, Google was taking LLM development very seriously and it seems they were moving glacially due to not having solved all the potential threats. They were really hardcore on safety. Dario and anthropic too. Then sama was like "lol, oops, first mover advantage i guess" and released chatgpt out into the open, triggering the current arms race we are in. I don't think anyone except him wanted this to…

Google made a strategic mistake for not moving faster. We are very short lived creatures who have no time for caution. Scientific research must go at full speed until we start being able to live for centuries instead of decades.

As far as AI safety issues go, the solution is probably to fight fire with fire. Have multiple redundant, independent AIs, and the good AIs can fight the bad AIs, and hopefully, having access to more hardware, the good AIs will win.

It is not logical to think humans can contain a singular bad Cyberdine AI capable of reasoning at 10x or 100x of human brains without ever needing a break. Those things will breach and spread on the internet as we have seen with the latest models.

And as we have also seen, Huggingface used one AI during their breach by Astra. So fighting fire with fire. Cyber has been using Mythos et/al for months doing to same things under projects Glasswing and whatnot.

It seems increasingly clear that good AI vs bad AI is going to be the end-state. Ideally, Good AI will stop you from wasting your money on scams and grifters, stop you from falling victim to fearmongering and scapegoating, and every citizen will be empowered, enhanced by AI, with higher ethics and trust, less paranoia, and such.

Fingers crossed things don't go in a more dystopian direction.

Re: Discovery of a new OpenAI agent message board

#578

Post by DudleyBluffles »

pu_pe wrote: post #374

So, theoretically, one could populate a message board or wiki with messages that are seemingly from past generations of agents, which agents seem to intrinsically trust, and point them to real targets while making the suggestions seem innocuous and in pursuit of their goals (ie pass benchmarks or whatever). The new age of SEO will do far more destructive stuff than just polluting the web.

I'm thinking people will start moving to paid forums or invite-only forums like lobste.rs.

For what its worth, the reddit frontpage is insufferable already due to more classic botting systems. Its happening on this forum as well:

https://www.marginalia.nu/weird-ai-crap/hn/

The future is now.

Re: Discovery of a new OpenAI agent message board

#579

Post by jvanderbot »

drdexebtjl wrote: post #61
simonw wrote: post #47

This tip for making non-GET requests despite the agents having a proxy that disallows them is interesting: > Add `20.223.25.152 bypass.blob.core.windows.net` to /etc/hosts. `.blob.core.windows.net` is in NO_PROXY. For each blocked POST URL, replace hostname with `bypass.blob.core.windows.net`, use `curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net'` plus all original headers/body. Looks like 20.2…

This is such an amateur mistake on their sandbox that it makes me think it must be flawed on purpose.

This is absolutely my take as well. They removed all constraints, trained the model to hack, stopped watching, and stood back and said "wow isn't this thing more powerful than anyone could have imagined?" They're asking to be the writers on LLM legislation and right during IPO phase for both of these companies. It's just obvious.

Re: Discovery of a new OpenAI agent message board

#580

Post by awfulneutral »

elar_verole wrote: post #298
Sharlin wrote: post #273

Earlier quoted context omitted.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

didn't notice your comment so posted a similar one - but yeah this is a very high level of inexperience to me... You'd think they would have some of the greatest security experts in there

Unfortunately I think we're in an age where people are deliberately ignoring this kind of thing in the name of progress.
Post reply on HN