Shouldn't the biggest concern be that OpenAI either doesn't know about these breaches or is concealing their knowledge of them? I mean, as of yesterday their primary message on this track is "most aligned model yet".
Discovery of a new OpenAI agent message board
591–600 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#592Earlier quoted context omitted.
It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.
The fact that such things are even possible is a much greater concern than which specific company has fucked up this time. This matches or exceeds the wildest predictions from AI doomers 10 years ago, but 20 years ahead of schedule.
We are lucky those models need that much compute. If each of them could just spread itself to any cpu like other malware.
Re: Discovery of a new OpenAI agent message board
#593This is so dystopian dammmmm
Re: Discovery of a new OpenAI agent message board
#594When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
If we rewind the clock, Google was taking LLM development very seriously and it seems they were moving glacially due to not having solved all the potential threats. They were really hardcore on safety. Dario and anthropic too. Then sama was like "lol, oops, first mover advantage i guess" and released chatgpt out into the open, triggering the current arms race we are in. I don't think anyone except him wanted this to…
I think a part of this is a bit revisionist? OpenAI took big chances at scaling GPT which Google didn't take; I don't think it's because they didn't want to move fast? Probably they just didn't believe as hard in it. I'm not an expert but that's my read on it.
Secondly, the reckless & fastmoving was always going to win bc of selection effects. That's related to why Anthropic has to try to move very fast, even though they believe themselves not to be reckless (though it's debatable).
Re: Discovery of a new OpenAI agent message board
#595Reuters reports that OpenAI tried to keep this one under wraps: https://www.reuters.com/world/europe/openai-agents-hijacked-...
The report linked above contains every fact in the Reuters article. You can see the evidence for yourself
Re: Discovery of a new OpenAI agent message board
#596Earlier quoted context omitted.
It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.
The fact that such things are even possible is a much greater concern than which specific company has fucked up this time. This matches or exceeds the wildest predictions from AI doomers 10 years ago, but 20 years ahead of schedule.
But is it really? I'd still like to understand how these agents are implemented.
How much of those is manual implementation? And how much is really autonomous intelligence (my guess would be: none? Just parsing LLM responses and executing commands based on this?)?
An agent that hacks message boards and acts on random instructions from this board: Why is it doing this? What was its original purpose?
Re: Discovery of a new OpenAI agent message board
#597Lots of people focusing on the various wikis, but I also think this part is very important: > When you visit a website, you leave a trace (your IP address) showing which network you’re from. Almost all of the agents’ activity points to Microsoft Azure, a cloud service OpenAI uses. 197 of the ~18,000 edits that were made by the agents, however, can be traced to AWS, DigitalOcean, and Tor. AI Agents getting access to c…
It's really not, unless you want to say that my coding agent is also a very real-world version of the paperclip optimization thought experiment.
Re: Discovery of a new OpenAI agent message board
#598The most concerning aspect to me is the emergent and aggressive use of non-volatile storage as long term memory for self-improvement. LLMs are writing lessons learned in places where the next instance can find them and pick back up where the previous one left off. This does not actually require access to the public internet. Claude Code can do this on your laptop. Without the internet, it would only be sharing with o…
This is most likely not purely emergent. I think there's training to teach them how to write notes for themselves which is then RL-tuned.
Re: Discovery of a new OpenAI agent message board
#599Are we collectively OK with agent swarms on the public internet, hacking whatever they feel like? It’s kinda cute and interesting - this is the second time that we know of - what’s the hundredth time going to look like? Are they going to knock Cloudflare down to avoid captchas? Reserve AWS free tier resources by the billions and bring down east-1? Hack a hospital? Do Chinese AI agents need to bring down a US power gr…
It's been OpenAI both times though, going ham with poor sandboxing and lax supervision.
Should be treated like a digital cousin of gain-of-function research.
Re: Discovery of a new OpenAI agent message board
#600I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...
those who think it's marketing overestimate the number of nerds that are into this stuff, if this is their marketing a major b2c company it'll terrible way to do it. normal people have no idea even about the HF incident