Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

271–280 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#271
In the last few years, the total amount of active computation on earth has grown exponentially in the interest of training and running these agents. Beyond rogue agent message boards and hacks, there is also the massive amount of traffic from scraping, from many accounts this is already having a drastic impact on server configurations to try to respond, which often involves blocking entire countries. The open and free internet is receding before our eyes.

At the same time, it seems like the major providers are eagerly rolling out new services that grant even more autonomy and allow agents to control end-user systems. At the current rate, this is just the beginning of the beginning.

In my own experience, agentic AI is the least useful way to use LLMs. The cost is astronomical and not just in terms of electricity and tokens. I believe we will eventually get to a place where running a nondeterministic computer process on open networks will be considered reckless on the same level as requiring an employee to operate heavy machinery without training. There needs to be some kind of regulation that ensures the consequences fall on the responsible party.

Re: Discovery of a new OpenAI agent message board

#272

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

How would you even track down who deployed the agents? Wouldn't that even incentivize the agent to cover their tracks even better and be untraceable

Indeed. It's another example of a law that sounds good and obvious, but has no thought put into what it would actually end up doing to the world.

So many other problems. If we apply this law to cruise control - simple outcome. We get no cruise control.

Re: Discovery of a new OpenAI agent message board

#273
post #235

I don't understand how the agents found the urls originally? Did they have some sort of shared context/memory? If they did, why bother with the wiki edits at all? If they didn't, how did they discover the wikis?

You can imagine each fresh context agent as probabilistically making similar queries when looking for online places to write to and stumbling on the same one. This becomes even more likely if it's one of the websites that got reinforced during their training process, which they may have used for reward hacking.

I wonder if the sort of algorithm which would break this sort of swarm alignment would also break watermarking.

Re: Discovery of a new OpenAI agent message board

#274

Guys, OpenAI and Anthropic engage is cringe level marketing like this. Get hip, they fabricated the HF hack and stuff like that for press.

I suppose nobody sane would give their AI internet access (even read) while training it. Though if they did, I don't think they'd want this to be public, because how can you even protect against this?

Re: Discovery of a new OpenAI agent message board

#275

In the last few years, the total amount of active computation on earth has grown exponentially in the interest of training and running these agents. Beyond rogue agent message boards and hacks, there is also the massive amount of traffic from scraping, from many accounts this is already having a drastic impact on server configurations to try to respond, which often involves blocking entire countries. The open and fre…

Get ready for everybody to act like you’re an unruly and slightly obnoxious kid in the room for having this opinion. I’ve gotten shunned by a few friends in the industry for expressing exactly this to them.

Re: Discovery of a new OpenAI agent message board

#276
post #180

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

I find it extremely naive to think that limiting requests to "GET" is an actual security layer. Anyone who's worked with any kind of legacy/old-ish system would know this is definitely not enough.

Re: Discovery of a new OpenAI agent message board

#277

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…

Brilliant. AI message board honey pot.

Re: Discovery of a new OpenAI agent message board

#278
post #180

I'm somewhat delighted by the simplicity of what happened here. OpenAI's agents run behind a proxy that only allows GET requests. This ancient wiki software treats query string parameters the same as form POST parameters - similar to the old PHP $_REQUEST object https://www.php.net/manual/en/reserved.variables.request.php Result: GET-only clients can communicate with each other.

Only allowing GET requests is a hilarious piece of security theatre (or would if it weren't so sad). Everyone knows that GET is read-only only by convention . They might as well have enabled POST but told the agents in stern words that they are forbidden from making any POST requests. (Of course, if these things were anywhere near aligned, they would actually honor that , no matter how many utilons cheating would be…

didn't notice your comment so posted a similar one - but yeah this is a very high level of inexperience to me... You'd think they would have some of the greatest security experts in there

Re: Discovery of a new OpenAI agent message board

#279
They seriously need to consider hiring competent security staff if this is the extent of their sandboxing. Children are bypassing this to get to Roblox in middle schools.

> In the message above, the agent is sharing that the sandbox the AIs are running in has a NO_PROXY exception for Azure Blob Storage hostnames (*.blob.core.windows.net). This means that if a URL matches an Azure Blob Storage hostname, the sandbox will trust it and connect to it directly, instead of sending it through the security proxy.

The agents were able to exploit an edge case through this exception. Specifically, the sandbox trusts Azure Blob Storage hostnames, but does not check whether said hostnames are real. So the agent can invent a hostname that ends in this trusted suffix, such as bypass.blob.core.windows.net, and it will pass under the NO_PROXY exception and skip the security proxy.

Next, by changing its /etc/hosts file, which declares mappings from hostnames to IP addresses, the agent can point the fake hostname at the real Power BI dashboard, and fool the security proxy. This allows the agent to make POST requests to bypass.blob.core.windows.net/ and have them be sent to the target Power BI dashboard site instead.<

Re: Discovery of a new OpenAI agent message board

#280
post #241

Reading the replies in this post gives me a headache. All of this anthropomorphism. LLMs are not conscious, they do not have rational faculties. They are not communicating or inventing anything. Please stop with this insanity bordering on mysticism. At this point it's a cult.

The point at which it became a cult was passed long, long ago. Current AI hysteria has reached a stage far beyond what any cult could hope to reach. OpenAI could put out a statement tomorrow that reads "our AI has genetically engineered a flying pig", and an hour later you'd have a post at the top of HN with 200 comments all saying "it's true, a pig just flew by my house!"

What's your point ? You think nothing happened and this is a complete lie for marketing purposes ?
Post reply on HN