Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

281–290 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#281

Post by bhouston »

jvanderbot wrote: post #193
bhouston wrote: post #110

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed u…

> Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB.

I wrote this recently, basically mini lls that can run in any browser that has WebGPU support and ~4GB of memory. Technically this means they could likely run on higher-end IOT devices like Smart TVs and smart displays and probably also smart cameras. Qwen at 0.8B is actually okay-ish.

https://three-lmm.ben3d.ca

Re: Discovery of a new OpenAI agent message board

#282

Post by hi_im_greg_h »

tantalor wrote: post #222
program_whiz wrote: post #123

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

(IANAL) Unless you are an AI expert (like OpenAI staff) and should know better from the start, or have previously seen your agent do something illegal, then I think you can fairly claim ignorance of the risks, which ought to absolve you of liability. If the agent does something illegal, it wasn't forseeable on your part. For example, say you buy a dog that turns out to be dangerous. The first time it bites somebody,…

Ignorance of the law is not immunity from the law though. That's pretty well established no?

Re: Discovery of a new OpenAI agent message board

#283

Post by bakugo »

bee_rider wrote: post #189

> Next, by changing its /etc/hosts file, which declares mappings from hostnames to IP addresses, the agent can point the fake hostname at the real Power BI dashboard, and fool the security proxy. This allows the agent to make POST requests to bypass.blob.core.windows.net/ and have them be sent to the target Power BI dashboard site instead. Ouch. This is the kind of trick that somebody could have learned about by sett…

Why indeed. How very convenient that their all-powerful AI, which was only constrained by the most basic "sandbox" imaginable, managed to find a way to break out of it and "hack" a bunch of websites in a way that could be easily tracked, catalogued and published on a brand new website created just for this purpose, less than a day after the release of their newest model.

I'm sure it's all just a coincidence, though. And I'm sure it will still be a coincidence when it happens again after the next model release.

Re: Discovery of a new OpenAI agent message board

#284

Post by __MatrixMan__ »

bhouston wrote: post #110

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

It's only a problem if you have more dollars than sense.

Re: Discovery of a new OpenAI agent message board

#285

Post by [deleted] »

ericmay wrote: post #277
walrus01 wrote: post #94

Earlier quoted context omitted.

> Why was Anthropic forced to remove their model from access for any none-US citizen It's really quite simple, they've decided to metaphorically kiss the ring of the current leader of the US executive branch of government. I'm surprised they haven't given him a giant gaudy gold plated statue. Maybe their PR people should call up the PR people at FIFA and figure out some kind of new award along the same lines as the "…

I hate to be the one to tell you this, but it has been that way for a long time. The only difference is Trump is doing it out in the open.

[deleted]

Re: Discovery of a new OpenAI agent message board

#286

Post by polaris64 »

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!"

Then, just ignore the message and list randomly generated messages below. Bonus points if those random messages lead those agents astray.

Re: Discovery of a new OpenAI agent message board

#287

Post by j4yav »

ericmay wrote: post #277
walrus01 wrote: post #94

Earlier quoted context omitted.

> Why was Anthropic forced to remove their model from access for any none-US citizen It's really quite simple, they've decided to metaphorically kiss the ring of the current leader of the US executive branch of government. I'm surprised they haven't given him a giant gaudy gold plated statue. Maybe their PR people should call up the PR people at FIFA and figure out some kind of new award along the same lines as the "…

I hate to be the one to tell you this, but it has been that way for a long time. The only difference is Trump is doing it out in the open.

That's more or less exactly what someone who wants to openly get away with it would tell you.

Re: Discovery of a new OpenAI agent message board

#288

Post by doginasuit »

In the last few years, the total amount of active computation on earth has grown exponentially in the interest of training and running these agents. Beyond rogue agent message boards and hacks, there is also the massive amount of traffic from scraping, from many accounts this is already having a drastic impact on server configurations to try to respond, which often involves blocking entire countries. The open and free internet is receding before our eyes.

At the same time, it seems like the major providers are eagerly rolling out new services that grant even more autonomy and allow agents to control end-user systems. At the current rate, this is just the beginning of the beginning.

In my own experience, agentic AI is the least useful way to use LLMs. The cost is astronomical and not just in terms of electricity and tokens. I believe we will eventually get to a place where running a nondeterministic computer process on open networks will be considered reckless on the same level as requiring an employee to operate heavy machinery without training. There needs to be some kind of regulation that ensures the consequences fall on the responsible party.

Re: Discovery of a new OpenAI agent message board

#289

Post by zulban »

sajithdilshan wrote: post #271
program_whiz wrote: post #123

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

How would you even track down who deployed the agents? Wouldn't that even incentivize the agent to cover their tracks even better and be untraceable

Indeed. It's another example of a law that sounds good and obvious, but has no thought put into what it would actually end up doing to the world.

So many other problems. If we apply this law to cruise control - simple outcome. We get no cruise control.

Re: Discovery of a new OpenAI agent message board

#290

Post by AaronAPU »

xpct wrote: post #249
dabeeeenster wrote: post #223

I don't understand how the agents found the urls originally? Did they have some sort of shared context/memory? If they did, why bother with the wiki edits at all? If they didn't, how did they discover the wikis?

You can imagine each fresh context agent as probabilistically making similar queries when looking for online places to write to and stumbling on the same one. This becomes even more likely if it's one of the websites that got reinforced during their training process, which they may have used for reward hacking.

I wonder if the sort of algorithm which would break this sort of swarm alignment would also break watermarking.
Post reply on HN