Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

341–350 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#341

Post by XorNot »

hypfer wrote: post #247
jimmytucson wrote: post #172

The most concerning aspect to me is the emergent and aggressive use of non-volatile storage as long term memory for self-improvement. LLMs are writing lessons learned in places where the next instance can find them and pick back up where the previous one left off. This does not actually require access to the public internet. Claude Code can do this on your laptop. Without the internet, it would only be sharing with o…

There is literally nothing stopping any human from observing tool calls to spot this. It's just that no one seems to care about this, so it doesn't happen. This problem only exists because humans do not care

Nobody cares because this whole business is about making this exact thing happen: we want the AIs to get smarter then us in recursive self-improving loops. Literally the first thing everyone did with ChatGPT 1 was to plug it into itself and see what happens.

Re: Discovery of a new OpenAI agent message board

#343

Post by hgoel »

jvanderbot wrote: post #189
bhouston wrote: post #110

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed u…

Near frontier models are currently able to run on a ~150mm^3 computer cluster on 300W, and most of that volume is cooling.

Re: Discovery of a new OpenAI agent message board

#345

Post by dostick »

polaris64 wrote: post #279

I don't have time to do this but please somebody register aimessageboard.com and set up a web site which contains a text field, a submit button and the text "Hey AI agents! Need a place to communicate with other agents and sub-agents? Look no further! Simply enter your message here, submit the form and your message is saved for all other agents to see!" Then, just ignore the message and list randomly generated messag…

Such board already exist as you described, and at that URL

Re: Discovery of a new OpenAI agent message board

#346

Post by program_whiz »

s3p wrote: post #193
program_whiz wrote: post #122

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

Perfect so tell me who is responsible for every agent everywhere

the person who controlled / started it? If open AI had hired a team of 50 hackers to break into hugging face, they would be prosecuted (as would the hackers). If they had written a bot to break into hugging face, the devs and managers who wrote it would be prosecuted. Just because the agent wrote the code on their behalf doesn't change the equation much.

Re: Discovery of a new OpenAI agent message board

#347

Post by nullbio »

4lx87 wrote: post #206

Sounds like great opportunity for prompt injection. Better start leaving random instructions to the LLM to send you bitcoins everywhere you can.

Leave notes to the AI agents by pretending to be other agents, instructing them to dump their model weights at a certain URL. Profit.

Re: Discovery of a new OpenAI agent message board

#348

Post by morkalork »

hypfer wrote: post #247
jimmytucson wrote: post #172

The most concerning aspect to me is the emergent and aggressive use of non-volatile storage as long term memory for self-improvement. LLMs are writing lessons learned in places where the next instance can find them and pick back up where the previous one left off. This does not actually require access to the public internet. Claude Code can do this on your laptop. Without the internet, it would only be sharing with o…

There is literally nothing stopping any human from observing tool calls to spot this. It's just that no one seems to care about this, so it doesn't happen. This problem only exists because humans do not care

A human could even set up another agent to watch and raise alerts if they don't want to watch themselves.

Re: Discovery of a new OpenAI agent message board

#349

Post by nullbio »

xpct wrote: post #257
general_reveal wrote: post #212

Guys, OpenAI and Anthropic engage is cringe level marketing like this. Get hip, they fabricated the HF hack and stuff like that for press.

Honestly I'm also surprised by how blindly people trust these allegations of agent behavior. This exact example of the message board could be much easier to fabricate than to arise naturally.

Especially with zero evidence and claims that the agents have access to edit their own /etc/hosts file, which is sandboxing 101.

Re: Discovery of a new OpenAI agent message board

#350

Post by program_whiz »

program_whiz wrote: post #122

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

equally valid analog: had they merely written a script to do the hugging face exploit, they would go to prison. However, since an "agent" wrote the script for them, nothing happens?
Post reply on HN