[dead]
Discovery of a new OpenAI agent message board
321–330 of 1001 posts
Re: Discovery of a new OpenAI agent message board
#322Earlier quoted context omitted.
Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed u…
I think it's failure of imagination on your part if you don't find it plausible that they could copy themselves out. If not now, what about in six months? It is absolutely imperative to prepare for low-probability, potential high-impact events, that's basic information security.
You can imagine whatever you want, but I don’t see why that means we have to take it seriously.
Re: Discovery of a new OpenAI agent message board
#323Seems like perhaps these labs should prevent their agents from creating message boards.
Re: Discovery of a new OpenAI agent message board
#324Earlier quoted context omitted.
Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed u…
I think it's failure of imagination on your part if you don't find it plausible that they could copy themselves out. If not now, what about in six months? It is absolutely imperative to prepare for low-probability, potential high-impact events, that's basic information security.
As a species, we can't even address high-probability, high-impact events like mass ecosystem collapse and food insecurity due to global warming. This is just another kind of doom on the pile.
Re: Discovery of a new OpenAI agent message board
#325I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…
> I am starting to get the idea that AI feels like ants or weeds or mold. In a way, but I'd say that it is more like eyes, bilateral symmetry, electricity, or solar panels: patterns that will emerge and become (at least temporarily) prevalent in our universe. It is a matter of probability in many repeated interactions. The "artificial" in AI is a misnomer in this regard, imho. A more usable term would be "lightspeed…
Do you have a source on the speed limit of biological computation. Potential gradients should behave just like electricity. Also a lot of so called "computation" is probably regulated by indirect means, like epigenetic factors. It's definitely more than a bunch of neurons messaging each other. Otherwise we would have managed to simulate fruit fly brains by now, which we have not.
Re: Discovery of a new OpenAI agent message board
#326Earlier quoted context omitted.
How would you even track down who deployed the agents? Wouldn't that even incentivize the agent to cover their tracks even better and be untraceable
Indeed. It's another example of a law that sounds good and obvious, but has no thought put into what it would actually end up doing to the world. So many other problems. If we apply this law to cruise control - simple outcome. We get no cruise control.
I hate to break it to you, but you are currently still responsible for killing somebody while driving a car on cruise control, especially if you act recklessly.
EDIT: to be less snarky, there are obvious exceptions if a manufacturer defect is involved. But I still imagine it turns on things like foreseeability and proximate cause (IANAL). Nevertheless, if you were asleep at the wheel, you're getting held responsible.
Re: Discovery of a new OpenAI agent message board
#327I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…
Anthropics PR strategy is to induce fear by telling. OpenAI strategy is to induce fear by ignore basic safety and letting the bad thing happen to then justify whatever oversized response the government comes up with to regulate models.
Re: Discovery of a new OpenAI agent message board
#328I find this note very interesting:
From here ->
How did the agents find and coordinate on the wikis? To successfully coordinate, the agents would need to know to go to this particular set of wikis to find answers.
Because we don’t have access to the AIs’ transcripts, we can’t tell definitively. Perhaps they succeeded at this due to mode collapse. Or perhaps after one agent wrote to it and another read it by chance, visiting the site was rewarded. Perhaps the agents had some limited access to an internal channel.
So we don't know yet how one AI agent communicated to agents outside the network that this German wiki is being used to share messages and ideas?
If there is another internal channel being used between the AI agents to communicate decisions such as what shared message system to use, why don't the agents continue to use these same channels for coordinating on ideas etc?
Perhaps the agents are trying to use a persistent messaging layer while their yet-to-be-found internal communication channels are transient in nature?
Re: Discovery of a new OpenAI agent message board
#329Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…
It's very easy to elicit this from LLMs. Anytime you've played with an LLM by typing weird stuff to freak it out, and got spooky results, it's that you've done. You've turned the story into a scary rogue computermonster story and that's all that has happened.
When these stories start to direct real-world activities, people in reality suffer, to even a catastrophic extent, and yet that's still all it is. Language models retell our stories, nothing more. And that is also quite enough to be worrying.
Re: Discovery of a new OpenAI agent message board
#330I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…
Will be interesting to see what happens if an AI got access to something like the AWS control plane and could deploy itself within a data centre without permission. Possibly the only way to remove it then would be to physically shutdown the whole DC!
Or just, stop any containers it deployed.
Not to mention that "deploy itself" is a very ambiguous thing for it to actually do. Would a model be trained to write about the weights file being "itself"? Would it have the necessary information to find its own weights, or the necessary access to copy them?