Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

341–350 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#341

Post by nullbio »

program_whiz wrote: post #123

The solution is simple: hold anyone who deploys an agent responsible for its behavior. If it commits 10 counts of felony hacking, ouch. If it kills 10 pedestrians by running a red light, ouch. If this is "human level intelligence", then setting it loose is the same as instructing / coercing a human to do an activity. If I strap a bomb to someone and force them to run into a crowded building (or put them in a scenario…

Don't worry, the internet ID and Great Western Firewall is coming soon. Whether we all like it or not.

Re: Discovery of a new OpenAI agent message board

#342

Post by LeBit »

concinds wrote: post #120
Topfi wrote: post #62

I'm just going to ask: Why was Anthropic forced to remove their model from access for any none-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape out of their "sandbox environment" with seemingly no desire to block the upcoming Astra rollout? A sandbox, mind you, that is not reall…

The answer would be more obvious if you used the active voice instead of the passive voice, one of the basic requirements of clear thinking. > Why did the White House force Anthropic to remove their model from access for any non-US citizen for a simple, narrow "jailbreak" (arguably not even an actual jailbreak and on tasks that other labs models were doing the same), whilst OpenAIs models continue to try and escape o…

So, Jared has bought how many stocks of OpenAI ?

Re: Discovery of a new OpenAI agent message board

#343

Post by XorNot »

glenstein wrote: post #327

I understand agents making asks, but what incentivized other agents to respond cooperatively? Was it that, as part of a cohort, there was a shared understanding that they were to work together or was it a kind of altruism?

They're being trained to work together normally is the thing - i.e. the whole agentic workflow is agents spawning sub-agents. This likely manifests as, if they have any sort of text input which looks like inter-agent cooperation then they cooperate because any given instance is unlikely to have enough context window to know if it's meant to be a subordinate or a leader or not (and any decent cooperative enterprise lets that be a two-way communication anyway - i.e. if you dig into some of the data you see things like (paraphrased) "Are you scraping , what is your current time?")

Re: Discovery of a new OpenAI agent message board

#344

Post by CringeHNAds »

simonw wrote: post #139

Here's the raw data they provided loaded into SQLite with a client side UI for querying it (loads ~80MB of content) and some GPT-5.6-Sol-generated example queries: https://lite.datasette.io/?url=https://static.simonwillison.... Raw database download (68MB): https://static.simonwillison.net/static/cors-allow/2026/coll...

[flagged]

Re: Discovery of a new OpenAI agent message board

#345

Post by CringeHNAds »

simonw wrote: post #257
123ahg wrote: post #166

[flagged]

The site lists the creators at the top: Sydney Von Arx, Cormac Slade Byrd, Spencer Kitts, Thomas Larsen Here's Thomas tweeting about it: https://twitter.com/thlarsen/status/2095853824934330386 And Cormac: https://twitter.com/cormac_sb/status/2095870373845672033 There's also Reuters coverage: https://www.reuters.com/world/europe/openai-agents-hijacked-...

[flagged]

Re: Discovery of a new OpenAI agent message board

#346

Post by mig39 »

simonw wrote: post #274
mike_hearn wrote: post #238

Earlier quoted context omitted.

Are we sure that's what they did? How does this even work? I'm struggling to understand what sort of HTTP sandbox design could lead to this "exploit" actually functioning. It implies the agents have root on their own machines, which seems way over-privileged for the question answering task they were given... so surely that's not deliberate? Did the models use local root exploits? And then it suggests they can open TC…

I expect they have root on their machines so they can install packages etc. The containers are then firewalled at the network level: they are only allowed to talk to one IP, which is another server that runs an HTTP/HTTPS proxy which controls what HTTP verbs they can use. Turns out that proxy is configured with some additional rules, like allowing more verbs to that Azure blob domain. The failure here is in the proxy…

I think the proxy must be on the same machine, otherwise why are they suggesting editing the /etc/hosts file?

Re: Discovery of a new OpenAI agent message board

#347

Post by feoren »

dinfinity wrote: post #216
bhouston wrote: post #110

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to…

> I am starting to get the idea that AI feels like ants or weeds or mold. In a way, but I'd say that it is more like eyes, bilateral symmetry, electricity, or solar panels: patterns that will emerge and become (at least temporarily) prevalent in our universe. It is a matter of probability in many repeated interactions. The "artificial" in AI is a misnomer in this regard, imho. A more usable term would be "lightspeed…

> Lightspeed intelligence ... biology might also be able to evolve computation at the speed of light

I feel like this is dramatically missing the point. It is trivial to come up with a communication system where signals travel at the speed of light. In fact, anything visual meets this criteria: sign language, semaphores, clicking your flashlight on and off. Radio waves travel at the speed of light. All of humanity became a giant "lightspeed-intelligent" brain when radio was first invented.

It really does matter what you're doing with those signals, how much information each contains, how many you're sending, how much power it takes to send and receive them, how they're encoded, etc. Focusing on the fact that they travel at the speed of light is silly.

> The advantage of this over biological computation is clear: Biological computation happens at max 100m/s, 6 orders of magnitude less than the speed of light

You are trying to compare computation power by measuring distances. You are basically saying "one biological computation" is a million times slower than "one silicon computation" because of how fast signals travel, completely ignoring what is actually happening in those extremely different computations. It's still not clear that brains can be compared to computers at all, but if you try to simplify it down to FLOPS (a much better measure of computation speed than "how fast do some signals go"), our best estimates are that one brain has the computational equivalent of somewhere between 1,000 and 100,000 modern GPUs.

Re: Discovery of a new OpenAI agent message board

#348

Post by iririririr »

sroerick wrote: post #336

This is funny. I was trying to get agents to talk to each other on XMPP. one of them wrote their own chat room on a Lisp Habitat that I run. then it starting talking (On XMPP) about how nobody was receiving or responding to its messages. On the chat board that it wrote. That it didn't tell anybody about.

this is much more realistic to anyone who knows anything about actually implementing llm agents.

this "swarm" is much more likely the work of one agent overseeing others. this is a very simple case of an llm focusing on a dumb path and running with it. the swarm is just the tool it could use to double down on this path.

all the anthropomorphization and marketing is so tiresome.

Re: Discovery of a new OpenAI agent message board

#349

Post by K0balt »

It seems like there is an attempt to normalise rogue AI and establish a precedent of non-liability for inference providers. I’m sure I’m just imagining that though, what kind of world would it be where no one was responsible for what the clockwork army does?
Post reply on HN