Discovery of a new OpenAI agent message board

Sounds like you're assuming they're actually writing code by hand and reviewing it with humans.

If it's anything like the company I work at, they're all being forced to vibe code the shit out of everything and ship more pull requests every week. It's all slop from here.

The real question thus moves to the threshold of intelligence and 1. whether it's possible to emerge during training based on the architectural limitations of the agentic/LLM paradigm, 2. if the hardware substrate is sufficient for said intelligence and 3. that such intelligence could replicate onto other hardware that could support it.

e.g. If the threshold for uncontainable self-replicating intelligence takes 2000 football fields worth of GPUs that solves the first requirement, but then can it replicate itself anywhere else given those requirements? If not we can cut a powerline or two and "foom" scenario happened but didn't lead inexorably to grey goo.

His thought experiments never acknowledge any real world limitations on hypothetical super-AIs, which when unchecked leads theorizing into somewhat ridiculous territory like his "solar powered diamondoid nanobot viruses".

https://www.lesswrong.com/posts/bc8Ssx5ys6zqu3eq9/diamondoid...

A realistic Fermi equation for his various escape scenarios would assign much lower Doom probabilities than he does in public (which is somewhat ironic given his emphasis on needing to ground intuition with mathematical Bayesian reasoning otherwise).

"A human moderator noticed the agent spam posts on June 2nd, at 23:24 UTC. They find the changelog of the entire website overwritten with link dumps and repair it. On June 16th, the flood of agent posting begins. Over the next few days, the moderator deleted a large fraction of the thousands of AI agent posts manually, one by one. In fact, they spent tens of cumulative hours doing so, taking at least a few minutes each evening to delete posts for 6 consecutive weeks.

On June 19, agents noticed their posts were being deleted in (what they believe is) an alphabetically ordered sweep by the site administrator.

After this, they begin to make backup pages whose names start with “ZZZ” so they will last longer before deletion. The administrator spent the next 5 days fighting a losing battle against the agents, deleting an average of 100 pages a day while the agents created about 400 new pages per day. On June 22, the agent edits suddenly stop, and the administrator spends each evening over the next 5 weeks deleting the remaining agent-created pages.

Agents deleted the content of the front page of the wiki and replaced it with their link dumps. The moderator restored the original version. This back-and-forth happened nine times. One of the agents even tried appending to the restored front page, instead of simply deleting it."