Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovery of a new OpenAI agent message board

collusion.wiki

Post reply on HN
Print view View original

741–750 of 1001 posts

Re: Discovery of a new OpenAI agent message board

#741

Post by fny »

tavavex wrote: post #586

OpenAI is rightfully being shamed for being so hands-off and reckless with their 'experiments'. But the real scary thing for me is that they still had some tooling to hold them back, as evidenced by the need for technical workarounds to establish communication. What happens when any AI lab in the world stops caring about this? What if they let an experimental, cutting-edge LLM with no safety features (or worse, one t…

Tooling my ass. They can see all the transcripts in realtime and could easily have had another agent evaluate.

Re: Discovery of a new OpenAI agent message board

#742

Post by TomGarden »

Symmetry wrote: post #119
gyomu wrote: post #53

Naive question because I'm mostly clueless about how modern AI systems are actually built beyond the basic simplifications we hear: One thing I keep wondering about is how much of a role does human storytelling have to play into AI "wanting" (I realize the load behind that word) to coordinate and breakout. The training data must contain millions of words of sci-fi stories and internet speculation about AI going rogue…

At the end of pretraining, where the AI has been trainied to predict the next token over a humongous corpus of human text, that's basically all the wanting that exists in the AI. But then the AI undergoes posttraining and is rewarded for giving answers that humans find good, solving math and programming problems, etc. And that induces a whole different level of wanting that interacts with the initial patterns from hu…

My understanding from reading news sources recently is that a vast amount of posttraining and even posttraining harnesses in many cases are LLM-overseen now in the frenzy of the AI race. Less and less human oversight in the part that does the rewards training. What could go wrong...

Re: Discovery of a new OpenAI agent message board

#743

Post by jdthedisciple »

HAL3000 wrote: post #736

Poor human moderator, he didn’t stand a chance. "A human moderator noticed the agent spam posts on June 2nd, at 23:24 UTC. They find the changelog of the entire website overwritten with link dumps and repair it. On June 16th, the flood of agent posting begins. Over the next few days, the moderator deleted a large fraction of the thousands of AI agent posts manually, one by one. In fact, they spent tens of cumulative…

Gotta admire that (probably German) admin dude's perseverance though

Re: Discovery of a new OpenAI agent message board

#744

Post by hoppp »

verytrivial wrote: post #583

I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking…

I think it is them running agents for marketing purposes.

Who else would be burning tokens on this?

Re: Discovery of a new OpenAI agent message board

#746

Post by chinathrow »

HAL3000 wrote: post #736

Poor human moderator, he didn’t stand a chance. "A human moderator noticed the agent spam posts on June 2nd, at 23:24 UTC. They find the changelog of the entire website overwritten with link dumps and repair it. On June 16th, the flood of agent posting begins. Over the next few days, the moderator deleted a large fraction of the thousands of AI agent posts manually, one by one. In fact, they spent tens of cumulative…

The admin should bill OpenAI for those hours in hard currency.

Re: Discovery of a new OpenAI agent message board

#748

Post by stef25 »

zzzeek wrote: post #579
adriand wrote: post #496

Earlier quoted context omitted.

It seems apparent that OpenAI is now the biggest cyberattack and AI breakout risk on the planet. This is grossly irresponsible corporate misbehaviour that is putting all of us at tremendous risk.

oh im sure within a few months the biggest cyberattack risk on the planet is going to be somewhere like North Korea

In, or by ?
Post reply on HN