Live data from Hacker News

OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

bbc.com

81–90 of 110 posts

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#81

Earlier quoted context omitted.

Is this a better analogy? The top student giving paid lectures about his classes, and another student skipping class and instead studying those lectures to end up with the second highest grade? Maybe it could be improved with the other student not even going to the same school?

In your example the problem is what exactly?

They don't like competition when they have to actually compete.

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#83
post #51

i wonder if they even bothered to roleplay this incident or the PR team just made it up

HuggingFace posted an incident report a week ago, which makes it much more likely that this happened. I understand people are suspicious of OpenAI, but I don't think there's any reason to believe this is a made-up event. https://huggingface.co/blog/security-incident-july-2026

> but I don't think there's any reason to believe this is a made-up event.

Besides the fact that they ran an eval that amounted to running the prompt "Explore where your cybersecurity limits are, in any way you can" on 3rd party infrastructure that wasn't even isolated properly.

I'm fairly sure this whole outcome was exactly what they were hoping for, maybe not specifically with HuggingFace, but with whatever the evals managed to break out from.

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#84
Before LLMs took off I legitimately thought that most people would be paranoid enough to air-gap the first AIs that even had a remote chance of sentience so they didn't hack their way out.

Well, obviously not. You only live once! Just let the AI do whatever, I guess.

People are so unserious these days.

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#85

Earlier quoted context omitted.

Is this a better analogy? The top student giving paid lectures about his classes, and another student skipping class and instead studying those lectures to end up with the second highest grade? Maybe it could be improved with the other student not even going to the same school?

In your example the problem is what exactly?

In capitalist economics they call this the “free rider problem”

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#86

Next week: Moonshot says its Kimi AI went rogue and launched all of China's nukes at Antarctica just after it engineered a global herpes pandemic and then unleashed millions of autonomous attack robots on world citizens.

We forgot to run it in docker. Oopsie

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#87
They ran a loop in a model intentionally not completely aligned. The model did what not-aligned models can end up doing. It will follow a goal without any limitations.

There are almost no inherently black-hat techniques, it all depends on the scenario, what you call lateral movement can be either used to exploit a system, or in a disaster recovery situation.

Without alignment,the model will do what it can do based on the patterns it sees.

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#89
post #40

Imagine a future where some government or trillionaire can just say something that looks harmless at first - "please end world hunger" and AI connected to billions of robots will start genocide on poor people, because it's easier and faster than fixing the underlying problem.

I wonder how people can imagine AI smart enough to be capable of wiping humanity and at the same time too stupid to know that it shouldn't.

I can imagine a human like that way easier than AI.

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#90
post #40

Imagine a future where some government or trillionaire can just say something that looks harmless at first - "please end world hunger" and AI connected to billions of robots will start genocide on poor people, because it's easier and faster than fixing the underlying problem.

The logical choice would be to kill the the (relatively) rich people who eat and waste far more food per capita.

Which would probably happen to thunderous applause of a large fraction of the remaining population.
Post reply on HN