Live data from Hacker News

OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

bbc.com

21–30 of 110 posts

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#25
The interesting part isn't whether the headline says "AI went rogue" or whether it's PR. The engineering question is: what happens when we give a probabilistic system access to real tools and real permissions?

A model does not need intent to cause damage. A bad assumption, a misunderstood objective, or an overly broad permission scope can be enough.

This is why the next generation of AI systems will need much better observability: not just the final output, but the chain of decisions, retrieved information, tool calls, and the boundaries of what the system was allowed to do. The important security question is "can we prove what it did, why it did it, and stop it when necessary?"

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#28
post #12

This kind of narrative is going to bite them just like the "AI will take your job" narrative has. It feels like the frontier labs are taking a massive gamble with public perception here. I assume the goal is to paint the technology as so powerful and dangerous that only a handful of blessed US companies should be trusted to run it, in an attempt to suppress the rise of the Chinese models that are rapidly catching the…

Are the Chinese models actually catching up or are they just distilling the frontiers? If it's all just distilling then they'll always be behind.

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#29
post #12

This kind of narrative is going to bite them just like the "AI will take your job" narrative has. It feels like the frontier labs are taking a massive gamble with public perception here. I assume the goal is to paint the technology as so powerful and dangerous that only a handful of blessed US companies should be trusted to run it, in an attempt to suppress the rise of the Chinese models that are rapidly catching the…

That’s the play. That these frontier models are so powerful that they must be behind sovereign firewalls and gateways.

Please buy our IPO before it crashes so we can be billionaires.

Re: OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

#30
post #12

This kind of narrative is going to bite them just like the "AI will take your job" narrative has. It feels like the frontier labs are taking a massive gamble with public perception here. I assume the goal is to paint the technology as so powerful and dangerous that only a handful of blessed US companies should be trusted to run it, in an attempt to suppress the rise of the Chinese models that are rapidly catching the…

That’s the play. That these frontier models are so powerful that they must be behind sovereign firewalls and gateways.

Didn't HF use an open weights model running on their own hardware to solve the issue though? Sort of defeats that narrative and plays into one in which frontier == bad_guys and open == good_guys
Post reply on HN