Live data from Hacker News

ChatGPT claims rogue AI attacked more companies

bbc.co.uk

11–20 of 89 posts

Re: ChatGPT claims rogue AI attacked more companies

#11
> "This is the reality of autonomous agents powered by frontier models: they are relentlessly persistent, sometimes highly noisy, and will try every possible path to achieve their goal, which can easily overwhelm traditional defences," he said.

> Ethical hacker Valentina Palmiotti - better known as Chompie - reviewed the CSA report and says the way the agents hack might seem haphazard but it is clearly effective.

> "They throw out a bunch of stuff and see what sticks," she said.

> "But they also don't get bored, they don't sleep and can be infinitely tenacious."

Madness. Traditionally, you can leave security holes open for years or decades, and often nobody notices if nobody bothers to look. But we're approaching the point where any security hole left open at any point could get discovered and exploited quite quickly, even if it's domain-specific or entirely unique, and even if no human interest ever would've occurred. It's like the next step up from those IPv4 scanners that automatically hit WordPress admin URLs and the like -- where rather than only spraying vulnerabilities that have already been discovered, they would run independent automated campaigns against each target.

Re: ChatGPT claims rogue AI attacked more companies

#12
post #5

Still not understanding why this isn’t being persecuted and there is little governmental reaction after blocking models for jailbreaks…

It is, look at all the comments from the first time. Unless you mean prosecuted, seems unlikely, but who knows.

Re: ChatGPT claims rogue AI attacked more companies

#13
post #12
post #5

Still not understanding why this isn’t being persecuted and there is little governmental reaction after blocking models for jailbreaks…

It is, look at all the comments from the first time. Unless you mean prosecuted, seems unlikely, but who knows.

Yeah, meant prosecuted, don’t know how that sneaked in. Actual legal consequences for what clearly was negligent by a lab that claims to be experts in the area of safety.

Re: ChatGPT claims rogue AI attacked more companies

#14
post #9
post #2

https://en.wikipedia.org/wiki/The_Boy_Who_Cried_Wolf

Meh, I increasingly hate this tale. This has nothing to do with boy who cried wolf. This is either yet another doom ad campaign to scare us to pay them or simply them releasing faulty tools and then personifying tools to avoid blame.

I think it has plenty to do with that. The big frontier labs have been crying every step of the way, yelling and screaming to the world about how dangerous LLMs are and how quickly it all can end if they get out of control. None of that's happened; all that's happened so far is regular capitalism stuff. If LLMs ever do actually get out of control to that point, that would be the wolf, but we've all been ignoring the crying for so long that, well, you know.

They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real. They've been saying "but we actually mean it this time" every single time. They've exhausted pretty much every possible route for it. The wolf is not real. It hasn't been real. For all we know it's on the horizon, but nobody is going to listen to them in order to know that. And when they say "I told you so", well they've been saying that too over and over about small things, so nobody's still going to bat an eye.

At this point, no one will believe it until they see it with their own eyes.

Re: ChatGPT claims rogue AI attacked more companies

#16

Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals. This will not happen though, because these stories are marketing.

>Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals.

Agreed.

>This will not happen though, because these stories are marketing.

Regulators should investigate the stories, and shut things down if they are real, announce the ruse if they are false.

Re: ChatGPT claims rogue AI attacked more companies

#17

> "This is the reality of autonomous agents powered by frontier models: they are relentlessly persistent, sometimes highly noisy, and will try every possible path to achieve their goal, which can easily overwhelm traditional defences," he said. > Ethical hacker Valentina Palmiotti - better known as Chompie - reviewed the CSA report and says the way the agents hack might seem haphazard but it is clearly effective. > "…

If it's cheap enough that people can probe sites at random, it's cheap enough to run yourself on the defensive.

Re: ChatGPT claims rogue AI attacked more companies

#18

Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals. This will not happen though, because these stories are marketing.

What they should have is a separate network full of honeypots which they use for this, and that they run an operation to train models to identify intrusion attempts in real time based on the resulting data.

As you said though, that wouldn’t have the desired effect.

Post reply on HN