Live data from Hacker News

ChatGPT claims rogue AI attacked more companies

bbc.co.uk

31–40 of 89 posts

Re: ChatGPT claims rogue AI attacked more companies

#32
How on earth is not the police involved at this point to literally pull the plug on the unethical OpenAI security experiments?! This is bananas, a company is effectively telling the world they're unable to safely contain their experiments, and not only back in 2024, but again just now, and with multiple victims at that too!

I think the whole "AI will take over the world and enslave humanity" (or whatever the doomerism is today) is a bit over the top, but at very least we should contain the companies who clearly demonstrate they cannot handle containing what they're experimenting with, when what they work on breaks containment over and over again. Where are the people who are supposed to be keeping the public safe? Alarm bells should be going off all over the place at this point.

Re: ChatGPT claims rogue AI attacked more companies

#33
post #5

Still not understanding why this isn’t being persecuted and there is little governmental reaction after blocking models for jailbreaks…

Same reason road vehicles haven't been banned despite killing hundreds of thousands every single year for over a century

Re: ChatGPT claims rogue AI attacked more companies

#34
post #29

Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals. This will not happen though, because these stories are marketing.

They should shut them down immediately then investigate. Extraordinary claims need extraordinary measures. If it’s rubbish those stories will stop instantly.

> They should shut them down immediately then investigate.

Yeah, I don't understand either. If there was a "hitman for hire" service on the clearweb, the police would shut it down first, then ask questions. Now we have a huge company effectively letting AI agents without guardrails run amok on 3rd party infrastructure, and the police is doing nothing?

Re: ChatGPT claims rogue AI attacked more companies

#35

> The agents repeated actions that they had already completed - a sign of an agentic AI losing its thread and context. > The agents also hallucinated reams of incoherent commands and text and were sloppy and did not cover their tracks well. ASI works in mysterious ways.

Why not complete the quote?

> But among the errors and strange behaviour, Hugging Face warned the AI agents made brilliant technical moves and were able to rapidly adapt to new scenarios in the days-long hack.

Re: ChatGPT claims rogue AI attacked more companies

#37
post #5

Still not understanding why this isn’t being persecuted and there is little governmental reaction after blocking models for jailbreaks…

Same reason road vehicles haven't been banned despite killing hundreds of thousands every single year for over a century

Respectfully, that’s such a nonsensical comparison I don’t even know where to start.

Re: ChatGPT claims rogue AI attacked more companies

#38
post #31
post #5

Still not understanding why this isn’t being persecuted and there is little governmental reaction after blocking models for jailbreaks…

As for prosecution, none of the victims in this case have any interest in pressing charges.

Isn’t prosecution independent of victims?

Re: ChatGPT claims rogue AI attacked more companies

#39

Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals. This will not happen though, because these stories are marketing.

> This will not happen though, because these stories are marketing.

The magnitude and the complexity of the cynicism displayed by some people when it comes to AI risks is mind-blowing.

It's like if the NRA reported on school shootings and people said "oh, they probably fake these shootings to make guns sound dangerous and sell more of them".

OpenAI could report that its AI started spontaneously generating illegal porn and sending it to people and you'd still think it was a marketing stunt.

Re: ChatGPT claims rogue AI attacked more companies

#40
post #22

> The agents repeated actions that they had already completed - a sign of an agentic AI losing its thread and context. > The agents also hallucinated reams of incoherent commands and text and were sloppy and did not cover their tracks well. ASI works in mysterious ways.

Anyone that thinks the current agent systems will get us to AGI or ASI is either delusional or isn’t actually using them. This is entirely separate from them having uses.

That first statement is great, because it's generic and self-defensive enough to apply regardless of what happens in the future. If current LLMs with small modifications to the architecture does lead to AGI, they're clearly not "current agent systems" anymore. Witty :)

With that said, I do agree with you, they're highly productive to certain workflows, and personally a great help for oh so many things, but they're also really, really dumb and the average person (and even general developer) really misunderstands how it all works and what can be relied on for vs not.

Post reply on HN