Live data from Hacker News

U of T researchers demonstrate AI worm could target any online device

utoronto.ca

41–50 of 53 posts

Re: U of T researchers demonstrate AI worm could target any online device

#41
post #28

Earlier quoted context omitted.

The abstract says: > The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks.

Thanks for pointing that out. I scanned the paper and found that in their main experiments, they use a shared GPU resource and do not copy LLMs to target machines. Apparently they did other experiments in the ablation study where they did copy LLMs. So it's even worse than I expected. The intended worm can spread through my thermostat, and when it reaches a GPU host, it can spread even harder. Fun times ahead.

You'll just have to starve it with a bunch of thermostats that lead it towards the GPU rich honey pot where you will extract it...

Re: U of T researchers demonstrate AI worm could target any online device

#42
post #35

In the paper they say that the worm uses either existing vulnerabilities that it has been trained on or new published vulnerabilities that it scrapes. 44% claimed success. The paper is a bit silent on why a such a worm would need an LLM. It seems that brute forcing all known vulnerabilities, script kiddie style on each new machine is about the same. But apparently that info is too dangerous to release ...

AV/EDRs are kinda lame but "brute force all known vulnerabilities" is definitely something they can detect.

Re: U of T researchers demonstrate AI worm could target any online device

#43
post #16

I'm reminded of the universal computer viruses of Steve Barnes' SF stories, which ended up infecting people too.

Doesn't Neal Stephenson's Snow Crash have a similar idea? IIRC, a computer infects human brains via language and sound. In the 2004 Battlestar Galactica series, the explanation for why the Galactica was the only ship that survived a massive Cylon attack seems more and more likely. The ship was old and wasn't fully connected to the human's command and control systems and so the Cylon virus couldn't reach it.

"Memes" they called them in Snow Crash (1992).

In a very real way there ARE malicious AI agents working tirelessly to create and spread memes via language and sound to alter our brain software.

Re: U of T researchers demonstrate AI worm could target any online device

#45
post #33
post #25

ANY online device? Even assuming AI can find vulnerabilities in every operating system, there's no indication that this is actually true beyond a "here's how it could work" This is the same nonsense that lead to article saying researchers had created a wormhole when all they had done was draw one. I have a microcontroller with an ROM disk (i.e., physically read only). You're telling me that an AI can find a way aroun…

I mean, if it's online it has a network/wireless card and a TCP stack along with at least some amount of RAM, so yea, in theory unless the programming is perfect it could be exploited. Now, it's not going to be used to run AI, but could very well get used in a DDOS or something like that.

I'm relieved my entire network consists of old computers and small, low power network devices. The only dedicated GPUs you'll find is discarded old junk in a cardboard junk box.

Being poor can have advantages :)

Re: U of T researchers demonstrate AI worm could target any online device

#46
post #28

Earlier quoted context omitted.

The abstract says: > The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks.

Thanks for pointing that out. I scanned the paper and found that in their main experiments, they use a shared GPU resource and do not copy LLMs to target machines. Apparently they did other experiments in the ablation study where they did copy LLMs. So it's even worse than I expected. The intended worm can spread through my thermostat, and when it reaches a GPU host, it can spread even harder. Fun times ahead.

This reminded me of the geth from mass effect. They get smarter as more geth "agents" network together.

What if there is a worm that spread through thermostats and another that spread through smart fridges and they finally infect a laptop with a gpu. They can exchange notes while they are there. Fun times

Re: U of T researchers demonstrate AI worm could target any online device

#47
post #11
post #7

Did people doubt that this was theoretically possible? Seems self-evident to me. The interesting thing will be seeing it in the real world rather than in a controlled environment where they deliberately made all devices on the network have a known vulnerability.

There’s a difference between speculation and measurement, especially since you’d have people making arguments like saying that open models aren’t powerful/fast enough to work. Demonstrating this is a useful warning to everyone (most of the industry) who’s been slacking on internal defenses because they don’t think a well-resourced attacker will target them.

It's not really a realistic measurement though is it?

Re: U of T researchers demonstrate AI worm could target any online device

#48
post #25

ANY online device? Even assuming AI can find vulnerabilities in every operating system, there's no indication that this is actually true beyond a "here's how it could work" This is the same nonsense that lead to article saying researchers had created a wormhole when all they had done was draw one. I have a microcontroller with an ROM disk (i.e., physically read only). You're telling me that an AI can find a way aroun…

RAM-only viruses exist.

Re: U of T researchers demonstrate AI worm could target any online device

#49
post #11

Earlier quoted context omitted.

There’s a difference between speculation and measurement, especially since you’d have people making arguments like saying that open models aren’t powerful/fast enough to work. Demonstrating this is a useful warning to everyone (most of the industry) who’s been slacking on internal defenses because they don’t think a well-resourced attacker will target them.

It's not really a realistic measurement though is it?

It shows that the idea can work in a somewhat realistic scenario. That doesn’t mean we’re all doomed but it’s a good wake up call for anyone who’s been slacking on internal controls or patching.

Re: U of T researchers demonstrate AI worm could target any online device

#50
Of course this is possible, but to use a botnet for intelligence compute is going to be slow. That's one thing we have going for us—it's going to take a long time for this bug to fumble through your network. But rest assured, eventually, it will pwn you.
Post reply on HN