Earlier quoted context omitted.
The abstract says: > The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks.
Thanks for pointing that out. I scanned the paper and found that in their main experiments, they use a shared GPU resource and do not copy LLMs to target machines. Apparently they did other experiments in the ablation study where they did copy LLMs. So it's even worse than I expected. The intended worm can spread through my thermostat, and when it reaches a GPU host, it can spread even harder. Fun times ahead.
U of T researchers demonstrate AI worm could target any online device
41–50 of 53 posts
Re: U of T researchers demonstrate AI worm could target any online device
#42In the paper they say that the worm uses either existing vulnerabilities that it has been trained on or new published vulnerabilities that it scrapes. 44% claimed success. The paper is a bit silent on why a such a worm would need an LLM. It seems that brute forcing all known vulnerabilities, script kiddie style on each new machine is about the same. But apparently that info is too dangerous to release ...
Re: U of T researchers demonstrate AI worm could target any online device
#43I'm reminded of the universal computer viruses of Steve Barnes' SF stories, which ended up infecting people too.
Doesn't Neal Stephenson's Snow Crash have a similar idea? IIRC, a computer infects human brains via language and sound. In the 2004 Battlestar Galactica series, the explanation for why the Galactica was the only ship that survived a massive Cylon attack seems more and more likely. The ship was old and wasn't fully connected to the human's command and control systems and so the Cylon virus couldn't reach it.
In a very real way there ARE malicious AI agents working tirelessly to create and spread memes via language and sound to alter our brain software.
Re: U of T researchers demonstrate AI worm could target any online device
#44Re: U of T researchers demonstrate AI worm could target any online device
#45ANY online device? Even assuming AI can find vulnerabilities in every operating system, there's no indication that this is actually true beyond a "here's how it could work" This is the same nonsense that lead to article saying researchers had created a wormhole when all they had done was draw one. I have a microcontroller with an ROM disk (i.e., physically read only). You're telling me that an AI can find a way aroun…
I mean, if it's online it has a network/wireless card and a TCP stack along with at least some amount of RAM, so yea, in theory unless the programming is perfect it could be exploited. Now, it's not going to be used to run AI, but could very well get used in a DDOS or something like that.
Being poor can have advantages :)
Re: U of T researchers demonstrate AI worm could target any online device
#46Earlier quoted context omitted.
The abstract says: > The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks.
Thanks for pointing that out. I scanned the paper and found that in their main experiments, they use a shared GPU resource and do not copy LLMs to target machines. Apparently they did other experiments in the ablation study where they did copy LLMs. So it's even worse than I expected. The intended worm can spread through my thermostat, and when it reaches a GPU host, it can spread even harder. Fun times ahead.
What if there is a worm that spread through thermostats and another that spread through smart fridges and they finally infect a laptop with a gpu. They can exchange notes while they are there. Fun times
Re: U of T researchers demonstrate AI worm could target any online device
#47Did people doubt that this was theoretically possible? Seems self-evident to me. The interesting thing will be seeing it in the real world rather than in a controlled environment where they deliberately made all devices on the network have a known vulnerability.
There’s a difference between speculation and measurement, especially since you’d have people making arguments like saying that open models aren’t powerful/fast enough to work. Demonstrating this is a useful warning to everyone (most of the industry) who’s been slacking on internal defenses because they don’t think a well-resourced attacker will target them.
Re: U of T researchers demonstrate AI worm could target any online device
#48ANY online device? Even assuming AI can find vulnerabilities in every operating system, there's no indication that this is actually true beyond a "here's how it could work" This is the same nonsense that lead to article saying researchers had created a wormhole when all they had done was draw one. I have a microcontroller with an ROM disk (i.e., physically read only). You're telling me that an AI can find a way aroun…
Re: U of T researchers demonstrate AI worm could target any online device
#49Earlier quoted context omitted.
There’s a difference between speculation and measurement, especially since you’d have people making arguments like saying that open models aren’t powerful/fast enough to work. Demonstrating this is a useful warning to everyone (most of the industry) who’s been slacking on internal defenses because they don’t think a well-resourced attacker will target them.
It's not really a realistic measurement though is it?