WormGPT – The Generative AI Tool Cybercriminals Are Using
1–10 of 61 posts
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#2> GPT-J is the LLM, the old one from 2021
Thats very interesting.
The infamous Pygmalion 6B is a GPT-J finetune, predating the LLM craze. Yet its decent in its roleplaying niche.
But the LLaMA 13B version, with instruct finetuning, is massively better, even with dataset errors that allegedly messed up its performance. In fact, a chat with Metharme 13b, where it made some very introspective logical jumps, was my first real LLM "Wow!" moment.
And Airoboros-Chronos 33b is leagues ahead of that.
If someone in that forum has a 3090, and trains LLaMA 33b on that dataset + a instruct dataset off huggingface... Yeah, that would be terrifying.
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#3> GPT-J is the LLM, the old one from 2021 Thats very interesting. The infamous Pygmalion 6B is a GPT-J finetune, predating the LLM craze. Yet its decent in its roleplaying niche. But the LLaMA 13B version, with instruct finetuning, is massively better, even with dataset errors that allegedly messed up its performance. In fact, a chat with Metharme 13b, where it made some very introspective logical jumps, was my first…
Phishing emails self select, I don't think generative ML would make a difference. Those "typos" are sometimes intentional. The scammers are not illiterate. They are more than capable of using Grammarly if they wanted to.
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#4> GPT-J is the LLM, the old one from 2021 Thats very interesting. The infamous Pygmalion 6B is a GPT-J finetune, predating the LLM craze. Yet its decent in its roleplaying niche. But the LLaMA 13B version, with instruct finetuning, is massively better, even with dataset errors that allegedly messed up its performance. In fact, a chat with Metharme 13b, where it made some very introspective logical jumps, was my first…
Phishing emails self select, I don't think generative ML would make a difference. Those "typos" are sometimes intentional. The scammers are not illiterate. They are more than capable of using Grammarly if they wanted to.
But what if emails can be mass produced at 15+ words/second, on a gaming PC?
And what if they can very convincingly and quickly reply?
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#5the best educational resource on this topic is the pictured forum, which is obfuscated to the reader, which proves the article to be clickbait to me
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#6This is basically about bad actors using LLMs to generate better emails; however you could also automate actual conversations at scale which is what I thought this article was going to be about.
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#7Earlier quoted context omitted.
Phishing emails self select, I don't think generative ML would make a difference. Those "typos" are sometimes intentional. The scammers are not illiterate. They are more than capable of using Grammarly if they wanted to.
But what if emails can be mass produced at 15+ words/second, on a gaming PC? And what if they can very convincingly and quickly reply?
A bunch of random thoughts:
Why would being able to write different emails help? Wouldn't they just reuse the same email over and over again, maybe just changing the addressee's name?
Also, now I wonder what the typical response rate is for phishing emails? Are they already getting too many responses to keep up with or are they already managing just fine without bots?
I think LLMs might be better for scams carried out over instant messaging services like Discord because that probably requires greater attention and flexibility on the part of the scammer.
Still, has the number of people that could conceivably be scammed over an email or instant messaging-based scam increased? That is, will LLMs, by virtue of their response quality, responsiveness, or some other salient quality, increased the pool of people who could fall for these scams? If the answer is no, then I don't see how increased scale and throughput for human-quality text generation will really matter much.
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#8the best educational resource on this topic is the pictured forum, which is obfuscated to the reader, which proves the article to be clickbait to me
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#9As usual, LLMs are far too wordy. That urgent email from the CEO was way to long. Bezos sent others in to a tizzy with just a question mark.
Re: WormGPT – The Generative AI Tool Cybercriminals Are Using
#10Earlier quoted context omitted.
But what if emails can be mass produced at 15+ words/second, on a gaming PC? And what if they can very convincingly and quickly reply?
A bunch of random thoughts: Why would being able to write different emails help? Wouldn't they just reuse the same email over and over again, maybe just changing the addressee's name? Also, now I wonder what the typical response rate is for phishing emails? Are they already getting too many responses to keep up with or are they already managing just fine without bots? I think LLMs might be better for scams carried ou…
My guess is that being able to generate lots of different text helps to bypass existing spam and phishing protections.
Think of it as a large scale A/B test of circumvention techniques against both human and automated scam detection systems.
Done right, it can effectively keep getting better on its own. Then add phone scamming and various other interactive scams once it is good enough. TBH, it is pretty scary where this will inevitably head.