The golden age of scammers: AI-powered phishing

Post by savanaly » Wed, Jul 17, 2024, 1:48 PM UTC

jjulius wrote: post #87

dbspin wrote: post #75

I'm kind of amazed how slow 'AI phishing' has been to roll out. The technology for customised text based attacks at scale has been available at least since Llama was open sourced. The tech for custom voice and image based attacks is basically there too with whisper / tortoise and stable diffusion - though clearly more expensive to render. I'm honestly not sure why social networks aren't being leveraged more to target…

Forgive my ignorance, but why are we surprised that voice messages aren't being spoofed more often? Doesn't this require a pretty darn decent dataset for training? Unless they've got a ton of videos of themselves shared on a public social media profile, I don't know that this is going to be a thing.

Post by Grimblewald » Wed, Jul 17, 2024, 2:25 PM UTC

reginald78 wrote: post #80

HPsquared wrote: post #60

Earlier quoted context omitted.

Also the URL "security scanner" things on corporate email systems. The user can't see the URL by hovering over it.

I was about to complain about this. I take security training to inspect URLs and then Microsoft safe link or whatever it's called gives me a twenty line long URL filled with random characters. I have to trust it works since they took my manual inspection ability away.

Post by cess11 » Wed, Jul 17, 2024, 2:33 PM UTC

Der_Einzige wrote: post #82

dbspin wrote: post #75

I'm kind of amazed how slow 'AI phishing' has been to roll out. The technology for customised text based attacks at scale has been available at least since Llama was open sourced. The tech for custom voice and image based attacks is basically there too with whisper / tortoise and stable diffusion - though clearly more expensive to render. I'm honestly not sure why social networks aren't being leveraged more to target…

It appears that criminals really are stupid, and thank goodness for it!

Post by tijtij » Wed, Jul 17, 2024, 2:46 PM UTC

diatone wrote: post #89

dbspin wrote: post #75

I'm kind of amazed how slow 'AI phishing' has been to roll out. The technology for customised text based attacks at scale has been available at least since Llama was open sourced. The tech for custom voice and image based attacks is basically there too with whisper / tortoise and stable diffusion - though clearly more expensive to render. I'm honestly not sure why social networks aren't being leveraged more to target…

It’s not enough for the method to be possible, it has to be economical. If it’s not prevalent yet it means one of two things: (1) cheaper methods work too well for now to even bother with sophisticated approaches (2) it’s too expensive to be worth the effort It would be interesting to plot out the cost of an integrated AI stack for this over time.

Post by nerdponx » Wed, Jul 17, 2024, 3:02 PM UTC

smeej wrote: post #77

nonrandomstring wrote: post #62

Earlier quoted context omitted.

> Tbh the browser/email client makers are complicit in these phishing attempts for hiding the URLs and the actual email addresses. It's worse. Research "Scamicry". Big business now is so fake, such a grift, drenched in PR deception, and lacking integrity and trustworthiness, there isn't much space left between what is "legitimate" and what is a scam. If businesses like Google or Facebook hide URLs and email addresses…

I beat that drum so long it turned into beating my head against a wall. The last two companies I worked for insisted that customer account security was the highest priority, but as soon as I said we needed to stop hiding links to our own website behind Hubspot tracking URLs so we don't train our customers to click links that look like gobbledygook garbage, the marketing team melted down and it became clear where user…