Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

vice.com

Post reply on HN
Print view View original

41–50 of 60 posts

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#41

Post by squeaky-clean »

gruez wrote: post #37
userbinator wrote: post #31

Earlier quoted context omitted.

Basically everything from the demoscene... AVs have also turned into moral police with their detection of cracks/keygens/"potentially unwanted software". Detecting worms, ransomware, and the like is one thing, but I think going beyond that crosses a line.

That might work for technical users like yourself, but non-technical people need an opinionated detection engine. If the computer has some sort of software that hijacks search queries, should that be removed? Chances are the user didn't want the software there, but you can't know for sure. What about a RAT? Maybe they wanted it there to monitor their kids.

I don't think this is what they were referring to. I've had an anti-virus nuke the directory for a pirated and cracked game. All because there was a keygen.exe detected in the directory.

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#42

Post by jgalentine007 »

some_random wrote: post #36

The actual research: https://skylightcyber.com/2019/07/18/cylance-i-kill-you/ Actual summary, Cylance has a model that uses strings as a feature, and by taking all the strings from the whitelisted files and catting them onto the malware sample one can subvert the ensemble model completely.

Which is something a novice computer user might try when 'hacking'. Really Cylance?

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#43

Post by mruts »

sourabhkt wrote: post #18

and what most of the companies try to sell as AI is not anywhere close.

I mean, is deep learning AI? Is word2vec AI? I would say yes, but I guess some would say no.

But it doesn’t really matter because no one has a good definition of AI. Perhaps the best one would be passing the Turing test, but that has problems as well.

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#44

Post by ma2rten »

jgalentine007 wrote: post #42
some_random wrote: post #36

The actual research: https://skylightcyber.com/2019/07/18/cylance-i-kill-you/ Actual summary, Cylance has a model that uses strings as a feature, and by taking all the strings from the whitelisted files and catting them onto the malware sample one can subvert the ensemble model completely.

Which is something a novice computer user might try when 'hacking'. Really Cylance?

Some early spam filters could also be tricked using the same technique. I think PG might have written about this.

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#45

Post by AlexandrB »

Whenever I’ve used AV I have found it to be worse than useless. Only detections are false positives, uses tons of systems resources, breaks commonly used apps.

I generally agree with Tavis Ormandy[1] that AV products often just increase the attack surface. Especially for applications that already have a high security focus (e.g. browsers). I’m somewhat surprised to see so many AV believers in the HN comments section.

[1] http://blog.cmpxchg8b.com/2016/03/security-software-certific...

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#46

Post by self_awareness »

AlexandrB wrote: post #45

Whenever I’ve used AV I have found it to be worse than useless. Only detections are false positives, uses tons of systems resources, breaks commonly used apps. I generally agree with Tavis Ormandy[1] that AV products often just increase the attack surface. Especially for applications that already have a high security focus (e.g. browsers). I’m somewhat surprised to see so many AV believers in the HN comments section.…

Neither you nor Tavis are taking into account non-technical people. Nobody forces you to use AV -- if you feel you have the skill to defend yourself, do yourself a favor (and us), and remove AV from your boxes. But leave non-technical people alone.

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#47

Post by gregmac »

jm4 wrote: post #38
Trias11 wrote: post #3

I literally make customers disable cylance on their servers or we are not guaranteeing servers operations. Too many buggy conflicts.

How does that usually work out? Personally, I would throw out a vendor who asked me to disable my AV. I don't mind whitelisting a directory or making certain adjustments, but it's hard to take them seriously if they can't work with AV. Cylance isn't that bad. I ran it for 3 years. The false positives were annoying, but it also stopped a lot of nasty stuff that our traditional AV wasn't detecting. I'm in an environmen…

What type of "nasty stuff" did it stop? Are you talking servers or workstations here? How did that nasty stuff get far enough in that it had to be "stopped" by AV?

(I don't mean this to sound snarky, I'm genuinely interested in what type of real-world infections are happening and being detected. My experience is limited to internet-facing endpoints that have no security or are horribly outdated -- which is typically a bigger failure of IT to begin with -- and non-technical end-users running random executables from e-mails or shady websites)

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#48

Post by michaelmrose »

self_awareness wrote: post #46
AlexandrB wrote: post #45

Whenever I’ve used AV I have found it to be worse than useless. Only detections are false positives, uses tons of systems resources, breaks commonly used apps. I generally agree with Tavis Ormandy[1] that AV products often just increase the attack surface. Especially for applications that already have a high security focus (e.g. browsers). I’m somewhat surprised to see so many AV believers in the HN comments section.…

Neither you nor Tavis are taking into account non-technical people. Nobody forces you to use AV -- if you feel you have the skill to defend yourself, do yourself a favor (and us), and remove AV from your boxes. But leave non-technical people alone.

Herein lies the challenge. The user that is so ill trained that AV is a great idea actually needs an environment locked down sufficiently that he can't ruin his own day not a tool to analyze in real time whether he is doing so right now.

Those more skilled would be better off just using their wits.

The set between is empty.

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#49

Post by Barrin92 »

jointpdf wrote: post #30
Barrin92 wrote: post #21

Earlier quoted context omitted.

that's mostly because the bar is so unbelievably low that "it works 95% of the time" is good enough to please consumers. In security or anything that requires engineering rigor where catching the exception is exactly what matters ML is virtually useless, or even worse actively harmful.

That’s not true, there’s an entire subfield of stats/ML dedicated to anomaly detection.

But even outliers in ML systems are pretty much always outliers of first order, that is to say they're outliers in a predictable way, they're often significantly different from their environment.

The sort of outliers that concern security problems are pretty much always idiosyncratic by design because the people that create them know how easy it is to create adversarial examples for machines.

There's a human ingenuity to genuine edge cases that ML is ill suited to figure out because ML by design draws conclusions from patterns. My prediction is that we'll very soon see the same problem in fields like autonomous driving. Every time we see ML attack complex human domains, the "last 2%" seem intractable.

Re: Researchers easily trick Cylance's AI Antivirus to think Malware is 'Goodware'

#50

Post by jm4 »

gregmac wrote: post #47
jm4 wrote: post #38

Earlier quoted context omitted.

How does that usually work out? Personally, I would throw out a vendor who asked me to disable my AV. I don't mind whitelisting a directory or making certain adjustments, but it's hard to take them seriously if they can't work with AV. Cylance isn't that bad. I ran it for 3 years. The false positives were annoying, but it also stopped a lot of nasty stuff that our traditional AV wasn't detecting. I'm in an environmen…

What type of "nasty stuff" did it stop? Are you talking servers or workstations here? How did that nasty stuff get far enough in that it had to be "stopped" by AV? (I don't mean this to sound snarky, I'm genuinely interested in what type of real-world infections are happening and being detected. My experience is limited to internet-facing endpoints that have no security or are horribly outdated -- which is typically…

Mostly crap that was targeted at users via email. Weird droppers, infected documents, PDFs loaded with some exploit. Lots of stuff that doesn't show up on Virus Total. We see a lot of stuff that attacks unpatched software and there's quite a bit that gets through the email gateway. We see frequent spear phishing scenarios where it's not uncommon for the attacker to email back and forth for a while under some pretext before delivering a malicious payload. Several situations where a party we are communicating with has been breached and the attacker will use them as a jumping off point to get to us - basically a business email compromise scenario where they jump into the middle of a conversation or document transaction with a malicious payload. We patch too, block a lot of connections at the firewall and have other layers, but it's nice when the endpoint protection stops it.
Post reply on HN