Interesting post. Maybe to complete about adversarial examples in medicine https://medium.com/fitchain/attacking-deep-learning-models-3...
Attacks against machine learning – an overview
11–20 of 66 posts
Re: Attacks against machine learning – an overview
#12Very related: about a year ago, I wrote about weaknesses of neural networks specifically: https://matt.life/papers/security_privacy_neural_networks.pd... With powerful machine learning systems, we need to think about security a little differently. See especially the section 4.8 about function approximation: > Given a task for which no discrete algorithm is known to solve, there is a good chance a neural network can a…
Very well said, never thought about it in this way. It also nullifies a lot of propriety risk scoring models, like credit scores. I wonder what research is done around this for automated trading systems? I see an “attacker” that creates models who’s only purpose is to force another financial institution to make unprofitable trades based on reverse engineering the other traders trading modes. Eventually, if not already happening, trading becomes machines attacking other machines.
Re: Attacks against machine learning – an overview
#13Very related: about a year ago, I wrote about weaknesses of neural networks specifically: https://matt.life/papers/security_privacy_neural_networks.pd... With powerful machine learning systems, we need to think about security a little differently. See especially the section 4.8 about function approximation: > Given a task for which no discrete algorithm is known to solve, there is a good chance a neural network can a…
Very well said, never thought about it in this way. It also nullifies a lot of propriety risk scoring models, like credit scores. I wonder what research is done around this for automated trading systems? I see an “attacker” that creates models who’s only purpose is to force another financial institution to make unprofitable trades based on reverse engineering the other traders trading modes. Eventually, if not alread…
>Eventually, if not already happening, trading becomes machines attacking other machines.
Welcome to high frequency trading. You’re a bit late to the party though (around 15 years).
Re: Attacks against machine learning – an overview
#14It'd be great if there was a service that you could sign up for, which would "deceive" Facebook, Twitter, and other social media websites by producing false information about you. For example, if I don't want FB to know what movies I'm interested in, how about liking "random" movie pages on FB? If I don't want FB to know about my political orientations, how about run with the hare and hunt with the hounds?
AdNauseum[1] is that "service", but Google blocked it from the Chorome Add On Store for obvious reasons. [1]: https://adnauseam.io/
One of the many reasons I switched over to Firefox. Additionally, AdNauseum is an amazing product name.
Re: Attacks against machine learning – an overview
#15It'd be great if there was a service that you could sign up for, which would "deceive" Facebook, Twitter, and other social media websites by producing false information about you. For example, if I don't want FB to know what movies I'm interested in, how about liking "random" movie pages on FB? If I don't want FB to know about my political orientations, how about run with the hare and hunt with the hounds?
Not a happy answer but such a service would raise the noise floor but would otherwise not have much of an effect unless it was massively adopted to the point that the original signal was insignificant compared to the noise.
Re: Attacks against machine learning – an overview
#16I just watched a presentation about using deep learning to detect cheaters in CounterStrike: Go ( https://youtu.be/ObhK8lUfIlc ) and the question he didn't seem to have an answer for was data poisoning -- what if the cheaters all volunteer to be on the anti-cheater jury? Of course they are cross checking juror reliability ratings and stuff, but it's definitely a treadmill.
Thankfully, the majority of CSGO players (and overwatch reviewers) are not cheaters.
Re: Attacks against machine learning – an overview
#17I just watched a presentation about using deep learning to detect cheaters in CounterStrike: Go ( https://youtu.be/ObhK8lUfIlc ) and the question he didn't seem to have an answer for was data poisoning -- what if the cheaters all volunteer to be on the anti-cheater jury? Of course they are cross checking juror reliability ratings and stuff, but it's definitely a treadmill.
This is just like all the bad actors buying all the fast computers with BitCoin
Re: Attacks against machine learning – an overview
#18It'd be great if there was a service that you could sign up for, which would "deceive" Facebook, Twitter, and other social media websites by producing false information about you. For example, if I don't want FB to know what movies I'm interested in, how about liking "random" movie pages on FB? If I don't want FB to know about my political orientations, how about run with the hare and hunt with the hounds?
I suggested this to an EFF lawyer at Defcon and they hated the idea. Perhaps I worded it wrong, it makes sense to me. Signal to noise ratio. You should have a legal right to submit false information if you feel a service might harm you at some point.
Re: Attacks against machine learning – an overview
#19It'd be great if there was a service that you could sign up for, which would "deceive" Facebook, Twitter, and other social media websites by producing false information about you. For example, if I don't want FB to know what movies I'm interested in, how about liking "random" movie pages on FB? If I don't want FB to know about my political orientations, how about run with the hare and hunt with the hounds?
I suggested this to an EFF lawyer at Defcon and they hated the idea. Perhaps I worded it wrong, it makes sense to me. Signal to noise ratio. You should have a legal right to submit false information if you feel a service might harm you at some point.
Interesting, what did they hate about it?
Re: Attacks against machine learning – an overview
#20One simple way to minimize impact of these attacks is our work called Pixel Deflection (CVPR 2018 Spotlight). Here is a short (4 min) video introduction to the idea https://youtu.be/VgjOXJ9QKWo