Attacks against machine learning – an overview • Hacker News Board

Re: Attacks against machine learning – an overview

#61

Post by sacado2 » Mon, Jun 11, 2018, 1:07 PM UTC

pferde wrote: post #59

sacado2 wrote: post #55

Earlier quoted context omitted.

Exactly. In France people have been convicted because they "liked" illegal opinions. As if the fact that such a thing as an illegal opinion exists was not enough of a problem, it's been decided by justice that the semantic of a "like" was "I make this opinion mine".

Can you please share some links on this? All I could find was a similar case in Thailand: https://www.theguardian.com/world/2015/dec/10/thai-man-arres...

I found that, but this is in French, obviously: http://www.leparisien.fr/rozay-en-brie-77540/rozay-en-brie-c...

My own (approximate) translation of parts of the text:

"Sur Facebook, le trentenaire avait apposé un «J’aime» sur une image d’un combattant de Daesh brandissant la tête décapitée d’une femme. Il a été condamné à trois mois de prison avec sursis." --> "On Facebook, the man in his thirties had clicked "like" on a picture of an ISIS fighter holding the head of a beheaded woman. He was given a 3-month suspended prison sentence".

"«Quand on met J’aime, c’est que l’on considère que ce n’est pas choquant ou que l’on adhère», considère pour sa part Jean-Baptiste Bougerol, le substitut du procureur de la République." --> ""When you click "like" on something, you consider it's not shocking or you agree with it"", said the prosecutor".

Re: Attacks against machine learning – an overview

#62

Post by gwern » Mon, Jun 11, 2018, 2:19 PM UTC

mlthoughts2018 wrote: post #37

jacquesm wrote: post #15

Earlier quoted context omitted.

Not a happy answer but such a service would raise the noise floor but would otherwise not have much of an effect unless it was massively adopted to the point that the original signal was insignificant compared to the noise.

Even then it’s questionable if it would be effective. Over time even a faint behavioral signature will become transparent, because things that deviate from the true behavioral signature do so in random ways, which could essentially “cancel” out if your model for quantifying behavioral characteristics is well-specified. Meanwhile the “true” behaviors would “add” over time. It would become like any other signal jamming…

> I took a class with Scott Aaronson once where he mentioned the idea that the natural enemy of machine learning is cryptography.

He cites Rivest for that one: https://people.csail.mit.edu/rivest/pubs/Riv91.pdf

Re: Attacks against machine learning – an overview

#63

Post by poulsbohemian » Mon, Jun 11, 2018, 10:02 PM UTC

bhnmmhmd wrote: post #3

It'd be great if there was a service that you could sign up for, which would "deceive" Facebook, Twitter, and other social media websites by producing false information about you. For example, if I don't want FB to know what movies I'm interested in, how about liking "random" movie pages on FB? If I don't want FB to know about my political orientations, how about run with the hare and hunt with the hounds?

What's funny is how bad most of these services are at identifying people. They seem to think I'm a southeast Asian female (white dude here) because I happen to read a particular article, buy a particular item (probably as a gift), etc. Turn on, tune in, drop out...

Re: Attacks against machine learning – an overview

#64

Post by lurquer » Mon, Jun 11, 2018, 10:13 PM UTC

bhnmmhmd wrote: post #3

It'd be great if there was a service that you could sign up for, which would "deceive" Facebook, Twitter, and other social media websites by producing false information about you. For example, if I don't want FB to know what movies I'm interested in, how about liking "random" movie pages on FB? If I don't want FB to know about my political orientations, how about run with the hare and hunt with the hounds?

In the late 90s-early 00s, at a corporate job, several of us used a script to have the browser load a random page every few minutes. (I didn't write the script... I think the pages were just random links from search results based on random words.) Anyway, the thousands of pages loaded masked the NSFW pages. We figured we'd have plausible deniability. "You say I visited Xxx.com? I don't know... I think my computer has a virus or something. It's always loading up random stuff. Let me see the log... Yep... Just as I thought... It says I visited 24,239 pages on Tuesday. Heck, that's not even possible!"

Re: Attacks against machine learning – an overview

#65

Post by itronitron » Mon, Jun 11, 2018, 10:39 PM UTC

pferde wrote: post #50

itronitron wrote: post #43

Earlier quoted context omitted.

In order to wash out the signal, all the service would need to do is 'like everything'. In addition to masking your interests it would also grind their algorithms to a halt if enough people did that. A lot of these algorithms gain performance due to the sparsity of the data, so if everything became connected it would negatively impact the performance of their algorithms. Anyone know how to get, or compile, a list of…

Until you get into problems (legal or personal, doesn't matter) for "liking" stuff related to child porn, terrorist propaganda or, I don't know, scientology, without even knowing about it, because it was done on your behalf by this "like automaton".

anything likable on Facebook shouldn't be illegal, or illegal to like

Re: Attacks against machine learning – an overview

#66

Post by taurine » Wed, Jun 13, 2018, 4:13 PM UTC

Bromskloss wrote: post #60

taurine wrote: post #52

Earlier quoted context omitted.

There are not any, since there are no public stock market models worth copying, and no stock market model takes external input. But if they did (you could give a time-series to a model in the cloud, and it would give you predictions) then it would be possible. Copying models is a problem for cloud-hosted pay-per-prediction image classification, not for constantly retrained stock market models that don't take external…

I thought it would be about observing the behaviour of a system that is trading on the market. The input to the system would consist, for example, of other people's trades.

It is referring to https://arxiv.org/abs/1609.02943

What you are referring to is possible, but is not "copying" per se, just trying to infer what the system is doing (inverse RL), and then exploit that/make it do mistakes. If you are not HFT it is very difficult to distinguish bots from humans, so you'd have a hard time even finding a target.