Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-Consensual Intimate Image Pilot

newsroom.fb.com

Post reply on HN
Print view View original

31–40 of 184 posts

Re: Non-Consensual Intimate Image Pilot

#31

Post by praneshp »

leggomylibro wrote: post #28
alexggordon wrote: post #24

> To establish which image is of concern, people will be asked to send the image to themselves on Messenger. > Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it. This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to us…

I really don't understand why anyone uses Facebook. You have a phone, which can communicate with your friends. It's only remotely useful for its near-monopoly on event planning, and imo that's a job for the trustbusters.

> I really don't understand why anyone uses Facebook.

You really cannot? Like, can you not imagine one person, who doesn't realize (or worse, and more likely IMO, does not care) about the privacy implications?

It's not hard to understand at all, I think.

Re: Non-Consensual Intimate Image Pilot

#32

Post by Pulcinella »

orastor wrote: post #13

To everyone asking why they need a human to review to image before hashing it: it's currently the only way to prevent abuse of this system -- Without this, everyone would upload random images that would get taken down, effectively mass-trolling

At most, they would only need a human to review after the hash matches, not before.

Facebooks plan is awful.

Re: Non-Consensual Intimate Image Pilot

#33

Post by danbruc »

alexggordon wrote: post #24

> To establish which image is of concern, people will be asked to send the image to themselves on Messenger. > Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it. This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to us…

This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees.

You can view and search your message history on Facebook, so there is not really any question that they have and can read all messages. I never explicitly checked it but at least I never noticed any gaps, i.e. missing messages because I sent them using the Messenger app.

I believe to remember that Messenger is supposed to use Axolotl Ratchet for end-to-end encryption but that is hard to reconcile with the availability of your message history on facebook.com. So maybe it's - quote - end-to-end - unquote - between the phone and the server?

I never really thought about that. Or is it just not available or disabled in my Windows Phone version of the app?

Re: Non-Consensual Intimate Image Pilot

#34

Post by evgen »

BinaryIdiot wrote: post #21
Apreche wrote: post #12

Ok, so now all the harassers out there are just going to edit a few pixels on the images before uploading, so the hash isn't caught. All the same tricks use don YouTube to avoid the copyright bot, will work here as well.

Yeah I'm curious how their hashing mechanism works. Like, is it a straight up sha hash of the image or something a little more sophisticated that wouldn't be fooled by minor edits? Though in the latter case can that even technically be considered hashing? I guess it all depends on how they're doing it which brings us back to my first question...

It is called PhotoDNA and was developed by Microsoft. This is the same tech that is used to find child porn images even when people go to various lengths to prevent simple hashing from being able to determine photo similarity. It works surprisingly well at the task from what I have seen/heard.

Re: Non-Consensual Intimate Image Pilot

#35

Post by alexggordon »

danbruc wrote: post #33
alexggordon wrote: post #24

> To establish which image is of concern, people will be asked to send the image to themselves on Messenger. > Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it. This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to us…

This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees. You can view and search your message history on Facebook, so there is not really any question that they have and can read all messages. I never explicitly checked it but at least I never noticed any gaps, i.e…

It's not that infeasible to have a web platform that encrypts users messages, just each client gets an encryption key. Look at Telegram, or even FB Messenger end to end encryption. I'm not saying they ever said they didn't have the ability to do it, I'm saying it's weird that they would use "message snooping" as a form of user photo submission.

Re: Non-Consensual Intimate Image Pilot

#36

Post by FRex »

BinaryIdiot wrote: post #21
Apreche wrote: post #12

Ok, so now all the harassers out there are just going to edit a few pixels on the images before uploading, so the hash isn't caught. All the same tricks use don YouTube to avoid the copyright bot, will work here as well.

Yeah I'm curious how their hashing mechanism works. Like, is it a straight up sha hash of the image or something a little more sophisticated that wouldn't be fooled by minor edits? Though in the latter case can that even technically be considered hashing? I guess it all depends on how they're doing it which brings us back to my first question...

Hash by strictest definition is just a function that maps any input to fixed size output. There is a special subcategory of hashes that are locality sensitive and change a little when input changes a little, just like there is a special subcategory of crypto hashes. See: https://en.wikipedia.org/wiki/Locality-sensitive_hashing

Technology for searching for similar images (I have no idea how it works interally) already exists and is very widely deployed. You can easily find an image similar (uncropped, cropped, with different filter, greyed, ungreyed, with a logo in corner, etc.) to your input image on https://tineye.com/ and https://images.google.com .

Re: Non-Consensual Intimate Image Pilot

#37

Post by evgen »

pault wrote: post #4

This is insane. If they can fingerprint your images, why can't they provide a tool for processing the image and sending the fingerprint only? Can you imagine how big of a target a tailor made database of blackmail material linked to facebook accounts would be?

In addition to the reasons other have suggested, one reason that this is not done client side is that the PhotoDNA tech that is used for the fingerprinting is not something that Facebook can share or make available. It's primary use is in matching posted images with known databases of child porn images, so providing a mechanism for someone to reverse engineer the system and develop a means of bypassing it is not going to happen.

Re: Non-Consensual Intimate Image Pilot

#38

Post by LukaAl »

As other has pointed out, this is extremely creepy. The easiest solution would have been this one:

- The user uses a JS solution to hash the images on the client, without the image being uploaded

- She compiles a form with additional information (e.g: capture her account, reasons for uploading, suspect person sharing the picture)

- The picture is saved in the DB as un-verified revenge porn.

- The first time someone uploads a picture that matches the hash, the pic is quarantined and the specially trained individual manually check them

- A scoring system could be used to check the reliability of the submission. If multiple photos marked revenge porn get rejected, the control becomes ex-post. For even more violations, the user get banned from using the tool and should directly contact Facebook. Submitting the same hash that has been rejected, will count as a "red mark"

Now, I understand this system is very complex, what Facebook has done is an MVP and as a product manager, this is what I prefer. But considering the issue (revenge porn, not something I necessarily want to test the impact on retention :-) ). Also, yes, it requires resources, but Facebook has a problem with trust lately, better to do the best...

[edited for formatting]

Re: Non-Consensual Intimate Image Pilot

#39

Post by danbruc »

alexggordon wrote: post #35
danbruc wrote: post #33

Earlier quoted context omitted.

This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees. You can view and search your message history on Facebook, so there is not really any question that they have and can read all messages. I never explicitly checked it but at least I never noticed any gaps, i.e…

It's not that infeasible to have a web platform that encrypts users messages, just each client gets an encryption key. Look at Telegram, or even FB Messenger end to end encryption. I'm not saying they ever said they didn't have the ability to do it, I'm saying it's weird that they would use "message snooping" as a form of user photo submission.

When I send messages using the Messenger app - which is supposed to use end-to-end encryption, isn't it? - I can see and search those messages on facebook.com. Something does not add up here. And if they have the keys on their servers, then it's rather pointless to use end-to-end encryption in the first place.

Re: Non-Consensual Intimate Image Pilot

#40

Post by evgen »

tonyarkles wrote: post #6
moyix wrote: post #3

Earlier quoted context omitted.

From the fine article: "We store the photo hash—not the photo—to prevent someone from uploading the photo in the future. If someone tries to upload the image to our platform, like all photos on Facebook, it is run through a database of these hashes and if it matches we do not allow it to be posted or shared."

Is there a way for me to audit that the image is never stored? (No) It seems like that's a huge honeypot; someone compromises the "upload your naughty photos here" endpoint and has an endless supply of "non-consensual intimate images". Edit: Thinking about it a bit, I'd be way more comfortable with the idea of the image hashes being computed on-device and only the hashes being sent to the server. This opens a differe…

While this (client-side hashing) would be better in terms of privacy protection for this specific case it is not going to happen because Facebook is not allowed to perform the specific photo hashing client-side as this would expose the hashing mechanism to analysis.
Post reply on HN