Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-Consensual Intimate Image Pilot

newsroom.fb.com

Post reply on HN
Print view View original

111–120 of 184 posts

Re: Non-Consensual Intimate Image Pilot

#111

Post by spicyj »

alexggordon wrote: post #24

> To establish which image is of concern, people will be asked to send the image to themselves on Messenger. > Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it. This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to us…

I don’t think that’s accurate. I believe for this specific feature you need to specifically report the message from within the Messenger UI, similarly to how you would report spam or other abuse. Only then is it sent to the human reps. I don’t think other messages are ever read.

I’m a FB employee and it’s definitely not possible for employees to easily snoop on random messages. Even if you found some way that isn’t properly protected, you’d get promptly fired if caught.

Re: Non-Consensual Intimate Image Pilot

#112

Post by syrrim »

LukaAl wrote: post #95
moyix wrote: post #78

Earlier quoted context omitted.

Honestly, I doubt that most of these sorts of algorithms would survive concerted attacks – that's why they tend to be closely guarded. Alex Stamos (Facebook's CISO) implies this is why they can't do it client-side: https://twitter.com/alexstamos/status/928646228472078336

From when security through obscurity is a good idea?

This is in general an unsolvable problem, and the only possible security is through obscurity. Images are on a spectrum, and so in order to hash an image, you must quantize the spectrum. Between two images, there is a continuum of mixed images. Using bisection, you can identify the point where it goes from matching to not matching. You can then upload the non-matching variant. Because of the way hashing works, this will be indistinguishable from any other non-matching image. Therefore, this creates an easy method to bypass facebook's filter while sacrificing minimal quality. As long as facebook matches against hashes, there is way to prevent bypassing of this form. All they can do is make it as difficult as possible.

Re: Non-Consensual Intimate Image Pilot

#113

Post by [deleted] »

evgen wrote: post #41
LukaAl wrote: post #38

As other has pointed out, this is extremely creepy. The easiest solution would have been this one: - The user uses a JS solution to hash the images on the client, without the image being uploaded - She compiles a form with additional information (e.g: capture her account, reasons for uploading, suspect person sharing the picture) - The picture is saved in the DB as un-verified revenge porn. - The first time someone u…

Now figure out a way to do step #1 (user does a hash client-side) without making it trivial for someone else to create a filter that adds enough noise to invalidate step #4 (uploaded pics that match a hash are quarantined.)

[deleted]

Re: Non-Consensual Intimate Image Pilot

#114

Post by theptip »

ghshephard wrote: post #92
Xeoncross wrote: post #60

The better solution is to have the image hashed and NOT sent to facebook. If the hash matches an existing image [A]. If the hash isn't found [B]. [A]: human can review the image they already have [B]: Facebook waits until someone uploads an image that matches and then reviews the image (as normal) but with a marker alerting the problem. The benefit is that people NOT affected won't have to upload lots of images of th…

The obvious reason why Facebook didn't go down this path is one of scale - It's easy (and altogether likely) for an attacker to pollute the hashes with innocuous pictures. But, even more importantly, once you reveal your hashing system, it's relatively trivial for people to work around them and modify the pictures in such a way as to render the hashing system useless. Has to be done server side for these reasons.

Could you elaborate why you think it's easy to pollute hashes/fingerprints in this scheme? It seems to me that it would be hard to do reliably, since each hash submitted is going to be linked to an individual account. If an account submitted a bunch of hashes that matched non-explicit images, all further hash submissions from that user account could be rejected. (In the GP's proposed scheme a human can still verify a sample of all positive matches, to validate if they are true or false positives).

I think the point about revealing your fingerprinting system is a good one. But I wonder if the equilibrium point would still be better than what we have now, i.e. unsophisticated posters would get caught, and sophisticated ones might not, vs. what we have now where there is no way to catch any of these posts.

Re: Non-Consensual Intimate Image Pilot

#115

Post by trjordan »

Xeoncross wrote: post #60

The better solution is to have the image hashed and NOT sent to facebook. If the hash matches an existing image [A]. If the hash isn't found [B]. [A]: human can review the image they already have [B]: Facebook waits until someone uploads an image that matches and then reviews the image (as normal) but with a marker alerting the problem. The benefit is that people NOT affected won't have to upload lots of images of th…

There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065

The salient bit:

> A quick note to everybody who says 'calculate the hash locally':

> A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques.

> B) Humans need to review to prevent adversarial reporting.

Further down the thread, they discuss further that PhotoDNA and similar algorithms are, in particular, not resistant to being tricked. Until they are, it's another layer to prevent adversaries from fully decomposing how they're hashing it, which is a good thing.

It's a trade-off, for sure, but this is a hard problem with no cut and dry answer. Facebook already has the images in question, because you have to be sending them to somebody in order to mark them as NCII. Asking yet another Facebook service to look at them without storing them seems reasonable to me. You may disagree.

Re: Non-Consensual Intimate Image Pilot

#116

Post by yjftsjthsd-h »

15155 wrote: post #97
yjftsjthsd-h wrote: post #94

Earlier quoted context omitted.

Excellent! Then it should be short work for you to photoshop my face out so it won't be blocked. Everybody wins.

Why is Facebook obligated to fulfill your desires for privacy by censoring content you have no legal claim to?

I think the claim is that I should have a legal claim to my likeness.

Note that I'm not personally convinced of this, I just felt that it was really weak to say that it was fine because the faces aren't an important part of the picture.

Re: Non-Consensual Intimate Image Pilot

#118

Post by leggomylibro »

praneshp wrote: post #31
leggomylibro wrote: post #28

Earlier quoted context omitted.

I really don't understand why anyone uses Facebook. You have a phone, which can communicate with your friends. It's only remotely useful for its near-monopoly on event planning, and imo that's a job for the trustbusters.

> I really don't understand why anyone uses Facebook. You really cannot? Like, can you not imagine one person, who doesn't realize (or worse, and more likely IMO, does not care) about the privacy implications? It's not hard to understand at all, I think.

Okay, fine, I don't want to understand; it jives against the charitable way that I want to think about people.

I guess there's a whole lot of cognitive dissonance between how fucking stupid People reliably are and how kind and admirable persons can be, though - I should probably just get used to it.

Re: Non-Consensual Intimate Image Pilot

#119

Post by gdulli »

f_allwein wrote: post #117

Even if this worked as intended, what’s the point? The person wanting to spread such images could just use another platform to do so, or modify the image slightly... Would this really stop much abuse?

The same hubris that lets Facebook believe they can do this without it backfiring on them is the same hubris that lets them believe that theirs is the only platform that matters.

Re: Non-Consensual Intimate Image Pilot

#120

Post by valuearb »

trjordan wrote: post #115
Xeoncross wrote: post #60

The better solution is to have the image hashed and NOT sent to facebook. If the hash matches an existing image [A]. If the hash isn't found [B]. [A]: human can review the image they already have [B]: Facebook waits until someone uploads an image that matches and then reviews the image (as normal) but with a marker alerting the problem. The benefit is that people NOT affected won't have to upload lots of images of th…

There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065 The salient bit: > A quick note to everybody who says 'calculate the hash locally': > A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques. > B) Humans need to review to prevent adversarial reporting. Further down the thread…

It's not a hard problem. Both A & B are bogus reasons.

A) Any hash is going to be easily circumvented by anyone who owns Photoshop or any image editor. It's only going to catch those dumb enough not to modify the images, ie. most everyone. Doing the hash server side isn't the big problem though, because if you trust the Facebook App then by default you should trust the Facebook web applications.

B) The big problem is you take care of adversarial reporting at the end, not the beginning. If they get a hit on a reported hash, only then do they tell the photo owner and ask for proof that it's their image.

It's bonkers that Facebook expects people terrified of their photos being circulated to send them those images so a Facebook employee can eye fondle them.

This is just par for the course for Facebook, as a business they are so tone death and oblivious to basic human behavior and feelings that it seems like the entire company is infected with the same alleged autism of it's founder.

Post reply on HN