Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-Consensual Intimate Image Pilot

newsroom.fb.com

Post reply on HN
Print view View original

131–140 of 184 posts

Re: Non-Consensual Intimate Image Pilot

#131

Post by LukaAl »

_red wrote: post #104
LukaAl wrote: post #95

Earlier quoted context omitted.

From when security through obscurity is a good idea?

You should go put a $100 bill on the dashboard of your car along with your laptop on the front seat. Report back the results. snark-mode off: We use obscurity every day and its a completely valid layer of security.

> We use obscurity every day and its a completely valid layer of security.

Not sure I agree with that, most of the time when we do that is because we don't want to spend the time to have better security. And then we get burned. To your example of the 100$ bill: at my parents home with the car parked in the garage? No problem to do that at all. Out on the street in SF? No. I don't trust my glass enough as a security measure. But I don't leave money at all, is not security through obscurity.

But we are going OT. The problem that is raised is that they need necessarily security through obscurity. And we have two problems:

- How really robust are these algorithms? How long before we will see people abusing them?

- Have you thought hard enough about how this system could work? E.g: have a partial hashing made client-side and the final one on the server? Or a situation where the server code is open-sourced without the model to calculate the hash? That would allow for external review without disclosing the hash. Yes, you still need warranties that Facebook is using that code, but you could have a trusted third party certifying the program.

My point is, the person who designed this program didn't really understand the problem. The problem is not revenge porn. The problem is Facebook reputation. And this solution is totally deaf.

Re: Non-Consensual Intimate Image Pilot

#132

Post by dredmorbius »

15155 wrote: post #128
dredmorbius wrote: post #105

Earlier quoted context omitted.

Consider that this situation could extend or apply to humans, animals, or even plants. Does a rhinocerous have a right to be forgotten? https://www.reddit.com/r/dredmorbius/comments/25ll1v/does_a_...

I think this is a separate philosophical debate: operating under current legal frameworks in the United States of America, you do not have the right to prevent me from publishing my photo (with no demonstrable harm being done.)

The purpose of a discussion is to determine whether or not those legal frameworks are appropriate to the situation.

The law is a tool, not an ends.

Re: Non-Consensual Intimate Image Pilot

#133

Post by scoot »

> Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image

At first I wondered "Why?!" Why does a "specially trained" person have to look at the very same intimate images that their users wish to prevent being shared.

Then it struck me that without review, any image could be blocked, simply by submitting it through this process. This would also explain why the offline hashing approaches being suggested aren't practical - they're just too prone to abuse by flagging images that are not of a sexual nature, but that you would like to see blocked for whatever reason.

An ML approach to masking the most sensitive portions of the submitted images would both protect the user, and reduce the burden on those tasked with reviewing the submitted images.

If this could be handled on the client, and sent with a verifiable hash of the original image, that would solve the majority of concerns raised in the discussions here.

Re: Non-Consensual Intimate Image Pilot

#134

Post by reificator »

JauntyHatAngle wrote: post #123
Dylan16807 wrote: post #106

Earlier quoted context omitted.

That system works the same way as what they're implementing. Someone reviews the raw image and enters the hash into the system.

>Someone reviews the raw image and enters the hash into the system. Now that is a job I really would not want to have. Talk about mentally scarring.

Isn't that one of the highest turnover jobs that exists? It's really hard to keep people coming into work when their work is to review twisted content all day.

Re: Non-Consensual Intimate Image Pilot

#135

Post by dude01 »

wellboy wrote: post #84

Ok, I had to laugh. FB is literally the worst company to make this service with the worst privacy reputation ever. Then the service makes the worst mistake that they actually upload nudes to their platform, PLUS they have a human look at it, which adds another layer of security leaks. Did they do this intentionally or did they have such a huge lack of common sense when they decided how to design this and nobody in th…

I think they want to see how much can they get away with. Let's see if we can actually get people to send us all their private porn.

Re: Non-Consensual Intimate Image Pilot

#136

Post by yawaramin »

valuearb wrote: post #120
trjordan wrote: post #115

Earlier quoted context omitted.

There was an excellent Twitter thread on why they didn't do this: https://twitter.com/alexstamos/status/928740488395608065 The salient bit: > A quick note to everybody who says 'calculate the hash locally': > A) Photo fingerprinting algorithms are usually not included in clients to prevent the development of circumvention techniques. > B) Humans need to review to prevent adversarial reporting. Further down the thread…

It's not a hard problem. Both A & B are bogus reasons. A) Any hash is going to be easily circumvented by anyone who owns Photoshop or any image editor. It's only going to catch those dumb enough not to modify the images, ie. most everyone. Doing the hash server side isn't the big problem though, because if you trust the Facebook App then by default you should trust the Facebook web applications. B) The big problem is…

Hey, please don't use phrasing like '... infected with ... autism'. First of all, autism isn't an infection, and second it dehumanises people with autism when you associate them with negative connotations like not being aware of human feelings and 'infecting' others. They are people too, don't dump on them.

Re: Non-Consensual Intimate Image Pilot

#137

Post by kelnos »

dmitrygr wrote: post #64
evgen wrote: post #40

Earlier quoted context omitted.

While this (client-side hashing) would be better in terms of privacy protection for this specific case it is not going to happen because Facebook is not allowed to perform the specific photo hashing client-side as this would expose the hashing mechanism to analysis.

> this would expose the hashing mechanism to analysis Security through obscurity. Works every time (tm)

I mean, it worked pretty well for Google's ranking algorithm, so much so that it spawned an entire industry (SEO, which still doesn't let you get away with much in the way of black-hat type stuff).

Even just the rate-limiting aspect of keeping the algo on the server (in this case behind a human, but that's even not necessary) is useful: I can't sit offline with the algo and keep beating it with slightly different images until the hash changes enough to be considered unique.

Re: Non-Consensual Intimate Image Pilot

#139

Post by alsetmusic »

pavel_lishin wrote: post #130
kelnos wrote: post #121

Earlier quoted context omitted.

A) I think we're talking about modification-resistant fingerprinting methods, not just a simple hash of the data. B) That's a support nightmare; it would be nearly trivial to start submitting false reports from a ton of accounts without establishing a pattern so those accounts could easily be ignored. Regardless, how do you prove that you "own an image" (what does that even mean?) and that it was reported as an "inti…

> That's a support nightmare; it would be nearly trivial to start submitting false reports from a ton of accounts without establishing a pattern so those accounts could easily be ignored. What's stopping me from flooding false reports now ? Arguably, only reviewing the image they already have once a hash matches is less work, since they wouldn't have to review all incoming reports, only matched ones.

> Arguably, only reviewing the image they already have once a hash matches is less work, since they wouldn't have to review all incoming reports, only matched ones.

They would still need to investigate reports that didn’t match known hashes, but you’re right that this would be beneficial. Images that match a known hash could be prioritized during review.

Re: Non-Consensual Intimate Image Pilot

#140

Post by beager »

giobox wrote: post #51
moyix wrote: post #3

Earlier quoted context omitted.

From the fine article: "We store the photo hash—not the photo—to prevent someone from uploading the photo in the future. If someone tries to upload the image to our platform, like all photos on Facebook, it is run through a database of these hashes and if it matches we do not allow it to be posted or shared."

All you can do is take FB at their word that the image is destroyed irrevocably. Given what we know about data retention policies at big tech firms, I'm not so sure I would feel confident taking the press release at face value. I'd really like to see a white paper or similar outlining the specifics of how this is being handled. I'd also have questions around what steps have been taken to prevent rogue Facebook employ…

I don’t know. I think a whitepaper might just be a drier way to lie about what they’re really doing.
Post reply on HN