Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-Consensual Intimate Image Pilot

newsroom.fb.com

Post reply on HN
Print view View original

21–30 of 184 posts

Re: Non-Consensual Intimate Image Pilot

#21

Post by BinaryIdiot »

Apreche wrote: post #12

Ok, so now all the harassers out there are just going to edit a few pixels on the images before uploading, so the hash isn't caught. All the same tricks use don YouTube to avoid the copyright bot, will work here as well.

Yeah I'm curious how their hashing mechanism works. Like, is it a straight up sha hash of the image or something a little more sophisticated that wouldn't be fooled by minor edits? Though in the latter case can that even technically be considered hashing? I guess it all depends on how they're doing it which brings us back to my first question...

Re: Non-Consensual Intimate Image Pilot

#22

Post by tantalor »

BinaryIdiot wrote: post #20
marrone12 wrote: post #9

Why do they need a human representative to hash the image? I don't understand why this can't all be done automatically. Creeps me out that a real human would see intimate pictures before they block them.

I'm _assuming_ the only reason a human representative is doing this is so the system doesn't get abused. Like say I upload a photo claiming it is of myself but it's actually the advertisement a competitor of mine is using. If they didn't verify it was something that SHOULD be removed they'd automatically remove my competitor's posting of their advertisement every time they posted it. Granted I really wish this was al…

they'd automatically remove my competitor's posting of their advertisement every time they posted it

Since that will be very rare case why not make that the human-required step?

Re: Non-Consensual Intimate Image Pilot

#23

Post by sp332 »

beaconstudios wrote: post #14
sp332 wrote: post #10

Earlier quoted context omitted.

Revenge porn isn't all about nudity. And this does cover Messenger (and Instagram). The point isn't to get an image taken down after the fact, but to proactively block it from being posted or sent in the first place.

I thought facebook would run posted images through their AI nudity filter before allowing them to be posted? It's good that this covers messenger as I've read stories in the past of people being blackmailed over it, but surely there must be a better way than asking people to upload their private images to facebook. Plus, what's the case with younger facebook members? Do they upload the images and are therefore sharin…

I didn't know Facebook had an AI nudity filter, I thought it was all based on reported images. They softened their stance on some things, especially after censoring a Pulizter-prize winning photo from Vietnam, and eventually changed their policy on breastfeeding too https://www.facebook.com/help/340974655932193/

I'm sure they don't have a legal exception for underage photos, but those laws do vary from state to state.

Re: Non-Consensual Intimate Image Pilot

#24

Post by alexggordon »

> To establish which image is of concern, people will be asked to send the image to themselves on Messenger.

> Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it.

This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to user (even though it might limited to messages to oneself), and exposes that ability to its employees.

While I stopped using Facebook a long time ago, Zuckerberg's quote, "they 'trust me'; dumb fucks" seems relevant here. If I were Facebook, I would do this in a different, much more privacy conscious way.

1. Obviously, Facebook is already hashing all images on its platforms and storing the hashes. Given that, let the user, using a frontend only platform, upload an image and generate the hash. Make sure this image is a real photo (not a meme or something heavily photoshopped) in addition to making sure:

1a) The image contains a human[0]. Fairly easily doable for most situations.

1b) If not, let the user know that "we can't automatically detect a human in this photo, do you want to still submit this photo for removal, this could get a strike against your account if (not in some list of reasons for removal)". Maybe the photo has a picture of a credit card or something.

2. Submit the hash to the backend. Given that hash is not used on the platform past a significant amount of views, automatically ban the photo, and allow other users to petition for reinstatement of the photo, knowing that a non-valid petition for reinstatement (the user doesn't have rights to the photo) COULD result in a strike also.

3. Have humans review the petitions, and in cases where the users explicitly allow, the images.

Doing this would limit another human seeing the intimate photo in well over 90% of the cases I'd bet. In addition to that, even if the photo is shared with another human, it would let the end user decide if they'd like a Facebook employee to view the image too. IE, basic user privacy. Come on Facebook. Dumb fucks.

[0] very doable, https://trackingjs.com/

Re: Non-Consensual Intimate Image Pilot

#25

Post by pjc50 »

BinaryIdiot wrote: post #20
marrone12 wrote: post #9

Why do they need a human representative to hash the image? I don't understand why this can't all be done automatically. Creeps me out that a real human would see intimate pictures before they block them.

I'm _assuming_ the only reason a human representative is doing this is so the system doesn't get abused. Like say I upload a photo claiming it is of myself but it's actually the advertisement a competitor of mine is using. If they didn't verify it was something that SHOULD be removed they'd automatically remove my competitor's posting of their advertisement every time they posted it. Granted I really wish this was al…

I'd assume that would be dealt with by an appeals process on the other end; if you post an image and it's flagged, you should be able to say "this is obviously not porn" and get it posted and the original false claimer should lose trust points.

Re: Non-Consensual Intimate Image Pilot

#26

Post by leggomylibro »

marrone12 wrote: post #9

Why do they need a human representative to hash the image? I don't understand why this can't all be done automatically. Creeps me out that a real human would see intimate pictures before they block them.

Because otherwise the platform will not be able to store or display images at all.

Report, report, report, report, report - you know someone will do it, or make a bot to.

Or they would need to put actual effort into developing a novel solution to a new-ish problem. But they're a large incumbent, so...no.

Re: Non-Consensual Intimate Image Pilot

#27

Post by baddox »

Apreche wrote: post #12

Ok, so now all the harassers out there are just going to edit a few pixels on the images before uploading, so the hash isn't caught. All the same tricks use don YouTube to avoid the copyright bot, will work here as well.

I’m not saying that people won’t be able to fool it, but I’m sure it would require way more than just changing a few pixels. I’ve seen YouTube videos that were clearly edited to circumvent the copyright bot, and the video transformations were so significant that the videos were essentially unwatchable.

Re: Non-Consensual Intimate Image Pilot

#28

Post by leggomylibro »

alexggordon wrote: post #24

> To establish which image is of concern, people will be asked to send the image to themselves on Messenger. > Once we receive this notification, a specially trained representative from our Community Operations team reviews and hashes the image, which creates a human-unreadable, numerical fingerprint of it. This clearly implicates that Facebook has, almost without reserve, the ability to read messages from user to us…

I really don't understand why anyone uses Facebook.

You have a phone, which can communicate with your friends. It's only remotely useful for its near-monopoly on event planning, and imo that's a job for the trustbusters.

Re: Non-Consensual Intimate Image Pilot

#29

Post by BinaryIdiot »

pjc50 wrote: post #25
BinaryIdiot wrote: post #20

Earlier quoted context omitted.

I'm _assuming_ the only reason a human representative is doing this is so the system doesn't get abused. Like say I upload a photo claiming it is of myself but it's actually the advertisement a competitor of mine is using. If they didn't verify it was something that SHOULD be removed they'd automatically remove my competitor's posting of their advertisement every time they posted it. Granted I really wish this was al…

I'd assume that would be dealt with by an appeals process on the other end; if you post an image and it's flagged, you should be able to say "this is obviously not porn" and get it posted and the original false claimer should lose trust points.

Problem is it's significantly easier and faster to create noise than it is to clean up the noise. Dropping the up front review in favor of an appeals only system sounds impossible to handle at Facebook scale, IMO. Every time you shut someone's account down for losing too much trust, 10 others would have already replaced it.

Re: Non-Consensual Intimate Image Pilot

#30

Post by BinaryIdiot »

tantalor wrote: post #22
BinaryIdiot wrote: post #20

Earlier quoted context omitted.

I'm _assuming_ the only reason a human representative is doing this is so the system doesn't get abused. Like say I upload a photo claiming it is of myself but it's actually the advertisement a competitor of mine is using. If they didn't verify it was something that SHOULD be removed they'd automatically remove my competitor's posting of their advertisement every time they posted it. Granted I really wish this was al…

they'd automatically remove my competitor's posting of their advertisement every time they posted it Since that will be very rare case why not make that the human-required step?

Why would that be a "very rare" case? Facebook and Twitter have (or have had) hundreds of thousands of bots and constant abuses, why would this feature be rarely abused?
Post reply on HN