Live data from Hacker News

Bumble releases open-source A.I. feature to help combat unsolicited nudes

bumble.com

71–80 of 255 posts

Re: Bumble releases open-source A.I. feature to help combat unsolicited nudes

#72

> AI tool to detect unsolicited lewd images Does this mean it can also detect "send nudes" to avoid false positives on solicited lewd images? Or does the UI have a on/off toggle for currently-accepting-lewd-images?

If it does I hope they were considerate enough of our Australian friends to have it detect "sapnu puas" as well.

Re: Bumble releases open-source A.I. feature to help combat unsolicited nudes

#73
post #50
post #32

Can they open source their algorithirm so I can figure out why I get no matches

does anyone actually use bumble and enjoy it? serious question

had a great time on it, culminating in meeting my wife

Re: Bumble releases open-source A.I. feature to help combat unsolicited nudes

#75

A lot of their copy seems to indicate the priority is detecting dick picks. I realize those are the vast, vast majority of "unsolicited lewd images", but I do wonder how capable their model is at detecting pussy pics, if at all.

The dev team had a large training set for male anatomy due to availability and a large training set for female anatomy due to determination.

Re: Bumble releases open-source A.I. feature to help combat unsolicited nudes

#76

Many years ago (ca. 2009?) at a previous employer we had an enterprise email scanner which included a content filtering feature which could block "pornographic images". We ended up turning it off as it had too many false positives since it was an insurance company that dealt with injuries so a lot of legitimate injury photos would get classed as porn and blocked. I think it was basically looking at how much of an ima…

> aggressively block photos of golden retrievers

That gave me a good laugh

Re: Bumble releases open-source A.I. feature to help combat unsolicited nudes

#77

I wonder if there is a way to use this to finetune an image generating AI to generate images that the AI would think is lewd but is not. (similar to what was done with a Yahoo model in 2016: https://news.ycombinator.com/item?id=12756462 ) Archive of the original post (NSFW?): https://web.archive.org/web/20161022003102/https://open_nsfw...

More likely is people making adversarial dick pics to get past the filter, which is very quick to do when you have the parameters of a differentiable model. I noticed they were careful to state that they are releasing a model instead of the model they are using though.

> adversarial dick pics

Well, I wasn’t planning on encountering that phrase today, perhaps ever to be honest

Re: Bumble releases open-source A.I. feature to help combat unsolicited nudes

#78

Funnily enough one of my SFW pictures got detected recently on bumble and my working theory is that they detected my phone as a bulge. It might be fun to try to test what type of content it misidentifies. Edit: missing "to"

20 years ago when candy bar phones were popular i designed a dildo-shaped phone case – a sort of “push up bra” for men. Not sure why my product never took off.

Re: Bumble releases open-source A.I. feature to help combat unsolicited nudes

#80

I wonder if there is a way to use this to finetune an image generating AI to generate images that the AI would think is lewd but is not. (similar to what was done with a Yahoo model in 2016: https://news.ycombinator.com/item?id=12756462 ) Archive of the original post (NSFW?): https://web.archive.org/web/20161022003102/https://open_nsfw...

More likely is people making adversarial dick pics to get past the filter, which is very quick to do when you have the parameters of a differentiable model. I noticed they were careful to state that they are releasing a model instead of the model they are using though.

Granted, I don't understand the psychology of people who send dick pics so I could be wrong about this, but I get the impression it's not a deeply considered thing conducive to spending lots of effort on. Adversarial dick pics are probably a rare threat just because most men won't be bothered.
Post reply on HN