You either die an MVP or live long enough to build content moderation • Hacker News Board

Re: You either die an MVP or live long enough to build content moderation

#191

Post by nerd_light » Tue, Sep 28, 2021, 8:05 PM UTC

Alir3z4 wrote: post #110

nerd_light wrote: post #88

Earlier quoted context omitted.

I think they meant (and I am interested in hearing about) appealing a "block" decision that was made by your automation. If I'm a real human and trying to post a "good" post, but the model classifies it as bad and automatically blocks it, how do I appeal that decision? Can I? Or is my post totally blocked with no recourse?

Oh got it. Thanks for clarification. When a post gets published, it will be send to machine learning image via REST. If bad, the post will be kept as Draft. A new record gets created in another database table to keep track them, the accuracy rate was recorded as well. This was made to make sure no irreversible action was done on the good content. Blogs with more than 1 year of history would not go through moderation…

Thanks for the detailed response! And nice to hear how much you've managed to keep humans involved in the process. I used to work on a content review automation system for a big company, so it's always fun to hear about how others handle similar cases.

And there's a lot of overlap between how that system acted and what you're describing. It makes we wonder if there's space for a company that offers this sort of model training + content tagging + review tooling capability as a service, or if there's too many variation on what "good" and "bad" input is to make it generalizable.

Re: You either die an MVP or live long enough to build content moderation

#192

Post by quantumBerry » Tue, Sep 28, 2021, 8:09 PM UTC

jjulius wrote: post #188

quantumBerry wrote: post #172

Earlier quoted context omitted.

I suppose in the sense that Gabby Pettito was moderated off the internet, Ross Ulbricht was moderated off of the internet and into a cage permanently for the heinous crime of facilitating voluntarily peaceful trade. Tor marketplaces were definitely not gone for years, the same content just moved under new banners. You can literally find the same content and more on WHM today as you did under Ulbricht's banner before…

>I suppose in the sense that Gabby Pettito was moderated off the internet, Ross Ulbricht was moderated off of the internet and into a cage permanently for the heinous crime of facilitating voluntarily peaceful trade. Oh hello, strawman. >Tor marketplaces were definitely not gone for years, the same content just moved under new banners. You can literally find the same content and more on WHM today as you did under Ulb…

>Oh hello, strawman.

Glad to know you finally admit that being kidnapped by a 3rd party is not really what most of us think as "moderation", and thus you have made a straw man. Although in the strict sense I guess it is true that moderation could merely mean some 3rd party entity came along and violently kept me away from communicating. If you don't like me posting cat pictures on reddit, you could crack my skull or lock me in a cage and steal my PC and you would have "moderated" me but I wouldn't call that reddit moderation.

Re: You either die an MVP or live long enough to build content moderation

#193

Post by wffurr » Tue, Sep 28, 2021, 8:09 PM UTC

jonshariat wrote: post #22

I'll never forget having to be a moderator for a somewhat popular forum back in the day and oh man did I learn how a few people can make your life hell. One thing not mentioned many times in these discussions are the poor moderators. Having to look at all that stuff, some of which can be very disturbing or shocking (think death, gore, etc as well as the racy things) really takes a toll on the mind. The more automatio…

Someone still ends up reviewing images for the ML training dataset.

That's still a huge improvement over every mod everywhere seeing the same images repeatedly, but someone has to make the call at some point.

Re: You either die an MVP or live long enough to build content moderation

#194

Post by beachy » Tue, Sep 28, 2021, 8:10 PM UTC

phkahler wrote: post #133

Here's a thought. Have a platform where identity is verified. Users can post publicly or within their circle. Any illegal or fraudulent content can be handled by the legal system due to the lack of anonymity. Beyond that, let users form groups for topics like reddit. Where does this fall down?

I would do that but make it pseudonymous.

i.e. illegal or fraudulent content is clearly owned by the user irongoat, and you know who irongoat is (at least you know their email, IP address) but no-one on your site knows who irongoat is.

If the content is bad enough, authorities will get in touch for you to tell them what email address irongoat is associated with.

But as long as the content is OK, irongoat can say what he/she wants, with no PII visible.

Re: You either die an MVP or live long enough to build content moderation

#195

Post by pier25 » Tue, Sep 28, 2021, 8:11 PM UTC

Alir3z4 wrote: post #128

pier25 wrote: post #117

Earlier quoted context omitted.

Jesus... what a nightmare. Were these all using your free plans?

Oh yes. Never ending. We still keep the free plans even through there are abusers, but that won't be a reason to retire it. So many people using for legitimate reasons and keeping their personal writing there. It's unfortunate, but well, it happens.

Why do you think they were using your platform to do this?

Is it because these scammers were non technical?

Or maybe the anonymity provided by your platform?

Re: You either die an MVP or live long enough to build content moderation

#196

Post by beachy » Tue, Sep 28, 2021, 8:13 PM UTC

view wrote: post #63

I'm building a human content moderation service as an API for developers. It's not fully ready yet (expected next week) but people can sign up and start exploring the docs. I'd love to hear your feedback and any features you might want to see: https://moderationsystem.com/

Feedback: I went to the site with one question in mind - what type of moderation do you use?

e.g. would your services be useful to a christian social network? Or to a goth band forum? In the US? In Iceland?

i.e. how do you sync your service to the style, nuance and voice of your customer?

Re: You either die an MVP or live long enough to build content moderation

#197

Post by caseyross » Tue, Sep 28, 2021, 8:15 PM UTC

GCA10 wrote: post #84

Can't help but think back to W. Edwards Deming's distinction between after-the-fact efforts to "inspect" quality into the process -- as opposed to before-the-fact efforts to build quality into the process. OP offers a first-rate review (strategy + tactics!) for the inspection approach. But, the unspoken alternative is to rethink the on-ramp to content-creation privileges, so that only people with net-positive value t…

Absolutely this. Don't build the ship and then run around plugging leaks --- plan out the ship well enough to prevent leaks in the first place.

This is hard, and rare, because it requires predicting how all sorts of different people are going to interact with the community. Traditionally, this hasn't been something that the people who start software companies are particularly interested in, or good at. And a laser focus on user growth only compounds the problem.

Re: You either die an MVP or live long enough to build content moderation

#198

Post by dkh » Tue, Sep 28, 2021, 8:16 PM UTC

sneak wrote: post #151

This article opens up with an anecdote about policing morality and imposing one's own local norms on the legal speech of others. NSFW is a euphemism. Please don't push your American puritanical worldview on every user of your UGC platform.

Okay, so disregarding that label, most platforms will have a target or focus or niche (well, the ones that want a chance of surviving, anyway) and they will thus be very wise to tailor the rules around that, and create the conditions ideal for fostering that type of content.

For instance, if you were starting, say, a TikTok-esque video app but for super-quick tutorial videos, wouldn't it make sense for upload criteria to require it be some sort of tutorial, stay within some time limit, and probably not be just a gratuitous video of a bunch of people having sex? Call it whatever you want -- "NSFW" is just a shortcut, a heuristic that most people understand the meaning of regardless of whether it is actually safe or unsafe at their place of work. But there can be no denying that platforms/communities serving some interest or demographic will have their own unique requirements, their policies and standards will reflect this, and very often this will preclude "NSFW" content.

A lot of people get bent out of shape about this and view these sorts of policies solely as some sort of censorship issue, but many fail to realize that most of the time it's just about creating the ideal conditions for the community/platform to take hold.

Re: You either die an MVP or live long enough to build content moderation

#199

Post by bogwog » Tue, Sep 28, 2021, 8:18 PM UTC

duxup wrote: post #68

So true. In HN articles where we discuss social media moderation there's often this idea that "they shouldn't be doing this at all". But I think for most companies and users ... they won't like what a completely moderation free site looks like. So here we are with this painful problem. I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without…

> I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without the land of bots and dishonest users and so forth. But that obviously brings its own issues.

That sounds like it could be done in a way that isnt terrible. As a user, you sign up with an identity provider by submitting personal documents and/or an interview to prove that you’re a real person.

Then, when you sign up to an app/service, you login with the ID provider and come up with a username for that service.

The ID provider does not give the website provider any of your personal information; they just verify that you exist (and you login their their secure portal)

The identity providers could further protect privacy by automatically deleting all of your personal documents from their databases as soon as the verification process is complete. They could also have a policy to not store any logs, such as the list of services you’ve signed up for.

This could still be gamed (ex: a phone scammer tricking someone into getting verified with a provider to get a valid ID), but itd make things much harder and costlier.

Am I missing anything obvious that would make this a terrible idea?

Re: You either die an MVP or live long enough to build content moderation

#200

Post by jjulius » Tue, Sep 28, 2021, 8:19 PM UTC

quantumBerry wrote: post #192

jjulius wrote: post #188

Earlier quoted context omitted.

>I suppose in the sense that Gabby Pettito was moderated off the internet, Ross Ulbricht was moderated off of the internet and into a cage permanently for the heinous crime of facilitating voluntarily peaceful trade. Oh hello, strawman. >Tor marketplaces were definitely not gone for years, the same content just moved under new banners. You can literally find the same content and more on WHM today as you did under Ulb…

>Oh hello, strawman. Glad to know you finally admit that being kidnapped by a 3rd party is not really what most of us think as "moderation", and thus you have made a straw man. Although in the strict sense I guess it is true that moderation could merely mean some 3rd party entity came along and violently kept me away from communicating. If you don't like me posting cat pictures on reddit, you could crack my skull or…

... wow. Talk about going from 0-100 entirely too fast.

I was talking specifically about sites such as Silk Road and others being taken offline (which is exactly what you were talking about, too), not once did I mention his arrest nor did I allude to it. Glancing at your username, I seem to recall previous comments from you in threads about drug use being legalized. On the broad topic of drug legalization - again - you and I agree, but you would do well to prevent your biases from creeping in and causing you to misunderstand posts and/or lash out at others.