You either die an MVP or live long enough to build content moderation • Hacker News Board

Re: You either die an MVP or live long enough to build content moderation

#211

Post by dkh » Tue, Sep 28, 2021, 8:44 PM UTC

BeFlatXIII wrote: post #162

dkh wrote: post #81

It drives me absolutely nuts when I encounter a video platform upstart that has not adequately prepared (or prepared at all) for the inevitable onslaught of undesirable and illegal content that users will soon start uploading if the platform has really any traction at all. No UGC site/app is immune. Even when prepared, it is an eternal, constantly-evolving battle as users find more clever ways to try to hide their up…

> 2. People uploading things like pirated content tend to do so in bulk. You might see people uploading hundreds of videos of TV shows or whatever. It may exceed legitimate uploads in the early days of a platform. You do not want to burden users with this level of moderation Not to mention that viewers aren't likely to flag the complete discography camrip of My Little Pony unless they're stupid or have an axe to grin…

Exactly. Setting aside all legal concerns and whatever anyone's philosophy is about piracy or moderated content, you still have the enormous concern about what kind of community you are fostering and what kind of people you are attracting based on what content you allow to be surfaced.

Re: You either die an MVP or live long enough to build content moderation

#212

Post by Nasrudith » Tue, Sep 28, 2021, 8:46 PM UTC

snowwrestler wrote: post #135

It’s not covered in this post, but IP infringement is another moment in life when content moderation becomes necessary. You have to be above a certain scale for large IP owners to notice or care, but if you’re growing and allow users to upload media, you’ll eventually need to start handling DMCA requests at minimum. Also worth noting that the infamous Section 230 is what allows companies to take these sort of best-ef…

What is infamous about it? People have to lie constantly to attack it and create outright alternative universes with distinctions that don't really exist.

Re: You either die an MVP or live long enough to build content moderation

#213

Post by throwawaysea » Tue, Sep 28, 2021, 8:46 PM UTC

Sohcahtoa82 wrote: post #189

rtkwe wrote: post #71

It's really funny every time a "we don't censor" platform pops up catering too the American right they speed run going from moderation == censorship to we're moderating our platform in record time. Turns out moderation is really important to make a platform for a community.

If you create a platform with absolutely zero censorship, you become a repository for child porn. I participated in Freenet many years ago because I liked its ideas (And thought it would have been a nice way to pirate games without my ISP being able to know), but it got a reputation for being used for CP, and I promptly deleted it, because I want no part in that. If you merely censor illegal content, you will become…

> If you merely censor illegal content, you will become a home for disinformation and ultra right-wing conspiracies. See Parler.

What do you count as disinformation and why is it a problem? If you disagree with something you can ignore it and move on, or engage with it and respond with your own counter-argument. It doesn't seem like a problem that reduces the viability of the entire platform. It is also strange to me that you seem to think a lack of censorship only favors "ultra right-wing" conspiracies. I saw a lot of disinformation about policing being spread throughout 2020 without much evidence. Those who pushed those narratives did not face any moderation for their misinformation. I recall as well when Twitter, Medium, and others banned discussions of the lab leak theory. The pro-moderation crowd unwittingly aided in the CCP's avoidance of accountability and smeared some very rational speculation as disinformation. I don't think I want anyone - whether the government, powerful private companies, or biased moderators - to become the arbiters of permitted opinions.

> In either case, and especially the first, you're likely to get kicked off your hosting platform and get a lot of attention from the government.

It's also curious to me that you mention Parler, because January 6th was organized more on other platforms than on Parler. Silicon Valley acted in unison against Parler because they share the same political biases among their leaders and employees, and because they share that degree of monopolistic power (https://greenwald.substack.com/p/how-silicon-valley-in-a-sho...). The darker part of this saga is that sitting members of the US government pressured private companies (Apple, Google, Amazon) to censor the speech of their political adversaries by banning Parler (https://greenwald.substack.com/p/congress-escalates-pressure...), in what can only be called an abuse of power and authority. When tech companies are facing anti-trust scrutiny and regulatory pressure on other issues, why wouldn't they seek favor by doing the incoming government's bidding and deplatforming Parler? I feel like the actions observed in the Parler saga are less about moderation and more about bias and power.

Re: You either die an MVP or live long enough to build content moderation

#214

Post by v_london » Tue, Sep 28, 2021, 8:49 PM UTC

chefandy wrote: post #101

infogulch wrote: post #77

Earlier quoted context omitted.

If you can identify text written with mixed glyphs just ban it outright. Normal users don't use text like this, the pure binary presence of such "homomorphic" text at all is probably a better signal for spam than whatever your neural net when running it after normalization.

> Normal users don't use text like this I think that depends on the users. People copying and pasting bits of text that was in English or another common language— think documentation, code, news articles, tweets, etc.— with a different character set could be problematic. Also, 𝒮ℴ𝓂ℯ 𝒜𝓅𝓅𝓈 marketed as "𝔽𝕠𝕟𝕥𝕤 𝕗𝕠𝕣 𝕤𝕠𝕔𝕒𝕝 𝕞𝕖𝕕𝕚𝕒" would be ℭ𝔞𝔲𝔤𝔥𝔱 𝔲𝔭 𝔦𝔫 𝔱𝔥𝔦𝔰. (math symbols) A user base with…

Bizarrely, I'm seeing recruiters using this on LinkedIn.

Re: You either die an MVP or live long enough to build content moderation

#215

Post by Nasrudith » Tue, Sep 28, 2021, 8:50 PM UTC

duxup wrote: post #68

So true. In HN articles where we discuss social media moderation there's often this idea that "they shouldn't be doing this at all". But I think for most companies and users ... they won't like what a completely moderation free site looks like. So here we are with this painful problem. I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without…

I suspect that even if we could we would wind up with a disturbing "reverse Turing test". The dark suggestion that there isn't any difference between us and other malfunctioning machine learning arrangements trained by huge data sets. They may be objectively homo sapiens with honest identities devoted to something which makes them act indistinguishable from a bot.

Re: You either die an MVP or live long enough to build content moderation

#216

Post by Animats » Tue, Sep 28, 2021, 8:50 PM UTC

Roblox is working on a "moderation" system that can ban a user within 100ms after saying a bad word in voice. But their average user is 13 years old.

Interestingly, Second Life, the virtual world built of user-created content, does not have this problem. Second Life has real estate with strong property rights. Property owners can eject or ban people from their own property. So moderation, such as it is, is the responsibility of landowners. Operators of clubs ban people regularly, and some share ban lists. Linden Lab generally takes the position that what you and your guests do on your own land is your own business, provided that it isn't visible or audible beyond the parcel boundary. This works well in practice.

There are more and less restrictive areas. There's the "adult continent", which allows adult content in public view. But there's not that much in public view. Activity is mostly in private homes or clubs. At the other extreme, there's a giant planned unit development (60,000 houses and growing) which mostly looks like upper-middle class American suburbia. It has more rules and a HOA covenant. Users can choose to live or visit either, or both.

Because it's a big 3D world, about the size of Greater London, most problems are local. There's a certain amount of griefing, but the world is so big that the impact is limited. Spam in Second Life consists of putting up large billboards along roads.

Second Life has a governance group. It's about six people, for a system that averages 30,000 to 50,000 concurrent connected users. They deal mostly with reported incidents that fall into narrow categories. Things like someone putting a tree on their property that has a branch sticking out into a road and interferes with traffic.

There's getting to be an assumption that the Internet must be heavily censored. That is not correct. There are other approaches. It helps that Second Life is not indexed by Google and doesn't have "sharing".

Re: You either die an MVP or live long enough to build content moderation

#217

Post by ljm » Tue, Sep 28, 2021, 8:51 PM UTC

mrkurt wrote: post #74

tdeck wrote: post #58

Earlier quoted context omitted.

For adversarial problems like this, a shadowban approach can sometimes be necessary. Perhaps people can still see their blogs but GoogleBot gets blocked from indexing them, or they only appear to someone with the spammer's cookies. That way it takes them longer to catch on and evade the model. Of course, that means you'll need to at least spot check your bans because you can't rely on legit users escalating to you.

We shadow ban abusive users on Fly.io and it works great. Everything seems to work right up until they try to connect to a deployed app. It took me a while to realize that ramping up the frustration level is, itself, a helpful deterrent.

I remember an old mailing list discussion on sourcehut, because sourcehut provides a build service that you can use for automation.

The decision from sircmpwn was, at the end, to charge money for the service. Charging money and KnowYourCustomer will kill most exploits dead.

In this sense, this is turning the frustration level to 11. You can use the service to a certain extent, without frustration, but if you want to get serious then you're going to have to jump through some hoops.

Dedicated people will still find a way through, but you've cut off 95% of the flow and killed the low-effort attempts. Now, you can focus on the serious shit.

Re: You either die an MVP or live long enough to build content moderation

#218

Post by PeterisP » Tue, Sep 28, 2021, 8:53 PM UTC

bogwog wrote: post #199

duxup wrote: post #68

So true. In HN articles where we discuss social media moderation there's often this idea that "they shouldn't be doing this at all". But I think for most companies and users ... they won't like what a completely moderation free site looks like. So here we are with this painful problem. I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without…

> I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without the land of bots and dishonest users and so forth. But that obviously brings its own issues. That sounds like it could be done in a way that isnt terrible. As a user, you sign up with an identity provider by submitting personal documents and/or an interview to prove that you’re a real…

Your idea becomes useless for deterrence from the "automatically deleting" point. The only benefits for having users who are "real person with an honest identity" accrue when people either can't make many accounts (so banning an user actually bans them instead of simply makes them get a new account) or when you can identify them in case of fraud or child sexual abuse material or some such.

So at the very least the "identity provider" absolutely needs to keep a list of all real identities offered to a particular service; otherwise the bad actors will just re-verify some identity as many times as needed.

But if you give up the "hard privacy" requirement, then technically it's possible. It would also mean that the identity provider would sometimes get subpoenas to reveal these identities.

Re: You either die an MVP or live long enough to build content moderation

#219

Post by Nasrudith » Tue, Sep 28, 2021, 8:53 PM UTC

throwawaysea wrote: post #206

duxup wrote: post #68

So true. In HN articles where we discuss social media moderation there's often this idea that "they shouldn't be doing this at all". But I think for most companies and users ... they won't like what a completely moderation free site looks like. So here we are with this painful problem. I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without…

The problem is where the line is drawn. Arguments against explicitly illegal content (like CSAM) or unhelpful content (like spam) are used to justify content moderation that goes far beyond. Moderation on the biggest social media platforms (like Facebook, Twitter, TikTok, etc.) includes more than just basic moderation to make the platform viable. They include a number of elements that are more like censorship or prop…

Propaganda and bias are ultimately subjective notions. Suggest that kings are no different from the rest of us today aside from what their parents did to seize power and it would be slammed as wicked propaganda denying their divine right. Heck just suggest that disfavored groups are people!

Re: You either die an MVP or live long enough to build content moderation

#220

Post by jjulius » Tue, Sep 28, 2021, 8:54 PM UTC

quantumBerry wrote: post #208

jjulius wrote: post #200

Earlier quoted context omitted.

... wow. Talk about going from 0-100 entirely too fast. I was talking specifically about sites such as Silk Road and others being taken offline (which is exactly what you were talking about, too), not once did I mention his arrest nor did I allude to it. Glancing at your username, I seem to recall previous comments from you in threads about drug use being legalized. On the broad topic of drug legalization - again - y…

I apologize, maybe you are not familiar with the details of the silk road. Ross Ulbricht was the administrator and creator of the silk road, allegedly. It's quite probable that without his arrest, it would have persisted even if on newly acquired hardware. I would argue his arrest was integral in these violent thugs "moderating" silk road away like the mob "moderates" away their competition. Instead, after his arrest…

Yeah, again, you're injecting your own biases as you create assumptions about my comments, rather than stopping to ask what I mean before you fly off the handle. See how I used "moderated" in quotes in my very first response? That suggests that I'm using the term rather loosely.

All I've said was that that's how illegal content is moderated on the internet - it is removed. Silk Road was removed, AlphaBay was removed, DarkMarket was removed, many others have been removed, and many more will continue to be removed even if Tor makes that a slow process. At no point did I bring up whether or not I thought it was "right" to remove them, or to treat Ulbricht in that manner (again, you're assuming I don't know what happened). I said "moderating" with quotes, for lack of a better word.

If something's illegal - even if you and I think it shouldn't be - then it's typically going to be removed at some point, even if it takes a while because something like Tor makes it difficult. And in that sense, yes, the internet is "moderated" for that content. That's all I've said/argued, and I truly don't understand how that is so difficult for you to grasp.

>It's quite probable that without his arrest, it would have persisted even if on newly acquired hardware. ... Instead, after his arrest the content ended up on new platforms rather than the Silk Road platform.

For implying that I don't know what happened, you seem to be forgetting that other Silk Road staff started Silk Road 2.0 after his arrest, but that was also shut down.

>What are you talking about? You feel attacked because your poorly constructed argument was laid open.

Nope. You allow your biases to creep in to your poor interpretations of other people's comments, and then angrily respond to them as such. My initial response was simple, but your strongly held beliefs have clouded your responses.