Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You either die an MVP or live long enough to build content moderation

mux.com

Post reply on HN
Print view View original

101–110 of 429 posts

Re: You either die an MVP or live long enough to build content moderation

#101

Post by chefandy »

infogulch wrote: post #77
Y_Y wrote: post #25

Earlier quoted context omitted.

If your eyes can "normalize" a unusual symbols to a common one to make an English word then so can a lookup table. I feel like this isn't a case where you'd reach first for a neural net.

If you can identify text written with mixed glyphs just ban it outright. Normal users don't use text like this, the pure binary presence of such "homomorphic" text at all is probably a better signal for spam than whatever your neural net when running it after normalization.

> Normal users don't use text like this

I think that depends on the users. People copying and pasting bits of text that was in English or another common language— think documentation, code, news articles, tweets, etc.— with a different character set could be problematic.

Also, 𝒮ℴ𝓂ℯ 𝒜𝓅𝓅𝓈 marketed as "𝔽𝕠𝕟𝕥𝕤 𝕗𝕠𝕣 𝕤𝕠𝕔𝕒𝕝 𝕞𝕖𝕕𝕚𝕒" would be ℭ𝔞𝔲𝔤𝔥𝔱 𝔲𝔭 𝔦𝔫 𝔱𝔥𝔦𝔰. (math symbols) A user base with young people getting bounced or shadow banned for trying to express themselves or distinguish themselves from their peers would be like ಠ_ಠ (Kannada letter ttha)

I think targeting the language they're using is a better bet.

¯\_(ツ)_/¯ (Hirigana letter tsu)

Re: You either die an MVP or live long enough to build content moderation

#102

Post by commandlinefan »

dkh wrote: post #81

It drives me absolutely nuts when I encounter a video platform upstart that has not adequately prepared (or prepared at all) for the inevitable onslaught of undesirable and illegal content that users will soon start uploading if the platform has really any traction at all. No UGC site/app is immune. Even when prepared, it is an eternal, constantly-evolving battle as users find more clever ways to try to hide their up…

> It was a year before the resulting back-and-forth with federal agencies

You're blaming lack of content moderation and not a law enforcement system that holds you responsible for something you had no control over when it actually failed to do its own job in this case?

Re: You either die an MVP or live long enough to build content moderation

#103

Post by canadapups »

I run an online marketplace. It's a constant battle against scammers putting up fake items to sell. While I do run "content moderation" to identify the scams, the fakes are identical or nearly identical to the real items. Content moderation isn't the solution for me. As other commenters point out it's just war of attrition or cycle of escalation from a few bad actors.

The only effective method I have now is fingerprinting (i.e., invading users privacy). Browsers are becoming more privacy oriented so at time goes on fingerprinting will be less effective, with more people being scammed online. I don't think those that want privacy at all costs understand the trade off.

In a few months, I will move to an voluntary fingerprinting/identification scheme soon (like GDPR cookies opt in). Where you identify yourself or don't use my website... which may leave me as a "die an MVP" example.

Re: You either die an MVP or live long enough to build content moderation

#104

Post by unethical_ban »

commandlinefan wrote: post #96
duxup wrote: post #68

So true. In HN articles where we discuss social media moderation there's often this idea that "they shouldn't be doing this at all". But I think for most companies and users ... they won't like what a completely moderation free site looks like. So here we are with this painful problem. I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without…

> they won't like what a completely moderation free site looks like. So you say, but we've never actually had a chance to see one. We have seen content moderation slippery-slope its way to highly opinionated censorship... every time it's been tried.

This is false. 4chan, some early reddit, voat, and a number of other sites.

Moreover, that's the point: it's as i.possible to have a healthy anonymous forum available to the world as it is to have a large society with no laws or government, that isn't dystopian.

Re: You either die an MVP or live long enough to build content moderation

#105

Post by commandlinefan »

oblio wrote: post #21
quantumBerry wrote: post #7

The internet itself is unmoderated in any useful sense for content, yet it has lived longer than most of these cheesy "moderated" products that seek to impose their morality on you.

Unmoderated? Probably 99% of traffic goes to the same top 500 sites which are heavily moderated.

Maybe so (that sounds believable, anyway). What he's saying is that the other 1% is unmoderated because there's no central authority [1]. The problem here isn't that people will share bad things if you don't stop them, the problem is that you're in a position of being held responsible for something outside your control. If it's illegal, it should be reported (or found by law enforcement whose job it is to enforce the law) and if it's offensive, offer some user-side filtering.

[1] this is starting to change, though - Amazon took Parler offline completely at the hosting level. Although they eventually found another hosting provider, it's not unimaginable that in the near future, service providers will collaborate to moderate the underlying traffic itself.

Re: You either die an MVP or live long enough to build content moderation

#106

Post by nothrowaways »

Alir3z4 wrote: post #17

Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc. Rolled out a machine learning model and trained it on the database. 99% of them vanished. Next day, the machine didn't work and success rate was around 5%. Found out, they have learned the trick and now using symbols from…

I would recommend you a different approach, such as using metadata like their location etc.

Re: You either die an MVP or live long enough to build content moderation

#107

Post by ziml77 »

commandlinefan wrote: post #96
duxup wrote: post #68

So true. In HN articles where we discuss social media moderation there's often this idea that "they shouldn't be doing this at all". But I think for most companies and users ... they won't like what a completely moderation free site looks like. So here we are with this painful problem. I kinda wish there was an imaginary "real person with an honest identity" type system that did exist where we could interact without…

> they won't like what a completely moderation free site looks like. So you say, but we've never actually had a chance to see one. We have seen content moderation slippery-slope its way to highly opinionated censorship... every time it's been tried.

There used to be a fair chance that you'd stumble on CSAM on 4chan. Without filtering and aggressive moderation, that's what ends up happening (yes 4chan did have moderation to delete the stuff back then and dish out IP bans, but it wasn't fast enough to save people from seeing those things)

Re: You either die an MVP or live long enough to build content moderation

#109

Post by jcun4128 »

htrp wrote: post #70
jfengel wrote: post #55

If you host blobs for free, somebody is going to use you as their host. Even if you just hosted audio, I'm sure somebody will quickly come along with a steganography tool to hide their content on your site (and use your bandwidth). Similarly, if you make compute power available, people will use you to mine cryptocurrency. Even if all you host is text, somebody will come along to be abusive. When you put a computer on…

> If you host blobs for free, somebody is going to use you as their host. Even if you just hosted audio, I'm sure somebody will quickly come along with a steganography tool to hide their content on your site (and use your bandwidth). This feels like something more of a theoretical example cited versus something that has happened. Do you have any examples of steganography being used as bandwidth redirection/hosting?

Recently saw a post about using imgur to host websites by the website code embedded in images (steganography?)

Re: You either die an MVP or live long enough to build content moderation

#110

Post by Alir3z4 »

nerd_light wrote: post #88
Alir3z4 wrote: post #78

Earlier quoted context omitted.

Not sure if I understand correctly, but if you mean how re-training and deployment would be. Nothing fancy tbh. For the first several deployment while taking care of edge case and debugging, all manually on my own laptop and shot into cluster as a docker image. Later, when starting to classify more content on the platform itself: - Webhook will trigger the CI to train the model with new ham and spam content. - A new…

I think they meant (and I am interested in hearing about) appealing a "block" decision that was made by your automation. If I'm a real human and trying to post a "good" post, but the model classifies it as bad and automatically blocks it, how do I appeal that decision? Can I? Or is my post totally blocked with no recourse?

Oh got it.

Thanks for clarification.

When a post gets published, it will be send to machine learning image via REST.

If bad, the post will be kept as Draft.

A new record gets created in another database table to keep track them, the accuracy rate was recorded as well.

This was made to make sure no irreversible action was done on the good content.

Blogs with more than 1 year of history would not go through moderation but no action was being taken, just recording the accuracy for future reference.

Later, someone from our team (me usually) would check them by eye and pull trigger on them, they would go into make the training better.

If something would pass the moderation but it was indeed spam, would go into another iteration.

We had to do this for over a month, through the time, the success was around 99%, no blogs would be wiped by machine classification from our database unless confirmed by someone.

That time the whole model was trained for that specific content. Later it get into other type of spams. Which we trained different models.

Overall, the the machine actions were logged, content/users/blogs would get labeled and bad marks on them.

They would be displayed in a report page, until someone make the final decision, through the whole time, the user would be shadow banned (shadow banning didn't help though) and their content would not be published.

Post reply on HN