Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You either die an MVP or live long enough to build content moderation

mux.com

Post reply on HN
Print view View original

41–50 of 429 posts

Re: You either die an MVP or live long enough to build content moderation

#41

Post by shadowgovt »

Ages ago on This Week in Tech, Leo Laporte and perennial guest John C. Dvorak discussed the notion (in the context of Yahoo Groups at the time) that nearly every hosted content platform starts default-open to maximize the size of their user base but eventually passes over the "horse porn event horizon." Sooner or later, a critical mass of users with specific predilections would find the forum and use it for communicating on their, uh, topic of choice. And the owner at that point had two options:

- Do nothing and let the forum stay as open as it had been

- Ever be ad-supported or supported via mainstream sponsorship or partnership, or be purchased by a bigger company

... Few hosts choose the first option.

Re: You either die an MVP or live long enough to build content moderation

#42

Post by devmor »

As someone working on a platform with content moderation as a core feature, it is so much work. I thoroughly understand why so many platforms ignore it for so long.

Thankfully, we have some nice tools these days. I use Google's Perspective API to automatically hold back text input for manual moderation, which takes a lot of the man hours out of it for my moderation team.

The rest is handled by the users of the platform themselves, and metrics about content reports to curtail abuse.

Re: You either die an MVP or live long enough to build content moderation

#43

Post by bliteben »

oblio wrote: post #20
Alir3z4 wrote: post #17

Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc. Rolled out a machine learning model and trained it on the database. 99% of them vanished. Next day, the machine didn't work and success rate was around 5%. Found out, they have learned the trick and now using symbols from…

Do you have a mechanism for appealing the automated process?

what are they google? wait google doesn't even have that

Re: You either die an MVP or live long enough to build content moderation

#44

Post by Alir3z4 »

vadfa wrote: post #23
Alir3z4 wrote: post #17

Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc. Rolled out a machine learning model and trained it on the database. 99% of them vanished. Next day, the machine didn't work and success rate was around 5%. Found out, they have learned the trick and now using symbols from…

What software/libraries have you used for your machine learning moderation system?

tensorflow and scikit-learn to train and build the model.

On the front FastAPI (behind uvicorn) to accept calls via REST API.

Deployed via docker.

To be honest, tensorflow and scikit-learn may not be the right fit for everything.

Every situation needs different approach and different solution.

Worth nothing, the most time consuming part was dealing with data itself and not model training or machine learning.

In couple of hours you'd notice you're starting at charts and tuning parameters.

Re: You either die an MVP or live long enough to build content moderation

#45

Post by nradov »

Y_Y wrote: post #25
Alir3z4 wrote: post #17

Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc. Rolled out a machine learning model and trained it on the database. 99% of them vanished. Next day, the machine didn't work and success rate was around 5%. Found out, they have learned the trick and now using symbols from…

If your eyes can "normalize" a unusual symbols to a common one to make an English word then so can a lookup table. I feel like this isn't a case where you'd reach first for a neural net.

Those are known as homoglyphs. This issue has been studied and best practices are documented. There's no complete solution but it can be mitigated.

https://en.wikipedia.org/wiki/Homoglyph

Re: You either die an MVP or live long enough to build content moderation

#47

Post by godshatter »

godshatter wrote: post #40

I guess I would rather see a system that focuses on scanning all images for illegal content (presumably there are services where you can hash the image and check for known child porn images, for example), and focus on tagging all other images for certain things (like David Hasselhof's bare chest or whatever concerns your users). Give the users tools to flag images as illegal content, or for misapplied or missing tags…

That downvote was fast. See, user moderation works :)

Re: You either die an MVP or live long enough to build content moderation

#48

Post by YetAnotherNick »

Alir3z4 wrote: post #17

Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc. Rolled out a machine learning model and trained it on the database. 99% of them vanished. Next day, the machine didn't work and success rate was around 5%. Found out, they have learned the trick and now using symbols from…

How do you detect the ground truth for training the model? Do you manually label it?

Re: You either die an MVP or live long enough to build content moderation

#49

Post by Alir3z4 »

Y_Y wrote: post #25
Alir3z4 wrote: post #17

Around 1 year ago we got hit badly on our [blogging platform][0] by people/groups submitting fake customer support description of other big companies, either being Microsoft, Facebook, Comcast etc. Rolled out a machine learning model and trained it on the database. 99% of them vanished. Next day, the machine didn't work and success rate was around 5%. Found out, they have learned the trick and now using symbols from…

If your eyes can "normalize" a unusual symbols to a common one to make an English word then so can a lookup table. I feel like this isn't a case where you'd reach first for a neural net.

Yeah, then someone has to create or find that whole table and make.

The initial problem wasn't those symbols but the content itself, the symbols and special characters came into the problem later.

Later on as mentioned in my original comment, that they would use positive content from other blog posts that were published/passed the moderation to mix up their bad content.

Probably could use a different method, but at that time needed something quick and fast and it worked and still works with very little tweaking.

Although we don't have massive amount of threats or abusers anymore to exactly know the effect, but again, so far it works.

That time, they would coming several thousands per minute, IP blocking, range blocking, USER AGENT, captcha or anything such didn't work on them.

Re: You either die an MVP or live long enough to build content moderation

#50

Post by MattGaiser »

baybal2 wrote: post #14

What about not treating users like kiddies needing supervision? I think the Silicone Valley MVP TrendySpeak crowd needs to open eyes to the realities of life. Slashdot is a very peculiar case of near no spam filtering, yet very good user content moderation Kuro5 was also kind of interesting with its old user rating system.

Turn on your ability to see dead comments if it is not on already. Even HN would be a landfill without moderation.
Post reply on HN