The naughty username checking system used by Twitch
1–10 of 348 posts
Re: The naughty username checking system used by Twitch
#2Path in the leak is:
safety-ml\offensive-usernames\data_pull\sql\bad.sql
Highlights:
CREATE OR REPLACE FUNCTION is_tragedy (VARCHAR) RETURNS BOOLEAN STABLE AS $$
SELECT replace($1,'_','') LIKE '%george%floyd%'
$$ LANGUAGE SQL;
CREATE OR REPLACE FUNCTION is_derogatory (VARCHAR) RETURNS BOOLEAN STABLE AS $$
SELECT replace($1,'_','') LIKE '%retard%'
$$ LANGUAGE SQL;
Re: The naughty username checking system used by Twitch
#3> is_marijuana
Looks like Twitch doesn't like weed.
Re: The naughty username checking system used by Twitch
#4This script feels like this is something that:
- should be in its own application with its own rules engine so you dont accidentally whack a bunch of userames
- I would have done in the past and cringe when people ask me to update it.
Re: The naughty username checking system used by Twitch
#5I have a hard time believing this was / is the real version used. It doesn't seem broad enough. More likely it was a kind of smoketest that made sure that a more automated keyword checker was working.
It does remind me of the XKEYSCORE (Snowden leaks) that used keywords to bubble up potential threats from emails etc https://www.businessinsider.com/nsa-prism-keywords-for-domes... .
Re: The naughty username checking system used by Twitch
#6This script feels like this is something that: - should be in its own application with its own rules engine so you dont accidentally whack a bunch of userames - I would have done in the past and cringe when people ask me to update it.
I feel bad for the Amazon employee who came into work one day with this project sitting on their desk.
Re: The naughty username checking system used by Twitch
#7Path in the leak is: safety-ml\offensive-usernames\data_pull\sql\bad.sql Highlights: CREATE OR REPLACE FUNCTION is_tragedy (VARCHAR) RETURNS BOOLEAN STABLE AS $$ SELECT replace($1,'_','') LIKE '%george%floyd%' $$ LANGUAGE SQL; CREATE OR REPLACE FUNCTION is_derogatory (VARCHAR) RETURNS BOOLEAN STABLE AS $$ SELECT replace($1,'_','') LIKE '%retard%' $$ LANGUAGE SQL; Now we know we can make as many goergefloyd accounts a…
safety-ml - I assume this is some attempt at training a machine learning algorithm to find bad names.
Re: The naughty username checking system used by Twitch
#8looks like it is limited to English only. There is whole world of non-English offense out there. Like using English characters to make national offensive words as well as using national characters to make English offensive words.
Re: The naughty username checking system used by Twitch
#9> CREATE OR REPLACE FUNCTION is_hateful (VARCHAR) RETURNS BOOLEAN STABLE AS $$
>...
>OR replace($1,'_','') SIMILAR TO '%(kill|keel|hang|burn)%(black|bl4ck|black|jew|trans|gay|african|afrikan|minorit|asian|nig|n1g)%'
Protected classes:
black, jewish, trans, gay, african, asian, minorities
Non-protected classes:
white
Typical Twitch (D-CA). All (D-CA) companies really. Under the law (in the USA), all races, and sexes are equally protected. On big tech platforms it's only one slice of demographics that harassment and hatred is tolerated toward, even encouraged. Do they actually think this is helping? Singling out one demographic for abuse? Do they think this is progress? That people won't notice? That it will never backfire?
Re: The naughty username checking system used by Twitch
#10looks like it is limited to English only. There is whole world of non-English offense out there. Like using English characters to make national offensive words as well as using national characters to make English offensive words.
It also includes a couple of Italian swearwords in the function `is_blasphemy`