Great article, fun service. Still a part of me feels sad that we spend brain power on things like this.
Emoji and Deep Learning
21–30 of 55 posts
Re: Emoji and Deep Learning
#22Reading this was somewhat bothersome in that my browser (Chrome on Linux) doesn't render emoji. Is there a standard font that supports emoji that could be installed?
Re: Emoji and Deep Learning
#23It's also a little... racist.
If you feed it with emojis it spits out other emojis
(I was testing if it could spit out text from emoji input)
But what happens if you change the skintone of the emojis?
White arm: http://i.imgur.com/KTNky0O.png Obvious connection to sports, sunglasses(like saying "cool" in this context)
Black arm: http://i.imgur.com/uXtSRfc.png Policeman searching something, a location marker(search location?)
Re: Emoji and Deep Learning
#24I have to say this was one of the more "fun" ML articles I have read lately. Excellent visualizations as well. Great job!
Thanks! I was tempted to go way more technical but we decided it would be fun to let more people get some intuitive grasp of these kinds of algorithms.
It was really effective! It was great to be able to visually "walk" through the state space. If you do another, more technical post digging into the details then I would definitely be interested.
Re: Emoji and Deep Learning
#25It's also a little... racist. If you feed it with emojis it spits out other emojis (I was testing if it could spit out text from emoji input) But what happens if you change the skintone of the emojis? White arm: http://i.imgur.com/KTNky0O.png Obvious connection to sports, sunglasses(like saying "cool" in this context) Black arm: http://i.imgur.com/uXtSRfc.png Policeman searching something, a location marker(search lo…
racism is in the eye of the beholder. You've injected a lot of non-standard meaning into those emoji.
Re: Emoji and Deep Learning
#26Why not train the RNN to directly predict emojis, instead of projecting everything to semantic space and picking the closest emoji? Seems like that would help with the problem of emojis with multiple meanings in different contexts. With this model, they could only be in a single point in semantic space.
Re: Emoji and Deep Learning
#27It's also a little... racist. If you feed it with emojis it spits out other emojis (I was testing if it could spit out text from emoji input) But what happens if you change the skintone of the emojis? White arm: http://i.imgur.com/KTNky0O.png Obvious connection to sports, sunglasses(like saying "cool" in this context) Black arm: http://i.imgur.com/uXtSRfc.png Policeman searching something, a location marker(search lo…
(I am also a dev for Dango)
This is definitely a concern and something we've though about but not yet fully solved. The neural net is trained on real world data which unfortunately includes various types of questionable, racist, sexist, etc content. We already blacklisted emoji combination that are too often triggered in racist ways. However such a system is very difficult to audit completely.
Your example comparing different skin tone modifiers is a good one that we hadn't thought of. I've made a note of it so we can try and improve.
Re: Emoji and Deep Learning
#28Why not train the RNN to directly predict emojis, instead of projecting everything to semantic space and picking the closest emoji? Seems like that would help with the problem of emojis with multiple meanings in different contexts. With this model, they could only be in a single point in semantic space.
The RNN does in fact directly predict emoji. It outputs a vector of length 1624 (the number of emoji) containing the score associated with each emoji given the input text. This vector of probabilities is what can be though of as the point in semantic space.
The issue of multiple meanings is that if you strongly predict an ambiguous emoji (say the prayer emoji) how do you then extrapolate what concept is contained in the sentence (e.g. was the person saying "thanks" or "high five" or "please").
[I'm also a Dango dev]
Re: Emoji and Deep Learning
#29Extremely neat, but I really don't understand the point of the app (Dango) that all this engineering is for the sake of. If I'm using an emoji, it's either instead of words, or to clarify words that could be taken multiple ways (e.g. sarcasm.)
Who are these people that type a sentence (with a single meaning, clear-cut enough for Dango to detect), and then want to add a redundant pictorial representation of the same words they just typed?
Re: Emoji and Deep Learning
#30It's also a little... racist. If you feed it with emojis it spits out other emojis (I was testing if it could spit out text from emoji input) But what happens if you change the skintone of the emojis? White arm: http://i.imgur.com/KTNky0O.png Obvious connection to sports, sunglasses(like saying "cool" in this context) Black arm: http://i.imgur.com/uXtSRfc.png Policeman searching something, a location marker(search lo…
Yeah this is a real challenge with AI systems, they reflect both our worst and our best back at us, we're still struggling with the best way to deal with this.
Although in this particular case it's actually just a bug: Dango gets confused by any skin tone modifier character, since they're not supported on Android (our target platform). Try putting in a "white" arm and you'll see the same results. They're actually just our "Dango is confused by this input" results.
We should fix the bug, of course!