Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emoji and Deep Learning

getdango.com

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Emoji and Deep Learning

#31

Post by wxs »

spaky wrote: post #28
joefkelley wrote: post #26

Why not train the RNN to directly predict emojis, instead of projecting everything to semantic space and picking the closest emoji? Seems like that would help with the problem of emojis with multiple meanings in different contexts. With this model, they could only be in a single point in semantic space.

The RNN does in fact directly predict emoji. It outputs a vector of length 1624 (the number of emoji) containing the score associated with each emoji given the input text. This vector of probabilities is what can be though of as the point in semantic space. The issue of multiple meanings is that if you strongly predict an ambiguous emoji (say the prayer emoji) how do you then extrapolate what concept is contained in…

So yeah: we can focus on vectors at different levels of the net and these are in some sense different semantic spaces. In the article I talk about a level immediately before it projects onto the emoji vectors. If you look at the output after the projection (and do a softmax) you get a probability distribution across all emoji. This would be a different space in which each axis is an emoji, rather than the emoji being points distributed around the space.

Re: Emoji and Deep Learning

#32

Post by wxs »

derefr wrote: post #29

Extremely neat, but I really don't understand the point of the app (Dango) that all this engineering is for the sake of. If I'm using an emoji, it's either instead of words, or to clarify words that could be taken multiple ways (e.g. sarcasm.) Who are these people that type a sentence (with a single meaning, clear-cut enough for Dango to detect), and then want to add a redundant pictorial representation of the same w…

Yeah so this is a legitimate concern. Of course sometimes it's fun to say "let's eat pizza :pizza_emoji:", but that's not hugely valuable.

However, Dango's training data includes people using Emoji to augment rather than repeat their sentence. So if there are two different interpretations and an emoji could disambiguate, the ideal is that Dango has seen people use that phrase both ways and, and that it suggests both possibilities and you can pick the one that you meant. In many cases this works now, in many cases we still have work to do.

It also suggests based on messages sent to you, so if there are a couple different replies it can show you them all (although this feature still needs work).

Re: Emoji and Deep Learning

#33

Post by keyle »

This is really cool. But half the fun for me is to pick the emojis at the end of the message. And they "add" to the mood of my message, they don't "amplify" it. Hence this wouldn't work for me most of the time o_0 ;(

Re: Emoji and Deep Learning

#34

Post by wxs »

the_watcher wrote: post #20
wxs wrote: post #15

Earlier quoted context omitted.

We start supporting them as soon as they're available! Obviously we have a lot less training data early on but we can lean on some heuristics early on until it builds up. Unfortunately in the app we can't give you emoji that your phone doesn't support so we don't always show all the results.

If you have (or build) a slack integration, would you be able to include custom emoji's? Or is that not enough input?

Yeah we'd need to do some more work. But this is similar to stickers and GIFs: there are many fewer examples for any given sticker or GIF, which is why we do transfer-learning approaches as discussed in the article.

So there's a good chance we could get it to work! We've not focused on that possibility… yet.

Re: Emoji and Deep Learning

#35

Post by wxs »

keyle wrote: post #33

This is really cool. But half the fun for me is to pick the emojis at the end of the message. And they "add" to the mood of my message, they don't "amplify" it. Hence this wouldn't work for me most of the time o_0 ;(

Well you can search in Dango, too ;)

But yeah our main focus is suggestions. You can use Dango concurrently with the normal emoji keyboard, of course! It can just sit there showing you emoji you might not know about "ambiently"

Re: Emoji and Deep Learning

#36

Post by joefkelley »

wxs wrote: post #31
spaky wrote: post #28

Earlier quoted context omitted.

The RNN does in fact directly predict emoji. It outputs a vector of length 1624 (the number of emoji) containing the score associated with each emoji given the input text. This vector of probabilities is what can be though of as the point in semantic space. The issue of multiple meanings is that if you strongly predict an ambiguous emoji (say the prayer emoji) how do you then extrapolate what concept is contained in…

So yeah: we can focus on vectors at different levels of the net and these are in some sense different semantic spaces. In the article I talk about a level immediately before it projects onto the emoji vectors. If you look at the output after the projection (and do a softmax) you get a probability distribution across all emoji. This would be a different space in which each axis is an emoji, rather than the emoji being…

Awesome, thanks for clarifying. So does the training optimize some property of the "semantic" layer immediately before the final emoji prediction layer? Or does it just optimize accuracy of emoji prediction directly?

And then the t-SNE projection shown in the article is based on this same layer (one before prediction)?

Re: Emoji and Deep Learning

#37

Post by the_watcher »

wxs wrote: post #34
the_watcher wrote: post #20

Earlier quoted context omitted.

If you have (or build) a slack integration, would you be able to include custom emoji's? Or is that not enough input?

Yeah we'd need to do some more work. But this is similar to stickers and GIFs: there are many fewer examples for any given sticker or GIF, which is why we do transfer-learning approaches as discussed in the article. So there's a good chance we could get it to work! We've not focused on that possibility… yet.

I figured - even a Slack integration as is would be pretty cool though.

Re: Emoji and Deep Learning

#39

Post by [deleted] »

hollythebeaver wrote: post #23

It's also a little... racist. If you feed it with emojis it spits out other emojis (I was testing if it could spit out text from emoji input) But what happens if you change the skintone of the emojis? White arm: http://i.imgur.com/KTNky0O.png Obvious connection to sports, sunglasses(like saying "cool" in this context) Black arm: http://i.imgur.com/uXtSRfc.png Policeman searching something, a location marker(search lo…

[deleted]
Post reply on HN