Viewing profile — maxlam

That's definitely an interesting idea -- it seems this would allow for boundaries that "change" along with the data (instead of having static boundaries as it is). Would be interes…

Haven't tried this but this is definitely a good idea for visualizing what's going on!

Hm what do you mean? I'm not quite seeing how to differentiate with respect to the quantization steps.

Absolutely, please do!

Training these quantized word vectors has to be done in full precision (so no memory gains during training the word vectors). But when you save them to disk every value is either -…

Here's the one for man: ['man', 'woman', 'boy', 'handsome', 'stranger', 'gentleman', 'young', 'drunkard', 'devil', 'lonely', 'lady', 'lad', 'drunken', 'beggar', 'kid', 'effeminate'…

Thanks for the compliments! And thanks also for bringing up the debiasing! It's interesting to see what people care about and any conversation that leads to new ideas is a plus.

Hm not sure, would need to think more about this -- definitely an interesting idea though!

You can kind of see that words that are similar have similar looking vector values (that's why there are vertical stripes of yellow / black). But you're right in that most of it ju…

Definitely tried to figure out if the dimensions mean anything -- as far as I can tell they don't really mean much :(

Interesting, definitely need to think about debiasing. Seems like it won't really work straight out of the box since it'd destroy the 1 bit-ness of the vectors. Though if only a fe…

The idea is that you can kind of capture the "meaning" of a word with a sequence of numbers (a vector) -- and then you use these vectors for machine learning tasks to do cool stuff…

You're definitely right, the quantization function and its values definitely have an impact on performance. For 1 bit I think I tried something like -1/+1, -.5/+.5, -.25/+.25, -.33…

Main reason I did it this way is because Facebook's DrQA (which I evaluate the vectors on for the SQuAD task) uses case sensitive vectors. Was a tough decision between choosing whe…

This might be because "Artist" has an uppercase "A" -- I trained all the word vectors to be case sensitive so "Artist" is not the same as "artist" (which should be closer to "man" …

Oops, thought you meant the graphs (with the dotted/solid lines) in the writeup. If you're referring to the image under "Visualizing Quantized Word Vectors" then each row is a word…

Yeah, I should definitely put more detail in the writeup -- thanks for the feedback! What's happening with figure 1a (epochs vs google accuracy) is that as you train for more epoch…