Viewing profile — jalammar

Hi HN, This is the first in a series of articles I'm writing to introduce devs to practical applications of large NLP language models (for text generations like GPT and for languag…

It's widely used in recommenders based on embeddings. See: https://github.com/spotify/annoy https://github.com/facebookresearch/faiss

Hi HN, I created this cheat sheet and video as high-level guidance to the major categories of ML explainability research and techniques. It's an area I've been working in the last …

I wouldn't trust any model to generate text for customers yet. Not even the largest GPT3. There are no guarantees on what they will output and could be damaging to your business. Y…

Hugging Face has that service https://huggingface.co/pricing

I didn't come across one yet personally

Hello HN, author here. Language models are absolutely fascinating tools. I believe it would pay for software engineers to have a sense of their capabilities and how they function. …

Hey, I feel you! I'm an intuitive learner as well. I wouldn't have been able to learn much in ML if it weren't for people who write and visualize and make the methods accessible to…

I actually started with PCA. But NMF proved more understandable since negative dimensions in PCA are hard to interpret. I didn't consider UMAP, but would be interested to see how i…

Interesting. Thanks for sharing your notes on the higher layers. Allow me to repost that to the discussion board on github. I do get your point on interpretation. This work is just…

These are AI explanation methods. They belong to the toolbox which would include LIME, Shapley values...etc. Input saliency is a gradient-based explanation method.

Wonderful! Thanks! I am curious about those recent O(L) attention transformers (see slide 106 of http://gabrielilharco.com/publications/EMNLP_2020_Tutorial__... ). If these methods…

I'd love to look at your group's visualizations! Is it a private repo? because the link doesn't open up. It never stops to blow my mind that we can represent words and concepts in …

Thanks! Please let me know if you have any feedback!

What I found most fascinating is identifying neuron firing patterns corresponding to linguistic properties: e.g. groups of neurons that fire in response to verbs, or pronounds. Scr…

Thank you!

Hi HN, Author here. I had been fascinated with Andrej Karpathy's article ( https://karpathy.github.io/2015/05/21/rnn-effectiveness/ ) -- especially where it shows neurons being act…

Apple Keynote

Author here. Thank you. I feel an important element of this type of writing is what complexity to show and what to hide at different points. "Magic" is just to say "don't worry abo…