Viewing profile — jayalammar

We actually just wrote a book with your profile in mind -- especially if by "AI" you're especially interested in LLMs and if you're a visual learner. It's called Hands-On Large Lan…

This is my sense as well. Text generation LLMs haven't been the best source of embeddings for other downstream use cases. If you're optimizing for token embeddings (e.g., for NER, …

Contribution page: https://sites.google.com/cohere.com/aya-en/home

That's beautiful! Hope you're getting to do more of these!

Additional ones that come to mind now are: Transformer Feed-Forward Layers Are Key-Value Memories https://arxiv.org/abs/2012.14913 The Dual Form of Neural Networks Revisited: Conne…

Thank you so much (and others for your kind messages). Glad you found them useful! Writing is the best way for me to learn, I find.

This is a field I find fascinating. It's generally the research field of Machine Learning Interpretability. The BlackboxNLP workshop is one of the main places for investigating thi…

The goal is to make the materials as accessible as possible. So we're definitely not limited to the structure of a typical university course and are happy to iterate on it. I appre…

The landing page is technically the course overview. I'd love to hear what you think would've made it more engaging for you. We can probably pull up some of the visuals to it as a …

Hi HN, We've just published a lot of original, visual, and intuitive explanations of concepts to introduce people to large language models. It's available for free with no sign-up …

I'm the author of https://jalammar.github.io/illustrated-transformer/ and have spent years since introducing people to Transformers and thinking of how best to communicate those co…

How would you add that data? As new columns you mean? Or add the paragraph headings to the text of the paragraphs before embedding them?

There's a lot you can do with the vectors themselves without needing to embed any more text (e.g., clustering, exploration, visualization after dimensionality reduction...etc). Her…

Cohere actually trains its own models and they are not based on models from other providers [I work at Cohere]. Your prompt suggestion is a good one for LLMs as a whole. Any inform…

For Cohere, make sure you're using Command-Xlarge-Nightly. Otherwise, you may be prompting a Base LLM expecting the behavior of a different kind of LLM (an instruction-tuned chat m…

Hi. Author here. This is the first in a series I've been writing for a while to help orient people about useful perspectives to have in catching up to all that's happening in AI/ML…

They're trained and focused on language data, actually, not code specifically. There are both generation models and multilingual text embedding models (100+ languages, single model…

We train and serve large models at cohere.ai. We've shared some optimization techniques here: https://txt.cohere.ai/running-large-language-models-in-produ...

Agreed. "Stable Diffusion with Diffusers" and "The Annotated Diffusion Model" were excellent and are linked in the article. The code in Diffusers was also a good reference.

I updated the post to say "multi-dimensional array". In a context like this, we use tensor because it allows for any number of dimensions (while vector/ array is only one, matrix i…

Two diffusion processes are involved: 1- Forward Diffusion (adding noise, and training the Unet to predict how much noise is added in each step) 2- Generating the image by denoisin…