11–20 of 38 posts
I almost believe that if I know how make an LLM prompt and how to make an API call to OpenAI, Mistral, Claude 3, or together.ai, then as an application programmer, I can skip this whole book. I see people posting project specifications asking for NLP, named entity extraction, etc. But most things in those jobs look like they could be handled by an LLM, possibly even smaller than 7b, and probably more robustly. My oth…
For comparison, both Bishop's new deep learning book and Simon Price's Understanding deep learning get straight to transformers and only mention RNN/LSTM in passing - they were written just as state space models and hybrid RNN/transformer models started showing good results.
Why Hidden Markov Models are in the related links (7 years ago)? Did the PDF name change?
Chapter 10 looks like a good, self-contained intro to Transformers and LLMs https://web.stanford.edu/~jurafsky/slp3/10.pdf
I almost believe that if I know how make an LLM prompt and how to make an API call to OpenAI, Mistral, Claude 3, or together.ai, then as an application programmer, I can skip this whole book. I see people posting project specifications asking for NLP, named entity extraction, etc. But most things in those jobs look like they could be handled by an LLM, possibly even smaller than 7b, and probably more robustly. My oth…
I almost believe that if I know how make an LLM prompt and how to make an API call to OpenAI, Mistral, Claude 3, or together.ai, then as an application programmer, I can skip this whole book. I see people posting project specifications asking for NLP, named entity extraction, etc. But most things in those jobs look like they could be handled by an LLM, possibly even smaller than 7b, and probably more robustly. My oth…
It is not written for application programmers, it is a machine learning book.
The nice thing is performance would be different based on the human getting it. It kind of preserves some unique element.
I almost believe that if I know how make an LLM prompt and how to make an API call to OpenAI, Mistral, Claude 3, or together.ai, then as an application programmer, I can skip this whole book. I see people posting project specifications asking for NLP, named entity extraction, etc. But most things in those jobs look like they could be handled by an LLM, possibly even smaller than 7b, and probably more robustly. My oth…
The problem is exacerbated by a very big token dictionary. Every new token you generate is independently sampled (from tokens outside of the context size). If you have any task that requires joint distribution modeling LLM will fail and HMMs/CRFs will succeed. Of course, it depends when the problem will manifest and I do not know that, but skipping the whole book is not recommended.
For example, there were approaches for many language tasks in that book that used CNNs (instead of something more principled) and were extremely successful (despite the lack of joint modeling). Who knows when this accumulation of errors becomes measurable.
As long as your generation of tokens fits inside the context size, you're modeling everything jointly. But I guess you need to be aware that when you drop the furthest tokens to generate next ones, the performance can drastically drop (multiplicative accumulation of errors).
Earlier quoted context omitted.
That’s kinda like saying we don’t need to teach CS students anything about algorithms. I don’t think that book was ever aimed at application engineers in the first place.
Well, I can see it as being useful to give aspiring ML researchers a survey and jumping off point for possibly digging deeper. But I also think when they created that book originally it was about giving ML practitioners tools they would use.
I almost believe that if I know how make an LLM prompt and how to make an API call to OpenAI, Mistral, Claude 3, or together.ai, then as an application programmer, I can skip this whole book. I see people posting project specifications asking for NLP, named entity extraction, etc. But most things in those jobs look like they could be handled by an LLM, possibly even smaller than 7b, and probably more robustly. My oth…
It was the first edition of this book that, when I read it 20 years ago, got me "hooked" on computer science as a science rather than as just a fun thing to do (which I had been doing for several years prior). When I met Dan Jurafsky, 10 years later, I definitely felt none of that disappointment that often goes with meeting one of your "childhood heroes" and thanked him for writing the book and the impact it had on m…
Although much in computer science is reinventing the wheel, LLM so far strikes me as a tool that doesn't have a ton of analogues in history.