81–90 of 139 posts
Earlier quoted context omitted.
It is a very common word used in Nigerian style English which was a very common place they were outsourcing RLHF tasks to. A sibling comment has a link but it is also easy to google.
As a non native speaker, I didn't know the word "delve" but now I know this word. I think internet community is learning from LLM?
> learning from LLM
If anyone wants to delve into machine learning, one of the superb resources I have found is, Stanfords "Probability for computer scientists"( https://www.youtube.com/watch?v=2MuDZIAzBMY&list=PLoROMvodv4... ). It delves into theoretical underpinnings of probability theory and ML, IMO better than any other course I have seen. (Yeah, Andrew Ng is legendary, but his course demands some mathematical familarity with linear…
Apparently the word “delve” is the biggest indicator of the use of ChatGPT according to Paul Graham.
Hm... Saw that, I have used it multiple times in my comment. I was just trying to convey the meaning.
What is right use of word? What would be right word to use here?
Earlier quoted context omitted.
Apparently the word “delve” is the biggest indicator of the use of ChatGPT according to Paul Graham.
Non native speaker here. Will remember this. Hm... Saw that, I have used it multiple times in my comment. I was just trying to convey the meaning. What is right use of word? What would be right word to use here?
If anyone wants to delve into machine learning, one of the superb resources I have found is, Stanfords "Probability for computer scientists"( https://www.youtube.com/watch?v=2MuDZIAzBMY&list=PLoROMvodv4... ). It delves into theoretical underpinnings of probability theory and ML, IMO better than any other course I have seen. (Yeah, Andrew Ng is legendary, but his course demands some mathematical familarity with linear…
Looks nice - are there written versions?
There is also book[2] for excellent caltech course[3].
[1] https://chrispiech.github.io/probabilityForComputerScientist...
[2] https://www.amazon.com/Learning-Data-Yaser-S-Abu-Mostafa/dp/...
Earlier quoted context omitted.
The scaling is brutal. If you have a 20k word vocabulary and want to do 3 grams, you need a 20000^3 matrix of elements (8 trillion). Most of which is going to be empty. GPT and friends cheat by not modeling each word separately, but a large dimensional “embedding” (just a vector if you also find new vocabulary silly). The embedding represents similar words near each other in this space. The famous king-man-queen exam…
You shouldn't need to allocate every possible combination !_! if you dynamically add new pairs/distance as you find them. Im talkin simple for loops.
https://web.stanford.edu/~jurafsky/slp3/3.pdf
Essentially you just count every n-gram that's actually in the corpus, and "fill in the blanks" for all the 0s with some simple rules for smoothing out the probability.
Agreed, but PAC-Bayes or other descendants of VC theory is probably not the best explanation. The notion of algorithmic stability provides a (much) more compelling explanation. See [1] (particularly Sections 11 and 12) [1] https://arxiv.org/abs/2203.10036
DNNs do not have special generalization powers. If anything, their generalization is likely weaker than more mathematically principled techniques like the SVM. If you try to train a DNN to solve a classical ML problem like the "Wine Quality" dataset from the UCI Machine Learning repo [0], you will get abysmal results and overfitting. The "magic" of LLMs comes from the training paradigm. Because the optimization is wo…
Earlier quoted context omitted.
Apparently the word “delve” is the biggest indicator of the use of ChatGPT according to Paul Graham.
Non native speaker here. Will remember this. Hm... Saw that, I have used it multiple times in my comment. I was just trying to convey the meaning. What is right use of word? What would be right word to use here?
If anyone wants to delve into machine learning, one of the superb resources I have found is, Stanfords "Probability for computer scientists"( https://www.youtube.com/watch?v=2MuDZIAzBMY&list=PLoROMvodv4... ). It delves into theoretical underpinnings of probability theory and ML, IMO better than any other course I have seen. (Yeah, Andrew Ng is legendary, but his course demands some mathematical familarity with linear…
There is a course reader for CS109 [1]. You can download pdf version of this.
There is also book[2] for excellent caltech course[3].
[1] https://chrispiech.github.io/probabilityForComputerScientist...
[2] https://www.amazon.com/Learning-Data-Yaser-S-Abu-Mostafa/dp/...