Kind of weird not to see β̂ = (XᵀX)⁻¹Xᵀy
Important machine learning equations
31–39 of 39 posts
Re: Important machine learning equations
#32> This blog post has explored the most critical equations in machine learning, from foundational probability and linear algebra to advanced concepts like diffusion and attention. With theoretical explanations, practical implementations, and visualizations, you now have a comprehensive resource to understand and apply ML math. Point anyone asking about core ML math here—they’ll learn 95% of what they need in one place…
[deleted]
Re: Important machine learning equations
#33Earlier quoted context omitted.
What does this comment have to do with the previous comment, which talked about supervised learning?
Reread the comment "Backprop is just a way to compute the gradients of the weights with respect to the cost function, not an algorithm to minimize the cost function wrt. the weights." What does the word supervised mean? It's when you define a cost function to be the difference between the training data and the model output. Aka something like (f(x)-y)^2 which is simply the quadratic difference between the result of t…
regarding "supervised", it is a bit of a small nuance.
Traditional "supervised" training, required the dataset to be annotated with labels (good/bad, such-and-such a bounding box in an image, ...) which cost a lot of human labor to produce.
When people speak of "unsupervised" training, I actually consider it a misnomer: its historically grown, and the term will not go away quickly, but a more apt name would have been "label-free" training.
For example consider a corpus of human written text (books, blogs, ...) without additional labels (verb annotations, subject annotations, ...).
Now consider someone proposing to use next-token prediction, clearly it doesn't require additional labeling. Is it supervised? Nobody calls it supervised under the current convention, but actually one may view next-token prediction on a bare text corpus as a trick to turn an unlabeled dataset into trillions of supervised prediction tasks. Given this N-gram of preceding tokens, what does the model predict as the next token? And what does the corpus actually say as next token? Lets use this actual next token as if it were a "supervised" (labeled) exercise.
Re: Important machine learning equations
#34Earlier quoted context omitted.
Reread the comment "Backprop is just a way to compute the gradients of the weights with respect to the cost function, not an algorithm to minimize the cost function wrt. the weights." What does the word supervised mean? It's when you define a cost function to be the difference between the training data and the model output. Aka something like (f(x)-y)^2 which is simply the quadratic difference between the result of t…
regarding "supervised", it is a bit of a small nuance. Traditional "supervised" training, required the dataset to be annotated with labels (good/bad, such-and-such a bounding box in an image, ...) which cost a lot of human labor to produce. When people speak of "unsupervised" training, I actually consider it a misnomer: its historically grown, and the term will not go away quickly, but a more apt name would have been…
That's also why LeCun promoted the term "self-supervised" a while ago, with some success.
Re: Important machine learning equations
#35> This blog post has explored the most critical equations in machine learning, from foundational probability and linear algebra to advanced concepts like diffusion and attention. With theoretical explanations, practical implementations, and visualizations, you now have a comprehensive resource to understand and apply ML math. Point anyone asking about core ML math here—they’ll learn 95% of what they need in one place…
Apart from the “—“, what else gives it away? Just asking from a non-native perspective.
As someone who tended to use "—" in a lot of my writing naturally before, the prevalence of its usage by LLMs frustrate me a lot. I now have to rewrite things that felt natural just so no one will think I'm an LLM.
Re: Important machine learning equations
#36Earlier quoted context omitted.
Apart from the “—“, what else gives it away? Just asking from a non-native perspective.
Not op, but it is very clearly the final summary telling the user that the post they asked the AI to write is now created.
I stopped reading the post before that and went back to check. It's so blatant...especially when it mentions visualizations.
> With theoretical explanations, practical implementations, and visualizations, you now have a comprehensive resource to understand and apply ML math. Point anyone asking about core ML math here—they’ll learn 95% of what they need in one place!
Re: Important machine learning equations
#37@dang, I think we need to start talking about letting AI slop get to the front page and how the community can help flag things as best as we can.
Re: Important machine learning equations
#38@dang, I think we need to start talking about letting AI slop get to the front page and how the community can help flag things as best as we can.
At least in this case the community likes AI slop. It got 284 points so far.
Re: Important machine learning equations
#39Presenting information theory as a series of independent equations like this does a disservice to the learning process. Cross-entropy and KL-divergence are directly derived from information entropy, where InformationEntropy(P) represents the baseline number of bits needed to encode events from the true distribution P, CrossEntropy(P, Q) represents the (average) number of bits needed for encoding P with a suboptimal d…
Agree 100% with this. It gives the illusion of understanding, like when a precocious 6 year old learns the word "precocious" and feels smart because they have can say it. Or any movie with tech or science with .
While I can share the sentiment, my small experience teaching (and studying the same area for over a decade) suggests that giving students a trivial formula to play with "as is" helps motivate its future usage well. It is difficult to teach everything important about X in one go, knowledge is accumulated in layers.