The Matrix Calculus You Need for Deep Learning

Post by victor106 » Fri, Jun 29, 2018, 11:55 AM UTC

archgoon wrote: post #11

roel_v wrote: post #2

So I have a question somewhat related to this that I never knew where/who to ask (well actually I asked a few mathematicians at a university I work with whose answers I couldn't understand - their answers were almost as impenetrable as the Wikipedia page, and some engineering scientists who I thought would be more into 'applied math' but they didn't know. So I'm hoping some data science people reading this would bett…

So, when you perform a regression of any sort, what you're doing is saying "Hey, I want to find parameters X,Y,Z, etc, that make this curve best fit the data that I have". One interpretation of 'best fit' is 'minimize the mean squared error'. So regression is just a minimization problem. You're trying to find the values that minimize f(X,Y,Z...) And, well, that means that you just want to find values for X,Y,Z such t…

Post by hotwire » Fri, Jun 29, 2018, 12:15 PM UTC

cimmanom wrote: post #16

TJSomething wrote: post #9

Earlier quoted context omitted.

Those Wikipedia pages are kind of awful for pedagogy, but they have the right equations, so I won't cover those. Say we have a curve that corresponds to how good of a fit your model is. We want to try to find the maximum on that curve. However, calculating every point of the curve is too expensive, so we want to minimize the number of points we have to check. So, we start with a guess as to the highest point on the c…

Thank you for a super clear explanation that was easy to grok even with no more math background than (extremely rusty) high school calculus.

Post by ghaff » Fri, Jun 29, 2018, 12:36 PM UTC

TJSomething wrote: post #9

roel_v wrote: post #2

So I have a question somewhat related to this that I never knew where/who to ask (well actually I asked a few mathematicians at a university I work with whose answers I couldn't understand - their answers were almost as impenetrable as the Wikipedia page, and some engineering scientists who I thought would be more into 'applied math' but they didn't know. So I'm hoping some data science people reading this would bett…

Those Wikipedia pages are kind of awful for pedagogy, but they have the right equations, so I won't cover those. Say we have a curve that corresponds to how good of a fit your model is. We want to try to find the maximum on that curve. However, calculating every point of the curve is too expensive, so we want to minimize the number of points we have to check. So, we start with a guess as to the highest point on the c…

Post by [deleted] » Fri, Jun 29, 2018, 1:19 PM UTC

ghaff wrote: post #25

TJSomething wrote: post #9

Earlier quoted context omitted.

Those Wikipedia pages are kind of awful for pedagogy, but they have the right equations, so I won't cover those. Say we have a curve that corresponds to how good of a fit your model is. We want to try to find the maximum on that curve. However, calculating every point of the curve is too expensive, so we want to minimize the number of points we have to check. So, we start with a guess as to the highest point on the c…

Unfortunately, mathematics is one of the areas where Wikipedia is pretty awful in general. The articles seem mostly written for people who pretty much already understand the topic in question. Of course, you always have to assume some knowledge base but the stereotypical jargon-filled Wilipedia approach is particularly off-putting in this area.

Post by Myrmornis » Fri, Jun 29, 2018, 2:13 PM UTC

ghaff wrote: post #25

TJSomething wrote: post #9

Earlier quoted context omitted.

Those Wikipedia pages are kind of awful for pedagogy, but they have the right equations, so I won't cover those. Say we have a curve that corresponds to how good of a fit your model is. We want to try to find the maximum on that curve. However, calculating every point of the curve is too expensive, so we want to minimize the number of points we have to check. So, we start with a guess as to the highest point on the c…

Unfortunately, mathematics is one of the areas where Wikipedia is pretty awful in general. The articles seem mostly written for people who pretty much already understand the topic in question. Of course, you always have to assume some knowledge base but the stereotypical jargon-filled Wilipedia approach is particularly off-putting in this area.