> If you’re interested read on, if you’re not, see yourself out.
Huh? Doesn't this apply to literally every piece of writing ever produced? The reader is always free to leave.
Linear Regression
31–40 of 71 posts
Re: Linear Regression
#32The next level: https://en.wikipedia.org/wiki/Symbolic_regression
Do you have personal experience with Symbolic regression? If yes, please share them briefly. What are advantages and disadvantages in comparison to other predictive algorithms? Did it showed to you better results than other predictive algorithms?
Re: Linear Regression
#33If you've got an interesting nonlinear cost function to minimise and can compute the gradient, you can do a lot worse than plugging the thing into L-BFGS-B and letting it optimise it.
e.g.
https://en.m.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%...
https://docs.scipy.org/doc/scipy/reference/generated/scipy.o...
What'll that give you compared to a simple gradient descent? It'll attempt to accelerate convergence by estimating an approximation of the Hessian matrix - all the second order partial derivatives, and use a line search algorithm to figure out a good step size for each step instead of using an arbitrary constant step size aka "learning rate". The "L-" variation of the algorithm will use a limited amount of memory when approximating the Hessian matrix, which might help if your cost function has a larger number of parameters. The "-B" variations of the algorithm will also let you set upper and/or lower bounds on each variable that will be respected during the search.
Re: Linear Regression
#34Two comments here, and I am sorry if they come across as mean: 1) I know this article isn't aimed at me, but I do truly hate the excessive emojis. 2) More substantively: not once does the word "statistics" enter here. There is not a single illustration of the idea behind linear regression, which is staggeringly simple: it is just finding the best linear fit. Anyone who looks at a 2D scatterplot can do an approximate…
> I do truly hate the excessive emojis.
I am more in the camp of actually always finding them excessive. Especially in more serious, and possibly, interesting posts.
Re: Linear Regression
#35Two comments here, and I am sorry if they come across as mean: 1) I know this article isn't aimed at me, but I do truly hate the excessive emojis. 2) More substantively: not once does the word "statistics" enter here. There is not a single illustration of the idea behind linear regression, which is staggeringly simple: it is just finding the best linear fit. Anyone who looks at a 2D scatterplot can do an approximate…
> I do truly hate the excessive emojis. I am more in the camp of actually always finding them excessive. Especially in more serious, and possibly, interesting posts.
Me too, but I am focusing more on the battle than the war :)
Re: Linear Regression
#36> Regression is any algorithm that takes a collection of inputs and predicts an output. I get that the author is writing for a certain audience, but this is a gross over-simplification.
Depends on the tradition you are coming from. In machine learning it really is just that. A synonym for continuous-valued function approximation based on training data. You may say it's only regression if it fits your favorite framework, like giving confidence values or goodness of fit etc, but that's not true in general. There so many variants, like Bayesian probabilistic regression, nonparametrics, neural nets, ran…
It’s a nonstandard definition at best. While regression is a name given to some algorithms (linear and logistic for example), this is an artifact of the statistical foundations of ML. Its standard use in ML is to describe the problem, not the solution: regression is predicting a continuous value and classification is predicting discrete categories.
Either use is ok, but e.g. a tree model predicting if an animal is a dog or a cat is not regression by any definition.
Re: Linear Regression
#37> Regression is any algorithm that takes a collection of inputs and predicts an output. I get that the author is writing for a certain audience, but this is a gross over-simplification.
Depends on the tradition you are coming from. In machine learning it really is just that. A synonym for continuous-valued function approximation based on training data. You may say it's only regression if it fits your favorite framework, like giving confidence values or goodness of fit etc, but that's not true in general. There so many variants, like Bayesian probabilistic regression, nonparametrics, neural nets, ran…
> Depends on the tradition you are coming from.
Statistics :)
Re: Linear Regression
#38Two comments here, and I am sorry if they come across as mean: 1) I know this article isn't aimed at me, but I do truly hate the excessive emojis. 2) More substantively: not once does the word "statistics" enter here. There is not a single illustration of the idea behind linear regression, which is staggeringly simple: it is just finding the best linear fit. Anyone who looks at a 2D scatterplot can do an approximate…
> I do truly hate the excessive emojis. I am more in the camp of actually always finding them excessive. Especially in more serious, and possibly, interesting posts.
I would have loved to believe that they are a fad and will pass away, but I just found myself using them in the same way that I used to use phpBB emoticons.
I think they're never going to go away, but at least we can try to eradicate them from professional discourse and education.
Re: Linear Regression
#39This is one of the most condescending things I've read in years
Yeah, I get the impression this is article is more to show off how smart the writer is, rather than about effective instruction. If you can actually read the article, you have no need for the content because you've probably encountered gradient descent already in your maths training (or could figure it out).
Also, there's no concrete examples of this algorithm in place for students to go through and built intuition and a working understanding. Even the most dense math text books I've read have problems for the reader to work through.
Re: Linear Regression
#40I would prefer an introduction via the Moore-Penrose Pseudo-Inverse. It's a lot easier, imho
I agree. What is the purpose of applying gradient descent to a linear regression problem? I mean, the pseudo inverse _is_ a closed form solution! Is it maybe more efficient for very large problems? I just don't get it.