B = (X^TX)^(-1)X^TY anyone?
Put a pseudoinverse in there :)
Performing Linear Regression Using Ruby
11–20 of 44 posts
Re: Performing Linear Regression Using Ruby
#12...and at the end he shows you how to do the same thing with two lines of R.
It's very useful to code basic statistical algorithms yourself so you understand how they work, but for any serious analysis you'll get more reliable and performant results with a library.
Re: Performing Linear Regression Using Ruby
#13[deleted]
Re: Performing Linear Regression Using Ruby
#14Ruby is great for data prep, basic calculations, web app development, and scraping aggregating data and R for visualization. I find them to be a joyful combination. Great for small data sets, quick estimations, and various small projects. Larger data sets and performance intensive operations are better handled in Python (or Java, C++, etc). Lots of statistical analysis is way below the threshold of Hadoop and company…
I mostly agree with this, because I do like Ruby even though I don't use it. The missing link here, and the reason Python gets more love from the data community, is that Python scales down to the smaller data sets as well as it handles big ones. (Not sure if you ment it couldn't, but the distinction you make implies that.)
Python is surprisingly heavy-duty. But my kingdom for a seamlessly distributed or parallelized version of NumPy/SciPy! How nice would it be to just enter "C = A * B", with A living as a sparse CSC across many nodes?
Re: Performing Linear Regression Using Ruby
#15Earlier quoted context omitted.
I mostly agree with this, because I do like Ruby even though I don't use it. The missing link here, and the reason Python gets more love from the data community, is that Python scales down to the smaller data sets as well as it handles big ones. (Not sure if you ment it couldn't, but the distinction you make implies that.)
Python is surprisingly heavy-duty. But my kingdom for a seamlessly distributed or parallelized version of NumPy/SciPy! How nice would it be to just enter "C = A * B", with A living as a sparse CSC across many nodes?
Would Disco (http://discoproject.org/) work for you?
Re: Performing Linear Regression Using Ruby
#16Earlier quoted context omitted.
I mostly agree with this, because I do like Ruby even though I don't use it. The missing link here, and the reason Python gets more love from the data community, is that Python scales down to the smaller data sets as well as it handles big ones. (Not sure if you ment it couldn't, but the distinction you make implies that.)
Python is surprisingly heavy-duty. But my kingdom for a seamlessly distributed or parallelized version of NumPy/SciPy! How nice would it be to just enter "C = A * B", with A living as a sparse CSC across many nodes?
we, http://continuum.io/, are working on this.
Re: Performing Linear Regression Using Ruby
#17B = (X^TX)^(-1)X^TY anyone?
Did somebody say B = (X'X)^(-1)X'Y?
Here is one of my favorite OLS-related abuses of linear algebra: y = XB + e ---> e = y - XB ---> e = y - X(X'X)^(-1)X'Y ---> e = [I - X(X'X)^(-1)X'Y]y = My
Now use the two idempotent matrices to compute the SSR: e'e = (My)'(My) ---> e'e = y'M'My ---> e'e = y'MMy ---> e'e = y'My
Re: Performing Linear Regression Using Ruby
#18B = (X^TX)^(-1)X^TY anyone?
In computing (X^TX)^(-1) if the number of features is large then it can be slow as computing the inverse of a matrix is slow. Also unless you use pseudo inverse (pinv in octave) you need to take care of degenerate cases. However if you use Regularization i.e replace the (X^TX)^(-1) with (X^TX + lambda*W)^(-1),
where lambda is the regularization parameter
and W is a matrix of the form:
|0 0 0|
|0 1 0|
|0 0 1|
This ensures that the matrix is now invertible. Regularization takes care of overfitting.
P.S I'm a ml n00b doing Machine Learning course on Coursera so I might be unaware of more practical knowledge of the above. :D
Re: Performing Linear Regression Using Ruby
#19We do regressions with rb-gsl. require 'gsl' x = GSL::Vector.alloc(array_of_x_values) y = GSL::Vector.alloc(array_of_y_values) c0, c1, cov00, cov01, cov11, chisq, status = GSL::Fit::linear(x, y) It's not nearly as much work, and it's much faster than doing it in pure Ruby. :) (It also does weighted regressions and exponential fitting, among a host of other things. That wheel's gone done been invented already.)
As pointed out by Tony Arkles in blog post comments, GSL is a bit restrictive in it's licensing:
GSL can be used internally ("in-house") without restriction, but only redistributed in other software that is under the GNU GPL.
Re: Performing Linear Regression Using Ruby
#20Math stuff in ruby can be greatly speeded up by using the NArray library http://narray.rubyforge.org/ Method list here: http://narray.rubyforge.org/SPEC.en