How does this compare to, say "Introduction to Statistical Learning" and "Elements of Statistical Learning" by Trevor et al? As I understand, the former is also supposed to be a concise introduction to statistical concepts while the latter offers a more rigorous treatment. Where does this book fall in between?
All of Statistics, by Larry Wassserman (2013) [pdf]
41–50 of 55 posts
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#42Can someone compare this with Hastie and Tibshirani (https://lagunita.stanford.edu/courses/HumanitiesSciences/Sta...) ? I wonder which one is more practical
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#43Can someone compare this with Hastie and Tibshirani ( https://lagunita.stanford.edu/courses/HumanitiesSciences/Sta... ) ? I wonder which one is more practical
The Hastie/Tibshirani book (http://www-bcf.usc.edu/~gareth/ISL/) is very practical.
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#44Happy to see a book like this trending on hn, especially with a sentence like: "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." in it's preface. I definitely agree, since I wasted a lot of time doing fruitless surgeries before I went and learned about band aids in depth. From my look at…
> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." Having studied both statistics and neural networks, I'm not sure if I completely agree with that quote. There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural net…
> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid."
Or like programming in C without also knowing assembly and compiler theory? Or flying a plane without having a degree in aerodynamics?
I think we can extract a lot of use from high level frameworks that abstract away much of the gritty statistics and math. For most applications all we need is to have well behaved, well tested libraries and some basic intuition about how they work.
Fortunately, in machine learning almost everything is a function from inputs X to outputs y, and we know how functions work from programming. It's easy to integrate in apps. The devil is in hyperparameter tuning, but we can get away with good initializations and some measure of web research.
In time people will just use precomputed neural nets for standard tasks like Syntaxnet (text parsing), Inception (image classification) ore use web APIs to hosted services (less secure for sensitive data). We make those better, maybe fine tune them to our needs and get away with it 100x faster.
There is also work in automated hyperparameter search. Machine learning could become a black box when they get good enough.
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#45Earlier quoted context omitted.
> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." Having studied both statistics and neural networks, I'm not sure if I completely agree with that quote. There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural net…
> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." Or like programming in C without also knowing assembly and compiler theory? Or flying a plane without having a degree in aerodynamics? I think we can extract a lot of use from high level frameworks that abstract away much of the gritty s…
Straw man going on here. The problem is, whilst you can get numbers out without understanding the statistics underneath, you can easily misinterpret that output if you don't understand it.
Look at how p values are used in science journals for an example of poor stats knowledge affecting real life outputs.
There are some things in life that do require you to do the requisite reading. Things based on stats fall into that camp.
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#46Happy to see a book like this trending on hn, especially with a sentence like: "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." in it's preface. I definitely agree, since I wasted a lot of time doing fruitless surgeries before I went and learned about band aids in depth. From my look at…
> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." Having studied both statistics and neural networks, I'm not sure if I completely agree with that quote. There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural net…
I would agree with you and I also did research on neural networks when I was in college. I'm not sure if those downvoting you have actually done/seen neural network research. Deep learning is pretty engineering driven (as opposed to theory driven), right now.
I have a pretty weak understanding of statistics and from my perspective it was very common for grad students to also have a weak understanding (of course, those working in statistical learning theory had a strong understanding). This is a pretty ordinary occurrence - it shouldn't be surprising - this is sort of like pointing out that many theoretical statisticians have poor coding skills.
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#47Earlier quoted context omitted.
> "Using fancy tools like neural nets, boosting, and support vector machines without understanding basic statistics is like doing brain surgery before knowing how to use a band-aid." Having studied both statistics and neural networks, I'm not sure if I completely agree with that quote. There are lots of neural network applications that have little to do with statistics (image recognition with convolutional neural net…
Image classification has everything to do with statistics: you're guessing the probability distribution over the classes conditional on the input image vector; the model is trained through a process of statistical inference (using gradient descent).
You have stated something that someone with a high school-level (e.g. superficial) understanding of probability/statistics would understand. Most research in neural networks requires only a very superficial level of statistics.
There are certainly many areas of neural networks where statistics is important (more theoretical areas), but those don't form the core of the research field.
Also, calling (stochastic) gradient descent a form of statistical inference, while technically correct, is a ridiculous stretching of the term. No researcher considers SGD to be a statistical inference algorithm.
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#48Can someone say in few sentences what Statistics is all about? I can't shake off the feeling that it is just glorified curve fitting. Edit: Please stop the down votes, just an electrical engineer here, with one basic course in Probability and Stat. :)
Neural nets are glorified curve fitting. The are curves parameterized by the weight matrix. The weight matrix is relatively massive (e.g. 1M DOF), which makes the family of curves it generate essentially almost fluid like a piece of yarn. Now given a small amount of data, and a programmable piece of string, how well can you fit the data? Turns out the string is higher dimensional than the data, so you can fit any curve you like. The trick, it avoiding overfitting. Overfitting is the yarn warping its shape to fit noise that has no intrinsic meaning. That's what cross validation prevents ... overfitting. Stop moving the yarn to match the training data better when it fails to improve an independent performance test. Thats what machine learning is... figuring out algorithms that don't overfit and have some ability to generalize onto data not seen before. It's still basically glorified curve fitting.
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#49Who is this book supposed to be for? Given the heavy emphasis on formalism (theorem, proof, theorem, proof, theorem, proof), and the lack of a single example that actually computes a number, I hazard a guess that this book is not for people who actually want to apply statistics to real problems. A while back I had to teach myself Fisher matrices and the Cramér–Rao bound to solve a problem I was working on. I quickly…
I found this book to be a godsend. I never took statistics and always wanted to better understand the deep conceptual ideas in the field. I had so many frustrating experiences with books that came highly recommend to me, and turned out to be not what I wanted at all. They spend chapters and chapters beating around the bush, conversationally talking about general ideas around data management and measurement bias and research design and different ways of charting data sets.
I cannot tell you how frustrating this was for me. I wanted just the meat: the core mathematical concepts on which statistical models and inferences are built. Don't tell me a folksy story about gathering soil samples, show me the tools and what they can do, both their power and their limitations. I can think for myself about how to apply those concepts.
I loved this book for being exceptionally clear and terse. I was hooked from the first sentence: "Probability is a mathematical language for quantifying uncertainty." That one sentence makes the concept clear in a way that the entire chapter on probability from "Statistics in a Nutshell" (http://www.amazon.com/Statistics-Nutshell-Sarah-Boslaugh/dp/...) did not.
I'm not someone who thrives on theorems and proofs, I thrive on concepts. And I found this book dense with clear explanations of the key concepts.
Re: All of Statistics, by Larry Wassserman (2013) [pdf]
#50While All of Statistics is wonderful in its genre, it really isn't a good place to start to learn statistics. Firstly because it focuses very heavily on the theory and contains very little on practical modeling. Secondly because the theory isn't even necessarily going to be very enlightening: frequentist statistics is a mathematical tour de force, using every possible hack you can think of to be able to draw statistical conclusions from nothing more than a few pen and paper calculations, but as a result frequentist theory won't actually give you any sort of deeper insight into the core theoretical foundations of probability and statistics.
If you're new to statistics, try Allen Downey's http://greenteapress.com/thinkstats2/index.html or Brian Blais' http://web.bryant.edu/~bblais/statistical-inference-for-ever.... Both are free.
Then, go in depth on regression. Not just feeding in the numbers and getting back a fitted model, but actually knowing how everything works, what the common issues are, how to interpret the estimates and so on. Once you've got that down, read Regression Modeling Strategies by Harrell to go really in depth.
Or if you're really just interested in prediction, Hastie and Tibshirani is wonderful of course.