Recommended Self-Study Path for Statistics

The only advanced textbook on statistics that I would recommend is: Kennedy, Peter (2008) Guide to Econometrics, 6E

TL;DR What to focus on? Some statistics advice: Don't be a liar. Don't be a biased idiot. Don't fuck up. The software should handle the rest.

Nuance is a fair serious question. And this could easily turn into a debate of semantics or philosophy (will add links at bottom tho^).

But what I meant was statistics in practice isn't about proof of some truth but about chance of disproof. An analogy in jurisprudence: there is a difference between "not guilty" and "innocent".

Someone may or may not be "innocent". There's even presumption of innocence. But then in practice, lawyers give evidence to a jury to decide beyond a reasonable doubt if someone is "guilty" or "not guilty".

What's the focus? It sure looks like the work is more focused on "not guilty" vs "innocent".

Furthermore, in statistics there are errors...eg statistical errors, random errors, systematic errors, type 1 errors, non-sampling errors...lots of errors. You can't eliminate them. But you can be aware of them and reduce them where possible.

Now, statistical software deals with errors to the extent statistics techniques exist and the technology can handle the process. Sort of like spellcheck.

But software can't fix everything. Most importantly it can't fix if the person using software is an idiot.

Too many times I have looked like an idiot for sending an email where spellcheck put the wrong word. What to do? I could write a new algo to make spellcheck better or I can just double check the email next time.

^Links to semantics and philosophy stuff: Some fields try to have precise, official definitions for words like "error" and "accuracy".

See ISO 5725 or longer list of examples on wikipedia: https://en.wikipedia.org/wiki/Accuracy_and_precision

Of course, philosophy also addresses the nuances. Way more fun to read than ISO technical documentation.

Short list of philosophy of statistics issues on wiki: https://en.wikipedia.org/wiki/Philosophy_of_statistics.

Better, longer list, which is worth reading as it includes more interesting and broader philosophy of science issues: http://plato.stanford.edu/entries/statistics/

If lists of philosophies are overwhelming and you want one random example of it...What is the probability the sun rises tomorrow?

Long post. Lastly, a joke: 'A physicist, an engineer, and a statistician go duck hunting. They spot a duck in the distance and the physicist takes the first shot, but just misses left. The engineer shoots next, but just misses right. The statistician yells, “we got it!”'.

[Edit] At this point I might as well add Buffet's 2 rules for investing: "Rule No. 1: Never lose money. Rule No. 2: Never forget rule No.1”