Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seeing Theory: A Visual Introduction to Probability and Statistics

seeing-theory.brown.edu

Post reply on HN
Print view View original

51–60 of 94 posts

Re: Seeing Theory: A Visual Introduction to Probability and Statistics

#51

Post by jibal »

mturmon wrote: post #29
jtsuken wrote: post #14

"Don't trust your intuition". This should be the basis for all teaching in statistics and probability. If all goes wrong, it should be the one thing everyone remembers from their statistics education. And yet year after year, everyone is starting with E(X)=sum(x*P(x)) and has no idea what it was about afterwards. With calculus and linear algebra your gut feel is about right no average. You can quickly get a feel for…

Wow, strong disagree. Once you develop intuition, probability is really quite intuitive. This kind of course should be working to develop this intuition — like the conditional probability examples and the CLT examples. The computational examples inline really help here. The Monte Hall problem is more of a curiosity than a fundamental principle! (Was a TA in undergrad engineering probability for 2 years, saw my share…

"Once you develop intuition, probability is really quite intuitive"

That's a tautology.

Plenty of studies, such as the work by Kahneman and Tversky, show that humans by default have incorrect statistical intuitions. These faulty intuitions are hard to overcome, even by a considerable amount of training.

> The Monte Hall problem is more of a curiosity than a fundamental principle!

It's quite straightforward conditional probability. That so many people, including trained mathematicians, get it wrong is quite illustrative. And it's not unique ... the coins and drawers problem is similar, and one can craft many others. MH is not a mere curiosity, it's simply well known.

Re: Seeing Theory: A Visual Introduction to Probability and Statistics

#53

Post by Eugeleo »

qsort wrote: post #41
Eugeleo wrote: post #33

Earlier quoted context omitted.

I’ll copy my comment from other place in this thread, because I think it might be relevant here. I feel that most math subjects are treated either as full-on “fluff” (e.g. calculus, all computing, no theory building) or full-on theory (real analysis). A combination of intuition AND rigor is hard to come by. With that said... What textbook(s?) would you recommend for a thorough self-learning of statistics? I’m looking…

> I feel that most math subjects are treated either as full-on “fluff” (e.g. calculus, all computing, no theory building) or full-on theory (real analysis) My background is computer science and I had a similar experience. Just a caveat: I'm not arguing that we should stop teaching theory, quite the contrary: most of the times we err on the side of the fluff. In particular, the fact that many reputable institutions ar…

> My background is computer science and I had a similar experience. Just a caveat: I'm not arguing that we should stop teaching theory, quite the contrary: most of the times we err on the side of the fluff.

Bioinformatics at my uni is just the typical CS minus some hardware stuff + molecular biology minus some chemistry stuff; in other words, I'm pretty close to CS, too. And I share your opinion — at least for me, I don't believe things until I see them proven.

> My point is just that lack of intuition/operative knowledge will lead to your theoretical knowledge of the field being less in-depth and generally less helpful to you.

In addition, building an intuition can help make the understanding come faster. To give you an example, I can stare at a proof for half a day and _then_ finally get it, but one clever diagram or a descriptive commentary can save me hours of pushing through the dense text — without cutting down on the rigour (as the proof is still there). Unfortunately, it seems that maths textbooks mostly come only with the former, or the latter, but not both.

> I honestly don't think it really matters what book you are studying as an introduction to a subject

I agree that it probably doesn't matter from the content POV (i.e. the basic definitions and theorems will be there), but it could matter if we take the intuition into account.

For example, in real analysis, there's baby Rudin, but there's also all sorts of books that include all (or most of) the content, but supply it with better commentary and/or illustrations to drive the point home quicker. And I'd day that's a pretty established field, too; probably more so than stats, in fact.

Re: Seeing Theory: A Visual Introduction to Probability and Statistics

#54

Post by st1x7 »

will_pseudonym wrote: post #28
jtsuken wrote: post #14

"Don't trust your intuition". This should be the basis for all teaching in statistics and probability. If all goes wrong, it should be the one thing everyone remembers from their statistics education. And yet year after year, everyone is starting with E(X)=sum(x*P(x)) and has no idea what it was about afterwards. With calculus and linear algebra your gut feel is about right no average. You can quickly get a feel for…

On the topic of the Monty Hall problem, what helped me "believe" it more was if you change it to 1,000,000 doors, still with only 1 car, and the rest goats. You choose 1 door. The host then opens up 999,998 other doors, which all contain goats. So there are 2 doors left. Your door, and the only other door the host didn't open. Do you feel at a gut level that you should switch?

I see this argument a lot and for some reason it doesn't help me with the intuition at all. If you (wrongly) get caught up on the fact that the remaining door and your pick have the same initial probabilities of being a car, then you'll still think that switching doesn't make a difference even in the million-door case.

Here's what works for me:

- the switching strategy always gives you the opposite of your initial choice

- the initial probabilities are 2/3 goat and 1/3 car so by switching you get 2/3 car and 1/3 goat

Re: Seeing Theory: A Visual Introduction to Probability and Statistics

#55

Post by jhncls »

will_pseudonym wrote: post #28
jtsuken wrote: post #14

"Don't trust your intuition". This should be the basis for all teaching in statistics and probability. If all goes wrong, it should be the one thing everyone remembers from their statistics education. And yet year after year, everyone is starting with E(X)=sum(x*P(x)) and has no idea what it was about afterwards. With calculus and linear algebra your gut feel is about right no average. You can quickly get a feel for…

On the topic of the Monty Hall problem, what helped me "believe" it more was if you change it to 1,000,000 doors, still with only 1 car, and the rest goats. You choose 1 door. The host then opens up 999,998 other doors, which all contain goats. So there are 2 doors left. Your door, and the only other door the host didn't open. Do you feel at a gut level that you should switch?

What's unintuitive about the Monty Hall problem is the difference between mathematical Monty and a psychological Monty. It is easy to imagine Monty almost only opening doors in case the guest choose the car, and not opening anything in case they choose a goat. So, when presented with the choice, a cautious guest will hesitate to change. If, however, the guest knows from previous shows that Monty will ALWAYS open a goat door, it is still mentally hard to change the cautious strategy.

Re: Seeing Theory: A Visual Introduction to Probability and Statistics

#56

Post by blablablerg »

Eugeleo wrote: post #31

What textbook(s?) would you recommend for a thorough self-learning of statistics? I’m looking for both intuition _and_ mathematical rigor — not all proofs, but not all fluff either. I’m a bioinformatics student and I will have a semester of combined probability/stats some time this year, but I think that won’t be enough to support me given my preference for DS-based bioinformatics jobs. I’m reading Feller right now f…

I am doing harvard 110 now (free lectures on youtube) and I am working through his book Introduction in probability.

Maybe it is too basic for you, but It is focused on the intuition part and I can recommend it!

Re: Seeing Theory: A Visual Introduction to Probability and Statistics

#59

Post by jonsen »

jtsuken wrote: post #34
mturmon wrote: post #29

Earlier quoted context omitted.

Wow, strong disagree. Once you develop intuition, probability is really quite intuitive. This kind of course should be working to develop this intuition — like the conditional probability examples and the CLT examples. The computational examples inline really help here. The Monte Hall problem is more of a curiosity than a fundamental principle! (Was a TA in undergrad engineering probability for 2 years, saw my share…

I can't argue with "once you develop intuition, probability is intuitive". I was arguing that lessons starting with E(X)=... basically stop the majority of people from getting to the point, where they see how their "initial intuition" is wrong. Convincing as many people as possible that statistical intuition is not something we are born with should be the key priority of any probability and statistics class. Monte Ha…

> Monte Hall was one example.

Monty Hall is not a good example, unless it is explicitly stated that Monty knows where the car is and that he deliberately opens a door with a goat. Just look at the discussions in the comment here.

Re: Seeing Theory: A Visual Introduction to Probability and Statistics

#60

Post by jonsen »

jibal wrote: post #51
mturmon wrote: post #29

Earlier quoted context omitted.

Wow, strong disagree. Once you develop intuition, probability is really quite intuitive. This kind of course should be working to develop this intuition — like the conditional probability examples and the CLT examples. The computational examples inline really help here. The Monte Hall problem is more of a curiosity than a fundamental principle! (Was a TA in undergrad engineering probability for 2 years, saw my share…

"Once you develop intuition, probability is really quite intuitive" That's a tautology. Plenty of studies, such as the work by Kahneman and Tversky, show that humans by default have incorrect statistical intuitions. These faulty intuitions are hard to overcome, even by a considerable amount of training. > The Monte Hall problem is more of a curiosity than a fundamental principle! It's quite straightforward conditiona…

> It's quite straightforward ...

No it is not, unless it is explicitly stated that Monty knows where the car is and that he deliberately opens a door with a goat. Just look at the discussions in the comment here.

Post reply on HN