Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Stein's Paradox (2021)

joe-antognini.github.io

Post reply on HN
Print view View original

11–20 of 50 posts

Re: Understanding Stein's Paradox (2021)

#11

Post by wavemode »

credit_guy wrote: post #10

Stein's paradox is bogus. Somebody needs to say that. Here's one wikipedia example: > Suppose we are to estimate three unrelated parameters, such as the US wheat yield for 1993, the number of spectators at the Wimbledon tennis tournament in 2001, and the weight of a randomly chosen candy bar from the supermarket. Suppose we have independent Gaussian measurements of each of these quantities. Stein's example now tells…

(Disclaimer, stats noob here) - I thought the point was that, you have a better chance of being -overall- closer to the mean (i.e., the 3D euclidean distance between your guess and the mean would be the smallest, on average), even though you may not necessarily have improved your odds of guessing any of the single individual means.

So it's not that "you get a better estimate of the US wheat yield by estimating also the number of Wimbledon spectators and the weight of a candy bar in a shop", it's simply that you get a better estimate for the combined vector of the three means. (Which, in this case, the vector of the three means is probably meaningless, since the three data sets are entirely unrelated. But we could also imagine scenarios where that vector is meaningful.)

Am I misunderstanding something?

Re: Understanding Stein's Paradox (2021)

#12

Post by jprete »

My intuition is that the problem is in using squares for the error. The volume of space available for a given distance of error in 3-space is O(N^3) the magnitude of the error, so an error term of O(N^2) doesn't grow fast enough compared to the volume that can contain that magnitude of error.

But I really don't know, it's just an intuition with no formalism behind it.

Re: Understanding Stein's Paradox (2021)

#13

Post by pfortuny »

hcks wrote: post #9
pfortuny wrote: post #4

I do not get it: if variance is too large, a random sample is very little representative of the mean. As simple as that? Now the specific formula may be complicated. But otherwise I do not understand the “paradox”? Or am I missing something?

In the 1D case the single point will be the best estimator for the mean no matter what the variance is

OK, this is a good reply, and very informative, many thanks. Now I understand why the thing is paradoxical.

Re: Understanding Stein's Paradox (2021)

#14

Post by credit_guy »

wavemode wrote: post #11
credit_guy wrote: post #10

Stein's paradox is bogus. Somebody needs to say that. Here's one wikipedia example: > Suppose we are to estimate three unrelated parameters, such as the US wheat yield for 1993, the number of spectators at the Wimbledon tennis tournament in 2001, and the weight of a randomly chosen candy bar from the supermarket. Suppose we have independent Gaussian measurements of each of these quantities. Stein's example now tells…

(Disclaimer, stats noob here) - I thought the point was that, you have a better chance of being -overall- closer to the mean (i.e., the 3D euclidean distance between your guess and the mean would be the smallest, on average), even though you may not necessarily have improved your odds of guessing any of the single individual means. So it's not that "you get a better estimate of the US wheat yield by estimating also t…

You are most likely right.

I am personally bothered by the way it is presented as a "paradox", with the implication that it would have real world applications.

I have zero doubts that you can't improve the estimate of the US wheat yields by looking at some other unrelated things, like candy bars. Presenting the result as if it a real "improvement" is false advertisement.

On the other hand, if we look at related observations, then the improvement is not a paradox at all. Let's say I want to estimate the average temperature in the US and in Europe. They are related, and combining the estimates will result to a better result, to nobody's surprise.

Re: Understanding Stein's Paradox (2021)

#15

Post by falseprofit »

credit_guy wrote: post #14
wavemode wrote: post #11

Earlier quoted context omitted.

(Disclaimer, stats noob here) - I thought the point was that, you have a better chance of being -overall- closer to the mean (i.e., the 3D euclidean distance between your guess and the mean would be the smallest, on average), even though you may not necessarily have improved your odds of guessing any of the single individual means. So it's not that "you get a better estimate of the US wheat yield by estimating also t…

You are most likely right. I am personally bothered by the way it is presented as a "paradox", with the implication that it would have real world applications. I have zero doubts that you can't improve the estimate of the US wheat yields by looking at some other unrelated things, like candy bars. Presenting the result as if it a real "improvement" is false advertisement. On the other hand, if we look at related obser…

Since when does “paradox” imply real world application?

In your last paragraph, what you’re describing is just inference based on correlation, which is unrelated to this topic.

Re: Understanding Stein's Paradox (2021)

#16

Post by mmmmpancakes »

credit_guy wrote: post #10

Stein's paradox is bogus. Somebody needs to say that. Here's one wikipedia example: > Suppose we are to estimate three unrelated parameters, such as the US wheat yield for 1993, the number of spectators at the Wimbledon tennis tournament in 2001, and the weight of a randomly chosen candy bar from the supermarket. Suppose we have independent Gaussian measurements of each of these quantities. Stein's example now tells…

No it is not bogus, you just don't know much stats apparently.

Re: Understanding Stein's Paradox (2021)

#18

Post by mitthrowaway2 »

kgwgk wrote: post #6
mitthrowaway2 wrote: post #3

Sorry, I'm siding with the physicists here. If you're going to declare that your seemingly arbitrary choice of coordinate system is actually not arbitrary and part of your prior information about where the mean of the distribution is suspected to be, you have to put that in the initial problem statement.

There is nothing magical about the origin, the shrinkage can be done towards any point and in fact when estimating multiple means it's customary to move each point closer to their average. https://www.math.drexel.edu/~tolya/EfronMorris.pdf

There is something magical about the origin when the result does not respect translational symmetry.

In fact, in a real world setting I would probably use my first measurement to define the origin, having no other reference to reach for.

Re: Understanding Stein's Paradox (2021)

#19

Post by kgwgk »

mitthrowaway2 wrote: post #18
kgwgk wrote: post #6

Earlier quoted context omitted.

There is nothing magical about the origin, the shrinkage can be done towards any point and in fact when estimating multiple means it's customary to move each point closer to their average. https://www.math.drexel.edu/~tolya/EfronMorris.pdf

There is something magical about the origin when the result does not respect translational symmetry. In fact, in a real world setting I would probably use my first measurement to define the origin, having no other reference to reach for.

What does not respect translational symmetry?

You have an estimator. If you apply shrinkage towards the origin you have another estimator. If you apply shrinkage towards [42, 42, ..., 42] you have yet another estimator. Etc. Is it a problem that different estimators produce different results?

Post reply on HN