When the U.S. air force discovered the flaw of averages
1–10 of 115 posts
Re: When the U.S. air force discovered the flaw of averages
#2[deleted]
Re: When the U.S. air force discovered the flaw of averages
#3The concept of no members of a group fitting into the average range for all observations is reminiscent of the 'curse of dimensionality'. Can anyone with a data science background make the connection here?
Secondly, this seems to explain why everyone hates autocorrect.
Re: When the U.S. air force discovered the flaw of averages
#4Is it just me or does this sound oddly familiar to machine learning recommendation systems?
ie: show an item based on the average of all the other items that everyone else has looked at.
Re: When the U.S. air force discovered the flaw of averages
#5The concept of no members of a group fitting into the average range for all observations is reminiscent of the 'curse of dimensionality'. Can anyone with a data science background make the connection here? Secondly, this seems to explain why everyone hates autocorrect.
Yeah, in this case if the 10 traits are independent and the chance of falling in the "average range" is 1/3 for any one trait then the probability of any one soldier falling the the average range for 10 traits is (1/3)^10 = 1/59049.
Re: When the U.S. air force discovered the flaw of averages
#6The concept of no members of a group fitting into the average range for all observations is reminiscent of the 'curse of dimensionality'. Can anyone with a data science background make the connection here? Secondly, this seems to explain why everyone hates autocorrect.
> The concept of no members of a group fitting into the average range for all observations is reminiscent of the 'curse of dimensionality'. Can anyone with a data science background make the connection here?
? It would seem like you have made the connection already. Although I don't see it.
Re: When the U.S. air force discovered the flaw of averages
#7The concept of no members of a group fitting into the average range for all observations is reminiscent of the 'curse of dimensionality'. Can anyone with a data science background make the connection here? Secondly, this seems to explain why everyone hates autocorrect.
Yeah, in this case if the 10 traits are independent and the chance of falling in the "average range" is 1/3 for any one trait then the probability of any one soldier falling the the average range for 10 traits is (1/3)^10 = 1/59049.
That's assuming that they're independent, though, which is certainly false. I'm curious by how much.
Re: When the U.S. air force discovered the flaw of averages
#8The concept of no members of a group fitting into the average range for all observations is reminiscent of the 'curse of dimensionality'. Can anyone with a data science background make the connection here? Secondly, this seems to explain why everyone hates autocorrect.
Yeah, in this case if the 10 traits are independent and the chance of falling in the "average range" is 1/3 for any one trait then the probability of any one soldier falling the the average range for 10 traits is (1/3)^10 = 1/59049.
But you might naturally assume a strong correlation between the middle range of each dimension - the interesting conclusion is that it's not nearly as strong as our intuition suggests.
Re: When the U.S. air force discovered the flaw of averages
#9The concept of no members of a group fitting into the average range for all observations is reminiscent of the 'curse of dimensionality'. Can anyone with a data science background make the connection here? Secondly, this seems to explain why everyone hates autocorrect.
Yes, this can be made precise: For example, consider a uniform distribution on a d-dimensional cube with side length 1.
Only a (1/2)^d fraction of the mass is within 0.25 distance of the average in every coordinate.
In high dimensions, almost all the mass is "near the boundary" in at least one coordinate.
Re: When the U.S. air force discovered the flaw of averages
#10Is it just me or does this sound oddly familiar to machine learning recommendation systems? ie: show an item based on the average of all the other items that everyone else has looked at.
[deleted]