This is silly. If overflow is really a concern, use Welford’s algorithm to compute a running mean. It avoids the sum -> float overflow issue.
Calculating the mean of a list of numbers (2016)
81–90 of 96 posts
Re: Calculating the mean of a list of numbers (2016)
#82Earlier quoted context omitted.
Thanks for thoughtful reply. But the Kahan method aside, the idea would be to start with the smallest numbers, not the largest.
Yes, I understand that. >> if you have a list of values of which some are very large and some are very small, sort-and-add will sum up all of the small numbers into one large number before adding that sum to another large number I don't quite follow the point you're making -- can you elaborate?
Intuitively:
1e100 + 1e-100 = 1e100
Because of rounding error.
(This is true for some sufficiently large and small exponent.)
Re: Calculating the mean of a list of numbers (2016)
#83reduce(lambda x,y: (x+y)/2.0, numlist) Why wouldn't this work?
Just work out the math for N = 3 and you'll see why your solution is wrong.
X = Mean([a,b,c]) = (a+b+c)/3
Y = YourFunction([a,b,c]) = (((a+b)/2)+c)/2 = (a+b+2c)/4
X does not equal Y. It doesn't matter if the reduce is left or right, btw, it's still wrong.
Re: Calculating the mean of a list of numbers (2016)
#84No nasty tricks - these aren’t NaN or Infinity
OK but then the first example is a naive computation of the mean of two enormous numbers.
All this really illustrates is that you need to be aware of the range of your inputs and choose a solution that accomodates that.
Re: Calculating the mean of a list of numbers (2016)
#85What about a "reduce" technique? Average the numbers in equal-sized chunks, then average those averages. You could even chunk the chunk averages and repeat the process as many levels down as you want to, and chunks could be as small as 2 each. I guess this still assumes that the largest number in the original list is less than or equal to the maximum floating point value, but otherwise you stay roughly in the same sp…
That produces an unavoidable special case when the number of elements in the list is a prime number.
Just pad the end of the list with 0s, but keep the original size of the list.
Re: Calculating the mean of a list of numbers (2016)
#86Earlier quoted context omitted.
Yes, I understand that. >> if you have a list of values of which some are very large and some are very small, sort-and-add will sum up all of the small numbers into one large number before adding that sum to another large number I don't quite follow the point you're making -- can you elaborate?
Intuitively: 1e100 + 1e-100 = 1e100 Because of rounding error. (This is true for some sufficiently large and small exponent.)
You don't need to go that far:
julia> 1e16 + 1 == 1e16
true
julia> 1e19 + 1000 == 1e19
trueRe: Calculating the mean of a list of numbers (2016)
#87Earlier quoted context omitted.
Yes, I understand that. >> if you have a list of values of which some are very large and some are very small, sort-and-add will sum up all of the small numbers into one large number before adding that sum to another large number I don't quite follow the point you're making -- can you elaborate?
Intuitively: 1e100 + 1e-100 = 1e100 Because of rounding error. (This is true for some sufficiently large and small exponent.)
So what? That's true regardless of whether you add small numbers in before or after the big one. If you have a lot of small numbers that add to 1e-2, and a couple of big numbers that add to 1e+200, it is totally irrelevant what order you add the numbers in, because all of the small numbers together have exactly zero influence on the sum.
But we're talking about cases where the sum of the small numbers is large enough to be detectable when measured against the large numbers, even if no individual small number is that large.
Re: Calculating the mean of a list of numbers (2016)
#88Why not convert to integer (saving residuals), computing mean of integers via long division and adding means of residuals (arbitrary precision sum)?
Re: Calculating the mean of a list of numbers (2016)
#89An interesting article describing a useful framework. In addition to the largest numbers that floating point can handle, a related issue can come up with numbers that are not near the edge. A good student of floating point numbers will notice that there is a good chance you will get a different result summing a list of floating point numbers if they are sorted in a different order. This is due to the rounding error t…
A solution more accurate than sorting before adding is to place your numbers into a priority queue and repeatedly add the smallest two numbers and re-insert the result into the queue. This helps handle the case where you have many similarly valued numbers and your running sum becomes large enough relative to your numbers to cause the same rounding errors.
Re: Calculating the mean of a list of numbers (2016)
#90An interesting article describing a useful framework. In addition to the largest numbers that floating point can handle, a related issue can come up with numbers that are not near the edge. A good student of floating point numbers will notice that there is a good chance you will get a different result summing a list of floating point numbers if they are sorted in a different order. This is due to the rounding error t…
A solution more accurate than sorting before adding is to place your numbers into a priority queue and repeatedly add the smallest two numbers and re-insert the result into the queue. This helps handle the case where you have many similarly valued numbers and your running sum becomes large enough relative to your numbers to cause the same rounding errors.
Isn't a priority queue implicitly sorted?