31–40 of 96 posts
This is the TXR Lisp interactive listener of TXR 214. Quit with :quit or Ctrl-D on empty line. Ctrl-X ? for cheatsheet. 1> (defun mean (seq) (/ (sum seq) (len seq))) mean 2> (mean '(8.988465674311579e+307 8.98846567431158e+307)) ** out-of-range floating-point result ** during evaluation at expr-1:1 of form (sum seq) Works for me; error is caught. Breaking news: floating-point has a limited range! Detailed story at 11…
An 'online' mean calculation might resolve some of the common issues. Something like - def mean(ls): mu = 0.0 for i, x in enumerate(ls): mu = i/(i+1) * mu + x/(i+1) return mu
I suspect something similar could be done here, even if it would be a few times slower, to get an ulp-accurate mean regardless of the order/size of the individual elements.
If you're reaching these values then it's extremely likely that either: (a) You're doing something wrong (usually, not standardizing your data, etc.) which means that you're not thinking about the fact that computers have finite numerical precision and adjusting your problem accordingly (e.g., have a wide dynamic range of numbers). Or, (b) your problem is pretty ill-conditioned and there's probably no solving it in a…
You're right that there are issues with precision and floating point math (the article touches on this a little), but the implementation is definitely wrong if it promises to work like in the case of Python's statistics library (which was correctly using the fractions library to store arbitrary numbers, but converted to a fraction in the wrong order).
What about a "reduce" technique? Average the numbers in equal-sized chunks, then average those averages. You could even chunk the chunk averages and repeat the process as many levels down as you want to, and chunks could be as small as 2 each. I guess this still assumes that the largest number in the original list is less than or equal to the maximum floating point value, but otherwise you stay roughly in the same sp…
Unfortunately, by averaging the averages you skew the results. Average of averages does not produce the same result as averaging the whole list.
What about a "reduce" technique? Average the numbers in equal-sized chunks, then average those averages. You could even chunk the chunk averages and repeat the process as many levels down as you want to, and chunks could be as small as 2 each. I guess this still assumes that the largest number in the original list is less than or equal to the maximum floating point value, but otherwise you stay roughly in the same sp…
Unfortunately, by averaging the averages you skew the results. Average of averages does not produce the same result as averaging the whole list.
This is the TXR Lisp interactive listener of TXR 214. Quit with :quit or Ctrl-D on empty line. Ctrl-X ? for cheatsheet. 1> (defun mean (seq) (/ (sum seq) (len seq))) mean 2> (mean '(8.988465674311579e+307 8.98846567431158e+307)) ** out-of-range floating-point result ** during evaluation at expr-1:1 of form (sum seq) Works for me; error is caught. Breaking news: floating-point has a limited range! Detailed story at 11…
#lang racket/base
(define (mean seq) (/ (apply + seq) (length seq)))
(mean '(#e8.988465674311579e+307 #e8.98846567431158e+307))
What about a "reduce" technique? Average the numbers in equal-sized chunks, then average those averages. You could even chunk the chunk averages and repeat the process as many levels down as you want to, and chunks could be as small as 2 each. I guess this still assumes that the largest number in the original list is less than or equal to the maximum floating point value, but otherwise you stay roughly in the same sp…
An interesting article describing a useful framework. In addition to the largest numbers that floating point can handle, a related issue can come up with numbers that are not near the edge. A good student of floating point numbers will notice that there is a good chance you will get a different result summing a list of floating point numbers if they are sorted in a different order. This is due to the rounding error t…
Is this method more accurate than Kahan summation? And if so, is it worth the extra cost of sorting?