11–20 of 96 posts
This article highlights and issue with floating point numbers, a substantial use case for data scientists (and as such, I value the input). How do REPLs and databases handle this edge case?
reduce(lambda x,y: (x+y)/2.0, numlist) Why wouldn't this work?
This article highlights and issue with floating point numbers, a substantial use case for data scientists (and as such, I value the input). How do REPLs and databases handle this edge case?
This article highlights and issue with floating point numbers, a substantial use case for data scientists (and as such, I value the input). How do REPLs and databases handle this edge case?
If you when calculating an average actually reach overflow in a double without messing up the precision first and making the calculation worthless in the first place, some numbers in the list of numbers is bogusly big anyway.
This is the TXR Lisp interactive listener of TXR 214.
Quit with :quit or Ctrl-D on empty line. Ctrl-X ? for cheatsheet.
1> (defun mean (seq) (/ (sum seq) (len seq)))
mean
2> (mean '(8.988465674311579e+307 8.98846567431158e+307))
** out-of-range floating-point result
** during evaluation at expr-1:1 of form (sum seq)
Almost any non-trivial calculation that overflows can be massaged into something that avoids overflow. One important motivation behind floating point is to have enough practical range so that you don't run into overflow.
If this is an actual practical problem someone faces, the tool to reach for is wider floating-point, not to mess with the code. A bigger "E" than +307 is a thing.
Also, multi-precision floating-point is a thing.
def mean(ls):
mu = 0.0
for i, x in enumerate(ls):
mu = i/(i+1) * mu + x/(i+1)
return muusing known stable numeric methods? Kahan-Babuška summation?
An 'online' mean calculation might resolve some of the common issues. Something like - def mean(ls): mu = 0.0 for i, x in enumerate(ls): mu = i/(i+1) * mu + x/(i+1) return mu
precision = Float::DIG + 1
require 'bigdecimal'
a = BigDecimal(8.98846567431158e+307, precision)
b = BigDecimal(8.988465674311579e+307, precision)
mean = (a + b) / 2
puts "Works" if mean > [a, b].min && mean Works
Works for small floats too:
smalls = [1.390671161567e-309, 1.390671161567e-309, 1.390671161567e-309].map { BigDecimal(@1, precision) }
mean = smalls.sum / smalls.size
puts "Works" if mean >= smalls.min && mean Works
An interesting article describing a useful framework. In addition to the largest numbers that floating point can handle, a related issue can come up with numbers that are not near the edge. A good student of floating point numbers will notice that there is a good chance you will get a different result summing a list of floating point numbers if they are sorted in a different order. This is due to the rounding error t…