Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Examples of floating point problems

jvns.ca

Post reply on HN
Print view View original

91–100 of 179 posts

Re: Examples of floating point problems

#91

Post by toolslive »

in computer graphics, some people try to accumulate the transformations in 1 matrix. So A_{n+1} = T_{n} * A_{n} where T_{n} is a small transformation like a rotation around an axis

They learn by experience that they also slowly accumulate errors and end up with a transformation matrix A that's no longer orthogonal and will skew the image.

Or people try to solve large lineair systems of floating points with a naive Gaussian elimination approx and end up with noise. Same with naive iterative eigen vector calculations.

Re: Examples of floating point problems

#92

Post by toolslive »

toolslive wrote: post #91

in computer graphics, some people try to accumulate the transformations in 1 matrix. So A_{n+1} = T_{n} * A_{n} where T_{n} is a small transformation like a rotation around an axis They learn by experience that they also slowly accumulate errors and end up with a transformation matrix A that's no longer orthogonal and will skew the image. Or people try to solve large lineair systems of floating points with a naive Ga…

moreover.... floating point addition is not commutative, nor is is associative.

Re: Examples of floating point problems

#93

Post by abnry »

maximilianburke wrote: post #83
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

I think the concept of NaNs are sound, but I think relying on them is fraught with peril, made so by the unobvious test for NaN-ness in many languages (ie, "if (x != x)"), and the lure of people who want to turn on "fast math" optimizations which do things like assume NaNs aren't possible and then dead-code-eliminate everything that's guarded by an "x != x" test. Really though, I'm a fan, I just think that we need be…

NaNs are incredibly helpful in numpy. Often I want to take a mean over a collection of lists but the size of each list isn't constant. Np.nanmean works great.

Re: Examples of floating point problems

#94

Post by raphlinus »

owisd wrote: post #77

My 'favourite' is that the quadratic formula -b±sqrt(b²-4ac)/2a falls apart when you solve for the positive solution using floating point for cases where ε=b²/4ac is small, the workaround being to use the binomial expansion -b/2a*(0.5ε-0.125ε²+O(ε³))

This one is fun! However, that particular formula is not my favorite, as it doesn't behave nicely when a is small. The art here is deciding what to special case.

The way I approach this[1] is to choose the sign of the ± to be the same as -b to compute the first root x1. Then the second root is c/(a * x1). There are a few other checks in the code for things overflowing, but that basically gives you very accurate answers across the range.

This is explained a bit in a good Math Exchange post[2]. Jim Blinn's "How to solve a quadratic equation" also makes good reading.

Wait til you get to cubics and quartics.

[1]: https://docs.rs/kurbo/latest/src/kurbo/common.rs.html#116-16...

[2]: https://math.stackexchange.com/questions/866331/numerically-...

Re: Examples of floating point problems

#95

Post by BeetleB »

> addition isn’t associative (x + (y + z)) is different from (x + y) + z))

A thousand thanks for not saying "addition is not commutative".

(Addition is commutative in floating point. It merely is not associative).

Re: Examples of floating point problems

#96

Post by thfuran »

inetknght wrote: post #85
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> This means that NaNs propagate from the source of the original NaN to the final printed result. An exception would be better. Then you immediately get at the first problem instead of having to track down the lifetime of the observed problem to find the first problem.

Yeah, or using a type system that lets you avoid making NaN a float to both force the case to be handled and prevent NaN from leaking into the math.

Re: Examples of floating point problems

#97

Post by jcranmer »

inetknght wrote: post #85
WalterBright wrote: post #65

> NaN/infinity values can propagate and cause chaos NaN is the most misunderstood feature of IEEE floating point. Most people react to a NaN like they'd react to the dentist telling them they need a root canal. But NaN is actually a very valuable and useful tool! NaN is just a value that represents an invalid floating point value. The result of any operation on a NaN is a NaN. This means that NaNs propagate from the…

> This means that NaNs propagate from the source of the original NaN to the final printed result. An exception would be better. Then you immediately get at the first problem instead of having to track down the lifetime of the observed problem to find the first problem.

You can get an exception! Just enable FP exceptions for the invalid exception, and compile all of your code such that it's FP-exception aware, and you can get the exception.

Re: Examples of floating point problems

#98

Post by defrost »

svat wrote: post #58
sacrosancty wrote: post #50

Earlier quoted context omitted.

If you have even less time, just think of them as representing physical measurements made with practical instruments and the math done with analog equipment. The common cause of floating point problems is usually treating them as a mathematical ideal. The quirks appear at the extremes when you try to to un-physical things with them. You can't measure exactly 0 V with a voltmeter, or use an instrument for measuring th…

Thanks, I actually edited my post (made the second paragraph longer) after seeing your comment. The "physical" / "analog" idea does help in one direction (prevents us from relying on floating-point numbers in unsafe ways) but I think it brings us too close to the "superstition" end of the spectrum, where we start to think that floating-point operations are non-deterministic, start doubting whether we can rely on (say…

> whether we can rely on (say) the operation 2.0 + 3.0 giving exactly 5.0 (we can!)

Can we rely upon 2.3 + 2.3 giving exactly 4.6 though?

Can we rely upon LargeInt.0 + LargeInt.0 giving exactly 2xLargeInt.0 for all integers?

Re: Examples of floating point problems

#99

Post by jcranmer »

jcparkyn wrote: post #81
WalterBright wrote: post #74

Earlier quoted context omitted.

> An option/maybe type would solve this much better. NaN's are already an option type, although implemented in hardware. The checking comes for free. > An exception would be much better You can configure the FPU to cause an Invalid Operation Exception, but I personally don't find that attractive.

> NaN's are already an option type, although implemented in hardware The compromise with this is that it makes it impossible to represent a non-optional float, which leads to the same issues as null pointers in c++/java/etc. The impacts of NaN are almost certainly not as bad (in aggregate) as `null`, but it'd still be nice if more languages had ways to guarantee that certain numbers aren't NaN (e.g. with a richer set…

> The impacts of NaN are almost certainly not as bad (in aggregate) as `null`, but it'd still be nice if more languages had ways to guarantee that certain numbers aren't NaN (e.g. with a richer set of number types).

The problem with that is that to guarantee arithmetic does not result in a NaN, you need to guarantee that 0 and infinity are not valid values, and those values can still arise from underflow/overflow of regular computation. Basically, there's no subset of floating-point numbers that forms a closed set under +, -, *, or / that doesn't include NaN. So you can define FiniteF32 (e.g.), but you can't really do anything with it without the result becoming a full-on float.

Post reply on HN