Live data from Hacker News

Beware of Fast-Math

simonbyrne.github.io

71–80 of 233 posts

Re: Beware of Fast-Math

#71

The worst thing that strikes fear into me is seeing floating points used for real world currency. Dear god. So many things can go wrong. I always use unsigned integers counting number of cents. And if I gotta handle multiple currencies, then I'll use or make a wrapper class.

How do you store negative numbers?

Re: Beware of Fast-Math

#72
post #54

Earlier quoted context omitted.

> I get the feeling that the real problem here are the IEEE specs themselves. Well, all standards are bad when you really get into them, sure. But no, the problem here is that floating point code is often sensitive to precision errors. Relying on rigorous adherence to a specification doesn't fix precision errors, but it does guarantee that software behavior in the face of them is deterministic. Which 90%+ of the time…

> (consider gimbal lock: "fixing" that requires understanding quaternions or some other orthogonal orientation space, and that's hard!) Or just avoiding gimbal lock by other means. We went to the moon using Euler angles, but I don't suppose there's much of a choice when you're using real mechanical gimbals.

That is the "tuning" solution. And mostly it works by limiting scope of execution ("just don't do that") and if that doesn't work by having some kind of recovery method ("push this button to reset", probably along with "use this backup to recalibrate"). And it... works. But the bug is still a bug. In software we prefer more robust techniques.

FWIW, my memory is that this was exactly what happened with Apollo 13. It lost its gyro calibration after the accident (it did the thing that was the "just don't do that") and they had to do a bunch of iterative contortions to recover it from things like the sun position (because they couldn't see stars out the iced-over windows).

NASA would have strongly preferred IEEE doubles and quaternions, in hindsight.

Re: Beware of Fast-Math

#73
post #36
post #31

I get the feeling that the real problem here are the IEEE specs themselves. They include a huge bunch of restrictions that each individually aren't relevant to something like 99.9% of floating point code, and probably even in aggregate not a single one is relevant to a large majority of code segments out in the wild. That doesn't mean they're not important - but some of these features should have been locally opt-in,…

How does IEEE 754 prevent auto-vectorisation?

The spec doesn’t prevent auto-vectorization, it only says the language should avoid it when it wants to opt in to producing “reproducible floating-point results” (section 11 of IEEE 754-2019). Vectorizing can be implemented in different ways, so whether a language avoids vectorizing in order to opt in to reproducible results is implementation dependent. It also depends on whether there is an option to not vectorize. If a language only had auto-vectorization, and the vectorization result was deterministic and reproducible, and if the language offered no serial mode, this could adhere to the IEEE spec. But since C++ (for example) offers serial reductions in debug & non-optimized code, and it wants to offer reproducible results, then it has to be careful about vectorizing without the user’s explicit consent.

Re: Beware of Fast-Math

#74
post #71

The worst thing that strikes fear into me is seeing floating points used for real world currency. Dear god. So many things can go wrong. I always use unsigned integers counting number of cents. And if I gotta handle multiple currencies, then I'll use or make a wrapper class.

How do you store negative numbers?

Maybe as in accounting, one column for benefits, one for debts?

Re: Beware of Fast-Math

#75

“Nothing brings fear to my heart more than a floating point number.” - Gerald Jay Sussman Is there any IEEE standards committee working on FP alternative for examples Unum and Posit [1],[2]. [1] Unum & Posit: https://posithub.org/about [2] The End of Error: https://www.oreilly.com/library/view/the-end-of/978148223986...

Is this sarcasm? If not, the proposed posit standard, IEEE P3109.

The current P3109 draft has no posits in it.

Re: Beware of Fast-Math

#76
post #4

I helped design an API for "algebraic operations" in Rust: https://github.com/rust-lang/rust/issues/136469 >, which are coming along nicely. These operations are 1. Localized, not a function-wide or program-wide flag. 2. Completely safe, -ffast-math includes assumptions such that there are no NaNs, and violating that is undefined behavior . So what do these algebraic operations do? Well, one by itself doesn't do much…

-ffast-math is actually something like 15 separate flags, and you can use them individually if you want. 3 of them are "no NaNs," "no infinities," and "no subnormals." Several of the other flags allow you to treat math as associative or distributive if you want that.

The library has some merit, but the goal you've stated here is given to you with 5 compiler flags. The benefit of the library is choosing when these apply.

Re: Beware of Fast-Math

#78
post #41
post #36

Earlier quoted context omitted.

How does IEEE 754 prevent auto-vectorisation?

If you write a loop `for x in array { sum += x }` Then your program is a specification that you want to add the elements in exactly that order, one by one. Vectorization would change the order.

The bigger problem there is the language not offering a way to signal the author’s intent. If an author doesn’t care about the order of operations in a sum, they will still write the exact same code as the author who does care. This is a failure of the language to be expressive enough, and doesn’t reflect on the IEEE spec. (The spec even does suggest that languages should offer and define these sorts of semantics.) Whether the program is specifying an order of operations is lost when the language offers no way for a coder to distinguish between caring about order and not caring. This is especially difficult since the vast majority of people don’t care and don’t consider their own code to be a specification on order of operations. Worse, most people would even be surprised and/or annoyed if the compiler didn’t do certain simplifications and constant folding, which change the results. The few cases where people do care about order can be extremely important, but they are rare nonetheless.

Re: Beware of Fast-Math

#79

The worst thing that strikes fear into me is seeing floating points used for real world currency. Dear god. So many things can go wrong. I always use unsigned integers counting number of cents. And if I gotta handle multiple currencies, then I'll use or make a wrapper class.

Wrappers are good even when non dealing with multiple currencies since in many places some transactions are in fractions of cents, so depending on the usecase may need to push that decimal a few places out.

I always have a wrapper class to put the logic of converting to whole currency units when and if needed, as well as when requirements change and now you need 4 digits past the decimal instead of 2, etc.

Re: Beware of Fast-Math

#80

The worst thing that strikes fear into me is seeing floating points used for real world currency. Dear god. So many things can go wrong. I always use unsigned integers counting number of cents. And if I gotta handle multiple currencies, then I'll use or make a wrapper class.

I've been having an interesting challenge relating to this recently. I'm trying to calculate costs for LLM usage, but the amounts of money involved are so tiny. Gemini 1.5 Flash 8B is $0.0375 per million tokens!

Should I be running my accounting system on units of 10 billionths of a dollar?

Post reply on HN