11–20 of 33 posts
It's really hard for me to think that this is a good solution. Normal users should be using Float64 (where there is no similar solution), and Float32 should only be used when Float64 computation is too expensive (e.g. GPUs). In such cases, it's hard to believe that doing the math in Float64 and converting will make anyone happy.
Single-precision is too expensive for GPUs, unfortunately.
It's worth noting that the C standard explicitly disclaims correct rounding for these IEEE 754 functions (C23§F.3¶20). Also, there's a group of people who have been running tests on common libms, reporting their current accuracy states here: https://members.loria.fr/PZimmermann/papers/accuracy.pdf (that paper is updated ~monthly).
The 2012 discovery of the Higgs boson at the ATLAS experiment in the LHC relied crucially on the ability to track charged particles with exquisite precision (10 microns over a 10m length) and high reliability (over 99% of roughly 1000 charged particles per collision correctly identified).
In an attempt to speed up the calculation, researchers found that merely changing the underlying math library (which should only affect at most the last bit) resulted in some collisions being missed or misidentified.
I was a user contributing to the LHC@Home BOINC project[2], where they ran into similar problems. They simulated beam stability, so iterated on the position of the simulated particles for millions of steps. As normal in BOINC each work unit is computed at least three times and if the results don't match the work unit is queued for additional runs.
They noticed that they got a lot of work units that failed the initial check compared to other BOINC projects. Digging into it they noticed that if a work unit was computed by the same CPU manufacturer, ie all Intel CPUs, then they passed as expected. But if the work unit had been processed by mixed CPUs, ie at least one run on Intel and one run on AMD, they very often disagreed.
That's when they discovered[3] this very issue about how the rounding of various floating point functions differed between vendors.
After switching to crlibm[4] for the elementary functions they used the mixed-vendor problem went away.
[1]: https://www.davidhbailey.com/dhbtalks/dhb-icerm-2020.pdf
[2]: https://en.wikipedia.org/wiki/LHC@home
[3]: https://accelconf.web.cern.ch/icap06/papers/MOM1MP01.pdf
[4]: https://ens-lyon.hal.science/ensl-01529804/file/crlibm.pdf
It's really hard for me to think that this is a good solution. Normal users should be using Float64 (where there is no similar solution), and Float32 should only be used when Float64 computation is too expensive (e.g. GPUs). In such cases, it's hard to believe that doing the math in Float64 and converting will make anyone happy.
Or when you're bottlenecked on memory and want to store each number in four bytes instead of eight.
Floating-point is hard, and standards seem like they cater to lawyers rather than devs. But a few things are slightly misleading in the post. 1. It correctly quotes the IEEE754-2008 standard: > A conforming function shall return results correctly rounded for the applicable rounding direction for all operands in its domain and even points out that the citation is from "Section 9. *Recommended* operations" (emphasis mi…
The key thing is there are only 2*32 float32s so you can check all of them. It sounds to me like the author did this, and realized they needed some tweaks for correct answers with log.
Would working with unums side-step the issue for math problems?
It's worth noting that the C standard explicitly disclaims correct rounding for these IEEE 754 functions (C23§F.3¶20). Also, there's a group of people who have been running tests on common libms, reporting their current accuracy states here: https://members.loria.fr/PZimmermann/papers/accuracy.pdf (that paper is updated ~monthly).
That Zimmermann paper is far more useful than the article. Notably LLVM libm is correctly rounded for most single precision ops. Notable omission are crlibm/rlibm/core-math etc libs which claim to be more correct, but I suppose we can already be pretty confident about them.
It's really hard for me to think that this is a good solution. Normal users should be using Float64 (where there is no similar solution), and Float32 should only be used when Float64 computation is too expensive (e.g. GPUs). In such cases, it's hard to believe that doing the math in Float64 and converting will make anyone happy.
Single-precision is too expensive for GPUs, unfortunately.
It's worth noting that the C standard explicitly disclaims correct rounding for these IEEE 754 functions (C23§F.3¶20). Also, there's a group of people who have been running tests on common libms, reporting their current accuracy states here: https://members.loria.fr/PZimmermann/papers/accuracy.pdf (that paper is updated ~monthly).
That Zimmermann paper is far more useful than the article. Notably LLVM libm is correctly rounded for most single precision ops. Notable omission are crlibm/rlibm/core-math etc libs which claim to be more correct, but I suppose we can already be pretty confident about them.
https://github.com/J-Montgomery/rfloat/blob/8a58367db32807c8...