Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Herbie: Automatically improve imprecise floating point formulas

herbie.uwplse.org

Post reply on HN
Print view View original

31–40 of 47 posts

Re: Herbie: Automatically improve imprecise floating point formulas

#31

Post by pavpanchekha »

andrepd wrote: post #23
perching_aix wrote: post #20

Earlier quoted context omitted.

Could you substantiate that a bit more? I don't see what'd be hard to understand about it at a skim.

Is it an arithmetic average of relative error over the given range? Because if yes then it can be misleading, and potentially a bad meshes to rank alternatives (though the HTML report includes a graph over the input range, which is quite nice, so I'm talking only about the accuracy number). In the limit, an alternative with 10x better accuracy when x>10^150 and 10x worse in 1 Furthermore, floats have underflow to 0 a…

Documented here but yes it's an average, of something similar to but not exactly the same as relative error: https://herbie.uwplse.org/doc/latest/error.html

It's true that averages can be misleading but we encourage users to think about it instead as a percentage of inputs. In practice the error distribution is very bimodal, the two modes being "basically fine" (a few ulps of error) and "garbage" (usually 0 instead of some actual value)

Re: Herbie: Automatically improve imprecise floating point formulas

#32

Post by AlotOfReading »

andrepd wrote: post #17
lifthrasiir wrote: post #14

Earlier quoted context omitted.

That does make sense, because a half of all available fp numbers are less than 1 in their magnitude. In particular there should be a plenty of numbers x such that |x| << 1 so x + 1 ~= 1; in fact, the proportion should be just shy of 50%.

That is indeed one of the problems with IEEE floats. There are only 10^80 atoms in the universe, and a Planck length is 1^-60th of the radius of the universe. But 64-bit floats have an absurd range of over 10^±300! Worse than that, notice that there are as many bit patterns in the never-used range between 10^300 and 10^301 as there are in the super-important range between 1 and 10! Super wasteful. Not to mention the…

Note that the logarithmic distribution of float density is also key to certain kinds of efficient hardware float implementations, because it means you can use the fixed mantissa bits alone as table indices. Unums have proven difficult to build efficient HW implementations for.

IEEE floats have a few warts like any other 1980s standard, but they're a fantastic design.

Re: Herbie: Automatically improve imprecise floating point formulas

#33

Post by irchans »

FabHK wrote: post #21
lifthrasiir wrote: post #14

Earlier quoted context omitted.

That does make sense, because a half of all available fp numbers are less than 1 in their magnitude. In particular there should be a plenty of numbers x such that |x| << 1 so x + 1 ~= 1; in fact, the proportion should be just shy of 50%.

Not really. 1+x/2, however, would be a good approximation to sqrt(1+x) for small (in absolute value) x.

How about:

(* Mathematica Notation, Assume x>0 ) If[ x order x^2 error ) If[ x> 10^10, Sqrt[x], ( order 1/Sqrt[x] error ) (else) Sqrt[x +1] ] ] ( I guess the If statements take too much time. *)

Re: Herbie: Automatically improve imprecise floating point formulas

#34

Post by bee_rider »

pavpanchekha wrote: post #30
bee_rider wrote: post #5

I wonder, is there a way to only request reformulations that don’t involve branches? The tool already seems quite nice, but that might be a good feature. Also, I’m not sure I understand the speedup. Is it latency or throughput?

Author here. The speed up is modeled throughput, though the model is relatively naive. It's possible to disable branches by turning off the regimes flag, see https://herbie.uwplse.org/doc/1.0/options.html

Nice!

What’s uwplse mean? I mixed up the letters and misread it as ulp-wise which works for the project, haha.

Re: Herbie: Automatically improve imprecise floating point formulas

#35

Post by Rexxar »

LiamPowell wrote: post #12
yossi_peti wrote: post #4

Earlier quoted context omitted.

The precondition on the link you shared has -1 <= x && x <= 1, so 99 is way outside of that range. But even so, testing for x=1, which is supposed to be inside that range, 0.5 doesn't seem tolerably close to 0.4142.

I have a suspicion that the accuracy number is the mean of accuracies over all valid floats in the range (or something approximating that), which is going to be weighted towards zero where the accuracy is higher, and perhaps where sqrt near 1 has some artefacts.

The input should be the range and the distribution of probability on this range. Intuitively we have a tendency to assume an uniform probability for range [-1, 1] which is not the case if we check every doubles.

Re: Herbie: Automatically improve imprecise floating point formulas

#36

Post by fluorinerocket »

yossi_peti wrote: post #4
fluorinerocket wrote: post #2

How useful is this when you are using numbers in a reasonable range, like 10^-12 to 10^12? Generally I try to scale my numbers to be in this range, whether by picking the right units or scaling constraints and objectives when doing nonlinear programming/ optimization. Like looking at this example, https://herbie.uwplse.org/demo/b070b371a661191752fe37ce0321c... It is claimed that for the function f(x) =sqrt(x+1) -1 Ac…

The precondition on the link you shared has -1 <= x && x <= 1, so 99 is way outside of that range. But even so, testing for x=1, which is supposed to be inside that range, 0.5 doesn't seem tolerably close to 0.4142.

You're right but still, big error. I get its averaging over the range, and the floats are not uniformly distributed.

Maybe the thing to optimize the expression for is the minimize the maximum error, and not the average error. I think that's what I would care about

Re: Herbie: Automatically improve imprecise floating point formulas

#38

Post by pavpanchekha »

bee_rider wrote: post #34
pavpanchekha wrote: post #30

Earlier quoted context omitted.

Author here. The speed up is modeled throughput, though the model is relatively naive. It's possible to disable branches by turning off the regimes flag, see https://herbie.uwplse.org/doc/1.0/options.html

Nice! What’s uwplse mean? I mixed up the letters and misread it as ulp-wise which works for the project, haha.

University of Washington Programming Languages and Software Engineering (research group).

I'm not at UW any more, I'm now at Utah, but some of the Herbie team is at UW and they provide the infrastructure

Re: Herbie: Automatically improve imprecise floating point formulas

#39

Post by fouronnes3 »

Working with tensor datatypes in numerical computing, I've been wondering if it would be possible to somehow add an extra dimension to tensors that would serve as the "floating point precision" dimension, instead of a data type. After all why couldn't the bit depth be one of the tensor dims? Maybe it would be possible to implement arbitrary floating point precision that way?

Re: Herbie: Automatically improve imprecise floating point formulas

#40

Post by summarity »

fouronnes3 wrote: post #39

Working with tensor datatypes in numerical computing, I've been wondering if it would be possible to somehow add an extra dimension to tensors that would serve as the "floating point precision" dimension, instead of a data type. After all why couldn't the bit depth be one of the tensor dims? Maybe it would be possible to implement arbitrary floating point precision that way?

This is somewhat in line with the approach taken by some softfloat libraries, e.g. https://bigfloat.org/architecture.html
Post reply on HN