Herbie: Automatically improve imprecise floating point formulas • Hacker News Board

Re: Herbie: Automatically improve imprecise floating point formulas

#41

Post by andrepd » Sat, Apr 04, 2026, 7:10 PM UTC

AlotOfReading wrote: post #32

andrepd wrote: post #17

Earlier quoted context omitted.

That is indeed one of the problems with IEEE floats. There are only 10^80 atoms in the universe, and a Planck length is 1^-60th of the radius of the universe. But 64-bit floats have an absurd range of over 10^±300! Worse than that, notice that there are as many bit patterns in the never-used range between 10^300 and 10^301 as there are in the super-important range between 1 and 10! Super wasteful. Not to mention the…

Note that the logarithmic distribution of float density is also key to certain kinds of efficient hardware float implementations, because it means you can use the fixed mantissa bits alone as table indices. Unums have proven difficult to build efficient HW implementations for. IEEE floats have a few warts like any other 1980s standard, but they're a fantastic design.

> Unums have proven difficult to build efficient HW implementations for

Valid point, but not quite true anymore. It comes down basically to the latency of count_leading_ones/zeros for decoding the regime, on which everything else depends. But work has been done in the past ~2ish years and we can have posit units with lower latency than FP units of the same width! https://arxiv.org/abs/2603.01615

> IEEE floats have a few warts like any other 1980s standard, but they're a fantastic design.

Hmm I don't know if I would call it a fantastic design x) The "standard" is less a standard than a rough formalisation of a specific FPU design from back in the 1980s, and that design was in turn not really the product of a forward thinking visionary but something to fit the technical and business constraints of that specific piece of hardware.

It has more than a few warts and we can probably do much better nowadays. That's not really a diss on IEEE floats or their designers, it's just a matter of fact (which honestly applies to very many things which are 40 years old, let alone those designed under the constraints of IEEE754).

Re: Herbie: Automatically improve imprecise floating point formulas

#42

Post by bhargavkk » Sat, Apr 04, 2026, 9:03 PM UTC

fouronnes3 wrote: post #37

This is very interesting! I made an interval arithmetic calculator a few months ago [0]. I wonder if you could combine the two techs to make something even more powerful. [0] https://victorpoughon.github.io/interval-calculator/

Herbie uses interval arithmetic a lot! The team has a in-house developed interval arithmetic library you can use at: https://crates.io/crates/rival3.

Re: Herbie: Automatically improve imprecise floating point formulas

#43

Post by AlotOfReading » Sat, Apr 04, 2026, 9:05 PM UTC

andrepd wrote: post #41

AlotOfReading wrote: post #32

Earlier quoted context omitted.

Note that the logarithmic distribution of float density is also key to certain kinds of efficient hardware float implementations, because it means you can use the fixed mantissa bits alone as table indices. Unums have proven difficult to build efficient HW implementations for. IEEE floats have a few warts like any other 1980s standard, but they're a fantastic design.

> Unums have proven difficult to build efficient HW implementations for Valid point, but not quite true anymore. It comes down basically to the latency of count_leading_ones/zeros for decoding the regime, on which everything else depends. But work has been done in the past ~2ish years and we can have posit units with lower latency than FP units of the same width! https://arxiv.org/abs/2603.01615 > IEEE floats have a…

I'm sure you're much more knowledgeable about this than I am, but that's kind of my point. A month old preprint is the first thing to compare to implementations of a mildly evolved, warty old standard from 40 years ago. I consider that fantastic.

Thanks for the paper though. Looking forward to reading it more closely when I have time.

Re: Herbie: Automatically improve imprecise floating point formulas

#44

Post by andrepd » Sun, Apr 05, 2026, 12:32 AM UTC

AlotOfReading wrote: post #43

andrepd wrote: post #41

Earlier quoted context omitted.

> Unums have proven difficult to build efficient HW implementations for Valid point, but not quite true anymore. It comes down basically to the latency of count_leading_ones/zeros for decoding the regime, on which everything else depends. But work has been done in the past ~2ish years and we can have posit units with lower latency than FP units of the same width! https://arxiv.org/abs/2603.01615 > IEEE floats have a…

I'm sure you're much more knowledgeable about this than I am, but that's kind of my point. A month old preprint is the first thing to compare to implementations of a mildly evolved, warty old standard from 40 years ago. I consider that fantastic. Thanks for the paper though. Looking forward to reading it more closely when I have time.

Absolutely! IEEE floats are ubiquitous in software and hardware. I bet good money they will still be around 40 more years into the future :) and any alternative has to fight their ubiquity.

Better alternatives have been proposed for a long time though. Posits are very nice, and even they are almost 10 years old now :p

Re: Herbie: Automatically improve imprecise floating point formulas

#45

Post by lutusp » Sun, Apr 05, 2026, 4:45 AM UTC

ashton314 wrote: post #10

lutusp wrote: post #7

[flagged]

You didn’t even look at what the tool does, did you? > If the issue is that people write bad floating-point expressions, a code-writing tutorial would be a better solution. Yeah you are just criticizing this without even looking at it. Shame.

> You didn’t even look at what the tool does, did you?

On the contrary, I did exactly that. It proactively intervenes where mathematical knowledge would be a better remedy overall. It shields programmers from their ignorance.

If floating-point code is correctly written, it can't possibly serve a useful purpose.

> Yeah you are just criticizing this without even looking at it.

See above -- don't jump to conclusions.

Re: Herbie: Automatically improve imprecise floating point formulas

#46

Post by lutusp » Sun, Apr 05, 2026, 4:59 AM UTC

dented42 wrote: post #9

lutusp wrote: post #7

[flagged]

Did you read what this does? Because I get the feeling you didn’t… This isn’t a library, you don’t include in your application, and it doesn’t try to replace an understanding of floating point issues on the programmers part. Is this comment written by AI?

> This isn’t a library, you don’t include in your application, and it doesn’t try to replace an understanding of floating point issues on the programmers part.

If that were true, it would serve no purpose, since competently written floating-point expressions are already optimal, given the well-understood limitations of modern floating-point processing.

> Is this comment written by AI?

That's a non sequitur that resolves nothing, and a remark that would get you disqualified in a formal debate.

Ah -- I get it. In modern times, if someone composes coherent prose, and since no mere mortals can do that any more, the reply must have originated with AI.

A reply like yours leaves the originator in the position of needing to prove a negative, which is impossible, which is why it breaks the time-honored rules of formal debate.

Re: Herbie: Automatically improve imprecise floating point formulas

#47

Post by notpushkin » Wed, Apr 08, 2026, 6:06 AM UTC

irchans wrote: post #33

FabHK wrote: post #21

Earlier quoted context omitted.

Not really. 1+x/2, however, would be a good approximation to sqrt(1+x) for small (in absolute value) x.

How about: (* Mathematica Notation, Assume x>0 ) If[ x order x^2 error ) If[ x> 10^10, Sqrt[x], ( order 1/Sqrt[x] error ) ( else ) Sqrt[x +1] ] ] ( I guess the If statements take too much time. *)

HN formatter ate up the asterisks in your code comments :) To paste code blocks here, prepend each line with 2 or more spaces:

  (* Mathematica Notation, Assume x>0 *)
  If[ x  10^10,     Sqrt[x],   (* order 1/Sqrt[x] error *)
  (* else *)         Sqrt[x+1] ] ]
  
  (* I guess the If statements take too much time. *)