Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Gustafson’s crusade to replace floating point with something better

nextplatform.com

Post reply on HN
Print view View original

71–80 of 201 posts

Re: John Gustafson’s crusade to replace floating point with something better

#71

Post by Zardoz84 »

it_aint_that wrote: post #31
dragontamer wrote: post #24

Earlier quoted context omitted.

Yes and no. 1. Yes -- On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time, even though the machine did things 80-bits at a time). 2. No -- Modern x86 systems use SSE registers, which are 64-bits. There are a whole s…

x87 has been functionally superseded (largely by various SSE iterations), but it is still supported for backwards-compatibility. Modern applications and compilers largely do not use x87, but they can, and x87 results depending on hidden bits persists in modern x86_64 CPUs.

DLang allow to use it explicitly by using Real type, I instead of Double and Float types.

Re: John Gustafson’s crusade to replace floating point with something better

#72

Post by myaH »

xhgdvjky wrote: post #50
savant_penguin wrote: post #16

"(even the same computation on the same system can produce different results for floats)" wait...what? Is this real?

this is not allowed by the standard. it is false in practice too

The correct comment at the bottom again, while everyone else misinterprets IEEE.

Re: John Gustafson’s crusade to replace floating point with something better

#73

Post by [deleted] »

coldtea wrote: post #64
enriquto wrote: post #62

Posits and other floating variants are seriously cool, and Gustafson work is amazing. Sadly, the guy has a very annoying writing style that makes him sound like a crackpot. The advantages of posits would shine much more if they were not mixed with ridiculous language (posits are floating point numbers, thus they cannot replace them) and outlandish claims (IEEE floating point is deterministic, to the apparent contradi…

> IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson What about this then, which somebody comments below: "On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time,…

[deleted]

Re: John Gustafson’s crusade to replace floating point with something better

#74

Post by [deleted] »

jacobolus wrote: post #9
jchw wrote: post #5

Obviously for any 32 bit value there’s only 2^32 possible values. So logically, a better number format is all about the distribution of values and reducing redundancy. It sounds like the basic idea here is that rather than a fixed amount of significant figures, you get more precision in the middle. Meanwhile, you can also get larger exponents. Is that right?

The ideas are (1) gradual instead of hard overflow [IEEE floats do gradual underflow via denormals], (2) crowding representable numbers more precisely around 1 while letting very large or very small numbers get less and less precise, vs. floating point which (except for denormals) is scale free within its range, (3) making a format which can be extended by just adding bits without new definitions, so that you can get…

[deleted]

Re: John Gustafson’s crusade to replace floating point with something better

#75

Post by yholio »

Do these techniques make it easier or harder to implement hardware floating point units?

Storage and transmission speed have progressed exponentially while execution units have become the bottleneck. A floating point format that is 20% denser or more accurate but that requires a 2x number of gate delays to implement is major step backwards, except maybe in highly specialized applications.

Re: John Gustafson’s crusade to replace floating point with something better

#76

Post by theclaw »

jjtheblunt wrote: post #43
_kst_ wrote: post #35

Earlier quoted context omitted.

UNIX timestamps are normally stored as 32-bit or 64-bit signed integers, not as floating-point. If you want better than 1-second precision, then the type "struct timespec" (specified by POSIX) gives you nanosecond precision. Fixed-point types can also be used in languages that support them.

I thought unsigned in 64 bits, holding the number of nanoseconds since the Unix 0 time (which might be 1970, but i forget).

It’s a signed type so dates before 1970 can be represented

Re: John Gustafson’s crusade to replace floating point with something better

#77

Post by bsder »

coldtea wrote: post #66
bsder wrote: post #53

Earlier quoted context omitted.

> Quackery, mythomany,.. nobody wanted to hear about anything but IEEE standards. Mostly because there is no good evidence that what is being proposed is better. This isn't the olden days when it was difficult to demonstrate on a large enough CPU and dataset. Today we have cloud computing. If you create something better, you can demonstrate it by putting it into a numerics application and blow everybody away. The CFD…

> Mostly because there is no good evidence that what is being proposed is better. Err, it's math, there doesn't need to be "evidence". They can do the calculations for themselves and see...

It's not my job to prove your assertion.

If you want me to believe something, the jobs is yours to cough up the convincing evidence.

Re: John Gustafson’s crusade to replace floating point with something better

#79

Post by bsder »

ricardobeat wrote: post #61
bsder wrote: post #53

Earlier quoted context omitted.

> Quackery, mythomany,.. nobody wanted to hear about anything but IEEE standards. Mostly because there is no good evidence that what is being proposed is better. This isn't the olden days when it was difficult to demonstrate on a large enough CPU and dataset. Today we have cloud computing. If you create something better, you can demonstrate it by putting it into a numerics application and blow everybody away. The CFD…

This paper on UK weather simulation, cited in the article, seems like pretty good evidence: https://posithub.org/conga/2019/docs/13/1100-MilanKlower.pdf

That only really compares posits to Float16 in a domain where the paper admits roundoff error mostly isn't really an issue even at single precision.

I'd be much more interested to see how these handle stiff systems with multiple time constants. That's a domain where everything has problems and improvement is going to actually move some things from completely infeasible to actually simulatable.

That's a much stronger use case. And you don't have to worry about implementation efficiency since the improvement ratio is "infinity"--you went from can't do it at all to actually being able to do it.

Re: John Gustafson’s crusade to replace floating point with something better

#80

Post by DougBTX »

ambrop7 wrote: post #52

> It also does away with rounding errors What? Surely it does not have infinite precision with a finite number of bits, and also doesn't seem to be a rational number representation.

Hm, perhaps the answer is a subtle definition of "rounding error".

A rounding error is when you have a number, lets say 0.75, and due to rounding it is recorded as 1.00. The "rounding error" is 0.25.

An alternative to rounding to 1.00 would be to have a mechanism which says "the value is between 0.50 and 1.50". This way, there is no actual rounding, as it doesn't commit to a rounded value, so there is technically no "rounding error".

A neat advantage of recording an interval rather than rounding is that the "error" is preserved in the data through arithmetic, so if there is some following code that runs x * 100, a rounding mechanism would say "the value is 100" whereas an interval mechanism would say "the value is between 50 and 150". Then, if the user only looks at the output, it will be clear that there is a wide error range and something needs to be fixed, rather than the output indicating a precise answer when really it suffers from significant rounding errors.

Post reply on HN