Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Gustafson’s crusade to replace floating point with something better

nextplatform.com

Post reply on HN
Print view View original

101–110 of 201 posts

Re: John Gustafson’s crusade to replace floating point with something better

#101

Post by stkdump »

desdiv wrote: post #38
martinpw wrote: post #28

Earlier quoted context omitted.

> On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results I spent a long time once debugging an issue arising from this. Code was basically: double x = y; [...] if (x>y) fail; Nothing in between the assignment and the check changed the value of x or y, yet the check triggered the fail condition. Turned out to be that one of…

Yes, that's a hard bug to catch, but if the mantra of "every floating point comparison must use a context-appropriate epsilon" is followed, then it wouldn't have happened in the first place.

I hate that mantra, as it produced more bugs and inaccuracies in the code I have to work with than necessary. Actually you should be able to rely on the equality of a value with its copy no matter what. We are past the FPU days, so I think we can let go of these mindless epsilons sprinkled over the code.

(Even during the FPU days, every ; should have rounded each value to the size of the variables, unless you use a special compiler switch to make FP math faster)

Re: John Gustafson’s crusade to replace floating point with something better

#102

Post by AlanSE »

ambrop7 wrote: post #52

> It also does away with rounding errors What? Surely it does not have infinite precision with a finite number of bits, and also doesn't seem to be a rational number representation.

Yes, this is the question I come to. The article betrays its own claim...

> Gustafson said that for training, 32-bit floating point is overkill and in some cases doesn’t even perform as well as the smaller 16-bit posits

If posits have a bit number, then they are not of variable accuracy. This is simply sloppy explaining.

At some point, rounding has to happen (call it an interval if you want, but you're not being helpful). As a programmer, I WANT rounding to happen. I depend on it. You don't break out the floats unless you are prepared for some information to be lost. If you try to never lose any information, then you wind up with a monstrosity like Maple and Mathematica in which all steps need to be curated by hand to occasionally reduce the giant glob of fractions and implicit solves into something concrete, but imperfect.

I read the article and took a glance at https://www.johndcook.com/blog/2018/04/11/anatomy-of-a-posit..., but I'm still completely unable to answer this question of how accuracy is shed.

Re: John Gustafson’s crusade to replace floating point with something better

#103

Post by pfortuny »

ambrop7 wrote: post #52

> It also does away with rounding errors What? Surely it does not have infinite precision with a finite number of bits, and also doesn't seem to be a rational number representation.

That is unfortunately a mistake on the writer’s side: posits do not claim that anywhere in their specification or elsewhere.

Sloppy journalism.

Re: John Gustafson’s crusade to replace floating point with something better

#104

Post by 314 »

coldtea wrote: post #64
enriquto wrote: post #62

Posits and other floating variants are seriously cool, and Gustafson work is amazing. Sadly, the guy has a very annoying writing style that makes him sound like a crackpot. The advantages of posits would shine much more if they were not mixed with ridiculous language (posits are floating point numbers, thus they cannot replace them) and outlandish claims (IEEE floating point is deterministic, to the apparent contradi…

> IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson What about this then, which somebody comments below: "On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time,…

”This behavior might not be IEEE defined, but even as IEEE undefined (and implementation dependent), it still qualifies those IEEE compliant implementations as non-deterministic.”

Although IEEE is non-deterministic for other reasons (the transcendental example below is a good illustration) this argument is incorrect.

Performing operations on a non-IEEE 80-bit representation and then using a non-IEEE comparison is not a valid argument about non-determinism in IEEE.

Re: John Gustafson’s crusade to replace floating point with something better

#105

Post by melinoe »

This has always been interesting to me. It's been awhile since I read about it last, though, and I remember reading some criticisms or concerns that posits (or unums? what's the difference?) would end up being slower for some reason. I don't remember the arguments though, or where I saw them; I think the idea was that there were some edge cases that were common enough in practice that overall it would slow things down. It would be nice to see a balanced discussion of the ideas (pros and cons).

Re: John Gustafson’s crusade to replace floating point with something better

#106

Post by Gustafnot »

xvilka wrote: post #4

Here is the official site[1] of the project. There is a request[2] to add it in the Scryer Prolog[3] (ISO Prolog implementation in Rust). And implementations in Rust[4] itself, and Julia[5] language. [1] https://posithub.org/index [2] https://github.com/mthom/scryer-prolog/issues/6 [3] https://github.com/mthom/scryer-prolog [4] https://gitlab.com/burrbull/softposit-rs [5] https://juliacomputing.com/blog/2016/03/29/un…

He will keep failing to replace IEEE floating point as long as he insists on making NEGATIVE infinity the same as POSITIVE infinity.

Also, IEEE 754 floating point standard guarantees the results of addition, subtraction, multiplication, division, and square root to be the exact correctly rounded value, ie a deterministic result, contrary to what he says.

Re: John Gustafson’s crusade to replace floating point with something better

#108

Post by mcv »

nestorD wrote: post #94
mcv wrote: post #90

This sounds really good. I find floating points completely unusable for any situation where accuracy is important. It's fine when being vaguely in the right ballpark is good enough, but I don't want to have to deal with 2 + 4.1 = 6.1000000001, or x/1000000 + y/1000000 != (x+y)/1000000.

Well, those will not be solved by posits, it is still an approximate floating point format. What it does is redistribution the precision to what Gustafson considers a better default and dropping edge cases in order to get more bits for precision. edit: One of several examples found in the "Posits: the good, the bad and the ugly" paper linked in the thread : 10.0 * 2.0 = 16.0 in posit8

Really? My impression from the article was that this was exactly one of the things posits were supposed to fix, at least for numbers with small exponents.

I'm not sure how 10.0 * 2.0 = 16.0. I'm not sure what posit8 means, but it can only be correct if it switches halfway from base 8 representation to base 10, which is a bit weird, but at least the calculation is correct. (Otherwise it would be so incorrect to be unusable for anything.)

Re: John Gustafson’s crusade to replace floating point with something better

#109

Post by gwbas1c »

> Better yet, he claims the new format is a “drop-in replacement” for standard floats, with no changes needed to an application’s source code.

Claims like that are best taken with a grain of salt:

> It also does away with rounding errors, overflow and underflow exceptions, subnormal (denormalized) numbers, and the plethora of not-a-number (NaN) values. Additionally, posits avoids the weirdness of 0 and -0 as two distinct values.

Ok, so posits will probably work fine as a drop-in replacement when my application makes simple use of floats. But, assuming my application is doing non-trivial math, it's probably aware of the above edge cases. Thus, dropping in posits might have lots of weird side effects where I had to work around weird side effects of floats.

Re: John Gustafson’s crusade to replace floating point with something better

#110

Post by phkahler »

DougBTX wrote: post #80
ambrop7 wrote: post #52

> It also does away with rounding errors What? Surely it does not have infinite precision with a finite number of bits, and also doesn't seem to be a rational number representation.

Hm, perhaps the answer is a subtle definition of "rounding error". A rounding error is when you have a number, lets say 0.75, and due to rounding it is recorded as 1.00. The "rounding error" is 0.25. An alternative to rounding to 1.00 would be to have a mechanism which says "the value is between 0.50 and 1.50". This way, there is no actual rounding, as it doesn't commit to a rounded value, so there is technically no…

I like posits, but they do not encode intervals. His older ideas on that are crap and that's part of the problem getting posits accepted IMO.
Post reply on HN