Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Gustafson’s crusade to replace floating point with something better

nextplatform.com

Post reply on HN
Print view View original

121–130 of 201 posts

Re: John Gustafson’s crusade to replace floating point with something better

#121

Post by epr »

dragontamer wrote: post #24
savant_penguin wrote: post #16

"(even the same computation on the same system can produce different results for floats)" wait...what? Is this real?

Yes and no. 1. Yes -- On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time, even though the machine did things 80-bits at a time). 2. No -- Modern x86 systems use SSE registers, which are 64-bits. There are a whole s…

Common Lisp over here like:

  * (+ (+ 1 1) (expt 2 256))
  115792089237316195423570985008687907853269984665640564039457584007913129639938

  * (+ 1 (+ 1 (expt 2 256)))
  115792089237316195423570985008687907853269984665640564039457584007913129639938

Re: John Gustafson’s crusade to replace floating point with something better

#122

Post by toolslive »

DannyB2 wrote: post #118
antonyme wrote: post #111

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions! So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitf…

Considering that back in the 1980's we figured out that you should use some kind of 'integer' or 'bcd' based type for financial calculations; it is astonishing that by almost 2020 people are making these same mistakes over and over again. A 64-bit integer is big enough to express the US National Debt in Argentine Pesos.

> it is astonishing that by almost 2020 people are making these same mistakes over and over again.

It's actually logical: the number of developers doubles roughly every 5 years. It means that half of the developers have less than 5 years of experience. If they don't teach you this in school (university), you will have to learn from someone who knows, but chances are the other developers are as clueless as you.

Re: John Gustafson’s crusade to replace floating point with something better

#123

Post by Sean1708 »

mcv wrote: post #108
nestorD wrote: post #94

Earlier quoted context omitted.

Well, those will not be solved by posits, it is still an approximate floating point format. What it does is redistribution the precision to what Gustafson considers a better default and dropping edge cases in order to get more bits for precision. edit: One of several examples found in the "Posits: the good, the bad and the ugly" paper linked in the thread : 10.0 * 2.0 = 16.0 in posit8

Really? My impression from the article was that this was exactly one of the things posits were supposed to fix, at least for numbers with small exponents. I'm not sure how 10.0 * 2.0 = 16.0. I'm not sure what posit8 means, but it can only be correct if it switches halfway from base 8 representation to base 10, which is a bit weird, but at least the calculation is correct. (Otherwise it would be so incorrect to be unu…

Here's the paper they mentioned: https://hal.inria.fr/hal-01959581v3/document

Re: John Gustafson’s crusade to replace floating point with something better

#124

Post by rrss »

antonyme wrote: post #111

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions! So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitf…

Replacing all the IEEE 754 hardware with posits won't fix this, though.

If you don't care about performance, then the actual solution has no dependency on hardware:

1. Replace the default format for numbers with a decimal point in suitably high level languages with a infinite precision format.

2. Teach people using other languages about floating point and how they may want to use integers instead.

The end. No multi-generation hardware transition required.

IMO, IEEE 754 is an exceptionally good format. It has real problems, but they aren't widely known to people unfamiliar with floats (e.g. 1.0 + 2.0 != 3.0 isn't one of them).

Re: John Gustafson’s crusade to replace floating point with something better

#125

Post by dmd »

rrss wrote: post #124
antonyme wrote: post #111

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions! So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitf…

Replacing all the IEEE 754 hardware with posits won't fix this, though. If you don't care about performance, then the actual solution has no dependency on hardware: 1. Replace the default format for numbers with a decimal point in suitably high level languages with a infinite precision format. 2. Teach people using other languages about floating point and how they may want to use integers instead. The end. No multi-g…

I'm pretty sure that 1.0 + 2.0 == 3.0 in IEEE 754. :) Now, 0.1 ...

Re: John Gustafson’s crusade to replace floating point with something better

#126

Post by FabHK »

Gustafnot wrote: post #106
xvilka wrote: post #4

Here is the official site[1] of the project. There is a request[2] to add it in the Scryer Prolog[3] (ISO Prolog implementation in Rust). And implementations in Rust[4] itself, and Julia[5] language. [1] https://posithub.org/index [2] https://github.com/mthom/scryer-prolog/issues/6 [3] https://github.com/mthom/scryer-prolog [4] https://gitlab.com/burrbull/softposit-rs [5] https://juliacomputing.com/blog/2016/03/29/un…

He will keep failing to replace IEEE floating point as long as he insists on making NEGATIVE infinity the same as POSITIVE infinity. Also, IEEE 754 floating point standard guarantees the results of addition, subtraction, multiplication, division, and square root to be the exact correctly rounded value, ie a deterministic result, contrary to what he says.

Not so sure. I mean, yes, what you say is right, but there are problems nevertheless, see eg Wikipedia:

> Reproducibility

> The IEEE 754-1985 allowed many variations in implementations (such as the encoding of some values and the detection of certain exceptions). IEEE 754-2008 has strengthened up many of these, but a few variations still remain (especially for binary formats). The reproducibility clause recommends that language standards should provide a means to write reproducible programs (i.e., programs that will produce the same result in all implementations of a language), and describes what needs to be done to achieve reproducible results.

https://en.wikipedia.org/wiki/IEEE_754#Reproducibility

Re: John Gustafson’s crusade to replace floating point with something better

#127

Post by FabHK »

tosh wrote: post #88

found this from Kahan in 2016 https://people.eecs.berkeley.edu/~wkahan/UnumSORN.pdf https://people.eecs.berkeley.edu/~wkahan/EndErErs.pdf https://people.eecs.berkeley.edu/~wkahan/SORNers.pdf

Thanks, came to say the same thing - wondered what William "Father of IEEE 754" Kahan had to say about this.

I haven't had time to digest it all, but he is certainly critical of Gustafson's proposals. Not sure though the articles linked above cover the latest and greatest Unum III. At any rate, I'd pay close attention to Kahan's critique.

Re: John Gustafson’s crusade to replace floating point with something better

#128

Post by ethbro »

dmd wrote: post #125
rrss wrote: post #124

Earlier quoted context omitted.

Replacing all the IEEE 754 hardware with posits won't fix this, though. If you don't care about performance, then the actual solution has no dependency on hardware: 1. Replace the default format for numbers with a decimal point in suitably high level languages with a infinite precision format. 2. Teach people using other languages about floating point and how they may want to use integers instead. The end. No multi-g…

I'm pretty sure that 1.0 + 2.0 == 3.0 in IEEE 754. :) Now, 0.1 ...

For reference to what they're talking about, the helpful https://0.30000000000000004.com/

Re: John Gustafson’s crusade to replace floating point with something better

#129

Post by FabHK »

jacobolus wrote: post #9
jchw wrote: post #5

Obviously for any 32 bit value there’s only 2^32 possible values. So logically, a better number format is all about the distribution of values and reducing redundancy. It sounds like the basic idea here is that rather than a fixed amount of significant figures, you get more precision in the middle. Meanwhile, you can also get larger exponents. Is that right?

The ideas are (1) gradual instead of hard overflow [IEEE floats do gradual underflow via denormals], (2) crowding representable numbers more precisely around 1 while letting very large or very small numbers get less and less precise, vs. floating point which (except for denormals) is scale free within its range, (3) making a format which can be extended by just adding bits without new definitions, so that you can get…

> floating point which (except for denormals) is scale free within its range

Presumably that's what you mean by scale free, but let's spell it out:

It's denser around 0, then has uniform density between, say, 1/8 and 1/4, half that density between 1/4 and 1/2, half that between 1/2 and 1, half that between 1 and 2, etc. etc.

If x = 1.0e22, you can add a million and it's still the same.

Re: John Gustafson’s crusade to replace floating point with something better

#130

Post by hinkley »

gwbas1c wrote: post #109

> Better yet, he claims the new format is a “drop-in replacement” for standard floats, with no changes needed to an application’s source code. Claims like that are best taken with a grain of salt: > It also does away with rounding errors, overflow and underflow exceptions, subnormal (denormalized) numbers, and the plethora of not-a-number (NaN) values. Additionally, posits avoids the weirdness of 0 and -0 as two dist…

I think we can assume that the people who are sophisticated enough to “use floats correctly” are also sophisticated enough to know the article is not aimed at them.

Your boss, on the other hand, may try to armchair architect you, which is a real (and underreported) problem,

Post reply on HN