John Gustafson’s crusade to replace floating point with something better • Hacker News Board

Re: John Gustafson’s crusade to replace floating point with something better

#111

Post by antonyme » Tue, Jul 09, 2019, 1:20 PM UTC

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions!

So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitfalls of IEEE floats. Intel chips have had BCD support in machine code; I'm surprised nobody has made a decent fixed point lib that is widely used already.

Re: John Gustafson’s crusade to replace floating point with something better

#112

Post by mkettn » Tue, Jul 09, 2019, 1:27 PM UTC

QuickToBan wrote: post #12

xvilka wrote: post #4

Here is the official site[1] of the project. There is a request[2] to add it in the Scryer Prolog[3] (ISO Prolog implementation in Rust). And implementations in Rust[4] itself, and Julia[5] language. [1] https://posithub.org/index [2] https://github.com/mthom/scryer-prolog/issues/6 [3] https://github.com/mthom/scryer-prolog [4] https://gitlab.com/burrbull/softposit-rs [5] https://juliacomputing.com/blog/2016/03/29/un…

That's great, but if the hardware doesn't support it, then wouldn't the implementation would be slow?

As far as I read https://posithub.org/docs/Posits4.pdf there are FPGA implementations. Also in systems where floating point arithmetic isn't supported in hardware (i.e. Arduino Uno) you rely on soft fp.

Re: John Gustafson’s crusade to replace floating point with something better

#113

Post by [deleted] » Tue, Jul 09, 2019, 1:29 PM UTC

enriquto wrote: post #62

Posits and other floating variants are seriously cool, and Gustafson work is amazing. Sadly, the guy has a very annoying writing style that makes him sound like a crackpot. The advantages of posits would shine much more if they were not mixed with ridiculous language (posits are floating point numbers, thus they cannot replace them) and outlandish claims (IEEE floating point is deterministic, to the apparent contradi…

[deleted]

Re: John Gustafson’s crusade to replace floating point with something better

#114

Post by IEEE754 » Tue, Jul 09, 2019, 1:32 PM UTC

antonyme wrote: post #111

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions! So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitf…

> Most think of it as base-10 [...] I'm surprised nobody has made a decent fixed point lib that is widely used already.

Note that fixed radix point does not solve the common issues with representing rational base 10 fractions. A base10 fixed radix solution would, but so would IEEE754's decimal64 spec, which would eliminate representation error when working exclusively in the context of base10 e.g finance, but these are not found in common hardware and do not help reduce propagation of error due to compounding with limited precision in any base.

Re: John Gustafson’s crusade to replace floating point with something better

#115

Post by roel_v » Tue, Jul 09, 2019, 1:32 PM UTC

OskarS wrote: post #89

roel_v wrote: post #65

Earlier quoted context omitted.

Lots of researchers cargo cult floating point programming. I had a Fortran program a few months ago where the author did "if var > 0.99 and var < 1.01 then" etc., where "var" was an integer. I tried to search back where and how that was ever done, but no scenario made any sense ("var" was a categorical variable and always had been). So I went back to some of the original authors and there too they looked at me like I…

In all likelyhood, the reason that's there isn't because a programmer was an idiot. Almost certainly, that variable was originally a float, then some time later another programmer came in and refactored it into an int, but missed fixing this condition, since it technically "works" for ints as well. This is a good argument for stricter typechecking than anything else. In Rust, a condition like that wouldn't compile be…

It was the same in the situation I was talking about; all numbers that would always be integers because of their nature (and in the data source they came from, they were integers, there was no way this could ever have been floating point, or that the values ever had any non-0 digits after the comma). And well, they probably weren't 'idiots' in the absolute sense, in fact I'm 100% sure that they are much smarter (and/or just plain better humans in other ways) than I am in many respects.

My point was that many researchers can only cobble together somewhat barely working software by taking snippets from their undergraduate textbooks (or googling/stack overflowing, for those under 40), and have no idea about many/most of the underlying principles. If only they would recognize that, and leave the software development to professionals, instead of treating it as an inconsequential implementation detail that is beneath them. Then again, if they did, they wouldn't have to pay me what they do to fix it up afterwards. So meh?

Re: John Gustafson’s crusade to replace floating point with something better

#116

Post by Mikhail_K » Tue, Jul 09, 2019, 1:33 PM UTC

> It also does away with rounding errors, overflow and underflow exceptions, subnormal (denormalized) numbers, and the plethora of not-a-number (NaN) values.

All of those are important and useful features. Presenting their absence as some kind of advantage shows that Gustafson has no clue.

Re: John Gustafson’s crusade to replace floating point with something better

#117

Post by pfortuny » Tue, Jul 09, 2019, 1:36 PM UTC

Mikhail_K wrote: post #116

> It also does away with rounding errors, overflow and underflow exceptions, subnormal (denormalized) numbers, and the plethora of not-a-number (NaN) values. All of those are important and useful features. Presenting their absence as some kind of advantage shows that Gustafson has no clue.

The article is too sloppy on details: these can be found in the specifiation and are not what the journalist writes.

Re: John Gustafson’s crusade to replace floating point with something better

#118

Post by DannyB2 » Tue, Jul 09, 2019, 1:37 PM UTC

antonyme wrote: post #111

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions! So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitf…

Considering that back in the 1980's we figured out that you should use some kind of 'integer' or 'bcd' based type for financial calculations; it is astonishing that by almost 2020 people are making these same mistakes over and over again.

A 64-bit integer is big enough to express the US National Debt in Argentine Pesos.

Re: John Gustafson’s crusade to replace floating point with something better

#119

Post by rrss » Tue, Jul 09, 2019, 1:41 PM UTC

dang wrote: post #34

A thread from 2015: https://news.ycombinator.com/item?id=9943589 2016: https://news.ycombinator.com/item?id=11573172 2017: https://news.ycombinator.com/item?id=15617633 https://news.ycombinator.com/item?id=14669913 Many other articles (but not many comments): https://hn.algolia.com/?sort=byDate&dateRange=all&type=story... https://hn.algolia.com/?sort=byDate&dateRange=all&type=story...

The proposed format has changed several times since 2015, so not all those threads are discussing the same thing as this one.

Posits are Type III Unums IIRC, the original proposal was with Type I Unums.

Previous proposals were variable length formats, which is pretty much a nonstarter for a variety of reasons. This proposal (posits) is a fixed length format with variable length fields.

Re: John Gustafson’s crusade to replace floating point with something better

#120

Post by nestorD » Tue, Jul 09, 2019, 1:43 PM UTC

mcv wrote: post #108

nestorD wrote: post #94

Earlier quoted context omitted.

Well, those will not be solved by posits, it is still an approximate floating point format. What it does is redistribution the precision to what Gustafson considers a better default and dropping edge cases in order to get more bits for precision. edit: One of several examples found in the "Posits: the good, the bad and the ugly" paper linked in the thread : 10.0 * 2.0 = 16.0 in posit8

Really? My impression from the article was that this was exactly one of the things posits were supposed to fix, at least for numbers with small exponents. I'm not sure how 10.0 * 2.0 = 16.0. I'm not sure what posit8 means, but it can only be correct if it switches halfway from base 8 representation to base 10, which is a bit weird, but at least the calculation is correct. (Otherwise it would be so incorrect to be unu…

I am serious (and recommend reading the paper I quoted to have a good understanding of the trade-of offered by posits).

Overall, posits are a new trade-of that will give you better precision (nothing exact, it is still an approximation) when you manage to keep all your number in a small range. Once you get out of that range precision drops significantly (whereas the precision of classical floating points drops gradually).

Posit8 are equivalent to 8 bits floating points (minifloats) making them an easy target for pathological cases but the example still illustrate the fact that, contrary to floating-point arithmetic, multiplication by a multiple of two is not exact with posits (one of several good properties we take for granted and would lose when switching to posits).