Posits seem great, but LLNL seems to favor ZFP[1], not posits.[2] Maybe new chips should then implement both posits and ZFP.
[1] https://github.com/LLNL/zfp
[2] https://helper.ipam.ucla.edu/publications/bdcws2/bdcws2_1504...
John Gustafson’s crusade to replace floating point with something better
11–20 of 201 posts
Re: John Gustafson’s crusade to replace floating point with something better
#12Here is the official site[1] of the project. There is a request[2] to add it in the Scryer Prolog[3] (ISO Prolog implementation in Rust). And implementations in Rust[4] itself, and Julia[5] language. [1] https://posithub.org/index [2] https://github.com/mthom/scryer-prolog/issues/6 [3] https://github.com/mthom/scryer-prolog [4] https://gitlab.com/burrbull/softposit-rs [5] https://juliacomputing.com/blog/2016/03/29/un…
That's great, but if the hardware doesn't support it, then wouldn't the implementation would be slow?
Re: John Gustafson’s crusade to replace floating point with something better
#13Obviously for any 32 bit value there’s only 2^32 possible values. So logically, a better number format is all about the distribution of values and reducing redundancy. It sounds like the basic idea here is that rather than a fixed amount of significant figures, you get more precision in the middle. Meanwhile, you can also get larger exponents. Is that right?
[deleted]
Re: John Gustafson’s crusade to replace floating point with something better
#14Here is the official site[1] of the project. There is a request[2] to add it in the Scryer Prolog[3] (ISO Prolog implementation in Rust). And implementations in Rust[4] itself, and Julia[5] language. [1] https://posithub.org/index [2] https://github.com/mthom/scryer-prolog/issues/6 [3] https://github.com/mthom/scryer-prolog [4] https://gitlab.com/burrbull/softposit-rs [5] https://juliacomputing.com/blog/2016/03/29/un…
That's great, but if the hardware doesn't support it, then wouldn't the implementation would be slow?
Besides claimed efficiency improvements, posits are claimed to have advantages for implementing numerical algorithms correctly compared to IEEE floats. A big part of Gustafson's book The End of Error is devoted to that, claiming to show that a range of numerical algorithms are implemented better with posits vs IEEE floats (better meaning some mix of clearer, easier to produce stability, better error bounds, etc.). It's something of a competitor to interval arithmetic in that respect [1].
A 'softposit' implementation can at least let people experiment with writing posit algorithms to investigate the claimed algorithm-design benefits, even if it's not going to beat hardware floats in speed.
[1] The interval arithmetic people don't seem very happy with his comparisons though: http://frederic.goualard.net/publications/MR3329180.pdf
Re: John Gustafson’s crusade to replace floating point with something better
#15Posits seem great, but LLNL seems to favor ZFP[1], not posits.[2] Maybe new chips should then implement both posits and ZFP. [1] https://github.com/LLNL/zfp [2] https://helper.ipam.ucla.edu/publications/bdcws2/bdcws2_1504...
Unrelated as far as I can tell. Posits are an alternative floating point format, not an array compression algorithm. The “compression” benefits from being able to use posit floats instead of ieee754 doubles, because of the better precision.
Re: John Gustafson’s crusade to replace floating point with something better
#16"(even the same computation on the same system can produce different results for floats)"
wait...what? Is this real?
Re: John Gustafson’s crusade to replace floating point with something better
#17"(even the same computation on the same system can produce different results for floats)" wait...what? Is this real?
Not really. I suspect they are referring to the original x87 FP functions with their odd 80 bit FP stack and operations, where the exact result depends on when intermediate results are written to RAM and when they are kept in the FP stack, which might depend on intransparent things like compiler optimisations.
Re: John Gustafson’s crusade to replace floating point with something better
#18"(even the same computation on the same system can produce different results for floats)" wait...what? Is this real?
Not really. I suspect they are referring to the original x87 FP functions with their odd 80 bit FP stack and operations, where the exact result depends on when intermediate results are written to RAM and when they are kept in the FP stack, which might depend on intransparent things like compiler optimisations.
They could refer to parallel systems, most likely GPUs, where it is common to have slightly different results even on the same operation, on the same system.
This is due to the fact that several pipelines will do calculations in parallel and depending on which pipeline ends first (which can depend on many factors like current temperature) the sums can happen in a different order, leading to different rounding results.
Re: John Gustafson’s crusade to replace floating point with something better
#19[deleted]
Re: John Gustafson’s crusade to replace floating point with something better
#20Earlier quoted context omitted.
Not really. I suspect they are referring to the original x87 FP functions with their odd 80 bit FP stack and operations, where the exact result depends on when intermediate results are written to RAM and when they are kept in the FP stack, which might depend on intransparent things like compiler optimisations.
They could refer to parallel systems, most likely GPUs, where it is common to have slightly different results even on the same operation, on the same system. This is due to the fact that several pipelines will do calculations in parallel and depending on which pipeline ends first (which can depend on many factors like current temperature) the sums can happen in a different order, leading to different rounding results…
I've never heard of that happening on a GPU. Can you give an example?