Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Gustafson’s crusade to replace floating point with something better

nextplatform.com

Post reply on HN
Print view View original

161–170 of 201 posts

Re: John Gustafson’s crusade to replace floating point with something better

#161

Post by piadodjanho »

nestorD wrote: post #158
piadodjanho wrote: post #151

Earlier quoted context omitted.

The innovative idea from posits are they use Golomb-Rice prefix to encode exponent numbers. The Golomb Rice prefix let you encode exponent closer to zero using less space. For example, the posit16 with nbits=16 and es=1 encodes the exponent like: 01.0 = 0 01.1 = 1 001.0 = 2 001.1 = 3 0001.0 = 4 The format has a normal exponent field with es bits that encodes exponent in binary. When it overflows, it encodes the carry…

> They likely compared the binnary64 with posit16 using the accumulator (aka quire). Which I always find deceptiv because nothing stops us from using a quire with classical floating point arithmetic. It is in fact a comparaison betwen a summation and a compensated summation : it is more precise because the algorithm is different not because of posits or floats. They tell you that the quire is even faster than a tradi…

> They tell you that the quire is even faster than a traditional sum while being more precise but reading the associated reference reveals that it only hold with specific hardware which could also be used to use the quire with floats.

Precisely. Once the posit in unpacked they are indistinguishable from a floating point. It is not fair let posit use a massive accumulator while working with a tiny ieee-754 floating point accumulator. Like I said before, the precision of a number represented in binnary64 is greater than posit8. This comparation ignores the biggest advantages of posit: efficient data format.

> (and, arguably, I would love to know that every programmer is aware of a solid implementation of compensated/exact summation/dot-product and uses it when appropriate)

I like the idea of making the accumulator type (quire) accessible to the programmer. I think this brings awareness of the underlying hardware implementation to the average programmer.

Re: John Gustafson’s crusade to replace floating point with something better

#162

Post by kazinator »

enriquto wrote: post #62

Posits and other floating variants are seriously cool, and Gustafson work is amazing. Sadly, the guy has a very annoying writing style that makes him sound like a crackpot. The advantages of posits would shine much more if they were not mixed with ridiculous language (posits are floating point numbers, thus they cannot replace them) and outlandish claims (IEEE floating point is deterministic, to the apparent contradi…

> posits are floating point numbers, thus they cannot replace them

Everyone who reads that who is cognitively typical will understand "floating-point" (which, by the way, needs a dash, if we are going to nitpick) as a nickname for "IEEE 754 floating-point" or "traditional floating-point", rather than an insinuation that posits are, in contrast, fixed-point or something other than floating-point.

Re: John Gustafson’s crusade to replace floating point with something better

#163

Post by kps »

ScottFree wrote: post #152
antonyme wrote: post #111

Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions! So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitf…

> Very few developers truly understand floating point representation. Where would one go to better understand how floating points are represented?

“What every computer scientist should know about floating point” by David Goldberg. Readily available free online.

Re: John Gustafson’s crusade to replace floating point with something better

#164

Post by xenadu02 »

enriquto wrote: post #156
piadodjanho wrote: post #147

Earlier quoted context omitted.

I am implementing a high performance posit arithmetic unit for uni thesis. The idea of using Golomb-Rice prefix tree to encode the exponent is genius. Having said that. I totally agree his papers make a lot of bogus claim. For instance, the claim that "posits can be a direct replacement of floating point" is not true. There are a few tricks, such the Fast Invertion Square Root that abuses the floating point format to…

I'm eager to read your thesis, then! Please, make sure to include a (short) historical account from a neutral point of view. Most notably, Kahan's rejection of unums/posits mixes mostly reasonable criticism with a few points that are not relevant; and then Gustafson holds on to these irrelevant points and replies with his usual word salad. It is quite a sad state of affairs for what could be an enlightening scientifi…

Papers have already been written. This one implemented posits on an FPGA: https://hal.inria.fr/hal-02131982

The summary:

These architectures are evaluated on recent FPGA hardware and compared to their IEEE-754 counterpart. The standard 32 bits posit adder is found to be twice as large as the corresponding floating-point adder. Posit multiplication requires about 7 times more LUTs and a few more DSPs for a latency which is 2x worst than the IEEE-754 32 bit multiplier.

Re: John Gustafson’s crusade to replace floating point with something better

#165

Post by vanderZwan »

piadodjanho wrote: post #151
vanderZwan wrote: post #145

The curious skeptics here might want to check out the videos from CoNGA'19[0]. There are talks about posits being used and tried out in the wild with impressive results. For example, according to Millan Klöwer, 16 bit posits could be accurate enough to replace 64 bit floats in certain climate modelling problems[1]. EDIT: just realized that the example was mentioned in the article, with a link to the slides. Still, th…

The innovative idea from posits are they use Golomb-Rice prefix to encode exponent numbers. The Golomb Rice prefix let you encode exponent closer to zero using less space. For example, the posit16 with nbits=16 and es=1 encodes the exponent like: 01.0 = 0 01.1 = 1 001.0 = 2 001.1 = 3 0001.0 = 4 The format has a normal exponent field with es bits that encodes exponent in binary. When it overflows, it encodes the carry…

> They likely compared the binnary64 with posit16 using the accumulator (aka quire).

As far as I can tell, Klöwer didn't. He mentions around the 13 minute mark (slide 9) that he used the SigmoidNumbers software package for Julia. From the looks of the examples he gives, if he used the quire, it must have happened implicitly. Which IIRC is not how using the quire works.

He did rescale all his inputs to minimize rounding errors that way, and he mentions that this has benefits for posits that floats don't have (because of the tapered precision of posits).

Re: John Gustafson’s crusade to replace floating point with something better

#166

Post by xenadu02 »

rrss wrote: post #119
dang wrote: post #34

A thread from 2015: https://news.ycombinator.com/item?id=9943589 2016: https://news.ycombinator.com/item?id=11573172 2017: https://news.ycombinator.com/item?id=15617633 https://news.ycombinator.com/item?id=14669913 Many other articles (but not many comments): https://hn.algolia.com/?sort=byDate&dateRange=all&type=story... https://hn.algolia.com/?sort=byDate&dateRange=all&type=story...

The proposed format has changed several times since 2015, so not all those threads are discussing the same thing as this one. Posits are Type III Unums IIRC, the original proposal was with Type I Unums. Previous proposals were variable length formats, which is pretty much a nonstarter for a variety of reasons. This proposal (posits) is a fixed length format with variable length fields.

There have been several papers and blog posts written about the more recent posits proposals.

https://marc-b-reynolds.github.io/math/2019/02/06/Posit1.htm... https://hal.inria.fr/hal-01959581v2 https://hal.inria.fr/hal-02131982

The bottom line is they are a tradeoff; they certainly aren't purely better than IEEE floats. Plus they seem to have a non-trivial hardware cost (one of the linked papers shows the adder is twice as large as the equivalent IEEE adder and has double the latency!)

Re: John Gustafson’s crusade to replace floating point with something better

#167

Post by DubiousPusher »

titzer wrote: post #60
DubiousPusher wrote: post #46

Earlier quoted context omitted.

> Modern x86 systems use SSE registers Are your sure of this? It's my understanding that SSE registers require the use of a special API and standard floating point operations do not use them. But the last time I worked with them was writing a SIMD vector library 7 years ago.

> It's my understanding that SSE registers require the use of a special API and standard floating point operations do not use them. No, SSE registers are used by most compilers that do floating point these days. They support all the usual IEEE float math operations and a host of bit twiddling operations as well as vector operations. The vector operations do still require using compiler intrinsics in C++, although som…

Ah yes. This is what I was remembering. For vector operations you have to use special intrinsics

Re: John Gustafson’s crusade to replace floating point with something better

#168

Post by dmd »

piadodjanho wrote: post #160
dmd wrote: post #125

Earlier quoted context omitted.

I'm pretty sure that 1.0 + 2.0 == 3.0 in IEEE 754. :) Now, 0.1 ...

They are not the same thing, but they are close enough most of the time. The issue with floating point arrises when comparing very close numbers.

whooosh

Re: John Gustafson’s crusade to replace floating point with something better

#169

Post by piadodjanho »

enriquto wrote: post #156
piadodjanho wrote: post #147

Earlier quoted context omitted.

I am implementing a high performance posit arithmetic unit for uni thesis. The idea of using Golomb-Rice prefix tree to encode the exponent is genius. Having said that. I totally agree his papers make a lot of bogus claim. For instance, the claim that "posits can be a direct replacement of floating point" is not true. There are a few tricks, such the Fast Invertion Square Root that abuses the floating point format to…

I'm eager to read your thesis, then! Please, make sure to include a (short) historical account from a neutral point of view. Most notably, Kahan's rejection of unums/posits mixes mostly reasonable criticism with a few points that are not relevant; and then Gustafson holds on to these irrelevant points and replies with his usual word salad. It is quite a sad state of affairs for what could be an enlightening scientifi…

I will get in touch once I finish the section on Posit. I would appreciate some feedback! I'm writing in my language, but hopefully the google translator will work well.

Gustation is being too ambitious trying to reinvent the whole floating point framework. I think he should focus in what really matters: the compact encoding. A lot of application could save memory bandwidth using some kind of posit inspired format. A pack/unpack unit that converted the posit encoding to floating point inside the core would could be game changing.

He also proposed a good idea on code ergonomic: Make the accumulator a type (quire in the posit). Making the programmer aware of the accumulator can lead to better to code.

By the way, reading the Kahan work in floating point I could understand why Gustafson want's to avoid the IEEE. The Kahan implementation favored Intel, not necessarily the community.

Anyway. My point of views are from the hardware side of the things. The format seems to have some arithmetic properties that might be useful for mathematicians.

Re: John Gustafson’s crusade to replace floating point with something better

#170

Post by piadodjanho »

xenadu02 wrote: post #164
enriquto wrote: post #156

Earlier quoted context omitted.

I'm eager to read your thesis, then! Please, make sure to include a (short) historical account from a neutral point of view. Most notably, Kahan's rejection of unums/posits mixes mostly reasonable criticism with a few points that are not relevant; and then Gustafson holds on to these irrelevant points and replies with his usual word salad. It is quite a sad state of affairs for what could be an enlightening scientifi…

Papers have already been written. This one implemented posits on an FPGA: https://hal.inria.fr/hal-02131982 The summary: These architectures are evaluated on recent FPGA hardware and compared to their IEEE-754 counterpart. The standard 32 bits posit adder is found to be twice as large as the corresponding floating-point adder. Posit multiplication requires about 7 times more LUTs and a few more DSPs for a latency whi…

There are a few FPGA implementations. The posit hub website provides a table with all of them. Each has different trade-offs. The implementation I working on is focused on hardware reuse and high frequency operation.
Post reply on HN