Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

johngustafson.net

Post reply on HN
Print view View original

41–50 of 53 posts

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#41

Post by dnautics »

stephencanon wrote: post #34

Posits are actually quite reasonable, but there's a lot of either ignorance or disingenuousness in this article, which is really too bad. I wish that John would ditch the hyperbole and solicit feedback from other experts, because posits are not a bad idea, but the presentation continues to give him the trappings of a crank. I'll unpack just the first example that jumped out at me: > Currently, half-precision (16-bit)…

I'm just going to be blunt here. John and I have decided that we need to be more marketing savvy after he's had trouble with several rounds pitching other floating point formats. Posits are just an intermediate step to try to build acceptance for valids, so there's a lot of effort put into branding.

A couple of points: 2-4x faster means 2x faster in dot product based simd and 4x faster in matrix simd, assuming that your bottleneck is memory throughput.

The sigmoid function realistically isn't a bottleneck in general, but you gotta admit it is pretty cool to have a ~zero clock cycle approximation. (I've tried to rein in John a bit on this one)

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#43

Post by dnautics »

IshKebab wrote: post #42
dnautics wrote: post #38

Earlier quoted context omitted.

Yep!

So... not exactly what I suggested then?

oh whoops, reading comprehension fail! Yeah that is an interesting idea that crossed my mind! I'm a little bit hesitant to do that because 1) infinities are not really that important to have, and 2) if we have them we want them to be exact values in the Valid representation, which any value ending in a binary 1 would not be.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#44

Post by stephencanon »

dnautics wrote: post #41
stephencanon wrote: post #34

Posits are actually quite reasonable, but there's a lot of either ignorance or disingenuousness in this article, which is really too bad. I wish that John would ditch the hyperbole and solicit feedback from other experts, because posits are not a bad idea, but the presentation continues to give him the trappings of a crank. I'll unpack just the first example that jumped out at me: > Currently, half-precision (16-bit)…

I'm just going to be blunt here. John and I have decided that we need to be more marketing savvy after he's had trouble with several rounds pitching other floating point formats. Posits are just an intermediate step to try to build acceptance for valids, so there's a lot of effort put into branding. A couple of points: 2-4x faster means 2x faster in dot product based simd and 4x faster in matrix simd, assuming that y…

If you're bound by memory throughput, you can't go beyond a 2x speedup (there's 1/2 as much data to move in an 8b format, whether it's in vectors or matrices doesn't matter). I still don't see any reasonable expectation for 4x.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#45

Post by louithethrid »

Dylan16807 wrote: post #33
louithethrid wrote: post #32

Earlier quoted context omitted.

Cost of a NaN Operation on chip- same as every other operation, as NaN just is handled and returned like another floating point value - always resulting in a new NaN value- thus a error invalidates all resulting wrong results. Cost of a Interrupt: 100 ns to 1 microseconds (Quora) Sorry, that solution is simply not interesting for most implementations where floats are used. There are sensors which in realtime hammer o…

> Cost of a Interrupt: 100 ns to 1 microseconds That sounds like the time it takes to do a context switch to the OS. A math error interrupt doesn't need to do a context switch. The cost doesn't need to be any higher than a branch misprediction at 10-20 cycles.

I stand corrected, sorry, it was late at night and yes of course its a floating point operation that goes sour, which in assembly would be handled and then the result passed to the programm handling. Still expensive when encountered in mass though, on a micro controler.

Thanks for putting it right, before the missinformation could spread.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#47

Post by geezerjay »

dbcurtis wrote: post #22
Dylan16807 wrote: post #17

> There are no “NaN” (not-a-number) bit representations with posits; instead, the calculation is interrupted, and the interrupt handler can be set to report the error and its cause, or invoke a workaround and continue computing, but posits do not make the logical error of assigning a number to something that is, by definition, not a number. This simplifies the hardware considerably. What a strange claim. Outputting a…

Interrupts on NaNs are extremely expensive to implement on vector and GPU processors. We learned that in the Cray-1 days. Handling the exception in a vector iperation kicks off an ultra-expensive context switch. A NaN does not break the flow.

> Interrupts on NaNs are extremely expensive to implement on vector and GPU processors.

That's ok, because NaNs are exceptional responses to numerical errors thrown during number crunching, indicating that something failed unexpectedly and the output is crap.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#48

Post by geezerjay »

VikingCoder wrote: post #19
Dylan16807 wrote: post #17

> There are no “NaN” (not-a-number) bit representations with posits; instead, the calculation is interrupted, and the interrupt handler can be set to report the error and its cause, or invoke a workaround and continue computing, but posits do not make the logical error of assigning a number to something that is, by definition, not a number. This simplifies the hardware considerably. What a strange claim. Outputting a…

> This is not to say that NaNs are a good or bad method, but they're definitely not expensive to implement. I saw him explain that in float, there a too many bit representations that amount to "NaN". According to a Stack Overflow I found: "IEEE 754 standard defines 16,777,214 32-bit floating point values as NaNs, or 0.4% of all possible values." That's "expensive" in terms of losing bits of expressiveness that could…

> That's "expensive" in terms of losing bits of expressiveness that could have been used to represent actual numbers.

The double precision floating point format already represents between 15 and 17 significant digits, and those who, for some reason, believe they need more expressiveness can already use the illusive and rarely needed 80-bit extended precision floating point format.

0.4% of anything is nothing.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#49

Post by deepnotderp »

trsohmers wrote: post #8
jamesaross wrote: post #5

Earlier quoted context omitted.

John Gustafson is on the board of Rex Computing, a small semi startup. They claim to have taped out last year, but I don't know if the chip has been validated or if it had posits in silicon, but I know that is one of their longer term goals. I find Gustafson's UNUMS 2.0 more compelling than the first version.

Founder of REX Computing here, we taped out in July of 2016 and got our silicon brought up and working back in February. I gave a talk at Stanford showing our hardware and a tiny bit of software: https://www.youtube.com/watch?v=ki6jVXZM2XU Type 2 unums are pretty much entirely deprecated by type 3 unums (now given the name 'posits' as referred to in the OP's linked paper)... they are basically superior in every way,…

Does the Rex Neo have posits?

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#50

Post by [deleted] »

Dylan16807 wrote: post #17

> There are no “NaN” (not-a-number) bit representations with posits; instead, the calculation is interrupted, and the interrupt handler can be set to report the error and its cause, or invoke a workaround and continue computing, but posits do not make the logical error of assigning a number to something that is, by definition, not a number. This simplifies the hardware considerably. What a strange claim. Outputting a…

[deleted]
Post reply on HN