Live data from Hacker News

Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

johngustafson.net

21–30 of 53 posts

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#21

> There are no “NaN” (not-a-number) bit representations with posits; instead, the calculation is interrupted, and the interrupt handler can be set to report the error and its cause, or invoke a workaround and continue computing, but posits do not make the logical error of assigning a number to something that is, by definition, not a number. This simplifies the hardware considerably. What a strange claim. Outputting a…

> Outputting a NaN is easier than raising an interrupt

That's probably why the recent RISC-V architecture doesn't even give the option of raising an interrupt on floating point errors ("As allowed by the standard, we do not support traps on floating-point exceptions in the base ISA, [...]"). Going even farther, it also doesn't raise an interrupt on integer divide by zero (https://content.riscv.org/wp-content/uploads/2017/05/riscv-s...). That probably simplifies out-of-order implementations, since the only things that can trap then are the instruction decoder (instruction fetch errors and invalid instructions), the load/store instructions, and special instructions (system calls, breakpoints, and like).

(It also doesn't propagate NaN payloads, always returning the same canonical NaN whenever an operation produces a NaN).

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#22

> There are no “NaN” (not-a-number) bit representations with posits; instead, the calculation is interrupted, and the interrupt handler can be set to report the error and its cause, or invoke a workaround and continue computing, but posits do not make the logical error of assigning a number to something that is, by definition, not a number. This simplifies the hardware considerably. What a strange claim. Outputting a…

Interrupts on NaNs are extremely expensive to implement on vector and GPU processors. We learned that in the Cray-1 days. Handling the exception in a vector iperation kicks off an ultra-expensive context switch. A NaN does not break the flow.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#23

> There are no “NaN” (not-a-number) bit representations with posits; instead, the calculation is interrupted, and the interrupt handler can be set to report the error and its cause, or invoke a workaround and continue computing, but posits do not make the logical error of assigning a number to something that is, by definition, not a number. This simplifies the hardware considerably. What a strange claim. Outputting a…

Why don't they just change the current +/-inf to NaN, and then have the adjacent values be -inf or +inf. Or is that exception hard to implement?

Great observation! That's a compuational mode we are discussing. It's actually trivial to implement.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#24

> There are no “NaN” (not-a-number) bit representations with posits; instead, the calculation is interrupted, and the interrupt handler can be set to report the error and its cause, or invoke a workaround and continue computing, but posits do not make the logical error of assigning a number to something that is, by definition, not a number. This simplifies the hardware considerably. What a strange claim. Outputting a…

Interrupts on NaNs are extremely expensive to implement on vector and GPU processors. We learned that in the Cray-1 days. Handling the exception in a vector iperation kicks off an ultra-expensive context switch. A NaN does not break the flow.

It seems like an antipattern. If your execution time is bound by an "ultra expensive context switch" occurring when you hit a Nan, then something is wrong with your algorithm.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#25
post #13

The LINPACK benchmark feels a bit like bragging. Of course you can demand an exact dot-product operation for any implementation of your standard, and then use it to compute an exact vector-matrix product, but floats could do the same if it were part of the standard. The fact that it requires a 1024-bit accumulator makes me doubt that it would be used for a massively parallel implementation e.g. in GPUs. The overall i…

The accumulator is pipelineable. For most graphics and ml applications you probably don't need the exact dot product. You probably do want it for scientific applications... There is a trade-off in performance, as always, an appeal to calculation speed though falls to the retort of, sure, you can calculate wrong answers with higher throughput if you wished.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#26
post #13

The LINPACK benchmark feels a bit like bragging. Of course you can demand an exact dot-product operation for any implementation of your standard, and then use it to compute an exact vector-matrix product, but floats could do the same if it were part of the standard. The fact that it requires a 1024-bit accumulator makes me doubt that it would be used for a massively parallel implementation e.g. in GPUs. The overall i…

[deleted]

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#27

> There are no “NaN” (not-a-number) bit representations with posits; instead, the calculation is interrupted, and the interrupt handler can be set to report the error and its cause, or invoke a workaround and continue computing, but posits do not make the logical error of assigning a number to something that is, by definition, not a number. This simplifies the hardware considerably. What a strange claim. Outputting a…

Why don't they just change the current +/-inf to NaN, and then have the adjacent values be -inf or +inf. Or is that exception hard to implement?

here is the spec we drafted a few months ago on exactly what you suggest.

https://github.com/interplanetary-robot/SigmoidNumbers/blob/...

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#28

Earlier quoted context omitted.

Why don't they just change the current +/-inf to NaN, and then have the adjacent values be -inf or +inf. Or is that exception hard to implement?

Great observation! That's a compuational mode we are discussing. It's actually trivial to implement.

Another option could be to borrow a trick from floating point, and have a separate status register with bits that represent the kinds of errors encountered. Before a many-step calculation, software could clear these bits, and if any of them is set after the calculation, there was an error somewhere. Of course, since you don't have a NaN, you have to specify the bit pattern which will be returned by each kind of failing operation (like RISC-V does with integer divide by zero).

That doesn't help if the result could be partially valid, but if a "square root of minus one" or similar anywhere in the calculation means the result is invalid (and would be a NaN under floating point), sticky error bits in a status register could be sufficient.

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#29
post #28

Earlier quoted context omitted.

Great observation! That's a compuational mode we are discussing. It's actually trivial to implement.

Another option could be to borrow a trick from floating point, and have a separate status register with bits that represent the kinds of errors encountered. Before a many-step calculation, software could clear these bits, and if any of them is set after the calculation, there was an error somewhere. Of course, since you don't have a NaN, you have to specify the bit pattern which will be returned by each kind of faili…

question: How do you access this status register using standard C? (because if you can't, basically no one will use it)

Re: Beating Floating Point at Its Own Game: Posit Arithmetic [pdf]

#30
post #28

Earlier quoted context omitted.

Another option could be to borrow a trick from floating point, and have a separate status register with bits that represent the kinds of errors encountered. Before a many-step calculation, software could clear these bits, and if any of them is set after the calculation, there was an error somewhere. Of course, since you don't have a NaN, you have to specify the bit pattern which will be returned by each kind of faili…

question: How do you access this status register using standard C? (because if you can't, basically no one will use it)

You use feclearexcept() and fetestexcept() from fenv.h: https://linux.die.net/man/3/fenv
Post reply on HN