Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Gustafson’s crusade to replace floating point with something better

nextplatform.com

Post reply on HN
Print view View original

91–100 of 201 posts

Re: John Gustafson’s crusade to replace floating point with something better

#91

Post by fluffything »

enriquto wrote: post #62

Posits and other floating variants are seriously cool, and Gustafson work is amazing. Sadly, the guy has a very annoying writing style that makes him sound like a crackpot. The advantages of posits would shine much more if they were not mixed with ridiculous language (posits are floating point numbers, thus they cannot replace them) and outlandish claims (IEEE floating point is deterministic, to the apparent contradi…

> IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson

How can that be? The IEEE standard does not place many requirements on the precision of the transcendental functions, and AFAICT a transcendental doesn't even need to return the exact same answer twice. That is, it is correct for a compiler to, if it can, evaluate a transcendental at compile-time with infinite precision, and if it cannot, call a run-time function that computes it. Math libraries can be dynamically linked, and all can produce different results here, so it is not that a compiler can go out of its way to make things deterministic either.

Re: John Gustafson’s crusade to replace floating point with something better

#92

Post by fluffything »

dragontamer wrote: post #24
savant_penguin wrote: post #16

"(even the same computation on the same system can produce different results for floats)" wait...what? Is this real?

Yes and no. 1. Yes -- On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time, even though the machine did things 80-bits at a time). 2. No -- Modern x86 systems use SSE registers, which are 64-bits. There are a whole s…

> . No -- Modern x86 systems use SSE registers, which are 64-bits. There are a whole slew of configuration options, but if Windows / Linux does their job, you should have the same rounding-errors across your program

Knock, knock. Who is there? Floating-point contraction.

Re: John Gustafson’s crusade to replace floating point with something better

#94

Post by nestorD »

mcv wrote: post #90

This sounds really good. I find floating points completely unusable for any situation where accuracy is important. It's fine when being vaguely in the right ballpark is good enough, but I don't want to have to deal with 2 + 4.1 = 6.1000000001, or x/1000000 + y/1000000 != (x+y)/1000000.

Well, those will not be solved by posits, it is still an approximate floating point format. What it does is redistribution the precision to what Gustafson considers a better default and dropping edge cases in order to get more bits for precision.

edit: One of several examples found in the "Posits: the good, the bad and the ugly" paper linked in the thread : 10.0 * 2.0 = 16.0 in posit8

Re: John Gustafson’s crusade to replace floating point with something better

#95

Post by Lerc »

Aardwolf wrote: post #87

Choosing to have -infinity and various nans can be done independently from choosing to add regime bits, one could as well design this system to include them by adding more special values. I didn't find a resource specifying how many exponent bits to actually use? (The 'es' value) I don't see anything different in this system than regular floats that would ensure consistent results across machines/compilers/...? It's…

I just read the paper. Standard es bit counts are covered in section 7.2

Long story short: es=log2(nbits) -3

Re: John Gustafson’s crusade to replace floating point with something better

#96

Post by account42 »

hrydgard wrote: post #47
DubiousPusher wrote: post #46

Earlier quoted context omitted.

> Modern x86 systems use SSE registers Are your sure of this? It's my understanding that SSE registers require the use of a special API and standard floating point operations do not use them. But the last time I worked with them was writing a SIMD vector library 7 years ago.

Compilers now exclusively target the SSE registers (with scalar SSE instructions like mulss), and they did 7 years ago too.

https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html

  -mfpmath=unit
  
      Generate floating-point arithmetic for selected unit unit. The choices for unit are:
  
      ‘387’
  
          Use the standard 387 floating-point coprocessor present on the majority of chips and emulated otherwise. Code compiled with this option runs almost everywhere. The temporary results are computed in 80-bit precision instead of the precision specified by the type, resulting in slightly different results compared to most of other chips. See -ffloat-store for more detailed description.
  
          This is the default choice for non-Darwin x86-32 targets.

Re: John Gustafson’s crusade to replace floating point with something better

#97

Post by dnautics »

wwwigham wrote: post #54
dnautics wrote: post #22

Earlier quoted context omitted.

having implemented it, with unoptimized verilog (ok, i wrote a verilog generator to generate it), it requires about 30% fewer LUTs on a FPGA relative to berkeley hardfloat implementation.

Is the variable length regime handling that much easier to deal with (space-wise) than the NaN and subnormal handling needed in IEEE floats? I'd think that the regime scheme would effectively be equivalent to creating a multitude of different-width subnormal routes. Is it really the NaN handling that kills IEEE float performance?

It's basically a barrel shifter; for addition you're going to need it anyways. Multiplication is a bit nastier, but most of multiplier gates are the adder gates anyways. I made a useful insight that negative numbers are basically the same as positives, with a "minus two" invisible bit.

Here is a sample 8-bit multiplier. All code was generated using a verilog DSL I wrote in Julia for the specific purpose. All verilog is tested by transpiling to c using verilator and mounting the shared object into a Julia runtime with a Julia implementation.

https://github.com/interplanetary-robot/mullinengine/blob/ma...

Re: John Gustafson’s crusade to replace floating point with something better

#98

Post by Lerc »

yorwba wrote: post #85
coldtea wrote: post #64

Earlier quoted context omitted.

> IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson What about this then, which somebody comments below: "On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time,…

Those are not problems a new number format can fix though. The issues with x87 precision are due to compilers freely converting between different representations without this being indicated in the program. Posits also can't achieve associativity in all conditions, you need to use "quire" accumulators with higher precision. However, if the decision of using quires were left to the compiler, you'd end up with the exac…

The observation that all of the +-*/ operations in a basic block can be reduced to a single round with the quire would get you a reasonable way with the compiler alone.

Overall I'm not sold on the quire thing. It seems like a great big accumulator is a solution that is fairly independent from the floating point format.

Re: John Gustafson’s crusade to replace floating point with something better

#99

Post by dnautics »

yaantc wrote: post #82
yholio wrote: post #75

Do these techniques make it easier or harder to implement hardware floating point units? Storage and transmission speed have progressed exponentially while execution units have become the bottleneck. A floating point format that is 20% denser or more accurate but that requires a 2x number of gate delays to implement is major step backwards, except maybe in highly specialized applications.

From [1]: " The standard 32 bits posit adder is found to be twice as large as the corresponding floating-point adder. Posit multiplication requires about 7 times more LUTs and a few more DSPs for a latency which is 2x worst than the IEEE-754 32 bit multiplier." It's for an FPGA implementation. This being said, I object to your premise: transmission and memory storage are getting comparatively more costly vs computati…

I'm not sure the inria team used the correct optimization for posit addition. You do yourself a disservice by treating a posit like a float and bifurcate the cross (negative/positive) from the negative/negative and positive/positive branches; since posits are twos complements the posit adder should be smaller, not bigger.

Re: John Gustafson’s crusade to replace floating point with something better

#100

Post by zeckalpha »

dnautics wrote: post #21
fhars wrote: post #17

Earlier quoted context omitted.

Not really. I suspect they are referring to the original x87 FP functions with their odd 80 bit FP stack and operations, where the exact result depends on when intermediate results are written to RAM and when they are kept in the FP stack, which might depend on intransparent things like compiler optimisations.

it doesn't happen much in practice, but the official IEEE spec is silent about it, so you can toss it into whatever bin contains 'undefined' behaviour of C.

Are any C undefined behaviors explicitly undefined spec?
Post reply on HN