Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Gustafson’s crusade to replace floating point with something better

nextplatform.com

Post reply on HN
Print view View original

171–180 of 201 posts

Re: John Gustafson’s crusade to replace floating point with something better

#171

Post by klingonopera »

mkettn wrote: post #112
QuickToBan wrote: post #12

Earlier quoted context omitted.

That's great, but if the hardware doesn't support it, then wouldn't the implementation would be slow?

As far as I read https://posithub.org/docs/Posits4.pdf there are FPGA implementations. Also in systems where floating point arithmetic isn't supported in hardware (i.e. Arduino Uno) you rely on soft fp.

Benchmark performance analysis of accuracy and speed on AVR platforms would indeed be very interesting and an actual, possible, immediate implementation scenario...

Re: John Gustafson’s crusade to replace floating point with something better

#172

Post by theamk »

_kst_ wrote: post #35
theamk wrote: post #32

This may be good for numerical simulation, but I doubt this will ever replace common ieee754 float in general purpose programs. Two problems I can see right away: - Sometimes, one needs precision for numbers not close to 1. A UNIX timestamp is a good example -- for the current date, it provides ~1uS resolution in 64-bit ieee754. I could not calculate what the resolution would be for posits, but I suspect much worse.…

UNIX timestamps are normally stored as 32-bit or 64-bit signed integers, not as floating-point. If you want better than 1-second precision, then the type "struct timespec" (specified by POSIX) gives you nanosecond precision. Fixed-point types can also be used in languages that support them.

Yeah, timespec (or time_interval) is a proper way to go, but it is quite a pain to work with -- you need a library even if you just want to subtract the numbers.

On the other hand, floating-point time is pretty common in scripting languages -- for example Python has time.time(); ruby has Time.now.to_f. It is not perfect, but great for smaller scripts: fool proof (except for the precision loss), roundtrips via any serialization format, and easy to understand. And no timezone problems at all!

Re: John Gustafson’s crusade to replace floating point with something better

#173

Post by nwallin »

coldtea wrote: post #64
enriquto wrote: post #62

Posits and other floating variants are seriously cool, and Gustafson work is amazing. Sadly, the guy has a very annoying writing style that makes him sound like a crackpot. The advantages of posits would shine much more if they were not mixed with ridiculous language (posits are floating point numbers, thus they cannot replace them) and outlandish claims (IEEE floating point is deterministic, to the apparent contradi…

> IEEE floating point is deterministic, to the apparent contradiction of many sentences written by Gustafson What about this then, which somebody comments below: "On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time,…

If you have an 80 bit object, convert it to a 64 bit object, then convert it back to an 80 bit object, and you expect to drop zero bits, of course you'll run into problems. Posits can't fix these problems.

Desktop graphics cards intentionally ignore IEEE 754 rules if it allows them to squeeze out a little bit of performance. (or die space, or power consumption depending on the application) If the bad guy is drawn half a pixel out of position or if his shirt is slightly the wrong shade of red, nobody cares. But if your graphics card draws at 72 frames per second and your competitor's cards draws at 73, every publication is going to say their card is better than yours. Posits won't fix this either.

These problems you point out aren't technical problems, they're institutional ones. IMHO it's generally a bad idea to try to fix institutional problems with technical solutions, but I went try to stop you from attempting it.

Re: John Gustafson’s crusade to replace floating point with something better

#174

Post by piadodjanho »

vanderZwan wrote: post #165
piadodjanho wrote: post #151

Earlier quoted context omitted.

The innovative idea from posits are they use Golomb-Rice prefix to encode exponent numbers. The Golomb Rice prefix let you encode exponent closer to zero using less space. For example, the posit16 with nbits=16 and es=1 encodes the exponent like: 01.0 = 0 01.1 = 1 001.0 = 2 001.1 = 3 0001.0 = 4 The format has a normal exponent field with es bits that encodes exponent in binary. When it overflows, it encodes the carry…

> They likely compared the binnary64 with posit16 using the accumulator (aka quire). As far as I can tell, Klöwer didn't. He mentions around the 13 minute mark (slide 9) that he used the SigmoidNumbers software package for Julia. From the looks of the examples he gives, if he used the quire, it must have happened implicitly. Which IIRC is not how using the quire works. He did rescale all his inputs to minimize roundi…

Hmm. Their results seems to good to be true without using quire. Look:

The two largest/smallest exponent posit16 can represent are:

min exp: 2^{ (+14Note those numbers only have the implicit bit as significant. They don't have any space left to encode any other information other than the sign and regime.

While the double (binary64) can represent a much larger range of exponent:

min exp: 2^{ (-1023-2047-1) } * (1+0) = 2^(-1022) max exp: 2^{ (-1023+2047-1) } * (1+0) = 2^(1023)

Also, all doubles have 52 bits of precision while the posit have at most 16-1-2-1=12 maximum bits of precision.

Their significant are encoded slightly differently though. I'm not sure if this would be enough to achieve such different result without quire.

The scaling he mention could be done very easily introducing a parameter bias on FPGA implementation. I might add that to my work.

I pinged Klöwer on twitter.

Re: John Gustafson’s crusade to replace floating point with something better

#175

Post by ScottFree »

kps wrote: post #163
ScottFree wrote: post #152

Earlier quoted context omitted.

> Very few developers truly understand floating point representation. Where would one go to better understand how floating points are represented?

“What every computer scientist should know about floating point” by David Goldberg. Readily available free online.

Thanks!

Re: John Gustafson’s crusade to replace floating point with something better

#176

Post by vanderZwan »

piadodjanho wrote: post #174
vanderZwan wrote: post #165

Earlier quoted context omitted.

> They likely compared the binnary64 with posit16 using the accumulator (aka quire). As far as I can tell, Klöwer didn't. He mentions around the 13 minute mark (slide 9) that he used the SigmoidNumbers software package for Julia. From the looks of the examples he gives, if he used the quire, it must have happened implicitly. Which IIRC is not how using the quire works. He did rescale all his inputs to minimize roundi…

Hmm. Their results seems to good to be true without using quire. Look: The two largest/smallest exponent posit16 can represent are: min exp: 2^{ (+14 Note those numbers only have the implicit bit as significant. They don't have any space left to encode any other information other than the sign and regime. While the double (binary64) can represent a much larger range of exponent: min exp: 2^{ (-1023-2047-1) } * (1+0)…

You clearly know more about this topic than me, thanks for taking the time to explain your viewpoints.

> I pinged Klöwer on twitter.

Probably the most sensible, easiest way to clear this up, haha :)

Re: John Gustafson’s crusade to replace floating point with something better

#177

Post by mooman219 »

One annoying part about floats is the abuse of the NaN space. Some runtimes like SpiderMonkey, JSC, and LuaJIT abuse the NaN space to store pointers in doubles. This practice is often called nan-boxing, but has a few variants. A more efficient use of bits like with Posits will break this.

As for the claim about a FPU taking up less space and power for posits than floats, facebook made that claim: https://code.fb.com/ai-research/floating-point-math/

Re: John Gustafson’s crusade to replace floating point with something better

#178

Post by seppel »

mcv wrote: post #90

This sounds really good. I find floating points completely unusable for any situation where accuracy is important. It's fine when being vaguely in the right ballpark is good enough, but I don't want to have to deal with 2 + 4.1 = 6.1000000001, or x/1000000 + y/1000000 != (x+y)/1000000.

The problem you describe here arises from using binary fractions, that is, a power of 2 as a denominator. You cannot represent the decimal fraction 0.1 as a binary fraction. You would have the same problem representing 1/3 with decimal fractions. It just does not work.

You can solve it by switching to decimal floating points, they are defined by IEEE as well:

https://en.wikipedia.org/wiki/Decimal_floating_point

Re: John Gustafson’s crusade to replace floating point with something better

#179

Post by sitkack »

garmaine wrote: post #15
QuickToBan wrote: post #11

Posits seem great, but LLNL seems to favor ZFP[1], not posits.[2] Maybe new chips should then implement both posits and ZFP. [1] https://github.com/LLNL/zfp [2] https://helper.ipam.ucla.edu/publications/bdcws2/bdcws2_1504...

Unrelated as far as I can tell. Posits are an alternative floating point format, not an array compression algorithm. The “compression” benefits from being able to use posit floats instead of ieee754 doubles, because of the better precision.

Posits also take less space for the same total accuracy, they use less space on disk, ram and less memory bandwidth.

Re: John Gustafson’s crusade to replace floating point with something better

#180

Post by sitkack »

yholio wrote: post #75

Do these techniques make it easier or harder to implement hardware floating point units? Storage and transmission speed have progressed exponentially while execution units have become the bottleneck. A floating point format that is 20% denser or more accurate but that requires a 2x number of gate delays to implement is major step backwards, except maybe in highly specialized applications.

> transmission speed

Memory bandwidth relative to instruction to instruction latency has gone way down. Memory is slower now relative to the amount of math one can do.

We are also swimming in extra silicon, so even if what you are saying is true (2x the number of gates, all adding to the delay), posits would still be a win.

You are arguing from an arbitrary what-if position, it doesn't look good.

Post reply on HN