Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Floating Point Visually Explained (2017)

fabiensanglard.net

Post reply on HN
Print view View original

51–60 of 101 posts

Re: Floating Point Visually Explained (2017)

#51

Post by piadodjanho »

thechao wrote: post #40
piadodjanho wrote: post #34

Earlier quoted context omitted.

Ackchyually... The IEEE-754 has a lot of redundant representation. Not where you would expect though. Caveat: Those features are invaluable for some niche applications, but not for the average joe. To start. Every IEEE-754 float has two zero representation: one for positive zero and another negative negative zero (sic). The special numbers are another source of redundancy. The the double format, have about 9,007,199,…

I'm mixed on Gustafson's posit stuff. For me, the only thing I'd change for fp would be: 1. -0 now encodes NAN. 2. +inf/-inf are all Fs with sign: 0x7FFFFFFF, 0xFFFFFFFF. 3. 0 is the only denorm. Which does four good things: 1. Gets rid of the utter insanity which is -0. 2. Gets rid of all the redundant NANs. 3. Makes INF "look like" INF. 4. Gets rid of "hard" mixed denorm/norm math. And one seriously bad thing: 1. L…

He also propose the use of an opaque register to accumulate (quire), in contrast to the transparent float register (its a mess, each compiler does what it think is best).

When working with numbers that exceed the posit representation you use the quire to accumulate. At the end of the computation you convert again to posit to store in memory, or store the quire in memory.

In C, it would look like something like:

    posit32_r a, b;
    quire_t q;
    
    q = a; // load posit into quire
    
    q = q + b; // accumulate in quire
    
    a = q; // load quire into posit
> The rest of Gustafson's stuff always sounds like crazy-people talk, to me.

I've read all his papers on posit and agree. But I do believe the idea of encoding exponent with golomb-rice is actually very good and suit most users. The normalization hardware (used in the subtraction operation) can be easily repurposed to decode the exponent and shift the exponent.

But the quire logic (fixed point arithmetic) might use more area than a larger float-point. But maybe in power usage it pays of.

Re: Floating Point Visually Explained (2017)

#52

Post by splittingTimes »

That's why when I did numerical simulation of electron Dynamics in semiconductors during my phD we never used straight SI units (m, s, kg, etc), but instead expressed all physical natural constants in nm, fs, eV, etc. That way all relevant constants had numerical values between 1 and 10 which stabilized the simulations a lot.

Re: Floating Point Visually Explained (2017)

#53

Post by 0-_-0 »

Fabien Sargland is the same guy who wrote 2 books on a deep dive into 2 game engines: Wofenstein 3D and Doom which are great reading and I really recommend them to the HN crowd (can be downloaded for free):

https://fabiensanglard.net/gebbwolf3d/

https://fabiensanglard.net/gebbdoom/

Re: Floating Point Visually Explained (2017)

#54

Post by BeetleB »

randyrand wrote: post #50
romwell wrote: post #35

Earlier quoted context omitted.

I think this is worth saying about the leading bit (why 0.001 is not a valid mantissa): In binary, we always know the first non-zero digit of any number - so there's no need to write it down. We know it's 1 because, well, binary. So we don't waste space, and only write down all the other digits, and then use the exponent to put the 'point' into the right place. We save a bit of space doing this.

Thats not the right way of thinking about it. Sure, the first non-zero digit of any binary number is 1, but who is to say that the number has a bit that isn't 0? Couldn't the number be zero? The mantissa can only be a value from [1,2). i.e. in scientific notation: exponent * mantissa, the mantissa cannot be less than 1, or >= 2 in floating point. So given that the mantissa can go from 000000 to 111111, what should th…

> Thats not the right way of thinking about it. Sure, the first non-zero digit of any binary number is 1, but who is to say that the number has a bit that isn't 0? Couldn't the number be zero?

When I was taught scientific notation in middle school, high school, and at college, it was always explicitly stated that:

1. You can have only one digit before the decimal point.

2. That digit cannot be 0 (unless your number is 0, of course). So 0.3 * 10^5 is not scientific notation.

This is no different. The "twist" is that there is only one possible non-zero bit, whereas in decimal it could be [1-9].

I think this explanation is neat. However, the "usual" formulation is just scientific notation, with the optimization that one bit is redundant.

Personally, I prefer the alternative notation: M * 2^(exponent-precision+1), with M being a p-bit integer. It's easier to work with when you know that M is always an integer, and you don't need to deal with fractions in base 2. In fact, FP made a lot more sense when I took this formulation in decimal, and worked with that.

Re: Floating Point Visually Explained (2017)

#55

Post by swimfar »

splittingTimes wrote: post #52

That's why when I did numerical simulation of electron Dynamics in semiconductors during my phD we never used straight SI units (m, s, kg, etc), but instead expressed all physical natural constants in nm, fs, eV, etc. That way all relevant constants had numerical values between 1 and 10 which stabilized the simulations a lot.

I had never heard of this idea before. Do you have any references to this technique? Also, depending on what equations you're using, aren't you constrained to using a consistent system of units?

Re: Floating Point Visually Explained (2017)

#56

Post by BeetleB »

bogomipz wrote: post #4
commandlinefan wrote: post #2

Wow, that's a much easier way to convert from decimal to floating point than I had ever seen. He doesn't mention why biased notation is used (i.e. why the exponent is stored as 127+E): it's used so that if you sort positive numbers as if they were integers, they'll still end up in the right order.

>"He doesn't mention why biased notation is used (i.e. why the exponent is stored as 127+E): it's used so that if you sort positive numbers as if they were integers, they'll still end up in the right order." Could you elaborate on this? Maybe an example? This sounds interesting but I'm failing to grasp it.

The exponent is 8 bits. That translates to 0 through 255. This means your "window" begins from [1,2], [2,4], etc.

That sucks. It means I cannot represent any number between 0 and 1.

Ideally, we want the exponent to be somewhat symmetric, which in this case means going from -127 to 127. To translate 0 to 255 to that interval, you subtract 127. You are simply shifting your exponent so that the allowed values for your exponent is symmetric about 0.

Now your window starts with [2^-127, 2^-126] and so on.

(I may have an off by one error here).

Re: Floating Point Visually Explained (2017)

#57

Post by sigjuice »

piadodjanho wrote: post #48
anaphor wrote: post #44

Earlier quoted context omitted.

It's an implicit 1 or 0 (because it's binary floating point). The implicit 0 is for subnormal numbers. That's the part people usually don't explain when they're first introducing it. It's literally {1,0}.xxxxxx where x is also a 1 or 0. I.e. a binary floating point number in scientific notation. I've seen a lot of explanations that kind of gloss over that part of it (not saying you don't understand it, just that even…

In practice, subnormal are very rarely used. Most compiler disable subnormals when compiling with anything other than -O0. It takes over a hundred cycle to complete an operation. Demo: #include int main () { volatile float v; float acc = 0; float den = 1.40129846432e-45; for (size_t i; i With -01: $ gcc float.c -o float -O1 && time ./float ./float 8.93s user 0.00s system 99% cpu 8.933 total With -O0: $ gcc float.c -o…

Shouldn't i be initialized?

Re: Floating Point Visually Explained (2017)

#58

Post by BeetleB »

piadodjanho wrote: post #34
jimmaswell wrote: post #33

I remember wondering at first how you know this won't have multiple representations and cover every number - it's because the mantissa can't reach 2, and if it was 2 it would be the same as adding to the exponent, so you get the full range between any two exponents.

Ackchyually... The IEEE-754 has a lot of redundant representation. Not where you would expect though. Caveat: Those features are invaluable for some niche applications, but not for the average joe. To start. Every IEEE-754 float has two zero representation: one for positive zero and another negative negative zero (sic). The special numbers are another source of redundancy. The the double format, have about 9,007,199,…

> If that wasn't bad enough, since the magnitude of the numbers follows a normal distribution (someone whose name I forgot's law), the most significant bits of the exponent field are very rarely used. The IEEE-754 encoding is suboptimal.

But isn't that accounted for by the fact the floating point number distribution is non-uniform? Half of all floating point numbers are between -1 and 1.

Re: Floating Point Visually Explained (2017)

#60

Post by Someone »

saagarjha wrote: post #20
Someone wrote: post #16

Earlier quoted context omitted.

https://wiki.c2.com/?IeeeSevenFiftyFour : “IEEE 754 […] Has the interesting and useful property that two's complement comparisons of the underlying bit pattern of any two IEEE 754 numbers will have the same result as comparing the numbers that are represented” That means that, if you interpret the bits of a float/double as an int32/int64, increase that integer by one, and then interpret the bits of the result as a fl…

This has more to do with the mantissa than the biased exponent.

No, it has to do with bothequally.

Changing a 0 bit to a 1 in an integer increases its value, and by the rule I gave, should also increase the value of the floating point number with the same bit pattern. It doesn’t matter whether that flipped bit is in the mantissa or the exponent.

That requires the use of the biased number in the exponent. In particular, the “all zeroes” bit pattern for the exponent must be the representation for the lowest possible exponent, and the “all ones” one that for the highest. It cannot be the ‘normal’ two’s complement representation of -1.

Post reply on HN