Live data from Hacker News

How many floating-point numbers are in the interval [0,1]?

lemire.me

31–40 of 158 posts

Re: How many floating-point numbers are in the interval [0,1]?

#31
post #10
post #6

Article says 1,056,964,609 / 1,056,964,610 Quick Java program says 1,065,353,216 / 1,065,353,217 Which one is right? Or are Java floats just "different"? static { float a = 0; long count = 0; while (a

nextAfter is probably also including the denormals (an additional 2^23 values near 0).

Yep:

  1,056,964,609 - 1,065,353,216 = 8388607 = 2^23 - 1
because the denormal that is all-zero is already present.

Re: How many floating-point numbers are in the interval [0,1]?

#32
I had a similar question a while ago: how many decimal digits does it take to represent the smallest possible 80 bit float. The answer is ~16000:

  #include 

  int main(int argc, char **argv)
  {
        unsigned long long int mant, exp ;

        mant = 0;
        exp = 1;

        typedef struct _bitfield80 {
                unsigned int sign:1;
                signed int exp:15;
                unsigned long long int mant:64;
        } bitfield80;

        union {
                long double a;
                bitfield80 b;
        } union80;

        union80.b.sign = 0;
        union80.b.mant = mant;
        union80.b.exp = exp;

        printf ("%0.17000Lf\n", union80.a);
        return 0;
  }

Re: How many floating-point numbers are in the interval [0,1]?

#33
post #27

The correct answer is 1,065,353,216. This is easy to work out yourself if you remember one basic, handy property of floats: adjacent floats are adjacent in bit representation, except -0.0f and 0.0f. For example, 0x00000000 is +0.0f. 0x00000001 is the smallest non-zero positive float. 0x00000002 is the second smallest. The only exception to this is -0.0f and +0.0f, which are 0x80000000 and 0x00000000. The rule works w…

the question is [0, 1] so wouldn't that include +/-0 and 1, so 3 more?

Re: How many floating-point numbers are in the interval [0,1]?

#34

> Of all the float-pointing point numbers your computer can represent, a quarter of them lie in [0,1]. Many people think floating point numbers are magically precise. They're not. They're far more accurate at lower magnitudes and precision fades as you work with larger values.

>Many people think floating point numbers are magically precise.

Even worse, it kicks in earlier than one might naively think. People are often surprised to hear a value as simple as 0.1 has no matching floating point representation.

(For those interested, http://www.exploringbinary.com/why-0-point-1-does-not-exist-... has a nicely illustrated explanation.)

Re: How many floating-point numbers are in the interval [0,1]?

#35
post #4

for many applications, consider randomizing in [1,2) and then subtracting one. Although many of the fp values in [0,1) will be inaccessible, you are guaranteed uniformity, and also the lattice that you're drawing from will be fixed.

I'm interested in this as I maintain a little floating point RNG project, but I suspect you are somehow mislead on this.

Simple RNGs can discard significant bits during math operations to help get random-like data, eg. the "middle-square method" or the lower bits of Math.sin(seed++) can be used as practically random. The little rng I developed and tested discards 0 to 3 MSbs erratically in its state as part of its generation scheme. But getting a library random function to return [1,2] and then subtracting 1 can barely affect overall uniformity of distribution. At best its just discarding 1 bit of potential entropy for more regular and i suspect louder noise floor.

Re: How many floating-point numbers are in the interval [0,1]?

#36
post #32

I had a similar question a while ago: how many decimal digits does it take to represent the smallest possible 80 bit float. The answer is ~16000: #include int main(int argc, char **argv) { unsigned long long int mant, exp ; mant = 0; exp = 1; typedef struct _bitfield80 { unsigned int sign:1; signed int exp:15; unsigned long long int mant:64; } bitfield80; union { long double a; bitfield80 b; } union80; union80.b.sign…

Depends what you mean by “represent”.

The smallest positive “denormal” 80-bit float is (I think) 2^(2 − 2^(14)). I just represented it using 5 decimal digits and two minus signs. :-)

Re: How many floating-point numbers are in the interval [0,1]?

#37

(I invoke my pedant-pass.) As formulated in the title, "How many floating-point numbers are in the interval [0,1]?" you could argue that this is the cardinality of the Real Numbers. What the article says it is really talking about are single-precision IEEE 754 floating-point numbers. However, I could define any number of my own floating-point representations at various sizes. The cardinality of all possible floating…

Assuming "floating point" refers to things fitting the IEEE754 spec at some precision, I'm pretty sure there are still only countably many of them.

After all, for any specific size of mantissa and exponent, there will be a finite number of floats of that size, and there are a countable number of options for mantissa and exponent (corresponds to N²), thus the number of floating point values is countable.

Alternatively, each of them corresponds to an arbitrary length bitstring, in addition to a pair of numbers defining the mantissa and exponent, which would put them in bijection with N³ and thus also be countable.

EDIT: to add to that, I believe that one cannot in any meaningful way encode uncomputable values (in the sense that even if one introduces distinguished bitstrings intended to "encode" a specific¹ uncomputable value one can't do anything other than treat is as a distinguished value, and especially one can't perform arithmetic on it or print its digits or similar), so you'll still be limited to a countable set of floating point numbers even with other tricks.

¹ If that term even has meaning when dealing with uncomputable numbers...

Re: How many floating-point numbers are in the interval [0,1]?

#38

Honestly, I think IEEE 754 floating point numbers are a pretty bad way of dealing with real numbers and just cause tons of headaches that every math library has to deal with. Even high level programmers aren't shielded from the NaN nonsense. It would be great if we had an underlying implementation we could ignore and that allowed us to think at a more mathematical level. I actually found something recently on this to…

Wikipedia provides a pretty good summary:

William Kahan claims the following issues with unums:

1. Unum computation does not always deliver correct results.

2. The description of unums sidesteps using unum for solving calculus problems.

3. Unums can be expensive in terms of time and power consumption.

4. Each computation in unum space is likely to change the bit length of the structure. This requires either unpacking them into a fixed-size space, or data allocation, deallocation, and garbage collection during unum operations, similar to the issues for dealing with variable-length records in mass storage.

5. Unums provide only two kinds of numerical exception, quiet and signaling NaN (Not-a-Number).

6. Unum computation may deliver overly loose bounds from the selection of an algebraically correct but numerically unstable algorithm.

7. The costs and benefits of unum over short precision floating point for problems requiring low precision are not obvious.

8. Solving differential equations and evaluating integrals with unums guarantee correct answers but may not be as fast as methods that usually work.

Point #4 makes implementation tricky and slow. Point #6 (from what I understand) means unum operations tend to guarantee correctness by being overly vague - by denoting results that FP can produce using ranges that are just too big to be useful.

Re: How many floating-point numbers are in the interval [0,1]?

#39

> Of all the float-pointing point numbers your computer can represent, a quarter of them lie in [0,1]. Many people think floating point numbers are magically precise. They're not. They're far more accurate at lower magnitudes and precision fades as you work with larger values.

Correct me if I'm wrong, but my understanding is that a consequence of this is that it's better to store, for example, angles as radians from -pi to pi rather than as degrees between 0.0 and 360.0, in order to take advantage of all that precision between 0 and 1.

Re: How many floating-point numbers are in the interval [0,1]?

#40
post #27

The correct answer is 1,065,353,216. This is easy to work out yourself if you remember one basic, handy property of floats: adjacent floats are adjacent in bit representation, except -0.0f and 0.0f. For example, 0x00000000 is +0.0f. 0x00000001 is the smallest non-zero positive float. 0x00000002 is the second smallest. The only exception to this is -0.0f and +0.0f, which are 0x80000000 and 0x00000000. The rule works w…

the question is [0, 1] so wouldn't that include +/-0 and 1, so 3 more?

Probably shouldn’t include –0, which is used to represent underflow of negative numbers.
Post reply on HN