Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How many floating-point numbers are in the interval [0,1]?

lemire.me

Post reply on HN
Print view View original

111–120 of 158 posts

Re: How many floating-point numbers are in the interval [0,1]?

#111

Post by jxy »

yason wrote: post #108

I don't like the divisions, though, and all the analysis that comes with it. A way that is intuitively more robust would be to generate 23 or 52 random bits from an integer RNG for the mantissa (given you trust you integer RNG is unbiased). For example, take a random uint64_t, clear the sign bit, rewrite the exponent with a predefined value, and let the mantissa be untouched. That will give you a random floating poin…

Hitting each representable floating point number with an equal probability does not generate an approximately uniform distribution of [0,1). For a sufficiently small positive number, eps, you have to have equal probabilities of numbers falling in [0,eps) and [0.5,0.5+eps) for a uniform distribution, and using random bits would fail this requirement.

Re: How many floating-point numbers are in the interval [0,1]?

#112

Post by Certhas »

Ummm... maybe I am mistaken, but as the fp numbers are not uniform in [0,1] chances are I don't want to pick one at random. Rather I want to pick a fp representation of a random real between 0 and 1. In that case the readers suggested strategy seems fine.

Am I wrong?

Re: How many floating-point numbers are in the interval [0,1]?

#113

Post by rtpg »

greglindahl wrote: post #86
dbcurtis wrote: post #84

Earlier quoted context omitted.

Exceptions are utterly painful for vectorized operations. Goodbye Tensorflow if you say goodbye to NaNs

That's an excellent explanation for why scientific computing has been using vectors for a long time and has traditionally faulted when hitting the first NaN. Is there something about Tensorflow that makes propagating NaNs useful? And if so, why are you generalizing from Tensorflow to all "vectorized operations"?

my guess is that this is a performance argument. If you don't have to check the results, your circuits to do huge vector calculations can be simpler.

Re: How many floating-point numbers are in the interval [0,1]?

#114

Post by petters »

Certhas wrote: post #112

Ummm... maybe I am mistaken, but as the fp numbers are not uniform in [0,1] chances are I don't want to pick one at random. Rather I want to pick a fp representation of a random real between 0 and 1. In that case the readers suggested strategy seems fine. Am I wrong?

The author says in a comment

> If hitting exactly zero is much less probable than hitting exactly 0.5, then I would argue that you do not have a uniform distribution…

But due to the non-uniformity of floating point numbers, I think the author is wrong here.

We can think of each floating point number as an interval. The probability that it is picked should ideally be equal to the size of the interval.

Re: How many floating-point numbers are in the interval [0,1]?

#115

Post by lightedman »

vog wrote: post #109
mtklein wrote: post #27

The correct answer is 1,065,353,216. This is easy to work out yourself if you remember one basic, handy property of floats: adjacent floats are adjacent in bit representation, except -0.0f and 0.0f. For example, 0x00000000 is +0.0f. 0x00000001 is the smallest non-zero positive float. 0x00000002 is the second smallest. The only exception to this is -0.0f and +0.0f, which are 0x80000000 and 0x00000000. The rule works w…

The correct answer is 1,065,353,217. You have an off-by-one error in the following line of argument: > The number of values between the two is 0x3f800000 - 0x00000000 == 0x3f800000 This should be corrected to: > The number of values between the two is 0x3f800000 - 0x00000000 == 0x3f800001 In general, the number of numbers in the closed integral interval [a, b] is not b-a, but 1+b-a. For example, the closed interval […

I was under the impression (after several thousand attempts) that the correct answer is infinity, limited only by your addressable memory and bitspace.

Re: How many floating-point numbers are in the interval [0,1]?

#116

Post by yiyus »

jacobolus wrote: post #36
ldarby wrote: post #32

I had a similar question a while ago: how many decimal digits does it take to represent the smallest possible 80 bit float. The answer is ~16000: #include int main(int argc, char **argv) { unsigned long long int mant, exp ; mant = 0; exp = 1; typedef struct _bitfield80 { unsigned int sign:1; signed int exp:15; unsigned long long int mant:64; } bitfield80; union { long double a; bitfield80 b; } union80; union80.b.sign…

Depends what you mean by “represent”. The smallest positive “denormal” 80-bit float is (I think) 2^(2 − 2^(14)). I just represented it using 5 decimal digits and two minus signs. :-)

The Berry paradox takes this a bit further.

It may appear that "the least integer not nameable in fewer than nineteen syllables" is 111777. But "the least integer not nameable in fewer than nineteen syllables" is itself a name consisting of only eighteen syllables, and therefore the least integer not nameable in fewer than nineteen syllables can be named using only eighteen syllables (!).

There are alternative expressions, for example wikipedia mentions "the smallest positive integer not definable in fewer than twelve words" and "the smallest positive integer not definable in under sixty letters".

Re: How many floating-point numbers are in the interval [0,1]?

#117

Post by dlubarov »

mcav wrote: post #62
dlubarov wrote: post #60

Earlier quoted context omitted.

> An exception is the same NaN, only less convenientlying packaged. I prefer an exception because I'd rather have my code fail fast, and immediately point me close to the source of the bug, rather than letting a NaN or Infinity propagate through my code base and cause some harder-to-debug problems down the line. That's the main thing I don't like about JavaScript. 1/0 is Infinity, Math.acos(2) is NaN, Object().foo is…

The JS behavior regarding 1/0, etc, is the same as C: http://www.gnu.org/software/libc/manual/html_node/Infinity-a...

Yes, but the issue comes up more often in JS and PHP since they don't have native integer division.

Re: How many floating-point numbers are in the interval [0,1]?

#118

Post by dlubarov »

golergka wrote: post #102
dlubarov wrote: post #60

Earlier quoted context omitted.

> An exception is the same NaN, only less convenientlying packaged. I prefer an exception because I'd rather have my code fail fast, and immediately point me close to the source of the bug, rather than letting a NaN or Infinity propagate through my code base and cause some harder-to-debug problems down the line. That's the main thing I don't like about JavaScript. 1/0 is Infinity, Math.acos(2) is NaN, Object().foo is…

> I'd rather have my code fail fast Doesn't it depend on the product in question? In some cases, you want to fail fast and loud. In others, you want to recover at all costs. Sometimes a single calculation going 1/0 means the whole program is failing. Sometimes it's a completely ignorable, minor error, not even worth checking for. Can we please not have a "default" way of handling errors? None fit all.

> Can we please not have a "default" way of handling errors? None fit all.

Well language designers have to make the choice. I think overall, the best approach is Python's behavior, where both 1/0 and 1.0/0.0 trigger a ZeroDivisionError.

Re: How many floating-point numbers are in the interval [0,1]?

#119

Post by detaro »

dlubarov wrote: post #118
golergka wrote: post #102

Earlier quoted context omitted.

> I'd rather have my code fail fast Doesn't it depend on the product in question? In some cases, you want to fail fast and loud. In others, you want to recover at all costs. Sometimes a single calculation going 1/0 means the whole program is failing. Sometimes it's a completely ignorable, minor error, not even worth checking for. Can we please not have a "default" way of handling errors? None fit all.

> Can we please not have a "default" way of handling errors? None fit all. Well language designers have to make the choice. I think overall, the best approach is Python's behavior, where both 1/0 and 1.0/0.0 trigger a ZeroDivisionError.

They could give options. E.g. have a "throwing float" and a "NaN float" as distinct data types, or some other way to parametrize the behaviour. Is that a good idea? I don't know.

Re: How many floating-point numbers are in the interval [0,1]?

#120

Post by golergka »

dlubarov wrote: post #118
golergka wrote: post #102

Earlier quoted context omitted.

> I'd rather have my code fail fast Doesn't it depend on the product in question? In some cases, you want to fail fast and loud. In others, you want to recover at all costs. Sometimes a single calculation going 1/0 means the whole program is failing. Sometimes it's a completely ignorable, minor error, not even worth checking for. Can we please not have a "default" way of handling errors? None fit all.

> Can we please not have a "default" way of handling errors? None fit all. Well language designers have to make the choice. I think overall, the best approach is Python's behavior, where both 1/0 and 1.0/0.0 trigger a ZeroDivisionError.

> Well language designers have to make the choice.

Of course. I meant, no "default" that we would consider the "right way" for all the languages.

Post reply on HN