Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How many floating-point numbers are in the interval [0,1]?

lemire.me

Post reply on HN
Print view View original

71–80 of 158 posts

Re: How many floating-point numbers are in the interval [0,1]?

#71

Post by astrodust »

greenleafjacob wrote: post #68
RandomOpinion wrote: post #34

Earlier quoted context omitted.

> Many people think floating point numbers are magically precise. Even worse, it kicks in earlier than one might naively think. People are often surprised to hear a value as simple as 0.1 has no matching floating point representation. (For those interested, http://www.exploringbinary.com/why-0-point-1-does-not-exist-... has a nicely illustrated explanation.)

This isn't true with e.g. decimal64.

Those sorts of numbers aren't floating point, they're typically fixed point.

Re: How many floating-point numbers are in the interval [0,1]?

#72

Post by dnautics »

SamReidHughes wrote: post #29
dnautics wrote: post #4

for many applications, consider randomizing in [1,2) and then subtracting one. Although many of the fp values in [0,1) will be inaccessible, you are guaranteed uniformity, and also the lattice that you're drawing from will be fixed.

How do randomize in [1,2)? You might as well just generate numbers from [0, 2^53) or [0, 2^24) and divide. If you didn't do that on the way to generating in [1,2), you're going to have fine-grained non-uniformity anyway.

f() = (rand() + 1.0) - 1.0

and pray that your compiler isn't clever! Or you can actually randomize the fraction bits.

Re: How many floating-point numbers are in the interval [0,1]?

#73

Post by dnautics »

jfaucett wrote: post #13

Honestly, I think IEEE 754 floating point numbers are a pretty bad way of dealing with real numbers and just cause tons of headaches that every math library has to deal with. Even high level programmers aren't shielded from the NaN nonsense. It would be great if we had an underlying implementation we could ignore and that allowed us to think at a more mathematical level. I actually found something recently on this to…

so I'm the author of one of the first usable unum implementations. We're actually moving to something called sigmoid numbers which are even better. The lecture doesn't cover "valid mode" but that's more like the "unum". At the end of the video, there's a demonstration where I show some very interesting results concerning machine learning.

https://www.youtube.com/watch?v=aP0Y1uAA-2Y

You can try out sigmoid numbers in julia here:

https://github.com/interplanetary-robot/SigmoidNumbers

Also, I am working on some verilog implementations, so we may have this in hardware rather soon.

A few things about sigmoid numbers: 1) they have fixed sizes, so you don't have to worry about keeping track of things (which was a pain). 2) they have a property I call "isomorphic structure" - which means that zero-padding does not affect the value, and truncation gives you the nearest value at a lower bit resolution. 3) they really do perform better than floating points in several applications. There's a higher bit-per-bit entropy for 'most things you want to do', so the information content is richer. The distribution of the numbers is "more human" - that is to say at a certain point you stop caring about the details around the number, and more care that it's 'really big' or 'really small'; and the distribution reflects that.

Re: How many floating-point numbers are in the interval [0,1]?

#74

Post by dnautics »

nine_k wrote: post #45
jfaucett wrote: post #13

Honestly, I think IEEE 754 floating point numbers are a pretty bad way of dealing with real numbers and just cause tons of headaches that every math library has to deal with. Even high level programmers aren't shielded from the NaN nonsense. It would be great if we had an underlying implementation we could ignore and that allowed us to think at a more mathematical level. I actually found something recently on this to…

If NaN is nonsense, what would you have 1/0 or acos(2) to produce? (An exception is the same NaN, only less convenientlying packaged.)

you throw an error and stop the calculation, and let the programmer decide what to do.

Re: How many floating-point numbers are in the interval [0,1]?

#75

Post by deepnotderp »

jcranmer wrote: post #59
jfaucett wrote: post #13

Honestly, I think IEEE 754 floating point numbers are a pretty bad way of dealing with real numbers and just cause tons of headaches that every math library has to deal with. Even high level programmers aren't shielded from the NaN nonsense. It would be great if we had an underlying implementation we could ignore and that allowed us to think at a more mathematical level. I actually found something recently on this to…

A cursory examination of the documentation for and against the unum stuff seems to indicate a vehement ad hominem argument running between Gustafson and Kahan, which makes it somewhat hard to find objective information. The unum stuff does seem to be greatly oversold, though. Fundamentally, there is a problem in mathematics: some equations naturally blow up tiny uncertainties into major ones. Indeed, in chaotic syste…

> I didn't see a response from Gustafson to the latter accusation--the slide deck I see has him shifting from an argument about the power usage involved in storing 64 bits of "unnecessary" precision to a commentary about the parallelizability of his approach (no comment about the work-efficiency!).

Gustafson has addressed this. His argument essentially boils down to the fact that moving 64 bits to memory costs far more than doing the actual computation. Therefore, you have to spend more transistors doing the computation for unums, but you spend much less energy overall, since you have to transport less energy.

Re: How many floating-point numbers are in the interval [0,1]?

#76

Post by SamReidHughes »

strainer wrote: post #70
SamReidHughes wrote: post #65

Earlier quoted context omitted.

Well. That'll do it. Probably no slower than the integer to float conversion too. It gets you multiples of 2^-52 instead of 2^-53 though.

I see your talking 64 bit floats there which is my habit too. Having only 24 bit mantissa float32s seem insufficient for producing practically uniform variates - the missing 2^24th can be spotted averaging just a billion or so of them. I dont know if it has been improved recently but last year Chrome's Math.random was only putting 32 bits into its float64 - the missing four-billionth can be suggested from the average…

Yeah, I really think a better way is to imagine generating a uniform real and rounding -- generating [0,1) instead of [0,1] is just sick. If you do rand() * k + n you can get [n, n+k] anyway, so it's not a good primitive to get a half-open interval with. You can generate an integer n in [1, 2^25], then take (n >> 1) * (1 / (float)(1 << 24)) to get a pretty good [0,1].

Re: How many floating-point numbers are in the interval [0,1]?

#77

Post by dasfasf »

astrodust wrote: post #8

> Of all the float-pointing point numbers your computer can represent, a quarter of them lie in [0,1]. Many people think floating point numbers are magically precise. They're not. They're far more accurate at lower magnitudes and precision fades as you work with larger values.

Every number system in which all numbers occupy the same amount of space must have this property (namely, that there are roughly as many positive numbers above 1 as below 1, and therefore that there is greater precision near 0) or the accuracy of the function x->1/x will suffer.

Re: How many floating-point numbers are in the interval [0,1]?

#78

Post by protomikron »

brianberns wrote: post #52
rnhmjoj wrote: post #50

Earlier quoted context omitted.

Ideally I would restrict all functions to their natural domain using refinement types. This ways 1/0 simply doesn't type check and you don't have to bother assigning it some nonsensical value. However I can see something like this becoming much more a hassle than a benefit when things get complicated enough.

I'm not very familiar with refinement types. How would that work? For example, how would you write a function that answers the mean value of an array of numbers? You could define Array.Length to be of type NonNegativeInteger, but that still includes 0. Alternatively, you'd have to define a NonEmptyArray type that excludes arrays of length 0, but that seems less than useful.

> Alternatively, you'd have to define a NonEmptyArray type that excludes arrays of length 0, but that seems less than useful.

I think the usefulness of a specific NonEmptyArray type is controversial, and I would say such types could be extremely useful. AFAIK there have been discussions if Haskell's Prelude (part of the standard library providing a name space for convenient functions and types, similar to the functions you get in e.g. Python like map, zip, filter, range, etc.) should be more save by not advocating too generic types and functions like the standard `map`, `head` and `tail` implementations.

Functions like `head` and `tail` can be seen as problematic as they can fail and cause runtime errors (the head and tail of an empty list cause a runtime error), but I do not think they already changed that (or if it's even a topic of discussion at the moment).

It's a little bit a matter of taste. Think about Unix' "head" program that by default prints maximal 10 lines of a file. How should we deal with the following?

    $ touch empty.txt
    $ head -n1 empty.txt
Currently this works - you do not get output and $? == 0, but one could also say "head" should panic here.

Re: How many floating-point numbers are in the interval [0,1]?

#79

Post by mark-r »

drv wrote: post #10
murkle wrote: post #6

Article says 1,056,964,609 / 1,056,964,610 Quick Java program says 1,065,353,216 / 1,065,353,217 Which one is right? Or are Java floats just "different"? static { float a = 0; long count = 0; while (a

nextAfter is probably also including the denormals (an additional 2^23 values near 0).

And the author explicitly stated he was excluding the denormals.

Re: How many floating-point numbers are in the interval [0,1]?

#80

Post by mark-r »

amelius wrote: post #5
santaclaus wrote: post #3

What about nan? The test '!(x 1)' will say yes, while 'x >= 0 && x Or -0 for that matter.

Common sense approach: like the name says, NaN is not a number. The question was about floating point numbers, not floating point values. So I suppose we shouldn't count NaN. Further, -0 (an arbitrarily small number less than 0) doesn't logically fall in the range [0, 1], so it is excluded.

There's no need to exclude -0.0, since mathematically and by all tests available to the floating point unit it's equal to +0.0.
Post reply on HN