I can't believe I'm the only person to ask "what bit depth?" There is a pretty big assumption baked into the title. Edit: hi, I see that I made a mistake, can someone correct my misunderstanding?
IEEE 754 floats are 32-bit. Doubles are 64-bit.
How many floating-point numbers are in the interval [0,1]?
131–140 of 158 posts
Re: How many floating-point numbers are in the interval [0,1]?
#132Ummm... maybe I am mistaken, but as the fp numbers are not uniform in [0,1] chances are I don't want to pick one at random. Rather I want to pick a fp representation of a random real between 0 and 1. In that case the readers suggested strategy seems fine. Am I wrong?
The author says in a comment > If hitting exactly zero is much less probable than hitting exactly 0.5, then I would argue that you do not have a uniform distribution… But due to the non-uniformity of floating point numbers, I think the author is wrong here. We can think of each floating point number as an interval. The probability that it is picked should ideally be equal to the size of the interval.
Yes. The effect of having a coarser partition around a mesh point is countered by the effect of a higher frequency of hitting that partition. This actually is the essence of importance sampling. For the purposes of numerical integration, as long as the integrand is relatively smooth with respect to the partition size, there shouldn't raise any serious problem. On the contrary, if the integrand does vary wildly between two adjacent floating point numbers, then the real issue here is that the precision (single, double, quadruple, etc.) used is not fine enough, rather than the deviation from uniform distribution.
Re: How many floating-point numbers are in the interval [0,1]?
#133Earlier quoted context omitted.
Even 'better' just store them in fixed point representation.
If all you're dealing with is angles, 64-bit quantization of the 360° space will give you far more accuracy than you'll ever need, and it will be absolutely uniform throughout.
True enough... my mind is in the embedded space where say a good enough representation in say 16-bit is desirable.
Re: How many floating-point numbers are in the interval [0,1]?
#134Honestly, I think IEEE 754 floating point numbers are a pretty bad way of dealing with real numbers and just cause tons of headaches that every math library has to deal with. Even high level programmers aren't shielded from the NaN nonsense. It would be great if we had an underlying implementation we could ignore and that allowed us to think at a more mathematical level. I actually found something recently on this to…
so I'm the author of one of the first usable unum implementations. We're actually moving to something called sigmoid numbers which are even better. The lecture doesn't cover "valid mode" but that's more like the "unum". At the end of the video, there's a demonstration where I show some very interesting results concerning machine learning. https://www.youtube.com/watch?v=aP0Y1uAA-2Y You can try out sigmoid numbers in…
Those sigmoid numbers look interesting. Is there an article somewhere that explains how they work?
Re: How many floating-point numbers are in the interval [0,1]?
#135The correct answer is 1,065,353,216. This is easy to work out yourself if you remember one basic, handy property of floats: adjacent floats are adjacent in bit representation, except -0.0f and 0.0f. For example, 0x00000000 is +0.0f. 0x00000001 is the smallest non-zero positive float. 0x00000002 is the second smallest. The only exception to this is -0.0f and +0.0f, which are 0x80000000 and 0x00000000. The rule works w…
The discrepancy with the number in the article is that you're including 2^23 mantissae with exponent 0. In any case, the interesting bit is that half of floats are in the interval [-1, 1].
> the interesting bit is that half of floats are in the interval [-1, 1]
This is not so surprising once you realize that this just means you have the same precision for 1/x as for x, which makes a lot of sense for scientific calculations.
Re: How many floating-point numbers are in the interval [0,1]?
#136Earlier quoted context omitted.
The discrepancy with the number in the article is that you're including 2^23 mantissae with exponent 0. In any case, the interesting bit is that half of floats are in the interval [-1, 1].
> the interesting bit is that half of floats are in the interval [-1, 1] This is not so surprising once you realize that this just means you have the same precision for 1/x as for x, which makes a lot of sense for scientific calculations.
I'll clarify mostly for my sake (I hadn't thought about this before)
f(x) = 1/x takes takes numbers in the range (-1, 1) to numbers outside that range, and vice versa. The representable floats are split evenly between those two sets
Re: How many floating-point numbers are in the interval [0,1]?
#137The more I read about the gnarly details of floating point specs and implementations, the more they look like a messy hack. I think it'd be nice to just use rational numbers in my programs.
Re: How many floating-point numbers are in the interval [0,1]?
#138Earlier quoted context omitted.
This is not true, and I can think of at least two arguments. First, the space of all computable reals is countable[1] (i.e. numbers which can be approximated by a sequence of rational numbers produced by a computable function). Note that this subsumes the irrational numbers as well. Second, all floating point representations would individually have at most the cardinality of the natural numbers, even if the represent…
Second, all floating point representations would individually have at most the cardinality of the natural numbers All of that is irrelevant. I'm using the cardinality of all possible floating point representations. Give me a real in [0,1] that you say doesn't have a representation, and I will give you a floating point representation that will include it. (Using an operation that's just cut and paste on the IEEE one.)
>Give me a real in [0,1] that you say doesn't have a representation, and I will give you a floating point representation that will include it.
pi
Re: How many floating-point numbers are in the interval [0,1]?
#139> Of all the float-pointing point numbers your computer can represent, a quarter of them lie in [0,1]. Many people think floating point numbers are magically precise. They're not. They're far more accurate at lower magnitudes and precision fades as you work with larger values.
Correct me if I'm wrong, but my understanding is that a consequence of this is that it's better to store, for example, angles as radians from -pi to pi rather than as degrees between 0.0 and 360.0, in order to take advantage of all that precision between 0 and 1.
Useful answer: Use a double.
Really, 52 bits of mantissa is more than enough for representing anything. You have to be wary of error building up on calculations, not of original representation errors (unless you are living on the edge; and I'd tell you: don't).
Useless answer: No, the precision is fixed for any magnitude. When you multiply your values, you will also multiply the interval, and there are as many useful numbers inside that new, larger, interval than were inside the old, smaller one.
Re: How many floating-point numbers are in the interval [0,1]?
#140for many applications, consider randomizing in [1,2) and then subtracting one. Although many of the fp values in [0,1) will be inaccessible, you are guaranteed uniformity, and also the lattice that you're drawing from will be fixed.
Generate integral values, then move them into the [0,1) interval.