What is the practical application of this?
Currently working on monetization strategy.
Doing A/B tests with ads.
Looking for first round investors.
How many floating-point numbers are in the interval [0,1]?
91–100 of 158 posts
Re: How many floating-point numbers are in the interval [0,1]?
#92Earlier quoted context omitted.
Only a subset of scientific codes benefit from denormals, and it does not appear that implementing them is that big of a deal, given that 4-stage pipelines can do it. The pipeline depth to memory is a lot bigger than 4.
If you consider every iterative algorithm that solves systems of nonlinear equations a subset you cam ignore.... SPICE, linpack, fishpack.... sorry, denormals are essential to today's algorithms. As to "no big deal".... show me your code. Don't worry, I'll be able to understand it. After 20 years as a CPU designer I've learned how to understand bit-bashing.
The iterative algorithms for all of the scientific fields that I've worked on don't fall within the subset that you're describing, so no, I'm not going to agree that you're speaking about the majority of today's algorithms. Weather, climate, finite differencing, FFT... nope. Implicit systems, sure. Radar cross-sections, sure.
And as a CPU designer, are you disagreeing that most FP units today take 4 cycles (fully pipelined) (add and multiply, of course)? And that main memory is a lot farther away than that?
Given 20 years of experience, you missed out on the Cray PVP machines that were the start of this sub-thread. But the cycle counts I'm giving are modern ones. Cray did eventually implement IEEE in these machines without any significant problems, but that was more than 20 years ago.
Re: How many floating-point numbers are in the interval [0,1]?
#93Earlier quoted context omitted.
Thanks! To be just a bit more precise, I am claiming that all possible floating point systems, taken together, would contain the same number of values in [0,1] as there are points on the real line.
Nobody uses "floating point number" to mean "a number that could theoretically exist in a floating point system I will invent for you after you tell me the number". That's not being pedantic, that's insisting on a wrong definition.
Nobody uses "floating point number" to mean "a number that could theoretically exist in a floating point system I will invent
Put a full stop there -- Oh yes, people do mean that! Most of the time, people are referring to a bit representation in a specific system like IEEE's.
a floating point system I will invent for you after you tell me the number"
The above part of the sentence isn't a definition of "floating point number." It's how I use the common notion of "floating point number" in my argument. Your objection only looks like it works because you conflate the two concepts. If you don't conflate the two concepts, then you are arguing that most low level programming that deals with 32 bit floats doesn't actually deal with "floating point numbers." That is a reasonable assertion for a reasonable set of definitions. However, it's not the one I'm using, which also fits the reality of the mental models most people are actually using.
In terms of program correctness, you can find many examples where using an abstract concept of "floating point" instead of the concrete representation will produce errors. So then why is it "the correct one" as you say?
Re: How many floating-point numbers are in the interval [0,1]?
#94Honestly, I think IEEE 754 floating point numbers are a pretty bad way of dealing with real numbers and just cause tons of headaches that every math library has to deal with. Even high level programmers aren't shielded from the NaN nonsense. It would be great if we had an underlying implementation we could ignore and that allowed us to think at a more mathematical level. I actually found something recently on this to…
You say it's pretty bad and nonsense, yet you haven't found (or even looked very hard) about alternatives. So maybe don't jump to the conclusion that floating point is terrible just yet :)
Re: How many floating-point numbers are in the interval [0,1]?
#95Earlier quoted context omitted.
Second, all floating point representations would individually have at most the cardinality of the natural numbers All of that is irrelevant. I'm using the cardinality of all possible floating point representations. Give me a real in [0,1] that you say doesn't have a representation, and I will give you a floating point representation that will include it. (Using an operation that's just cut and paste on the IEEE one.)
Chaitin's constant for Turing Machines. Also note that I mentioned the cardinality of all representations in the next sentence.
Sorry, you missed it. You could formulate the proof with only 32 bit FP numbers. I've created a function that maps any real number, computable or not, to a notional FP representation. The function doesn't actually have to be computable to show a bijection. I just have to show there is a way to do the mapping.
You would have to come up with a mighty strange notion of "floating point representation" that allowed you to have more distinct representations than computable functions
That is what I think I did. Let's say that you have a real number X in [0,1] that you say isn't in the set of possible "floating point representations." I could just posit a reformulation of the IEEE 32 bit FP where NaN is used to represent that number instead. There you have your mapping. Now reset the universe to the state it was 5 minutes ago. In that alternate reality, you say you have a different real number. We also call that real number X and in that other reality I posit a reformulation of the IEEE 32 bit where NaN is used to represent that number X instead.
Just because such a set isn't computable, that doesn't mean it isn't conceivable in a way that can show a bijection.
Re: How many floating-point numbers are in the interval [0,1]?
#96(I invoke my pedant-pass.) As formulated in the title, "How many floating-point numbers are in the interval [0,1]?" you could argue that this is the cardinality of the Real Numbers. What the article says it is really talking about are single-precision IEEE 754 floating-point numbers. However, I could define any number of my own floating-point representations at various sizes. The cardinality of all possible floating…
Yes, for every number, one can come up with a floating point format that can represent it exactly (proof that stretches the meaning of 'float format' a bit: given x, the one-bit float format with bit value 1 meaning 'x', and bit value 0 meaning 'not x' is a representation that represents x exactly), but the cardinality of the set of arbitrarily length bit sequences is smaller than that of the reals in [0,1], so there…
so there isn't a single floating point representation that can represent all of the numbers in [0,1]
I agree! What I've done is to propose (an admittedly non-computable) set of floating point representations for which you can show a bijection to the real numbers. So if you accept my stretched meaning, which you do so above, then we can answer the title's question with "the cardinality of the real numbers."
Re: How many floating-point numbers are in the interval [0,1]?
#97(I invoke my pedant-pass.) As formulated in the title, "How many floating-point numbers are in the interval [0,1]?" you could argue that this is the cardinality of the Real Numbers. What the article says it is really talking about are single-precision IEEE 754 floating-point numbers. However, I could define any number of my own floating-point representations at various sizes. The cardinality of all possible floating…
Assuming "floating point" refers to things fitting the IEEE754 spec at some precision, I'm pretty sure there are still only countably many of them. After all, for any specific size of mantissa and exponent, there will be a finite number of floats of that size, and there are a countable number of options for mantissa and exponent (corresponds to N²), thus the number of floating point values is countable. Alternatively…
Assuming "floating point" refers to things fitting the IEEE754 spec at some precision
Not my assumption!
Re: How many floating-point numbers are in the interval [0,1]?
#98Slightly OT but regarding floats - I was wondering if we could get by just with an exponent and ditch the mantissa. Each number would be represented as an integer exponent of 2^-n for n corresponding to the precision.
Re: How many floating-point numbers are in the interval [0,1]?
#99Earlier quoted context omitted.
The Cray 1 was done before anyone realized the importance of denormals. And denormals are gruesome to implement on vector machines anyway.
Only a subset of scientific codes benefit from denormals, and it does not appear that implementing them is that big of a deal, given that 4-stage pipelines can do it. The pipeline depth to memory is a lot bigger than 4.
Only a subset of scientific codes benefit from denormals, and it does not appear that implementing them is that big of a deal, given that 4-stage pipelines can do it.
Maybe I'm misunderstanding your terminology, but it seems like you are saying that operations involving denormals have the same latency as normal floating point multiplications and additions. At least for multiplication on Intel chips through Haswell, I think the current case is that subnormals between 0 and FLT_MIN still have abysmal performance --- 100+ cycles of penalty.
Here's Bruce Dawson from a few years ago:
Performance implications on SSE
Intel handles NaNs and infinities much better on their SSE FPUs than on their x87 FPUs. NaNs and infinities have long been handled at full speed on this floating-point unit. However denormals are still a problem.
On Core 2 processors the worst-case I have measured is a 175 times slowdown, on SSE addition and multiplication.
On SandyBridge Intel has fixed this for addition – I was unable to produce any slowdown on ‘addps’ instructions. However SSE multiplication (‘mulps’) on Sandybridge has about a 140 cycle penalty if one of the inputs or results is a denormal.
https://randomascii.wordpress.com/2012/05/20/thats-not-norma...
And here's the overview from a slightly more recent paper:
C. Subnormal Performance Variability
Due to the complex nature of the floating point numbers, processors struggle to handle certain inputs efficiently. In particular, it is well understood that operating on subnormal values can cause extreme performance issues, including slowdowns of up to 100× [19]. As an example, on a Core i7 processor using SSE instructions, performing standard mul- tiply between two normal numbers takes 4 clock cycles, whereas the same multiply given a subnormal input takes over 200 clock cycles.
https://cseweb.ucsd.edu/~hovav/dist/subnormal.pdf
Are you saying this has been fixed in recent (or non-Intel) chips? Or maybe you were considering only NaN and Inf when you said denormals?
Re: How many floating-point numbers are in the interval [0,1]?
#100Earlier quoted context omitted.
Chaitin's constant for Turing Machines. Also note that I mentioned the cardinality of all representations in the next sentence.
Sorry, you missed it. You could formulate the proof with only 32 bit FP numbers. I've created a function that maps any real number, computable or not, to a notional FP representation. The function doesn't actually have to be computable to show a bijection. I just have to show there is a way to do the mapping. You would have to come up with a mighty strange notion of "floating point representation" that allowed you to…
Even when you move the goalposts (a floating point representation where you can't perform any operations with the numbers you "represent"?) there is still at least one big hole in your trick.
The set of describable real numbers in any particular theory (e.g ZFC) is also countable (because there are a countable number of formulas of any kind), and the set of all formal theories (even including the ones which cannot model the real numbers) is still countable. That's already giving you a lot of leeway with the term "real number". So now you'd have to call your floating point representation a representation of a real number when you can't even specify the X, no matter how abstract or lengthy the description.
In the future if you're going to use your "pedant-card" like that, do it someplace that isn't HN where you'll run headlong into a lot of people who actually know a few basic things about cardinality.