181–190 of 201 posts
Earlier quoted context omitted.
x87 has been functionally superseded (largely by various SSE iterations), but it is still supported for backwards-compatibility. Modern applications and compilers largely do not use x87, but they can, and x87 results depending on hidden bits persists in modern x86_64 CPUs.
yet it is still used for arithmetic with "long double"
Earlier quoted context omitted.
yet it is still used for arithmetic with "long double"
Wouldn't that resovle the issue? long double contains the whole 80 bit state, not much space for undefined bits left.
One annoying part about floats is the abuse of the NaN space. Some runtimes like SpiderMonkey, JSC, and LuaJIT abuse the NaN space to store pointers in doubles. This practice is often called nan-boxing, but has a few variants. A more efficient use of bits like with Posits will break this. As for the claim about a FPU taking up less space and power for posits than floats, facebook made that claim: https://code.fb.com/…
I would say that this is certainly my favorite feature of IEEE floats :)
The irreproducibility of IEEE 754 float calculations is well documented... on Wikipedia, by William Kahan, and in an excellent paper by David Monniaux titled "The pitfalls of floating-point computations". It is amazing that this is tolerated, but IEEE 754 has done a great deal to lower the expectations of computer users regarding mathematically correct behavior.
The posit approach is not merely a format but also the Draft Standard. Whereas floats can arbitrarily use "guard bits" to covertly do calculations with greater accuracy, the posit standard rules that out. Whereas the float standard recommends that math functions like log(x), cos(x) etc. be correctly rounded, the draft posit standard mandates that they be correctly rounded (or else they have to use a function name that clarifies that they are not the correctly-rounded function). By the draft posit standard, you cannot do anything not specified in the source code (like noticing that a multiply and an add could be fused into a multiply-add with deferred rounding, so calling fused multiply-add without telling anyone). The source code completely defines what the result will be, bitwise, or it is not posit-compliant. It cannot depend on internal processor flags, optimization levels, or special hardware with guard bits to improve accuracy; this is what corrupted the IEEE 754 Standard and made it an irreproduci ble environment to this day.
The claim that posits is a "drop-in" replacement for floating point needs a lot of clarification, and this is unfortunately left out of much of the coverage of the ida. Clearly, if an algorithm assigns a hexadecimal value to encode a real value, that will need work to port from IEEE floats to posits. The math libraries need to be rewritten, as well as scanf and printf in C and their equivalent for other languages. However, a number of researchers have found that they can substituted a posit representation for a float representation of the same size, and they get more accurate results with the same number of bits. I call that "plug-and-play" replacement; yes, there are a multitude of side effects that might need to be managed, but it's nothing like the jarring change, say, of moving from serial execution to parallel execution. It's really pretty easy, and it's easy to build tools that catch the 'gotcha' cases.
Some here have suggested the use of rational number representation, or said that there are redundant binary representations of the same numerical value. Unlike floats, posits do not have redundancy. I suspect someone is confused by the Morris approach to adjusting the tradeoff between fraction bits and exponent bits, which produces many redundant wa6s to express the same mathematical value.
Perfect additive associativity is available, as an option, with the quire. If needed. Multiplicative associativity is available, as an option, by calling fused multiply-multiply in the draft posit standard. Because quire operations appear to be both faster (free of renormalization and rounding) and more accurate (exact until converted back to posit form), I am puzzled regarding why anyone would want to do things more slowly and with less accuracy.
Kulisch blazed the way with his exact dot product; unfortunately, any exact dot product based on IEEE floats will have an accumulator with far too many bits (like 4,224 for IEEEE double precision) and an accumulator that is just a bit larger than a power-of-two size. The "quire" of posits is always a power-of-two, much more hardware-friendly. It's 128 bits for 16-bit posits, and 512 bits for 32-bit posits, the width of a cache line on x86, or a an AVX-512 instruction.
"A little knowledge is a dangerous thing." In evaluating posit arithmetic, please use more than what you see in a ycombinator blog. You might discover that there are several decades of careful decision-making behind the design of posit arithmetic. And unlike Kahan, I subject my ideas to critical review by the community and learn from their input. The 1985 IEEE format is grossly overdue for a change.
Very few developers truly understand floating point representation. Most think of it as base-10, and put in horrific kludges and workarounds when they discover it doesn't work as they (wrongly) expected. I shudder to think how many e-commerce sites use `float` for financial transactions! So as far as I'm concerned, whatever performance cost these alternate methods may have, it would be well worth it to avoid the pitf…
In the paper "Do Developers Understand IEEE Floating Point" the authors surveys phd student and faculty members from computer science and found out that your observation is true: most people don't know how fp works. They have the survey online at [1] in case you want to see how much you know about fp behavior. [1] http://presciencelab.org/float
Earlier quoted context omitted.
Unrelated as far as I can tell. Posits are an alternative floating point format, not an array compression algorithm. The “compression” benefits from being able to use posit floats instead of ieee754 doubles, because of the better precision.
Posits also take less space for the same total accuracy, they use less space on disk, ram and less memory bandwidth.
Earlier quoted context omitted.
Yes and no. 1. Yes -- On old x86 systems, the x87 registers were 80-bits and the bottom bits were undefined. You could theoretically have a 1-bit difference on some results depending on what the bottom 26 bits (that were undefined, because you had 64-bit floats most of the time, even though the machine did things 80-bits at a time). 2. No -- Modern x86 systems use SSE registers, which are 64-bits. There are a whole s…
Common Lisp over here like: * (+ (+ 1 1) (expt 2 256)) 115792089237316195423570985008687907853269984665640564039457584007913129639938 * (+ 1 (+ 1 (expt 2 256))) 115792089237316195423570985008687907853269984665640564039457584007913129639938
The point is to demonstrate a property of IEEE floats, not the language. I'd expect CL has some means of using IEEE floats too, and with the appropriate syntax, the same issues could be demonstrated.
Earlier quoted context omitted.
> They likely compared the binnary64 with posit16 using the accumulator (aka quire). As far as I can tell, Klöwer didn't. He mentions around the 13 minute mark (slide 9) that he used the SigmoidNumbers software package for Julia. From the looks of the examples he gives, if he used the quire, it must have happened implicitly. Which IIRC is not how using the quire works. He did rescale all his inputs to minimize roundi…
Hmm. Their results seems to good to be true without using quire. Look: The two largest/smallest exponent posit16 can represent are: min exp: 2^{ (+14 Note those numbers only have the implicit bit as significant. They don't have any space left to encode any other information other than the sign and regime. While the double (binary64) can represent a much larger range of exponent: min exp: 2^{ (-1023-2047-1) } * (1+0)…
> @milankloewer Would mind joining this thread on HN: https://news.ycombinator.com/item?id=20392612 We are discussing the results of your work with posit.
> Happy to join. In short: I did not use any quires so far. All simulations are entirely based on 16bit posits, but I compare them to Float16 and not Float64. Tricks are: Scaling and rewriting algorithms to avoid very large and very small numbers, that's it.
Obviously for any 32 bit value there’s only 2^32 possible values. So logically, a better number format is all about the distribution of values and reducing redundancy. It sounds like the basic idea here is that rather than a fixed amount of significant figures, you get more precision in the middle. Meanwhile, you can also get larger exponents. Is that right?
I believe this has even fewer allowable states because there's more than one way to represent the same number
Earlier quoted context omitted.
Reproducibility is an orthogonal issue to Posits vs IEEE754. Most developers prefer speed over reproducibility, and are encouraged to use denormal-to-zeros, fast-math optimizations, fused MAC, approximated square root function and whatever else is available to achieve results. The IEEE754 standard provides a guarantee for deterministic results, and many multi-precision and interval arithmetic libraries depend on this…
> Most developers prefer You're free to voice your own opinion, but I take some issue with people asserting theirs as if they speak for "most developers". Especially if it comes from a new account with a name like "Gustafnot". That doesn't exactly scream "unbiased" to me.
If you really want to claim the opposite, do you have evidence? Or experience that it’s true?