Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

9999999999999999.0 – 9999999999999998.0

geocar.sdf1.org

Post reply on HN
Print view View original

241–250 of 274 posts

Re: 9999999999999999.0 – 9999999999999998.0

#241

Post by BeeOnRope »

leiroigh wrote: post #150
TFortunato wrote: post #121

Earlier quoted context omitted.

How do you mean? The x86-64 instruction set / abi specifies long doubles as 80-bits, and still supports them ...

And nobody uses this terrible mis-feature in practice, everything runs via 64 bit xmm registers. Rightly so, because programmers want their optimizing compiler to decide when to put a variable on the stack and when to elide a store/load cycle by keeping it in a register. With 80 bit precision, this makes a semantic difference and you end up in volatile hell.

To be fair, the problem you describe isn't inherent to 80-bit floating point values. If you use 80-bit values in your ABI or language definition, it won't occur - it occurs when you try to user a wider type to implement a narrower type, e.g., implementing 64-bit floats (as specified in the ABI or language) with 80-bit operations.

In that case, the extra precision is present and "carried across" operation when registers or the dedicated floating point stack is used, but is discarded when values are stored to a narrower 64-bit location. So the problem is one really of mismatch between the language/ABI size and the supported hardware size. Of course, 80 bits isn't a popular floating point size any more in modern languages, so this happens a lot.

Re: 9999999999999999.0 – 9999999999999998.0

#242

Post by dannypgh »

josefx wrote: post #232
dannypgh wrote: post #127

Earlier quoted context omitted.

At some point, your test switches from testing the code, to testing the machine the code runs on. That likely happens before 120 days.

Given issues like the intel fdiv bug that may make sense to test if you really want to avoid running into hardware specific bugs.

Maybe, but I'd rather a test suite that's designed to test hardware, rather than overloading some code's unit tests.

I think most unit tests are best served by testing key values- e.g. values before and after any intended behavior change, values that represent min/max possible values, values indicative of typical use.

The unit test can serve as documentation of what the code is intended to do, and meaninglessly invoking every unit test over the range of floats obscures that.

There are certainly cases where all values should be tested, but I don't think that's all cases.

Re: 9999999999999999.0 – 9999999999999998.0

#243

Post by tntn »

vanderZwan wrote: post #238
nestorD wrote: post #217

Earlier quoted context omitted.

To my understanding, experiments with Unums showed shortcoming that Gustafson didn't anticipate and lead to Posits which drop the fixed length constraint. Doing that makes improving precision a lot easier but at the cost of computation time. Overall I am not convinced that the current implementation is optimal but it is a very good trade-off between speed and precision.

> Doing that makes improving precision a lot easier but at the cost of computation time. Not quite. The difference in computation time is the current the lack of hardware support, not something inherent to the underlying encoding method. So in practice you are right, but in, for example, embedded contexts without floating point hardware, the performance advantages of IEEE floats should disappear (especially if using…

Even if hardware support existed, it seems like a variable length encoding has some inherent overhead relative to a fixed length encoding. If you have a "base length" of e.g. 32 bits and occasionally expand to 64, there's an inherent cost there in both computation and memory, presumably for greater precision. Perhaps that overhead could be minimal with hardware support, but it seems it must have some.

Re: 9999999999999999.0 – 9999999999999998.0

#244

Post by svat »

tasty_freeze wrote: post #126
svat wrote: post #71

Earlier quoted context omitted.

> considering the problem is to fit the reals into 64/32/16 bits and have fast math Floating-point numbers (and IEEE-754 in particular) are a good solution to this problem, but is it the right problem? I think the "minimum of surprises" part isn't true. Many programmers develop incorrect mental models when starting to program, and get no feedback to correct them until much later (when they get surprised). It is true…

When I went to university in 1982, one of the lower level courses was called "Numerical Methods". It went over all of the issues related to precision, stability, as well as a host of common numerical integration and approximation methods. I'm just a sample size of one, but isn't this kind of class a requirement for CS majors?

From a 1998 interview with William Kahan, the “father” of IEEE-754 floating point (emphasis mine):

> My reasoning was based on the requirements of a mass market: A lot of code involving a little floating-point will be written by many people who have never attended my (nor anyone else's) numerical analysis classes. We had to enhance the likelihood that their programs would get correct results. At the same time we had to ensure that people who really are expert in floating-point could write portable software and prove that it worked, since so many of us would have to rely upon it. There were a lot of almost conflicting requirements on the way to a balanced design.

I imagine that the number of people writing code without having taken a numerical methods class has only increased since the late 1970s being talked about, or even in the two decades since that interview.

Re: 9999999999999999.0 – 9999999999999998.0

#245

Post by hakfoo »

CamouflagedKiwi wrote: post #237
dotancohen wrote: post #233

Earlier quoted context omitted.

Even with money I use integers. Instead of dollars (or local currency), I store values internally as pennies (or local equivalent 1/100 of main currency). Sometimes when working with interest I'll need to work with floats, and some databases I have values stored as DECIMAL(8,2) instead of INT, but for the most part I've saved quite a few headaches by keeping my values in INTs.

That seems okay until you need to track sub-penny accuracy somewhere, then you have a big problem. Lots of applications don't need that, but a surprising amount do, so it's not a global solution.

But there's still a minimum significant value which can be defined from the problem space. Do DECIMAL(16,8) or whatever.

In some situations, there may even be industry or legal standards as to what can be considered rounding noise.

Re: 9999999999999999.0 – 9999999999999998.0

#246

Post by twtw »

hedora wrote: post #178
twtw wrote: post #39

I don't understand all the crap that IEEE 754 gets. I appreciate that it may be surprising that 0.1 + 0.2 != 0.3 at first, or that many people are not educated about floating point, but I don't understand the people who "understand" floating point and continue to criticize it for the 0.1 + 0.2 "problem." The fact is that IEEE 754 is an exceptionally good way to approximate the reals in computers with a minimum number…

I took the table to be a handy guide to where arbitrary precision is the default vs. hw accelerated math. Filtered by languages I care about, I guess I have no choice but to learn perl 6 if I want correct (but presumably slow) floating point with elegant syntax (my taste might not match yours). I’d be curious to know what the random GPU languages and new vector instruction sets do with this computation. I don’t think…

Can't comment on the situation with other GPU languages, but CUDA on GPUs since fermi are 754 compliant, with the exception that certain status flags are unavailable.

Re: 9999999999999999.0 – 9999999999999998.0

#247

Post by twtw »

sgt101 wrote: post #212
twtw wrote: post #46

Earlier quoted context omitted.

Julia has built in rationals (as do a few other languages). I'm not aware of any language (other than Wolfram) that defaults to storing something like 0.1 as 1/10 - i.e. uses the decimal constant notation for rationals, rather than having some secondary syntax or library.

Yes, but the Julia REPL produces 2.0 as an answer and casting both these to BigInt doesn't work either.

Thus my second paragraph. You have to opt in to use other formats.

Casting to bigint doesn't work because the problem occurs when converting the decimal constant in the source to floating point. You would have to convince the parser to parse the constant as something besides a float.

Re: 9999999999999999.0 – 9999999999999998.0

#248

Post by simonbyrne »

TFortunato wrote: post #207
simonbyrne wrote: post #184

Earlier quoted context omitted.

Essentially there are two different sets of floating point instructions on x86 and x86-64: - the x87 instructions, which descend from the original 8087 coprocessor (and have 80-bit registers), and - the SSE instructions, which descend from the Pentium MMX feature set, are faster, support SIMD operations, and can be fully pipelined. The x87 instructions are basically for legacy compatibility, or if you manually use lo…

Yep - and there are absolutely some cases where you do want to manually use it which is why the x86_64 ABI on SysV (used by Linux and OS X, still specifies the long double type as 80-bits, and why GCC and Clang will still emit these instructions when long doubles are used! (Sorry, this is more for the folks who aren't familiar with this, since it seems like you are familiar, but I didn't want it to seem like this isn…

I agree that extended precision can be very useful, though I think the failing was on the software side: basically languages and compilers didn't provide useful constructs to control things like register spilling (which caused the truncation of the extended precision).

The current hardware trends seem to be providing instructions for compensated arithmetic, like FMA and "2sum" operations. I think this is ultimately a better solution, and will make it possible to give finer control of accuracy (though there will still be challenges on the software/language side of how to make use of them).

Re: 9999999999999999.0 – 9999999999999998.0

#249

Post by Gibbon1 »

zokier wrote: post #91
coddingtonbear wrote: post #70

Earlier quoted context omitted.

Is that really true? In my experience, 99.9% of the time I don't need an exact number; the vanishingly few times when I have such a need (almost entirely calculations involving currency), using a fixed point representation is simple enough.

I think there is argument to be made for high-level languages defaulting for arbitrary precision math ("make it correct first, fast second"). But considering that we are still fumbling around with fixed-width integers and that is much simpler domain after all, I don't hold my breath on "solving" the problem of reals any time soon.

That's exactly my point, why is the default a lossy format? And consider the distinction between variables and calculations. Formats like IEEE754 are designed for performing fast high accuracy transformations on matrices. I have no complaints about that. But the default arbitrary number format should be able to store exact integers and ratio's.

Re: 9999999999999999.0 – 9999999999999998.0

#250

Post by bigiain »

dotancohen wrote: post #234
bigiain wrote: post #182

Earlier quoted context omitted.

I did a web project in the gambling space ~10 years back - we were legally required to perform all calculations as integers in ten thousandths of a cent (or millionths of a dollar). We chose to _not_ do _any_ calculations client side in Javascript...

Which regulation is that? I've worked on financial applications, but not gambling, and I've not heard of this regulation. I should probably know about it!

Australian, or possibly Tasmanian state regs. This would have been around 2011 or so (The Samsung Galaxy S2 was the "top of the range Android phone" at the time...)
Post reply on HN