Live data from Hacker News

Log is non-monotonic in PHP and Lua

purplesyringa.moe

31–40 of 58 posts

Re: Log is non-monotonic in PHP and Lua

#31

> Everyone already knows floating-point operations are imprecise and it wouldn’t be fun to blog about. Wellll... Yes and no. And I want to nitpick "FP ops are imprecise" because it's important sometimes. It does come up in Lua - Lua uses 64-bit double floats for everything, _even array indexing_, because they have 53 bits of mantissa and they're guaranteed to represent all 32-bit integers with 100% precision. I just…

> The usual "0.1 + 0.2 != 0.3" is _not_ imprecision.

Correct. That’s using the wrong base. Decimal floating point doesn’t have that problem.

Re: Log is non-monotonic in PHP and Lua

#32
post #3

Recently I was reading an article on the EML operator (exp-min-log) that uses exponentiation and logarithm to build elementary math functions including arithmetic operations. There was a table of results from testing across languages. Language Result of 2 x 3 Error --- Node.js v25.3.0 6.000000000000000 0 Python 3.9.6 6.000000000000001 8.88e-16 PHP 8.5.1 6.000000000000001 8.88e-16 Go 1.26.2 6.000000000000000 0 Rust 6.…

I assume these results are all amd64. Is this the same on arm64?

Re: Log is non-monotonic in PHP and Lua

#33

Earlier quoted context omitted.

The dilemma is hypothetical (because we now know the exact precision requirement for those functions over all 64-bit doubles) but let's assume that we don't know that and also we happen to use decimals instead. What if, some f(x) is known to be, say, 123.4999...9997 with 1,000 fractional digits? You have to calculate at least 1,000 decimal digits of f(x) in order to correctly round, before that we only know it's betw…

> because we now know the exact precision requirement for those functions over all 64-bit doubles Do you have a link or something on this?

I'm on a mobile right now so I can't give the exact paper link but you can learn about the CORE-MATH project [1].

[1] https://core-math.gitlabpages.inria.fr/

Re: Log is non-monotonic in PHP and Lua

#34

> Everyone already knows floating-point operations are imprecise and it wouldn’t be fun to blog about. Wellll... Yes and no. And I want to nitpick "FP ops are imprecise" because it's important sometimes. It does come up in Lua - Lua uses 64-bit double floats for everything, _even array indexing_, because they have 53 bits of mantissa and they're guaranteed to represent all 32-bit integers with 100% precision. I just…

Thank you! I completely agree with this, floats are treated as a lot more magical than they actually are. I myself often rely on their exact guarantees for fun tricks (e.g. fast precise intfloat conversion: https://purplesyringa.moe/blog/fast-limited-range-conversion...). But while IEEE-754 is very precise, some subtleties arise when you add library functions to the mix -- many libm's and userland libraries don't guarantee 0.5 ulp precision for certain operations and don't document the guaranteed precision either, at which point you're left guessing and saying "well, I guess I should treat floats as magic in this case after all". I added "it's not imprecision" to step around this whole question.

Re: Log is non-monotonic in PHP and Lua

#35
post #25

There's a widespread misconception (not shared by the article author) that floating point arithmetic is "imprecise" in the sense that the result is off by some amount of random noise. On the contrary, IEEE floating point results are precisely specified to produce the closest representable value to the mathematically exact result. For efficiency reasons, library functions (and sometimes, sadly, hardware implementation…

> On the contrary, IEEE floating point results are precisely specified to produce the closest representable value to the mathematically exact result. Not sure about Rust, but C++ only guarantees that for four arithmetic operations and `sqrt`. Not `pow`, not `log`. Not even when when all inputs/outputs are integers. Sometimes you can even get `(int)pow(10, 2) == 99`, which I believe is fully standard-compliant.

Yeah, those operations (+fma) are the only ones for which IEEE 754 mandates correct rounding.

Calculating correct rounding for other functions without hardware support is both challenging to implement and is often really slow. And that's before one gets into special functions. Or, even worse, inverses of special functions. Sometimes you might be lucky when those don't over/underflow around the edges of the domain

Re: Log is non-monotonic in PHP and Lua

#36
post #25

There's a widespread misconception (not shared by the article author) that floating point arithmetic is "imprecise" in the sense that the result is off by some amount of random noise. On the contrary, IEEE floating point results are precisely specified to produce the closest representable value to the mathematically exact result. For efficiency reasons, library functions (and sometimes, sadly, hardware implementation…

Thank you! This point has been driving me mad for the last couple of years.

While answering a sibling comment, I found a paper analyzing various libms for their precision: https://homepages.loria.fr/pzimmermann/papers/glibc238-20230... (2023). I only skimmed it, but it looks like only LLVM's libm guarantees 0.5 ulp for every supported single-precision operation, and everyone else is wildly off. That's a pretty good result, though -- it means there's at least one reasonably compliant libm :)

Another sibling comments says that guaranteeing 0.5 ulp for double-precision operations is nigh impossible (and then another says it is after all). I don't have the knowledge to confirm which is true, but it's possible that this is the best we can get.

Re: Log is non-monotonic in PHP and Lua

#37
post #3

Recently I was reading an article on the EML operator (exp-min-log) that uses exponentiation and logarithm to build elementary math functions including arithmetic operations. There was a table of results from testing across languages. Language Result of 2 x 3 Error --- Node.js v25.3.0 6.000000000000000 0 Python 3.9.6 6.000000000000001 8.88e-16 PHP 8.5.1 6.000000000000001 8.88e-16 Go 1.26.2 6.000000000000000 0 Rust 6.…

I assume these results are all amd64. Is this the same on arm64?

The original article [^1] didn't specify the CPU, but I gathered the code examples (and added one for Lua, untested) if anyone is curious to try with arm64. The test case to compare the margin of error was eml_mul(2, 3).

JavaScript

    const eml = (x, y) => Math.exp(x) - Math.log(y);
    const emlLn = (x) => eml(1, eml(eml(1, x), 1));
    const emlMul = (x, y) => eml(emlLn(x) + emlLn(y), 1);
    const emlAdd = (x, y) => emlLn(eml(x, 1) * eml(y, 1));
Python

    import math

    def eml(x, y):
        return math.exp(x) - math.log(y)
    def eml_ln(x):
        return eml(1, eml(eml(1, x), 1))
    def eml_mul(x, y):
        return eml(eml_ln(x) + eml_ln(y), 1)
    def eml_add(x, y):
        return eml_ln(eml(x, 1) * eml(y, 1))
PHP

    function eml(float $x, float $y): float {
        return exp($x) - log($y);
    }
    function eml_ln(float $x): float {
        return eml(1, eml(eml(1, $x), 1));
    }
    function eml_mul(float $x, float $y): float {
        return eml(eml_ln($x) + eml_ln($y), 1);
    }
    function eml_add(float $x, float $y): float {
        return eml_ln(eml($x, 1) * eml($y, 1));
    }
Go

    func eml(x, y float64) float64 {
        return math.Exp(x) - math.Log(y)
    }
    func emlLn(x float64) float64 {
        return eml(1, eml(eml(1, x), 1))
    }
    func emlMul(x, y float64) float64 {
        return eml(emlLn(x)+emlLn(y), 1)
    }
    func emlAdd(x, y float64) float64 {
        return emlLn(eml(x, 1) * eml(y, 1))
    }
Rust

    fn eml(x: f64, y: f64) -> f64 {
        x.exp() - y.ln()
    }   
    fn eml_ln(x: f64) -> f64 {
        eml(1.0, eml(eml(1.0, x), 1.0))
    }
    fn eml_mul(x: f64, y: f64) -> f64 {
        eml(eml_ln(x) + eml_ln(y), 1.0)
    }
    fn eml_add(x: f64, y: f64) -> f64 {
        eml_ln(eml(x, 1.0) * eml(y, 1.0))
    }
Lua

    local math = require("math")

    function eml(x, y)
        return math.exp(x) - math.log(y)
    end
    function eml_ln(x)
        return eml(1, eml(eml(1, x), 1))
    end
    function eml_mul(x, y)
        return eml(eml_ln(x) + eml_ln(y), 1)
    end
    function eml_add(x, y)
        return eml_ln(eml(x, 1) * eml(y, 1))
    end
---

[^1]: https://lilting.ch/en/articles/eml-single-operator-elementar...

Re: Log is non-monotonic in PHP and Lua

#38
post #28

I've tested this with both LuaJIT and Lua 5.5. It appears to only affect LuaJIT. Worth mentioning in the article I'd say.

That's interesting! On my PC it reproduces under Lua 5.5 (`log_a x > log_b x`), but not under LuaJIT (`log_a x = log_b x`). I took a look at LuaJIT's implementation (https://github.com/LuaJIT/LuaJIT/blob/faaf663340347a78b22ed9...) and noticed that it always uses the `ln x / ln a` formula -- or, rather, `log_2 x / log_2 a`, which is just as correct I guess. Have you perhaps misinterpreted the results? (I do think it's valuable to add that this doesn't apply to LuaJIT though.)

Re: Log is non-monotonic in PHP and Lua

#39

Earlier quoted context omitted.

> because we now know the exact precision requirement for those functions over all 64-bit doubles Do you have a link or something on this?

I'm on a mobile right now so I can't give the exact paper link but you can learn about the CORE-MATH project [1]. [1] https://core-math.gitlabpages.inria.fr/

> CORE-MATH Mission: provide on-the-shelf high performance open-source mathematical functions with correct rounding that can be integrated into current mathematical libraries (GNU libc, Intel Math Library, AMD Libm, Newlib, OpenLibm, Musl, Apple Libm, llvm-libc, Microsoft libm, CUDA libm, ROCm)

That's beautiful. Here's the Git repo with implementation.

https://gitlab.inria.fr/core-math/core-math/-/blob/master/RE...

Re: Log is non-monotonic in PHP and Lua

#40

> Everyone already knows floating-point operations are imprecise and it wouldn’t be fun to blog about. Wellll... Yes and no. And I want to nitpick "FP ops are imprecise" because it's important sometimes. It does come up in Lua - Lua uses 64-bit double floats for everything, _even array indexing_, because they have 53 bits of mantissa and they're guaranteed to represent all 32-bit integers with 100% precision. I just…

> The usual "0.1 + 0.2 != 0.3" is _not_ imprecision. Correct. That’s using the wrong base. Decimal floating point doesn’t have that problem.

On the other hand, decimal floating point would fail with 1/3 + 1/3 != 2/3.

The general rule is that, in base b, you have finite positional ("decimal") representations of numbers p/q where q is a divisor of bˆn for some n. So e.g. 1/10 doesn't have a finite binary representation because 10 = 2 * 5, and that factor 5 ensures 10 is never a divisor of a power of 2. Likewise, 3 is coprime with 10, so you can't represent 1/3 in decimal.

Post reply on HN