Live data from Hacker News

All elementary functions from a single binary operator

arxiv.org

301–310 of 317 posts

Re: All elementary functions from a single binary operator

#301
This article inspired us to test whether EML trees can be trained using gradient descent for symbolic regression: instead of searching for the right tree, is it possible to optimize one from start to finish?

We implemented the EML trees as a differentiable module of PyTorch. Each leaf is a softmax mixture over {0, 1, x}, and the tree is evaluated from the bottom up. The entire system can be trained with Adam.

Results: 7 of the 7 elementary functions (exp, ln, sqrt, x², x³, 1/x, sin) converge with ≤24 parameters at depth 3. Three (exp, ln, sqrt) achieve an RMSE The main challenge is depth scaling. Random initialization at depth 4 always diverges: the exp() strings create towering exponential growth that cancels out the gradients. We tested 12 initialization strategies; only hierarchical hot-starting (training depth n-1 first) works. sin(x²) gets 12.9x better at depth 4 vs depth 3.

Two honest negative results: (1) The trained trees use continuous softmax mixtures, not discrete leaf assignments; therefore, numerical approximations, not exact formulas, are obtained for everything except exp(x). (2) A 49-parameter MLP and PySR outperform it in MSE by orders of magnitude. It's not a practical tool — it just shows that gradient descent can work on S → 1 | eml(S, S) without needing sin, exp, etc. as primitives.

Paper: https://doi.org/10.5281/zenodo.19592926 Code: https://github.com/seetrex-ai/monolith

Re: All elementary functions from a single binary operator

#302
post #57

This is neat, but could someone explain the significance or practical (or even theoretical) utility of it?

second, please help us laypeople here

It's potentially useful for computer algebra with complex numbers - we might be able to simplify formulas using non-standard methods, but instead via pattern matching. We might use this to represent exact numbers internally, and only produce an inexact result when we later reduce the expression.

Consider it a bit like a "church encoding" for complex numbers. I'll try to demonstrate with an S-expression representation.

---

A small primer if you're not familiar. S-expressions are basically atoms (symbols/numbers etc), pairs, or null.

    S = 
      | 
      | (S . S)      ;; aka pair
      | ()           ;; aka null
    
There's some syntax sugar for right chains of pairs to form lists:

    (a b c)          == (a . (b . (c . ()))   ;; a proper list
    (a b . c)        == (a . (b . c))         ;; an improper list
    (#0=(a b c) #0#) == ((a b c) (a b c))     ;; a list with a repeated sublist using a reference
---

So, we have a function `eml(x, y) and a constant `1`. `x` and `y` are symbols.

Lets say we're going to replace `eml` with an infix operator `.`, and replace the unit 1 with `()`.

    C = 
      | 
      | (C . C)      ;; eml
      | ()           ;; 1
We have basically the same context-free structure - we can encode complex numbers as lists. Let's define ourselves a couple of symbols for use in the examples:

    ($define x (string->symbol "x"))
    ($define y (string->symbol "y"))
And now we can define the `eml` function as an alias for `cons`.

    ($define! eml cons)

    (eml x y)
    ;; Output: (x . y)
We can now write a bunch of functions which construct trees, representing the operations they perform. We use only `eml` or previously defined functions to construct each tree:

    ;; e^x

        ($define! exp     ($lambda (x) (eml x ())))
        
        (exp x)
        ;; Output: (x)
        ;; Note: (x) is syntax sugar for (x . ())

    ;; Euler's number `e`

        ($define! c:e     (exp ()))
        
        c:e          
        ;; Output: (())
        ;; Note: (()) is syntax sugar for (() . ())

    ;; exp(1) - ln(x)

        ($define! e1ml    ($lambda (x) (eml () x)))
        
        (e1ml x) 
        ;; Output: (() . x)

    ;; ln(x)

        ($define! ln      ($lambda (x) (e1ml (exp (e1ml x)))))
        
        (ln x)
        ;; Output: (() (() . x))

    ;; Zero

        ($define! c:0      (ln ()))
        
        c:0
        ;; Output: (() (()))

    ;; -infinity

        ($define! c:-inf   (ln 0))
        
        c:-inf
        ;; Output: (() (() () (())))

    ;; -x
        
        ($define! neg      ($lambda (x) (eml c:-inf (exp x))))

        (neg x)
        ;; Output: ((() (() () (()))) x)
        
    ;; +infinity
    
        ($define! c:+inf   (neg c:-inf))
        
        c:+inf
        ;; Output: (#0=(() (() () (()))) #0#)
        
    ;; 1/x
    
        ($define! recip    ($lambda (x) (exp (eml c:-inf x))))
        
        (recip x)
        ;; Output: (((() (() () (()))) . x))
  
    ;; x - y
    
        ($define! sub      ($lambda (x y) (eml (ln x) (exp y))))
    
        (sub x y)
        ;; Output: ((() (() . x)) y)
    
    ;; x + y
    
        ($define! add      ($lambda (x y) (sub x (neg y))))
    
        (add x y)
        ;; Output: ((() (() . x)) ((() (() () (()))) y))
    
    ;; x * y
    
        ($define! mul      ($lambda (x y) (exp (add (ln x) (exp (neg y))))))
        
        (mul x y)
        ;; Output: (((() (() () (() . x))) (#0=(() (() () (()))) ((#0# y)))))
        
    ;; x / y
    
        ($define! div      ($lambda (x y) (exp (sub (ln x) (ln y)))))
        
        (div x y)
        ;; Output: (((() (() () (() . x))) (() (() . y))))
        
    ;; x^y
    
        ($define! pow      ($lambda (x y) (exp (mul x (ln y)))))
  
        (pow x y)
        ;; Output: ((((() (() () (() . x))) (#0=(() (() () (()))) ((#0# (() (() . y))))))))
  
I'll stop there, but we continue for implementing all the trig, pi, etc using the same approach.

So basically, we have a way of constructing trees based on `eml`

Next, we pattern match. For example, to pattern match over addition, extract the `x` and `y` values, we can use:

    ($define! perform-addition
        ($lambda (add-expr)
            ($let ((((() (() . x)) ((() (() () (()))) y)) add-expr))
                (+ x y))))  

    ;; Note, + is provided by the language to perform addition of complex numbers

    (perform-addition (add 256 512))
    ;; Output: 768
So we didn't need to actually compute any `exp(x)` or `ln(y)` to perform this addition - we just needed to pattern match over the tree, which in this case the language does for us via deconstructing `$let`.

We can simplify the defintion of perform-addition by expanding the parameters of a call to `add` as the arguments to the function:

    ($define! $let-lambda
        ($vau (expr . body) env
            ($let ((params (eval expr env)))
                (wrap (eval (list* $vau (list params) #ignore body) env)))))
                
    ($define! perform-addition
        ($let-lambda (add x y)
            (+ x y)))

    ($define! perform-subtraction
        ($let-lambda (sub x y)
            (- x y)))


    ($define! sub-expr (sub 256 512))
    ;; Output: #inert
    sub-expr
    ;; Output: ((() (() . 256)) 512)

    (perform-subtraction sub-expr)
    ;; Output: -256

There's a bit more work involved for a full pattern matcher which will take some arbitrary `expr` and perform the relevant computation. I'm still working on that.

Examples are in the Kernel programming language, tested using klisp[1]

[1]:https://github.com/dbohdan/klisp

Re: All elementary functions from a single binary operator

#303

Earlier quoted context omitted.

I think this is the novel bit: > This includes constants such as e, pi, and i ; arithmetic operations including addition, subtraction, multiplication, division, and exponentiation as well as the usual transcendental and algebraic functions .

And those come from the infinite series needed to compute exp and ln. They’re just as much work either way. The exp and ln way are vastly costlier for every op, including simply adding 1 and 2.

You are taking `exp` and `ln` to require infinite series, because you aren't taking his operator as primitive.

Re: All elementary functions from a single binary operator

#304

This isn't unique, or even the least compute way to do this. For example, let f(x,y) = 1/(x-y). This too is universal. I think there's a theorem stating for any finite set of binary operators there is a single one replacing it. write x#y for 1/(x-y). x#0 = 1/(x-0) = 1/x, so you get reciprocals. Then (x#y)#0 = 1/((1/(x-y)) - 0) = x-y, so subtraction. it's common problem to show in any (insert various algebraic structu…

1/(x-y) has nothing remotely like the power of his operator. The guy is not stupid.

Re: All elementary functions from a single binary operator

#305

This isn't unique, or even the least compute way to do this. For example, let f(x,y) = 1/(x-y). This too is universal. I think there's a theorem stating for any finite set of binary operators there is a single one replacing it. write x#y for 1/(x-y). x#0 = 1/(x-0) = 1/x, so you get reciprocals. Then (x#y)#0 = 1/((1/(x-y)) - 0) = x-y, so subtraction. it's common problem to show in any (insert various algebraic structu…

1/(x-y) has nothing remotely like the power of his operator. The guy is not stupid.

It generates the same class of functions. Read the comments and links in this thread.

Re: All elementary functions from a single binary operator

#306
post #296

Earlier quoted context omitted.

His paper misses infinitely many such functions.

Oh, glad you are still here. Because I kept wondering about 1/(x-y), and came to the conclusion it actually cannot do nearly as much as eml. So maybe you could confirm if I understood your assumptions correctly and help me to sort it out overall. In your original post you were kinda hand-wavy about what we have except for x # y := 1/(x-y), but your examples make it clear you also assume 0 exists. Then it's pretty obv…

All of these are standard fare in abstract algebra classes, and I didn’t care to write it all out. Once you have the “inverse” operations - and reciprocal, the entire structure follows, for a large set of objects, whether N or Q or R or C or finite fields or division rings, and a host of other structures. So I only wrote - and 1/x

Then, subtraction is (x#y)#0 = x-y. Reciprocal is x#0 = 1/x. Addition follows from x+y=x-((x-x)-y). This used the additive identity 0.

Multiplication follows from

x^2= x-1/(1/x + 1/(1-x)), so we can square things. Then -2xy = (x-y)^2 -x^2 - y^2 is constructible. Then we can divide by -2 via x/-2 = 1/((0-1/x)-1/x), and there’s multiplication. In terms of #, this expression only needed the constant 1, which is the multiplicative identity.

Now mult and reciprocal give x * 1/y = x/y, division.

Any nontrivial ring needs additive and multiplicative identities 1!=0, which are the only constants needed above. If you assume this is Q or R or C, it may be possible to derive one from the other, not sure. But if you’re in these fields, you know 0 and 1 exist.

Then any element of Q is a finite set of ops. R can be constructed in whatever way you want: Dedekind cuts, Cauchy sequences, whatever usual constructions. Or assume R exists, and compute in it via the f(x,y).

This also works over finite fields (eml does not), division rings, even infinite fields of positive characteristic, function fields (think elements are ratio of polynomials), basically any algebraic object with the 4 ops.

Re: All elementary functions from a single binary operator

#307

Earlier quoted context omitted.

> no gain other than it’s pretty Conceptual elegance is worth something, isn't it? I don't mean just aesthetic pleasure, as in recreational mathematics, but there's often (not always) value in being able to succinctly describe a wide range of phenomena with a small number of primitives. It could open up new ways of understanding or working that wasn't possible before. Not saying this specific discovery fits the descr…

I feel sad when pragmatics comes in scientific discussions... that's not what science should be (I think). But I value the discussion this paper is bringing to distinct contexts (even outside academia). That by itself adds value to this work.

i am outisde of academia, i am told there are significant findings here at monogate.dev . just started vibe coding from the paper.

appreciate any feedback

Re: All elementary functions from a single binary operator

#309

Earlier quoted context omitted.

ah, the paper acknowledges this. my bad for jumping to the diagrams!

On page 11, the paper explicitly states: > EML-compiled formulas work flawlessly in symbolic Mathematica and IEEE754 floating-point… This is because some formulas internally might rely on the following properties of extended reals: ln 0 = −∞, e^(−∞) = 0. And then follows with: > But EML expressions in general do not work ‘out of the box’ in pure Python/Julia or numerical Mathematica. Thus, the paper’s completeness cl…

I spent a few days playing around with this work in Lean and his central claim is provably wrong..

The main problem is that singularities infect everything, and you can't have an equational theory without rewrite/substitution rules that aren't grounded unless you can decide if an arbitrary EML tree is zero, which is undecidable in elementary functions (because of sine).

Basically it's only valid on a undecideable subset. All of his numerical tests are carefully crafted to avoid singularities which are exactly where it fails, and the singularities are all over the place -- in particular in subtraction (which shouldn't have any!). He wants to sort of compile it to actually computable functions and use those instead, but there is no equational theory possible that you can build with this.

You can't restrict it to the positive reals either or it's not closed, it's trivially easy to get to complex or negative numbers.

Using extended reals doesn't fix it, you just get different undefined terms (-inf + inf = 0).

It's quite pretty! I love the idea or I wouldn't have spent so much time on it. It just doesn't work, and none of his other candidates will work either because they all have ln(0).

The only sense in which it's true which is that it can generate the elementary functions that it can generate, which is just tautological. It can in no sense generate all of them.

Even his verification doesn't work because they set it up to check the narrow bands where it's valid, outside of that it's a mess.

Re: All elementary functions from a single binary operator

#310

Earlier quoted context omitted.

On page 11, the paper explicitly states: > EML-compiled formulas work flawlessly in symbolic Mathematica and IEEE754 floating-point… This is because some formulas internally might rely on the following properties of extended reals: ln 0 = −∞, e^(−∞) = 0. And then follows with: > But EML expressions in general do not work ‘out of the box’ in pure Python/Julia or numerical Mathematica. Thus, the paper’s completeness cl…

The author does address this further on page 14 of SI and provides an alternative of: −z = 1 − (e − ((e − 1) − z))

So here is the problem -- we have two constructions of -z.

Whether or not this shows up in a tree somewhere if you try to compose functions together is probably undecideable.

Either they aren't equal and you've broken any tree that includes that construction of -z anywhere in it, _or_ you have two trees which _are_ equal, but disagree on their value at every point

Any rule that tries to rewrite one form to the other is unsound

The lack of any equational theory makes a lot of claims about it fairly nonsensical.

Post reply on HN