Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All elementary functions from a single binary operator

arxiv.org

Post reply on HN
Print view View original

241–250 of 317 posts

Re: All elementary functions from a single binary operator

#241

Post by tgtweak »

tripletao wrote: post #233
tgtweak wrote: post #219

Earlier quoted context omitted.

For basic arithmetic, this is not required nor would it be faster, as it is not likely advantageous for bulk static transcendal functions. Where this becomes interesting is when combining them OR when chaining them where today they must come back out to the main process for reconfiguration and then re-issued. Practical terms: Jacobian (heavily used in weather and combustion simulation): The transcendental calls, most…

> The core of this is the Arrhenius rate, (A × T^n × exp(-E_a/(R×T))), which involves an exponentiation, a division, a multiplication, and an exponential. On a GPU, that's multiple SFU calls chained with ALU ops. In an EML tree, the whole expression compiles to a single tree that flows through the pipeline in one pass. I think you're missing the reason why the GPU kicks you out of the fast path when you need that spe…

There is a huge market for "its faster" at the cost of efficiency, but I don't think your claim that an EML hardware block would be inherently less inefficient than the same workload running on a GPU. If you think it would be, back it up with some numbers.

A 10-stage EML pipeline would be about the size of an avx-512 instruction block on a modern CPU, in the realm of ~0.1mm2 on a 5nm process node (collectively including the FMA units behind it), at it's entirety about 1% of the CPU die. None of this suggests that even a ~500 wide 10-stage EML pipeline would be consuming anywhere near the power of a modern datacenter GPU (which wastes a lot of it's energy moving things from memory to ALU to shader core...).

Not sure if you're arguing from a hypothetical position or practical one but you seem to be narrowing your argument to "well for simple math it's less efficient" but that's not the argument being made at all.

Re: All elementary functions from a single binary operator

#243

Post by dang »

DoctorOetker wrote: post #99
dang wrote: post #77

Earlier quoted context omitted.

Thanks! that's what it looked like to me too.

I agree: I don't understand what happened, but the first "View PDF" resulted in a PDF where the hyperlinks to the figures didn't work. Upon closer inspection it wasn't v1 at all, thats a PNAS article. I am unable to remove the EDIT:... line in my original comment at this time...

No worries! We definitely want the best link so such comments are helpful.

Re: All elementary functions from a single binary operator

#244

Post by DoctorOetker »

jmyeet wrote: post #190
DoctorOetker wrote: post #30

EDIT: please change the article link to the most recent version (as of now still v2), it is currently pointing to the v1 version which misses the figures. I'm still reading this, but if this checks out, this is one of the most significant discoveries in years. Why use splines or polynomials or haphazardly chosen basis functions if you can just fit (gradient descent) your data or wave functions to the proper computati…

This isn't all that significant to anyone who has done Calculus 2 and knows about Taylor's Series. All this really says is that the Taylor's expansions of e^x and ln x are sufficient to express to express trig functions, which is trivially true from Euler's formula as long as you're in the complex domain. Arithmetic operations follow from the fact that e^x and ln x are inverses, in particular that e^ln(x) = x. Taylor…

There is a huge number of people who understand Taylor series, know how to compute them, and the things you can do with Taylor (and other kinds of) expansions. Yet none of us identified that this binary operation spans that lot of them, but I'm willing to read references to predating observations of the same kind. The author does mention alternative systems (incomplete in some specific sense) in the paper.

I did however keep thinking there was a lot of attention to trying to include special constants even though we don't know that much about these constants yet, while comparatively little attention went to say elliptic integrals etc.

When aiming for a wide catch, you'd include some of those esoteric functions, or erf() etc...

I also wished they had attempted to find a representation for derivative and integrals.

Re: All elementary functions from a single binary operator

#245

Post by xpe »

noobermin wrote: post #232
xpe wrote: post #188

Earlier quoted context omitted.

A quick meta-take here: it is hard to assess the level of expertise here on HN. Some might be just tangentially interested, other might have degrees in the specific topic. Others might maintain a scientific computing library. Domains vary too: embedded systems, robotics, spacecraft navigation, materials modeling, or physics simulation. Until/unless people step up and fill the gaps somehow, we have little notion of id…

I do computational electromagnetism, specifically plasma simulation. In the field solver and particle pushers (I mainly do explicit codes meaning we just approximate the derivatives numerically) we only do taylor expansion so that the derivatives are essentially second order accurate. We don't bother going further, although I can, because in my domain, being more "accurate" as a function of step size (dx in approxima…

Thanks for sharing that! Carry on :)

Re: All elementary functions from a single binary operator

#246

Post by lemonwaterlime »

SideQuark wrote: post #230

This isn't unique, or even the least compute way to do this. For example, let f(x,y) = 1/(x-y). This too is universal. I think there's a theorem stating for any finite set of binary operators there is a single one replacing it. write x#y for 1/(x-y). x#0 = 1/(x-0) = 1/x, so you get reciprocals. Then (x#y)#0 = 1/((1/(x-y)) - 0) = x-y, so subtraction. it's common problem to show in any (insert various algebraic structu…

I’m fairly certain that the difference between the approaches is that the f(x,y) function you mentioned requires limits to represent certain concepts while the eml approach is essentially a tree or a chain of computations meant to represent a model of a system.

Re: All elementary functions from a single binary operator

#247

Post by DoctorOetker »

thaumasiotes wrote: post #131
adrian_b wrote: post #123

Earlier quoted context omitted.

I think that you may have replied before I saved my entire response, so I am not sure how much of it you had read before replying yourself. I have replied to your last statement: > "you can use the second parameter of a binary function to identify a unary function just as you can use the fourth parameter of a quaternary function to identify a trinary one." As I have explained above, what you propose does not work. It…

> As I have explained above, what you propose does not work. It works in functions with 3 or more parameters, but it does not work in binary functions, because you cannot make binary functions from unary functions (without using some auxiliary binary functions). I have no idea what you're trying to say. If you can use one parameter to identify a desired function, then obviously you can use a function of arity n+1 to…

he is saying that if you reserve the second argument of a binary operator as a "function selection indicator", that you have restricted yourself to an alphabet of unary functions. This means that you could at most effectively model some unary functions, but not arbitrary expression involving +,x,-,/, ^, etc.

Unless you had hit upon a very magical binary function where certain special values of the second parameter happens to coincide with useful unary functions, without those values trampling on a useful binary mode or region of your binary function, but the search space for such a special binary function is so large that you shouldn't demand us to disprove the existence, but rather employ your non-surprisal at the EML result and challenge you to present such a binary function, so we can challenge you to demonstrate how it captures binary functions like addition,products, exponentiation with arbitrary base etc.

So, can we see your construction, or if you refuse to present one, we may conclude you have implicitly reconsidered your position and understand the theoretical elegance this EML (and presumably many other) basis brings?

Re: All elementary functions from a single binary operator

#248

Post by tripletao »

tgtweak wrote: post #241
tripletao wrote: post #233

Earlier quoted context omitted.

> The core of this is the Arrhenius rate, (A × T^n × exp(-E_a/(R×T))), which involves an exponentiation, a division, a multiplication, and an exponential. On a GPU, that's multiple SFU calls chained with ALU ops. In an EML tree, the whole expression compiles to a single tree that flows through the pipeline in one pass. I think you're missing the reason why the GPU kicks you out of the fast path when you need that spe…

There is a huge market for "its faster" at the cost of efficiency, but I don't think your claim that an EML hardware block would be inherently less inefficient than the same workload running on a GPU. If you think it would be, back it up with some numbers. A 10-stage EML pipeline would be about the size of an avx-512 instruction block on a modern CPU, in the realm of ~0.1mm2 on a 5nm process node (collectively includ…

> you seem to be narrowing your argument to "well for simple math it's less efficient" but that's not the argument being made at all.

What? Unless the thing you want to compute happens to be exactly that eml() function (no multiplication, no addition, no subtraction unless it's an exponential minus a log, etc.) or almost so, it is unquestionably less efficient. If you believe otherwise, then please provide the eml() implementation of a practically useful function of your choice (e.g. that Arrhenius rate). Then we can count the superfluous transcendental function evaluations vs. a conventional implementation, and try to understand what benefit could outweigh them.

> A 10-stage EML pipeline would be about the size of an avx-512 instruction block on a modern CPU

Can you explain where you got that conclusion? And what do you think a "10-stage EML pipeline" would be useful for? Remember that the multiply embedded in your Arrhenius rate is already 8 layers and 12 operations.

Also, can you confirm whether you're working with an LLM here? You're making a lot of unsupported and oddly specific claims that don't make sense to me, and I'm trying to understand where they're coming from.

Re: All elementary functions from a single binary operator

#250

Post by pama »

karpathy wrote: post #207

All possible 36 distinct level-2 eml functions of one variable (the first 18 of them with entirely Real outputs, the other 18 with "intermediate" complex-valued components): https://imgur.com/a/K7AoOFi

It would be fun to catalogue all one-variable functions that can be represented as binary trees of fixed depth in this way and then encode the trees in binary. Lots of these functions in old math book tables would look very different with a plain hash lookup and the identities in such books would prove themselves.
Post reply on HN