Trap – Transformers in APL
github.com
Trap – Transformers in APL
1–10 of 32 posts
Re: Trap – Transformers in APL
#2> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, the Co-dfns project compiles APL code for CPUs and GPUs, exploiting the data parallel essence of APL to achieve high performance. Second, APL also almost entirely dispenses with the software-specific "noise" that bloats code in other languages, so APL code can be directly mapped to algorithms or mathematical expressions on a blackboard and vice versa, which cannot be said of the majority of programming languages. Finally, APL is extremely terse; its density might be considered a defect by some that renders APL a cryptic write-once, read-never language, but it allows for incredibly concise implementations of most algorithms. Assuming a decent grasp on APL syntax, shorter programs mean less code to maintain, debug, and understand.
This is really cool. At about 150 lines, terse indeed. And it makes sense that of course APL could work well with gpus, but I’m kind of surprised there’s enough of it still out in the wild so that there’s already a reliable tool chain for doing this.
Re: Trap – Transformers in APL
#3> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, th…
> APL code can be directly mapped to algorithms or mathematical expressions on a blackboard and vice versa
After looking at the code, I find this claim questionable.
Re: Trap – Transformers in APL
#4> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, th…
> APL code can be directly mapped to algorithms or mathematical expressions on a blackboard and vice versa After looking at the code, I find this claim questionable.
After looking at HN comments for years, I find this low effort dismissal downvoteable.
APL was originally a rewrite and normalisation of traditional math notation for use on blackboards. Before it was anything to do with computers it was linear algebra without all the bizarre precedence rules and with some common useful operations.
Re: Trap – Transformers in APL
#5> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, th…
> APL code can be directly mapped to algorithms or mathematical expressions on a blackboard and vice versa After looking at the code, I find this claim questionable.
Name checks out.
Re: Trap – Transformers in APL
#6> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, th…
> APL could work well with gpus
I've seen at least an APL implementation running on top of Julia, thanks to macros.
Julia has good GPU support, and it makes it easy to compose that support with any library.
However, kdb+ and q, which are APL descendants, have good GPU support already: https://code.kx.com/q/interfaces/gpus. But licenses are not cheap...
Re: Trap – Transformers in APL
#7> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, th…
> APL code can be directly mapped to algorithms or mathematical expressions on a blackboard and vice versa After looking at the code, I find this claim questionable.
APL was invented by Iverson as a blackboard notation because he felt the existing notation was awkward/insufficent for describing computation/algorithms
Re: Trap – Transformers in APL
#8> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, th…
> APL code can be directly mapped to algorithms or mathematical expressions on a blackboard and vice versa After looking at the code, I find this claim questionable.
[deleted]
Re: Trap – Transformers in APL
#9> Though APL may strike some as a strange language of choice for deep learning
It sure did to me, even as someone who has written (a trivial amount of) J. But the argument that follows is more than convincing.
Re: Trap – Transformers in APL
#10Earlier quoted context omitted.
> APL code can be directly mapped to algorithms or mathematical expressions on a blackboard and vice versa After looking at the code, I find this claim questionable.
APL was invented by Iverson as a blackboard notation because he felt the existing notation was awkward/insufficent for describing computation/algorithms
linear algebra notation is the real notation (on a blackboard). aka the language of AI. the medium is just the message format.