Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Trap – Transformers in APL

github.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Trap – Transformers in APL

#11

Post by koolala »

nextos wrote: post #6
photonthug wrote: post #2

> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, th…

> APL could work well with gpus I've seen at least an APL implementation running on top of Julia, thanks to macros. Julia has good GPU support, and it makes it easy to compose that support with any library. However, kdb+ and q, which are APL descendants, have good GPU support already: https://code.kx.com/q/interfaces/gpus . But licenses are not cheap...

GPUs can even run APL as a higher level programming language. It's the only abstract language I've ever heard to run on a GPU. Arrays in, Arrays out. A gpu is array programming hardware.

I hope one day its normal like the 1000s of CPU languages. Would be nice to have more than 10 gpu languages.

Re: Trap – Transformers in APL

#12

Post by sakras »

> Though APL may strike some as a strange language of choice for deep learning

I've actually spent the better part of last year wondering why we _haven't_ been using APL for deep learning. And actually I've been wondering why we don't just use APL for everything that operates over arrays, like data lakes and such.

Honestly, APL is probably a good fit for compilers. I seem to remember a guy who had some tree-wrangling APL scheme, and could execute his compiler on a GPU. But I can't find it now.

Re: Trap – Transformers in APL

#13

Post by tlack »

sakras wrote: post #12

> Though APL may strike some as a strange language of choice for deep learning I've actually spent the better part of last year wondering why we _haven't_ been using APL for deep learning. And actually I've been wondering why we don't just use APL for everything that operates over arrays, like data lakes and such. Honestly, APL is probably a good fit for compilers. I seem to remember a guy who had some tree-wrangling…

I believe the tree-wrangler you mentioned is Aaron Hsu, author of the Co-dfns APL "compiler" that Trap uses.

Here are some videos related to his work: https://www.youtube.com/playlist?list=PLDU0iEj6f8duXzmgnlGX4...

Co-dfns was most recently discussed on Hacker News 3 months ago: https://news.ycombinator.com/item?id=40928450

Re: Trap – Transformers in APL

#14

Post by smartmic »

It would be good if the APL dialect in which this is implemented is mentioned on the front page. I implemented some things in GNU APL which is an (almost) complete implementation of ISO standard 13751, based primarily on APL2. More common and modern is the proprietary Dyalog APL, which I assume is used here (and which is also free for personal use).

Re: Trap – Transformers in APL

#15

Post by pjmlp »

koolala wrote: post #11
nextos wrote: post #6

Earlier quoted context omitted.

> APL could work well with gpus I've seen at least an APL implementation running on top of Julia, thanks to macros. Julia has good GPU support, and it makes it easy to compose that support with any library. However, kdb+ and q, which are APL descendants, have good GPU support already: https://code.kx.com/q/interfaces/gpus . But licenses are not cheap...

GPUs can even run APL as a higher level programming language. It's the only abstract language I've ever heard to run on a GPU. Arrays in, Arrays out. A gpu is array programming hardware. I hope one day its normal like the 1000s of CPU languages. Would be nice to have more than 10 gpu languages.

StarLisp on the Connection Machine, which is kind of early attempt to what GPU became.

Futhark is another more recent example.

Re: Trap – Transformers in APL

#16

Post by anonzzzies »

nextos wrote: post #6
photonthug wrote: post #2

> Though APL may strike some as a strange language of choice for deep learning, it offers benefits that are especially suitable for this field: First, the only first-class data type in APL is the multi-dimensional array, which is one of the central object of deep learning in the form of tensors. This also signifies that APL is by nature data parallel and therefore particularly amenable to parallelization. Notably, th…

> APL could work well with gpus I've seen at least an APL implementation running on top of Julia, thanks to macros. Julia has good GPU support, and it makes it easy to compose that support with any library. However, kdb+ and q, which are APL descendants, have good GPU support already: https://code.kx.com/q/interfaces/gpus . But licenses are not cheap...

There is the famous '17 line' compiler here https://scholarworks.iu.edu/dspace/items/3ab772c9-92c9-4f59-...

(I got the t-shirt)

Re: Trap – Transformers in APL

#17

Post by anonzzzies »

There is this on https://shakti.com (the 'new k' from Arthur);

k-torch llm(61) 14M 2 14 6 288 288 x+l7{l8x{x%1+E-x}l6x}rl5x+:l4@,/(hvi,:l3w)Ss@S''h(ki,:ql2w)mql1w:rl0x (18M 2 32000 288)

which apparently can run on the gpu someone told me on discord (but i'm not sure if it's true or not).

Re: Trap – Transformers in APL

#18

Post by skruger »

smartmic wrote: post #14

It would be good if the APL dialect in which this is implemented is mentioned on the front page. I implemented some things in GNU APL which is an (almost) complete implementation of ISO standard 13751, based primarily on APL2. More common and modern is the proprietary Dyalog APL, which I assume is used here (and which is also free for personal use).

It is Dyalog APL.

Re: Trap – Transformers in APL

#19

Post by abrudz »

smartmic wrote: post #14

It would be good if the APL dialect in which this is implemented is mentioned on the front page. I implemented some things in GNU APL which is an (almost) complete implementation of ISO standard 13751, based primarily on APL2. More common and modern is the proprietary Dyalog APL, which I assume is used here (and which is also free for personal use).

It is indeed Dyalog APL (evident by certain features used, and also what Co-dfns requires). And yes, I agree, especially since this uses the ".apl" file extension of GNU APL, rather than the ".apls" that Dyalog uses for shell scripts. Oddly enough, the "⎕IO←0" appears outside the ":Namespace" which means it cannot be used by other APL code.

Re: Trap – Transformers in APL

#20

Post by pistoleer »

anonzzzies wrote: post #16
nextos wrote: post #6

Earlier quoted context omitted.

> APL could work well with gpus I've seen at least an APL implementation running on top of Julia, thanks to macros. Julia has good GPU support, and it makes it easy to compose that support with any library. However, kdb+ and q, which are APL descendants, have good GPU support already: https://code.kx.com/q/interfaces/gpus . But licenses are not cheap...

There is the famous '17 line' compiler here https://scholarworks.iu.edu/dspace/items/3ab772c9-92c9-4f59-... (I got the t-shirt)

That's a phd dissertation with hundreds of pages, wrong link maybe?
Post reply on HN