APL and Array Programming
forums.fast.ai
APL and Array Programming
1–10 of 29 posts
Re: APL and Array Programming
#2For anyone interested in array programming, the person who made k eventually created a query language called q:
Now days we refer to tables-within-a-language as "dataframes". These are widely available in R, pandas, Spark, etc.
The q language inspired me to write my own language a few years back that added static typing to dataframes:
Re: APL and Array Programming
#3I recently did a project using Dyalog APL. I really wanted to love it.
On the pro side, the community was great and I loved the language, but only to a point. There are way too many instances of utter astonishment at behaviors that seemed to be just how the language works. I had a lot of trouble finding good, mature libraries for things I’ve taken for granted in other languages.
I’ve found that Julia provides 98% of the Array Programming value in a modern “batteries included” package.
Re: APL and Array Programming
#4For anyone interested in array programming, the person who made k eventually created a query language called q: https://code.kx.com/q/ref/ Now days we refer to tables-within-a-language as "dataframes". These are widely available in R, pandas, Spark, etc. The q language inspired me to write my own language a few years back that added static typing to dataframes: https://www.empirical-soft.com
Your work might predate this, but spark/scala now has good support for typed dataframes (they’re called DataSets).
Re: APL and Array Programming
#5Well this is unexpected - didn't think my little study group would appear on HN! :O Well anyhoo, all are welcome to join; details in the linked thread (including playlist of last week's videos so you can catch up).
As it happens, my interview on The Array Cast was just published today:
https://www.arraycast.com/episodes/episode31-jeremy-howard
In it I discuss my thoughts on array programming, including in PyTorch, numpy, Tensorflow, and other libraries and languages. If you haven't seen it before, The Array Cast is a really great podcast including interviews with some of the most interesting folks in the array programming world. My fave episode so far is the one with Aaron Hsu (@arcfide) of co-dfns fame:
https://www.arraycast.com/episodes/episode19-aaron-hsu
My personal interest in APL is for teaching (my daughter and her friend who I tutor really like learning math via APL, and we've successfully covered territory that previously I'd had no luck making progress on using more traditional methods) and for studying notation (which is what APL was originally designed for).
Re: APL and Array Programming
#6APL seems like a great language for computer graphics! Has anyone tried their hands at the same?
Re: APL and Array Programming
#7For anyone interested in array programming, the person who made k eventually created a query language called q: https://code.kx.com/q/ref/ Now days we refer to tables-within-a-language as "dataframes". These are widely available in R, pandas, Spark, etc. The q language inspired me to write my own language a few years back that added static typing to dataframes: https://www.empirical-soft.com
Very cool, this approach will inevitably catch on, I think.
I built a similar typed PL and compiler for Morgan Stanley back in 2013: https://github.com/morganstanley/hobbes
Re: APL and Array Programming
#8This makes me remember the time when I was still codegolfing in K. It was a breeze from other languages that I have learned and still is.
Re: APL and Array Programming
#9For anyone interested in array programming, the person who made k eventually created a query language called q: https://code.kx.com/q/ref/ Now days we refer to tables-within-a-language as "dataframes". These are widely available in R, pandas, Spark, etc. The q language inspired me to write my own language a few years back that added static typing to dataframes: https://www.empirical-soft.com
Your work might predate this, but spark/scala now has good support for typed dataframes (they’re called DataSets).
Does Spark's Dataset have something like the "inferSchema" option when reading a CSV file? Everything I've read in Spark's documentation makes me think that Datasets require an explicit type provided by the user.
Empirical can infer a CSV's schema at compile time. If the file path can be determined at compile time, then the Empirical compiler will sample the CSV file and determine an appropriate type before the user's code is ever run. No need for an explicit type from the user, and yet we still have static typing.
Re: APL and Array Programming
#10For anyone interested in array programming, the person who made k eventually created a query language called q: https://code.kx.com/q/ref/ Now days we refer to tables-within-a-language as "dataframes". These are widely available in R, pandas, Spark, etc. The q language inspired me to write my own language a few years back that added static typing to dataframes: https://www.empirical-soft.com
Very cool, this approach will inevitably catch on, I think. I built a similar typed PL and compiler for Morgan Stanley back in 2013: https://github.com/morganstanley/hobbes
I read about Hobbes years ago. The embeddability certainly caught my attention.
I wrote Empirical specifically for the use case of wanting to read a CSV file while inferring the type at compile time. Basically, a ton of compile-time function evaluation mixed with type providers.