Live data from Hacker News

The Array Cast – A podcast about the array programming languages

arraycast.com

51–60 of 141 posts

Re: The Array Cast – A podcast about the array programming languages

#51
post #27

Earlier quoted context omitted.

Chrome now provides on-device powered live captions (which hooks into any chrome originating audio) - chrome://settings/accessibility -> toggle "Live Captions"[1] which could help alleviate some of the limitations for audio impaired viewers 1: https://support.google.com/chrome/answer/10538231?hl=en

> on-device powered live captions I hate this. What were they thinking about? Why not a damn text file that people can grep?

probably because that's a very niche usecase and most people just want some video captions :)

(don't get me wrong, what you describe would be cool and useful! but i can't imagine a lot of people would use it)

Re: The Array Cast – A podcast about the array programming languages

#52

Earlier quoted context omitted.

What the actual f? It's like someone threw up the noise that modems make during initial connection onto an electric typewriter from the 1960s, and then explained their intention using quotes from a Lovecraft novel.

I bet Chinese and Japanese look like that to someone who knows only English too.

As some one who only knows English. No that is not what Chinese and Japanese look like to me.

That's why I prefer APL and its special symbols. It's still utterly inscrutable if you don't know it but at it looks intentional. Those symbols shift your mindset or reframe what you're looking at.

Re: The Array Cast – A podcast about the array programming languages

#53
post #41

Earlier quoted context omitted.

Unfortunately, I have no idea about the implementation and what optimizations are done in J or APL for which operations :( I know that there are hardcoded "fast path" expressions (particular combinations of operations) which have much better performance than more general expressions doing the same thing, so it might be that the optimization happens at that level. OTOH, your example is very verbose when compared to J'…

My example is verbose in order to clearly communicate the principle. I can trivially shorten it to [1, 2, 3].map(a => a * 10) if I wanted to literally carry out that task alone. [[0, 1, 2], [3, 4, 5], [6, 7, 8]].flatMap(a => a * 10)

I think this is pseudo-code? `flatMap` and `=>` for lambdas look like Scala, but there are no array literals using `[]` there. Assuming you mean Scala-like semantics, your second example wouldn't work at all:

    scala> Array(Array(1,2),
                 Array(4,5)).flatMap(a => a * 10)
                                                       ^
           error: value * is not a member of Array[Int]
You would need to write it like this:

    scala> Array(Array(1,2),
                 Array(4,5)).flatMap(a => a.map(x => x * 10))
But then you'd get a flattened array of ints, not array of arrays of ints:

    res6: Array[Int] = Array(10, 20, 40, 50)
So to get the same result as

    (i. 2 2) * 10
You'd need to write:

    scala> Array(Array(1,2),
                 Array(4,5)).map(a => a.map(x => x * 10))
    res7: Array[Array[Int]] = Array(Array(10, 20), Array(40, 50))
...which is more verbose, no? :) EDIT: obviously, I mean "map in map" part, not the Array initialization!

The problem with list comprehensions and `map`, `filter` and friends is that they work very well for flat lists, or lists of lists in some specific circumstances (ie. when you can use `flatMap`). 2D arrays in the general case, and arrays with higher dimensions are really hard to work with using these primitives, unless you have some clever overloads, like what Clojure does. I think Haskell also has a solution for this, but I don't know it enough to comment further, unfortunately :)

Re: The Array Cast – A podcast about the array programming languages

#54
post #44
post #27

Earlier quoted context omitted.

Chrome now provides on-device powered live captions (which hooks into any chrome originating audio) - chrome://settings/accessibility -> toggle "Live Captions"[1] which could help alleviate some of the limitations for audio impaired viewers 1: https://support.google.com/chrome/answer/10538231?hl=en

Samsung's bastard version of Android had a similar "Automated Subtitles" feature. It's decent for watching videos with the phone on silent, but it's pretty crap when there are lots of proper nouns and unusual jargon, as I imagine this podcast has.

So does stock Android, at least the second (?) latest version. (I can never keep track, but I think my phone was eol'ed before the latest version ...)

Re: The Array Cast – A podcast about the array programming languages

#55

I had thought of APL as something from computing pre-history, with its bizarro custom keyboard, but I learned that APL and other array languages are apparently alive and well. Will subscribe to the podcast. Two quotes the hosts brought up stuck with me: (at 15:05) "A language that doesn't change the way you think is not a language worth learning". From Alan Perlis [1], and his Epigrams in Programming (#19) [2] (at 16…

k (and the closely related q) is the main language used in industry, particularly at investment banks and hedge funds. It can be a bit of a shock to realise there are people in London earning in excess of £1000/day (pretty good for London) working in a language where well-written code looks like this[1]: us:{$[#i:&{(y~*K)&"*"~\*x}':x;@[x;i;:[;,"_"]];x]} It's like discovering a whole different world of software develo…

What part of finance uses this language? Is it in widespread use, or is like Goldman Sachs' proprietary language (I forget the name)

Re: The Array Cast – A podcast about the array programming languages

#56

Earlier quoted context omitted.

k (and the closely related q) is the main language used in industry, particularly at investment banks and hedge funds. It can be a bit of a shock to realise there are people in London earning in excess of £1000/day (pretty good for London) working in a language where well-written code looks like this[1]: us:{$[#i:&{(y~*K)&"*"~\*x}':x;@[x;i;:[;,"_"]];x]} It's like discovering a whole different world of software develo…

What part of finance uses this language? Is it in widespread use, or is like Goldman Sachs' proprietary language (I forget the name)

Various parts, particularly in markets, from pricing quants to high-frequency traders. It is fairly widespread. Barclays, JPM, UBS, Morgan Stanley, HSBC are some of the big names, then you have loads of smaller firms.

Re: The Array Cast – A podcast about the array programming languages

#57
post #41

Earlier quoted context omitted.

My example is verbose in order to clearly communicate the principle. I can trivially shorten it to [1, 2, 3].map(a => a * 10) if I wanted to literally carry out that task alone. [[0, 1, 2], [3, 4, 5], [6, 7, 8]].flatMap(a => a * 10)

I think this is pseudo-code? `flatMap` and `=>` for lambdas look like Scala, but there are no array literals using `[]` there. Assuming you mean Scala-like semantics, your second example wouldn't work at all: scala> Array(Array(1,2), Array(4,5)).flatMap(a => a * 10) ^ error: value * is not a member of Array[Int] You would need to write it like this: scala> Array(Array(1,2), Array(4,5)).flatMap(a => a.map(x => x * 10)…

What I can't understand is what would J do if I tell it to sum these two arrays:

    1 2 3 4 5
    6 7 8 
I.e. 5 elements and 3 elements

Re: The Array Cast – A podcast about the array programming languages

#58

Earlier quoted context omitted.

k (and the closely related q) is the main language used in industry, particularly at investment banks and hedge funds. It can be a bit of a shock to realise there are people in London earning in excess of £1000/day (pretty good for London) working in a language where well-written code looks like this[1]: us:{$[#i:&{(y~*K)&"*"~\*x}':x;@[x;i;:[;,"_"]];x]} It's like discovering a whole different world of software develo…

What the actual f? It's like someone threw up the noise that modems make during initial connection onto an electric typewriter from the 1960s, and then explained their intention using quotes from a Lovecraft novel.

I am going to tell you something fantastic, but first, I want to explain some things about this:

    us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]}
The first is that there's a typo in what bidirectional wrote. The above is correct. The second, is what it is. Once I have explained that, I can tell you the fantastic thing.

k syntax is very simple. There's just a few forms you need to be aware of:

    f x
which applies x to f.

    a f b
which is apply f to the two arguments a and b, and:

    f[a;b;c]
which allows you to do three arguments. You can write the first one as f[x] and the second as f[a;b] if you like even more consistency. f can be an "operator" -- that is a symbol. The symbols ' / and \ are special and called adverbs. These adverbs have a special form if followed by a colon, so ': is different than ' and has nothing to do with : or '. I think Arthur just ran out of keys on the keyboard. Once you have those, parenthesis () and braces {} have some special syntax, just like double-quotes " do.

With the syntax explained, let us try to understand what we are looking at.

us: is how we start assignment. You can say "us gets" if you like (the colon can be pronounced). {} braces surround a lambda, this one takes a single argument "x" (the first argument). $[a;b;c] is cond like in lisp; if a then b else c. # means count. i: is another assignment.

& means where -- the argument to which is going to be a bitmap like 000100b or 01101b or something like that, and where returns the indices of the set bits; the former example being the list 3, the latter example being the three-element list 1 2 4.

Another lambda comes next: We can see it takes two arguments because there's an x and a y in there (y is the second argument). We can get a clue as to what it expects because the following adverb ': means each-prior. This tells us "x" is going to be a list of things, and this lambda is going to consume them pairwise. If given the list {(x;y)}':"iliketacos" we get the result:

     {(x;y)}':"iliketacos"
    i 
    li
    il
    ki
    ek
    te
    at
    ca
    oc
    so
y is the "previous" value, and "x" is the current value. The "where" before it tells us we want to know the indices where the condition inside is true. Let's try and understand that condition.

y~*K is in parenthesis. Parenthesis group, so we execute them first (just like in other languages). We're looking for a situation where the previous value is the first (that's what asterisk means here) of K. What is K?

    K:("select";"distinct";"partition";"from";"where";"group";"having";"order";"limit")
So we're looking for a value (x) whose previous (y) is the first of K which is "select". The "&" that follows here is "and" - Arthur likes to overload operators since there aren't many symbols on the keyboard and this is something you get used to.

So you can read {(y~*K)&"*"~*x}':x as simply trying to find the sequences "select star" -- given a list ("select"; "*"; "from"; "potato") you get 0100b and from ("select"; "*"; "from"; "("; "select"; "*"; "from"; "potato; ")") you get 01000100b. I think the attempt is to disambiguate the asterisks in the sql:

    select * from tacos where cat=4*42
but sql is a strange and irregular language, so this kind of thing is necessary. Back to our query:

    us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]}
i is going to be the locations of the asterisks following select. If the count of that is nonzero; we're going to do the @-part, and if not, we're just going to return x.

    @[x;i;f]
is called amend. It returns x, but at indices i, we apply them to f, so it's x[i]:f[x[i]] which is pretty cool. f in this case is a projection, of "gets" (the function colon) with the second-argument bound to an underscore. That is:

    :[;,"_"]
is just a function. That's how @[x;i;:[;,"_"] replaces all of the asterisks that follow select with an a "_"

Almost. It's actually a list of length one, rather than the scalar "_". I haven't read everything in sql.k but this is probably important elsewhere.

Ok. Now that I have explained what this is and what it does, I am ready to tell you something fantastic. I read this:

    us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]}
as:

"us gets a function, that finds the indices of asterisk following the first element of K, and then replaces the things at those indices with underscores"

Literally. From left, to right. Just that fast. And I only program in k part-time. That's not the fantastic thing. The fantastic thing is that by learning to read k, I am almost miraculously able to read other languages faster. This:

    copied=False
    for i in range(1,len(a)):
      if a[i] == "*" and a[i-1] == "select":
        if not copied:
          copied = True
          a = a[:]
        a[i] = "_";
gives me some grief for being so irregular and gross, and I have to look up range/xrange and len and memorise a much more complex set of rules for syntax, and I have to track the order of things carefully and so on, but I have places in my brain, made by k, for those things, and so I am able to absorb code in other languages faster.

If that does not amaze you, I do not think you have considered the ramifications of what I said. I can suggest maybe reading it again (or maybe actually reading what I wrote instead of skipping to the punchline), but if after two or three tries you are still lost, maybe you can ask a question and I can try to answer it.

Re: The Array Cast – A podcast about the array programming languages

#59
post #57

Earlier quoted context omitted.

I think this is pseudo-code? `flatMap` and `=>` for lambdas look like Scala, but there are no array literals using `[]` there. Assuming you mean Scala-like semantics, your second example wouldn't work at all: scala> Array(Array(1,2), Array(4,5)).flatMap(a => a * 10) ^ error: value * is not a member of Array[Int] You would need to write it like this: scala> Array(Array(1,2), Array(4,5)).flatMap(a => a.map(x => x * 10)…

What I can't understand is what would J do if I tell it to sum these two arrays: 1 2 3 4 5 6 7 8 I.e. 5 elements and 3 elements

       1 2 3 + 4 5 6 7
    |length error
    |   1 2 3    +4 5 6 7
In general - depends on the operation. Sometimes the verb checks that both operands have the same rank and dimensions, sometimes the shorter/smaller side gets applied column-wise (or cell-wise/page-wise):

       (2 2 $ 1 2 3 4)  NB. $ means "reshape"
    1 2
    3 4
       (2 2 $ 1 2 3 4) + 1 2
    2 3
    5 6
Sometimes the shorter side is repeated as much as needed to get the correct length, and sometimes the shorter side gets extended with 0s, and sometimes the longer side gets truncated:

       (2 2 $ 1 2 3 4 5 6)
    1 2
    3 4
       (2 4 $ 1 2 3 4 5 6)
    1 2 3 4
    5 6 1 2
 
To be perfectly honest: it's very unintuitive to me and I have to check the docs pretty often to see how the given verb behaves outside of the simplest case (ie. when the rank and dimensions match). But, I learned J as a hobby and never invested enough time into it to really learn all the built-ins, nor did I try using J outside of toy examples, so maybe this behavior becomes convenient once you internalized it?

EDIT: forgot to mention, you can also control the effective rank of the verb, like this:

       
the `<` verb means "box", without the `"n` part you'd get the last value, which is a 1-dimensional array of boxes of length 1 (with 2x2 array inside the box). By selecting rank of the verb, you can decide on which level the verb should be applied: with `"0` it's applied to the smallest possible chunks of the array (cells - you get 2x2 array of boxes), with `"1` you apply the verb to bigger chunks (rows), and so on, for input arrays of any dimensions (so if you have a 3x2x2 array, `"2` will apply the verb to the 3 2x2 arrays).

Re: The Array Cast – A podcast about the array programming languages

#60
post #50
post #17

Earlier quoted context omitted.

Fyi if you weren't aware ... most podcasts don't have text of the audio because high-quality (accurate) transcription of podcasts costs money . Example rates: https://www.google.com/search?q=podcast+transcription+servic... So this thread's podcast of 52 minutes of a complex technical topic with multiple speakers could cost ~$200. A programming-related podcast is already a niche topic with a tiny audience and an Array…

I make transcripts of all my work using Descript. It uses Google's speech-to-text algo (same as the one in youtube presumably) and gives you a transcript you can then edit. It costs $15/month I believe, and you have to spend some time editing the transcript that realistically won't be read by many, but it works pretty well ime (no affiliation besides being a happy customer)

Thanks for bringing Descript to my attention. Do you use any of the production aspects of it?
Post reply on HN