Live data from Hacker News

J Notation as a Tool of Thought

hillelwayne.com

51–60 of 60 posts

Re: J Notation as a Tool of Thought

#51

This reminded me very much of working in R or working with Numpy. I however always thought having an array as the primitive in R was great only because R is focused on statistics and data science. My understanding is that J claims to be general purpose programming and as such I’m surprised the paradigm holds.

If you run the J GUI, written in J, it is going with blow your mind about how general purpose it is...

I wish somebody would write a walkthrough of writing something like a piece of GUI to show that off.

(I keep trying to figure out J and ... sliding off ... every time I need to do something "normal" ... which is annoying and the fact I'm reduced to wishcasting in HN comments as a solution makes me disappointed in both myself and the universe)

Re: J Notation as a Tool of Thought

#52

Earlier quoted context omitted.

> Rank Is actually implicitly done by numpy using a mechanism called broadcasting. Numpy's broadcasting is scalar conformability, to which the rank operator (and general conformability) provides a general case. Example in j: ] x =. i. 2 3 4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 ] y =. i. 3 4 0 1 2 3 4 5 6 7 8 9 10 11 x + y NB. this will error because + expects that, if its arguments' shapes ar…

What is prefix/suffix agreement? Could you give a simple example of why prefix beats suffix? I'm curious and have never heard these terms before, and googling didn't help.

Sibling explains the difference. To explain why prefix is better—first let's look at the base case where argument shapes match. Let's assume that addition is defined on scalars; we don't have to explain that 11=5+6. Then let's consider vectors: there are two forms of scalar broadcasting for vectors. The first is addition of two equal-length vectors, for which each item of the left argument will be broadcast to the corresponding item of the right argument: 10 14=3 5+7 9. The second is the addition of a scalar to a vector (or vice versa), where the scalar is matched up with each item of the vector: 10 14=6 10+4.

For the first case, we can say that:

  R[i] = x[i] + y[i]  (when x and y are vectors)
(Where i is any valid array subscript; and R, x, and y are the names conventionally given to the result, left argument, and right argument of some function, respectively.)

Assume that we extend our scalar rule to arbitrary dimensions (that is, scalar+n-dimensional array will do what we expect)—there is actually a good reason for this, but we'll just assume it for now; it's a pretty intuitive rule.

Then the simple recursive rule I gave above gives you prefix agreement for any two argument shapes; just replace 'vector' with 'nonscalar'. Here are the rules for suffix agreement:

  R[i] = x[i] + y[i]   (when x and y have the same rank)
  R[i] = x    + y[i]   (when y has bigger rank)
  R[i] = x[i] + y      (when x has bigger rank)
The prefix agreement rule is simple: add each element of x to its corresponding element in y. The suffix agreement rule is much more complex (3 rules, as opposed to just 1), and with higher-ranked arrays it gets harder to reason about which elements go together.

There's an even deeper synergy, though, which goes between prefix agreement and forks. Forks are a generalisation of a convention from calculus. There's a convention that, given functions f and g, (f+g) is a legal function such that (f+g)(x) f(x)+g(x). (And ditto for other arithmetic operators.) Apl does not distinguish between user-defined functions (like f) and infix builtins (like +); x f y denotes the calling of function f with arguments x and y, same as x + y denotes the calling of function + with arguments x and y. So the f+g rule is generalised: we say that, given any functions f, g, and h, (f g h) x is equivalent to (f x) g (h x).

What does this have to do with prefix agreement? Well, all we have to do is remember that an array, as a relation of indices to elements, is very close to a function. In fact, I think that it's appropriate to say that an array is semantically a function, though its syntactic role is different. So saying x[i] is like saying 'apply function x to argument i'. Which leads very nicely—right into prefix agreement:

  (f g h) x   (f x)  g (h x)
  . . . 
  (x + y)[i]  (x[i]) + (y[i])

Re: J Notation as a Tool of Thought

#53
post #26

The article has the footnote > APL was the first language to use “monad” as a term. The popular FP meaning only appeared thirty years later. in its, to me odd, use of the word "monad". This is incorrect. The "popular FP meaning" arose because it's a very special case of the standard concept from mathematics [1]. The mathematical terminology harkens back to the 1950s, and thus predates APL. In Haskell, a monad is prec…

I think "the first language to" is the operative bit.

Yes, the mathematics predates it, but in context I think "only appeared" clearly means "only appeared in a programming context".

(I guess whether you accept this depends on how you're rank ordering different types of pedantry, and exercise I am happy to leave to the discretion of the reader)

Re: J Notation as a Tool of Thought

#55
post #47
post #29

Earlier quoted context omitted.

So can you give an example of something that can be expressed simply in APL/J/K that can't be expressed simply with numpy? And to be clear I don't think "less characters" makes the expression more simple. Maybe "less statements" or "less operators"

Here's Conway's Game of Life by Arthur Whitney (creator of K): life:{3=a-x*4=a:2{+(0+':x)+1_x,0}/x} I'm guessing a numpy implementation would be between one and two orders of magnitude more verbose, even if you're just comparing the number of operations.

It doesn't quite seem fair to compare numpy to one of the most famous K code golf's ever created, but here is my attempt, 140 characters compared to Arthur's 30, so 5 times longer. Not worry about padding the edges would cut it to 100. I don't think the gap is as wide as you think it is.

  def life(M):
    MP = np.pad(M,[(1,1),(1,1)])
    N = sum(np.roll(MP,(i,j),(0,1))
             for i in [1,0,-1]
             for j in [1,0,-1])
    return (3==N-MP*(4==N))[1:-1,1:-1]
Speedtest (presumably memory bound):

  100 x 100 matrix, numpy  0.2ms, K  0.2ms
  200 x 200 matrix, numpy  0.5ms, K  0.8ms
  500 x 500 matrix, numpy  5.2ms, K  8.0ms
  1000x1000 matrix, numpy 20.0ms, K 36.0ms
  5000x5000 matrix, numpy  0.5s , K  1.2s
Interestingly the K code is designed to return a boolean matrix, but operates much more slowly for me on a boolean matrix compared to an integer matrix, with the result that:

  q)\t klife M
Takes 1.2 seconds whilst

  q)\t klife klife M
Takes 24 seconds. So whilst the idea seems to be that klife is used with scan/over to run a number of iterations, it's actually a bad idea to run it more than once.

Re: J Notation as a Tool of Thought

#56
post #46

Earlier quoted context omitted.

That is fine. Most people simply cannot drive a Formula-1 car, but it would be stupid to conclude that a F-1 is a cursed car and nobody should want to drive one. Similarly, a few languages can only be used by highly trained people who understand well how it works. Lisp, Forth, and J are in that category of tools that require highly trained people. In their hand, one of these languages can do fantastic things, in the…

That is actually a nice analogy! No one, not even the pro formula-1 drivers will want to drive a f1 car in regular traffic. Much the same with these languages (apls, lisps, forth, etc); they have their place and role, but is better not used in regular open source or commercial codes.

That is absolutely false. I have seen absolutely horrible "clever" code hacks in Java production code - absolutely opaque and extremely complicated.

Lisp is definitely a good choice for 'regular open source or commercial code'.

Re: J Notation as a Tool of Thought

#57

Earlier quoted context omitted.

> there is no word on the algorithm - ' how is the array sorted' is the question that computer science is designed to solve, as opposed to 'what are the fixed points of a sorted array' ' [H]ow is the array sorted' is one question that computer science can be used to solve. However, assuming it has been solved (which, for many cases, it has), a much more interesting question is 'what can we do with a sorted array'; or…

My point was that general programming usually looks more like writing the sorting algorithm than simply calling it. Thus, I would be more interested in seeing real examples of how the properties of J allow you to write algorithms more concisely or otherwise better. The article showed how J solves some mathematical problems with built in functions or their built-in modifiers. But in general programming, you won't find…

> My point was that general programming usually looks more like writing the sorting algorithm than simply calling it. Thus, I would be more interested in seeing real examples of how the properties of J allow you to write algorithms more concisely or otherwise better.

Ah - I see. Here[1] is quicksort in j. John scholes's videos are excellent and very illustrative, though they deal with apl rather than j; conway's game of life[2] and sudoku[3].

> The article showed how J solves some mathematical problems with built in functions or their built-in modifiers. But in general programming, you won't find a function that does exactly what you wanted, maybe with a little adverb. You'll usually have to write that function yourself, to describe how it works in terms of some of those pre-existing functions.

Right. This is true, but you generally stay much closer to the builtins in j than you do in other language.

> Note that your second example could be expressed as something like

> x.OrderBy(y => y[1])

> In C#, assuming x is int[N][M]. I understand that J's version can work with many other shapes of X, but that is hard to appreciate without seeing example of how that comes in handy when writing some other (hopefully well known) algorithm.

Right; you mentioned sort predicates, so I was giving an example of a way in which you could sort according to another relation. My point is: being able to change the layout of your data so easily can obviate sort predicates. (Though I do think a sort adverb taking a predicate would be cool.)

1. https://code.jsoftware.com/wiki/Essays/Quicksort

2. https://www.youtube.com/watch?v=a9xAKttWgP4

3. https://www.youtube.com/watch?v=DmT80OseAGs

Re: J Notation as a Tool of Thought

#58
post #38

Earlier quoted context omitted.

I have only passing experience with numpy, but I suspect the following "shift a matrix in all 8 directions", used in the famous APL/J game of life solution [0][1], would require multiple statements as well as being more verbose in numpy (I'm guessing you'd need to use "roll" multiple times?): (>,{;~i:1) |. i.4 4 Try it online: https://tio.run/##y/r/PzU5I19Bw06n2rou08pQU6FGTyFTz0TB5P9/AA [0] https://www.youtube.com/wa…

This 100 character version is probably the cleanest. M = np.reshape(np.arange(16),(4,4)) np.array([np.roll(M,(i,j),(0,1)) for i in [1,0,-1] for j in [1,0,-1]])

That requires you to break out of the array paradigm and drop back down to procedural looping. Which is what you'd expect since numpy is array operations bolted onto python as a library and J is array operations baked into the language itself.

Re: J Notation as a Tool of Thought

#59

J and K remind me of the essay about the Lisp Curse[0], which mentions that the expressiveness of the language became a sort of Achilles heel in its culture. It talks about people writing their projects in Lisp and not expecting other people to adapt to their conventions or combine their efforts on one library, where every solution worked well enough initially - but only for one person, its author. In K the error mes…

It's about redundancy. Code in other languages has much more of it, and that is good for understandability, recognizability. With less redundancy it is easier to make errors both in writing AND reading of code. APL is like compressed code. The redundancy has been compressed out. Therefore reading a page of APL is like reading a book in other languages. Even though the text is shorter you still have to understand as m…

> With less redundancy it is easier to make errors both in writing AND reading of code.

I don't really see that. Boilerplate code is not helpful for clarity or correctness. It's just more code that you have to write that can introduce errors.

Re: J Notation as a Tool of Thought

#60
post #59

Earlier quoted context omitted.

It's about redundancy. Code in other languages has much more of it, and that is good for understandability, recognizability. With less redundancy it is easier to make errors both in writing AND reading of code. APL is like compressed code. The redundancy has been compressed out. Therefore reading a page of APL is like reading a book in other languages. Even though the text is shorter you still have to understand as m…

> With less redundancy it is easier to make errors both in writing AND reading of code. I don't really see that. Boilerplate code is not helpful for clarity or correctness. It's just more code that you have to write that can introduce errors.

The errors you introduce into boilerplate code are trivial syntax errors easy to spot by the parser. The difficult errors are the semantic ones because the parser can not catch them.

Think about human languages. They have large amounts of redundancy and for a good evolutionary reason I believe. It is easier to distinguish meaning when words around a given word anchor it into a specific context.

Redundant languages have a built-in error-correcting code built into them. They take more bits to transmit but those extra bits correct, and help detect errors.

Post reply on HN