The Array Cast – A podcast about the array programming languages • Hacker News Board

Re: The Array Cast – A podcast about the array programming languages

#111

Post by klibertp » Fri, May 21, 2021, 10:31 AM UTC

jiggawatts wrote: post #105

rscho wrote: post #101

Earlier quoted context omitted.

Because you're wrong. Do spend six months doing hobby array programming. Your view will change.

It's on my bucket list. But it seems so niche that I suspect I'll never use it. My only motivation is that I've been tempted to develop a toy programming language myself that is as high-level as Rust or C++ but explicitly designed for "wide" processing platforms such as GPUs or many-core processors with SIMD instruction sets. All I'm saying is that the concept of array programming -- like pure functional programming…

I just can't understand how even experienced programmers can fixate on syntax. In my - repeated tens of times by this point - experience the syntax is important for a few months (tops!) at the beginning of using a language, and then stops to matter almost completely. What experiences would make someone convinced otherwise? There's an argument against too high complexity of syntax, but most general-purpose languages out there are very similar in this regard...

Re: The Array Cast – A podcast about the array programming languages

#112

Post by the_only_law » Fri, May 21, 2021, 10:40 AM UTC

bidirectional wrote: post #24

an1sotropy wrote: post #22

I had thought of APL as something from computing pre-history, with its bizarro custom keyboard, but I learned that APL and other array languages are apparently alive and well. Will subscribe to the podcast. Two quotes the hosts brought up stuck with me: (at 15:05) "A language that doesn't change the way you think is not a language worth learning". From Alan Perlis [1], and his Epigrams in Programming (#19) [2] (at 16…

k (and the closely related q) is the main language used in industry, particularly at investment banks and hedge funds. It can be a bit of a shock to realise there are people in London earning in excess of £1000/day (pretty good for London) working in a language where well-written code looks like this[1]: us:{$[#i:&{(y~*K)&"*"~\*x}':x;@[x;i;:[;,"_"]];x]} It's like discovering a whole different world of software develo…

I recall a discussion on array programming languages on here a while back where someone claimed that an acquaintance of theirs was earning nearly 7 figures working on q/kdb+.

Re: The Array Cast – A podcast about the array programming languages

#113

Post by rak1507 » Fri, May 21, 2021, 10:45 AM UTC

Avshalom wrote: post #97

geocar wrote: post #86

Earlier quoted context omitted.

If this were me, and I needed a function like us, I would have written this: us:{$[x~(z;y);,"_";y]}[(*K;,"*")]': I would be interested in seeing anything that was shorter[1] and faster than that in any language , and I would be very curious to learn from anyone who could also do that faster than me. But I'm not a fetishist: I didn't learn k because it was cute, and I don't wake up every day looking for ways to rewrit…

okay so it's not really smaller because it uses UTF8 also it relies on order of operations to get rid of the parens around ⍵≡'*' but us←{('_'@(1+⍸2{(⍺≡⊃K)∧⍵≡'*'}/⍵))⍵} is one(1) character shorter.

⍵≡'*' should be ⍵≡,'*' like in the original k

us←⊃,⍥⊆2{(⍺≡⊃K)∧⍵≡,'*':,'_'⋄⍵}/⊢ is shorter anyway

Re: The Array Cast – A podcast about the array programming languages

#114

Post by rak1507 » Fri, May 21, 2021, 11:32 AM UTC

geocar wrote: post #86

kazinator wrote: post #82

Earlier quoted context omitted.

Let's do this using the same approach, "find indices and assign over them in a copy of the sequence": This is the TXR Lisp interactive listener of TXR 259. Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet. Do not operate heavy equipment or motor vehicles while using TXR. 1> (defun subst-select-* (list) (let ((indices (where (op starts-with '(select *)) (cons nil (conses list))))) (if indices (let (…

If this were me, and I needed a function like us, I would have written this: us:{$[x~(z;y);,"_";y]}[(*K;,"*")]': I would be interested in seeing anything that was shorter[1] and faster than that in any language , and I would be very curious to learn from anyone who could also do that faster than me. But I'm not a fetishist: I didn't learn k because it was cute, and I don't wake up every day looking for ways to rewrit…

Why do you pass in (*K;,"*") as x instead of hard-coding it in x~(z;y)? Clarity?

Re: The Array Cast – A podcast about the array programming languages

#115

Post by jiggawatts » Fri, May 21, 2021, 11:59 AM UTC

geocar wrote: post #104

jiggawatts wrote: post #100

Earlier quoted context omitted.

To me, this is infinitely more readable: let mut input = ["select", "*", "from", "potato"]; for i in 1..input.len() { if ["select","*"] == input[i-1..=i] { input[i] = "_"; } } If I could be bothered to dig up a Haskell compiler, I'm sure it's possible to do a one-liner list comprehension that is both terse and readable. If I was doing this in Rust, it's easy to create an iterator extension that does something like "m…

> To me, this is infinitely more readable: Yes, to you. Just like Chinese would be infinitely less readable to you, if you don't know Chinese. Do you think this is a deep observation? The real challenge is knowing whether it is worth it to learn k so that it becomes readable. - How many characters is it? This is a useful metric when you realise bug/defect rate is proportional to the physical size (in rows and columns…

> 126msec for 100k cycles.

Or to put it another way: 1,200 nanoseconds. That's about 3,000-5,000 instructions on a modern CPU. Believe it or not, that's actually pretty bad.

After jumping through some hoops to ensure that rustc doesn't just compile the whole thing down to a constant, I benchmarked my version as taking 15-20 nanoseconds per iteration. About 45-80 instructions!

I actually couldn't quite believe it myself, so I jumped through more hoops to ensure that it wasn't being optimised away, wasn't getting inlined too aggressively, etc... No change.

Ran it through Godbolt to inspect the assembly, and then I realised that, yes, modern languages, compilers, and CPUs really are this good!

Think about it: For the specific input example with 4 strings the algorithm boils down to: compare 7 bytes with 7 bytes, replace a pointer with another pointer, then compare 2 bytes with 2 bytes three times. That's about a hundred assembly instructions, or thereabouts.

Godbolt output:

    push   rbp
    push   r15
    push   r14
    push   r13
    push   r12
    push   rbx
    push   rax
    mov    r12,rdi
    mov    ebx,0x8
    xor    ebp,ebp
    lea    r14,[rip+0x3cb3c]        # 444e8 
    mov    r15,QWORD PTR [rip+0x3f3b5]        # 46d68 
    cs nop WORD PTR [rax+rax*1+0x0]
    nop    DWORD PTR [rax]
    cmp    rbp,0x2
    je     79f2 
    mov    r13,rbp
    mov    rdx,QWORD PTR [rbx+r14*1]
    cmp    rdx,QWORD PTR [r12+rbx*1]
    jne    79ec 
    lea    rbp,[r13+0x1]
    mov    rsi,QWORD PTR [r12+rbx*1-0x8]
    mov    rdi,QWORD PTR [rbx+r14*1-0x8]
    call   r15
    add    rbx,0x10
    test   eax,eax
    je     79c0 
    cmp    r13,0x2
    jb     7a07 
    lea    rax,[rip+0x3060e]        # 38007 
    mov    QWORD PTR [r12+0x10],rax
    mov    QWORD PTR [r12+0x18],0x1
    xor    ebx,ebx
    xor    ebp,ebp
    nop    DWORD PTR [rax+rax*1+0x0]
    cmp    rbp,0x2
    je     7a43 
    mov    r13,rbp
    mov    rdx,QWORD PTR [rbx+r14*1+0x8]
    cmp    rdx,QWORD PTR [r12+rbx*1+0x18]
    jne    7a3d 
    lea    rbp,[r13+0x1]
    mov    rsi,QWORD PTR [r12+rbx*1+0x10]
    mov    rdi,QWORD PTR [rbx+r14*1]
    call   r15
    add    rbx,0x10
    test   eax,eax
    je     7a10 
    cmp    r13,0x2
    jb     7a58 
    lea    rax,[rip+0x305bd]        # 38007 
    mov    QWORD PTR [r12+0x20],rax
    mov    QWORD PTR [r12+0x28],0x1
    mov    ebx,0x8
    xor    ebp,ebp
    nop
    cmp    rbp,0x2
    je     7a93 
    mov    r13,rbp
    mov    rdx,QWORD PTR [rbx+r14*1]
    cmp    rdx,QWORD PTR [r12+rbx*1+0x20]
    jne    7a8d 
    lea    rbp,[r13+0x1]
    mov    rsi,QWORD PTR [r12+rbx*1+0x18]
    mov    rdi,QWORD PTR [rbx+r14*1-0x8]
    call   r15
    add    rbx,0x10
    test   eax,eax
    je     7a60 
    cmp    r13,0x2
    jb     7aa8 
    lea    rax,[rip+0x3056d]        # 38007 
    mov    QWORD PTR [r12+0x30],rax
    mov    QWORD PTR [r12+0x38],0x1
    add    rsp,0x8
    pop    rbx
    pop    r12
    pop    r13
    pop    r14
    pop    r15
    pop    rbp
    ret    
    nop    WORD PTR [rax+rax*1+0x0]

Rust? It simply impresses.

Re: The Array Cast – A podcast about the array programming languages

#116

Post by jiggawatts » Fri, May 21, 2021, 12:10 PM UTC

klibertp wrote: post #111

jiggawatts wrote: post #105

Earlier quoted context omitted.

It's on my bucket list. But it seems so niche that I suspect I'll never use it. My only motivation is that I've been tempted to develop a toy programming language myself that is as high-level as Rust or C++ but explicitly designed for "wide" processing platforms such as GPUs or many-core processors with SIMD instruction sets. All I'm saying is that the concept of array programming -- like pure functional programming…

I just can't understand how even experienced programmers can fixate on syntax. In my - repeated tens of times by this point - experience the syntax is important for a few months (tops!) at the beginning of using a language, and then stops to matter almost completely. What experiences would make someone convinced otherwise? There's an argument against too high complexity of syntax, but most general-purpose languages o…

There's empirical evidence, loads of it, that this simply isn't true.

Syntax absolutely does matter, in all walks of life, not just programming.

We're not infinite, error-free computers like some sort of mathematical abstraction. We have squishy meat brains evolved to tell stories around a campfire.

As a random example: I learned Rust just for fun, and something that repeatedly caused hours of frustration is that unlike every other modern language, it allows identifier shadowing. This compiles and runs just fine:

    let a = 5;
    let a = "wat?"

Practically no other language allows this, because it is a recipe for errors.

Internally, compilers typically use single static assignment, so the above would be processed something like this:

    let a_1 = 5;
    let a_2 = "wat"

For a compiler to track this is no problem. For a human? It's a problem. Maybe not for trivial examples like this, but if dozens of identifiers are littered across a huge function it can be a challenge to keep track of which one is which if the names are the same but they're... not the same. Especially if the change is subtle.

We're not computers, that's why we make them out of silicon and metal and sell them for money.

Re: The Array Cast – A podcast about the array programming languages

#117

Post by smir » Fri, May 21, 2021, 12:25 PM UTC

specialist wrote: post #69

geocar wrote: post #58

Earlier quoted context omitted.

I am going to tell you something fantastic, but first, I want to explain some things about this: us:{$[#i:&{(y~*K)&"*"~*x}':x;@[x;i;:[;,"_"]];x]} The first is that there's a typo in what bidirectional wrote. The above is correct. The second, is what it is. Once I have explained that, I can tell you the fantastic thing. k syntax is very simple. There's just a few forms you need to be aware of: f x which applies x to f…

Wow. That's some explanation. Thanks. Do you use "rainbow brackets"? This example is first hit I found: https://kristofferc.github.io/OhMyREPL.jl/latest/features/ra... Such an obvious idea once you see it. Wish I had syntax coloring and rainbow brackets when I coded LISP for hire.

In J language single sided brackets are functions/verbs

Re: The Array Cast – A podcast about the array programming languages

#118

Post by jiggawatts » Fri, May 21, 2021, 12:30 PM UTC

rscho wrote: post #107

jiggawatts wrote: post #105

Earlier quoted context omitted.

It's on my bucket list. But it seems so niche that I suspect I'll never use it. My only motivation is that I've been tempted to develop a toy programming language myself that is as high-level as Rust or C++ but explicitly designed for "wide" processing platforms such as GPUs or many-core processors with SIMD instruction sets. All I'm saying is that the concept of array programming -- like pure functional programming…

https://futhark-lang.org/ Is the closest you'll get to that, I think. It took a whole team of big brains to make it, and the author stated it was highly non-trivial to make something like that.

Interesting, but not quite what I had in mind.

My idea was loosely based on a code search engine Google had for a while before they killed it off.

I'm not sure exactly how they did it, but I very strongly suspected that they implemented regular expressions over database indexes.

A simple b-tree index is just a set of sorted strings. If you have a lot of sorted strings and you squint at it, you can see how it picks out common prefixes, like a tree. (You can literally store it with the prefixes factored out, and then you have a trie.)

For a fairly wide range of regular expressions, and a tree of sorted strings, you can implement a search over the b-tree. E.g.: the search "[bx](a+)foo" can be implemented via finding the range starting with "b", then the range starting with "x". For each range, find the sub-range starting with "ba", "xa", etc...

Each step takes logarithmic time, and can be done in parallel. In theory, it takes only about a hundred times longer to find a ten matches in a million strings than it would take to match a single string.

Wouldn't it be nice if you could take an algorithm such as "match regex" that normally takes a 'scalar' string and have the compiler automatically create a version that can take an array of sorted values, like the database index?

The idea is that much like how array programming languages eschew scalars and pass arrays all over the place to gain efficiency, my language would pass sets all over the place, and gain a different type of efficiency.

Re: The Array Cast – A podcast about the array programming languages

#119

Post by rscho » Fri, May 21, 2021, 12:43 PM UTC

jiggawatts wrote: post #116

klibertp wrote: post #111

Earlier quoted context omitted.

I just can't understand how even experienced programmers can fixate on syntax. In my - repeated tens of times by this point - experience the syntax is important for a few months (tops!) at the beginning of using a language, and then stops to matter almost completely. What experiences would make someone convinced otherwise? There's an argument against too high complexity of syntax, but most general-purpose languages o…

There's empirical evidence, loads of it, that this simply isn't true. Syntax absolutely does matter, in all walks of life, not just programming. We're not infinite, error-free computers like some sort of mathematical abstraction. We have squishy meat brains evolved to tell stories around a campfire. As a random example: I learned Rust just for fun, and something that repeatedly caused hours of frustration is that unl…

Syntax can certainly be more or less suitable to a particular problem or even to humans in general, but (dis)allowing identifier shadowing is part of the semantics of the language (i.e. what the syntax means). One could say: "there should be syntactic cues if we are to allow variable shadowing". That would be a well-defined criticism of syntactic issues. BTW, most modern languages do allow identifier shadowing. The particularity of Rust and the object of your criticism is allowing it in the same scope as the definition.

But again, the only way you'll be able to apprehend array language notation is by writing enough of it to become somewhat fluent. It doesn't mean you have to like it, but remember it was the object of a Turing award and that's usually a good indicator of something relevant.

Re: The Array Cast – A podcast about the array programming languages

#120

Post by klibertp » Fri, May 21, 2021, 12:58 PM UTC

jiggawatts wrote: post #116

klibertp wrote: post #111

Earlier quoted context omitted.

I just can't understand how even experienced programmers can fixate on syntax. In my - repeated tens of times by this point - experience the syntax is important for a few months (tops!) at the beginning of using a language, and then stops to matter almost completely. What experiences would make someone convinced otherwise? There's an argument against too high complexity of syntax, but most general-purpose languages o…

There's empirical evidence, loads of it, that this simply isn't true. Syntax absolutely does matter, in all walks of life, not just programming. We're not infinite, error-free computers like some sort of mathematical abstraction. We have squishy meat brains evolved to tell stories around a campfire. As a random example: I learned Rust just for fun, and something that repeatedly caused hours of frustration is that unl…

> There's empirical evidence, loads of it, that this simply isn't true.

Source?

> unlike every other modern language, it allows identifier shadowing

1. All dynamically typed languages allow this, all/most REPLs even for statically typed languages allow this, even if regular code doesn't.

2. This is not syntax, at all, this is semantics of immutable value declaration+initialization.

> but if dozens of identifiers are littered across a huge function it can be a challenge to keep track of which one is which

Littering dozens of identifiers across huge functions is going to be a problem, no matter the syntax. It's a programmer's job to manage complexity by utilizing various kinds of techniques, including syntactic sugar (true), but also factoring the code into manageable chunks, using higher-level or better suited for the task at hand abstractions, and so on. Syntax on its own is, in my experience, the least impactful technique for managing complexity, at least before going into DSL-land (but then it's syntax+semantics).

Another thing worth mentioning is that lots of general-purpose languages give you exactly the same constructs, just spelled differently. Simple examples:

    for (x : list) { ... }
    foreach (x; list) { ... }
    for x


All the above denote exactly the same construct, with very little variation in the number of tokens. Can you tell me if you think one of them is better than the others - and if so, why?


    
      Permalink
      View on HN
      Reply on HN
      Top