Live data from Hacker News

The Array Cast – A podcast about the array programming languages

arraycast.com

131–140 of 141 posts

Re: The Array Cast – A podcast about the array programming languages

#131
post #123

Earlier quoted context omitted.

> 126msec for 100k cycles. Or to put it another way: 1,200 nanoseconds. That's about 3,000-5,000 instructions on a modern CPU. Believe it or not, that's actually pretty bad . After jumping through some hoops to ensure that rustc doesn't just compile the whole thing down to a constant, I benchmarked my version as taking 15-20 nanoseconds per iteration. About 45-80 instructions! I actually couldn't quite believe it mys…

I'm not sure getting the wrong answer fast is something to be proud of. Your rust code has a bug in it, is longer, and you spent more time on it.

Please elaborate on that bug. The rust function I provided replaces "select, *" correctly with "select, _". Is that not what it was supposed to do? I checked that it works with empty arrays, arrays with one entry, etc...

Thinking about it, this version boils down the logic to its barest essence:

    fn test(input: &mut [&str]) {
    
        let mut was_select = false;
        for i in input.iter_mut() {
            if was_select && *i == "*" { *i = "_"; was_select = false; }
            else { was_select = *i == "select"; }
        }
    }
It runs in 5.5 nanoseconds per iteration, which is just absurdly fast. That's about 200x faster than your K version!

What I like about this Rust version is that it reflects the approach that for a computer processor is optimal, yet it is high-level and readable. I could convert that "test" function easily enough to a generic "replace" function similar to a string search & replace, but one that can operate on any mutable iterator with the maximum possible efficiency, not just arrays.

Then, your K code that requires "careful reading" to slowly tease apart the meaning could be converted to a form that is practically prose:

    let mut input = ["select", "*", "bar", "potato"];
    replace( &mut input, &["select", "*"], &["select", "_"] );*

Re: The Array Cast – A podcast about the array programming languages

#132
post #123

Earlier quoted context omitted.

I'm not sure getting the wrong answer fast is something to be proud of. Your rust code has a bug in it, is longer, and you spent more time on it.

Please elaborate on that bug. The rust function I provided replaces "select, *" correctly with "select, _". Is that not what it was supposed to do? I checked that it works with empty arrays, arrays with one entry, etc... Thinking about it, this version boils down the logic to its barest essence: fn test(input: &mut [&str]) { let mut was_select = false; for i in input.iter_mut() { if was_select && *i == "*" { *i = "_"…

> The rust function I provided replaces "select, *" correctly with "select, _". Is that not what it was supposed to do?

No. It is not supposed to modify its input but return a copy.

Re: The Array Cast – A podcast about the array programming languages

#133
post #86

Earlier quoted context omitted.

If this were me, and I needed a function like us, I would have written this: us:{$[x~(z;y);,"_";y]}[(*K;,"*")]': I would be interested in seeing anything that was shorter[1] and faster than that in any language , and I would be very curious to learn from anyone who could also do that faster than me. But I'm not a fetishist: I didn't learn k because it was cute, and I don't wake up every day looking for ways to rewrit…

okay so it's not really smaller because it uses UTF8 also it relies on order of operations to get rid of the parens around ⍵≡'*' but us←{('_'@(1+⍸2{(⍺≡⊃K)∧⍵≡'*'}/⍵))⍵} is one(1) character shorter.

Is this the same language or a different one?

Re: The Array Cast – A podcast about the array programming languages

#134
post #132

Earlier quoted context omitted.

Please elaborate on that bug. The rust function I provided replaces "select, *" correctly with "select, _". Is that not what it was supposed to do? I checked that it works with empty arrays, arrays with one entry, etc... Thinking about it, this version boils down the logic to its barest essence: fn test(input: &mut [&str]) { let mut was_select = false; for i in input.iter_mut() { if was_select && *i == "*" { *i = "_"…

> The rust function I provided replaces "select, *" correctly with "select, _". Is that not what it was supposed to do? No. It is not supposed to modify its input but return a copy.

Granted, if replacing arbitrary sequences with arbitrary sequences, then copying is necessary, as this could result in the output length increasing.

However, for replacing scalars only, the in-place mutable version is in some sense superior: it doesn't force a memory allocation. Moreover it can be trivially converted into a copying version by simply wrapping it in a function that first copies the input, and then mutates it in-place. The reverse is not true: the copying version cannot be wrapped to create a non-allocating version.

To be honest, I wish more languages put this kind of effort into their standard libraries, but most have about 10% of what I would like to see. For example, this kind of "string matching" is really "sequence matching" and ought to be fully generic for any underlying comparable and copyable type, not just arrays of characters. String search algorithms like Boyer-Moore ought to be directly applicable to arrays of integers or enums, e.g.: for recognition of code patterns in lists of parsed tokens.

At least one person has done something like this for Rust: https://github.com/peterjoel/rust-iter-replace/blob/6c575eeb...

Similarly, there's the new InPlaceIterable, which is interesting but not quite enough to suit my taste: https://doc.rust-lang.org/std/iter/trait.InPlaceIterable.htm...

Re: The Array Cast – A podcast about the array programming languages

#135
post #132

Earlier quoted context omitted.

> The rust function I provided replaces "select, *" correctly with "select, _". Is that not what it was supposed to do? No. It is not supposed to modify its input but return a copy.

Granted, if replacing arbitrary sequences with arbitrary sequences, then copying is necessary, as this could result in the output length increasing. However, for replacing scalars only, the in-place mutable version is in some sense superior: it doesn't force a memory allocation. Moreover it can be trivially converted into a copying version by simply wrapping it in a function that first copies the input, and then muta…

> However, for replacing scalars only, the in-place mutable version is in some sense superior

I mean, we are talking about someone else's code and someone else's decisions. If we can change the rules, you're absolutely right we can do much much better.

But beware microbenchmarking too much: Giving the treatment you gave your rust to your entire program can be more than exhausting, it can actually end you up with a slower program simply because your program gets too big!

k makes a lot of compromises to stay small enough to keep both the interpreter and the application in L1, but whole-program speeds benefit from this treatment sometimes by factors of 1000x or more, and that's hard to show with these microbenchmarks as well.

> To be honest, I wish more languages put this kind of effort into their standard libraries, but most have about 10% of what I would like to see. For example, this kind of "string matching" is really "sequence matching" and ought to be fully generic for any underlying comparable and copyable type, not just arrays of characters

In APL, this is called ⍷ (pronounced "find") and sometimes even APL-ers momentarily forget it exists[1], but in k you always have to make it yourself, usually (as we did today) with ⍸ (where) and ≡ (match), but sometimes some other way[2], and this works on all the different data types k supports (including integers, enums, dates, times, symbols) and across multiple cores as well. You are right to predict it would be useful: It is useful :)

[1]: https://news.ycombinator.com/item?id=27229994

[2]: https://news.ycombinator.com/item?id=16851862

Re: The Array Cast – A podcast about the array programming languages

#136

Earlier quoted context omitted.

okay so it's not really smaller because it uses UTF8 also it relies on order of operations to get rid of the parens around ⍵≡'*' but us←{('_'@(1+⍸2{(⍺≡⊃K)∧⍵≡'*'}/⍵))⍵} is one(1) character shorter.

Is this the same language or a different one?

This is APL, k is a descendant of APL.

Re: The Array Cast – A podcast about the array programming languages

#137
post #60
post #50

Earlier quoted context omitted.

I make transcripts of all my work using Descript. It uses Google's speech-to-text algo (same as the one in youtube presumably) and gives you a transcript you can then edit. It costs $15/month I believe, and you have to spend some time editing the transcript that realistically won't be read by many, but it works pretty well ime (no affiliation besides being a happy customer)

Thanks for bringing Descript to my attention. Do you use any of the production aspects of it?

Yeah, it works really well, it's basically completely replaced what I used to use Audacity and Premiere for.

Re: The Array Cast – A podcast about the array programming languages

#138
post #77
post #63

Earlier quoted context omitted.

Mostly yes. Numbers (and character) are implemented as arrays with 0 dimensions. Text would be an array of characters with 1 dimension (the number of characters), and generally speaking the dimension of an array is a one dimensional list of non-negative integers. Many array languages also include an array type which is approximately the same as a C pointer to an array, with a bit of jargon thrown in to distinguish a…

From the J docs: https://www.jsoftware.com/help/dictionary/dx005.htm nub=: (i.@# = i.~) # ] 5!:2 ... so J functions have an array representation, at least.

Yes, each J function has several array representations.

Re: The Array Cast – A podcast about the array programming languages

#139
post #39
post #37

Earlier quoted context omitted.

Most compilers can inline statically defined closures in these contexts. And tracing JITs do this even when the closure is not define statically (but is stable). It's more about allowing the SIMD goodness without the ambiguity and restrictions of "scalar operators work on arrays" implemented naively.

Traditional array languages (APL,J) are not amenable to static compilation due to other dynamic issues, though. I think Dyalog APL is experimenting with a bytecode interpreter, but no JITs in sight either (that I know of). The very dynamic aspect of those languages makes that difficult. I'd love a compileable similar language, though. To replace R/Python for statistics that are sooooo annoying when exploring data due…

That's not really the issue.

It's certainly true that, depending on how the code is written, you may not be able to optimize out the language implementation from the compiled program. But that just turns into a link against a library with support for the language for your hypothetical compiled program.

That said, the compiler being hypothetical is a very real obstacle. But even there the problem is not that the language can't be compiled -- it's that no one has bothered to implement a compiler for it.

Anyways, if you throw in some ML type inference, and some tools for characterizing the resulting code and its interfaces, you can generate code from an array language which is quite similar to the code you would get from a variety of other languages.

Re: The Array Cast – A podcast about the array programming languages

#140
post #57

Earlier quoted context omitted.

I think this is pseudo-code? `flatMap` and `=>` for lambdas look like Scala, but there are no array literals using `[]` there. Assuming you mean Scala-like semantics, your second example wouldn't work at all: scala> Array(Array(1,2), Array(4,5)).flatMap(a => a * 10) ^ error: value * is not a member of Array[Int] You would need to write it like this: scala> Array(Array(1,2), Array(4,5)).flatMap(a => a.map(x => x * 10)…

What I can't understand is what would J do if I tell it to sum these two arrays: 1 2 3 4 5 6 7 8 I.e. 5 elements and 3 elements

Like was mentioned, the simplest behavior is treat this as an error case.

Other possibilities include:

(*) extending the length of the short array to match the length of the long array -- padding with zeros on the right -- before adding.

(*) finding all possible sums between pairs with one number from one array and the other number from the other array.

(*) cyclically extending the shorter array (probably not useful in this example, but since the example didn't come with a use case that's difficult to know for sure).

(*) Treating each array as a decimal number (or a number in some other base)

(*) combining the two array as a single list of numbers and summing everything to a single value

and... so on...

One point being that programming languages support arbitrarily complex operations.

Another point being that "sum" in this context carries some understandable ambiguity -- the sort of thing which we usually try to constrain with use cases or examples or similar elaboration.

Post reply on HN