Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Notation: Ken Iverson Centenary

kx.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: The Notation: Ken Iverson Centenary

#11

Post by sxp »

hpcjoe wrote: post #6
sxp wrote: post #4

> ”K programs routinely outperform hand-coded C. This is theoretically impossible. K compiles into C. Every k program has a C equivalent that runs exactly as fast. Yet it is true in practice. Why? Because it is easier to see your error in four lines of code than in four hundred lines of C.” How true is this? I hear this claim a lot but haven't seen any real benchmarks. E.g, https://benchmarksgame-team.pages.debian.ne…

Arthur's new company, Shakti, has some of the benchmarks they've talked about[1][2]. [1] https://shakti.sh/ [2] https://shakti.sh/benchmark/about?eula=shakti.com/license

That shows kdb is faster than BigQuery, Spark & other similar systems. But it doesn't compare K to C.

Re: The Notation: Ken Iverson Centenary

#12

Post by sxp »

kick wrote: post #8
sxp wrote: post #4

> ”K programs routinely outperform hand-coded C. This is theoretically impossible. K compiles into C. Every k program has a C equivalent that runs exactly as fast. Yet it is true in practice. Why? Because it is easier to see your error in four lines of code than in four hundred lines of C.” How true is this? I hear this claim a lot but haven't seen any real benchmarks. E.g, https://benchmarksgame-team.pages.debian.ne…

He wrote solutions for Shootout (predecessor to that Debian page) problems a few years ago; a lack of k in benchmarks is not due to a lack of k being volunteered: http://www.kparc.com/z/comp.k k is really fast. Half of the things on the Shakti mailing list are just Arthur getting really excited about how significantly he's beating x or y or z in performance and giving numbers for it. `grep`ping it now I see 40 in hal…

Thanks for that link. Do you know if the results are posted anywhere?

https://shakti.com/license.php says "Customer shall not... distribute ... any report regarding the performance of the Software benchmarks..." which I would need to agree to if I want to download the binaries at https://shakti.sh/benchmark/

Re: The Notation: Ken Iverson Centenary

#13

Post by anonu »

kick wrote: post #8
sxp wrote: post #4

> ”K programs routinely outperform hand-coded C. This is theoretically impossible. K compiles into C. Every k program has a C equivalent that runs exactly as fast. Yet it is true in practice. Why? Because it is easier to see your error in four lines of code than in four hundred lines of C.” How true is this? I hear this claim a lot but haven't seen any real benchmarks. E.g, https://benchmarksgame-team.pages.debian.ne…

He wrote solutions for Shootout (predecessor to that Debian page) problems a few years ago; a lack of k in benchmarks is not due to a lack of k being volunteered: http://www.kparc.com/z/comp.k k is really fast. Half of the things on the Shakti mailing list are just Arthur getting really excited about how significantly he's beating x or y or z in performance and giving numbers for it. `grep`ping it now I see 40 in hal…

How has your experience been with J for on disk time series DBs.

Re: The Notation: Ken Iverson Centenary

#14

Post by 7thaccount »

sxp wrote: post #4

> ”K programs routinely outperform hand-coded C. This is theoretically impossible. K compiles into C. Every k program has a C equivalent that runs exactly as fast. Yet it is true in practice. Why? Because it is easier to see your error in four lines of code than in four hundred lines of C.” How true is this? I hear this claim a lot but haven't seen any real benchmarks. E.g, https://benchmarksgame-team.pages.debian.ne…

I think you're missing the point. K certainly isn't faster than C. If you have a C guru, they should always be able to write faster code than K.

The difference is it should be a lot easier for someone like me (not a C guru) to write a K program that does something useful. I'll also be able to see the whole program as it will likely be a few lines long. If I wrote some C code, it would take me forever and take forever to debug. I see this as similar to any scripting language (Ex: Python).

Re: The Notation: Ken Iverson Centenary

#15

Post by jodrellblank »

haolez wrote: post #10
jodrellblank wrote: post #3

> " Iverson was more interested in how quickly a person could understand an algorithm " Has there been any study of whether J is quicker or slower than APL for this? Subjectively, APL is pretty and enticing in a way that J isn't. It's very strange to me that someone who promoted Iverson Notation as a better math notation, and wrote Notation as a Tool of Thought, could apparently completely switch notation from 30 yea…

I think the notation is not the only difference in J. I believe that tacit programming is one of the improvements proposed by J, but I might be wrong.

Tacit programming was sure brought to the foreground by J, but function trains apparently precede it by a couple of years, and have since been implemented in Dyalog APL and NARS 2000.

The idea of functions having rank, and modifying rank, was also made central to J, but has been included in Dyalog APL more recently - as much as they can without breaking backward compatibility, I think.

But it's the "notation" change I'm struck by - the linked article says "discuss the future prospects of “the notation” in its various forms." as if APL and J and K are the same notation, when they casually, visually, aren't, and in the sense that J differs from APL of 1990 with rank and tacit functions and linguistic naming and behaviours (gerunds, conjunctions, et al), they aren't the same notation semantically either.

Re: The Notation: Ken Iverson Centenary

#16

Post by moonchild »

kick wrote: post #8
sxp wrote: post #4

> ”K programs routinely outperform hand-coded C. This is theoretically impossible. K compiles into C. Every k program has a C equivalent that runs exactly as fast. Yet it is true in practice. Why? Because it is easier to see your error in four lines of code than in four hundred lines of C.” How true is this? I hear this claim a lot but haven't seen any real benchmarks. E.g, https://benchmarksgame-team.pages.debian.ne…

He wrote solutions for Shootout (predecessor to that Debian page) problems a few years ago; a lack of k in benchmarks is not due to a lack of k being volunteered: http://www.kparc.com/z/comp.k k is really fast. Half of the things on the Shakti mailing list are just Arthur getting really excited about how significantly he's beating x or y or z in performance and giving numbers for it. `grep`ping it now I see 40 in hal…

Have you looked at any of the opensource k impls? (Kona, ok, klong, ngn/k...)

Re: The Notation: Ken Iverson Centenary

#17

Post by kick »

moonchild wrote: post #16
kick wrote: post #8

Earlier quoted context omitted.

He wrote solutions for Shootout (predecessor to that Debian page) problems a few years ago; a lack of k in benchmarks is not due to a lack of k being volunteered: http://www.kparc.com/z/comp.k k is really fast. Half of the things on the Shakti mailing list are just Arthur getting really excited about how significantly he's beating x or y or z in performance and giving numbers for it. `grep`ping it now I see 40 in hal…

Have you looked at any of the opensource k impls? (Kona, ok, klong, ngn/k...)

If you look at my post history, it's actually primarily free software APL implementations! ngn/k is amazing. John Earnest's work with his own variants is really fantastic. I dislike klong and Kona.

Re: The Notation: Ken Iverson Centenary

#18

Post by kick »

sxp wrote: post #12
kick wrote: post #8

Earlier quoted context omitted.

He wrote solutions for Shootout (predecessor to that Debian page) problems a few years ago; a lack of k in benchmarks is not due to a lack of k being volunteered: http://www.kparc.com/z/comp.k k is really fast. Half of the things on the Shakti mailing list are just Arthur getting really excited about how significantly he's beating x or y or z in performance and giving numbers for it. `grep`ping it now I see 40 in hal…

Thanks for that link. Do you know if the results are posted anywhere? https://shakti.com/license.php says "Customer shall not... distribute ... any report regarding the performance of the Software benchmarks..." which I would need to agree to if I want to download the binaries at https://shakti.sh/benchmark/

You could check and see if it was ever posted on the Shootout site with archive.org, but when I checked coverage was spotty. The point of these benchmarks is everyone being able to run them themselves, though.

Re: The Notation: Ken Iverson Centenary

#19

Post by 082349872349872 »

jodrellblank wrote: post #15
haolez wrote: post #10

Earlier quoted context omitted.

I think the notation is not the only difference in J. I believe that tacit programming is one of the improvements proposed by J, but I might be wrong.

Tacit programming was sure brought to the foreground by J, but function trains apparently precede it by a couple of years, and have since been implemented in Dyalog APL and NARS 2000. The idea of functions having rank, and modifying rank, was also made central to J, but has been included in Dyalog APL more recently - as much as they can without breaking backward compatibility, I think. But it's the "notation" change…

Compared to the various instantiations of Pidgin Algol, they're the same. (obligatory "next 700" reference...)

Re: The Notation: Ken Iverson Centenary

#20

Post by geocar »

sxp wrote: post #4

> ”K programs routinely outperform hand-coded C. This is theoretically impossible. K compiles into C. Every k program has a C equivalent that runs exactly as fast. Yet it is true in practice. Why? Because it is easier to see your error in four lines of code than in four hundred lines of C.” How true is this? I hear this claim a lot but haven't seen any real benchmarks. E.g, https://benchmarksgame-team.pages.debian.ne…

> How true is this? I hear this claim a lot but haven't seen any real benchmarks ...

I think you're trying to read into the statement something other than Arthur meant; The last part of the quote is just as important as the first. Allow me to try and explain.

It's pretty easy for an experienced C programmer to beat K at some things, for example:

    int i,n;for(i=n=0;i
is faster (with gcc -O2) than the "equivalent":

    +/!1000000
which is more literally:

    int N=1000000;*a=malloc(N*sizeof(int)),i;
    for(i=0;i
but even an experienced C programmer will experience some fatigue trying to convert a K program to C in this way, and the K implementation is certainly faster than a literal translation (largely because it doesn't use malloc, but yes also because of careful vectorisation and parallelising of many of the operators).

It's my experience this difference adds up faster than anything else, and that's why a 20kloc C++ implementation of a HIBP checker was beat 10x by a 5-line k/q solution:

https://news.ycombinator.com/item?id=22467866

But make no mistake, I don't use k because it's fast, but because it makes me fast: five lines of code ain't squat to get right over 20k lines, so I'd prefer the k solution even if it were "only" just as fast as someone's C++ solution.

It does help tremendously though, that it's usually much much faster.

Post reply on HN