Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Arthur Whitney releases an open-source subset of K with MIT license

shakti.com

Post reply on HN
Print view View original

181–190 of 234 posts

Re: Arthur Whitney releases an open-source subset of K with MIT license

#181

Post by chuckadams »

freedomben wrote: post #145
oefrha wrote: post #128

Earlier quoted context omitted.

Seems weird to switch to develop faster and complain about people conflating the two aspects when this thread is clearly talking about runtime performance, triggered by the benchmark claims: > real-sql(k) is consistently 100 times faster (or more) than redshift, bigquery, snowflake, spark, mongodb, postgres, .. > same data. same queries. same hardware. anyone can run the scripts.

> Seems weird to switch to develop faster and complain about people conflating the two aspects when this thread is clearly talking about runtime performance, triggered by the benchmark claims It doesn't look to me like GP switched to develop and complained of conflation. The switch happened higher up the thread by wood_spirit, and GP just continued the conversation (and called out the tendency to conflate, without ca…

You can understand something fully and still call it weird. I’ve used perl for decades, some of the things it does are still just weird. As for fallacies, one of the first things I was taught in logic class was that using an argument’s formal structure (or lack thereof) to determine truth is itself a fallacy. Unsound != Untrue, and throwing around Latin names of fallacies doesn’t actually support an argument.

Re: Arthur Whitney releases an open-source subset of K with MIT license

#182

Post by nlitened »

tresclow wrote: post #157
wood_spirit wrote: post #149

Earlier quoted context omitted.

This is a bit of a tangent but it’s a thought experiment that I recently heard: Data pipeline A is written and maintained by a team in a type safe language with extensive unit tests. Data pipeline B be was written long ago by a scientist who has since left, in sql in a day. Both compute the same dataset, but B gets the answer correct. Which is the better pipeline, and why?

And how do you know B's answer is correct?

It makes all the money

Re: Arthur Whitney releases an open-source subset of K with MIT license

#183

Post by Qem »

IshKebab wrote: post #172
anonzzzies wrote: post #73

Earlier quoted context omitted.

> a sign that a programmer hasn't worked on anything substantial Maybe you want to check who he is?

I think what he meant is he hasn't had to work with other people's code. Other people have to work with his code.

This reminds me of Rorschach prison remarks.

Re: Arthur Whitney releases an open-source subset of K with MIT license

#184

Post by chuckadams »

omnicognate wrote: post #152
philipov wrote: post #148

Earlier quoted context omitted.

Which one do you recommend? The first one I found was "Structure and Interpretation of Computer Programs" https://web.mit.edu/6.001/6.037/sicp.pdf

That's a good place to start but it's primarily about programming itself. In "Structure and Interpretation of Classical Mechanics" and "Functional Differential Geometry" he applies his approach of using computer programs as a way of communicating concepts to humans to some fascinating maths and physics topics.

The SICP lecture series goes places that the book doesn’t: one of them involved encoding derivatives in a generic way that high-school math never covered. It was mind-blowingly elegant, which is of course why I completely forgot it.

Re: Arthur Whitney releases an open-source subset of K with MIT license

#185

Post by fragmede »

sph wrote: post #69
fragmede wrote: post #57

Earlier quoted context omitted.

haven't you ever written code with single letter variable names and it makes sense to you? and then been forced to read somebody else's code with single character variable names and found it completely inscrutable? this is just that on (a lot of) steroids

No I don't write entire C programs with single letter variables, because there is no way where "c" is more readable than "cnt" or "count". With the usual exception of "for(int i", x, and y variable inside small scopes. If I was paid by the hour to write C, then I'd use single letter variables too, but I'm too lazy to do twice the work, when I can make my life simpler. Simplicity is a virtue, there is nothing interest…

> With the usual exception of "for(int i", x, and y variable inside small scopes.

But then the scope grows as the code evolves and suddenly you've got 200 lines with a bunch of single variable names. If I'm not a sadist and I rename the code before submitting the PR, but there's definitely a flow state where I've been living the code for too long and what makes total sense to me looks like line noise to others, or even future me. (I was real good at perl, back in the day.)

Point being, Arthur Whitney writes like this, even if you and I can't comprehend it, and yes it's obtuse. I wouldn't even want to work with myself if I wrote code like this, but I'm not as smart as Arthur Whitney.

As you say though, simplicity is a virtue. This is simpler for Arthur Whitney, even if it's more complicated for the rest of us.

Re: Arthur Whitney releases an open-source subset of K with MIT license

#186

Post by msla »

rwillink wrote: post #17

He showed K @ Royal Society & bunch of apl & aplus guys were there. Someone asked , where are the comments? AW said comments get out of date with all the changes, if you can’t read the code you Shldnt be working on it. We then all looked at each other..

This immaturity is one reason I don't like array languages: They ape mathematical conventions but mathematicians are interested in communicating, if only with other mathematicians, and so write clearly and understandably. That source code (if you can call it that, as I'm not entirely convinced it's the form Whitney himself programs in) is the opposite of clear to the point of being comically dense, in every sense of the term.

Concision is the handmaiden of clarity.

Re: Arthur Whitney releases an open-source subset of K with MIT license

#187

Post by rscho »

nathan_compton wrote: post #177
rscho wrote: post #156

Earlier quoted context omitted.

Are you a quant, or doing very specialized numerical things that aren't in libs on 2D datasets? Then 10X yes. If not, no. Everything else will be better

I'm not a quant but having used "data science" languages, ocaml, J, R, etc, I strongly doubt that an array language offers any substantial advantages at this date. I could be wrong, of course, but it seems unlikely.

J only is a data science language in the sense that its core is suited to dataset manipulation, but it's severely lacking in modelling librairies. If you're doing exotic stuff and you know exactly what you're doing, I can see the rationale but otherwise it's R/Python any day. It makes plenty of sense for custom mathematical models, such as simulations though.

Re: Arthur Whitney releases an open-source subset of K with MIT license

#188

Post by zimpenfish »

gmfawcett wrote: post #175
Qem wrote: post #135

Earlier quoted context omitted.

> Any of the other open source K projects are better than this (more complete, less buggy, better documented). One thing that puzzles me, about array languages, is that despite several open source implementations already existing, like J, its surprisingly difficult to find them packaged in Linux repositories. For example, you can't just "apt install J", or "apt install gnu-apl" on Ubuntu. In J case, it seems the defa…

I could swear that I used to install j902 from apt on Ubuntu. Am I misremembering this?

Arch has j901 in AUR which is directly installable if you have a helper (like `rua`) (or you can download the `tar.gz` and run `makepkg`.)

Re: Arthur Whitney releases an open-source subset of K with MIT license

#189

Post by losvedir »

jeremiahbuckley wrote: post #114
dchest wrote: post #43

Everyone here charges too little for software. the free(MIT license) version is shakti.com/k/k.zip the supported version(supported and 10 to 100 times faster) is $100K per month(minimum) https://groups.google.com/g/shaktidb/c/5SPufca3mo4

They’ve been charging a that amount forever, it’s a crazy ask. But you’ll be happy to hear that the quoted price is about 80% off of the price in 2000, so take advantage of the discount. In 2000 it was $100K/month.

Inflation has been 80% since then, but that doesn't mean it's 80% off. $1000 in 2000 is $1800 today, so a discount of 44%. 80% off would imply it's $5000 today, but prices didn't 5x fortunately.

Good joke, though.

Re: Arthur Whitney releases an open-source subset of K with MIT license

#190

Post by IshKebab »

rscho wrote: post #120
IshKebab wrote: post #95

Earlier quoted context omitted.

I would probably use Matlab for that sort of stuff tbh. Is K faster than Matlab?

For using a heap of libs to compute something that has been done a thousand times? No. For writing a completely new data processing pipeline from scratch? Much, much faster! Array langs have decent performance, but that's not why they are used. They are used because you develop faster for the kind of problem they're good at. People always conflate those two aspects. I use J for scientific research.

Yeah to be clear I meant performance-wise. In terms of development speed it looks like it just goes to insane extreme on the "fast at the beginning / fast in the middle" trade-off. You know how dynamic typing can be faster to develop when you're writing a one-man 100 line script, but once you get beyond that the extra effort to add static types means you overtake dynamic typing.
Post reply on HN