Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

github.com

Post reply on HN
Print view View original

81–90 of 93 posts

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#81

Post by omaranto »

alfiedotwtf wrote: post #54
vessenes wrote: post #18

I have a longstanding fascination with K and other "modern" APL derivatives. There are a few intersecting truisms about coding that I believe: one is that people's working memory varies: some have an immense amount, some less. Humans definitely process spatially better than in time series (e.g. comparing side by side rather than turning over a page.) This implies you should prefer succinct code and languages because…

Awesome comment. In your opinion, what would be a language that comes closer to K’s functionality but at the same time be understandable to mere mortals?

In my opinion, the language that comes closest to K's functionality and is also understandable by mere mortals is K itself. It is obviously extremely close to K's functionality and is a very simple language, the only reason it doesn't seem simple is that most people are used to verbose languages. A couple of days of practice is enough to make K readable, in my experience.

Also, I am constantly amazed at how concise K is, easily rivaling not only conventional languages but also much larger array languages like APL or J. Arthur Whitney's taste in selecting primitives is out of this world.

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#82

Post by horsellama »

array languages are very fascinating and I'd love spending more time learning about them

I read that they are (K in particular) used in finance...would you reckon it would be easier to find job in that field having that in the CV? I know I cannot compete on the C++/Python side but maybe as a skilled K developer I could sneak in

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#83

Post by graemep »

scrawl wrote: post #77
graemep wrote: post #76

Earlier quoted context omitted.

I am interested, but one thing I have still not quite understood is where to use this family of languages. As far as I can tell they are best for data analysis but not for heavy numerical computation because most or all of them lack GPU support. is that right or have I got it wrong? Are there any other use cases?

q/kdb+ is used in finance (banking + funds) for heavy numerical computation every day. high-volume realtime data straight from markets, and petabyte/trillion-row historical DBs. it runs on CPU but computation easily parallelizes over cores/clusters. regarding use cases, see https://kx.com/resources/use-cases/

Thanks, that gives me better feel for it. Mostly analytics, good with large datasets, but probably not great for things where you get a big gain from GPU?

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#84

Post by omnicognate »

durumu wrote: post #78
mst wrote: post #74

Earlier quoted context omitted.

> a cool thing about q/kdb+ is that it takes so little code you don't have I$ issues I'm not sure what an I$ issue is - any chance of a bit more explanation? (also I'd love to hear about the tradeoff part in more detail but that's a bigger ask)

I$ is short for instruction cache ($ -> "cash" -> cache). Since kdb has so little code, more of it fits into the instruction cache at once.

I've encountered this idea that k's terseness somehow improves instruction cache use before. Can you explain further? It seems nonsensical, since instruction caching is about machine code, not source code. Why should it use the instruction cache better than any other JIT? Or is it interpreted, in which case "the terseness of the language improves cache use" might seem more of an admission than a boast... :-)

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#86

Post by CraigRo »

bwanab wrote: post #36
eismcc wrote: post #22

Earlier quoted context omitted.

Arthur Whitney is showing how to build an array language using C in array language form. AW is known for kdb+ which is often used in finance due to its extreme performance properties and ability for quants to quickly explore ideas. Personally, to get a better handle on how array languages worked, I implemented KlongPy which is a python implementation of Klong, which descends from K (which AW wrote). You have to play…

Ha! For those of us who were in the Morgan Stanley Fixed Income group for many years in the 1990s and 2000s AW is known for APLus which was used extensively for modeling and application work.

There were a lot of different and kind of weird attempts at this in the industry. The basic problem is that recalculating DAGs and monte-carlo simulations using a ton of floating point math was extremely expensive. Excel was way too slow, C++ required expert programmers and much better organization of programming teams to figure how to coordinate cached intermediate values. MS went with APL, which reduced a lot of the boilerplate and allowed for relatively easy memoization. GS went with SecDB which treated computation as memoized DAGs. Not sure what the others did

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#87

Post by mlochbaum »

omnicognate wrote: post #84
durumu wrote: post #78

Earlier quoted context omitted.

I$ is short for instruction cache ($ -> "cash" -> cache). Since kdb has so little code, more of it fits into the instruction cache at once.

I've encountered this idea that k's terseness somehow improves instruction cache use before. Can you explain further? It seems nonsensical, since instruction caching is about machine code, not source code. Why should it use the instruction cache better than any other JIT? Or is it interpreted, in which case "the terseness of the language improves cache use" might seem more of an admission than a boast... :-)

I say it's nonsensical (and yes, Ks are bytecode interpreted). https://mlochbaum.github.io/BQN/implementation/kclaims.html#...

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#88

Post by graemep »

userbinator wrote: post #62
vessenes wrote: post #18

I have a longstanding fascination with K and other "modern" APL derivatives. There are a few intersecting truisms about coding that I believe: one is that people's working memory varies: some have an immense amount, some less. Humans definitely process spatially better than in time series (e.g. comparing side by side rather than turning over a page.) This implies you should prefer succinct code and languages because…

One analogy I like to tell people who are overcome with shock and horror at APL-family languages is to compare it to someone used to Latin-family human languages looking at something like Chinese for the first time --- it's likewise totally "unreadable" at first glance, but then you realise that over a billion people can read and write that language fluently every day, many of which may also struggle with a Latin-fam…

Another is regular expressions. They look entirely incomprehensible if you see one with no prior knowledge, but once you get used to them they are fairly easy to understand.

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#89

Post by scrawl »

graemep wrote: post #83
scrawl wrote: post #77

Earlier quoted context omitted.

q/kdb+ is used in finance (banking + funds) for heavy numerical computation every day. high-volume realtime data straight from markets, and petabyte/trillion-row historical DBs. it runs on CPU but computation easily parallelizes over cores/clusters. regarding use cases, see https://kx.com/resources/use-cases/

Thanks, that gives me better feel for it. Mostly analytics, good with large datasets, but probably not great for things where you get a big gain from GPU?

what tasks are you thinking? i'm not a gpu expert

q is good with bulk operations on compact arrays; these are cache-friendly and the interpreter can utilize cache-level parallelism. and with q it's convenient to go from idea -> MVP in short time. it's a high-level language with functional features so expressing algos and complex logic is natural.

but it's interpreted and optimized for array ops. so really latency-critical (e.g. high-freq trading) or highly scalar logic will be done with C++. the trade-off is convenience of development.

Re: K/simple: a tiny K interpreter for educational purposes by Arthur Whitney

#90

Post by cachvico »

graemep wrote: post #83
scrawl wrote: post #77

Earlier quoted context omitted.

q/kdb+ is used in finance (banking + funds) for heavy numerical computation every day. high-volume realtime data straight from markets, and petabyte/trillion-row historical DBs. it runs on CPU but computation easily parallelizes over cores/clusters. regarding use cases, see https://kx.com/resources/use-cases/

Thanks, that gives me better feel for it. Mostly analytics, good with large datasets, but probably not great for things where you get a big gain from GPU?

Hedge funds use ML
Post reply on HN