Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Q Language

code.kx.com

Post reply on HN
Print view View original

111–120 of 161 posts

Re: The Q Language

#111

Post by beagle3 »

nkurz wrote: post #77
beagle3 wrote: post #47

Earlier quoted context omitted.

> Instruction cache misses are rarely going to be a limiting factor. k's performance is a combination of a lot of small things, each one independently doesn't seem to be that meaningful. And yet, the combination screams. The main interpreter core, for example, used to be When Python switched the interpreter loop from a switch to a threaded one, for example, they got ~20% speedup[0]; I wouldn't be surprised if the fit…

And yet, the combination screams. Yes, I presume it's very fast because of a number of smart design decisions. I would guess that the relatively small on-disk size of executable is a consequence of these decisions, rather than a cause of the high speed. And as you point it, it's really the design of the core interpreter that matters. When Python switched the interpreter loop from a switch to a threaded one, for examp…

I am not involved with k, and things might have changed significantly, but around the 2003-2005 timeframe, Arthur had very conclusive benchmarks that showed I-cache residence makes a huge difference (IIRC I-cache was just 8KB those days ...).

The people who surely know what difference it makes today are Nial Dalton and Arthur Whitney.

Re: The Q Language

#112

Post by petermora »

I worked in Pascal, C/C++, Java, Python, R, Nim, Clojure, Rust, Go, Js, and created various projects in these languages.

However, I couldn't get used to Q. I understand that it is fast, and I also started to like the functional programming aspect of it. But oh boy, there is no proper error messages (no, "`type" is not helpful). The short versions of various map and apply were handy, but there are no equivalent versions with longer names, therefore there is no way to write a code which is readable to a non Q-expert. The strange evaluation order made it impossible for me to read other people's code, I often started to add parentheses and checked whether the behavior changed. There is no debugger (I used KdbStudio). Once I used more than 16 local variables, which gave a runtime error with some strange unhelpful error message.

I'm not questioning its usefulness, but I think that it could be much more developer friendly without compromising speed.

Re: The Q Language

#113

Post by nkurz »

beagle3 wrote: post #111
nkurz wrote: post #77

Earlier quoted context omitted.

And yet, the combination screams. Yes, I presume it's very fast because of a number of smart design decisions. I would guess that the relatively small on-disk size of executable is a consequence of these decisions, rather than a cause of the high speed. And as you point it, it's really the design of the core interpreter that matters. When Python switched the interpreter loop from a switch to a threaded one, for examp…

I am not involved with k, and things might have changed significantly, but around the 2003-2005 timeframe, Arthur had very conclusive benchmarks that showed I-cache residence makes a huge difference (IIRC I-cache was just 8KB those days ...). The people who surely know what difference it makes today are Nial Dalton and Arthur Whitney.

around the 2003-2005 timeframe, Arthur had very conclusive benchmarks that showed I-cache residence makes a huge difference

That sounds quite plausible. The front-end of Intel processors (the parts that deal with making sure there is a queue of instructions ready to execute by the backend) has made some major advances since then. The biggest jumps were probably Nehalem in 2007, and then Sandy Bridge in 2009.

It's not that binary size no longer matters, but you almost have to go out of your way to make instruction cache misses be the tightest bottleneck on a hot path. And when it would be the bottleneck, the branch predictor and prefetch are so good that it's usually only a problem when combined with poor branch prediction, so it really only adds to the delay rather than causing it.

Re: The Q Language

#114

Post by marklgr »

feelin_googley wrote: post #101
oskarth wrote: post #27

Earlier quoted context omitted.

> This is only software I have ever seen that actually got smaller over time. I agree, this is admirable! Even if it is under different names (which I think is a far better approach), Sustrik does this too: http://250bpm.com/blog:50 He went from AMPQ (not his own creation) -> ZeroMQ -> nanomsg -> Libmill (essentially Go in C/UNIX style) Also, OpenBSD comes to mind (LibreSSL for example).

Enjoyed the blog post from Sustrik. I also found Hintjens' work in line with my own sensibilities, e.g., his libero code generator. That is perhaps a good example of "finished" software. Whenever I have suggested on HN that there is such a thing as "finished" software that is free of serious bugs, I get some resistance. There is a consistent knee-jerk reaction citing the same tired, old meme, "All software has bugs",…

> There is a consistent knee-jerk reaction citing the same tired, old meme, "All software has bugs", and "Software is never finished."

It's not as much of a meme as fairly good heuristics. Most software has bugs, or depend on other pieces of software who have bugs. Most software can be improved or integrate with new technologies that weren't prevalent a few years ago. If software is good, it generally has a solid user base (relative to its target audience), and people will ask for tweaks and features--author can rightfully reject most of them, but it's a rare thing that absolutely none of them merits to be accepted.

So, when you discover a project that might be useful to you but you never heard of it and it's been inactive for years, I still believe it's quite a good rule of thumb to be leery before committing serious time and efforts to using it in anger. Which doesn't mean that counter-examples don't exist, naturally.

Re: The Q Language

#115

Post by TurboHaskal »

pgtan wrote: post #21
pyvpx wrote: post #17

what makes kdb so special and why isn't there an open/libre alternative?

> why isn't there an open/libre alternative? see here: http://t3x.org/klong/

I like this "apljk" language one the most. It can feel at times a bit dumbed down compared to K, but it has a more regular syntax.

Re: The Q Language

#116

Post by easytiger »

petermora wrote: post #112

I worked in Pascal, C/C++, Java, Python, R, Nim, Clojure, Rust, Go, Js, and created various projects in these languages. However, I couldn't get used to Q. I understand that it is fast, and I also started to like the functional programming aspect of it. But oh boy, there is no proper error messages (no, "`type" is not helpful). The short versions of various map and apply were handy, but there are no equivalent versio…

> therefore there is no way to write a code which is readable to a non Q-expert.

If you could FD wouldn't be able to justify charging £1.5k/day per head consultancy fees for someone they just sent on a 1 week training course.

Re: The Q Language

#117

Post by geocar »

petermora wrote: post #112

I worked in Pascal, C/C++, Java, Python, R, Nim, Clojure, Rust, Go, Js, and created various projects in these languages. However, I couldn't get used to Q. I understand that it is fast, and I also started to like the functional programming aspect of it. But oh boy, there is no proper error messages (no, "`type" is not helpful). The short versions of various map and apply were handy, but there are no equivalent versio…

I have worked in C (twenty-five years), C++, Python, Perl (twenty years), JavaScript, Common Lisp (for ten years), OCaml, forth, postscript, and also q/Kdb.

I think a lot can be done to improve teaching q/Kdb.

The current “best practice” is to change you until it makes sense but this takes time- anywhere from 6 months to a couple years based on your other experiences. And you still have to want to “get it”.

I want that to be better.

However this is alien technology: the terseness is a feature. Limited locals are a feature. The evaluation order is a feature. These things actually help it go fast (as surprising as that is!)

We only just recently got “a debugger” and good error messages because of repeated complaints and wishes for it from newbies, but there is a reason experienced Kdb/q programmers never wanted it. Why doesn’t that reason prompt those newbies to figure out why?

Yes, ideally you would learn the way and the why we do things, but there are a number of serious hurdles to overcome: To you q/Kdb seems merely unpolished even if maybe “fast”. Is it worth it? “Maybe”, you suggest, but then there are crazy people like me telling you something preposterous; that q/Kdb is actually incredibly well crafted, highly readable and an absolute pleasure to use.

I'm also trying to say that everything is worth learning: Nim and Python and JavaScript are all basically the same thing. You learned one of them, you kindof learned them all, so adding another one feels like these things are easy to learn. Alien technology is alien though, you haven't learned any of it. How can we even talk to each other?

I'm hopeful: Lambda was tricky, but it snuck in to things. Can we get tables, views, and high code density?

We need to find something better- some better way to talk about it, but the peanut gallery is loud.

Re: The Q Language

#118

Post by throwaway7645 »

fusiongyro wrote: post #110
throwaway7645 wrote: post #76

Earlier quoted context omitted.

Dyalog appears to be more popular with conferences and more products, but it costs $1k ish for a commercial license and nobody else can run your code without a license and server licenses aren't cheap. It also pretty much needs a special keyboard and a key mapping. J is free for pretty much everything and uses standard characters (although I really like the APL characters). I think they're both nice.

Dyalog comes with a keyboard layout (on a Mac it just replaces the alt keys). It's quite easy to use. GNU APL's Emacs mode does the same thing, although mapped to super rather than alt (meta, in Emacs) by default. I'm aware of the licensing costs, I'm more curious about whether Dyalog is obtaining popularity versus J, and if so, why. Of course, three new people going to the Dyalog conference would be a 10% increase i…

Yea, I was just saying the key mappings can be a pain and your favorite keyboard probably doesn't have the APL symbols on it. The Dyalog IDE has a virtual keyboard, but I don't like those too much. If none of that bothers you, than no biggie. I'm guessing Dyalog has more production users and a bit more users than you see at the conference as they are typically held in the UK. J is free, so I bet a lot more people try it even though Dyalog has a free hobby license. J has a nice built in plotting library"viewmat" while Dyalog has sharpleaf. Both are nice, but sharpleaf has a GUI like doing charts in Excel. Dyalog can easily hook-in to .NET, so that is pretty helpful on Windows in the real-world. I'd agree it's a wash right now. What is your background and needs?

Re: The Q Language

#119

Post by greglindahl »

beagle3 wrote: post #64
vram22 wrote: post #51

Earlier quoted context omitted.

>My C code has become faster, simpler, shorter and less buggy after I dabbled in K. Interesting. Can you explain why you think that has happened?

Several things: 1. K (and the entire APL family) eschew many of the layers upon layers of abstraction that modern software engineering uses, whether they are justified or not. It turns out, that they are mostly not justified. K gently pushes you toward thinking in a lower level of "what's really happening here?"; I'm not sure I can give a good example here - but the world looks different after taking the red pill. e.…

I'm confused what the relation is between alt-right ideology ("red pill") and the usual things one does for high performance.

Re: The Q Language

#120

Post by throwaway7645 »

RodgerTheGreat wrote: post #105
feelin_googley wrote: post #104

Earlier quoted context omitted.

I believe the GUI idea is still alive, but today it can be web-based. See 1010data.com.

1010data's web UI isn't actually based on the K2-era GUI framework; it is architected as a reasonably conventional web application that happens to use K3 as a server-side language. The 1010data query language offers facilities for making data-driven UIs for interactive reports and the like, but any semantic similarity to the K2 GUI system is probably coincidental.

So 1010data's purpose is to sell analytics tools to kdb+ users? Does kdb+ not come with a simple way to pipe output to a chart? For the price, I'd hope they had something more than a REPL.
Post reply on HN