Live data from Hacker News

Impending kOS

archive.vector.org.uk

81–90 of 242 posts

Re: Impending kOS

#81
post #51

Earlier quoted context omitted.

What you're actually seeing is testimony: people saying they are seeing something amazing, and they aren't very good at explaining what they saw. Btw: k doesn't translate to C. It's actually a quite simple interpreter. The fact that it outperforms other languages so easily should be saying more about those languages than it should be saying anything about k.

So would k be even faster if it were compiled to machine code rather than being interpreted? Would that involve an unacceptable speed versus space trade-off? Or am I missing some crucial reason why k code has to be interpreted?

According to wikipedia's uncited 'performance characteristics' section of their page on K (programming language):

> The small size of the interpreter and compact syntax of the language makes it possible for K applications to fit entirely within the level 1 cache of the processor.

Sounds like the overhead's acceptable?

.

Edit: the following page (2002) says

> Even though K is an interpreted language, the source code is somewhat compiled internally, but not into abstract machine code like Java or Python. The interpreter can be invoked interactively or non-interactively, and you can also compile a source file down to a binary file for product distribution, if you wish.

http://www.kuro5hin.org/story/2002/11/14/22741/791

Re: Impending kOS

#82

The same story told again -- it's aim is to generate this magical atmosphere around a fast db engine and language that's deliberately obfuscated to make people who work in it feel smart, so they try to spread that it's the best. KDB/Q is a nice tool, not the holy grail how they put it. And it's not fast because they know something better - it's because it lacks almost any safety measure.

I totally agree. That's why I wrote a parser for the language to help me and my coworkers debug =) https://github.com/kiyoto/ungod

As I commented above, the syntax is horrible. But I still think it is an effective tool for certain problems.

Re: Impending kOS

#83
post #34

> kOS is coming. Nothing will be the same afterwards. There seems to be this strange idea going around that if we just get the right tool, everything else is going to change forever. I see this a lot with people trying to create IDEs that let non-programmers create programs without really knowing how to code. But the thing is, most people just don't have anything worth coding. The problem isn't that the tools don't e…

If I am from another planet, and say I don't know why programs are so big and slow and buggy, and the most complicated program you see I've produced is a glorified calculator, it's too easy to be patronising and say well, that's because you haven't done anything complicated . However if I then show you a programming language, a database engine (similar in capability to SQL but around 1000x faster), a graphical deskto…

I'm just a regular web and app developer, but would it be at all possible to write a layer on top of k that makes it at least more readable while retaining most of the speed advantage?

To me, part of being an engineer is not taking offense when being told there are potentially better ways to do things, but at the same time I can understand why some people jump to that kind of statement since at first glance it seems like a regression of sorts, especially when the industry has a continuing history of trying to lower the entry barrier. Even though underneath the programming concepts may be revolutionary, people may also be quick to balk at it when it's in such a form (hence the first question).

Re: Impending kOS

#84
post #31
post #21

Earlier quoted context omitted.

I disagree fundamentally. I have noticed every page I scroll causes a comprehensive loss of around 90%, so in reading something that is 10 pagefuls long, I might only be able to produce a tiny part of the program. Your milage may vary. I find not scrolling, and just moving my eyes, I rapidly absorb the program, and I find most bugs just by reading the code . This practice is absolutely impossible for me if I have to…

Yes, the research literature on software development has consistently found that code size is the best measurement of complexity and predictor of error rates. (Sorry I don't have citations handy but we've discussed this many times on HN, and there's a recent study in the book "Making Software" that adds to it.) What's interesting is how strongly this goes against what most people think they know about good programmin…

I just had to troubleshoot a small helper app that took some HTTP input and wrote to a DB. The code was in C# and had about 10 files spread over 3 namespaces, plus a separate test infrastructure project. All sorts of factory models were used to setup an "HTTP pipeline" and authentication modules. The problem I had to fix: after a server upgrade, authentication was broken.

After digging around for a while, I discovered there was no bug. The partner's client code had the auth disabled, and the pervious server was misconfigured to not require auth. All which would not have been a problem if the system just did an "if headers.auth != "Basic ..." - but buried in this forest of stuff, it was overlooked.

It seems that some developers just love their edifices. They build all this "infrastructure", expanding code by an order of magnitude or more. It's considered good and robust and so, so much writing online is dedicated to this pursuit. I think it gives those programmers a feeling of import, as if they're really architecting something, not just pushing a few form fields around.

Even on the line by line basis, it's shocking how they love verbosity. Type inference? Nope, that makes things too compact and hard to read. Higher order functions to wrap up common patterns? Too difficult to understand. I'm not sure if developers simply lack the tiny bit of extra intelligence, or if they've tried it and honestly concluded that overflowing verbosity is the key to readability. Either way, it's sad, and holding back progress slightly.

Re: Impending kOS

#85

The same story told again -- it's aim is to generate this magical atmosphere around a fast db engine and language that's deliberately obfuscated to make people who work in it feel smart, so they try to spread that it's the best. KDB/Q is a nice tool, not the holy grail how they put it. And it's not fast because they know something better - it's because it lacks almost any safety measure.

I'm very interested in hearing more. What do you mean by lacking safety? As in the DB invariants aren't? Or lack of durability? Or as in, easy-to-misuse query language?

And to help calibrate, what are your preferred languages and styles?

Re: Impending kOS

#86
post #39

There's a contradiction I always see in these pieces: they talk a lot about the importance of using the right data structure. But these languages get their incredible conciseness by not giving you any choice about your data structures; their array type is hardcoded into the language, and if you want to use something else then your code balloons.

K basically has 3 data shapes:

atom (int, float, char, date, symbol, ...)

list (one dimensional array of atoms, dicts, flips or lists)

dict (a map from one list to another)

There's also a flip, which exchanges the first two indexes applied to an item (so, e.g., it effectively transposes a list of lists) but it is just sugar (both syntactic and semantic).

You can trust Whitney that all of these are properly implemented, including appends.

It's not often that you actually need more. I've discovered this after using K for a while, and going back to python.

Back in my pre-K (ha!) C++ and Python day, I had an awful lot of classes everywhere. After using K for a while, my Python and C both have much much fewer (structs in C more often than python, as C is missing python's dict). And the code has gotten much shorter and more efficient. Arguably, more readable as well. And I've essentially dropped C++ for C, because the extra complexity is just not worth it.

Re: Impending kOS

#87
post #3

Here's the text editor they're talking about: http://www.kparc.com/edit.k The code is, well, not the easiest to understand.

Could someone who can parse (reverse engineer?) this write up an explanation? (if you exist)

I'll give this a shot. I'll try to explain what's in my mind as I read it as well.

First, get out the reference manual: http://kparc.com/k.txt and we'll do the first couple lines.

The sequence that goes f x applies x to f. this f is unary.

The sequence that goes x f y applies x and y to f. this f is binary (and just labelled verb).

Some things (adverbs) go f a x and apply f in some special way to x.

Last hint: You read this code from left to right (like english). Do not scan it.

Now let's dive in:

    c::a$"\n"
This says: c is a view of where a is a newline. That is, if "a" contains a file, then c is the offsets of each newline.

A view is a concept in k that is unusual in other languages: When "a" gets updated, then "c" will automatically contain the new values. This is similar to how a cell in Excel can refer to other cells and reflect whatever the value of those cells happens to be.

    b::0,1+c
This says: b is a view of zero join one plus c. That is, where "c" contains the offsets of each newline, 1+c would contain the beginning of each new line. We joint zero to the beginning because of course, the beginning of the file is also the beginning of a line.

    d::(#c),|/-':b
This says: d is the view of the (count of c) joined with the max reduce, of each pairs difference of b. That sounds like a lot, but "each pairs difference of b" (where b is the position of all the new lines) is also the length of each line, and the max reduce of that is the longest line. You might say that "d" is the dimensions of the file.

    i::x,j-b x:b'j
This says: i is a view of x (?) joined with j (?) minus b (the offset of the beginning of each line) applied to each x which is defined as the bin of j in b.

j hasn't been defined yet, but you can see that x is defined later in the definition, and used to the left. This is because K (like APL) executes from right to left just like other programming languages. you write a(b(c(d()))) in C and you're executing the rightmost code first (d) then applying that leftwards. We can do this with anything including definitions.

The other string thing is that we know that b is the offset of the beginning of each line, and yet we're applying something to it. This is because k does not distinguish between function application and array indexing. Think of all the times you write in JavaScript x.map(function(a) {return a[whatever] }) when you'd really just like to write x[whatever] -- k let's you do this. It's a very powerful concept.

On that subject of binning: b'j is going to find the index of the value of b that is smaller or equal to j. Since we remember that b is the offset of the beginning of each line, then if j is an offset in the file, then this will tell us which line it is on(!)

But we don't understand what j is yet; it's the next definition:

    j::*|k
This says: j is a view of the last (first reverse) of k. We don't know what k is yet.

    f:""
This says: f is defined as an empty string.

    g::a$f
This says: g is a view of the offsets of f (an empty string) in a. Initially this will be null, but other code will assign to f later making g a value.

Next line.

    s::i&s|i-w-2
This is very straightforward; & is and, and | is or. While excel doesn't let us use a cell in it's own definition, k does: It means the old value of s. So this is literally: the new view of s is i and the old view of s or i minus w (?) minus 2.

We don't know what w is yet.

    S:{s::0|x&d-w}
This is a function (lambda). x is the first argument. If we called S[5] it would be the same as setting s to zero or 5 and d (dimensions) minus w. Double-colon changes meaning here; it no longer means view, but set-global.

    px:{S s+w*x}
This requires some knowledge of g/z: http://kparc.com/z.txt

Note that px is defining a callback for when the pageup/pagedown routines are called. x will be 1 if we page down and -1 if we page up. It may now become clear that S is a setter of s that checks it somehow. When we understand what w is (later in the program) it will be absolutely clear, but pageup/pagedown are changing s by negative w when pageup, and positive w when pagedown.

    wx:{S s+4*x}
Consulting the g/z documentation, we can see this has to do with wheels. Note we modify s again relative to 4 times x; x is -1 when wheelup and 1 when wheeldown. It becomes clear that s is modified by the pageup/pagedown and the mouse wheel.

    cc:{9'*k_a}
Again: in the documentation, 9' stashes something in the clipboard. cc is a function that takes the first slice of offsets k (we still don't understand) in a (the file). The g/z documentation says that cc is the callback for control-C. This is expected as control-C traditionally copies things to the clipboard. Since the slice of offsets k in a are being saved in the clipboard, we may guess at this point that k will contain the current selection.

This process is time consuming, but it is to be expected: Learning english took a while at first, and often required consulting various dictionaries. Eventually you got better at it and could read somewhat quickly.

I don't know if you want to try the next few lines yourself to see if you can get a feel for it, or if you want to try one of the less dense examples:

* http://www.kparc.com/$/view.k * http://www.kparc.com/$/edit.k

... or if you want me to keep going like this, or if you want to ask a few questions. What are your thoughts?

Re: Impending kOS

#88
post #83
post #34

Earlier quoted context omitted.

If I am from another planet, and say I don't know why programs are so big and slow and buggy, and the most complicated program you see I've produced is a glorified calculator, it's too easy to be patronising and say well, that's because you haven't done anything complicated . However if I then show you a programming language, a database engine (similar in capability to SQL but around 1000x faster), a graphical deskto…

I'm just a regular web and app developer, but would it be at all possible to write a layer on top of k that makes it at least more readable while retaining most of the speed advantage? To me, part of being an engineer is not taking offense when being told there are potentially better ways to do things, but at the same time I can understand why some people jump to that kind of statement since at first glance it seems…

I think this "readable" bit is the theory behind q which you can download from kx.com; Some people think it is more readable. q/kdb+ supports websockets and has a built-in http server so it is very possible to make web apps with it.

However I think there is value in the dense coding style that has nothing to do with it's performance: it's that it reduces bugs and helps you think about the problem.

Re: Impending kOS

#89
post #27

Earlier quoted context omitted.

It's not done yet. This summer, Pierre and I got kOS to boot directly into g (the graphical interface; formally called z) with ISR, keymap, modesetting, basic filesystem, etc weighing in around 100 lines of C. That was pretty exciting. Could probably be done with less with some deeper changes to Arthur's code, but it's still very useful to run k under Linux. Oleg made a silly little game in kOS. Arthur and Oleg did s…

Quote from OA "Whitney sent Oleg and Pierre some of the C code he was working on, and notes on a problem he didn’t know how to solve. They emailed back a solution, coded in his style." Did Pierre and Oleg think their solution out in standard code first and then make it Whitney-like, or did they find themselves thinking in Whitneyese straight away? I imagine their teacher may have noticed Whitney tendencies and that i…

I do not think they code in what you call "standard code", but I don't know how "straight away" it was either.

Writing in a dense fashion facilitates thinking about the solution.

Re: Impending kOS

#90
post #14

Earlier quoted context omitted.

Note that his lines are Can you write this editor in one line of less than 6000 chars of javascript (without a "textarea")?

Why do you exclude textarea? It's a high-level functionality available in JS. That q code uses the high-level functionality of q. What's the difference?

No, it's part of webkit/blink/gecko, it's not part of JS. But that's not why I exclude textarea:

I exclude textarea for the same reason dictionaries define something without using the same word and base form.

And the argument is generally meaningless: You may just as well call emacs and say you are using "the operating system's high level functionality".

Post reply on HN