Live data from Hacker News

Interview with Ryan Dahl, Creator of Node.js

mappingthejourney.com

211–220 of 235 posts

Re: Interview with Ryan Dahl, Creator of Node.js

#211
post #97
post #69

Earlier quoted context omitted.

A lot of is more in what was taken away rather than what was added. As someone who is occasionally accused of being too pro-Go on HN... I actually have no problems thinking of Go as a really nice and refined 1990s language. Another type of "Java done right". (I say "another" because I think C# has a really good claim to that as well, albeit in a very different direction.) There's a place for that in the world. As muc…

> A lot of is more in what was taken away rather than what was added. Given the fact that C is basically the blueprint for Go, I'm not sure what things were taken away from C, syntactically speaking. It's more like Go added garbage collection to C, and a few other bells and whistles and told people : "this is 21th century programming", without actually thinking about what C got wrong at first place.

"I'm not sure what things were taken away from C, syntactically speaking."

Pointer arithmetic. Go has "pointers" but they're really more references. Some other things depend on how you look at them; Go "removes" manual memory management from the language by adding GC, it "removes" the fact that in C values are essentially untyped and very easy to penetrate down to that untyped layer by making all values carry their type around regardless, they "remove" the limitations on function pointers to provide real closures, they "remove" trivial array overflow by adding checked access, there's a lot of rough edges removed from C at the language level even if it required "adding" some runtime support.

I mean, by the time you've removed pointer arithmetic and manual memory management, you are by no stretch of the imagination in C anymore.

Re: Interview with Ryan Dahl, Creator of Node.js

#212

I can't help but feel like Google gently encouraged him to promote Go over Node. Or that being told by coworkers (goworkers?) for years-on-end that Go is better than Node has had an effect. Especially considering he talks about green threads like he's not quite sure what he's talking about. At any rate, there are many innovations where the creators weren't fully aware of their impact and eventually come to hate their…

Node lets you do single thread non blocking.

Go lets you do multi threaded non blocking.

That's a pretty substantial delta in the server world where you can easily have 24+ cores available.

Memory and cpu footprint of go is also drastically lower to node as it is a compiled language. That makes a difference when you are running in a cloud paying per GB of memory and per core.

I say this while actively developing in node.

Re: Interview with Ryan Dahl, Creator of Node.js

#213

Earlier quoted context omitted.

> People have a short memory, but at the time getting single to double digits of requests per second on a beefy server was entirely typical. Umm, what? It was not a challenge to get to double digits of requests per second on a beefy server in 2008, at least not in any popular language (including PHP and Ruby). It was (and still is) easy to scale requests of "blocking IO" languages by spawning additional threads at th…

I said entirely typical , not that it was a dick-sized "challenge". A completely standard developer on .NET or PHP in 2008 was building pages that rendered at less than 10 requests per second. This is experience, not a guess, given that my role was improving the performance of those disasters. A completely average developer on node was building pages that was 10x to 100x better performance. Secondly, simply spawning…

> Secondly, simply spawning threads is laughably non scalable

How else are you going to do it with a non-threaded language? I've ran well into the 10s of requests per second running "standard" PHP code on practically vanilla Apache configs. This was far from atypical and is no way any kind of challenge on a beefy machine.

My job was not to clean up disasters though. Maybe if your job was to clean up disasters, your typical was much different than mine?

Re: Interview with Ryan Dahl, Creator of Node.js

#214

Many are forgetting the initial reason node.js became popular. Consider the popular server-side landscape before node.js. It was dominated by Java, Python, etc. The primary way these ecosystems handle concurrency is OS-level threading. There was nothing else in the popular languages. Each language had some niche library that did non-blocking I/O (Twisted for Python, Netty for Java), but these all had a critical flaw,…

> Many are forgetting the initial reason node.js became popular. Consider the popular server-side landscape before node.js. It was dominated by Java, Python, etc. The primary way these ecosystems handle concurrency is OS-level threading. There was nothing else in the popular languages. What? I am sorry but this is not true. Coroutines have been a concept for...well a long time: https://en.wikipedia.org/wiki/Coroutine…

Remember the "C10K challenge"? Show me what else could achieve those ten thousand concurrent connections back then (hint: no python).

Re: Interview with Ryan Dahl, Creator of Node.js

#215

Earlier quoted context omitted.

Stack growth is orthogonal to the M:N vs. 1:1 distinction. You can have large stacks with M:N or small stacks with 1:1.

Are you saying we could use a 1:1 threading model with growing stacks (by growing stack I mean a stack that starts for example at 2 KB and is grown as needed)?

Sure. There are syscalls that allow a great deal of customization as to stack placement.

Re: Interview with Ryan Dahl, Creator of Node.js

#216

Earlier quoted context omitted.

Are you saying we could use a 1:1 threading model with growing stacks (by growing stack I mean a stack that starts for example at 2 KB and is grown as needed)?

Sure. There are syscalls that allow a great deal of customization as to stack placement.

Thanks. I didn't know about that. What is the cost of context switching compared to userspace threading?

Re: Interview with Ryan Dahl, Creator of Node.js

#217
post #179
post #53

Earlier quoted context omitted.

A good example - I have no idea why unix filesystems are like that, and after some idle googling I still don't know. Can't even figure out where to start to find the history of those decisions. Certainly am finding what kind of stuff goes where.

Some old Unix devs/users ran out of disk space, so they mounted more disk to those points. This caused boot problems since some tools (like mount) needed to be available. All of that together, along with people blindly following convention, and you have the mess of the unix filesystem. Yep... [1] [1] http://lists.busybox.net/pipermail/busybox/2010-December/074...

oh my god, that's awful

Re: Interview with Ryan Dahl, Creator of Node.js

#218
post #23

A serious problem with Node was the callback pyramid of doom. This means deeply nested callbacks such that the indentation wanders steeply to the right. When you do loops or have exceptions programming got very confusing. In effect you have to do a continuation passing transform. It's not difficult but the results are utterly unreadable. For example a simple for loop could be transformed into recursive function becau…

When green threads, fibers etc. were a hot topic in the node.js community I was pretty involved in core development. As far as I remember, we were never so much against the idea of it, but we didn't believe an actually good implementation was possible. The primary concern was scalability. At the time supporting high concurrency networking (10K+ connected clients) was a top priority for us. The solutions proposed by L…

Just a small heads up: Bruno Jouhier's solution is a transpiler just doing the continuation passing style transform. In other words, one could program in a synchronous style and the code got transformed to callback style. Later he added features to be compatible with fibers.

Re: Interview with Ryan Dahl, Creator of Node.js

#219
post #82

Earlier quoted context omitted.

>because the threaded code blocks. Python and Java threads don't block on I/O, only on CPU. Event loops (i.e. node) also block on CPU. AFAIK, node's advantage over threads was just that it offered much higher concurrency for I/O constrained tasks for the same memory (and was easier to programme for people coming from the front-end world).

Python and Java threads block the thread on I/O. Which is wasteful, because you have a huge chunk of memory allocated for the stack of that thread, being reserved but not actually in use.

Yep, hence node's advantage of offering much higher concurrency for the same memory usage.

Re: Interview with Ryan Dahl, Creator of Node.js

#220
post #160

Earlier quoted context omitted.

It should also be noted that there's another statically typed language that provides multiplexed M:N threading (aka "green threads") and a programming style that looks synchronous while actually using asynchronous file and network IO behind the scenes (so all of the perf benefits of JavaScript without the awkward, nested callbacks and/or promises). The language I'm referring to is none other than Haskell (which has b…

Your still missing the point. It's not that it was possible to do so. It's that you could quickly write web services idomatically that did so by default. Your example shows a simple hello world and doesn't speak to the point being made. I am sure you can get a more relevant example going in Scotty, but it appears that it didn't show up until 2012.

> Your still missing the point.

No, I'm not missing the point.

> Your example shows a simple hello world and doesn't speak to the point being made.

Hmm... Let's do an analysis:

* My example demonstrates reading from a given file descriptor (FD 0 -- stdin -- in this case), and that reading happens asynchronously (as I had pointed out, and even provided white papers describing how this works). Again, this is despite the syntax looking like a naive, synchronous program. The GHC Run Time System handles asynchronously reading from the FD and scheduling threads as appropriate such that OS threads aren't tied up waiting on what would otherwise be a synchronous read(2).

* As a bonus, it also demonstrates writing to standard out.

Now, one could take exception to the fact that my example doesn't give any evidence that similar asynchrony should be expected in a web stack. One would be wrong: as anyone with, say, as little as a year of experience writing web applications on POSIX systems should know, the Berkeley sockets API exposes sockets as file descriptors, so the same generic async event handling system in GHC's RTS applies equally well to networked IO. GHC and Node (via libuv) uses similar primitive kernel APIs at the end of the day, the only difference being that Haskell doesn't require that you reify the callbacks in the source language.

Idiomatic? Check.

By default? Check.

So no, I think I do get the point, but if you still disagree, I'm all ears.

If my tone sounds harsh, it's because I spent the time to provide links to two papers and even gave a small example of what the syntax would look like for a minimal program that demonstrates IO. In turn, you're telling me that I'm missing the point. I would argue that, if you read either of those two papers, and you knew about the implications with respect to networked IO (if you didn't, you could easily research the Berkeley sockets API, kqueue(2), poll(2), etc -- all mentioned in the papers), you would see what I'm getting at with my example code and the papers referenced. I can only posit that you haven't read the papers, or if you did, you must have not done so in earnest. Hopefully you can see how I would be frustrated when I've put in time to research this space, and when I share something, I'm met with what (in appearance) amounts to a dissimilar level of diligence and, ultimately, an empty dismissal.

If something else is going on, I would be more than happy to issue an apology.

> I am sure you can get a more relevant example going in Scotty, but it appears that it didn't show up until 2012.

Open source web frameworks for Haskell go back to (at least as far as) 2009.

Happstack: (2009): https://hackage.haskell.org/package/happstack-server-0.1

Yesod (2010): https://hackage.haskell.org/package/yesod-0.0.0

Snap (2010): https://hackage.haskell.org/package/snap-0.3.0

I didn't provide a "hello world"-like example of a web app because my command-line example only requires a trivial inference on the part of the reader to realize that the same asynchronous treatment of FDs there applies equally well to sockets, and thus web app programming in general; I deemed a full fledged web app example as more complicated than necessary to get my point across.

For what it's worth, here's an example in Happstack, straight from the documentation:

  module Main where
  
  import Happstack.Server (nullConf, simpleHTTP, toResponse, ok)  

  main :: IO ()
  main = simpleHTTP nullConf $ ok "Hello, World!"
And ...

  $ curl http://localhost:8000/
  Hello, World!
Also, a note on tense:

> It's that you could quickly write web services idomatically that did so by default.

I wasn't questioning the rationale for using Node back in 2009. My point is this: eight years have passed since Node was released, and alternative solutions have since been released. Somehow the Node community (and the greater web dev community in general) are oblivious to these alternative approaches.

I suspect this is because most web devs see the callbacks as evidence that IO is asynchronous in Node, and at the same time lack the requisite experience to intuit how another language might implement asynchronous IO while providing a syntax that looks like the usual synchronous model. Now, they could actually read the research papers put out, but they don't, and so everyone keeps going on about the technological revolution that is Node and its approach to IO, oblivious to alternative solutions that (arguably) provide a more convenient programming model (and is, if we throw in support for multiple threads, more computationally powerful, too).

Such laziness and/or lack of curiosity is pervasive in the industry, and that's frustrating.

Post reply on HN