Live data from Hacker News

Hard-won lessons: Five years with Node.js

blog.scottnonnenberg.com

211–220 of 365 posts

Re: Hard-won lessons: Five years with Node.js

#211
post #15

I can't imagine choosing to write Javascript on the server, but considering its popularity I'm wondering if I'm wrong. So I'm curious as to the reasons people chose Node.js and whether you would recommend it, anybody willing to share their experiences?

To use words from Yaron Minsky, JS is no OCaml but it has a surprisingly decent "dynamic range", even though it doesn't look like it. (Yes, I'm pretty sure I'll end up being ridiculed for writing that, but... hear me out)

Its easy to quickly put up a prototype. There are a ton of libraries for everything, and most come with copy-pasteable examples. Yes, this is frowned upon, and with good reason - but for prototyping, its exactly what you want. The prototype you write will also work much better than your average dynamic language prototypes (except for Erlang).

With generators and async/await the code became fairly pedestrian and un-convoluted. The only additional element in the mix are the few await/yield keywords sprinkled around.

The language is flexible enough that you can also write code in FP style (higher order functions, combinators, etc). For example, RxJS code looks pretty natural.

ES6 and above has all the modern bells and whistles: decent module system, classes, short lambda syntax, template strings... As a result code is pretty much as pleasant to write as Ruby or Python

When your project grows large enough, you have the option to painlessly convert to TypeScript. Besides getting rid of one annoying JS flaw (implicit conversions), this will also improve development tooling tremendously. TypeScript again has a wide dynamic range: you can start out with a fairly lax type system that allows a lot and then turn the dial up by turning on more and more checks (no any types inferred, strict null checks, etc) TypeScript's type system is surprisingly powerful: generics, unions, intersections, type guards, discriminated unions, control flow analysis with exhaustiveness checks, string/integer literal types, "mapped types" - the list is quite large and growing: https://www.typescriptlang.org/docs/handbook/advanced-types.... . But what makes TS different is that it models and formalises idiomatic JavaScript well, so your existing code will not need to be adapted.

TypeScript is where client/server code sharing starts to shine. Shared types enable end-to-end type-checking. You can arrange your code in such a way that data fetching is injectable, and use the same code with fetch on the client and direct method calls on the server (thanks to the same interface). If you have a very demanding client-side app, there will definitely be a lot of code reuse opportunities. (Nowadays I suppose this is also possible with Scala.js and bucklescript)

The lack of threads is a mixed bag. Most people already covered the disadvantages very well so I'll mention some advantages. Its much easier to reason about stateful code, since you can treat each synchronous code chunk as uninterruptible, and the possible interleaving points are always obvious (e.g. await/yield keyword). There are also several techniques and libraries to work around the disadvantages (e.g. dnode for RPC to separate processes for intensive calculations) and while none of them are very convenient, they usually end up being what you have to do eventually anyway. Threads will only get you so far with servers running CPU intensive jobs - soon you will need more than one machine and then you can't take advantage of shared memory any longer.

If all else fails, you can probably throw more processes at it and put a load balancer. Thats an interesting project, and AFAIK not yet solved (at least not in the node ecosystem). You would need a load balancer that is aware of the current state of the node processes - round-robin or random algorithms will be far from optimal to be helpful. I started some work on this here: https://github.com/spion/least-latency-balancer but I'm sure that its not the best approach and that it can be improved a lot. Might be a good idea to look at what the OCaml folks have - I'm pretty sure they've been dealing with a somewhat similar problem due to the lack of multicore.

Another way to attack this problem is to have better monitoring for long CPU-bound tasks e.g. https://www.npmjs.com/package/long-task-detector

But I will admit that in this regard, there are languages/runtimes where the situation is far better: Haskell, Go, Erlang to name a few.

Re: Hard-won lessons: Five years with Node.js

#212
post #207

Earlier quoted context omitted.

> So what's that mysterious reason that makes writing dynamically typed language faster? Less ceremony.

But no ceremony is necessary in the cases where types can be inferred - and in the cases where types can't be inferred you probably do want to to be explicit about them even in a dynamic language.

It's not just the types, it's the whole APIs, culture etc.

Java, for example, is no picnic, whether it can now infer some types or not.

In Python I don't even need a main().

Re: Hard-won lessons: Five years with Node.js

#213
post #207

Earlier quoted context omitted.

But no ceremony is necessary in the cases where types can be inferred - and in the cases where types can't be inferred you probably do want to to be explicit about them even in a dynamic language.

It's not just the types, it's the whole APIs, culture etc. Java, for example, is no picnic, whether it can now infer some types or not. In Python I don't even need a main().

I agree Java is a high-ceremony language, but not all statically typed languages are like that.

Re: Hard-won lessons: Five years with Node.js

#214
post #151

Earlier quoted context omitted.

I would watch it with that "large talent pool". As is typical in the language du jour, there are tons of worthless imposters going around selling Node.js skills. However, it's even worse in the case of JavaScript because since it uses the "same language", some people assume that light front-end scripting qualifies as real backend development experience. "Tons of modules" is also a misnomer, because a much larger than…

> there are tons of worthless imposters going around selling Node.js skills. However, it's even worse in the case of JavaScript because since it uses the "same language", some people assume that light front-end scripting qualifies as real backend development experience. Seconded. I'm working for a company building an Ionic app, and they hired two junior programmers with "angular experience" (both of whom had built an…

similar experience with ionic. yeah, I don't know 'angular' (well, didn't) but have still been able to be very productive very quickly (20+ years as well).

and... the 'impostor' thing... quite true. easier for these people to slip in to larger companies, I think, but it's a problem all around.

I would expect someone with 20+ years of experience to be able to help juniors troubleshoot in any language/stack, though, whether they expect it or not. :)

Re: Hard-won lessons: Five years with Node.js

#215
post #205

Earlier quoted context omitted.

That's a way to do it. It is not the only way to do it, even if some people do it that way all the time. I might add a transpiler for Typescript to my stack soon, if that language turns out as pleasant to work with as I've been hearing. If I do, it'll be the first time I use one. And I've been working with Node, in production, since 2013.

Go away. I've invested weeks to learn something which at least resembles NodeJS best practices and now you come along and say "nope, we do it different" ... :( (That was a joke btw. Thanks for the info. The consensus seemed to be that Babel is the way to go, but if your variant works for you good to hear. I will still stay with it. I like ES6 imports in Node.)

Hey, you do you. I'm just here to say that not everyone agrees, or should, that real-world best practice necessarily involves a giant pile of transpilers and bundlers.

Re: Hard-won lessons: Five years with Node.js

#216

Earlier quoted context omitted.

Can you elaborate on the "major pain in the butt to monitor and keep Node.js running production-like" What sorts of issues did you face? Certainly .NET and Java have more tooling, but I haven't noticed a difference between keeping a Node process up as opposed to a Python one. Hard to compare to stateless PHP.

The default behavior for Node when there is an uncaught error is to crash the process, killing all requests in flight. Even if you go out of your way to stop this default behavior, errors still likely leave the process in an inconsistent state. Node can't just unwind a few stack frames like synchronous platforms. Do you read JSON from ajax post bodies? Do you access fields in that JSON without sanity checking it? Try…

The default error middleware built into my framework caught it and responded with a 502.

Re: Hard-won lessons: Five years with Node.js

#217

Earlier quoted context omitted.

There are certainly huge differences in the number of gotchas different plaforms have. I have been absolutely disappointed by Node's tremendous number of gotchas in deployment and massive complexity of build pipelines needed to get around the inherent issues of the platform. My deployment with Django, for example, have been much simpler to handle.

For your information: to assess what projects you might be referring to, I clicked on your profile, but your hackernews profile points me to a URL with a broken security certificate belonging to Ukrainians, and even when I bypass the security warnings in the browser, I am sent to a page which gives me the message "404 not found".

Hackernewsers was a thing from a few years ago, not his fault it's shut-down.

Re: Hard-won lessons: Five years with Node.js

#218
post #95

Earlier quoted context omitted.

JavaScript is a scripting language. So is Python and Ruby. As a scripting language, excluding all bindings (files, networking, etc), JavaScript is substantially faster than Ruby or Python. Now, in terms of bindings, bindings to libuv are also fast. So node is not so much of a problem in itself. However the problem is how the language is being used: To say you can write a serious library or server code because you kno…

I think a lot of that is baloney. Not all apps require the same degree of rigor. In fact most require quite little. If a library is widely used, that's a pretty good indication it's "safe".

What you describe is a functional prototype not a finished application.

A functional prototype is software that implements its functional requirements (e.g: features) but not its non-functional requirements (maintainability, stability, scalability, performance, configuration, monitoring, security, etc.)

What is "a lot of baloney" is people selling functional prototypes for the price of a finished application, or not knowing the difference.

Re: Hard-won lessons: Five years with Node.js

#219

Earlier quoted context omitted.

V8 is a world class compiler. Compared to ruby, Python, Erlang, php and that ilk it is lightning quick. (Pypy is a good match for it but it's not main street and you can't just use any module.) in terms of performance and popularity, v8/node stand pretty much alone as far as dynamic non-compiled environments. JavaScript has a lighter weight feel than Java and the jvm stack. Single threaded with a top notch event reac…

> Compared to ruby, Python, Erlang, php and that ilk it is lightning quick. Ain't nobody is going to compare performance against Erlang for numerical number crunch. It was never built for that. It's not even a fair comparison. It's like comparing a race car against a hybrid prius and saying the race car suck at mpg. They're built for different purposes. Ilk implies as if Erlang is some crap flavor flav tech. It is no…

>It was never built for that. It's not even a fair comparison.

No, but it means I can write relatively computationally intensive server side code in JavaScript rather than having to switch to C and use an FFI. It's always nice to have decent performance, even if it's irrelevant most of the time for web programming.

Erlang is great but its ecosystem is tiny compared to Node's, and as a programming language it's at least as quirky as JavaScript.

Re: Hard-won lessons: Five years with Node.js

#220

Earlier quoted context omitted.

> V8 is a world class compiler. Compared to ruby, Python, Erlang, php and that ilk it is lightning quick. Not to nitpick details, but V8 is a runtime , not a compiler. A compiler checks your code and tells you if it contains errors before you run it. V8 does not.

V8 is a JIT compiler, which incorporates a runtime and a compiler.

> V8 is a JIT compiler, which incorporates a runtime and a compiler.

So it's a compiler which incorporates a compiler? Compilerception much?

Seriously though: Can I tell V8 to compile my JS and tell me any errors found before I deploy it to production and runtime? Yes or no?

Just because V8 internally JIT-compiles the JS-code to something eventually executable which the machine can run doesn't change the fact that it's interpreter and execution-engine (also known as a "runtime") for Javascript code.

Just like Python and all those other platforms you listed as not being "compilers" (in which case you're absolutely right).

Post reply on HN