Live data from Hacker News

Hard-won lessons: Five years with Node.js

blog.scottnonnenberg.com

231–240 of 365 posts

Re: Hard-won lessons: Five years with Node.js

#231
post #144

Earlier quoted context omitted.

The multiprocess Node concurrency model is brittle and fault intolerant. The event loop scan actually has real bottlenecks at a certain level of fds in flight. A real scheduler really wins here; libuv sits below the knowledge of the runtime to really be optimal. You become CPU bound far sooner than you expect. With a global heap, you also become memory bound far sooner than you expect as well.

Your typical Node instance is using an order of magnitude less memory than the beastly JVM. We use Java microservices with Spring at my job and each "micro" service consumes close to 1GB of memory. I've never written a Node service that had more than 100MB footprint.

The default maximum heap size for the JVM on systems with 4GB or more of RAM is one gigabyte. Has this been changed in your microservices' startup parameters? If not, it is no surprise that Java uses the memory that you have allocated to it before garbage collecting.

Spring Boot microservices that don't do much shouldn't need more than 32MB heap -- see https://spring.io/blog/2015/12/10/spring-boot-memory-perform... for details.

Re: Hard-won lessons: Five years with Node.js

#232
This looks like how I used to work with BASIC back in the 80s:

> I finally added extremely verbose logging ... started adding logging ... Verbose logging to the rescue ... I had the right logging in place ... My first step was to jump in and add some key logging statements ... added extremely verbose logging

Weird to see people happy to be limited to such stone-age work-flows when fully capable debuggers have existed for almost all proper languages out there the last 30 years.

Based on this, it's hard not to state that current Node-developer are the new PHP-developers: Unsophisticated, completely unable to see when the tools at hand are lacking, and happy with a with whatever they can get running.

Re: Hard-won lessons: Five years with Node.js

#233

The biggest hard lessons I've had with Node.js have been around handling errors - when I first deployed https://www.findlectures.com , I was shocked to realize that uncaught errors could take down the whole site. Not handling error callbacks is also a big problem - TypeScript has been an awesome solution though, because it can show compilation errors if you screw up function arguments.

That's why you use a production manager like pm2

pm2 will restart the app on crash if I understood correctly how if works.

Re: Hard-won lessons: Five years with Node.js

#234
post #151

Earlier quoted context omitted.

> there are tons of worthless imposters going around selling Node.js skills. However, it's even worse in the case of JavaScript because since it uses the "same language", some people assume that light front-end scripting qualifies as real backend development experience. Seconded. I'm working for a company building an Ionic app, and they hired two junior programmers with "angular experience" (both of whom had built an…

similar experience with ionic. yeah, I don't know 'angular' (well, didn't) but have still been able to be very productive very quickly (20+ years as well). and... the 'impostor' thing... quite true. easier for these people to slip in to larger companies, I think, but it's a problem all around. I would expect someone with 20+ years of experience to be able to help juniors troubleshoot in any language/stack, though, wh…

> I would expect someone with 20+ years of experience to be able to help juniors troubleshoot in any language/stack, though, whether they expect it or not. :)

I can with one, and have cracked his management style too, so he's gone from being unproductive and the butt of office gossip to meeting schedules.

The other is new to this country, doesn't speak much English (e.g. not enough to ask questions) and understands little English too. At a loss for how to proceed...

Re: Hard-won lessons: Five years with Node.js

#235

Earlier quoted context omitted.

V8 is a world class compiler. Compared to ruby, Python, Erlang, php and that ilk it is lightning quick. (Pypy is a good match for it but it's not main street and you can't just use any module.) in terms of performance and popularity, v8/node stand pretty much alone as far as dynamic non-compiled environments. JavaScript has a lighter weight feel than Java and the jvm stack. Single threaded with a top notch event reac…

> Compared to ruby, Python, Erlang, php and that ilk it is lightning quick. Ain't nobody is going to compare performance against Erlang for numerical number crunch. It was never built for that. It's not even a fair comparison. It's like comparing a race car against a hybrid prius and saying the race car suck at mpg. They're built for different purposes. Ilk implies as if Erlang is some crap flavor flav tech. It is no…

[deleted]

Re: Hard-won lessons: Five years with Node.js

#236

Earlier quoted context omitted.

V8 is a JIT compiler, which incorporates a runtime and a compiler.

> V8 is a JIT compiler, which incorporates a runtime and a compiler. So it's a compiler which incorporates a compiler? Compilerception much? Seriously though: Can I tell V8 to compile my JS and tell me any errors found before I deploy it to production and runtime? Yes or no? Just because V8 internally JIT-compiles the JS-code to something eventually executable which the machine can run doesn't change the fact that it…

> So it's a compiler which incorporates a compiler? Compilerception much?

No, what I meant is that the concept of a JIT compiler incorporates both a runtime and a compiler.

> Seriously though: Can I tell V8 to compile my JS and tell me any errors found before I deploy it to production and runtime? Yes or no?

That's not the job of a compiler. It's a feature of compilation, but a compiler does not by definition have to compile your code before deployment. What would you be compiling the JS to? If you're wanting to check your syntax, your IDE should be able to do that, or you can use a compile-to-JS language like TypeScript. Yes, it's dumb I know - that's one of the many reasons I use TypeScript.

> Just because V8 internally JIT-compiles the JS-code to something eventually executable which the machine can run doesn't change the fact that it's interpreter and execution-engine (also known as a "runtime") for Javascript code.

You seem to be hung up on the idea of a compiler as something that checks your code is correct. It isn't - that functionality is incidental to the compiler's job, because it can't translate invalid syntax into whatever destination language it compiles to.

> Just like Python and all those other platforms you listed as not being "compilers" (in which case you're absolutely right).

I did no such thing. That was someone else. Any language which internally uses a bytecode representation is a JIT compiler, including Python, Java, and PHP. You could argue I'm being nitpicky, given that pretty much all performant interpreted languages use a bytecode representation - it's true in a sense, but I'm just trying to highlight that you seem to have a misunderstanding of what a compiler's job is.

Re: Hard-won lessons: Five years with Node.js

#237
post #109

Earlier quoted context omitted.

> And with Hotswap, you don't even need to redeploy anything. Except when you have to restart the JVM because you added a class or a whole bunch of other things that it can't hotswap in the new code.

Still better than any dynamically typed language (I mean any) where you always have to redeploy everything since none of these language support incremental compilation, let alone incremental reloading.

If I'm using a truly compiled language (eg. C++, D), which have incremental compilation, I still have to redeploy everything since 'everything' is a single executable.

If I'm using an interpreted language I only have to deploy the changed source files. Git can do that for you.

Re: Hard-won lessons: Five years with Node.js

#238

Earlier quoted context omitted.

I replied in a sister thread, but here are some new points that are specific to your questions. Node was decided upon before I joined the team, because some of the senior developers on the team were big members of the node team, and they also wanted to crank out something new in this greenfield project. In retrospect, and in my reading of the history, Node was sufficiently different and new enough that it was "ooh, s…

>For large production systems that need to be supported and maintained, it's awful I don't think it's changing as quickly as you say, especially over the time since ES6 released/acquired first-class support, but you are right, this has been an issue. >Javascript itself basically has objects that are just JSON objects. When you're passing them back and forth, you get the irresistible urge to add "optional" fields... Y…

> I think you maybe haven't looked on the language with fresh eyes for a while

I'm working on a very old node codebase - it's over 3 years old, which is absolutely ancient in terms of node and the pace of its development.

And you're correct, we're not using classes, promises, or async/await. However, this being a large system that's in production, we don't exactly have time to rewrite everything to take advantage of new features, and introducing another way to do something when we already have 5 ways of doing the same thing that are different with nuances seems silly.

And refactoring a dynamic language is just orders of magnitude harder than refactoring a static language. With a compiler, you have reasonable confidence that you've found every use of a particular function or field. With dynamic languages, you're never quite sure, especially with Javascript.

That's why I said that for a long-running production system, I would advise against Node. I'm sure something else would come down the pipe that would fix your particular complaint, but might introduce one of its own. And once you already have a large mass of code, you're sort of stuck with that version unless you have time to do a complete rewrite with the new coding paradigms that might introduce bugs. Heck, we tried to just use the latest version of Node, and all of our tests passed. When we tried to deploy it to production, we had to hastily roll back because it started sigseving after a while under load.

One of the largest advantages that people tout for node is that it's easy to write large quantities of code for. But that's also a curse for stuff you have to maintain, as it almost seems like you're taking a snapshot of the Node ecosystem at the time that you start writing the program, and it's extremely hard to update/upgrade it. Hence my dislike of it for any sort of long-lived production system.

EDIT: for an example for half baked node libraries, let's take a look at handling command line arguments. The two leading contenders are minimist and commander. Minimist gives you very few features, as befits its name. Fine. But if you want anything with more feature, you get to use commander. Commander doesn't really handle required arguments properly, always uses -h as a flag to trigger help (not configurable), and when you pass in two strings as flags to set the value of a named variable, it always sets the value of the one that's listed last. (e.g, if you want -version and -ver to indicate the same flag, the library parses what you pass into it but only sets the -ver variable when it parses things). And this is the second most used command line parsing library in node, as far as I can tell.

Re: Hard-won lessons: Five years with Node.js

#239

This looks like how I used to work with BASIC back in the 80s: > I finally added extremely verbose logging ... started adding logging ... Verbose logging to the rescue ... I had the right logging in place ... My first step was to jump in and add some key logging statements ... added extremely verbose logging Weird to see people happy to be limited to such stone-age work-flows when fully capable debuggers have existed…

In some cases here, he's talking about fixing bugs in a production. It's not so simple to attach a debugger to a live application serving 100s of requests. Break and you just 500'd all users.

In others granted he's working locally and probably could have found the problems without logging, but that just comes down to preference IMO.

Node does have a debugger though and recently added official (though experimental) support for attaching the Chrome DevTools debugger [1]. Doing so was doable via 3rd party packages previously.

Edit: updated as author wasn't only talking about in-production.

[1] https://nodejs.org/api/debugger.html#debugger_v8_inspector_i...

Re: Hard-won lessons: Five years with Node.js

#240
post #15

I can't imagine choosing to write Javascript on the server, but considering its popularity I'm wondering if I'm wrong. So I'm curious as to the reasons people chose Node.js and whether you would recommend it, anybody willing to share their experiences?

If you want to run the same code on the browser as on the client (for example in a scheme where you want to reduce latency by optimistically running server-code on the browser), it is nice to have Javascript on the server.

But you could also achieve this by transpiling another language to Javascript of course.

Post reply on HN