After a year of using Node.js in production • Hacker News Board

Re: After a year of using Node.js in production

#211

Post by teh_klev » Tue, May 03, 2016, 1:36 PM UTC

astrobe_ wrote: post #192

placebo wrote: post #98

I usually don't respond to anything which I feel is just another "language war" provocation, but whenever I see these type of reviews I'm mystified. I've developed in many languages and frameworks, both well known and lesser known - decades of client and server side of C/C++, Javascript, Lua, Java, Python, PHP, Perl, Lisp. Pascal (just to name a few) in projects of all sizes, and not once did I have the thought "this…

> I feel that blaming failure on the language used is like blaming a bad novel on the word processor used to write it. Sure. If you're a good novelist you can write something great even on a keyboard with a broken 'e' key (btw, your 'I' key is about to break; you should replace it asap). But novelists don't have deadlines and when the novel is done, they usually don't take reader requests to change this or that part…

> But novelists don't have deadlines and when the novel is done

I hate to "well actually" you :) , but a great number of authors have contracts that demand one or two novels a year over a fixed period of time. I know Iain Banks did with his publisher, he mentioned it at a one of his readings of a new but yet unpublished novel I was lucky to attend back in 2007'ish.

> they usually don't take reader requests to change this or that part of the story.

Even well known authors are reigned in by their editors and proof readers, and sometimes things have to be changed.

Re: After a year of using Node.js in production

#212

Post by Silhouette » Tue, May 03, 2016, 1:40 PM UTC

mgkimsal wrote: post #207

Silhouette wrote: post #78

Earlier quoted context omitted.

Ah, I see. In that case, I think we agree!

I think you're both saying something similar, but there's a "why" in there to connect: > Younger engineers are ignoring lessons learned Why ignoring them? Beyond the 'youth' factor directly, the signaling from the larger companies mentioned (google, etc) is that those lessons don't necessarily matter, and... look, google did XYZ, you can too (ignoring that they threw potentially 3-5x as many people at problem XYZ tha…

Yes, that's basically what I meant. I think some of the highly visible projects from tech giants today are succeeding despite their approach to software development, not necessarily because of it, but a lot of the new developers coming into the industry lack the experience to realise that and think of these projects as examples of how things should be done.

Every time I see some young developer posting on HN about how the code we write today only has to work for a year or maybe two at most, or how 10,000 lines of code is a large program, or how some high profile company's web site that is still fundamentally just forms and tables and an API for talking to a database is a complicated UI, I weep a little for the future of our industry and our lack of ambition. Of course things like performance and stability and standardisation and portability don't matter very much if all you ever do is hop from one small throwaway project to the next every few months.

I think the culture around HN is unfortunate in this respect, because from a commercial point of view, it is an alluring idea to build some sort of trivial MVP, try and get crazy amounts of funding, and then if (and only if) you succeed, to throw it all out and start over. As a business strategy for building the next WhatsApp or Instagram, that's rational and perhaps quite effective. But it also contributes to the everything-is-expendable mindset that plagues our industry, and since the overwhelming majority of software development projects don't have the luxury of starting over every five minutes, I think it breeds a lack of respect for professional programming and the skills required to build good software without the benefit of an effectively infinite funding source from benevolent investors (or from the goose that keeps laying golden eggs elsewhere in the organisation).

That might be OK for Facebook or Google, but I wonder how many other startups that fail could still have been very successful, albeit not in unicorn territory, if their developers had paid more attention to some of these issues. I wonder how much time humanity collectively wastes today just because software quality and reliability aren't taken seriously by too much of the industry.

Re: After a year of using Node.js in production

#213

Post by Silhouette » Tue, May 03, 2016, 2:02 PM UTC

spion wrote: post #138

Silhouette wrote: post #132

Earlier quoted context omitted.

Yes and no. The difficulty of getting Node itself up and running on the various platforms involved was a significant part of the problem. These weren't Linux boxes built around Xeons. It turns out that as soon as you go outside of the mainstream, the tools and documentation for building and running Node are awful compared to a lot of other languages. The lack of standardisation and stability and the relatively poor d…

> The tools and documentation for building and running Node are awful compared to a lot of other languages. Can you be more specific? > It's just a big runtime, and a lot of overhead because of the way libraries and dependencies are handled. Compared to what exactly? I don't see any mainstream dynamic language except Lua doing better. Ruby? Python? If you don't need native modules (which is actually the case in a sur…

Can you be more specific?

Sorry, it wasn't my part of the project so I can't offer much detail. I know they never got as far as a Node build that passed the full test suite on some platforms. Many of the problems seemed to be around how floating point is handled on different types of CPU.

Compared to what exactly? I don't see any mainstream dynamic language except Lua doing better. Ruby? Python?

The total footprint for Node and supporting packages was taking up a few GB more than the equivalents for some of those other languages you mentioned. If you're working on a system where your non-volatile storage is something like a microSD card or flash chip, "a few GB" can be a huge difference.

If you do need native modules, the situation is slightly worse than C or C++, but not significantly: you generally need to build on a machine with the same architecture (Setting up cross-compilation with C isn't very easy either)

Sorry, our experience has been very different on both counts. Building on a machine with the same architecture isn't necessarily a reasonable option if you're targeting platforms that aren't general purpose computers, and at best probably means jumping through a lot of hoops and figuring out a lot of the details for yourself. In contrast, cross-compiling a language like C is straightforward and reliable, the tools for doing it are mature and well documented, and many developers who have worked in embedded systems are familiar with the techniques.

P.S. Based on what you said, I have a hunch that most of your troubles came from this module: https://github.com/mapbox/node-sqlite3 - is that right?

No. The person doing the Node version was trying to use some of the NoSQL tools, and seemed to be spending much of their time documenting and implementing things you get as standard with a SQL database. The person who wrote the C based alternative just used SQLite as a demonstration of how much easier and more robust everything could be using a different database instead.

One of the striking ironies of the project was that although in principle a dynamic language like Node doesn't need a compilation step, just the JS source, while C code has to be compiled for every target platform, the reality was that getting the C code running was dramatically easier. A well-made C library like SQLite could be installed, cross-compiled and running on all the required platforms within literally a few minutes, while Node and several major packages were so flaky that the equivalent level of real world functionality was never achieved even after several months of work. Again just for comparison, some of those other dynamic languages you mentioned were in between but much closer to the shorter end of the spectrum. The Node ecosystem, in our experience, proved to be an exceptionally bad choice for this sort of work.

Re: After a year of using Node.js in production

#214

Post by placebo » Tue, May 03, 2016, 2:07 PM UTC

astrobe_ wrote: post #192

placebo wrote: post #98

I usually don't respond to anything which I feel is just another "language war" provocation, but whenever I see these type of reviews I'm mystified. I've developed in many languages and frameworks, both well known and lesser known - decades of client and server side of C/C++, Javascript, Lua, Java, Python, PHP, Perl, Lisp. Pascal (just to name a few) in projects of all sizes, and not once did I have the thought "this…

> I feel that blaming failure on the language used is like blaming a bad novel on the word processor used to write it. Sure. If you're a good novelist you can write something great even on a keyboard with a broken 'e' key (btw, your 'I' key is about to break; you should replace it asap). But novelists don't have deadlines and when the novel is done, they usually don't take reader requests to change this or that part…

> (btw, your 'I' key is about to break; you should replace it asap).

haha - touché, point taken :-)

Deadlines seem like a great excuse for compromising quality. Sure, life is complicated, the boss is demanding, the mortgage has to be paid, the children need to be supported etc. etc. but compromising quality and enthusiasm (they usually are correlated) because of the "terror" of a deadline will just leave you at the mercy of the next "terror", only this time you'll even have even less enthusiasm to fix the spaghetti. Doesn't sound like an enjoyable existence.

Of course, very few people have the privilege of never having to compromise, but it's never black or white and there are many more degrees of freedom to choose the path with more quality than are implied.

>"The advantages and limitations", what works and what doesn't, "areas to avoid" are precisely the point of that kind of review.

The "area to avoid" in the review is Node.js and considering that large and impressive projects have been written in it, it seems that this is another case of throwing the baby out with the bathwater.

Re: After a year of using Node.js in production

#215

Post by forty » Tue, May 03, 2016, 2:13 PM UTC

CJefferson wrote: post #153

kansface wrote: post #119

Earlier quoted context omitted.

> 2. I think that ALL of the other server-side language/framework combos (Ruby/Rails, Python/Django, C#/.NET, PHP/Symfony, etc.) make it significantly easier for a "merely-good" dev to make really good Web applications. All of those frameworks are threaded/synchronous - they are maybe 10 times easier to use and potentially 100x slower (less concurrent) than Node. The ruby equivalent isn't rails, its eventmachine. Mov…

I haven't found the overhead of threaded frameworks to be anywhere close to that -- in practice it can be close to even, maybe 2x slower. And a big advantage of threaded is that it is (obviously) easier to scale to multiple CPUs, and I don't have to worry about one connection taking more than a handful of milliseconds and jamming up my server. Node is fine, but I feel many of the speed benefits come from V8 vs cpytho…

Surprisingly (to me at least) scaling single threaded code to multiple CPUs is actually very easy: just spawn the node process N times (using cluster, naught, pm2 or similar project) and that's it.

Re: After a year of using Node.js in production

#216

Post by Corrado » Tue, May 03, 2016, 2:16 PM UTC

eagsalazar2 wrote: post #151

Corrado wrote: post #147

I agree with this article and find the Node.js community, and to a lesser extent Javascript itself, exhausting. It seems like every 2 minutes there is a "more" proper way to do something, which tells me that the architecture is not yet mature, even though it's pretty old by now. On a related note, it seems like every time you find something that doesn't quite work correctly or conveniently in Node.js there is a "fix"…

Yes, the Javascript world is quickly evolving both on the front end and backend, and of course that can be exhausting but I have to disagree with both conclusions that (a) this means the tech is immature, and (b) the community's readiness to make changes is "holding it back". The js world is very unique in its ability to evolve quickly and things have improved massively over the last few years. Now, apart from the ch…

I'm not saying that Node.js is bad and shouldn't be used. Rails was in the exact same boat several years ago; everything was changing so quickly that new people on the scene had to work pretty hard to keep up. Being on the other side of that process with Node.js, I can now understand the Rails newbie's frustration. :)

If you are comfortable with the constant change and evolution in a platform/language/framework, then by all means continue to use it. Just don't disparage people that come forward and point out the pains they have and say that everything is fine. Just my $0.02.

Re: After a year of using Node.js in production

#217

Post by co_dh » Tue, May 03, 2016, 2:29 PM UTC

welcome back :)

Re: After a year of using Node.js in production

#218

Post by auvrw » Tue, May 03, 2016, 2:53 PM UTC

Silhouette wrote: post #65

beders wrote: post #55

Now all of a sudden, having types and some standards to gather around doesn't sound like a bad idea anymore ;) I agree with one of the commenters: Lessons already learned by older engineers (who went through similar woes with other languages/tools) are being re-learned again and again. The software industry is in a sorry state. Unless you are a very disciplined team with a very strong sense of writing modular code, d…

Lessons already learned by older engineers (who went through similar woes with other languages/tools) are being re-learned again and again. If only that were true! I think the JS (and more generally web development) ecosystem today is more a case of those who do not learn from history being doomed to repeat it. The thing is, if you actually are Google or Facebook or Microsoft or Mozilla or Apple, you can throw huge a…

maybe i haven't read this thread closely enough ... the parent to this post mentions "lessons of the past" and its parent mentions typed languages, so uh, what lessons are being discussed here? certainty not "the past teaches that typed languages are best," b/c lisp

Re: After a year of using Node.js in production

#219

Post by 15155 » Tue, May 03, 2016, 4:08 PM UTC

Klathmon wrote: post #196

esailija wrote: post #148

Earlier quoted context omitted.

Startup speed is terrible with that approach, each browserified module adds like 10x more code as boilerplate than something like isArray implementation would take. Then every module dependency is resolved dynamically at runtime, which will also quickly become a performance problem even when you aren't using micromodules.

I haven't touched browserify, but with webpack what you said could not be more false... Webpack doesn't bundle at runtime, it doesn't add any amount of code overhead per module that I can easily measure, and it doesn't trash startup speed.

> it doesn't add any amount of code overhead per module

It's something like tens of bytes (minified), FWIW, for the closure that each module gets put into.

Re: After a year of using Node.js in production

#220

Post by Someone » Tue, May 03, 2016, 4:19 PM UTC

wrong_variable wrote: post #18

kcorbitt wrote: post #5

I've spent a lot of time writing Javascript on the front-end in the last year using both React and React Native. I've found the React ecosystem to be a sane, productive and enjoyable development environment. Interested in sharing more model logic between our front- and backends, I also investigated writing some new backend features using Node (we're currently developing with Rails). But after days of research and pla…

The same problems you cite for node.js are the reasons why a lot of devs love node.js. Its much easier to do your own research and find the best module to solve a particular problem you are having then to shoehorn into some larger monolithic framework. Also its a lot more fundamental then that - Node.js has prolly the fastest iteration cycle for any platform out there since its so easy to create your own module - it…

That Cambrian explosion is fun when you are programming, but if you are writing a product, you will want to make sure you don't pick a dinosaurus or trilobite to build it on.