Gotchas from Two Years with Node • Hacker News Board

Re: Gotchas from Two Years with Node

#81

Post by Rapzid » Tue, Apr 14, 2015, 1:59 PM UTC

wpietri wrote: post #76

Rapzid wrote: post #55

Earlier quoted context omitted.

I'm also unsure about this comment. It reads to me as "The main advantage of Go is.. that it makes it really easy to not have this problem." The OP talks about single core being an issue. It is not for Go due to the cooperative goroutine scheduling and function call pre-empting. You can still shoot yourself in the foot, but it's really easy to create an escape hatch by just making a function call or occasionally yiel…

> really easy to create an escape hatch by just making a function call or occasionally yielding to the scheduler... Having dealt with cooperative multitasking back in the dark ages, I definitely don't believe it is easy. With proper threads, you just write your code in a straightforward manner. With cooperative multitasking, you now have to be continuously imagining performance and sprinkling in otherwise useless cal…

Meh, it really doesn't come up that often. Every method call is an opportunity for the scheduler to run, not just specific ones. I find it is very rare indeed for a significant amount of work to be done without making method calls. Go also uses threads and makes parallel programming easy.. For me anyway.

Re: Gotchas from Two Years with Node

#82

Post by benmccann » Tue, Apr 14, 2015, 2:01 PM UTC

fennecfoxen wrote: post #72

ExpiredLink wrote: post #29

Why not Java EE? Everything out of the box. Tried, proven, standardized. Well suited for startups: http://www.adam-bien.com/roller/abien/entry/a_java_ee_startu...

Ah, yes, Java... the language that doesn't have niceties like hash literals, so the ecosystem adopted XML instead, and you'd use complex difficult-to-debug XML documents to metaprogram your frameworks because it was actually the least-painful way of doing things. (It's moved on slightly: annotations are the new XML. cough Jersey cough ) I mean, there's plenty of good stuff too, but... let's just not pretend that it's…

You can use XML in any language. No one forces you to use XML in Java. Yes, if you choose a crufty framework like J2EE or Spring then they'll use lots of XML. But you can just as easily choose a modern tools like Guice or Play Framework and not be forced to configure everything in XML.

Re: Gotchas from Two Years with Node

#83

Post by patsplat » Tue, Apr 14, 2015, 2:03 PM UTC

sanmon3186 wrote: post #59

aikah wrote: post #41

Earlier quoted context omitted.

why? PHP/ruby/python are successful too in the server space. Javascript,especially ES6, isn't worse than the formers. Devs should know by now that the dumbest tool that is good enough has good chances of being successful today. All these tech won't replace enterprise techs, but enterprise dev is a tiny percentage of all devs outthere.

>enterprise dev is a tiny percentage of all devs outthere I don't have specifics to quote but enterprise dev is definitely not a "tiny" percentage

Large percentage in cost. Tiny percentage in ROI. Blue ocean for disintermediation.

Re: Gotchas from Two Years with Node

#84

Post by vezzy-fnord » Tue, Apr 14, 2015, 2:10 PM UTC

davidw wrote: post #80

vezzy-fnord wrote: post #79

Earlier quoted context omitted.

RAILS or Node are easier and faster to handle compare to Scala or Erlang I don't know about Scala, but if one were to start from tabula rasa , I'd argue Erlang is easier to handle than Node. You have a standard set of patterns embedded in OTP and a well-defined process model purveying the entire language, whereas Node at its core being a reactor-based event loop means that you're bombarded by a variety of concurrency…

> Chicago Boss would be close competitors. I have commit access to Chicago Boss, and while it's a really cool effort, and is well suited to some niches like the one I'm using it for, it is very, very far away from being a competitor in terms of the flexibility and oodles of gems you get out of the box.

I'm actually using it myself currently. There are certainly rough edges (In fact, I think I'll be submitting a patch for boss_mail soon to tweak some of the default gen_smtp options that bit me). Migrations aren't as caliber or essential as in Rails, there is no asset pipeline, there isn't a scaffolding generator and so it must be filled manually, but it's still more-or-less a complete experience.

I don't know about "flexibility". It's much less rigid and opinionated than Rails, and it's easier to maintain modified versions of the source code in your project due to how Rebar handles dependencies. So I'd say it's pretty flexible.

Default library support, yes. But it's not that bad. Most major tasks are covered. The module system and resulting encapsulation (on top of runtime + OTP guarantees) means that using dumped libraries is easier and more reliable, so I had no qualms with reading and integrating, e.g. a wrapper to GraphicsMagick that serializes output to native Erlang proplists for batch image uploading.

The ETS session engine, the not strictly OO data mapper, functional tests, inbound mail server, in-memory MQ and model event watchers plus first-class WebSocket gen_servers are all nice perks.

My main concern is that commit activity has been dwindling down and the present lead maintainer (danikp, I think?) seems to be only sporadically active. But it can still be salvaged, I'd wager most of the serious users have private forks.

EDIT: By the way, out of curiosity, what "niches" are you using it for?

Re: Gotchas from Two Years with Node

#85

Post by untog » Tue, Apr 14, 2015, 2:12 PM UTC

uptownJimmy wrote: post #60

aikah wrote: post #41

Earlier quoted context omitted.

why? PHP/ruby/python are successful too in the server space. Javascript,especially ES6, isn't worse than the formers. Devs should know by now that the dumbest tool that is good enough has good chances of being successful today. All these tech won't replace enterprise techs, but enterprise dev is a tiny percentage of all devs outthere.

"...enterprise dev is a tiny percentage of all devs outthere." I may be terribly confused here, but I'm pretty sure this is the precise opposite of the facts. In terms of jobs, all the OSS languages TOGETHER are not as popular as Java alone, never mind adding in C#/.NET. It's a common misunderstanding, but one that seems unlikely to do a young coder any good...

It's HN bias. Enterprise rarely gets talked about (because by its nature it rarely does cutting edge things) but is still the huge mass of the iceberg under the water.

Re: Gotchas from Two Years with Node

#86

Post by zaphar » Tue, Apr 14, 2015, 2:13 PM UTC

jonpress wrote: post #78

wpietri wrote: post #69

Earlier quoted context omitted.

> In Node.js, it's really easy to identify process boundaries since the child_process module forces you to put code into different files and communicate via loosely coupled IPC channels. I find that approach troubling. Process boundaries are expensive, because you have to serialize and everything each time you cross a boundary. I also haven't used Go, but I think you can get great clarity with something like Akka's A…

Yes, socket/pipe-based IPC is more expensive than shared memory up to a point, but it's more scalable since you don't have to deal with locking (mutexes, semaphores) and the limits this imposes.

There is scaling and then there is scaling.

Sometimes you want to scale up to the box you are on and the most efficient way to do that is to use threads in a single process.

When you hit the limits of a single machine then it's time to start scaling out to multitple machines. You are right that then you have to start paying the communication/serialization costs and in return you get much greater scale.

However you don't have to start paying that cost until scale in that direction. You can get quite a lot out of multicore machine these days without having to pay serialization costs if you use threads.

Many times the serialization costs aren't worth the benefit unless you are getting a whole other box with another 32+ cores out of the deal. Paying it before you get that benefit isn't efficient engineering. And for some people choosing a language or framework that forces them to pay that cost before it's necessary is a bad idea.

Re: Gotchas from Two Years with Node

#87

Post by vosper » Tue, Apr 14, 2015, 2:21 PM UTC

Their first problem was sending customer requests directly into processing. They're collecting customer metrics - they should never be unable to handle a request because another request has consumed all the resources.

A much better way to do this would be to have a very lightweight API putting events into a stream (Kafka or, since they seems to be on AWS, Kinesis [1]). Let the stream absorb that crazy customer data, and let your data processing run full-speed from the stream. You'll get blips and slow downs, but it won't affect your ability to receive more data. Log out any errors or malformed data so that customers can see the problems. Do your profiling and optimisation, but avoid losing data.

[1] We're using Kinesis. Very easy to provision, does what it says on the box, and can easily handle thousands of requests per second.

Re: Gotchas from Two Years with Node

#88

Post by fennecfoxen » Tue, Apr 14, 2015, 2:29 PM UTC

benmccann wrote: post #82

fennecfoxen wrote: post #72

Earlier quoted context omitted.

Ah, yes, Java... the language that doesn't have niceties like hash literals, so the ecosystem adopted XML instead, and you'd use complex difficult-to-debug XML documents to metaprogram your frameworks because it was actually the least-painful way of doing things. (It's moved on slightly: annotations are the new XML. cough Jersey cough ) I mean, there's plenty of good stuff too, but... let's just not pretend that it's…

You can use XML in any language. No one forces you to use XML in Java. Yes, if you choose a crufty framework like J2EE or Spring then they'll use lots of XML. But you can just as easily choose a modern tools like Guice or Play Framework and not be forced to configure everything in XML.

> You can use XML in any language.

Ironically, Java actually ends up being one of the better languages for that; there end up being a lot of gaps in some of the others. Just a $0.02 example: Ruby tools like Nori and Crack that are designed to make idiomatic Ruby structures out of XML but have fundamental flaws in how they handle niceties like namespaces. (Nori can strip them out, or you can leave them in place as the short-name string, but you can't normalize a given namespace URI to a given string, so good luck if some tool that was sending you xmlns:ns0, xmlns:ns1, xmlns:ns2, etc starts messing up the order. I'd consider making a pull request, but chose to leave tech debt in that part of the stack instead.)

> Yes, if you choose a crufty framework like J2EE

I do believe that using J2EE was exactly the suggestion of the GP, unless there is a subtlety in the usage that I got wrong (I understand the 2 has been dropped in later EEs). And are the ones you suggest "tried, proven, standardized" like I was pitched?

And the cruftiness of the standardized framework is the even-more-fundamental answer to the original question of why-not-Java: it's not strictly because you can't do things with it well or quickly, it's because it has a crufty reputation.

Re: Gotchas from Two Years with Node

#89

Post by spion » Tue, Apr 14, 2015, 2:40 PM UTC

All these problems are solvable (streams by ditching built-in streams and replacing them with something decent, errors with promises + typescript, event loop blocking with a streaming JSON parser). But that still means that out of the box node is a pretty unsatisfying experience all around.

It took us a year to arrive at the solutions above, and many prominent members of the community scoffed at a number of them. Some are still scoffed at.

For example, everyone still thinks that promises should not catch errors, as if somehow throwing on typos is useful. Its not. The real solution here is a type system, the sensible error capturing model of promises that doesn't destroy all assumptions about code (unlike domains), and a sensible library like bluebird that reports rather than swallows up unhandled errors.

People also swore that streams are the best thing ever but the reality is that the built in streams are an organically grown design that accreted many flaws along the way. Most of them can be mended though and with streams 3 things are finally starting to be acceptable.

Re: Gotchas from Two Years with Node

#90

Post by astrodust » Tue, Apr 14, 2015, 2:46 PM UTC

spion wrote: post #89

All these problems are solvable (streams by ditching built-in streams and replacing them with something decent, errors with promises + typescript, event loop blocking with a streaming JSON parser). But that still means that out of the box node is a pretty unsatisfying experience all around. It took us a year to arrive at the solutions above, and many prominent members of the community scoffed at a number of them. Som…

After using Q for some time and recently discovering Bluebird, I've got to say Bluebird is the way it should have been done.

Great library and the API offers a lot of conveniences without being overwhelming.