Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gotchas from Two Years with Node

segment.com

Post reply on HN
Print view View original

101–110 of 125 posts

Re: Gotchas from Two Years with Node

#101

Post by mike_hearn »

bryanwb wrote: post #57
eva1984 wrote: post #6

Just curious though, I have seem quite a number of posts in the past year mentioned the switch from Node to Go, is there a pattern here?Why Go in particular?

because you can be productive in go in a couple weeks, whereas you will have spend a couple months w/ java/scala/erlang before you are productive.

That's a bold assertion. I don't see any reason why Go would be easier to pick up than Java. Scala, sure, maybe then as it's a much more complex and powerful language.

But a few years ago I effectively had to start re-learning Java from scratch (I knew the language but the whole ecosystem had moved on from when I first learned it), and it certainly didn't take months to get productive.

Re: Gotchas from Two Years with Node

#102

Post by 15155 »

vosper wrote: post #87

Their first problem was sending customer requests directly into processing. They're collecting customer metrics - they should never be unable to handle a request because another request has consumed all the resources. A much better way to do this would be to have a very lightweight API putting events into a stream (Kafka or, since they seems to be on AWS, Kinesis [1]). Let the stream absorb that crazy customer data,…

Yep. This technology is too cheap/readily available to "do it wrong."

Let the Kafka project handle the nuances of durable, performant queueing - that's not your business model.

Re: Gotchas from Two Years with Node

#103

Post by Macha »

Kurtz79 wrote: post #22
mhogomchungu wrote: post #8

Earlier quoted context omitted.

> Why Go in particular? Its the new shiny thing.

I think go and node are more or less of the same "age" and "shininess"? EDIT: Very much so, they were actually released to the public in 2009.

Ruby was first released in the mid 90s and became the shiny new thing in around 2005 when ten years old.

Sometimes the shiny newness is in the public eye rather than absolute terms.

Re: Gotchas from Two Years with Node

#104

Post by Kiro »

taternuts wrote: post #99
Kiro wrote: post #96

OT but can someone explain this? PHP: Starts reading from database, blocks everything else, returns data. Node: Starts reading from database, lets it go and continues with other stuff until the data is ready and then returns it. I understand it in theory but since Node is single-threaded, doesn't it need to use that thread for the database operations? Which means it's blocking the program until it's done anyway?

I believe what happens is it registers the functions callback on the event-loop which it'll check the next time around to see if it's done, then pushes the work to be handled in a separate pool of threads that libuv manages. When it's done, the next event-loop tick or the next time around it will execute the callback with the results. This is how asynchronous functions work anyways, but some functions are actually sy…

Thanks. So Node is actually using multiple threads behind the scenes or did I read that wrong? How do I know which kind of operations are pushed there by libuv?

Re: Gotchas from Two Years with Node

#105

Post by juliangregorian »

glibgil wrote: post #95
xhrpost wrote: post #93

Earlier quoted context omitted.

> "ditching built-in streams and replacing them with something decent" What have you found sufficient for this?

They just told you: Bluebird and streams 3. Did you get bored and stop reading?

Ooh, swing and a miss. Bluebird is not a streams replacement, streams 3 IS the built-in streams. Maybe save the snark for situations where you actually have a clue.

Re: Gotchas from Two Years with Node

#106

Post by philbo »

spion wrote: post #89

All these problems are solvable (streams by ditching built-in streams and replacing them with something decent, errors with promises + typescript, event loop blocking with a streaming JSON parser). But that still means that out of the box node is a pretty unsatisfying experience all around. It took us a year to arrive at the solutions above, and many prominent members of the community scoffed at a number of them. Som…

> event loop blocking with a streaming JSON parser

Apologies for taking this opportunity to shamelessly plug my streaming JSON parser, which I recently open-sourced here:

https://github.com/philbooth/bfj

Re: Gotchas from Two Years with Node

#107

Post by nemothekid »

mercurial wrote: post #48
nemothekid wrote: post #36

Earlier quoted context omitted.

> Most of the Node.js vs Go arguments are weak. It's surprising that Node.js is still outpacing Go in popularity in spite of all this slander. Thats a rather defensive position for node in what is a very rare use case for the language. It's unsurprising Node.js is still outpacing Go, given the large number of JS developers and the fact that Go is pretty much worthless for hosting front end web applications (you won't…

> the issue is one thread was has too much work The underlying issue is that when you have a consumer-facing API which accepts HTTP requests with a body, the first thing you should think about is limits. > Consider the following - what if both processes got tied up? Do you just start another process? Would it feasible or wise to run 1000 processes (no it wont)? However this is a problem that you won't come across in…

>If you get 5 MB of JSON in N different requests (N=number of cores) at the same time, I don't see go generating free CPU time out of thin air.

You don't, but the scheduler normally won't allow one thread to completely starve the cpu. Of course, its clear they should be using limits, however JVM, glibc threads scheduler or Go's green threads likely wouldn't allow a single thread to completely starve the CPU, eventually the scheduler will step in and divert resources to another thread.

Without limits in a threaded solution, you would see the latency increase, but you wouldn't see the application stop taking requests altogether.

However there are real benefits for having an event loop concurrency, so this shouldn't be taken as a reason one model is strictly better than another.

Re: Gotchas from Two Years with Node

#108

Post by rvirding »

Why am I getting this strong deja vu feeling? Hmm. Oh, now I know! These were exactly the type of problems we were attacking, and solving, 25 years ago when we were creating Erlang. How you build concurrent, fault-tolerant and non-blocking systems with low-latency. And about 10 years ago there came implementations of Erlang which handled multi-core transparently and provided things like load balancing automatically.

I would say you would have to have really extreme requirements to handle those type of things yourself.

Re: Gotchas from Two Years with Node

#109

Post by wpietri »

jonpress wrote: post #78
wpietri wrote: post #69

Earlier quoted context omitted.

> In Node.js, it's really easy to identify process boundaries since the child_process module forces you to put code into different files and communicate via loosely coupled IPC channels. I find that approach troubling. Process boundaries are expensive, because you have to serialize and everything each time you cross a boundary. I also haven't used Go, but I think you can get great clarity with something like Akka's A…

Yes, socket/pipe-based IPC is more expensive than shared memory up to a point, but it's more scalable since you don't have to deal with locking (mutexes, semaphores) and the limits this imposes.

If you don't want to deal with locking but are willing to pay extra memory cost, then you can just duplicate the data. A re/de-serialization step is a much more expensive way to do that.

Re: Gotchas from Two Years with Node

#110

Post by wpietri »

Rapzid wrote: post #81
wpietri wrote: post #76

Earlier quoted context omitted.

> really easy to create an escape hatch by just making a function call or occasionally yielding to the scheduler... Having dealt with cooperative multitasking back in the dark ages, I definitely don't believe it is easy. With proper threads, you just write your code in a straightforward manner. With cooperative multitasking, you now have to be continuously imagining performance and sprinkling in otherwise useless cal…

Meh, it really doesn't come up that often. Every method call is an opportunity for the scheduler to run, not just specific ones. I find it is very rare indeed for a significant amount of work to be done without making method calls. Go also uses threads and makes parallel programming easy.. For me anyway.

Ah, I didn't realize that it happened at every method call. That seems more manageable. Thanks.
Post reply on HN