Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node.js in Flame Graphs

techblog.netflix.com

Post reply on HN
Print view View original

171–180 of 259 posts

Re: Node.js in Flame Graphs

#171

Post by bcoates »

It's not just the extra lookups -- static in express is deceptively dog-slow. For every request it processes, it stats every filename that might satisfy the URL. This results in an enormous amount of useless syscall/IO overhead. This bit me pretty hard on a high-throughput webservice endpoint with an unnoticed extra static middleware. I wound up catching it with the excellent NodeTime service.

Now that I look at it, there's a TOCTOU bug on the fstat/open callback, too: https://github.com/tj/send/blob/master/index.js#L570-L605

This should be doing open-then-fstat, not stat-then-open.

Re: Node.js in Flame Graphs

#172

Post by drawkbox »

akkartik wrote: post #163
ChuckMcM wrote: post #130

The moneyquote: "We made incorrect assumptions about the Express.js API without digging further into its code base. As a result, our misuse of the Express.js API was the ultimate root cause of our performance issue." This situation is my biggest challenge with software these days. The advice to "just use FooMumbleAPI!" is rampant and yet the quality of the implemented APIs and the amount of review they have had varie…

You're assuming that your closed source vendors are perfectly aligned with you. In practice they almost inevitably seem to cause capture ( https://en.wikipedia.org/wiki/Regulatory_capture ). Open/closed is a red herring here. Projects slowing down as they succeed seems to be a universal phenomenon, from startups to civilizations. Specialization leads to capture. I think almost exclusively about how to fix this: http:…

I agree this is just part of the software iteration game. This has been Microsoft's modus operandi since they started. Microsoft still is with getting people to move to Azure/mobile, the cycling is just happening further up the stack.

Some of the iterations are good, some are just for control, mostly though it is to keep developers locked in and chasing rather than innovating. Software has to change and iterate though, but too slow or too fast can be harmful or the worthiness of iterations varies when platforms change.

On the topic of Node.js and Express though I think it is production solid and it is very fast (faster than php, ruby, python). I love it and I think here Netflix developers are at fault for not testing something a bit outside of the express default usage that is tested thoroughly. Their solution of changing to Restify so quick on a problem probably tells you how they got into this problem even if Restify is indeed better for this purpose. With any new change comes research/testing/possible problems open source or closed.

In the end this is why microframeworks, which both of the node frameworks are, do win out as they are easier to inspect and live on after the hype (monolithic frameworks not so much).

Re: Node.js in Flame Graphs

#173

Post by [deleted] »

tedchs wrote: post #44
vkjv wrote: post #4

> ...as well as increasing the Node.js heap size to 32Gb. > ...also saw that the process’s heap size stayed fairly constant at around 1.2 Gb. This is because 1.2 GB is the max allowed heap size in v8. Increasing beyond this value has no effect. > ...It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. It it is non-trivial (not possible?) to do this in O(1) for…

That seems like a pretty low size to me... how are people getting around this when they need to handle >1.2GB of data on Node?

[deleted]

Re: Node.js in Flame Graphs

#174

Post by spion »

andrewvc wrote: post #36
thedufer wrote: post #10

> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. Its actually quite clear - most routes are defined by a regex rather than a string, so there is no built-in structure (if there's a way at all) to do O(1) lookups in the routing table. A router that only allowed string route definitions would be faster but far less useful. I can't explain away the recursion…

A lot of people here are right, the right way is with an NFA. I just want to add that the solution is not even hard, you can do it with string concatenation and capture groups using regexps. Regexps are NFAs, and are highly optimized C code in just about every JS engine. If I have the routes /foo/bar and /foo/bar/(\d+) I can generate the regexp ((^\/foo\/bar$)|(^\/foo\/bar\/\d+$)) I'm not at all surprised, the qualit…

This indicates it probably wont be much faster in JS

http://jsperf.com/regex-list-vs-full

Most of the time in the single-regex case is likely spent on allocating the matches array (which for a router with 300 routes would have over 300 undefined elements on every execution)

V8 is not quite the typical dynamic language runtime. Code that would be dead slow in others is optimized really well by the amazing JIT.

Re: Node.js in Flame Graphs

#175

Post by andrewvc »

spion wrote: post #174
andrewvc wrote: post #36

Earlier quoted context omitted.

A lot of people here are right, the right way is with an NFA. I just want to add that the solution is not even hard, you can do it with string concatenation and capture groups using regexps. Regexps are NFAs, and are highly optimized C code in just about every JS engine. If I have the routes /foo/bar and /foo/bar/(\d+) I can generate the regexp ((^\/foo\/bar$)|(^\/foo\/bar\/\d+$)) I'm not at all surprised, the qualit…

This indicates it probably wont be much faster in JS http://jsperf.com/regex-list-vs-full Most of the time in the single-regex case is likely spent on allocating the matches array (which for a router with 300 routes would have over 300 undefined elements on every execution) V8 is not quite the typical dynamic language runtime. Code that would be dead slow in others is optimized really well by the amazing JIT.

Interesting, I'm looking at it right now, and it's saying that the multiple regexes are ~ 50% slower in both chrome and firefox. Am I missing something? It looks like the single regex actually is a significant win.

It is interesting how much allocation changes things though, very cool that you made this.

Re: Node.js in Flame Graphs

#176

Post by spion »

andrewvc wrote: post #61
mjs wrote: post #54

Earlier quoted context omitted.

It looks like routes are either strings (exact match) or regular expression objects. I don't think there's a way to combine regular expressions in JavaScript. I don't think this was an unreasonable implementation. Sure, it could be optimized, but no-one needed it enough to do the work. (Even Netflix didn't need a faster matcher, they needed a matcher that didn't slow down between the 1st match and the nth.)

What I'm saying is that the pipe (or) "|" is a cheap way to combine regexps. It has its limitations, but its better than an O(n) search through an array. The regexp parser can create an NFA that is far more optimized.

In JS, "far more" isn't quite that dramatically far as in other dynamic language runtimes. See http://jsperf.com/regex-list-vs-full

Re: Node.js in Flame Graphs

#177

Post by spion »

andrewvc wrote: post #175
spion wrote: post #174

Earlier quoted context omitted.

This indicates it probably wont be much faster in JS http://jsperf.com/regex-list-vs-full Most of the time in the single-regex case is likely spent on allocating the matches array (which for a router with 300 routes would have over 300 undefined elements on every execution) V8 is not quite the typical dynamic language runtime. Code that would be dead slow in others is optimized really well by the amazing JIT.

Interesting, I'm looking at it right now, and it's saying that the multiple regexes are ~ 50% slower in both chrome and firefox. Am I missing something? It looks like the single regex actually is a significant win. It is interesting how much allocation changes things though, very cool that you made this.

Yeah multiple regexes are slower, but the performance gain isn't too gigantic. If we had `regex.execIterable(string)` we could avoid allocating massive arrays and maybe get a really significant difference

Re: Node.js in Flame Graphs

#178

Post by andrewstuart2 »

quotemstr wrote: post #22
thedufer wrote: post #10

> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. Its actually quite clear - most routes are defined by a regex rather than a string, so there is no built-in structure (if there's a way at all) to do O(1) lookups in the routing table. A router that only allowed string route definitions would be faster but far less useful. I can't explain away the recursion…

Of course there's a faster way! Combine all the routes into a DFA, then run the DFA over the URL. It's guaranteed to run in constant space and O(n) (n=URL length) time! The union of any set of regular languages is itself a regular language. You can use Ragel[1] to build your automaton. [1] http://www.colm.net/open-source/ragel/

This approach will of course work, but you can't have a middleware stack with a defined order using that approach, unless I'm mistaken.

Sure, you could use all routes that match, but is there a way to specify the order for all handlers, and whether or not you should continue after one handler is done?

Re: Node.js in Flame Graphs

#179

Post by dagw »

gaving wrote: post #147
ecaron wrote: post #12

My biggest takeaway from this article is that Netflix is moving from Express to Restify, and I look forward to watching the massive uptick this has on https://github.com/mcavage/node-restify/graphs/contributors

not exactly highly endorsed https://twitter.com/tjholowaychuk/status/382084838179614721

While I kind of agree the restify gives you "almost nothing" when compared to some larger frameworks, in my experience "almost nothing" is basically all I need from my framework when writing REST APIs and restify has had me covered every time.

Re: Node.js in Flame Graphs

#180

Post by remon »

yourad_io wrote: post #77
remon wrote: post #67

Earlier quoted context omitted.

Why are these arguments in favor of Node.js over their current Java based stack? Java is both considerably more speedy for this sort of workload and there's a significantly larger pool of high end Java server developers compared to high end JavaScript server developers. Budget and brand recognition are not necessarily in favor of node.js over other stacks. That simply helps with getting the better engineers in genera…

They are arguments in favor of node.js being a reasonable tool for the job. > Java is both considerably more speedy for this sort of workload They're quoting 1ms latencies, that's pretty speedy. Maybe Java could do > there's a significantly larger pool of high end Java server developers compared to high end JavaScript server developers Oh, I would have thought there are far more JS* folks out there than Java. > Budge…

I'm not sure if I agree. Responding after 1ms is not the same (or even very much related to) routing a request taking up 1ms of cpu time. Although there are a ton of nuances that make the following a small oversimplification it does basically mean that purely for routing alone you have a hard cap on 1k lookups/sec/core. And building server software in any language takes a whole different set of skills and knowledge than knowing how to build front ends. I'm not arguing one is harder or more involved than the other but they are different enough to make language and paradigm familiarity and the only real gain when moving from front-end JS to node.
Post reply on HN