Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node.js in Flame Graphs

techblog.netflix.com

Post reply on HN
Print view View original

61–70 of 259 posts

Re: Node.js in Flame Graphs

#61

Post by andrewvc »

mjs wrote: post #54
andrewvc wrote: post #36

Earlier quoted context omitted.

A lot of people here are right, the right way is with an NFA. I just want to add that the solution is not even hard, you can do it with string concatenation and capture groups using regexps. Regexps are NFAs, and are highly optimized C code in just about every JS engine. If I have the routes /foo/bar and /foo/bar/(\d+) I can generate the regexp ((^\/foo\/bar$)|(^\/foo\/bar\/\d+$)) I'm not at all surprised, the qualit…

It looks like routes are either strings (exact match) or regular expression objects. I don't think there's a way to combine regular expressions in JavaScript. I don't think this was an unreasonable implementation. Sure, it could be optimized, but no-one needed it enough to do the work. (Even Netflix didn't need a faster matcher, they needed a matcher that didn't slow down between the 1st match and the nth.)

What I'm saying is that the pipe (or) "|" is a cheap way to combine regexps. It has its limitations, but its better than an O(n) search through an array. The regexp parser can create an NFA that is far more optimized.

Re: Node.js in Flame Graphs

#63

Post by dugmartin »

debacle wrote: post #11

Doesn't this seem like a bug in the express router? All of the additional routes in the array are dead (can't be routed to).

No, because the route list is also used for middleware. The recursive search is because middleware routes have a third next parameter that allows the search to run asynchronously.

Re: Node.js in Flame Graphs

#64

Post by not_kurt_godel »

gadders wrote: post #19
quarterto wrote: post #5

Earlier quoted context omitted.

"learnings" is a perfectly cromulent word: https://books.google.com/ngrams/graph?content=learnings&year...

Nope. You see an option for a plural here? http://www.merriam-webster.com/dictionary/learning

https://books.google.com/ngrams/graph?content=learnings&year...

Re: Node.js in Flame Graphs

#65

Post by meddlepal »

eikenberry wrote: post #59
general_failure wrote: post #49

Earlier quoted context omitted.

Netflix overhires quite a bit. They also pay their (very good) engineers very high salaries (possibly highest in the valley) which results is very low attrition. The net result is a lot of engineers having a lot of time and all these engineers experiment away on technology. In fact, they have quite a bit of NIH syndrome internally because the engineers have nothing to do.

I don't think you have those reasons in proper order. I think the ability of the engineers to experiment and use new tools is why they have the high level of retention. I've seen plenty of places that pay much higher than the competition yet have very high turnover due to the a conservative culture which dictates the tools and doesn't encourage experimentation.

Experimentation is good, but perhaps there is too much going on at some companies? Finding the right balance is a hard problem.

Re: Node.js in Flame Graphs

#66

Post by aturek »

general_failure wrote: post #49
personZ wrote: post #33

Earlier quoted context omitted.

Netflix seems to operate like a tech start-up that is trying to glue together a ragtag collections of often unsuitable solutions because of limited funding. It is a deeply perplexing company. Similar is LinkedIn, as an aside -- despite being fairly formidable now, I regularly have entire feeds disappear, their caching is abhorrent, they can't markup text properly, and so on. It seems very amateur hour, yet they regul…

Netflix overhires quite a bit. They also pay their (very good) engineers very high salaries (possibly highest in the valley) which results is very low attrition. The net result is a lot of engineers having a lot of time and all these engineers experiment away on technology. In fact, they have quite a bit of NIH syndrome internally because the engineers have nothing to do.

Curious, do you have some anecdotes about that? I've generally heard from coworkers who were at Netflix that they have pretty high attrition at the lower levels, with the "fire the bottom 10% every X months" rules.

Re: Node.js in Flame Graphs

#67

Post by remon »

yourad_io wrote: post #51
yourad_io wrote: post #39

Earlier quoted context omitted.

What are the arguments against node.js in their use case? Not looking to start any wars, but I was under the impression that if you know what you're doing* node.js is pretty awesome. This particular bug had to do with a misunderstanding regarding the express API. * for the most part: understand async and closures/memory leaks.

To give some arguments in favour: - "Here we see our latencies drop down to 1 ms and remain there after we deployed our fix." - They can hire from a vastly larger pool of developers. Since they've got both budget and brand recognition, they should have no problems hiring high end JS guys & gals.

Why are these arguments in favor of Node.js over their current Java based stack? Java is both considerably more speedy for this sort of workload and there's a significantly larger pool of high end Java server developers compared to high end JavaScript server developers. Budget and brand recognition are not necessarily in favor of node.js over other stacks. That simply helps with getting the better engineers in general.

Re: Node.js in Flame Graphs

#68

Post by thedufer »

andrewvc wrote: post #36
thedufer wrote: post #10

> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. Its actually quite clear - most routes are defined by a regex rather than a string, so there is no built-in structure (if there's a way at all) to do O(1) lookups in the routing table. A router that only allowed string route definitions would be faster but far less useful. I can't explain away the recursion…

A lot of people here are right, the right way is with an NFA. I just want to add that the solution is not even hard, you can do it with string concatenation and capture groups using regexps. Regexps are NFAs, and are highly optimized C code in just about every JS engine. If I have the routes /foo/bar and /foo/bar/(\d+) I can generate the regexp ((^\/foo\/bar$)|(^\/foo\/bar\/\d+$)) I'm not at all surprised, the qualit…

> I can generate the regexp ((^\/foo\/bar$)|(^\/foo\/bar\/\d+$))

And how would you know which one got matched? The regex match isn't going to tell you that. Also, it needs to recognize if multiple were matched, which is definitely not going to be done by the built-in regex matcher.

It's certainly possible, but pretending it's trivial isn't helping, either.

Re: Node.js in Flame Graphs

#69

Post by hit8run »

I would have written my apis in golang and not nodejs. Go is way faster in my experience and it feels leaner to create something because creating a web service can be productively doneout of box. Node apps tend to depend on thousands of 3rd party dependencies which makes the whole thing feel fragile to me.
Post reply on HN