Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node.js in Flame Graphs

techblog.netflix.com

Post reply on HN
Print view View original

31–40 of 259 posts

Re: Node.js in Flame Graphs

#31

Post by colanderman »

thedufer wrote: post #28
quotemstr wrote: post #22

Earlier quoted context omitted.

Of course there's a faster way! Combine all the routes into a DFA, then run the DFA over the URL. It's guaranteed to run in constant space and O(n) (n=URL length) time! The union of any set of regular languages is itself a regular language. You can use Ragel[1] to build your automaton. [1] http://www.colm.net/open-source/ragel/

Yeah, but what's n there? Isn't that going to be something like the sum of each route's length? That doesn't buy us anything (well, probably a smaller constant). Unless you go the NFA route, but I'm pretty sure that costs non-constant space.

No. n in the case of a DFA is the length of the input string. (i.e. the string being matched.)

So, in terms of the GP's post, yes, this is an O(1) (with respect to number of routes) solution.

Re: Node.js in Flame Graphs

#32

Post by StevePerkins »

joshuacc wrote: post #25
remon wrote: post #21

I wonder what the thought process was behind moving their web service stack (partially?) to node.js in the first place. For a company with the scale and resources of Netflix it's not exactly an obvious choice.

They went into this a bit on the NodeUp podcast: http://nodeup.com/seventyone

Are there some salient points that can be summarized here? That's an hour-and-twenty-minute podcast episode.

Re: Node.js in Flame Graphs

#33

Post by personZ »

remon wrote: post #21

I wonder what the thought process was behind moving their web service stack (partially?) to node.js in the first place. For a company with the scale and resources of Netflix it's not exactly an obvious choice.

Netflix seems to operate like a tech start-up that is trying to glue together a ragtag collections of often unsuitable solutions because of limited funding. It is a deeply perplexing company.

Similar is LinkedIn, as an aside -- despite being fairly formidable now, I regularly have entire feeds disappear, their caching is abhorrent, they can't markup text properly, and so on. It seems very amateur hour, yet they regularly publish "how it's done" documents that see wide applause despite often completely contradicting their prior missives.

Re: Node.js in Flame Graphs

#35

Post by thedufer »

colanderman wrote: post #31
thedufer wrote: post #28

Earlier quoted context omitted.

Yeah, but what's n there? Isn't that going to be something like the sum of each route's length? That doesn't buy us anything (well, probably a smaller constant). Unless you go the NFA route, but I'm pretty sure that costs non-constant space.

No. n in the case of a DFA is the length of the input string. (i.e. the string being matched.) So, in terms of the GP's post, yes, this is an O(1) (with respect to number of routes) solution.

Oh, right. My CS has gotten fuzzy - the tree gets enormous, but the runtime stays is O(n) on input length. Thanks for the explanation.

Re: Node.js in Flame Graphs

#36

Post by andrewvc »

thedufer wrote: post #10

> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. Its actually quite clear - most routes are defined by a regex rather than a string, so there is no built-in structure (if there's a way at all) to do O(1) lookups in the routing table. A router that only allowed string route definitions would be faster but far less useful. I can't explain away the recursion…

A lot of people here are right, the right way is with an NFA. I just want to add that the solution is not even hard, you can do it with string concatenation and capture groups using regexps. Regexps are NFAs, and are highly optimized C code in just about every JS engine.

If I have the routes /foo/bar and /foo/bar/(\d+) I can generate the regexp ((^\/foo\/bar$)|(^\/foo\/bar\/\d+$))

I'm not at all surprised, the quality of software in node is pretty low, I've seen numerous issues in node libs being just as boneheaded. I swear, the fact that the express devs overlooked a key optimization is crazy. Rails, by way of example, uses the Journey engine to solve this problem (https://github.com/rails/journey)

Re: Node.js in Flame Graphs

#37

Post by msmillie »

cataphract wrote: post #18
thedufer wrote: post #10

> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. Its actually quite clear - most routes are defined by a regex rather than a string, so there is no built-in structure (if there's a way at all) to do O(1) lookups in the routing table. A router that only allowed string route definitions would be faster but far less useful. I can't explain away the recursion…

On the other hand, without the recursion I don't see how the flame graph would have been helpful.

In flame graphs heavy iteration tends to give "wider" structures. Recall the stacks aren't ordered on the X axis by time, but by contents.

Re: Node.js in Flame Graphs

#38

Post by M2Ys4U »

gadders wrote: post #19
quarterto wrote: post #5

Earlier quoted context omitted.

"learnings" is a perfectly cromulent word: https://books.google.com/ngrams/graph?content=learnings&year...

Nope. You see an option for a plural here? http://www.merriam-webster.com/dictionary/learning

Dictionaries are descriptive, not prescriptive. They can only tell you if something is a word, not whether something isn't a word.

Re: Node.js in Flame Graphs

#39

Post by yourad_io »

remon wrote: post #21

I wonder what the thought process was behind moving their web service stack (partially?) to node.js in the first place. For a company with the scale and resources of Netflix it's not exactly an obvious choice.

What are the arguments against node.js in their use case?

Not looking to start any wars, but I was under the impression that if you know what you're doing* node.js is pretty awesome.

This particular bug had to do with a misunderstanding regarding the express API.

* for the most part: understand async and closures/memory leaks.

Re: Node.js in Flame Graphs

#40

Post by cdelsolar »

gadders wrote: post #14
mikeryan wrote: post #6

Earlier quoted context omitted.

OFFTOPIC: "Today, I want to share some recent learnings from performance tuning this new application stack." The word you want is "lessons". Jeez, not only is your comment offtopic and pedantic, it's also wrong.

I flagged it as Offtopic. But it is correct. "Learnings" is at best a pointless affectation, like saying "utilised" instead of "used". At worst, according to the OED, it's not even a word. http://english.stackexchange.com/questions/19227/plural-of-l... Also: http://www.merriam-webster.com/dictionary/learning No plural option.

I play tournament Scrabble and learnings is a valid word in it.
Post reply on HN