Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node.js in Flame Graphs

techblog.netflix.com

Post reply on HN
Print view View original

131–140 of 259 posts

Re: Node.js in Flame Graphs

#131

Post by andrewvc »

breuleux wrote: post #119
andrewvc wrote: post #103

Earlier quoted context omitted.

Ah, I did miss that, but it does still work. Run the following code below, you'll see multiple groups match. "/foo/bar/3".match(/((^\/foo\/bar\/.*)|(^\/foo\/bar\/(\d+)$)|(^\/baz))/) => Additionally, parsing out the number of capture groups in a regexp is simple, just looked for unescaped paren groups. You can do it once at route definition.

Actually, no: if you run this you will see three matching groups, but the first is always the whole match (this is in addition to the capture groups that are defined), the second is the (redundant) capture group you put around the whole thing, and the third is foo/bar/.*. /foo/bar/\d+ is not matched at all, even though it matches the input string.

You are 100% correct, I forgot about that extra group (that doesn't need to be there).

That being said, most frameworks don't have multiple matching routes, and I can't imagine that many people miss them. I've never used a framework that supported that, and I can't say I've ever felt I've missed out. Middleware is the right approach to that problem generally.

Re: Node.js in Flame Graphs

#132

Post by mjr578 »

BradRuderman wrote: post #108
mjr578 wrote: post #30

Earlier quoted context omitted.

We like the option of dynamically loading new routes, that point to new endpoints. We also have the ability to release new versions of our UI without redeploying (or restarting) our servers.

Ok you add a new route but how do you reference what code should be executed when that route is hit?

We have something that loads up, via requires, the action (or route) that should be run when a URL is encountered.

Re: Node.js in Flame Graphs

#133

Post by jonny_eh »

tedchs wrote: post #44
vkjv wrote: post #4

> ...as well as increasing the Node.js heap size to 32Gb. > ...also saw that the process’s heap size stayed fairly constant at around 1.2 Gb. This is because 1.2 GB is the max allowed heap size in v8. Increasing beyond this value has no effect. > ...It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. It it is non-trivial (not possible?) to do this in O(1) for…

That seems like a pretty low size to me... how are people getting around this when they need to handle >1.2GB of data on Node?

Native code modules I assume.

Re: Node.js in Flame Graphs

#134

Post by jonny_eh »

ecnahc515 wrote: post #57
sadkingbilly wrote: post #42

Earlier quoted context omitted.

Yes, but their original bug was from dynamically loading routes from an external source. I don't see how Express is to blame for this. Moving to Restify is not a solution, but they state having different reasons for moving (support for bunyan logging? But Express already supports this too).

the bug was related to dynamically loading routes, but the true cause was that express allowed duplicate handlers. They were loading routes dynamically correctly, that wasn't a problem, it was that when doing that express let them duplicate routes.

If they were loading duplicate routes, how was it they were "loading routes dynamically correctly"?

Re: Node.js in Flame Graphs

#135

Post by mjr578 »

codelucas wrote: post #58

> This turned out be caused by a periodic (10/hour) function in our code. The main purpose of this was to refresh our route handlers from an external source. This was implemented by deleting old handlers and adding new ones to the array. Unfortunately, it was also inadvertently adding a static route handler with the same path each time it ran. I don't understand the need of refreshing route handlers. Could someone ex…

We refresh periodically as we dynamically deploy new UI code, which can be accessed at new routes. (/home and /homeV2 for example) This allows us to not have to restart our servers or push out new server code just to serve a new UI at a different (or the same) route.

Re: Node.js in Flame Graphs

#136

Post by thedufer »

andrewvc wrote: post #131
breuleux wrote: post #119

Earlier quoted context omitted.

Actually, no: if you run this you will see three matching groups, but the first is always the whole match (this is in addition to the capture groups that are defined), the second is the (redundant) capture group you put around the whole thing, and the third is foo/bar/.*. /foo/bar/\d+ is not matched at all, even though it matches the input string.

You are 100% correct, I forgot about that extra group (that doesn't need to be there). That being said, most frameworks don't have multiple matching routes, and I can't imagine that many people miss them. I've never used a framework that supported that, and I can't say I've ever felt I've missed out. Middleware is the right approach to that problem generally.

The point is to be able to have middleware that only runs on certain urls. For example, you might have middleware that only runs on a particular version of your API (`regexp: ^/1\/.*/`). I've used it fairly extensively.

The point being, now you're describing a completely different, less powerful framework. Which, yes, of course it's faster. You'll notice that's what Netflix ended up doing - they moved to `restify` from `express`.

You might as well just only allow routes to be strings and then you have the constant-time map implementation that the article describes as the expected implementation.

Re: Node.js in Flame Graphs

#137

Post by thedufer »

andrewvc wrote: post #103
thedufer wrote: post #92

Earlier quoted context omitted.

> First off, you don't need to handle the 'multiple' case, since the "|" has precedence rules applied to it. Read my top-level post again - multiple routes can be called on the same request, so express has to be able to find all of the matches, not just the first one. This is a mistake in the original article, as the author doesn't appear to understand the power of express routers. > Second, you know which one matche…

Ah, I did miss that, but it does still work. Run the following code below, you'll see multiple groups match. "/foo/bar/3".match(/((^\/foo\/bar\/.*)|(^\/foo\/bar\/(\d+)$)|(^\/baz))/) => Additionally, parsing out the number of capture groups in a regexp is simple, just looked for unescaped paren groups. You can do it once at route definition.

> just looked for unescaped paren groups

Don't forget non-matching groups! Which brings up my point yet again - everyone is pretending this is a dead-simple problem to solve, but all of the proposed solutions are buggy. Not really an option for such a widely-used library.

I'm not claiming it's impossible. I'm claiming it's more difficult than anyone is willing to admit.

Re: Node.js in Flame Graphs

#138

Post by darkandbrooding »

ChuckMcM wrote: post #130

The moneyquote: "We made incorrect assumptions about the Express.js API without digging further into its code base. As a result, our misuse of the Express.js API was the ultimate root cause of our performance issue." This situation is my biggest challenge with software these days. The advice to "just use FooMumbleAPI!" is rampant and yet the quality of the implemented APIs and the amount of review they have had varie…

Please forgive me if I'm misinterpreting you, but the lament you make about open source software seems to me to be more about distributed systems. It is an interesting observation, and now that you've pointed it out I can observe the pattern at past and present employers. But I have seen that pattern on internal software, written by the company for the company. This makes me think the problem is an architectural one.

I hesitate to comment about the relative velocities of open source vs proprietary software because I do not have enough experience with commercial, third party software. My sample size is too small, but I'm inclined to agree with you.

I don't disagree with your Machiavellian conspiracy, either, but I've worked in marketing and advertising so I know that some of the villains are on the payroll. Maybe there needs to be a third category? There's open source software, written by someone who has no particular relationship to you. There's commercial software, written by someone who has a positive economic relationship with you. And then there is ... corporate?... software, written by someone who might think they have a zero sum relationship with you.

Re: Node.js in Flame Graphs

#139

Post by [deleted] »

ChuckMcM wrote: post #130

The moneyquote: "We made incorrect assumptions about the Express.js API without digging further into its code base. As a result, our misuse of the Express.js API was the ultimate root cause of our performance issue." This situation is my biggest challenge with software these days. The advice to "just use FooMumbleAPI!" is rampant and yet the quality of the implemented APIs and the amount of review they have had varie…

[deleted]
Post reply on HN