Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node.js in Flame Graphs

techblog.netflix.com

Post reply on HN
Print view View original

211–220 of 259 posts

Re: Node.js in Flame Graphs

#211

Post by tjholowaychuk »

Sounds like a documentation issue, or lack of a staging environment. I've written and maintained countless large Express applications and routing was never even remotely a bottleneck, thus the simple & flexible linear lookup. I believe we had an issue or two open for quite a while in case anyone wanted to report real use-cases that performed poorly.

Possibly worth mentioning, but there's really nothing stopping people from adding dtrace support to Express, it could easily be done with middleware. Switching frameworks seems a little heavy-handed for something that could have been a 20 minute npm module.

Re: Node.js in Flame Graphs

#212

Post by barrkel »

nostrademons wrote: post #187
akkartik wrote: post #163

Earlier quoted context omitted.

You're assuming that your closed source vendors are perfectly aligned with you. In practice they almost inevitably seem to cause capture ( https://en.wikipedia.org/wiki/Regulatory_capture ). Open/closed is a red herring here. Projects slowing down as they succeed seems to be a universal phenomenon, from startups to civilizations. Specialization leads to capture. I think almost exclusively about how to fix this: http:…

Yep, closed source doesn't solve the problem either. If you believe that just because you're paying money for someone to take responsibility for a problem, they will actually solve the problem in a way that's amenable to you...well, there are numerous closed-source software vendors looking to sell you something. In practice, the way to avoid this is to keep the software as simple as possible. Try to adjust to your us…

When a software provider I use starts ripping out features I relied upon, I start looking for an alternate provider, one that isn't so eager to kill features. And in particular, I try not to learn or rely on any new features, if there is past behaviour of feature removal by the provider.

It's better to be careful - very careful - about what you add, and to have a story for migration, than to remove features.

Re: Node.js in Flame Graphs

#213

Post by tjholowaychuk »

remon wrote: post #21

I wonder what the thought process was behind moving their web service stack (partially?) to node.js in the first place. For a company with the scale and resources of Netflix it's not exactly an obvious choice.

I share this thought, I'm not trolling, I really believe node is a bad solution for something like Netflix.

Node has its perks but for a money making machine that relies solely on being available and providing a good customer experience, not so much.

I can't imagine the ops nightmares at that size, one buggy code path and the entire cluster could be down. These are issues that drove me away from Node to Go, in my opinion Node has way too many issues to run in money-making scenarios.

Re: Node.js in Flame Graphs

#214

Post by rtpg »

quotemstr wrote: post #22
thedufer wrote: post #10

> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers. Its actually quite clear - most routes are defined by a regex rather than a string, so there is no built-in structure (if there's a way at all) to do O(1) lookups in the routing table. A router that only allowed string route definitions would be faster but far less useful. I can't explain away the recursion…

Of course there's a faster way! Combine all the routes into a DFA, then run the DFA over the URL. It's guaranteed to run in constant space and O(n) (n=URL length) time! The union of any set of regular languages is itself a regular language. You can use Ragel[1] to build your automaton. [1] http://www.colm.net/open-source/ragel/

This is the reason we need more people in JS land taking compiler courses and the like. There's a lot of tech out there that are speed sensitive yet do not apply the "best" solution to a problem solved in the 70s.

Re: Node.js in Flame Graphs

#215

Post by dmak »

mjr578 wrote: post #30
BradRuderman wrote: post #20

Why are they loading in routes from an external source? Is that normal, I have never seen that before.

We like the option of dynamically loading new routes, that point to new endpoints. We also have the ability to release new versions of our UI without redeploying (or restarting) our servers.

Could you elaborate in a bit on this? This is the first time I have heard of this, and I find it very interesting.

Re: Node.js in Flame Graphs

#216

Post by x0x0 »

nostrademons wrote: post #203
ChuckMcM wrote: post #196

Earlier quoted context omitted.

I agree with you that the greatest strength of open source is that you can just go read the code and (optionally) fix it. Netflix did this and it got them past their crisis. I think you missed my point though, which was that there isn't any sort of fitness function being applied to open source. In the closed source environment that it 'price' (caveat walled gardens). By paying for the software customers "vote with th…

There totally is a fitness function for open-source. People who find that an open-source program creates more hassles than it solves don't use it. Folks who blog about their hassles induce other people not to use it. There are a number of open-source projects - web.py, Mongo, Angular, Ember - that I am actively avoiding because IMHO their benefits to me don't outweigh risks or previous hassles experienced. Much bette…

   People who find that an open-source program creates more hassles than it 
   solves don't use it. Folks who blog about their hassles induce other people 
   not to use it.
Would that it were so, but people still use mongo. You seem to be positing perfect knowledge, whereas crummy software seems to creep in places either because of marketing or because it's the default in some system.

Docker, btw, is awesome. You can use it to replace virtualenv (which only handles python libs) and system lib isolation in a much more scriptable, more robustly deployable fashion. So you can take python projects developed at different times that use different revs of packages (and those packages can require different libs) and make deploying them way better. Try it, it's just awesome.

Re: Node.js in Flame Graphs

#217

Post by nostrademons »

x0x0 wrote: post #216
nostrademons wrote: post #203

Earlier quoted context omitted.

There totally is a fitness function for open-source. People who find that an open-source program creates more hassles than it solves don't use it. Folks who blog about their hassles induce other people not to use it. There are a number of open-source projects - web.py, Mongo, Angular, Ember - that I am actively avoiding because IMHO their benefits to me don't outweigh risks or previous hassles experienced. Much bette…

People who find that an open-source program creates more hassles than it solves don't use it. Folks who blog about their hassles induce other people not to use it. Would that it were so, but people still use mongo. You seem to be positing perfect knowledge, whereas crummy software seems to creep in places either because of marketing or because it's the default in some system. Docker, btw, is awesome. You can use it t…

The same problems with imperfect knowledge exist with closed-source software too, but they're worse because you can't examine the source yourself and the vendor often has an incentive not to let flaws come to light.

Re: Node.js in Flame Graphs

#218

Post by nostrademons »

barrkel wrote: post #212
nostrademons wrote: post #187

Earlier quoted context omitted.

Yep, closed source doesn't solve the problem either. If you believe that just because you're paying money for someone to take responsibility for a problem, they will actually solve the problem in a way that's amenable to you...well, there are numerous closed-source software vendors looking to sell you something. In practice, the way to avoid this is to keep the software as simple as possible. Try to adjust to your us…

When a software provider I use starts ripping out features I relied upon, I start looking for an alternate provider, one that isn't so eager to kill features. And in particular, I try not to learn or rely on any new features, if there is past behaviour of feature removal by the provider. It's better to be careful - very careful - about what you add, and to have a story for migration, than to remove features.

This depends on industry, of course - in consumer web it's much better to risk pissing off a few customers but make the majority of them happy than to keep all your existing customers but risk losing out on a new innovation that gives a competitor a toe-hold. Enterprise SaaS probably has different trade-offs, and software infrastructure probably different still.

This paradox, BTW, could be thought of as the full-employment theorem for entrepreneurs. As long as it is rational for a business to avoid change for fear of having to remove or support it later, then there will exist changes that a company with no customers and no codebase could implement that no incumbent would dare. Some of these are bound to be useful to some segment of the market, and that's why you get continued disruption in technology markets.

Re: Node.js in Flame Graphs

#219

Post by nmjohn »

tjholowaychuk wrote: post #213
remon wrote: post #21

I wonder what the thought process was behind moving their web service stack (partially?) to node.js in the first place. For a company with the scale and resources of Netflix it's not exactly an obvious choice.

I share this thought, I'm not trolling, I really believe node is a bad solution for something like Netflix. Node has its perks but for a money making machine that relies solely on being available and providing a good customer experience, not so much. I can't imagine the ops nightmares at that size, one buggy code path and the entire cluster could be down. These are issues that drove me away from Node to Go, in my opi…

> in my opinion Node has way too many issues to run in money-making scenarios.

You can say that about a lot of languages other than node. How many billion dollar companies have their software written in PHP - a language that many people would agree has far more glaring issues than node.

My point, is that I believe your comments miss the larger picture. There is far more involved with deciding which language to build a product with than "which language is best."

I don't disagree with you about Go being a great language - but for most companies, it is not even remotely practical to use. Hiring talented Go developers is hard because there are so few. Their current employees may or may not know Go, what do you do about that? Etc.

Re: Node.js in Flame Graphs

#220

Post by tjholowaychuk »

nmjohn wrote: post #219
tjholowaychuk wrote: post #213

Earlier quoted context omitted.

I share this thought, I'm not trolling, I really believe node is a bad solution for something like Netflix. Node has its perks but for a money making machine that relies solely on being available and providing a good customer experience, not so much. I can't imagine the ops nightmares at that size, one buggy code path and the entire cluster could be down. These are issues that drove me away from Node to Go, in my opi…

> in my opinion Node has way too many issues to run in money-making scenarios. You can say that about a lot of languages other than node. How many billion dollar companies have their software written in PHP - a language that many people would agree has far more glaring issues than node. My point, is that I believe your comments miss the larger picture. There is far more involved with deciding which language to build…

For sure, but jumping into Node from a clean slate is probably not the best idea, legacy is different.

If anything the comment on hiring highlights the use of Node being problematic. It's difficult to write robust systems in Node even as a seasoned Node developer.

Post reply on HN