Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node.js in Flame Graphs

techblog.netflix.com

Post reply on HN
Print view View original

191–200 of 259 posts

Re: Node.js in Flame Graphs

#191

Post by escape_goat »

TazeTSchnitzel wrote: post #152
elwell wrote: post #48

TIL, SVG's can display labels on element hover: http://cdn.nflximg.com/ffe/siteui/blog/yunong/200mins.svg Nice, contained way to show data like this.

SVGs can contain ECMAScript, video, canvases, animation, all sorts of things. They're the replacement for Flash that nobody seems to use.

Well, there was a good reason for that, for a long time: IE didn't even begin to support .svg until version 9, which means that it isn't a realistic option for deployment for anyone who needs to support pre-evergreen browsers.

I investigated them a year or two or so ago, because I found that AngularJS could work quite nicely within an .svg document, which opened up some exciting possibilities. My recollection is that at the time, there were some critical cross-browser problems with font rendering that made the topic very kludgy and complicated. I except that matters have improved since, but I do not know to what extent.

Re: Node.js in Flame Graphs

#192

Post by scoates »

chx wrote: post #169
ChuckMcM wrote: post #130

The moneyquote: "We made incorrect assumptions about the Express.js API without digging further into its code base. As a result, our misuse of the Express.js API was the ultimate root cause of our performance issue." This situation is my biggest challenge with software these days. The advice to "just use FooMumbleAPI!" is rampant and yet the quality of the implemented APIs and the amount of review they have had varie…

This is why I like choosing open source technologies with some sort of commercial support available. The best support I've ever gotten was from MySQL ab. (before Sun) -- the 10k we paid them for two years and three servers was affordable back then even for a small startup. I had a MySQL engineer (if memory serves he is now a MySQL community manager at Oracle) SSH into my server 34 minutes after a desperate call. Disc…

Morgan definitely knows what he's doing. (-;

Re: Node.js in Flame Graphs

#194

Post by bcoates »

remon wrote: post #181
bcoates wrote: post #89

Earlier quoted context omitted.

I'm a non-frontend dev who uses node.js for server/systems/embedded work. My preference for node is based primarily on its sane concurrency model. I basically use it as a handy scripting language for doing rapid prototyping of libuv programs. My on-paper plan is to fall back to libuv and C if I find myself really stuck for CPU, but this is something that's never actually happened to me. (I have had to abandon or modi…

I'm not disagreeing and that's a perfectly valid reason to go for node for smaller projects or for prototyping. What many people fail to realise is that the "concurrency" model (it is solved by simply not having concurrency) isn't a magic bullet. It comes with large drawbacks. Not a lot of things are complicated because people like it that way. Some things are as complicated as they need to be to be used effectively.…

An example of basic concurrency I have great difficulty doing correctly in Java:

  n = 0;
  avg = 0;

  function twothings() {
    thing1(function(ret) { ++n; avg += (ret - avg)/n; });
    thing2(function(ret) { ++n; avg += (ret - avg)/n; });
  }
I want to dispatch two I/O operations to run concurrently and have their completions modify some shared state in a serializable fashion, which can be read at any stage in the process and have a valid result. I shouldn't need to give up on nondeterministic simultaneous execution or faff about with locking mechanisms to get this.

Re: Node.js in Flame Graphs

#196

Post by ChuckMcM »

nissimk wrote: post #155
ChuckMcM wrote: post #130

The moneyquote: "We made incorrect assumptions about the Express.js API without digging further into its code base. As a result, our misuse of the Express.js API was the ultimate root cause of our performance issue." This situation is my biggest challenge with software these days. The advice to "just use FooMumbleAPI!" is rampant and yet the quality of the implemented APIs and the amount of review they have had varie…

Actually, open source allows something that is not possible with commercial software. You don't have to read all of the code and understand all of the implementation details, but you have that option if you need to. I think having the option is a great benefit as we can see from this example. When they started debugging they could reference the express source and figure out what was going on. If they were using some…

I agree with you that the greatest strength of open source is that you can just go read the code and (optionally) fix it. Netflix did this and it got them past their crisis.

I think you missed my point though, which was that there isn't any sort of fitness function being applied to open source. In the closed source environment that it 'price' (caveat walled gardens). By paying for the software customers "vote with their wallet" on the things they like/want/use. What is more if two folks are putting out the same capability they are incentivised to compete on a vague sense of value. That creates a 'culling' function which operates independently of the software creators (the only way to sell more is to be "better").

Open source doesn't have that forcing function, there isn't really even a reputation function involved where bad FOSS would give folks a bad reputation that a user could pick up on (yes we probably all know one or more committers who are jerks but that isn't quite the same thing). And there is no barrier to entry so we get multiple variations on the same conceptual solution (I've lost count of the number of CMS's or blogging packages there are out there for example).

Finally there is the various bits of entanglement, you wrote "And with regard to keeping up with changes, you always can remain with previous versions for some time to avoid the slowdown associated with shifting to new APIs." but that has not been my experience. As you're evolving your product interdependencies in the APIs and packages you are using force them to upgrade as well. There is no island of stability in the FOSS world (well maybe OpenBSD servers or something but I have not seen them in the 'stack') Backports only go on so long and suddenly you're no longer getting updates from your source as Ubuntu showed when they dropped updates for 12.x LTS. All those systems still shell shockable or heart bleedable, or something else. Because to upgrade that part, needs to upgrade the next part, and so on and so on until you've upgraded everything. And you pay that price again and again and again and again.

Much of this is just different than the other model which is not necessarily a bad model. But from a systemic point of view it is hard to partition the work and that leads to inefficiencies. A single change in an API implementation which results in dozens of engineering groups re-implementing stuff, versus a migration model that allows people to move over gradually. The promise of open source is "no capital expense" but I worry the down side will become "much higher operational expense". And when the opex cost of open source is > than the capex + opex cost of closed software, then closed software wins. Of the folks who have tried to prevent that equation from shifting in favor of closed source, I only see Redhat as a success story.

Re: Node.js in Flame Graphs

#197

Post by runT1ME »

nostrademons wrote: post #188
runT1ME wrote: post #186

Earlier quoted context omitted.

>What this means in practice is that companies that use open source extensively in their operation, become slower and slower to innovate as they are carrying the weight of a thousand different systems of checks on code quality and robustness, which people using closed source will start delivering faster and faster as they effectively partition the review/quality question to the person selling them the software and th…

It's funny, my first job out of college was at a startup with an ex-Sun CTO, and he insisted on a single monolithic codebase. After he left I tried to organize an (aborted) project to modularize the codebase, since many of the other engineers ran into it as well. Having worked at Google in the interim, and having a number of acquaintances at Microsoft (which uses the multiple-repository approach) I can see the pros a…

>When you have a number of interdependent modules, then every change request and new feature has to go through that module's owner. If it's not their top priority (and it won't be), then getting your work done suddenly has a hard dependency on a team who is...generally pretty unresponsive, at least from your POV.

Is that true? Again, I've never done development on that scale, so I could totally be wrong, but I'm seeing this a problem with how changes are handled and releases are published.

I do work on some Open Source projects that are used by some big companies, the project has very good reliability because of CI, specification testing, perf testing for regression to the point that merging in minor bug fixes and pushing a minor release is pretty trivial.

Let's say only some of the teams practice good module management, ok fine, you fork the repo, fix the changes, publish your fork as your new dependency and move along on your way.

Does this take discipline? Absolutely, but it severely reduces complexity in the code. It mandates a certain level of pureness by forbidding certain dependencies in the vast majority of the codebase. The upsides are tremendous when compared to the slight inconvenience of fixing the occasional bug in someone else's module.

Re: Node.js in Flame Graphs

#198

Post by [deleted] »

koenigdavidmj wrote: post #185
rudolf0 wrote: post #183

Earlier quoted context omitted.

I use RE2 and currently achieve this with regular string concatenation. Do you know if they provide an API for doing this without concatenation, and if it would be faster?

The header file is available at https://code.google.com/p/re2/source/browse/re2/set.h , and if I recall correctly, it merges all the regexes into one DFA so that it's faster than just concatenating all your patterns into one string.

[deleted]

Re: Node.js in Flame Graphs

#199

Post by seanmcdirmid »

nostrademons wrote: post #188
runT1ME wrote: post #186

Earlier quoted context omitted.

>What this means in practice is that companies that use open source extensively in their operation, become slower and slower to innovate as they are carrying the weight of a thousand different systems of checks on code quality and robustness, which people using closed source will start delivering faster and faster as they effectively partition the review/quality question to the person selling them the software and th…

It's funny, my first job out of college was at a startup with an ex-Sun CTO, and he insisted on a single monolithic codebase. After he left I tried to organize an (aborted) project to modularize the codebase, since many of the other engineers ran into it as well. Having worked at Google in the interim, and having a number of acquaintances at Microsoft (which uses the multiple-repository approach) I can see the pros a…

I learned a lot from this post; thanks! As a MS employee (but in research), it is fascinating to study these differences. I also believe that the single codebase view is inevitable given the need for agility in the modern marketplace.

Re: Node.js in Flame Graphs

#200

Post by gcv »

ChuckMcM wrote: post #130

The moneyquote: "We made incorrect assumptions about the Express.js API without digging further into its code base. As a result, our misuse of the Express.js API was the ultimate root cause of our performance issue." This situation is my biggest challenge with software these days. The advice to "just use FooMumbleAPI!" is rampant and yet the quality of the implemented APIs and the amount of review they have had varie…

This is precisely why — for some products in some industries — NIH is a reasonable strategy for writing good programs.
Post reply on HN