Saving 13M Computational Minutes per Day with Flame Graphs • Hacker News Board

Re: Saving 13M Computational Minutes per Day with Flame Graphs

#21

Post by tracker1 » Tue, Apr 12, 2016, 7:07 AM UTC

surrealvortex wrote: post #20

I'm currently using flame graphs at work. If your application hasn't been profiled recently, you'll usually get lots of improvement for very little effort. Some 15 minutes of work improved CPU usage of my team's biggest fleet by ~40%. Considering we scaled up to 1500 c3.4xlarge hosts at peak in NA alone on that fleet, those 15 minutes kinda made my month :) One thing to note once you eliminate the easy pickings is th…

Profiling in general is a really good thing when you're seeing odd load/timing/performance issues... I once found a project was storing its' configuration settings (loaded/cached from DB) in a really badly performing way, an in-memory datatable, with text queries instead of a hashtable (not my design).

A single call wasn't so bad, but the lookup was happening many hundreds of times per request adding seconds to some requests. Wild how much difference a relatively small thing can make.

Re: Saving 13M Computational Minutes per Day with Flame Graphs

#22

Post by chadlavi » Tue, Apr 12, 2016, 5:24 PM UTC

just think of the millions of dollars they could save if they stopped doing double spaces after sentences

Re: Saving 13M Computational Minutes per Day with Flame Graphs

#23

Post by surrealvortex » Wed, Apr 13, 2016, 7:16 AM UTC

tracker1 wrote: post #21

surrealvortex wrote: post #20

I'm currently using flame graphs at work. If your application hasn't been profiled recently, you'll usually get lots of improvement for very little effort. Some 15 minutes of work improved CPU usage of my team's biggest fleet by ~40%. Considering we scaled up to 1500 c3.4xlarge hosts at peak in NA alone on that fleet, those 15 minutes kinda made my month :) One thing to note once you eliminate the easy pickings is th…

Profiling in general is a really good thing when you're seeing odd load/timing/performance issues... I once found a project was storing its' configuration settings (loaded/cached from DB) in a really badly performing way, an in-memory datatable, with text queries instead of a hashtable (not my design). A single call wasn't so bad, but the lookup was happening many hundreds of times per request adding seconds to some…

That brings up another distinction - profilers don't distinguish between a method that takes very little time to run but is called very often and another method that is pretty expensive, but is not called very often.

Ultimately, we do care about the total time taken, but the approaches necessary for the two cases above are very different. In many cases, the method that is simply called very often will call for some type of caching solution in the caller, while the more expensive method will require retooling within the method itself.