Live data from Hacker News

Linux Performance Analysis

techblog.netflix.com

41–50 of 84 posts

Re: Linux Performance Analysis

#41
post #37

The link redirects to the blog frontpage for me, but https://media.netflix.com/en/tech-blog/linux-performance-ana... works.

Odd, that URL messes up the pre tags when I load it in Chrome or Firefox (desktop). http://techblog.netflix.com/2015/11/linux-performance-analys... looks right.

Re: Linux Performance Analysis

#42

Hmm, that java process has 0.227 TB of mapped virtual memory and 3090% CPU..

You sound like you doubt the readings. This is netflix, I assume they have some pretty hefty kit backing up their service. For a massive server the CPU reading might not be unusual. Maybe it has 32+ CPU cores and a multi-threaded java app is spinning most of them. Also remember that on a heavily loaded system the task that is reading CPU use is itself competeing for time. Timing issues and other monitoring vagueries…

  For a massive server the CPU reading might not be unusual. Maybe it has 
  32+ CPU cores and a multi-threaded java app is spinning most of them.
Yes, the article states that

  The %CPU column is the total across all CPUs;
  1591% shows that that java processes is consuming almost 16 CPUs.

Re: Linux Performance Analysis

#43
post #37

The link redirects to the blog frontpage for me, but https://media.netflix.com/en/tech-blog/linux-performance-ana... works.

Odd, that URL messes up the pre tags when I load it in Chrome or Firefox (desktop). http://techblog.netflix.com/2015/11/linux-performance-analys... looks right.

A blog redesign was launched today, which has messed up the pre tags. Hopefully fixed shortly...

edit: old version is (PDF) http://www.brendangregg.com/Articles/Netflix_Linux_Perf_Anal...

Re: Linux Performance Analysis

#46
This is yet another awesome contribution from Netflix.

Just curious though, anyone know of a similar type of writeup or post regarding windows?

I have an MSSQL box I want to kill with fire - but I would like to be able to measure its perf with as much insight as I might on a comparable linux box.

What is the best way to measure disk IO perf on a windows box, specifically?

Re: Linux Performance Analysis

#49
I had one of my EC2 instances (Ubuntu 14.04) lock up on Monday morning. I use it to run a Ruby on Rails app and to do a handful of ETL jobs. The website wasn't able to load and I wasn't able to SSH into the box. So, I went into the AWS console and rebooted the instance. This has happened 3 times in the last 6 months. It looks like the CPU spikes up to almost 100% around the same time when the instance locks up.

I'd love to be able to identify the root cause of the instance locking up. It seems like this article is more about the commands you'd run to assess the health of an active/working EC2 instance and not one that you're unable to SSH into. Any idea on how to identify the problem with my EC2 instance?

Re: Linux Performance Analysis

#50

> Pidstat is a little like top’s per-process summary, but prints a rolling summary instead of clearing the screen. This can be useful for watching patterns over time, and also recording what you saw (copy-n-paste) into a record of your investigation. top has a batch mode: "top -b" will print all processes periodically to the terminal, without starting the curses-style UI, providing the same "rolling summary".

"pidstat 1" prints active processes, making it much less noisy that top -b, and therefore highlights malevolent processes. I think the top equivalant would be "top -b -i", though that has more output and is a bit noisier.
Post reply on HN