Linux Performance Analysis
techblog.netflix.com
Linux Performance Analysis
1–10 of 84 posts
Re: Linux Performance Analysis
#2that's a pretty awesome summary. to that i would add:
* `iftop` that allows me to quickly check which network streams are hogging the machine (this is a little like sar -n DEV 1 but much more detailed!)
* `tcpdump -c 100 -vv`- poor man's alternative to iftop or systat if they're not available locally
* tail those logfiles, with systemd it's even easier with journald as all logfiles can be checked at once: `journalctl -xf -p notice | ccze -m ansi` (ccze colors the output, purely optional)
* probably more i am forgetting now
the great takeaways in the article for me are:
* pidstat: great to have an average, beats top for sure
* back to basics: hit dmesg, vmstat and iostat first, as you learned in the beginning! :)
Re: Linux Performance Analysis
#3Hmm, that java process has 0.227 TB of mapped virtual memory and 3090% CPU..
Re: Linux Performance Analysis
#4I'm wondering if there's a way to automate this kind of analysis to give you condensed, interesting points of a system performance status. À la powertop for power management.
Re: Linux Performance Analysis
#5I'm wondering if there's a way to automate this kind of analysis to give you condensed, interesting points of a system performance status. À la powertop for power management.
There is, look up Conky
Re: Linux Performance Analysis
#6I'm wondering if there's a way to automate this kind of analysis to give you condensed, interesting points of a system performance status. À la powertop for power management.
There is, look up Conky
Conky is for your desktop. We are talking about a server, which you connect to via ssh.
Re: Linux Performance Analysis
#7Hmm, that java process has 0.227 TB of mapped virtual memory and 3090% CPU..
You sound like you doubt the readings. This is netflix, I assume they have some pretty hefty kit backing up their service.
For a massive server the CPU reading might not be unusual. Maybe it has 32+ CPU cores and a multi-threaded java app is spinning most of them.
Also remember that on a heavily loaded system the task that is reading CPU use is itself competeing for time. Timing issues and other monitoring vagueries can make such readings noticably imprecise (though for CPU time usually in the downwards direction by missing tasks that started+worked+ended between readings).
The 233GB of memory seems high but there are possible explanations for this. A server with 32+ cores is not unlikely to have a lot of RAM too, the motherboard in my home server supports up to 128GB so perhaps that large java process genuinely does use more then 200. Also all that memory might not really be in use: it could have been allocated but never accessed so it isn't yet holding pages in RAM or swap for all of it.
Re: Linux Performance Analysis
#8I'm wondering if there's a way to automate this kind of analysis to give you condensed, interesting points of a system performance status. À la powertop for power management.
maybe monit or something similar (inspeqtor etc)
Re: Linux Performance Analysis
#9I'm wondering if there's a way to automate this kind of analysis to give you condensed, interesting points of a system performance status. À la powertop for power management.
nmon
Re: Linux Performance Analysis
#10I'm wondering if there's a way to automate this kind of analysis to give you condensed, interesting points of a system performance status. À la powertop for power management.
There is, look up Conky
Conky doesn't tell you what's wrong with your system (À la powertop 2.0). It just present the information differently.