scales: Greplin's new open source Python server metrics library
1–10 of 20 posts
Re: scales: Greplin's new open source Python server metrics library
#2We open sourced something similar called StatGrabber awhile back, including Perl and Python client libraries. We tend to avoid threads, and so it instead uses non blocking UDP (guaranteed delivery on localhost) to a collector daemon, which aggregates and then delivers the info to Ganglia for graphing. https://spideroak.com/code if anyone is curious.
The client modules simply emit non-blocking UDP packets and get on with their business, avoiding slowing down their response time. You can graph 4 types of stats: counters (ex: transactions) averages (ex: size of transactions), accumulators (ex: bandwidth used) and elapsed time (ex: time per transaction)
It's pretty nice to see Ganglia graphing system metrics along side of all the stats we have the backend emit. For example, one of the stats graphed are revenue events. There's clear relationship between network health issues and revenue.
Re: scales: Greplin's new open source Python server metrics library
#3We open sourced something similar called StatGrabber awhile back, including Perl and Python client libraries. We tend to avoid threads, and so it instead uses non blocking UDP (guaranteed delivery on localhost) to a collector daemon, which aggregates and then delivers the info to Ganglia for graphing. https://spideroak.com/code if anyone is curious. The client modules simply emit non-blocking UDP packets and get on w…
We wrote something similar at Brightcove to collect system statistics and publish them graphite. It runs as an independent service and focuses on OS (not application) level metrics.
Re: scales: Greplin's new open source Python server metrics library
#4We open sourced something similar called StatGrabber awhile back, including Perl and Python client libraries. We tend to avoid threads, and so it instead uses non blocking UDP (guaranteed delivery on localhost) to a collector daemon, which aggregates and then delivers the info to Ganglia for graphing. https://spideroak.com/code if anyone is curious. The client modules simply emit non-blocking UDP packets and get on w…
We wrote something similar at Brightcove to collect system statistics and publish them graphite. It runs as an independent service and focuses on OS (not application) level metrics. https://github.com/BrightcoveOS/Diamond
That looks really convenient and useful.
Re: scales: Greplin's new open source Python server metrics library
#5[deleted]
Re: scales: Greplin's new open source Python server metrics library
#6that reminds me to put 2 of three puzzle solutions online....
Re: scales: Greplin's new open source Python server metrics library
#7We open sourced something similar called StatGrabber awhile back, including Perl and Python client libraries. We tend to avoid threads, and so it instead uses non blocking UDP (guaranteed delivery on localhost) to a collector daemon, which aggregates and then delivers the info to Ganglia for graphing. https://spideroak.com/code if anyone is curious. The client modules simply emit non-blocking UDP packets and get on w…
<3 SpiderOak. Any reason you guys don't host or mirror your code on Github? Makes it much easier for us curious tinkerers to keep your stuff organized and 'top of mind' among the many other things we fork, grok, and hack at.
Re: scales: Greplin's new open source Python server metrics library
#8We open sourced something similar called StatGrabber awhile back, including Perl and Python client libraries. We tend to avoid threads, and so it instead uses non blocking UDP (guaranteed delivery on localhost) to a collector daemon, which aggregates and then delivers the info to Ganglia for graphing. https://spideroak.com/code if anyone is curious. The client modules simply emit non-blocking UDP packets and get on w…
<3 SpiderOak. Any reason you guys don't host or mirror your code on Github? Makes it much easier for us curious tinkerers to keep your stuff organized and 'top of mind' among the many other things we fork, grok, and hack at.
I'll second that - it was actually a bit of a surprise seeing a recent open source project where I had to download and extract an archive rather than browsing the source code online.
Re: scales: Greplin's new open source Python server metrics library
#9We open sourced something similar called StatGrabber awhile back, including Perl and Python client libraries. We tend to avoid threads, and so it instead uses non blocking UDP (guaranteed delivery on localhost) to a collector daemon, which aggregates and then delivers the info to Ganglia for graphing. https://spideroak.com/code if anyone is curious. The client modules simply emit non-blocking UDP packets and get on w…
> UDP (guaranteed delivery on localhost)
That's not something I've heard before. Is it generally true of localhost-UDP? OS Specific? Particular to your usage?
Re: scales: Greplin's new open source Python server metrics library
#10Correct me if I'm mistaken, but it seems like this wouldn't really work with a web server that has several worker processes like uwsgi in preforking mode. Each worker process would be sandboxed to its own STATS object and attempting to serve the HTTP/Graphite server in its own background thread; all of which would be trying to access the same port.