Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Measure Anything, Measure Everything (2011)

codeascraft.etsy.com

Post reply on HN
Print view View original

11–20 of 23 posts

Re: Measure Anything, Measure Everything (2011)

#11

Post by jashmenn »

huhtenberg wrote: post #2

Does anyone here use this?

Yes, at IFTTT we use this. I've been pleasantly surprised at the amount of traffic statsd can handle, even without sampling.

We also believe in measuring everything you can. We're interacting with many APIs across many boxes. Statsd + graphite are the tools we use to understand what's happing at runtime.

Graphite has a lot of warts, but it's really powerful once you get used to it. There are plenty of pretty interfaces you can put over graphite, but nothing really matches it for ease of ad-hoc queries.

Typically I'll use graphite to view ad-hoc metrics and build reports. When I find I'm repeatedly viewing a particular graphite report then I'll "hard-code" it in gdash [1] for the rest of the team.

We use this combo to track thousands of separate metrics and we've been pretty happy with it so far.

[1] https://github.com/ripienaar/gdash

Re: Measure Anything, Measure Everything (2011)

#12

Post by jmandzik »

I was inspired by this post when I first read it a few month ago. Since then, I've used a modified version of StatsD to send data to an in house realtime graphing engine. A lot of our tools are php backends, so it was super convenient to be able to drop the class in and start measuring things.

For anyone interested, I used wrote node.js process takes arbitrary statsd-compliant data point and serves a socket.io enabled front-end for 'zero-config', realtime graphing.

We've found it useful internally for taking quick measurements on various projects. I was going to productize or open source the whole thing, but then life got in the way. Maybe it will see the light of day someday.

Re: Measure Anything, Measure Everything (2011)

#13

Post by misiti3780 »

is it just me or is graphite a HUGE pain in the ass to setup? is there a better option for these graphs. I really want to use statsd in a few projects but it looks like im going to have to set aside 8 hours to figure out how graphite works (and i am a django developer)

Re: Measure Anything, Measure Everything (2011)

#14

Post by hoop »

misiti3780 wrote: post #13

is it just me or is graphite a HUGE pain in the ass to setup? is there a better option for these graphs. I really want to use statsd in a few projects but it looks like im going to have to set aside 8 hours to figure out how graphite works (and i am a django developer)

There are other graphing frontends now, but also you can check out companies like Librato who have statsd-compatible backends and do all the graphing and storage for you.

Re: Measure Anything, Measure Everything (2011)

#15

Post by pjungwir »

misiti3780 wrote: post #13

is it just me or is graphite a HUGE pain in the ass to setup? is there a better option for these graphs. I really want to use statsd in a few projects but it looks like im going to have to set aside 8 hours to figure out how graphite works (and i am a django developer)

I set up Graphite+StatsD for a project back in the spring of 2011, it was a ton of trouble. StatsD was easy, but Graphite was really hard. It was a challenge to find up-to-date documentation where big sections weren't marked simply "TODO". I also never solved a bug where my counters were getting graphed as tiny numbers like 0.02 rather than 1, despite repeatedly wiping the box and experimenting with different config settings. Things seemed to be a little better about 6 months later, but I still couldn't solve the 0.02 problem. I love the simplicity of StatsD, and I liked how Graphite let me use the URL API to build very useful dashboards for both technical and business metrics, but the setup was a big hassle. I spent days fighting with it, a big cost for a startup. I'd use StatsD again, but with a different database/frontend, probably something hosted.

Re: Measure Anything, Measure Everything (2011)

#16

Post by patrickod »

misiti3780 wrote: post #13

is it just me or is graphite a HUGE pain in the ass to setup? is there a better option for these graphs. I really want to use statsd in a few projects but it looks like im going to have to set aside 8 hours to figure out how graphite works (and i am a django developer)

If you're looking to use Graphite but don't want to manage it yourself then I highly recommend hostedgraphite.com. I and many friends at other companies use them and are very happy with the service. They're smart guys.

Re: Measure Anything, Measure Everything (2011)

#17

Post by Diederich »

misiti3780 wrote: post #13

is it just me or is graphite a HUGE pain in the ass to setup? is there a better option for these graphs. I really want to use statsd in a few projects but it looks like im going to have to set aside 8 hours to figure out how graphite works (and i am a django developer)

So I'm not being snarky in the least bit. Seriously. I'm just sharing.

I got graphite going on an Ubuntu installed on my macbook while in the air between SFO and ATL.

Literally, discovered it, downloaded it, and graphing metrics before we landed.

Now, that was, I think, in 2010. The last time I installed Graphite was a few months ago, and it seems like they have split the project up a bit. It took a bit longer.

NB: I'm neither a Python or Django 'guy'

Re: Measure Anything, Measure Everything (2011)

#19

Post by bsmith »

misiti3780 wrote: post #13

is it just me or is graphite a HUGE pain in the ass to setup? is there a better option for these graphs. I really want to use statsd in a few projects but it looks like im going to have to set aside 8 hours to figure out how graphite works (and i am a django developer)

I've been using StumbleUpon's OpenTSDB (http://opentsdb.net) with much success, and it looks like StatsD could easily be modified to work with it. The biggest pain with OpenTSDB I've had is that it requires an HBase/Hadoop backend, but a standalone, single-node install isn't terrible to setup.

Re: Measure Anything, Measure Everything (2011)

#20

Post by tbh »

patrickod wrote: post #16
misiti3780 wrote: post #13

is it just me or is graphite a HUGE pain in the ass to setup? is there a better option for these graphs. I really want to use statsd in a few projects but it looks like im going to have to set aside 8 hours to figure out how graphite works (and i am a django developer)

If you're looking to use Graphite but don't want to manage it yourself then I highly recommend hostedgraphite.com. I and many friends at other companies use them and are very happy with the service. They're smart guys.

Thanks Patrick!

Clickable link: https://www.hostedgraphite.com

(I'm one of the cofounders)

Post reply on HN