Great stuff, love node2dm, and didn't know about statsd + graphite.
Thanks! This was my first node.js project, would love any feedback on it.
Also, any reason you guys are not using MixPanel for storing events as well, besides of the costs?
Keeping Instagram up with over a million new users in twelve hours
11–20 of 54 posts
Re: Keeping Instagram up with over a million new users in twelve hours
#12Thought about adding a tool like newrelic.com to your toolset?
NewRelic's product screenshots look really nice, but 1) the hosted nature and 2) the price were turnoffs. We might revisit it at some point, though--there are a few features in there that would be awesome to have.
Had the same first impression, but compared to rolling your own solution it's still a very good ROI. And now I'm getting more and more servers and projects on it, because it's just so handy.
(Btw not connected in any way, jsut a very happy user that convinced enough companies to try them, that I think you would be a good match, too)
Re: Keeping Instagram up with over a million new users in twelve hours
#13Congratulations. Really impressive how solid you handled the Android onslaught.
Re: Keeping Instagram up with over a million new users in twelve hours
#14What sort of hosting do you use for your main Pg (and Redis) instances?
From the mentions of EBS snapshots in the article, I'm going to guess they're all on Amazon EC2.
Re: Keeping Instagram up with over a million new users in twelve hours
#15are you guys sharding redis? or does it all fit in a single machine?
Re: Keeping Instagram up with over a million new users in twelve hours
#16are you guys sharding redis? or does it all fit in a single machine?
We have a variety of Redis machines, some of them are in a consistent hash ring (the ones we're using for caching); some are using modulo-based hashing (the ones where losing data on adding more machines isn't an option), and some are just single-node installs.
Re: Keeping Instagram up with over a million new users in twelve hours
#17Does anyone has some info on the architecture required to maintain a service like this? Servers, db, etc?
Re: Keeping Instagram up with over a million new users in twelve hours
#18Thought about adding a tool like newrelic.com to your toolset?
NewRelic's product screenshots look really nice, but 1) the hosted nature and 2) the price were turnoffs. We might revisit it at some point, though--there are a few features in there that would be awesome to have.
I think New Relic saves us money - the amount of optimizations we've done because of them is just crazy and by now would have saved us a couple servers and hardware upgrades at least.
Their sales dudes like startups, we were only lightly funded when we started using them - garth@newrelic could probably help you.
Re: Keeping Instagram up with over a million new users in twelve hours
#19are you guys sharding redis? or does it all fit in a single machine?
We have a variety of Redis machines, some of them are in a consistent hash ring (the ones we're using for caching); some are using modulo-based hashing (the ones where losing data on adding more machines isn't an option), and some are just single-node installs.
How do you handle write replication? I haven't found any good document on how to do this (meaning, if the Redis master goes down, a slave should be promoted to master immediately).
Re: Keeping Instagram up with over a million new users in twelve hours
#20Does anyone has some info on the architecture required to maintain a service like this? Servers, db, etc?