Keeping Instagram up with over a million new users in twelve hours • Hacker News Board

Re: Keeping Instagram up with over a million new users in twelve hours

#51

Post by rkurian » Tue, Apr 10, 2012, 7:40 AM UTC

mikeyk wrote: post #31

lenn0x wrote: post #26

What kind of instances are you guys running for Redis/memcached? I am a bit surprised on the numbers here, but to be fair I don't do much in the virtualization world. With low cpu overhead, it sounds like you might be saturating the number of interrupts on the network card if it's not a bandwidth issue. Memcache can usually push 100-300k/s on an 8-core Westmere (could go higher if you removed the big lock). Redis on…

Those are really useful numbers--I think a lot of it can be chalked up to virtualization, but we should definitely explore more around IRQ pinning for queues. Any good starting points / reading, are you mostly using taskset?

Why do you guys use both memcache and Redis ? Redis also has LRU cache functionality.

Re: Keeping Instagram up with over a million new users in twelve hours

#52

Post by olegi » Wed, Apr 11, 2012, 12:59 PM UTC

Hello!

Question about quality insta-photos on Android.

I have JPG from SGS2 - http://kia4sale.narod.ru/insta/01.jpg

This is http://kia4sale.narod.ru/insta/02.jpg instaphoto (Earlybird) from Android version

This is http://distilleryimage9.instagram.com/662ade7483ce11e19e4a12... - instaphoto from SGS2 JPG but on iPhone 4.

Question: why instaphoto on Android version in blurry?

Thanks.

Re: Keeping Instagram up with over a million new users in twelve hours

#53

Post by andrewdunstan » Wed, Apr 11, 2012, 5:10 PM UTC

PGFouine is nice, but it needs a major do-over. It would be good written with a plpgsql backend running against database loaded csv log files, so that it could handle huge logs, unlike now.

Re: Keeping Instagram up with over a million new users in twelve hours

#54

Post by lenn0x » Sun, Apr 15, 2012, 5:56 AM UTC

rkurian wrote: post #51

mikeyk wrote: post #31

Earlier quoted context omitted.

Those are really useful numbers--I think a lot of it can be chalked up to virtualization, but we should definitely explore more around IRQ pinning for queues. Any good starting points / reading, are you mostly using taskset?

Why do you guys use both memcache and Redis ? Redis also has LRU cache functionality.

Because in-house we have a custom version of memcache. We rewrote memcache's slab allocator, and for some use cases, is better at memory efficiency than Redis.