Staircar: Redis-powered notifications

#1

Post by jcsalterego » Tue, Jul 19, 2011, 10:53 PM UTC

Staircar: Redis-powered notifications
engineering.tumblr.com

Re: Staircar: Redis-powered notifications

#2

Post by rdoherty » Wed, Jul 20, 2011, 12:07 AM UTC

Neat technology, but at first I thought I was going to read an article about some sort of messaging/event system. It's not; Staircar is basically a way to deal with high write to read ratio data. It's not for subscribing to or publishing events.

Re: Staircar: Redis-powered notifications

#3

Post by ozataman » Wed, Jul 20, 2011, 1:08 AM UTC

It reads like a hash-based sharding front-end for Redis rather than a layer for notification systems specifically.

Re: Staircar: Redis-powered notifications

#4

Post by mkjones » Wed, Jul 20, 2011, 1:27 AM UTC

This looks pretty cool. I'm curious how they deal with machine failure though - does Staircar coordinate some kind of redundancy, or when you lose a redis machine do you need to manually fix it and bring it back up with whatever the last checkpoint data was?

Re: Staircar: Redis-powered notifications

#5

Post by aaronbrethorst » Wed, Jul 20, 2011, 3:31 AM UTC

Do you have to watch out for hop-ons?

edit: no arrested development fans, i see...

Re: Staircar: Redis-powered notifications

#6

Post by troyk » Wed, Jul 20, 2011, 3:36 AM UTC

I'm curious as to the speed difference between redis current protocol and a resty-json loving http redis server, as the speed was more than enough for tumblr and going the http route has the benefit that any http client becomes a full blown redis client.

Re: Staircar: Redis-powered notifications

#7

Post by stephth » Wed, Jul 20, 2011, 4:53 AM UTC

aaronbrethorst wrote: post #5

Do you have to watch out for hop-ons? edit: no arrested development fans, i see...

It is a pretty good naming scheme.

Looking forward for steveholt, featherbottom, annyong and nevernude.

Re: Staircar: Redis-powered notifications

#8

Post by jrockway » Wed, Jul 20, 2011, 5:30 AM UTC

stephth wrote: post #7

aaronbrethorst wrote: post #5

Do you have to watch out for hop-ons? edit: no arrested development fans, i see...

It is a pretty good naming scheme. Looking forward for steveholt, featherbottom, annyong and nevernude.

There's always money in thebananastand.

Re: Staircar: Redis-powered notifications

#9

Post by aaronbrethorst » Wed, Jul 20, 2011, 5:54 AM UTC

stephth wrote: post #7

aaronbrethorst wrote: post #5

Do you have to watch out for hop-ons? edit: no arrested development fans, i see...

It is a pretty good naming scheme. Looking forward for steveholt, featherbottom, annyong and nevernude.

Steveholt!

Also: MrF, Cornballer, and Chickendance.

Re: Staircar: Redis-powered notifications

#10

Post by owyn » Wed, Jul 20, 2011, 5:58 AM UTC

Possibly early optimization syndrome? Redis is awesome, but the main problem here seems to be the amount of data (in the hundreds of gigs) and not scaling writes... The stated requirements are 7500 notifications per second but a single instance of redis can do 50k/sec updates on my mac...