21–30 of 37 posts
Interesting, but why not use redis pub-sub for the job queues instead of forwarding to RabbitMQ?
Durability & high availability?
Earlier quoted context omitted.
Durability & high availability?
Redis is durable with its bin logs, but if you're pushing many "jobs" through Redis, you will end up wanting to turn off bin logging because of the lag it introduces.
Earlier quoted context omitted.
Rabbit is durable and highly available as well. It can be clustered and it's confirmable queues allow for a high volume of writes while remaining durable.
This was my take away question. Redis can be used extremely effectively as a pool of job queues with failover. Perhaps RabbitMQ provides robust bidirectional messaging? While pooling Redis works well for one-way job submission (with each Redis instance being backed by some set of work consumers), making the process synchronous (whereby the consuming worker communicates back to the producer) is not so clearly handled…
This means that if you have part of your application that requires fast consistent GETs, and then another application does a slower SORT, UNION, DIFF, etc, on the same db or even other dbs on the same Redis server, EVERY other client request has to wait for this slower command to finish. http://redis.io/topics/latency
This is something that one really has to engineer around in order to use it in an environment that requires performance and consistent latency. In our case of 1000s req/s it was just unacceptable to have the latency be affected, sometimes by 10 times, by a slower command.
I do love all the sort, diff, union commands.
Last time I used Redis I was surprised to determine to my surprise that Redis was single threaded. Of course I could have just RTFM but I assumed incorrectly. This means that if you have part of your application that requires fast consistent GETs, and then another application does a slower SORT, UNION, DIFF, etc, on the same db or even other dbs on the same Redis server, EVERY other client request has to wait for thi…
If the datasets aren't disjoint, then you're trying to do fast and slow ops with the same data, which - if you need accurate values - is going to be mildly hairy even if multithreaded, since you'll need to somehow lock the data while you do the slow op (which will exclude the GETs, causing high latency), or you'll need some kind of transaction-based stable view to operate on (e.g. transactional memory?)
Redis is great... except for the fact it's (publicly) not ACID, so adding Redis in the mix and calling it "scaling SQL" is outright misleading, because it loses the very properties SQL exists to provide. Redis will enter into conflicts (where in this article's example, those locks won't "lock" the thing you're locking), and it'll lose minutes of committed operations on unexpected stops. Does that make Redis useless?…
Almost the entire post assumes you're not using it for durability. It's about making tradeoffs so the use of SQL can scale. If you want to be semantic, ACID will never be performant and scalable. The use of SQL on getsentry.com is already pushing Postgres to the limit's of what a locking/transactional database can do. The locks are a minor bullet point in a much larger picture. Redis is never going to generate "confl…
We manage this quite well at GigaSpaces (http://www.gigaspaces.com). I have some examples up at http://gigaspacesinanger.wordpress.com that show some use cases.
https://github.com/seivan/redis-friendships
Also you would think that rate limiting would be handled at the load balancing layer with Nginx, Apache, Layer7 etc. Way before it gets close to your app.
Not criticising Sentry for doing things a bit different. Redis is a fantastic technology.
Redis really is a fundamental building block for designing distributed systems these days. I was kind of surprised, but all these examples exist independently in the Zapier codebase as well (all backed by Redis). I've been meaning to open source our timeseries implementation for a while now, it is very similar to the linked article but uses a "{key}:YYYY:MM:DD:hh:mm:ss" pattern on hashes where you pick your stored gr…
Hey Bryan, just curious, are you guys not already using statsd/graphite? I think that at least you used to, since you contributed to my small script[0] to automate the installation of graphite... So I'm curious if/why it wasn't good enough, or whether this has different requirements that graphite wasn't suitable for? [0] https://github.com/gingerlime/graphite-fabric
Redis is great... except for the fact it's (publicly) not ACID, so adding Redis in the mix and calling it "scaling SQL" is outright misleading, because it loses the very properties SQL exists to provide. Redis will enter into conflicts (where in this article's example, those locks won't "lock" the thing you're locking), and it'll lose minutes of committed operations on unexpected stops. Does that make Redis useless?…
Never noticed anything like this and I've been using Redis for 3+ years.