Another way that we have used timeseries with Redis at Leftronic is ZSETs. The "score" is the timestamp and the key is a string like {"value": 42, "timestamp": 123456789}. That way you can have auto-sorting/replacement/insertion of timeseries points. Including the timestamp in the key is necessary so you can have duplicate values.
Scaling SQL with Redis
31–37 of 37 posts
Re: Scaling SQL with Redis
#32Earlier quoted context omitted.
Hey Bryan, just curious, are you guys not already using statsd/graphite? I think that at least you used to, since you contributed to my small script[0] to automate the installation of graphite... So I'm curious if/why it wasn't good enough, or whether this has different requirements that graphite wasn't suitable for? [0] https://github.com/gingerlime/graphite-fabric
Yep! Redis tends to be more for reading inflight rate/plan limiting, something I've not heard a lot about in conjunction with graphite (though it might be great!). We might bring back statsd/graphite for alerting/monitoring in general though, we've been looking for solutions there.
Thanks (also to Mike) for sharing. I can't comment about your specific use-case, but I imagine that unless your latency requirements are very demanding, graphite can play nicely here. Its rather expressive querying allows you to aggregate timeseries data pretty easily. Of course, Graphite is primarily used for monitoring and trending, but it's not limited to this use-case. Alerting isn't something it's particularly great at though, but there are other tools that plug in there.
Not trying to diss redis, I'm using redis as well and love it, but was just curious since you mentioned time-series data.
Re: Scaling SQL with Redis
#33Last time I used Redis I was surprised to determine to my surprise that Redis was single threaded. Of course I could have just RTFM but I assumed incorrectly. This means that if you have part of your application that requires fast consistent GETs, and then another application does a slower SORT, UNION, DIFF, etc, on the same db or even other dbs on the same Redis server, EVERY other client request has to wait for thi…
If the two datasets with different access speed requirements are disjoint, you can just run two instances of redis. One for the high-latency gruntwork, one for the low-latency GETs. If the datasets aren't disjoint, then you're trying to do fast and slow ops with the same data, which - if you need accurate values - is going to be mildly hairy even if multithreaded, since you'll need to somehow lock the data while you…
Very rare data access is disjoint, unless you're only doing key/value put/get. I think the interest of Redis is that it has many other features than simply put/get, and all those sorts, diff, etc typically would work a set of data that is being written in.
For sure having multiple instances will help some of this, but adds more complexity. Do you have your app write to multiple instances, and then read low latency from one, and read high latency from another? Is that data now consistent? Do you setup Redis replication and make sure that works right and then read from different replicas? Or perhaps you engineer some queue that does not block writes, groups them together and writes to Redis in a separate thread. Then you have to maintain all this and make sure it's correct, back it up, what are the corner cases, failure modes, etc.
From my experience, if you want to engineer things well, you end up essentially building out the same sub systems that a larger db engine has. Say Innodb. I'm smart enough to know that I'm not smart enough to build a one off complex system more correctly than really smart people that have been iterating over many years and improving things on something like innodb.
There are very rare, very specific cases where I would use redis over something else if I was building something realtime, large and important.
Re: Scaling SQL with Redis
#34I would be curious to compare this PostgreSQL + RabbitMQ + Redis solution with Cassandra. It is very well suited to time series data which is why it is so popular in advertising industries. Also you would think that rate limiting would be handled at the load balancing layer with Nginx, Apache, Layer7 etc. Way before it gets close to your app. Not criticising Sentry for doing things a bit different. Redis is a fantast…
We handle rate limiting with iptables, nginx, and Redis. Redis is the final state, but our goal is to make a sustainable and fast rate limiting solution which we can actually report metrics on. When things get dropped in iptables for example we have very little information, and Nginx is almost as low level as that.
Re: Scaling SQL with Redis
#35Earlier quoted context omitted.
If the two datasets with different access speed requirements are disjoint, you can just run two instances of redis. One for the high-latency gruntwork, one for the low-latency GETs. If the datasets aren't disjoint, then you're trying to do fast and slow ops with the same data, which - if you need accurate values - is going to be mildly hairy even if multithreaded, since you'll need to somehow lock the data while you…
Very rare data access is disjoint, unless you're only doing key/value put/get. I think the interest of Redis is that it has many other features than simply put/get, and all those sorts, diff, etc typically would work a set of data that is being written in. For sure having multiple instances will help some of this, but adds more complexity. Do you have your app write to multiple instances, and then read low latency fr…
That may be your experience but...
I suggest you google YouPorn's architecture.
I think its a domain/scale issue. It isn't a 'everything must become a more complex db engine to be engineered well' issue.
Re: Scaling SQL with Redis
#36Earlier quoted context omitted.
Redis is durable with its bin logs, but if you're pushing many "jobs" through Redis, you will end up wanting to turn off bin logging because of the lag it introduces.
Durable to me means the chance of losing state on unexpected shutdown is ruled out. It's not the case for Redis.
In the ACID sense, no it doesn't have write ahead logging or recovery. But it's good enough for government work.
Re: Scaling SQL with Redis
#37Earlier quoted context omitted.
This was my take away question. Redis can be used extremely effectively as a pool of job queues with failover. Perhaps RabbitMQ provides robust bidirectional messaging? While pooling Redis works well for one-way job submission (with each Redis instance being backed by some set of work consumers), making the process synchronous (whereby the consuming worker communicates back to the producer) is not so clearly handled…
Making something synchronous when it involves a job queue sounds like a recipe for disaster, IMO. Better to let both the consumer and producer act in a fire-and-forget manner with the original consumer producing a reply on a second queue which the original producer will eventually handle.
Our use case necessitated a synchronous interaction, but the message back was done exactly as you suggest - just uses a private second queue. The challenge was that while the initial job is fire-and-forget (or wait for a reply), the 2nd private queue was just a placeholder for a reliable messaging backend that needed to be implemented. It doesn't matter what worker gets the job, but it matters to whom the consumer of the job replies. I believe Redis provides enough primitives to make a highly reliable messaging system. We just have not done it.