Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis at Disqus

bretthoerner.com

Post reply on HN
Print view View original

21–30 of 42 posts

Re: Redis at Disqus

#21

Post by nikz »

When aggregating stats in this manner (by Day) how do people deal with Time Zones?

For instance, if I have one user in, say, NZST, their "Tuesday, 22 February" is still "Monday, 21 February" in PST - and the real issue is that the buckets are off. So you can't just store in UTC and then move it by whatever timezone offset, as then you are grabbing different "buckets".

I don't think that explanation is very clear (I had to draw a diagram to figure it out myself). Hopefully someone smarter than I am can figure it out anyway.

We've worked around it by just storing hour aggregates, but I'm interested in case someone else has a smart solution :)

Re: Redis at Disqus

#22

Post by bretthoerner »

nikz wrote: post #21

When aggregating stats in this manner (by Day) how do people deal with Time Zones? For instance, if I have one user in, say, NZST, their "Tuesday, 22 February" is still "Monday, 21 February" in PST - and the real issue is that the buckets are off. So you can't just store in UTC and then move it by whatever timezone offset, as then you are grabbing different "buckets". I don't think that explanation is very clear (I h…

You're right, when your finest granularity is "per day" you lose any real sense of what a day means to the user (and when it starts and ends).

Right now we don't have a solution. As it turns out (I guess) our users don't mind. In the future we want to provide certain (or all) stats also in the "last 24 hours" (rolling) time frame, which will help. One benefit of our Redis-powered analytics is that they're live. Even if your idea of a "day" is different than ours, you can see the count/totals/averages/etc updating in realtime (relative to the "day" we decide on). So users can get instant feedback if something is happening, but for the most part only care about day over day stats (which make the TZ matter a lot less).

Re: Redis at Disqus

#23

Post by DEinspanjer »

So frustrating to see all these people doing cool things with Redis and not having people free to do that stuff here. :) Any Redis hackers looking for a job or even some contract time? Big advantage is all the work will be open source and able to be shared and blogged.

Re: Redis at Disqus

#24

Post by bretthoerner »

DEinspanjer wrote: post #23

So frustrating to see all these people doing cool things with Redis and not having people free to do that stuff here. :) Any Redis hackers looking for a job or even some contract time? Big advantage is all the work will be open source and able to be shared and blogged.

I'm the author. What about a junior developer with an interest in Redis? This stuff isn't hard; I have recommendations. E-mail in profile. :)

Re: Redis at Disqus

#25

Post by sigil »

bretthoerner wrote: post #18
sigil wrote: post #17

Sharding: "We just take the modulo of the owning user's ID against the number of nodes we have to decide which node to read/write from/to." What's your procedure for adding new nodes to increase capacity? Would you have to take your redis cluster offline to redistribute data from all nodes over the new keyspace? I like the simplicity of your approach, but wonder if consistent hashing might be a bigger win in the long…

> What's your procedure for adding new nodes to increase capacity? Would you have to take your redis cluster offline to redistribute data from all nodes over the new keyspace? It's not easy, actually. The short answer is that we don't add capacity (because we've only needed to once, and we have tons of room to grow now). The long answer is that I have a switch I can flip that starts incrementing/adding data to a whol…

Thanks for the info!

> I'm not sure how consistent hashing helps when adding nodes to a "real" DB, either ... How does consistent hashing help with the migration?

Instead of backfilling all data to an entirely new cluster, you'd only backfill the small amount of data from the keyspace "stolen" by the new node, and expire the keys at the original locations. If you use M replicas of each node around the ring (typically M I'm still experimenting with this idea myself, and would also love to know if anyone's tried something similar with data store sharding (not just with caching).

Re: Redis at Disqus

#26

Post by bretthoerner »

sigil wrote: post #25
bretthoerner wrote: post #18

Earlier quoted context omitted.

> What's your procedure for adding new nodes to increase capacity? Would you have to take your redis cluster offline to redistribute data from all nodes over the new keyspace? It's not easy, actually. The short answer is that we don't add capacity (because we've only needed to once, and we have tons of room to grow now). The long answer is that I have a switch I can flip that starts incrementing/adding data to a whol…

Thanks for the info! > I'm not sure how consistent hashing helps when adding nodes to a "real" DB, either ... How does consistent hashing help with the migration? Instead of backfilling all data to an entirely new cluster, you'd only backfill the small amount of data from the keyspace "stolen" by the new node, and expire the keys at the original locations. If you use M replicas of each node around the ring (typically…

> you'd only backfill the small amount of data from the keyspace "stolen" by the new node

I think this is the part I'm not so sure about.

Say I have 100 stats, and of course each stat is per forum, per day (going back from 1 day to ... 5 years?). How do I know what keys were just "stolen"? Do I have my new-node code hash every possible key (all stats for all forums for all hours for all time) to see which might go to that node? And then it reverses that key to know what it "means" to backfill it? (I need to do that followup post as the way our data 'flows' in is applicable here)

Re: Redis at Disqus

#27

Post by iampims »

nikz wrote: post #21

When aggregating stats in this manner (by Day) how do people deal with Time Zones? For instance, if I have one user in, say, NZST, their "Tuesday, 22 February" is still "Monday, 21 February" in PST - and the real issue is that the buckets are off. So you can't just store in UTC and then move it by whatever timezone offset, as then you are grabbing different "buckets". I don't think that explanation is very clear (I h…

Flickr decided that UTC would be the default for their stats. As long as you stick to it, it's not that big of a problem.

Re: Redis at Disqus

#28

Post by bretthoerner »

iampims wrote: post #27
nikz wrote: post #21

When aggregating stats in this manner (by Day) how do people deal with Time Zones? For instance, if I have one user in, say, NZST, their "Tuesday, 22 February" is still "Monday, 21 February" in PST - and the real issue is that the buckets are off. So you can't just store in UTC and then move it by whatever timezone offset, as then you are grabbing different "buckets". I don't think that explanation is very clear (I h…

Flickr decided that UTC would be the default for their stats. As long as you stick to it, it's not that big of a problem.

Not if you're trying to save RAM (and operations to fetch said data) by storing stats per day. You'd need all stats to be per-hour in order for it work for any timezone.

Re: Redis at Disqus

#29

Post by sigil »

bretthoerner wrote: post #26
sigil wrote: post #25

Earlier quoted context omitted.

Thanks for the info! > I'm not sure how consistent hashing helps when adding nodes to a "real" DB, either ... How does consistent hashing help with the migration? Instead of backfilling all data to an entirely new cluster, you'd only backfill the small amount of data from the keyspace "stolen" by the new node, and expire the keys at the original locations. If you use M replicas of each node around the ring (typically…

> you'd only backfill the small amount of data from the keyspace "stolen" by the new node I think this is the part I'm not so sure about. Say I have 100 stats, and of course each stat is per forum, per day (going back from 1 day to ... 5 years?). How do I know what keys were just "stolen"? Do I have my new-node code hash every possible key (all stats for all forums for all hours for all time) to see which might go to…

> How do I know what keys were just "stolen"? Do I have my new-node code hash every possible key (all stats for all forums for all hours for all time) to see which might go to that node? And then it reverses that key to know what it "means" to backfill it?

Right, you'd have to iterate through all zset elements on the existing node, applying the consistent hash function to decide whether or not the element will be stolen by the new node.

If the element itself doesn't contain user id (or whatever you shard on) all bets are off.

Re: Redis at Disqus

#30

Post by jrockway »

bretthoerner wrote: post #28
iampims wrote: post #27

Earlier quoted context omitted.

Flickr decided that UTC would be the default for their stats. As long as you stick to it, it's not that big of a problem.

Not if you're trying to save RAM (and operations to fetch said data) by storing stats per day. You'd need all stats to be per-hour in order for it work for any timezone.

If you store by day UTC, you'd need two fetches to get a day in some other time zone. But if you store by hour, you need 24 fetches.
Post reply on HN