Why not store & update count in database. Rather Redis. Also I could never appreciate the redistogo. I can get much powerful redis instance on EC2 for much lesser price.
Snappy Dashboards with Redis
31–40 of 46 posts
Re: Snappy Dashboards with Redis
#32This is exactly what I'm building http://www.instahero.com to solve. You don't have to build your own infrastructure, just write the relevant bit of code (or select a template) and you have a dashboard.
Thanks for sharing. I agree with @bunkat here (though I didn't read the whole page). I find the language around analytics apps (including big boys like Mixpanel) to be generally vague. If you're catering to a primarily developer audience, you may consider just showing me how easy it is. StatHat does a good job of this.
Re: Snappy Dashboards with Redis
#33This doesn't make any sense to me. Why store the values in Redis and use Ruby to do simple math? If you're using Postgres already, just store the stats in Postgres and use an aggregate or window function to grab the stats. (And collect those stats with triggers, in the first place.) If you're using Mongo already, just grab your stats with a map reduce query.
I suppose it really depends on the use case. For example, there aren't probably any tables corresponding to an API call. If I wanted to collect stats on different API calls and other stuff not related to my models, I would prefer avoiding a database count update and would rather make a single incr call to a Redis key.
Re: Snappy Dashboards with Redis
#34I've been working on a dashboard that uses Redis as well. Just wondering, why not take advantage of sets/lists/zsets for your date-related keys. With lists you can do an easy LRANGE instead of that loop of GETs you're doing now.
Also, if you don't do this already, look into using bitsets to track users. As long as userID's are integers, it's real easy it saves a lot of space.
Re: Snappy Dashboards with Redis
#35This doesn't make any sense to me. Why store the values in Redis and use Ruby to do simple math? If you're using Postgres already, just store the stats in Postgres and use an aggregate or window function to grab the stats. (And collect those stats with triggers, in the first place.) If you're using Mongo already, just grab your stats with a map reduce query.
In general, absolutely. Redis tends to have very high write IO compared to your average DB though, so that's a definite advantage when gathering very-frequent actions.
Re: Snappy Dashboards with Redis
#36Earlier quoted context omitted.
I've never used SQL triggers. We host our Postgres database with Heroku. As a result, I think I've ruled out database level solutions. On a related note, it feels good to know that everything our app needs to run is in the code base. Back in my C# days, I remember relying on stored procedures that were configured manually at the database level. Rails fights that with migrations and the callback chain too, so I guess…
> We host our Postgres database with Heroku. As a result, I think I've ruled out database level solutions. What does hosting with Heroku have to do with using a trigger? > On a related note, it feels good to know that everything our app needs to run is in the code base. Except the database schema and any additional indexes you need to make it not perform terribly. All basic setup, just like creating triggers. > Rails…
Well thanks for the insights. I'll keep triggers in mind when I come across a problem like this in the future.
Re: Snappy Dashboards with Redis
#37This doesn't make any sense to me. Why store the values in Redis and use Ruby to do simple math? If you're using Postgres already, just store the stats in Postgres and use an aggregate or window function to grab the stats. (And collect those stats with triggers, in the first place.) If you're using Mongo already, just grab your stats with a map reduce query.
It looks like one of the things they're counting is clicks, so they could potentially have some pretty large datasets. I don't know how well Mongo's map-reduce works, but in Postgres, COUNT(star) [1] does not perform well for very large tables (e.g. 100 million rows). You wouldn't want to be doing a COUNT(star) once per minute for each customer that had their dashboard open on a plasma screen. Of course, there are ot…
Note that recent versions of Postgres can now COUNT against indices so no need to do a full table scan.
Re: Snappy Dashboards with Redis
#38Sounds like the typical caching approach to me, "avoid making big queries by caching your data in a ram-based key-value store" .. so this is pretty much common knowledge amongst webdevs since LiveJournal introduced memcached back in 2007?, or am i missing something ?
Re: Snappy Dashboards with Redis
#39This is a pretty standard way to handle counts and doesn't seem to have much to do with redis. In an SQL database you'd just do it with a trigger and your application wouldn't need to know a thing about it.
I've never used SQL triggers. We host our Postgres database with Heroku. As a result, I think I've ruled out database level solutions. On a related note, it feels good to know that everything our app needs to run is in the code base. Back in my C# days, I remember relying on stored procedures that were configured manually at the database level. Rails fights that with migrations and the callback chain too, so I guess…
For the record, Heroku doesn't affect your ability to use triggers. It's possible our ancient "shared database" infrastructure simply didn't support it, but the new starter tier plans certainly do.
Re: Snappy Dashboards with Redis
#40Earlier quoted context omitted.
Certainly you can implement this same pattern without Redis. Triggers in Postgres would be a reasonable way to do it. I didn't say you can't do this in Postgres, I said you can't do it with COUNT(). It does indeed sound like they're storing every click: that's precisely why using aggregate functions would be expensive.
You absolutely can do it without COUNT. Just increment a counter value, the same way they're doing it in Redis.
Why was this downvoted? Adding 1 to any number field is just as an atomic operation as incrementation is in Redis.