nitpick: "The values are almost inconsequential since THEIR just numbers" should be either THEY ARE or THEY'RE
Good god. I can't believe I missed that. Thanks. I'll make sure that gets fixed.
Snappy Dashboards with Redis
21–30 of 46 posts
Re: Snappy Dashboards with Redis
#22This is a pretty standard way to handle counts and doesn't seem to have much to do with redis. In an SQL database you'd just do it with a trigger and your application wouldn't need to know a thing about it.
I've never used SQL triggers. We host our Postgres database with Heroku. As a result, I think I've ruled out database level solutions.
On a related note, it feels good to know that everything our app needs to run is in the code base. Back in my C# days, I remember relying on stored procedures that were configured manually at the database level. Rails fights that with migrations and the callback chain too, so I guess that thinking has sunk in for me.
Thanks for the comment.
Re: Snappy Dashboards with Redis
#23This doesn't make any sense to me. Why store the values in Redis and use Ruby to do simple math? If you're using Postgres already, just store the stats in Postgres and use an aggregate or window function to grab the stats. (And collect those stats with triggers, in the first place.) If you're using Mongo already, just grab your stats with a map reduce query.
We definitely could have done this in Postgres.
Our API is a Rails project and though we do a lot of work that's "off the rails," so to speak, we do try to follow a convention where possible. A convention that's worked well for us is to use Postgres only for database-backed models in the app. So when it came time to solve this problem, Redis seemed like a good choice.
Map/reduce isn't an option here. First, it's too slow. Second, it's asynchronous by nature, which doesn't work well when trying to load a webpage. We did consider using it in the background, however, to generate the counter-caches.
Thanks for reading!
Re: Snappy Dashboards with Redis
#24This doesn't make any sense to me. Why store the values in Redis and use Ruby to do simple math? If you're using Postgres already, just store the stats in Postgres and use an aggregate or window function to grab the stats. (And collect those stats with triggers, in the first place.) If you're using Mongo already, just grab your stats with a map reduce query.
It looks like one of the things they're counting is clicks, so they could potentially have some pretty large datasets. I don't know how well Mongo's map-reduce works, but in Postgres, COUNT(star) [1] does not perform well for very large tables (e.g. 100 million rows). You wouldn't want to be doing a COUNT(star) once per minute for each customer that had their dashboard open on a plasma screen. Of course, there are ot…
To be fair, we may have over-engineered this solution. I've been doing some work with compound indexes with Mongo and they seem to be performing really well. Maybe I'll have to write a guest post for MongoHQ too ;)
Appreciate the time and thoughtful response.
Re: Snappy Dashboards with Redis
#25This doesn't make any sense to me. Why store the values in Redis and use Ruby to do simple math? If you're using Postgres already, just store the stats in Postgres and use an aggregate or window function to grab the stats. (And collect those stats with triggers, in the first place.) If you're using Mongo already, just grab your stats with a map reduce query.
Because they use MongoDB and as the usage of it increases, we'll see more and more people continue to "discover" ways to do things RDBMS' solved in the 80s.
Just like this also has nothing intrinsically to do with Redis and could be any regular ole key-value store.
Re: Snappy Dashboards with Redis
#26This is cool and, I believe, a good strategy for many things. > If Redis isn’t available, for whatever reason, we could rebuild the gaps from the canonical data in Mongo. We’ve never had to do this. I'm not sure this is so straight-forward. So, this is a guest post by Sqoot hosted by togo.io, who owns the redistogo.com. Can anyone explain redistogo.com's pricing? I understand the value in not running your own service…
I pinged the guys over at RedisToGo to get you some more clarity on the pricing.
Speaking for ourselves, we've historically preferred to have someone else manage our infrastructure. Setting up Redis is fairly trivial, I agree. However, setting up a secure machine out on the internet and making sure it's always there is less trivial. Rather than juggle an ops/engineer role, we just double down on engineering. I'm sure at some point this may need to change. Hopefully, when that day comes we can afford a sysadmin!
Re: Snappy Dashboards with Redis
#27This is exactly what I'm building http://www.instahero.com to solve. You don't have to build your own infrastructure, just write the relevant bit of code (or select a template) and you have a dashboard.
I think your value prop could be honed a bit. I tend to read more on a site than most people will, but I still couldn't get through the wall of text without losing interest and leaving. An example of 'write this code and get this!' would have kept me around.
Re: Snappy Dashboards with Redis
#28This is a pretty standard way to handle counts and doesn't seem to have much to do with redis. In an SQL database you'd just do it with a trigger and your application wouldn't need to know a thing about it.
I've never used SQL triggers. We host our Postgres database with Heroku. As a result, I think I've ruled out database level solutions. On a related note, it feels good to know that everything our app needs to run is in the code base. Back in my C# days, I remember relying on stored procedures that were configured manually at the database level. Rails fights that with migrations and the callback chain too, so I guess…
> We host our Postgres database with Heroku. As a result, I think I've ruled out database level solutions.
What does hosting with Heroku have to do with using a trigger?
> On a related note, it feels good to know that everything our app needs to run is in the code base.
Except the database schema and any additional indexes you need to make it not perform terribly. All basic setup, just like creating triggers.
> Rails fights that with migrations and the callback chain too, so I guess that thinking has sunk in for me.
The problem with doing stuff like this in a callback is that an additional query is sent to the database which can be a big performance problem if the insert load is high. I generally agree that complex logic should be avoided in triggers and is better left in the application code in most cases but incrementing a counter is about as simplistic as you can get.
Re: Snappy Dashboards with Redis
#29Nothing specific about redis or new here. Of course if you can easily cache a pre-calculated value, that'll save you time and CPU.
Well, to be fair, the cached counts are incremented by Redis through the incr command. So, no pre-calculation here. It can be interesting also to aggregate metrics of floats using the incrbyfloat command.
Re: Snappy Dashboards with Redis
#30I use Fnordmetric - Ruby/Redis dashboard. Works great.
That's slick.