The Evolution of Data at Reddit
redditblog.com
The Evolution of Data at Reddit
1–10 of 12 posts
Re: The Evolution of Data at Reddit
#2"Third party vendor". Is this Qubole? If so, would be nice to give him an Internet high-five by naming them :)
Re: The Evolution of Data at Reddit
#3I think nowadays they use Microsoft PowerBI for dashboards/reporting
Re: The Evolution of Data at Reddit
#4+1
Re: The Evolution of Data at Reddit
#5This is extremely interesting and I'm eager to see the follow-up.
I'm currently planning something very similar at my company and any reference to how other smaller-scale companies handle this when scaling is invaluable. Are there any other blogs from other companies which have talked about similar things?
Re: The Evolution of Data at Reddit
#6This is extremely interesting and I'm eager to see the follow-up. I'm currently planning something very similar at my company and any reference to how other smaller-scale companies handle this when scaling is invaluable. Are there any other blogs from other companies which have talked about similar things?
Just google for engineering blog. I know for sure Facebook and Google have good ones.
Re: The Evolution of Data at Reddit
#7This is extremely interesting and I'm eager to see the follow-up. I'm currently planning something very similar at my company and any reference to how other smaller-scale companies handle this when scaling is invaluable. Are there any other blogs from other companies which have talked about similar things?
Not a direct answer to your question, but I maintain an HN-like aggregator of engineering blogs here: https://engineered.at
These types of posts pop up there pretty frequently
Re: The Evolution of Data at Reddit
#8> The tool we used did not support using Hive as a backend, and given the speed of even relatively basic Hive queries, it wouldn’t make much sense to use that for reporting metrics.
Did Reddit try setting up Presto? Analytics, not just visualization, with Hive is really hard.
Re: The Evolution of Data at Reddit
#9I've had a great experience with Hive!
One thing that has made me sad is that there is that Rails / Ruby has little to no support. Many of the repos are over four to five years old.
Re: The Evolution of Data at Reddit
#10https://www.flydata.com/blog/hive-and-redshift-a-brief-compa...
Used Redshift for our large data needs. No reason to look to anything else, given our time-shifted data could easily be fit into Redshift. For larger data, we would have to have some other solution.