31–40 of 62 posts
I couldn't imagine why.
2 TB OMG, thats almost a decent sized SQL Server instance. Yeah, it should take about an hour or two to replicate. I'm assuming they have a 10Gb enet on their DB server.
They say they moved off ebs and onto local storage for postgres and saw a big increase in reliability and performance. I did the same for my site last year and it was great. This is one of the reasons why I haven't moved my Postgres databases to enterprisedb or heroku: they use ebs.
But how do you achieve data persistence in case of server crash? Snapshots are not reliable for that, slave db servers aren't foolproof either.
I'm not exactly sure how the failover system works in Postgres, the last time I setup replication on postgres it would only copy the WAL log after it was fully written, but I know they have a much more fine grained system now.
If you use SQL Server you can add a 3rd monitoring server and your connections failover to the new master pretty much automatically as long as you add the 2nd server to your connection string. Using the setup with a 3rd server can create some very strange failure modes though.
It's interesting to see that they're sticking with Cassandra, and that they're having a much better experience with 0.8. I've been hearing so many fellow coders in SF hate on Cassandra that I had stopped considering it for projects. Has anybody worked with 0.8 or 1.0? Would you recommend Cassandra? I got to work with Riak a lot while I was at DotCloud, but the speed issue was pretty frustrating (it can be painfully s…
Cassandra has enabled Reddit to manage a highly scalable distributed data store with a tiny staff. This is not to say it has been trouble free, but it has enabled them to do something that would have been infeasible without pioneers in this space (Cassandra, Riak, Voldemort, etc) making these tools available.
Earlier quoted context omitted.
I've always wondered why they don't contract out with other ad networks when they cannot fill the ad content themselves. Say for example their self serve ad can't fill the page request why not put in a google text ad link on the right side where the banner is? That to me seems like a straightforward way to massively increase revenues.
They dont do it because they really care about the user. Just sticking up random google ads isnt going to make anybody happier and with an internet savy crowd like reddit ad clicks are likley to be low. Sure very targeted ads like the ones that self-serve currently delivers work because its redditors advetising to redditors.
What about IndexTank? They dont talk about it in this blog post. Have they stopped using it?
They say they moved off ebs and onto local storage for postgres and saw a big increase in reliability and performance. I did the same for my site last year and it was great. This is one of the reasons why I haven't moved my Postgres databases to enterprisedb or heroku: they use ebs.
What a bummer. Amazon advertises EBS as being both faster and more reliable, but it sounds like they are delivering neither.
It's interesting to see that they're sticking with Cassandra, and that they're having a much better experience with 0.8. I've been hearing so many fellow coders in SF hate on Cassandra that I had stopped considering it for projects. Has anybody worked with 0.8 or 1.0? Would you recommend Cassandra? I got to work with Riak a lot while I was at DotCloud, but the speed issue was pretty frustrating (it can be painfully s…
Balancing your cluster requires a little bit more handholding, and if something goes wrong or you fuck it up, it can be pretty challenging. But most of the time it's pretty painless.
There are a lot of other warts though, the data model is slightly weird, the secondary indexing is slow, and eventual consistency is hard to wrap your head around, but it doesn't require much effort to run and operate a large cluster, and if that's important to you and your application, you should check it out.
The NoSQL space is pretty interesting, but there is no clear winner, each of the competing solutions have their own niche, their own specialities, so it's impossible to give general recommendations right now.
It's interesting to see that they're sticking with Cassandra, and that they're having a much better experience with 0.8. I've been hearing so many fellow coders in SF hate on Cassandra that I had stopped considering it for projects. Has anybody worked with 0.8 or 1.0? Would you recommend Cassandra? I got to work with Riak a lot while I was at DotCloud, but the speed issue was pretty frustrating (it can be painfully s…
The two phases we've seen are:
1/It's flexible and it works! Problem solved! 2/21st century called, they want their performance back.
The problem with phase 2 is that you may not be able to solve it by throwing more computing power at it.
Unfortunately if you really need map-reduce, at the moment I don't know what to recommend. Riak isn't better performance-wise and our product doesn't support map-reduce (yet).
However if you don't need map-reduce I definitively recommend not using Cassandra. There's a lot of non-relational databases out there that are an order of magnitude faster.