Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
1–10 of 16 posts
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#2This type of analysis really shows that dealing with failure is a highly complex problem with no easy answers.
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#3For me choosing a database involves carefully considering the value of data that will be housed within.
To what degree do I care about the data? How long would it take to recreate? What's the cost to my business if data is lost.
Time and time again, I end up choosing PostgreSQL for ANY data that I care about (i.e. anything that is not simply 'in-flight data'.
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#4This is a much more recent presentation by Kyle (the author of the linked article) with a more mature version of his Jepsen tool. I imagine he'll get around to a written version of his findings soon, until then it's worth the time to watch this and learn a bit about distributed systems, databases and testing. https://www.youtube.com/watch?v=XiXZOF6dZuE
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#5Great read! I think this is a first time I've seen a really good, structured failover test that contrasts postgre, redis, and mongo (I don't know much about Riak). It would have been interesting to see mysql in there as well.
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#6It would be cool to see how Hyperdex (http://hyperdex.org) holds up.
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#7FoundationDB did an extensive test with Jepsen, with results published here: https://foundationdb.com/blog/call-me-maybe-foundationdb-vs-...
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#8Were any issues logged, or these problems otherwise addressed, in the ten+ months since this was posted?
I wasn't able to find anything with some quick searches but hopefully others on this thread are more familiar with these projects.
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#9This is an awesome article. I would love to see the tests packaged in such a way that we could port them to other DB's easily and perhaps do something like the web framework shootout, but with database consistency.
Re: Jepsen: Testing Partition Tolerance of PostgreSQL, Redis, MongoDB, Riak (2013)
#10For me choosing a database involves carefully considering the value of data that will be housed within. To what degree do I care about the data? How long would it take to recreate? What's the cost to my business if data is lost. Time and time again, I end up choosing PostgreSQL for ANY data that I care about (i.e. anything that is not simply 'in-flight data'.
From the point of view of a "devops" developer - who has done hard time in QA and DBA as well - this requirement changes drastically when performance and scaling enter the equation. The scale slides quickly from "lose no data, ever" to "we can lose a few seconds of transactions if it speeds up the web page" to "we can lose a lot of data and still be OK, as long as we're still online".
Seconds, or even minutes worth of lost data do cost the company, but not nearly as much as poor performance. And unfortunately, developers tend to over-value data in the equation, leading to decisions which cause a company problems when it comes time to grow.
Ultimately, the best tools for ensuring business continuity (with few exceptions) is redundancy coupled with a set of proper backups.