This is great news. Historically I've accepted the risk of data loss and coded checks when needed. I will never rely on my database, regardless of which one I am using for complete data consistency. It is, however, nice to see strides being made towards even better robustness. Go MongoDB!
MongoDB 3.4.0-rc3
11–20 of 165 posts
Re: MongoDB 3.4.0-rc3
#12MongoDB receives a fair amount of criticism here but the company is a fantastic place to work. I'm proud that I was able to learn and grow as a developer alongside all of those who have been trying (and succeeding) to make a great database.
The team at MongoDB really cares a lot about making the best database product possible. I knew it when I was there and still think so after I've left.
Re: MongoDB 3.4.0-rc3
#13MongoDB 3.4 passes the rigorous and tough Jepsen test. Jepsen designs tests to make databases fail in terms of data consistency, correctness, and safety... MongoDB 3.4 passed through their newest tests. I think that this really shows how mature of a Database MongoDB is.
> I think that this really shows how mature of a Database MongoDB is. Or that it took this long for them to pass basic proficiency tests. How do other database systems fare with these tests?
You can check out the other databases he has tested here: http://jepsen.io/analyses
That being said, none of the big players in sql are there, so you can't size it up against postgres or mysql.
Re: MongoDB 3.4.0-rc3
#14MongoDB 3.4 passes the rigorous and tough Jepsen test. Jepsen designs tests to make databases fail in terms of data consistency, correctness, and safety... MongoDB 3.4 passed through their newest tests. I think that this really shows how mature of a Database MongoDB is.
> I think that this really shows how mature of a Database MongoDB is. Or that it took this long for them to pass basic proficiency tests. How do other database systems fare with these tests?
If you look at past Jepsen test mostly they fail, and sometimes they fail spectacularly. This is one of those things where if you don't have an automated test suite to continually verify correctness you will end up with bugs.
Re: MongoDB 3.4.0-rc3
#15Bigger news is that Jepsen tests are now part of the MongoDB continuous integration suite: https://evergreen.mongodb.com/build/mongodb_mongo_master_ubu...
Open and available for everyone to see, for every build of MongoDB. Is there another database that has this much transparency? (for every build)
Re: MongoDB 3.4.0-rc3
#16Earlier quoted context omitted.
> In addition, we show that the new v1 replication protocol has multiple bugs, allowing data loss in all versions up to MongoDB 3.2.11 and 3.4.0-rc4. While the v0 protocol remains broken, patches for v1 are available in MongoDB 3.2.12 and 3.4.0, and now pass the expanded Jepsen test suite. Wait, it is me not understanding what the abstract is trying to say?
I can only think that the article was a bit late and the new MongoDB version has the last word. Beyond this, the article is extremely valuable and it is mandatory to upgrade your MongoDB.
This analysis was a collaboration with MongoDB over the past three months. MongoDB was able to fix these issues in 3.4.0 because I found them in rc3 and rc4.
Re: MongoDB 3.4.0-rc3
#17It's been a long way from the "Call Me Maybe: MongoDB" post from years back. Aphyr/Kyle took them to task in so many ways for playing fast and loose with data integrity, and rightly so. MongoDB could have said, "that guy's full of BS, ignore him," but instead they did the smart thing and paid Kyle to help solve the problem.
n.b. I can't find the original "Call Me Maybe" post, but this later one [1] is similar.
Re: MongoDB 3.4.0-rc3
#18Earlier quoted context omitted.
I can only think that the article was a bit late and the new MongoDB version has the last word. Beyond this, the article is extremely valuable and it is mandatory to upgrade your MongoDB.
This analysis was a collaboration with MongoDB over the past three months. MongoDB was able to fix these issues in 3.4.0 because I found them in rc3 and rc4.
Sorry, my fault.
Re: MongoDB 3.4.0-rc3
#19This is great news. Historically I've accepted the risk of data loss and coded checks when needed. I will never rely on my database, regardless of which one I am using for complete data consistency. It is, however, nice to see strides being made towards even better robustness. Go MongoDB!
> I will never rely on my database, regardless of which one I am using for complete data consistency.
I can't imagine developing any software that involves relationships between entities that does not have data consistency. Check constraints, foreign keys, and data type validation all provide a minimum sanity level of the underlying data that allows your mind to focus on more important things. Otherwise you're entire application is going to be littered with those same sanity checks.
I'm not saying that all apps need that type of data store or that there isn't room in this world for NoSQL stores, I mean specifically that complicated interdependencies and validation checks lend themselves well to the relational model.
Re: MongoDB 3.4.0-rc3
#20MongoDB 3.4 passes the rigorous and tough Jepsen test. Jepsen designs tests to make databases fail in terms of data consistency, correctness, and safety... MongoDB 3.4 passed through their newest tests. I think that this really shows how mature of a Database MongoDB is.
While you are correct that 3.4 now passes, I feel like your interpretation here is a bit optimistic. It's not like I tested Mongo and it passed out of the box--it failed Jepsen tests, and not just with a read anomaly--it lost majority-acknowledged inserts. The v0 protocol still fails--it's fundamentally broken. The v1 bugs are fixed now, but that's a consequence of our collaboration.
I'd say the marker of maturity here is that MongoDB has put significant time and effort into correctness: they take clock skew and network partitions as serious failure modes, they've redesigned their replication protocol, added options for stronger reads, and invested in their own correctness test suite, and Jepsen tests, as a part of their CI process.