What I love here on HN is that even though everyone here seems to hate on MongoDB, it keeps on rolling, improving, and getting used in real companies. There may be "better" (for spurious defn of better) products like rethink or whatever, but folks just aren't using them. Get over it!
Call Me Maybe: MongoDB Stale Reads
91–100 of 150 posts
Re: Call Me Maybe: MongoDB Stale Reads
#92I seem to remember from a foundationDB talk that they first spent two years building a simulation environment to control everything from network to persistance for testing scenarios.
Does anyone know of any open-source project that would aim at doing the same, so that future NoSQL DB can finally be built on strong foundations ?
Re: Call Me Maybe: MongoDB Stale Reads
#93Does anyone have any references on how you could write a distributed database that met all ACID properties? Surely there's an academic paper that says that if you do A then B then C, you are guaranteed a certain level of consistency. We've developed a type of distributed database at my company, and I think it's pretty solid, but I need a broader familiarity with the available theory.
Consensus is the main hurdle - if you have multiple nodes that can be read from, then any values that are successfully written to the system must guarantee that those same values can be read sequentially.
Issues arise when network partitions interrupt communication between nodes; even if you require all nodes to send acks when writing, how do you deal with those acks not being received?
Re: Call Me Maybe: MongoDB Stale Reads
#94Earlier quoted context omitted.
After MongoDB published their write speed benchmarks based entirely on unacknowledged writes (e.g. how fast can you write to a socket?), it's been a long downhill ride with an immense amount of inexplicable ignorant support.
Can you post a link to these unacknowledged write benchmarks? I can't find them.
From my own memory, I don't recall 10gen ever posting misleading benchmarks. There were a bunch of other people who did so, and 10gen did little or nothing to try to shut that publicity down.
That said, I think that the 'how fast can you write to a socket?' default setting was probably intentionally put in place to make benchmarks look good.
Re: Call Me Maybe: MongoDB Stale Reads
#95The most interesting lessons from the Jepsen series: * You should never trust, and always verify, the claims made by database manufacturers. * Especially when those claims relate to data integrity. * Super-especially when every safety level provided by the manufacturer that includes the word "SAFE" is actually unsafe.
It seems easier to me to just go with a reputable database vendor.
Is there actually that much difference between MongoDB and e.g. Cassandra? The tone of the Call Me Maybe posts about both is quite different, but the bottom line seems to be the same: use CRDTs for everything or you will lose data.
Re: Call Me Maybe: MongoDB Stale Reads
#96From 2009 to 2012 I had a distributed database startup that competed with MongoDB. We used Paxos for replication and built the database with on-disk consistency guarantees --- like the ones this article looks for and rightly obsesses over --- in mind.
https://github.com/scalien/scaliendb
Outcome: you've never heard of ScalienDB; MongoDB brilliantly won by winning the hearts and minds of hackers and coders who don't care about such issues, but were able to get started quickly with Mongo (and got cool free cups at meetups). It turns out that's most engineers out there, definitely the initial critical mass to target for a database startup like Mongo.
Btw. the story behind Oracle is similar: early versions were basically write-only; read Ellison's book 'Softwar'. Of course there are other ways to get started: for example DBs coming out of academic research like Vertica seem to avoid this problem; in that case initial funding is basically provided by the gov't and when they create the company to commercialize they're already shooting for Enterprise contracts, skipping the opensource/community building phase of Mongo.
Re: Call Me Maybe: MongoDB Stale Reads
#97So what should users of MongoDB do? I'm asking because it is the main database used in Meteor and I'm very interested in Meteor. Should the general advice just be "store in MongoDB everything that doesn't require consistency and use Postgresql for everything else"?
The general advice should be: use PostgreSQL in case you are uncertain what to use. Watch some Youtube video's with Michael Stonebreaker (2014 Turing Award winner) and start getting disillusioned by the NoSQL hype.
Then, try to understand the mess Edgar Codd tried to fix in the '60s and '70s.
Re: Call Me Maybe: MongoDB Stale Reads
#98People really underestimate the value of Occasional Consistency. Occasionally Consistent databases, like MongoDB, are great for approximation algorithms, sublinear time algorithms, and similar applications.
The issue isn't that MongoDB is eventually consistent, it's that the documentation claims that in some cases it's strictly consistent[1] while Kyle found that: "MongoDB, even at the strongest consistency levels, allows reads to see old values of documents or even values that never should have been written." 1. http://docs.mongodb.org/manual/reference/glossary/#term-stri...
I didn't say Eventual Consistency. I said Occasional Consistency. MongoDB is has hard Occasional Consistency. Indeed, it is the most occasionally consistent database I know of. I once wrote a few million records into Mongo. It was consistent before the write, but never again after.
Great for sub-linear time algorithms! At that point, all my algorithms ran at less than O(n) on the size of the data I had written in.
From a business perspective, Occasional Consistency is also a very nice property if you are storing audit data for certain types of organizations. It gives complete plausible deniability about rule compliance.
Re: Call Me Maybe: MongoDB Stale Reads
#99The most interesting lessons from the Jepsen series: * You should never trust, and always verify, the claims made by database manufacturers. * Especially when those claims relate to data integrity. * Super-especially when every safety level provided by the manufacturer that includes the word "SAFE" is actually unsafe.
Actually the broader lesson would be to assume the worst in your application layer and try and remediate/verify wherever possible. If you look at his articles: Redis, PostgreSQL, Cassandra, ElasticSearch etc all had data consistency errors. And none of those have vendors making any claims. It's pretty sobering to say the least.
You misunderstand the results. PostgreSQL behaves as expected, and indeed, the only way it can behave. That's the Two General's Problem (http://en.wikipedia.org/wiki/Two_Generals%27_Problem). There is no way to solve it. PostgreSQL does as well as theoretically possible.
I understand you lost data with Oracle and Teradata.
1. Most big corporate vendors do not have systems which are very well designed. (1) The sale is made at a business level. (2) Most Oracle customers are not very tech companies, and have mixed quality employees. As a result, there is little pressure on building a robust, correct product, rather than one which meets a feature checklist. In addition, Oracle doesn't really recruit smart people (I know people who work there). It's just not very robust compared to something like PostgreSQL, which was written by Stonebraker, a legendary computer science professor and entrepreneur.
2. Still, more likely, the reason you lost data is because you didn't know what you were doing. Words like Eventual Consistency, ACID, Two-General's Problem, etc. are not just abstract. They have strict, formal meanings, and you need to understand what they do and do not guarantee. Otherwise, you will lose data again.
Re: Call Me Maybe: MongoDB Stale Reads
#100People really underestimate the value of Occasional Consistency. Occasionally Consistent databases, like MongoDB, are great for approximation algorithms, sublinear time algorithms, and similar applications.
Occasional Consistency sounds like a made up word.
I can't even google this term without it auto correcting to Eventually Consistency.
The wiki article, https://en.wikipedia.org/wiki/Consistency_model, doesn't even have such a term.
Doing a hard search term on google reveals nothing.