Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Call Me Maybe: MongoDB Stale Reads

aphyr.com

Post reply on HN
Print view View original

121–130 of 150 posts

Re: Call Me Maybe: MongoDB Stale Reads

#121

Post by fennecfoxen »

bakhy wrote: post #117

I must admit, I always feel like I am missing something in these discussions. Like I didn't get some memo... I just don't expect a DB like MongoDB to guarantee consistency. The whole story around NoSQL and the likes was to enable crazy horizontal scaling needed for the web. Phrases like "eventual consistency" flew around. It seems so logical - you lose consistency, gain scalability. But somehow, people simply started…

> It seems so logical - you lose consistency, gain scalability. ... And now, we're all bashing on MongoDB because it is - not consistent? What happened here? :)

There are ways to do "eventual consistency" responsibly. Mind you, it's obnoxiously tricky to do it right, even when someone has provided an underlying implementation that works exactly as promised. But if you design your data access patterns in the right way, the system can provide guarantees so that even if it doesn't have all your data at the moment, you can still ask questions about the the state of the data that is available, and get meaningful responses back that conform to a certain set of guarantees.

What happened here -- why we make fun of MongoDB -- is that it doesn't provide many promises like that, and even when it does, its implementation does a very, very bad job of delivering them ... and it doesn't even do a good job of delivering scalability. (It's basically a mmap()'d series of b-trees of BSON documents, so as soon as you run out of RAM, you're at risk of having the kernel swap out all your indicies instead of your data, whereupon performance craters. Oh, and the much-mocked global write lock has finally been replaced with a per-database write-lock in recent versions.)

In short, you sacrifice everything and gain... a modestly convenient API for document-storage, maybe.

Re: Call Me Maybe: MongoDB Stale Reads

#122

Post by tel »

hibikir wrote: post #118
tel wrote: post #104

Earlier quoted context omitted.

Depending on how large "software tests" is in your mind this can easily be not the case. Kyle often mentions the tool TLA+ which enables complete and total formal analysis of certain systems. This can be a test which is complete and therefore a positive proof of correctness. It's also not difficult to test smaller components of your system if you note that the state space here is small enough to be exhausted. This is…

Type systems and proofs have their limitations, if just because for a proof to be worth anything, I need a perfect description of what I want, and that doesn't make any sense outside of trivial cases. I once worked with a relatively well known, prove everything developer that you might know by name. He's written books and everything. He built an algorithm to make a distributed system of equal peers figure out when it…

Sure, never indicated otherwise I hope!

But that said, the style of analysis is totally different in each case. It's not true that you can never prove correctness. It's just a certain option with certain tradeoffs.

And there are success stories! Tons of older ones back when proof was a major component of compute programming. More modern ones like validated C kernels and compilers.

Re: Call Me Maybe: MongoDB Stale Reads

#123

Post by craigching »

jodah wrote: post #58
bkeroack wrote: post #4

If you are a database author and you get a bug report from Kyle, spend a long time thinking about it before closing the issue as invalid.

They're not the only ones: https://github.com/elastic/elasticsearch/issues/2488#issueco...

Actually elasticsearch took their jespen results pretty seriously and even have an ongoing status of their resiliency. See my post on this:

https://news.ycombinator.com/item?id=9418318

Re: Call Me Maybe: MongoDB Stale Reads

#124

Post by mdasen »

Maro wrote: post #96

From 2009 to 2012 I had a distributed database startup that competed with MongoDB. We used Paxos for replication and built the database with on-disk consistency guarantees --- like the ones this article looks for and rightly obsesses over --- in mind. https://github.com/scalien/scaliendb Outcome: you've never heard of ScalienDB; MongoDB brilliantly won by winning the hearts and minds of hackers and coders who don't c…

It's very true that MongoDB was able to target developers who didn't know or care about a lot of issues in the databases world. However, I wouldn't attribute Vertica's success to coming out of the academic world and being funded by the government.

MongoDB targets a part of the market that is well served. There are loads of other options that work well like PostgreSQL. If you were targeting a truly distributed transactional database, you might have been trying to build something that has that order-of-magnitude-better characteristics, but MongoDB doesn't seem like that.

Vertica, on the other hand, really targeted an area that wasn't well served and a decade after C-Store this area still isn't well served by anyone other than Vertica. ParAccel/Redshift isn't really comparable, though many go with it because Amazon will manage it for them.

For MongoDB, the question is "how can we win over developers' hearts and minds?" Why use Mongo? Well, you like the query DSL and get annoyed by joins, and they have a great website and documentation, and we all know that joins are slow so without them everything must scale nicely. But realistically, there are a bunch of other options that could be substituted for it.

And you're right that Vertica skipped the community aspect. The reason why is that something like Vertica isn't interesting if you don't have a lot of data. MongoDB targeted people who might have GBs of data that they wanted to store. There's a lot of people and community relations is going to matter, especially because there are lots of battle-tested alternatives. Since Vertica is most interesting when you have more data than can fit on a single machine's hard drives and might take hours to run analytics queries against, they're going to be targeting a much different market.

Vertica also had the advantage of being associated with the biggest names in databases. Stonebraker has a Turing award. Even beyond that, the names you'd think of when you think databases were involved in C-Store and Vertica.

If you're really creating something that has order-of-magnitude-better characteristics, you probably shouldn't target people that have 10GB of data to store. Someone once said something to the effect of: once there were more than 200 websites on the internet, it made more sense to target web frameworks at the websites that weren't the top 100 than solving for the problems of the top 100 sites. Most sites aren't going to need a massively distributed database. They might have dreams that the same codebase will scale from 100,000 visits/day to Facebook-scale, but they're just dreams. The fact is that for 10GB worth of data, your database probably wouldn't be better simply because that's already well-handled. In fact, distributed consensus on such small data would probably make your database worse. You really need to target people that you're solving a problem for.

Vertica just isn't interesting technology for the vast majority of users and so they don't sell to the vast majority of users. MongoDB is interesting for those who want an alternative DSL and such.

Re: Call Me Maybe: MongoDB Stale Reads

#125

Post by edejong »

narrator wrote: post #27

I knew something was funny with Mongo when all the api calls defaulted to writes not being guaranteed to sync to disk. Maybe for a use case like aggregate statistics gathering it would be ok to risk missing a few updates in a crash for the sake of speed, but to make that the default??

Would you want to miss the most important post-recovery data: when your system is under duress?

Re: Call Me Maybe: MongoDB Stale Reads

#126

Post by lazyloop »

threeseed wrote: post #38
narrator wrote: post #27

I knew something was funny with Mongo when all the api calls defaulted to writes not being guaranteed to sync to disk. Maybe for a use case like aggregate statistics gathering it would be ok to risk missing a few updates in a crash for the sake of speed, but to make that the default??

You know this configuration change was changed in November 2012. Do you think it's still relevant to be bringing this up ?

Actually, the defaults are still unsafe, just the marketing language has changed. Take the Node.js driver for example, it defaults to w=null and j=false. http://mongodb.github.io/node-mongodb-native/2.0/api/Db.html

Re: Call Me Maybe: MongoDB Stale Reads

#127

Post by jdcryans »

cpks wrote: post #98
jdcryans wrote: post #31

Earlier quoted context omitted.

The issue isn't that MongoDB is eventually consistent, it's that the documentation claims that in some cases it's strictly consistent[1] while Kyle found that: "MongoDB, even at the strongest consistency levels, allows reads to see old values of documents or even values that never should have been written." 1. http://docs.mongodb.org/manual/reference/glossary/#term-stri...

I didn't say Eventual Consistency. I said Occasional Consistency. MongoDB is has hard Occasional Consistency. Indeed, it is the most occasionally consistent database I know of. I once wrote a few million records into Mongo. It was consistent before the write, but never again after. Great for sub-linear time algorithms! At that point, all my algorithms ran at less than O(n) on the size of the data I had written in. Fr…

Heh, Poe's Law, etc :)

I guess you could also call it Quantum Consistency.

Re: Call Me Maybe: MongoDB Stale Reads

#128

Post by rspeer »

biggest_lou wrote: post #116
rspeer wrote: post #81

Earlier quoted context omitted.

I think Riak's theory is fine, but theory isn't enough. And they may have succumbed to the Osborne effect with Riak 2.0. Here's what I mean. Think of all the nice things you expect to come out of Riak's theoretical basis -- bulletproof distributed writes, for example. Well, the default last-write-wins writes aren't bulletproof. They clearly fail Jepsen [1]. And if you turned off last-write-wins, then you'd have to ha…

FWIW I did write a bunch of documentation on sibling resolution in Riak: http://docs.basho.com/riak/latest/dev/using/conflict-resolut...

So it looks like the conflict resolution strategies being recommended are "pick one arbitrarily", or in an advanced section, "keep the longer list"?

Not that it matters to me anymore, but it does sound like having all data in CRDTs is the only way to pass Jepsen. (Or to have your data be immutable, in which case Jepsen doesn't apply.)

Re: Call Me Maybe: MongoDB Stale Reads

#129

Post by jeffdavis »

cjbprime wrote: post #119
jeffdavis wrote: post #84

Earlier quoted context omitted.

To me that's not surprising. If the client connection drops, you might not know whether the transaction committed or not... the only way to know is to reconnect and inspect to see what happened. If you want to avoid that kind of problem, use 2PC. Do you (or the author) see this as a bug, or just something that might surprise people who haven't thought through the guarantees?

It looks like you didn't read the article either -- it describes how Postgres is using 2PC, but 2PC doesn't stop the server from committed something that the client is unaware of through missing the final ACK. Quoting the article: > The 2PC protocol says that we must wait for the acknowledgement message to arrive in order to decide the outcome. If it doesn’t arrive, 2PC deadlocks. It’s not a partition-tolerant protoc…

What is the significance of that though? As you say, it's not partition-tolerant, so while partitioned, the system is down. As soon as the network issue is resolved, you can determine the state of your transaction.

It would be foolish for a client to issue a COMMIT and then assume the transaction aborted because of a connection drop. The client should wait until the connection can be reestablished and determine the real transaction state before making a decision based on it.

It's the same issue as with a power failure during fsync. The durability of that transaction is indeterminate, but it doesn't matter because the system is down. Before the system comes back, it will go through recovery, and either find the commit record or not, thus getting back to a determinate state.

Re: Call Me Maybe: MongoDB Stale Reads

#130

Post by ccleve »

kasey_junk wrote: post #109
ccleve wrote: post #47

Does anyone have any references on how you could write a distributed database that met all ACID properties? Surely there's an academic paper that says that if you do A then B then C, you are guaranteed a certain level of consistency. We've developed a type of distributed database at my company, and I think it's pretty solid, but I need a broader familiarity with the available theory.

For the record calling something a "distributed database" is not nearly enough. What part of it is distributed? On what operations do you want to provide the ACID guarantees? What do you promise in the face of partition? I would be very skeptical of any database that was written by someone who didn't have a sound foundational understanding of distributed systems theory. This is quite simply one of the places in softw…

As well you should be. Fortunately, we're relying on some third-party libraries that are built on a solid theoretical foundation, and what we promise is limited. Our goal is to improve over time.
Post reply on HN