Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Call Me Maybe: MongoDB Stale Reads

aphyr.com

Post reply on HN
Print view View original

101–110 of 150 posts

Re: Call Me Maybe: MongoDB Stale Reads

#101

Post by GordyMD »

takeda wrote: post #80
cheeseprocedure wrote: post #67

Earlier quoted context omitted.

Agreed! As an example, HashiCorp included Jepsen test results in their Consul documentation: https://consul.io/docs/internals/jepsen.html

As Kyle himself mentioned (I think it might even be in relation to this document), Jepsen cannot prove that your software is safe, only prove that it isn't. He even called them out for modifying timeout value to pass the tests[1]. [1] https://aphyr.com/posts/316-call-me-maybe-etcd-and-consul

> Jepsen cannot prove that your software is safe, only prove that it isn't.

This doesn't just apply to Jepsen but to all software tests. You only ever test a finite set of scenarios, so you can't really ever guarantee your software is 'safe'/bug-free - only that it does not fail in common/expected scenarios.

Re: Call Me Maybe: MongoDB Stale Reads

#102

Post by uptownJimmy »

ivanb wrote: post #89

So what should users of MongoDB do? I'm asking because it is the main database used in Meteor and I'm very interested in Meteor. Should the general advice just be "store in MongoDB everything that doesn't require consistency and use Postgresql for everything else"?

I think Meteor is great, except for that one thing. I won't be touching it again until there if full support for one of the SQL technologies.

Re: Call Me Maybe: MongoDB Stale Reads

#103

Post by kokey »

iconfluence wrote: post #46
rdtsc wrote: post #41

Earlier quoted context omitted.

Actually, just use Riak. Those people know distributed bases. https://aphyr.com/posts/285-call-me-maybe-riak Now I hear they support consistency as well.

I think at one point the employees of Basho knew how to write distributed DBs - Riak is the most advanced AP DB from a distributed systems theory perspective. However, in recent months, their CEO, CTO and Chief Architect have left, as well as many of their prominent engineers. Worryingly, the new CTO seems content to make inane comments about "Data Gravity" [0]. [0] - http://www.kdnuggets.com/2015/03/interview-dave-m…

I had a bad experience with riak v2. I think something went quite wrong in the process when going from v1 to v2, especially around Riak Search.

Re: Call Me Maybe: MongoDB Stale Reads

#104

Post by tel »

GordyMD wrote: post #101
takeda wrote: post #80

Earlier quoted context omitted.

As Kyle himself mentioned (I think it might even be in relation to this document), Jepsen cannot prove that your software is safe, only prove that it isn't. He even called them out for modifying timeout value to pass the tests[1]. [1] https://aphyr.com/posts/316-call-me-maybe-etcd-and-consul

> Jepsen cannot prove that your software is safe, only prove that it isn't. This doesn't just apply to Jepsen but to all software tests. You only ever test a finite set of scenarios, so you can't really ever guarantee your software is 'safe'/bug-free - only that it does not fail in common/expected scenarios.

Depending on how large "software tests" is in your mind this can easily be not the case.

Kyle often mentions the tool TLA+ which enables complete and total formal analysis of certain systems. This can be a test which is complete and therefore a positive proof of correctness.

It's also not difficult to test smaller components of your system if you note that the state space here is small enough to be exhausted. This is a very important thing to look in to in serious, important components of a software system.

Finally, though it's futuretech today still, dependent types offer a general system for embedding proofs of correctness directly into code thus elevating positive proof to a computational artifact just like any other.

Re: Call Me Maybe: MongoDB Stale Reads

#105

Post by tel »

ivanb wrote: post #89

So what should users of MongoDB do? I'm asking because it is the main database used in Meteor and I'm very interested in Meteor. Should the general advice just be "store in MongoDB everything that doesn't require consistency and use Postgresql for everything else"?

You can write a DDP backend which is backed by Postgres. In such a case it ought to feel like Mongo is just serving materialized views of the genuine, consistent data. If you treat the Minimongo data that way—just consistent enough to show an image once—and verify all the writes on the server then you ought to be able to get by.

Re: Call Me Maybe: MongoDB Stale Reads

#106

Post by kasey_junk »

digitalzombie wrote: post #100
cpks wrote: post #16

People really underestimate the value of Occasional Consistency. Occasionally Consistent databases, like MongoDB, are great for approximation algorithms, sublinear time algorithms, and similar applications.

Occasional Consistency sounds like a made up word. I can't even google this term without it auto correcting to Eventually Consistency. The wiki article, https://en.wikipedia.org/wiki/Consistency_model , doesn't even have such a term. Doing a hard search term on google reveals nothing.

It was sarcasm...

Re: Call Me Maybe: MongoDB Stale Reads

#107

Post by kasey_junk »

teraflop wrote: post #74
ccleve wrote: post #47

Does anyone have any references on how you could write a distributed database that met all ACID properties? Surely there's an academic paper that says that if you do A then B then C, you are guaranteed a certain level of consistency. We've developed a type of distributed database at my company, and I think it's pretty solid, but I need a broader familiarity with the available theory.

Aside from reading papers, it's a good idea to look through the syllabus of a distributed systems course to get a broad idea of what the problem space looks like. Academic papers will talk about "minimal" problems like consensus, or desirable properties like sequential consistency, and expect you to already know why those concepts are important. If your experience is mostly hands-on, it may not be obvious how it all…

Or you are obviously violating proven theory...

Re: Call Me Maybe: MongoDB Stale Reads

#108

Post by tel »

Kiro wrote: post #86

This article is too technically advanced for me. As a casual MongoDB user, how do these problems affect me?

In replicated Mongo scenarios higher write volume increases the probability of inconsistent reads. What this means is that there's a chance that on some data—no matter how safely you attempt to write it—you'll end up in a totally inconsistent state for your system.

The actual impact of an inconsistent state is very hard to judge. It could be as minimal as having two different users just see something weird on their screen for a moment and then it goes away. It could even be totally avoided if your application handles inconsistent data well.

At the same time, it could also cause complete and nearly untraceable complete corruption of all data in your system. Who knows?

It'd be a bit like building a bridge using metal with a known defect. It'll probably work fine for a long time and depending on how and where that metal was used you might be alright.

Or you might have a complete structural integrity failure at any moment once stress starts ramping up and you'll just have to blame it wholesale on using bad materials.

Re: Call Me Maybe: MongoDB Stale Reads

#109

Post by kasey_junk »

ccleve wrote: post #47

Does anyone have any references on how you could write a distributed database that met all ACID properties? Surely there's an academic paper that says that if you do A then B then C, you are guaranteed a certain level of consistency. We've developed a type of distributed database at my company, and I think it's pretty solid, but I need a broader familiarity with the available theory.

For the record calling something a "distributed database" is not nearly enough. What part of it is distributed? On what operations do you want to provide the ACID guarantees? What do you promise in the face of partition?

I would be very skeptical of any database that was written by someone who didn't have a sound foundational understanding of distributed systems theory. This is quite simply one of the places in software engineering where subtle differences in promise, protocol and expectation can make a huge difference.

Re: Call Me Maybe: MongoDB Stale Reads

#110

Post by sciurus »

jodah wrote: post #58
bkeroack wrote: post #4

If you are a database author and you get a bug report from Kyle, spend a long time thinking about it before closing the issue as invalid.

They're not the only ones: https://github.com/elastic/elasticsearch/issues/2488#issueco...

It looks like that specific issue was closed because the other problems reported are being tracked in other issues.
Post reply on HN