Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Call Me Maybe: MongoDB Stale Reads

aphyr.com

Post reply on HN
Print view View original

61–70 of 150 posts

Re: Call Me Maybe: MongoDB Stale Reads

#61

Post by rdtsc »

scurvy wrote: post #56
iconfluence wrote: post #46

Earlier quoted context omitted.

I think at one point the employees of Basho knew how to write distributed DBs - Riak is the most advanced AP DB from a distributed systems theory perspective. However, in recent months, their CEO, CTO and Chief Architect have left, as well as many of their prominent engineers. Worryingly, the new CTO seems content to make inane comments about "Data Gravity" [0]. [0] - http://www.kdnuggets.com/2015/03/interview-dave-m…

Indeed. We're evaluating Riak CS and Swift. In a vacuum, I'd choose Riak CS any day. Knowing all of the drama at Basho, we're in a holding pattern at best and leaning (reluctantly) towards Swift. Basho needs to sell damn fast or just call it a day and open source the enterprise version.

This is kind of surprising to hear. I had no idea.

Well in that case have you heard about LeoFS? Wonder if it overlaps with any of the Riak CS features for you.

http://leo-project.net/leofs/

Re: Call Me Maybe: MongoDB Stale Reads

#62

Post by tootie »

meritt wrote: post #39
addisonj wrote: post #28

Mongo absolutely nailed creating a database that is easy to get started with and even do things that are traditionally more 'hard' such as replication. It is still super attractive for me to pick it up for small projects, even after dealing with its (many) pain points both in development and operational settings. Given this, it is so tragic to see how dismissive they have been in regards to the consistency issues tha…

After MongoDB published their write speed benchmarks based entirely on unacknowledged writes (e.g. how fast can you write to a socket?), it's been a long downhill ride with an immense amount of inexplicable ignorant support.

If they are making money, why would they care? There's lots of shitty software raking in huge license fees based on misplaced reputation.

Re: Call Me Maybe: MongoDB Stale Reads

#63

Post by tootie »

timmaxw wrote: post #24
lobo_tuerto wrote: post #20

I think it would be great to see one of these done for RethinkDB :)

RethinkDB engineer here. RethinkDB currently doesn't support automatic failover, so this test couldn't be performed for RethinkDB yet. But when we implement automatic failover we're planning to test it against Jepsen. That will probably be sometime in the next few months.

DB engineers living in mortal fear of Kyle is where we want to be.

Re: Call Me Maybe: MongoDB Stale Reads

#65

Post by chatman »

threeseed wrote: post #9
jxf wrote: post #7

The most interesting lessons from the Jepsen series: * You should never trust, and always verify, the claims made by database manufacturers. * Especially when those claims relate to data integrity. * Super-especially when every safety level provided by the manufacturer that includes the word "SAFE" is actually unsafe.

Actually the broader lesson would be to assume the worst in your application layer and try and remediate/verify wherever possible. If you look at his articles: Redis, PostgreSQL, Cassandra, ElasticSearch etc all had data consistency errors. And none of those have vendors making any claims. It's pretty sobering to say the least.

Apache Solr's jepsen tests have had very good results.

Re: Call Me Maybe: MongoDB Stale Reads

#66

Post by threeseed »

zapov wrote: post #51
threeseed wrote: post #36

Earlier quoted context omitted.

I was talking about data consistency more broadly. "However, two writes (215 and 218) succeeded, even though they threw an exception claiming that a failure occurred". This obviously isn't ideal behaviour from the perspective of a developer but isn't necessarily not correct. My broader point was that you need to assume the worst from your database in the application layer. Which I think you missed.

I read your post as FUD regarding Postgres. That article has various issues, for example, calling Postgres commit protocol as a special case of two phase commit is not really correct. Postgres has 2pc: http://www.postgresql.org/docs/9.2/static/sql-prepare-transa... but that was not tested. The described behavior is "expected" and "understood". Saying that "you should assume worst from your database" is not something…

I wasn't spreading FUD about anything. From the article there was issues with every databases expected behaviour. My point again was that you should expect and manage failure in your application layer. It's what sensible architecture looks like.

And ACID does NOT guarentee that you will not lose data. It is a theory not an implementation. I have lost data with both Oracle and Teradata due to bugs.

Re: Call Me Maybe: MongoDB Stale Reads

#67

Post by cheeseprocedure »

erewh0n wrote: post #37
bkeroack wrote: post #4

If you are a database author and you get a bug report from Kyle, spend a long time thinking about it before closing the issue as invalid.

Indeed, database vendors should aim to have their software "Jepsen certified".

Agreed!

As an example, HashiCorp included Jepsen test results in their Consul documentation: https://consul.io/docs/internals/jepsen.html

Re: Call Me Maybe: MongoDB Stale Reads

#68

Post by jodah »

jxf wrote: post #23

Question: How do I actually run Kyle's tests to see this for myself? (Not that I don't believe him, I just want to play around a bit.) When I run `lein install` and then `lein test`, I get: ╰─▶ ψ lein test Exception in thread "main" java.io.FileNotFoundException: Could not locate jepsen/db__init.class or jepsen/db.clj on classpath: , compiling:(mongodb/core.clj:1:1) at clojure.lang.Compiler.load(Compiler.java:7142) a…

Update the version of the "jepsen" dependency in project.clj to 0.0.3. You'll need to do this for each of the test projects you want to run.

FWIW, this was filed the other day: https://github.com/aphyr/jepsen/issues/52

Re: Call Me Maybe: MongoDB Stale Reads

#69

Post by ddlatham »

aidanhs wrote: post #22
jxf wrote: post #17

Earlier quoted context omitted.

That island of safety is a rapidly shrinking one. Jepsen covers more ground with each new post from Kyle. :)

Has there been any CP system passing a call-me-maybe test first time apart from Zookeeper [1]? [1] https://aphyr.com/posts/291-call-me-maybe-zookeeper

HBase. With the caveats that it's piggy backing off the success of Zookeeper, and the tests were not run by Kyle himself:

https://eng.yammer.com/call-me-maybe-hbase/

Re: Call Me Maybe: MongoDB Stale Reads

#70

Post by zapov »

threeseed wrote: post #66
zapov wrote: post #51

Earlier quoted context omitted.

I read your post as FUD regarding Postgres. That article has various issues, for example, calling Postgres commit protocol as a special case of two phase commit is not really correct. Postgres has 2pc: http://www.postgresql.org/docs/9.2/static/sql-prepare-transa... but that was not tested. The described behavior is "expected" and "understood". Saying that "you should assume worst from your database" is not something…

I wasn't spreading FUD about anything. From the article there was issues with every databases expected behaviour. My point again was that you should expect and manage failure in your application layer. It's what sensible architecture looks like. And ACID does NOT guarentee that you will not lose data. It is a theory not an implementation. I have lost data with both Oracle and Teradata due to bugs.

Well, you said that Postgres had data consistency errors (referencing Aphyr articles). This is not true (at least regarding that article).

Aphyr article about Postgres could be renamed to call-me-maybe-acid-db-over-the-network and could remain the same.

Post reply on HN