Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen Disputes MongoDB's Data Consistency Claims

infoq.com

Post reply on HN
Print view View original

31–40 of 416 posts

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#31

Post by scarface74 »

Smaug123 wrote: post #28
seemslegit wrote: post #14

"We found that due to these weak defaults, MongoDB’s causal sessions did not preserve causal consistency by default: users needed to specify both write and read concern majority (or higher) to actually get causal consistency. MongoDB closed the issue, saying it was working as designed, and updated their isolation documentation to note that even though MongoDB offers “causal consistency in client sessions”, that guara…

Kyle's point is that it's arguably valid but certainly unhelpful: the default settings are liable to lead to data loss. Moreover, he draws attention specifically to transactions as something which you would expect to make things safer, but in fact there's a rather arcane part of the documentation that notes that you need to manually specify both read and write concerns on every transaction individually if you want tr…

How is this any different than DynamoDB where you specify that you want either eventual consistency vs strong consistency? DDB also does eventual consistent reads by default.

Is the argument that Mongo’s documentation isn’t clear?

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#32

Post by Mister_Snuggles »

pletnes wrote: post #22
aeonsky wrote: post #20

Earlier quoted context omitted.

Postgres has terrible indexing with json. It doesn’t keep statistics so simple queries sometimes take much longer than expected due to query planner not knowing much about the data.

DB noob question: if you know that you should be indexing on a json attribute, can’t you put it into a «proper column» and index there?

There are a number of ways to do this:

* Extract the attributes you're interested in into their own columns, index these. With the extraction happening outside the database, this is the most flexible option.

* Similar to above, use a trigger to automatically extract these attributes.

* Also similar to above, used a generated column[0] to automatically extract these attributes.

* Create an index on the expression[1] you use to extract the attributes.

My use a JSON in PostgreSQL tends towards the first option. This works well enough for cases where documents are ingested and queried, but not updated. The last three options are automatic - add/change the JSON document and the extracted/indexed values are automatically updated.

[0] https://www.postgresql.org/docs/12/ddl-generated-columns.htm...

[1] https://www.postgresql.org/docs/12/indexes-expressional.html

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#33

Post by Smaug123 »

scarface74 wrote: post #31
Smaug123 wrote: post #28

Earlier quoted context omitted.

Kyle's point is that it's arguably valid but certainly unhelpful: the default settings are liable to lead to data loss. Moreover, he draws attention specifically to transactions as something which you would expect to make things safer, but in fact there's a rather arcane part of the documentation that notes that you need to manually specify both read and write concerns on every transaction individually if you want tr…

How is this any different than DynamoDB where you specify that you want either eventual consistency vs strong consistency? DDB also does eventual consistent reads by default. Is the argument that Mongo’s documentation isn’t clear?

I trust Kyle when he tells me that the behaviour he observes is surprising. From the analysis (https://jepsen.io/analyses/mongodb-4.2.6):

"In order to obtain snapshot isolation, users must be careful not only to set the read concern to snapshot for each transaction, but also to set write concern for each transaction to majority. Astonishingly, this applies even to read-only transactions."

"This behavior might be surprising, but to MongoDB’s credit, most of this behavior is clearly laid out in the transactions documentation… MongoDB offers database and collection-level safety settings precisely so users can assume all operations interacting with those databases or collections use those settings; ignoring read and write concern settings when users perform (presumably) safety-critical operations is surprising!"

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#34

Post by hartator »

toomuchtodo wrote: post #19
hartator wrote: post #17

[repost - asking for help] I am disappointed with the direction that MongoDB took this past few years. Going ACID shows in benchmarks [1] and it’s not advisable if you are using MongoDB for stats and queue. (No one uses MongoDB for financial transactions despite the changes.) And the recent change to a restrictive license is worrisome as well. I have been thinking of forking 3.4 and make it back to “true” open source…

Why not use PostgreSQL instead? It supports a JSON document data type natively. It also has exceptional stewardship as an open source project. Mongo should never be a first choice, but a last choice for edge cases.

> Why not use PostgreSQL instead? It supports a JSON document data type natively.

Yes, that's the thing, it's just a field type. It's not really that different than dumping your JSON in a TEXT column. MongoDB is fun because it's truly JSON - BSON - so you don't have to run migrations, you can store complex documents, and have a more object oriented way of storing your data than SQL.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#36

Post by macintux »

madhadron wrote: post #27

In the circles I run in, MongoDB is regarded as a joke and the company behind it as basically duplicitous. For example, they still list Facebook as their first user of MongoDB on their website, for example, but there is no MongoDB use in Facebook hasn't been for years (it came in only via a startup acquisition). I had the misfortune to use MongoDB at a previous job. The replication protocol wasn't atomic. You would f…

The joke I learned early on: "Migrating away from Mongo is trivial: wait long enough, and all your data will be gone anyway."

I imagine things are better now.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#37

Post by doublesCs »

scarface74 wrote: post #31
Smaug123 wrote: post #28

Earlier quoted context omitted.

Kyle's point is that it's arguably valid but certainly unhelpful: the default settings are liable to lead to data loss. Moreover, he draws attention specifically to transactions as something which you would expect to make things safer, but in fact there's a rather arcane part of the documentation that notes that you need to manually specify both read and write concerns on every transaction individually if you want tr…

How is this any different than DynamoDB where you specify that you want either eventual consistency vs strong consistency? DDB also does eventual consistent reads by default. Is the argument that Mongo’s documentation isn’t clear?

Oops. Turns out I was right.

https://news.ycombinator.com/item?id=23271211

The timing of this is absolutely beautiful.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#38

Post by [deleted] »

hartator wrote: post #34
toomuchtodo wrote: post #19

Earlier quoted context omitted.

Why not use PostgreSQL instead? It supports a JSON document data type natively. It also has exceptional stewardship as an open source project. Mongo should never be a first choice, but a last choice for edge cases.

> Why not use PostgreSQL instead? It supports a JSON document data type natively. Yes, that's the thing, it's just a field type. It's not really that different than dumping your JSON in a TEXT column. MongoDB is fun because it's truly JSON - BSON - so you don't have to run migrations, you can store complex documents, and have a more object oriented way of storing your data than SQL.

[deleted]

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#39

Post by scarface74 »

Smaug123 wrote: post #33
scarface74 wrote: post #31

Earlier quoted context omitted.

How is this any different than DynamoDB where you specify that you want either eventual consistency vs strong consistency? DDB also does eventual consistent reads by default. Is the argument that Mongo’s documentation isn’t clear?

I trust Kyle when he tells me that the behaviour he observes is surprising. From the analysis ( https://jepsen.io/analyses/mongodb-4.2.6 ): "In order to obtain snapshot isolation, users must be careful not only to set the read concern to snapshot for each transaction, but also to set write concern for each transaction to majority. Astonishingly, this applies even to read-only transactions." "This behavior might be su…

There is difference between “Mongo’s documentation sucks” and “Mongo is technically deficient”. The former can be corrected by updating the documentation.

Yes, I agree as far as the end user is concerned, they are losing data either way.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#40

Post by scarface74 »

doublesCs wrote: post #37
scarface74 wrote: post #31

Earlier quoted context omitted.

How is this any different than DynamoDB where you specify that you want either eventual consistency vs strong consistency? DDB also does eventual consistent reads by default. Is the argument that Mongo’s documentation isn’t clear?

Oops. Turns out I was right. https://news.ycombinator.com/item?id=23271211 The timing of this is absolutely beautiful.

So now we shouldn’t ever trust a project because they don’t have good technical writers?

I don’t have a dog in the Mongo fight. I haven’t done an implementation on top of it in years and probably the next time I do something with “Mongo” it will probably be AWS’s Document DB with Mongo support. That’s based on AWS’s own code and storage tier and doesn’t have the same characteristics as Mongo proper.

Post reply on HN