Live data from Hacker News

Jepsen Disputes MongoDB's Data Consistency Claims

infoq.com

331–340 of 416 posts

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#331
post #313

Ok, so defaults suck, marketing is misleading, documentation and error messages are not exactly obvious. Assuming you are already stuck in the soup, putting those issues aside and getting practical instead instead of throwing more fire on the discussion: If you set w: majority and r: linearizable/snapshot, both on collection, client and on transactions. Plus assuming you accept snapshot over Isolation. How bad are th…

"Informally, I would summarize the CAP theorem as: If the network is broken, your database won’t work."

- Dwight Merriman, former CEO, and "one of the original authors of MongoDB" [1]

A word to the wise suffices. Sometimes the word in question is implied by other words.

For those who get this oblique post, note that throwing the above bon mot in an interview session for a "distributed systems engineer" and asking for an opinion is a excellent way to differentiate between Peter Principle and Principal Engineer.

[1]: https://web.archive.org/web/20100903213540/http://blog.mongo...

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#332

Earlier quoted context omitted.

I like it, but I don't really really chime in on threads where it's mostly Mongo bashing and jokes, and I'm guessing others who like MongoDB do the same. But I'd think MongoDB the company increasing in revenue isn't totally related to the quality of MongoDB the database. In fact a lot of their products seem to be targeting the "I don't want to learn how to set it up and understand indexes" crowd.

I have a legacy app that uses CouchDB under the hood. It's not actually appropriate for how it's being used, but I can see how it could be used in an appropriate way. Over time I got to respect and even like CouchDB (just not the way we're using it ;-) ). However before CouchDB apparently we started with MongoDB and then migrated to Couch after problems with data integrity in Mongo (this was a long time ago and I've…

IMO, MongoDB is really good if all your indexes and your data (after sharding) can fit in memory, and you need queries that are a simple match query against a small number of keys or a range query against a date or float type. And also your data doesn't fit into a row/column structure very easily (otherwise go SQL). The aggregation framework, joins, other features are all nice to have for occasional use, but you can't be doing those 10k times a second.

It's great for things like a realtime layer of some app that merges data with a slower and more historical layer of data running on a SQL engine or something safer. Or for services that provide realtime or recent-time analytics, while storing your historical data somewhere else (see any patterns here so far? :P ). In my case the main usage is for an advertising bid and content serving engine, which was pretty much the ideal example use-case for MongoDB mentioned in books I read years ago when first learning it.

Just to note, yes the data integrity problems are "fixed", but only if you configure your instances properly and your read and write statements properly. It's not terribly hard to do, but I don't know if I would really go recommending MongoDB for newbies. If you know how to configure it properly for your data-safety needs, and would benefit from being able to have a flexible schema in early development.... I'd still maybe suggest looking at other document DBs unless you need the read/write speed Mongo can give on simple queries (and fresh projects probably do not need that)

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#333

Earlier quoted context omitted.

> I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”. This kind of thing is a scourge. I had a Chinese friend respond to something I said once by saying "that's nice". It looks so innocent... but it's really hard to overlook the fact that "that's nice" is a serious insult coming from a native speaker. I had to ask them to please…

Did the joke just went over my head or "that's nice" is a bad thing for real?

"That's nice" is a way of explicitly stating that you don't care to any degree about whatever the other person just said.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#334

Earlier quoted context omitted.

Did the joke just went over my head or "that's nice" is a bad thing for real?

No it's not, it depends mostly on the tone of the voice. It's often enough sarcastic, but really doesn't have to be.

Tone of voice does not exist in text.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#335

Earlier quoted context omitted.

Sends emails when scanned vs returned record ratio is greater than a threshold. Not quite sure how something like this would be different for MySQL/postgres. If an index is missing for you query pattern, wouldn't you create an index in MySQL/postgres?

> If an index is missing for you query pattern, wouldn't you create an index in MySQL/postgres? No. I’d consider adding an index. An index is not free, it comes at a cost and that cost may well be higher than the costs of not having that index. For example, if a reporting query that runs once every few hours is lacking an index, the cost of updating that index on every write (and the disk space/memory used) may well…

Exactly this. If you connect to your database and run a couple un-indexed queries to explore your data, any query which takes over 100ms will appear for the next 30 days in the "Performance Advisor" tab and it will offer you indexes to create targeted at that query. Just based on a query you did manually one time.

Just to note, this is referring to the features for a hosted databased on cloud.mongodb.com, and not something built into MongoDB the database.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#336
post #327

If you're looking for MongoDB done right, it does exist and it's called RethinkDB. For some reason it didn't catch on and become popular — but it's nicer, and most importantly, it doesn't lose your data. Data point: I have been running my production system (a fairly complex SaaS) on RethinkDB for the last 4 years.

RethinkDB is no longer supported, its major caveat.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#337
post #13

Earlier quoted context omitted.

Maybe they're too busy spending their MDB money. https://www.google.com/search?q=NASDAQ:+MDB

I genuinely am confused by comments like this. Are companies not supposed to invest money into their product, sales, people etc ? And why does being listed on the NASDAQ imply being flush with money ?

The culture changes to one of sales & corporatism from one of engineering and focus on product. Why would they be here?

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#338

Earlier quoted context omitted.

Why is storing all data normalized in a database important to you? Whenever I see an exceedingly complicated database schema, I know I'm most likely in for unfixable legacy DB warts and incessant schema updates for every little feature. Hyperbole aside, the best option often is somewhere in between. I find that a relational database with columns for primary keys/relations/anything used in WHERE statements in the norm…

It really depends on applications. In most databases I've seen, if information is unimportant it is just omitted and if it's important it's worth putting in a proper column to allow relational operations; and if important information is big it's usually unstructured or ad-hoc (e.g. log messages, PDF documents), not a "document" and particularly not JSON.

I disagree. Most of the information in a database is not something I'd want to query by (otherwise I'd be indexing every column, which I'm not), certainly not as part of the normal application.

What's the point of putting, say, every single user management field into columns in a "users" table? The regular application is never going to have to relate users by their CSRF token, or what their UI language is, or any of the other dozens of incidental details associated to a user, some visible, some implementation details.

What matters are things like the username, email, name - things the app needs to actually run relational operations on.

If you look at any real application, pretty much everyone has given up on trying to keep everything relational. That would be a massive pain in the ass and you'd end up with hundreds of columns in your users table. You'll find some form of key/value store attached to the user instead. And if you're going to do that, you might as well use a json field.

And if you do use a json field with a database engine that supports it well, like PostgreSQL, you'll find that it can be indexed if you need it anyway, and querying it is easier than joining a pile of tables implementing a KVS or two. Because yes, I might want to make a report on what users' UI language is some day, but that doesn't mean it has to be a column when Postgres is perfectly happy peeking into jsonb. And I don't need an index that will just cause unnecessary overhead during common update operations that don't use it, when I can just run reports on a DB secondary and not care about performance.

I designed an application in this manner and it has turned out exceedingly well for me. We have only had about a dozen schema changes total across the lifetime of the app. One of them involved some JSON querying to refactor a field out of JSON and into its own table (because requirements changed) and that was no problem to run just like any other database update. If we need to move things to columns we will, but starting off with an educated guess of what will need to be a column and dumping everything else into JSON has undoubtedly saved us a lot of complexity and pain.

One of our tables is just a single primary key and then a json blob. It stores event configuration. Why? Because it's always loaded entirely and there is never any reason to run relational ops on it. It's a huge json blob with lots of little details, only edited administratively, with nested fields and sub-fields (which is way more readable than columns, which are a flat namespace), including a sub-document that is actually a structure that drives generation of a pile of HTML forms. If I'd tried to normalize this I would have ended up with a dozen tables, probably more complexity than the entire rest of the DB, and a huge pile of code to drive it all, and every single application change that added an admin knob would've had to involve a dabatase schema change... All for what? Zero benefit.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#339

Earlier quoted context omitted.

No it's not, it depends mostly on the tone of the voice. It's often enough sarcastic, but really doesn't have to be.

Tone of voice does not exist in text.

Just like data consistency doesn’t exist in MongoDB

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#340

Earlier quoted context omitted.

No it's not, it depends mostly on the tone of the voice. It's often enough sarcastic, but really doesn't have to be.

Tone of voice does not exist in text.

Then you have to follow the principle of charity and assume they're being nice.
Post reply on HN