Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen Disputes MongoDB's Data Consistency Claims

infoq.com

Post reply on HN
Print view View original

321–330 of 416 posts

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#321

Post by throwaway_pdp09 »

HelloNurse wrote: post #305
kyllo wrote: post #128

Earlier quoted context omitted.

I was floored by this comment yesterday from one of their Developer Relations people: > Did any of you actually read the article? We are passing the Jepsen test suite and it was back in 2017 already. So, no, MongoDB is not losing anything if you know what you are doing. https://twitter.com/MBeugnet/status/1253622755049734150?s=20 Can you imagine saying the phrase "if you know what you are doing," in public, to your u…

Indirectly stating that they aren't good enough to use MongoDB properly could be offensive for thin-skinned developers, but it's only bad attitude. I'm much more concretely worried by a software design for which the authors (not hostile critics) consider "if you know what you are doing" an acceptable safety and quality standard for data integrity.

I've never used MongoDB and hopefully never will, but I have to defend that phrase in the larger context. A great number of programming problems I've seen have been to do with people not knowing what they're doing with complex tools, it really is that simple.

Actual example: a long time ago someone in my company introduced a race condition into their product because they didn't know about transaction isolation levels (or the locking facilities exposed by MSSQL). I can give many more, as I'm sure anybody here can.

All complex tools need large investment in time to understand. I suspect MongoDB's team are using that as an excuse but in general you must jnow what you're doing with any tool.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#322

Post by conistonwater »

pastacacioepepe wrote: post #294
thaumasiotes wrote: post #290

Earlier quoted context omitted.

> I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”. This kind of thing is a scourge. I had a Chinese friend respond to something I said once by saying "that's nice". It looks so innocent... but it's really hard to overlook the fact that "that's nice" is a serious insult coming from a native speaker. I had to ask them to please…

Did the joke just went over my head or "that's nice" is a bad thing for real?

No it's not, it depends mostly on the tone of the voice. It's often enough sarcastic, but really doesn't have to be.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#323

Post by bryanrasmussen »

kyllo wrote: post #207
jen20 wrote: post #177

Earlier quoted context omitted.

Firstly let me point out that this response is neither intended as a defence of MongoDB defaults which are atrocious, or of the company, who are arguably duplicitous. However I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”.

Perhaps, but the point is, if you're working in DevRel and you think your role is to defend the product from criticism, and you do it by placing the onus on the developer to figure out how to use your product safely, you've totally lost the plot.

I think personally it's an abusive tactic on their part, lots of devs are concerned with being smart, by saying if you know what you're doing you can get the insecure ones to shut up.

But obviously it is a tactic of the doomed.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#324

Post by bryanrasmussen »

pastacacioepepe wrote: post #294
thaumasiotes wrote: post #290

Earlier quoted context omitted.

> I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”. This kind of thing is a scourge. I had a Chinese friend respond to something I said once by saying "that's nice". It looks so innocent... but it's really hard to overlook the fact that "that's nice" is a serious insult coming from a native speaker. I had to ask them to please…

Did the joke just went over my head or "that's nice" is a bad thing for real?

I use Great so consistently sarcastic that my kids have unfortunately learned that Great means something not great.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#325

Post by tgsovlerkhgsel »

thomascgalvin wrote: post #53

You can tell a lot about a developer by their preferred database. * Mongo: I like things easy, even if easy is dangerous. I probably write Javascript exclusively * MySQL: I don't like to rock the boat, and MySQL is available everywhere * PostgreSQL: I'm not afraid of the command line * H2: My company can't afford a database admin, so I embedded the database in our application (I have actually done this) * SQLite: I'm…

How far does SQLite scale? Obviously not good for anything public facing with thousands of concurrent users, obviously good enough for something you only use yourself, but what about internal tools with a couple hundred users total (few of them concurrent) - where's the limit when it starts slowing down?

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#326

Post by squeaky-clean »

vosper wrote: post #318
neeleshs wrote: post #276

Earlier quoted context omitted.

Sends emails when scanned vs returned record ratio is greater than a threshold. Not quite sure how something like this would be different for MySQL/postgres. If an index is missing for you query pattern, wouldn't you create an index in MySQL/postgres?

My claim isn’t that indexes are a bad idea. The comment I replied to suggested that Mongo’s marketing targets people who don’t want to learn how to use indexes; my claim is that the reality of the hosted product is that it frequently suggests that you add indexes (presumably doing so helps with Mongo Corps hosting margins, as well as with performance [which is laughable even with indexes])

> my claim is that the reality of the hosted product is that it frequently suggests that you add indexes

But it auto-suggests what index to use and has a button for you to immediately apply it. I'd say it definitely intends for you to avoid learning how indexes in MongoDB work. The index suggestions it makes are often terrible.

> performance [which is laughable even with indexes]

It really depends on your use-case and how you can structure and query your data. For one such service I'm the lead on I haven't found anything faster that doesn't sacrifice the ability to do more generic queries when we occasionally need to. (Yes we've benchmarked against postgres json fields which is always everyone's first suggestion)

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#327

Post by jwr »

If you're looking for MongoDB done right, it does exist and it's called RethinkDB. For some reason it didn't catch on and become popular — but it's nicer, and most importantly, it doesn't lose your data.

Data point: I have been running my production system (a fairly complex SaaS) on RethinkDB for the last 4 years.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#328

Post by mping »

tgsovlerkhgsel wrote: post #325
thomascgalvin wrote: post #53

You can tell a lot about a developer by their preferred database. * Mongo: I like things easy, even if easy is dangerous. I probably write Javascript exclusively * MySQL: I don't like to rock the boat, and MySQL is available everywhere * PostgreSQL: I'm not afraid of the command line * H2: My company can't afford a database admin, so I embedded the database in our application (I have actually done this) * SQLite: I'm…

How far does SQLite scale? Obviously not good for anything public facing with thousands of concurrent users, obviously good enough for something you only use yourself, but what about internal tools with a couple hundred users total (few of them concurrent) - where's the limit when it starts slowing down?

Curiously, I just read this: https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-q...

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#329

Post by andreareina »

nieve wrote: post #293
__Joker wrote: post #282

Earlier quoted context omitted.

Interesting, is there a blog around discussing this in detail ? If not would be kind enough to go more into detail.

I don't think anyone posted about their particular system, but it's not unknown now. If you google "filesystem as a database" there are some relevant hits. One super simple and probably not ideal, but at least balanced version uses a hash of some primary key like customer row id as the file index, then partitions the items into directories with all permutations at each level (or only populated ones) based on successi…

That's how git stores its objects:

    18:35 $ tree .git/objects/
    .git/objects/
    ├── 02
    │   └── 9581d0c8ecb87cf1771afc0b4c2f1d9f7bfa82
    ├── 3b
    │   └── 97b950623230bd218cef6aebd983eb826b2078
    (...)
    ├── info
    └── pack
        ├── pack-b1fe2364423805afb6b1c03be0811c93b19dedc9.idx
        └── pack-b1fe2364423805afb6b1c03be0811c93b19dedc9.pack

    10 directories, 10 files

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#330

Post by tannhaeuser »

marcan_42 wrote: post #301
tasuki wrote: post #297

Earlier quoted context omitted.

Why is storing json in a database important to you? Whenever I see json fields in PostgreSQL/MySQL, I know I'm most likely in for inconsistent data and a world of pain.

Why is storing all data normalized in a database important to you? Whenever I see an exceedingly complicated database schema, I know I'm most likely in for unfixable legacy DB warts and incessant schema updates for every little feature. Hyperbole aside, the best option often is somewhere in between. I find that a relational database with columns for primary keys/relations/anything used in WHERE statements in the norm…

Are you saying fully normalized RDBs end up reimplementing key/value stores, and then suggest to just use a JSON blob instead? The reason why RDBs are normalized is to avoid update anomalies, and it isn't really difficult to understand. Technically, adding a column to a DB table is a matter of "ALTER TABLE T ADD (mycol NUMBER)", but it sounds like the real problem to be worked on is the organizational overhead going on. At least the way you're describing your use of JSON blobs in a relational DB really sounds like a guerilla tactic to sidestep DBAs gatekeeping your DB. JSON is just an accidental data serialization format supported by browsers OOTB. If you're using any other language in your backend, like Java, then you need additional serialization libs to read and write JSON anyway (like with every other format such as XML), and the convenience of JSON is lost. Though lately browser apps become so bloated with MVw frameworks and whatnot, I wonder if the original point of using JSON - that it can be handled as JavaScript object literal OOTB - is really all that important. If you've worked with MongoDB, you'll know that Mongo's JSON is not quite JSON, but uses additional ad-hoc constructs to re-introduce some typing (such as for dates and IDs).

So I have to agree with GP wondering why JSON is so important to people, and is even portrayed as a relief or saviour to developers. In my experience, JSON in a relational DB is always a sign of organizational failure, developers using the wrong tool for the job, or not knowing what they want or do.

Post reply on HN