Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB 3.4.0-rc3

jepsen.io

Post reply on HN
Print view View original

141–150 of 165 posts

Re: MongoDB 3.4.0-rc3

#141

Post by navalsaini »

x0x0 wrote: post #130
vacri wrote: post #125

Earlier quoted context omitted.

As a sysadmin, I got tired of the devs constantly ragging on Mongodb (the same folks that selected it before I was hired). I eventually got fed up and said: "why do we use it if you all hate it so much. Let's replace it. What do you want to use instead, it's easy for me to set up something new". Cue everyone going "ah, it's not so bad, really..." MongoDB is the Nickelback of databases: a reasonable act that's not goi…

Well, I'm a member of the MongoDB is worth quitting over club. I replaced it at a former employer with postgres. Eng waved bye to an endless stream of operational issues, and customers saw better uptimes and much much faster responses.

When was that and which version of mongodb? Do you suppose things have improved after 3.0 etc.

Re: MongoDB 3.4.0-rc3

#142

Post by _Codemonkeyism »

I know there are a lot of bad vibes on HN about MongoDB, but I also know several companies which MongoDB enabled with the right libraries to much faster iterate on "schema" and business requirements (in their growth phase) than with other databases at that time (some years ago). The schemaless approach of MongoDB suited their fast changing needs much better than e.g. RDBMs.

What currently still makes MongoDB nicer than e.g. Postgres JSON are the libraries embracing schemaless while JSON in PG libs still feels tacked on.

Recently I had some bad experience with MongoDB support though.

No experience with RethinkDB.

Re: MongoDB 3.4.0-rc3

#143

Post by pkolaczk »

aphyr wrote: post #119
pkolaczk wrote: post #115

Earlier quoted context omitted.

http://www.datastax.com/dev/blog/testing-apache-cassandra-wi...

Pretty sure Cassandra still uses local timestamps and last-write-wins, which means you can't do safe updates. Also pretty sure they didn't fix the timestamp-collision row-isolation issue I talked about, because it was on HN's front page a few months ago. ;-)

Using timestamps and last write wins is a well known and well documented behavior, so you must assume updates can be sometimes applied in a different order than they were submitted. However, reordering updates is not the same as rolling back writes as was in case of Mongo. Cassandra does not promise linearizability in this mode of operation (if not using LWTs), and this is a tradeoff to get better availability. There is no better way than last write wins if you want high availability and partition tolerance and don't want to pay the performance and availability price for a consensus algorithm like Paxos or Raft. And no, vector clocks do not solve this problem in practice at all.

As for the post on HN from a few months ago, I remember only one guy who mixed LWT and non LWT updates and was surprised with lack of linearizability. Not a Cassandra fault if somebody doesn't know what he's doing.

Re: MongoDB 3.4.0-rc3

#144

Post by nurettin »

deathanatos wrote: post #81
nurettin wrote: post #79

Earlier quoted context omitted.

In my experience, mongo lets you check the end result and try inserting again.

How do you expect to check the end result? The article's Jepsen analysis shows that both the v0 and v1 replication protocols (excepting the very latest version of v1 that appears to be in response to this) can result in acknowledged writes being lost. I.e., the DB tells you, for a write sent with a majority, that the write was successful — to a majority! Subsequently (and, if I understand the article, possibly not im…

Well, thanks for the question!

You check the result with getLastError which, as you described, can be used to ensure a majority agrees with the write. But you normally don't use getLastError that way. Because a majority might not even be concerned with that particular write. They are, after all, shards. Instead you check if primary got the write. If primary disconnects while you are checking, you catch the exception and try checking until a new primary is decided. And if your check result is not ok, you try inserting again. That's as reliable as it gets when inserting to any database including SQL databases that support transactions.

Re: MongoDB 3.4.0-rc3

#145

Post by ralfn »

dajohnson89 wrote: post #75
nomel wrote: post #61

Earlier quoted context omitted.

A database that doesn't need to guarantee data integrity will always be faster. How is the performance with the new v1 protocol? (still faster, I'm sure)

Sorry if this is a stupid question. But how important is guaranteed integrity, in practice? (BTW, is what you're referring to considered transactions?) Obviously, for things like banking you need guarantees, but I'm not sure how often reads/writes to mongodb just disappear out of thin air.

>but I'm not sure how often reads/writes to mongodb just disappear out of thin air.

Which would imply you have to restore from backups continuesly. Or else any bug you run into might be the result of data corruption.

Its the fact that you cant even tell. And its not just about loosing data. If you rely upon a consistent world view (i.e. schedule payment, jobs scheduling, vms provision or shipping, account systems) the risk is unacceptable.

Re: MongoDB 3.4.0-rc3

#146

Post by omginternets »

aphyr wrote: post #117
omginternets wrote: post #109

Earlier quoted context omitted.

Whoops, wrong link. Here's the correct one: https://news.ycombinator.com/item?id=13591048 It seems the problems were indeed "solved" and not solved .

I'm not exactly sure what you're trying to say, but perhaps I can help provide context: In 2013, I performed an unpaid analysis, in my nights and weekends, of MongoDB. I found a bug leading to the loss of acknowledged writes with majority write concern. Mongo fixed this bug within a few weeks. In 2015, I performed a followup test as a part of my work at Stripe. I confirmed dirty reads (which were already documented,…

Hi Kyle. I'm not trying to put words in anybody's mouth (though perhaps my tone suggested I was -- sorry!)

This is exactly the info I was looking for. Thank you.

Re: MongoDB 3.4.0-rc3

#147

Post by disgusting_guy »

nurettin wrote: post #144
deathanatos wrote: post #81

Earlier quoted context omitted.

How do you expect to check the end result? The article's Jepsen analysis shows that both the v0 and v1 replication protocols (excepting the very latest version of v1 that appears to be in response to this) can result in acknowledged writes being lost. I.e., the DB tells you, for a write sent with a majority, that the write was successful — to a majority! Subsequently (and, if I understand the article, possibly not im…

Well, thanks for the question! You check the result with getLastError which, as you described, can be used to ensure a majority agrees with the write. But you normally don't use getLastError that way. Because a majority might not even be concerned with that particular write. They are, after all, shards. Instead you check if primary got the write. If primary disconnects while you are checking, you catch the exception…

You describe it like it is simple but that is ridiculous number of steps to simply check your data was actually written to the database.

>that's as reliable as it gets when inserting into any database including SQL

The difference being in a SQL database you call commit and all this happens for you automatically

Re: MongoDB 3.4.0-rc3

#148

Post by Fiahil »

bogomipz wrote: post #110
Fiahil wrote: post #96

Earlier quoted context omitted.

Yes. Mongo is an operational nightmare. I mean, who in their right mind would use a product where the recommended solution[0] to resync a stale replica (which happen all the time after a long netsplit) is to "perform an initial sync". Which, of course, mean "please remove everything and type this command". Crazy. [0]: https://docs.mongodb.com/manual/tutorial/resync-replica-set-...

Indeed, and the only practical means of performing a compaction is to "rm -rf" the data directory and let it resync from another replica set member. This is not documented of course.

This is documented (okay, a little "hidden")! It's written black on white in their documentation (see the link above):

> A replica set member becomes “stale” when its replication process falls so far behind that the primary overwrites oplog entries the member has not yet replicated. The member cannot catch up and becomes “stale.” When this occurs, you must completely resynchronize the member by removing its data and performing an initial sync.

> MongoDB provides two options for performing an initial sync:

> Restart the mongod with an empty data directory and let MongoDB’s normal initial syncing feature restore the data

> Restart the machine with a copy of a recent data directory from another member in the replica set.

Note: the second option is not a real option when you're dealing with a 700GB database. By the time you finish the copy the oplog will be too big anyway. Thus, making all these steps completely pointless.

And that's why it's so bad. They even acknowledge the "correct" solution is to rm your data and resync.

Re: MongoDB 3.4.0-rc3

#149

Post by bogomipz »

Fiahil wrote: post #148
bogomipz wrote: post #110

Earlier quoted context omitted.

Indeed, and the only practical means of performing a compaction is to "rm -rf" the data directory and let it resync from another replica set member. This is not documented of course.

This is documented (okay, a little "hidden")! It's written black on white in their documentation (see the link above): > A replica set member becomes “stale” when its replication process falls so far behind that the primary overwrites oplog entries the member has not yet replicated. The member cannot catch up and becomes “stale.” When this occurs, you must completely resynchronize the member by removing its data and…

It's documented for the use case of "stale replica" but I was referring to the use case for when you want a compaction to reclaim disk space. For that they recommend the db.repairDatabase() option but that requires you to have twice the size of your db available available on whatever partition your database is on. That was I said "practical." But yes the procedure is the same.

Re: MongoDB 3.4.0-rc3

#150

Post by ralfn »

vacri wrote: post #125
k33n wrote: post #78

Earlier quoted context omitted.

> Its not like competent teams were using it in production. Right? I've heard that about 1000 times a day for 6 years. Usually the person stating it is snickering as if they are clued into some unknown secret. Mongo does work in production at many shops, and in many forms. Sometimes it's used as the main database, sometimes it's used to house specific slices of data, etc.

As a sysadmin, I got tired of the devs constantly ragging on Mongodb (the same folks that selected it before I was hired). I eventually got fed up and said: "why do we use it if you all hate it so much. Let's replace it. What do you want to use instead, it's easy for me to set up something new". Cue everyone going "ah, it's not so bad, really..." MongoDB is the Nickelback of databases: a reasonable act that's not goi…

CouchDB, RethinkDB, PostgreSQL.

>OMG I hate it!" somehow signals membership to some cool clique of connoisseurs

You do realize you are the one dragging identity into the mix.

I dont understand how this is a big ego debate. MongoDB isnt that relevant and it isnt Nickelback, its a homeopathetic database. That is: if you use it for something else than caching (store data you cant afford to loose, or load balance and use the database as main mutex to deal with all concurrency issues) that would arguably be a very irresponsible choice.

Hell i hate most databases, because its hard to get right yet some have interesting trade-offs (ElasticSearch, Cassandra, CockroachDB).

And its not a subjective or even analog discussion where databases are more or less consistent or more or less durable. They fsync or they dont. They use raft with majority consensus or they dont.

If you as a sysadmin judge these emperical facts based on your prejudices about the sort of people that would agree or disagree with you than you are much more like the cool clique of connoisseurs than the people at the other end of your finger.

Its engineering, not wine tasting. The shape of the world isnt a subjective thing anymore than the durability of a database that doesnt fsync.

Post reply on HN