Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB 3.4.0-rc3

jepsen.io

Post reply on HN
Print view View original

101–110 of 165 posts

Re: MongoDB 3.4.0-rc3

#101

Post by hmottestad »

ClayFerguson wrote: post #99

These 'from the ground up' totally all-new-code approaches to DBs are just a scary proposition. Think of the thousands of man-years of effort that went into building MySQL, testing its codebase, and perfecting it's robustness (fail-proof-ness). What does MongoDB bring that couldn't have 'built on top' of MySQL codebase, and used MySQL transational layer as it's underpinnings. Sure, MongoDB gets all its performance ga…

Probably because there has been a lot of development in distributed databases in the last few years. Raft is fairly new, for one.

Taking really old code and trying to make it do new things is very very hard. After you've invested all that time and effort making mysql better, it's still a database owned by oracle.

Think how Java would have turned out if Sun made it to be cross compiled to C rather than run in a VM.

Re: MongoDB 3.4.0-rc3

#102

Post by aphyr »

ClayFerguson wrote: post #99

These 'from the ground up' totally all-new-code approaches to DBs are just a scary proposition. Think of the thousands of man-years of effort that went into building MySQL, testing its codebase, and perfecting it's robustness (fail-proof-ness). What does MongoDB bring that couldn't have 'built on top' of MySQL codebase, and used MySQL transational layer as it's underpinnings. Sure, MongoDB gets all its performance ga…

What does MongoDB bring that couldn't have 'built on top' of MySQL codebase, and used MySQL transational layer as it's underpinnings.

There are two challenges here. One is that MongoDB has a different data model than MySQL: hierarchical, schema-less documents instead of uniformly-typed flat tables. It's possible to map one to the other--look at Postgres' support for JSON datatypes. That involves extensions to both the storage format and query language, but it's certainly doable.

The bigger problem is that MySQL's local transactional isolation is nice, but not particularly helpful in a distributed context. Distributed transactions--even single-document ones--are still tough to implement correctly and efficiently. As an example, consider Percona XtraDB, or MySQL+Galera Cluster. Both are building on a serializable single-node system--but in a distributed context, they wind up failing to provide snapshot isolation, let alone serializability.

Re: MongoDB 3.4.0-rc3

#103

Post by pkolaczk »

mat_keep wrote: post #76
radicalbyte wrote: post #71

Earlier quoted context omitted.

Rethink and Cassandra both fare far better in Jepsen. Depending on the exact requirements Postgres (in the right configuration) and Redis are better options. I'd argue that for 90% of MongoDB users - the group whose requirements were met by MySQL/MyISAM - would be better served by Postgres or MySQL with a proper storage engine.

Cassandra testing on Jepsen. Interesting definition of "fare far better in Jepsen" https://aphyr.com/posts/294-jepsen-cassandra

Cassandra had a minor bug with timestamps which was fixed relatively fast and now it passes Jepsen fully. There were also a few implementation issues around LWT (Paxos) but at that time Paxos was very new. There were no architectural / fundamental changes needed to correct these issues. Also, even before fixing, Cassandra was losing only a tiny fraction of data than Mongo was losing. So, yes, this is what I call "far better", even if not initially perfect.

Re: MongoDB 3.4.0-rc3

#104

Post by Kiro »

always_good wrote: post #91
laichzeit0 wrote: post #73

Earlier quoted context omitted.

Do you think MongoDB is a good choice (given how easy it is to use) when you only care that 99.999% of your data that you insert should end up in the database? That's my use case. Best-effort integrity. I mostly just want a DB can insert and query fast for documents and am not really concerned if I lose a few documents here and there.

Why wouldn't you just use anything else that can manage to insert/read data without losing it? I don't really understand the angle of "can I get away with it anyways, tho?"

Some of us are already using MongoDB and are not so keen on replacing it.

Re: MongoDB 3.4.0-rc3

#105

Post by squeaky-clean »

ClayFerguson wrote: post #99

These 'from the ground up' totally all-new-code approaches to DBs are just a scary proposition. Think of the thousands of man-years of effort that went into building MySQL, testing its codebase, and perfecting it's robustness (fail-proof-ness). What does MongoDB bring that couldn't have 'built on top' of MySQL codebase, and used MySQL transational layer as it's underpinnings. Sure, MongoDB gets all its performance ga…

> Also think of the power that would be available if there were some relational table capability (true real MySQL ACID) right inside MongoDB whenever it was needed, if they were the 'same animal', rather than having to use two totally and completely separate DBs

Just use Postgres (or several other options) then? It has all that built right in if it's what you need. But...

> if you need NoSQL and also ACID in an app, which 99% of apps DO NEED, at some point,

...I doubt 99% of apps really need this.

> once an app grows beyond the round-one funding startup-toy phase and MongoDB falls flat in it's RDB capabilities.

Or don't try to force MongoDB to behave like a RDB and you won't hit those problems? I just moved a system from Postgres to MongoDB, and it's running faster on way cheaper hardware now. Not because either database is inherently better than the other, but the use case lined up with mongodb perfectly and the old model was leveraging Postgres really poorly. Eventual consistency is fine, and I can denormalize certain data for faster reads because I know it won't be modified later.

Re: MongoDB 3.4.0-rc3

#106

Post by geocar »

radicalbyte wrote: post #98
deathanatos wrote: post #81

Earlier quoted context omitted.

How do you expect to check the end result? The article's Jepsen analysis shows that both the v0 and v1 replication protocols (excepting the very latest version of v1 that appears to be in response to this) can result in acknowledged writes being lost. I.e., the DB tells you, for a write sent with a majority, that the write was successful — to a majority! Subsequently (and, if I understand the article, possibly not im…

It depends. Given a small cluster of reliable nodes on a reliable network, these errors will occur extremely rarely. So rarely, in fact, that they'll be written off as "user error" by support. If you're a startup building a system which has to quickly and reliably scale from 3 > 3000 nodes in a year then the whole thing is likely to explode in your face. Twitter style. Now, if MongoDB was so superior that it was trul…

Fantastic market strategy, but it's still snake oil they're selling.

When you talk about growing, the biggest value in Open Source has been that you can start with something free but shit, and then as you make money then you can spend it on customizing that Open Source in a way that benefits you.

However there exist commercial offerings that are (and were) faster and better at MongoDB than MongoDB was: KDB could've handled Twitter, we never would've seen a fail whale, and it is a whole hell of a lot cheaper than the developers and the customizers, and the headache, and the fact that you're making something open source which ultimately benefits your competition.

Another way to think about it is by thinking about experts: If you've got a great startup idea, why would you want to make your odds 10% worse by introducing the possibility it'll fail, by using the cheapest hacky hack thing that has 10% chance of losing your data? Ask experts with data, and be honest with your budget and you'll do a lot better.

Re: MongoDB 3.4.0-rc3

#107

Post by throwawayish »

mushi wrote: post #26
koolba wrote: post #19

Earlier quoted context omitted.

> I will never rely on my database, regardless of which one I am using for complete data consistency. I can't imagine developing any software that involves relationships between entities that does not have data consistency. Check constraints, foreign keys, and data type validation all provide a minimum sanity level of the underlying data that allows your mind to focus on more important things. Otherwise you're entire…

I think maybe the doubt is based on case like https://aphyr.com/posts/282-jepsen-postgres which all systems are subject to.

Uh, no. That article is a bit of a long winded way to say that 2PC with timeouts is 2PC with timeouts.

Re: MongoDB 3.4.0-rc3

#108

Post by geocar »

lotyrin wrote: post #85
hendzen wrote: post #65

Earlier quoted context omitted.

That specific point was in reply to the GP's statement that "competent teams" weren't using it in production.

I don't think it should be taken as given that there's a correlation between competency and the size of an organization that a team exists within, and I don't think such a correlation, when combined with large organizations' usage of MongoDB would challenge the assertion that there exists an anticorrelation between team competency and use of MongoDB. Looking at the numbers, larger organizations straight-forwardly see…

I think you're looking at it wrong. It's not a popularity contest; I've seen billion dollar companies use fucking stupid tooling as well, but they still have the right processes where they don't lose data.

Mongo, on the other hand, loses data.

Re: MongoDB 3.4.0-rc3

#109

Post by omginternets »

omginternets wrote: post #44
jdcarter wrote: post #17

It's been a long way from the "Call Me Maybe: MongoDB" post from years back. Aphyr/Kyle took them to task in so many ways for playing fast and loose with data integrity, and rightly so. MongoDB could have said, "that guy's full of BS, ignore him," but instead they did the smart thing and paid Kyle to help solve the problem. n.b. I can't find the original "Call Me Maybe" post, but this later one [1] is similar. [1]: h…

>paid Kyle to help solve the problem. Any more information on this? How did he "solve" the problem? Edit: straight from the horse's mouth [0] [0] https://news.ycombinator.com/user?id=aphyr

Whoops, wrong link. Here's the correct one: https://news.ycombinator.com/item?id=13591048

It seems the problems were indeed "solved" and not solved.

Re: MongoDB 3.4.0-rc3

#110

Post by bogomipz »

Fiahil wrote: post #96
edgan wrote: post #84

Earlier quoted context omitted.

Actually, having been through their training, and dealt with their consultants. Their company culture is the problem. They had a pure-developer centric mindset, and no operations mindset. So at the time they had no good way to do backups. Their more modern solution is backups that they manage for you, which is even more crazy.

Yes. Mongo is an operational nightmare. I mean, who in their right mind would use a product where the recommended solution[0] to resync a stale replica (which happen all the time after a long netsplit) is to "perform an initial sync". Which, of course, mean "please remove everything and type this command". Crazy. [0]: https://docs.mongodb.com/manual/tutorial/resync-replica-set-...

Indeed, and the only practical means of performing a compaction is to "rm -rf" the data directory and let it resync from another replica set member. This is not documented of course.
Post reply on HN