MongoDB 3.4.0-rc3 • Hacker News Board

Re: MongoDB 3.4.0-rc3

#151

Post by geocar » Wed, Feb 08, 2017, 3:09 PM UTC

hendzen wrote: post #121

geocar wrote: post #106

Earlier quoted context omitted.

Fantastic market strategy, but it's still snake oil they're selling. When you talk about growing, the biggest value in Open Source has been that you can start with something free but shit, and then as you make money then you can spend it on customizing that Open Source in a way that benefits you. However there exist commercial offerings that are (and were) faster and better at MongoDB than MongoDB was: KDB could've h…

I have some actual experience with KDB and MongoDB so I'm going to have to call bullshit. How does KDB handle replication and failover? Or even high insert/update rates to datasets that exceed the size of memory? How do you shard KDB? KDB doesn't support unicode text. Do you plan to only have English speaking users? Yes, KDB excels at its relatively well defined niche of transforming and aggregating "smallish" (say 1…

> How does KDB handle replication and failover?

With -r and (in my case) SO_REUSEPORT. Most people use a dedicated gateway (have seen custom tomcat stuff and haproxy).

Meanwhile, MongoDB doesn't actually replicate reliably (acking then losing anyway) and failover can crash cascade in the naïve configuration.

> Or even high insert/update rates to datasets that exceed the size of memory?

This is literally the KDB tickerplant model. Have an RDB that flushes out regularly (daily) to an HDB.

You can also just write to a log `:log upsert ...

> How do you shard KDB?

Same way you shard anything else? By picking a key and directing the query to the appropriate server. h[(first md5 k) mod count h] "query..."

> KDB doesn't support unicode text.

UTF8 is fine.

The number of times I've needed the first 5 code points (and not the first 5 bytes or the first 5 characters) in my life is zero. All that half-baked Unicode support in various languages (like MongoDB) just makes people think that they've solved a problem that they really haven't.

> Yes, KDB excels at its relatively well defined niche of transforming and aggregating "smallish" (say 10 TB or less) numerical time series data. It would be a horrible choice for the backing store of a high throughput CRUD application...

I use it in one of those big CRUD databases (digital marketing and tele-lead tracking).

> What is it with KDB zealots thinking that KDB is the best database for every task? I swear, KDB is the Scientology of databases.

Because it solves problems they have.

Even when I don't use KDB I use a similar architecture because it's the correct architecture, because I've had these problems for a lot longer than I've had KDB.

If it doesn't solve every problem I have, that's because I have work to do, not because it isn't great at the problems it does solve, and I don't shout at my hammer because it isn't a spoon.

However MongoDB doesn't solve any problem I've ever had: I've never needed a bag of objects/filesytem that loses data, or a binary blob that I cannot query. It's so famously "web scalable" it has made a joke of the very idea of being scalable.

Re: MongoDB 3.4.0-rc3

#152

Post by atnnn » Wed, Feb 08, 2017, 4:18 PM UTC

Kiro wrote: post #57

rocky1138 wrote: post #40

Why would anyone use MongoDB when RethinkDB is available?

Is there anything like mlab.com for RethinkDB? I don't want to manage my own server.

There is https://www.compose.com/rethinkdb

Re: MongoDB 3.4.0-rc3

#153

Post by aphyr » Wed, Feb 08, 2017, 5:46 PM UTC

pkolaczk wrote: post #143

aphyr wrote: post #119

Earlier quoted context omitted.

Pretty sure Cassandra still uses local timestamps and last-write-wins, which means you can't do safe updates. Also pretty sure they didn't fix the timestamp-collision row-isolation issue I talked about, because it was on HN's front page a few months ago. ;-)

Using timestamps and last write wins is a well known and well documented behavior, so you must assume updates can be sometimes applied in a different order than they were submitted. However, reordering updates is not the same as rolling back writes as was in case of Mongo. Cassandra does not promise linearizability in this mode of operation (if not using LWTs), and this is a tradeoff to get better availability. There…

> assume updates can be sometimes applied in a different order than they were submitted

I suggest you re-read the analysis. Some databases can offer safe, generalized commutative updates; e.g. Riak. Cassandra can't: updates, in general, can be lost through reordering.

> There is no better way than last write wins if you want high availability and partition tolerance and don't want to pay the performance and availability price for a consensus algorithm like Paxos or Raft.

There is. There's a whole field of research devoted to this problem. http://hal.upmc.fr/inria-00555588/document

Re: MongoDB 3.4.0-rc3

#154

Post by aphyr » Wed, Feb 08, 2017, 5:52 PM UTC

ClayFerguson wrote: post #128

aphyr wrote: post #102

Earlier quoted context omitted.

What does MongoDB bring that couldn't have 'built on top' of MySQL codebase, and used MySQL transational layer as it's underpinnings. There are two challenges here. One is that MongoDB has a different data model than MySQL: hierarchical, schema-less documents instead of uniformly-typed flat tables. It's possible to map one to the other--look at Postgres' support for JSON datatypes. That involves extensions to both th…

The point is that MongoDB writes/reads directly to File system. It could have been (and still could be retro-fitted) to ride on top of an RDBMS for all it's actual IO, and create a hybrid. I'm aware of all the orthogonal points you raised which have nothing to do with ruling out RDBMS as I/O

http://www.slideshare.net/NorbertoLeite/mongodb-wiredtiger-i...

Re: MongoDB 3.4.0-rc3

#155

Post by nurettin » Wed, Feb 08, 2017, 6:04 PM UTC

disgusting_guy wrote: post #147

nurettin wrote: post #144

Earlier quoted context omitted.

Well, thanks for the question! You check the result with getLastError which, as you described, can be used to ensure a majority agrees with the write. But you normally don't use getLastError that way. Because a majority might not even be concerned with that particular write. They are, after all, shards. Instead you check if primary got the write. If primary disconnects while you are checking, you catch the exception…

You describe it like it is simple but that is ridiculous number of steps to simply check your data was actually written to the database. >that's as reliable as it gets when inserting into any database including SQL The difference being in a SQL database you call commit and all this happens for you automatically

>>You describe it like it is simple

ah, no. I did not.

Re: MongoDB 3.4.0-rc3

#156

Post by takeda » Wed, Feb 08, 2017, 9:22 PM UTC

hendzen wrote: post #88

takeda wrote: post #87

Earlier quoted context omitted.

In many ways MySQL is similar to MongoDB. Both started out being written by people who know nothing about databases and both threw away years of database research. Both gained popularity due to being accepted choice by web-based languages (PHP vs NodeJS) Both were faster than more established competition, only to turn out that both were losing data. Both turned out to be designed fundamentally wrong and had a replace…

You're comparing ISAM/MySAM (storage engine) to the MongoDB replication protocol. As a more relevant parallel MongoDB also replaced its original storage engine with one acquired from WiredTiger (BerkeleyDB founders). One big difference from a corporate strategy perspective is that MySQL let the replacement storage engine (InnoDB) fall in to the hands of Oracle. MongoDB was smart enough to make sure that they were the…

> You're comparing ISAM/MySAM (storage engine) to the MongoDB replication protocol. As a more relevant parallel MongoDB also replaced its original storage engine with one acquired from WiredTiger (BerkeleyDB founders).

My bad, MMAPv1 vs WiredTiger although I think it was obvious what I meant.

> One big difference from a corporate strategy perspective is that MySQL let the replacement storage engine (InnoDB) fall in to the hands of Oracle. MongoDB was smart enough to make sure that they were the acquirer, which puts them in control of their own destiny.

Not sure if that's relevant though, since whole MySQL became property of Oracle (after they acquired Sun).

Re: MongoDB 3.4.0-rc3

#157

Post by pkolaczk » Wed, Feb 08, 2017, 10:32 PM UTC

aphyr wrote: post #153

pkolaczk wrote: post #143

Earlier quoted context omitted.

Using timestamps and last write wins is a well known and well documented behavior, so you must assume updates can be sometimes applied in a different order than they were submitted. However, reordering updates is not the same as rolling back writes as was in case of Mongo. Cassandra does not promise linearizability in this mode of operation (if not using LWTs), and this is a tradeoff to get better availability. There…

> assume updates can be sometimes applied in a different order than they were submitted I suggest you re-read the analysis. Some databases can offer safe, generalized commutative updates; e.g. Riak. Cassandra can't: updates, in general, can be lost through reordering. > There is no better way than last write wins if you want high availability and partition tolerance and don't want to pay the performance and availabil…

You can have safe updates through clustering columns or if you really insist on destructive updates - through LWTs. With clustering columns you can easily achieve whatever is possible with vector clocks.

http://www.datastax.com/dev/blog/why-cassandra-doesnt-need-v...

As for the research you posted, there is no free lunch. Each of these strategies come with their own set of drawbacks. That's why Cassandra offers choice at a query level.

Re: MongoDB 3.4.0-rc3

#158

Post by hendzen » Thu, Feb 09, 2017, 12:04 AM UTC

geocar wrote: post #151

hendzen wrote: post #121

Earlier quoted context omitted.

I have some actual experience with KDB and MongoDB so I'm going to have to call bullshit. How does KDB handle replication and failover? Or even high insert/update rates to datasets that exceed the size of memory? How do you shard KDB? KDB doesn't support unicode text. Do you plan to only have English speaking users? Yes, KDB excels at its relatively well defined niche of transforming and aggregating "smallish" (say 1…

> How does KDB handle replication and failover? With -r and (in my case) SO_REUSEPORT. Most people use a dedicated gateway (have seen custom tomcat stuff and haproxy). Meanwhile, MongoDB doesn't actually replicate reliably (acking then losing anyway) and failover can crash cascade in the naïve configuration. > Or even high insert/update rates to datasets that exceed the size of memory? This is literally the KDB ticke…

> With -r and (in my case) SO_REUSEPORT. Most people use a dedicated gateway (have seen custom tomcat stuff and haproxy).

So as a KDB user you need to implement your own HA solution. That is strictly worse than MongoDB replication, even with its now-fixed bugs. Do you really think your homemade multi-master KDB system would pass Jepsen?

> This is literally the KDB tickerplant model. Have an RDB that flushes out regularly (daily) to an HDB.

Wat? That only works if data is immutable once written. Tweets are liked/deleted/etc. You could store an immutable log of user actions, but then you would have to reconstruct the current snapshot every time someone loads a timeline. It's entirely possible for someone to like/delete/RT an old tweet. Financial data is naturally partitioned because the order book clears at the end of every trading day - this doesn't apply to CRUD apps.

> UTF8 is fine

I think you misunderstand what I mean by unicode support. Does KDB support locale specific collations? Does it support normalization/canonicalization? Being able to index by code point is about 1% of the needed solution to build an i18n-proof product. Obviously that doesn't matter when you are dealing with normal KDB datasets like market data where e.g. asian names are represented with numbers.

> I use it in one of those big CRUD databases (digital marketing and tele-lead tracking).

Were you using it to store clickstream data? Or some other kind of immutable stream of events? That isn't really applicable to general CRUD applications.

Like I said - KDB is great for analyzing immutable streams of events. It's not a general purpose database for building CRUD applications. MongoDB tries to be a reasonable enough solution for many use cases, while KDB focuses on excelling at a small number. Both are valid approaches to building a database...

Re: MongoDB 3.4.0-rc3

#159

Post by hendzen » Thu, Feb 09, 2017, 12:07 AM UTC

gd1 wrote: post #138

hendzen wrote: post #121

Earlier quoted context omitted.

I have some actual experience with KDB and MongoDB so I'm going to have to call bullshit. How does KDB handle replication and failover? Or even high insert/update rates to datasets that exceed the size of memory? How do you shard KDB? KDB doesn't support unicode text. Do you plan to only have English speaking users? Yes, KDB excels at its relatively well defined niche of transforming and aggregating "smallish" (say 1…

>KDB doesn't support unicode text. Unicode (from 2011): http://code.kx.com/wiki/Cookbook/Unicode

I replied to a sibling with more details. Indexing by code point is only the smallest (and easiest to solve) part of the problem of dealing with non ASCII text.

Re: MongoDB 3.4.0-rc3

#160

Post by hendzen » Thu, Feb 09, 2017, 12:19 AM UTC

takeda wrote: post #156

hendzen wrote: post #88

Earlier quoted context omitted.

You're comparing ISAM/MySAM (storage engine) to the MongoDB replication protocol. As a more relevant parallel MongoDB also replaced its original storage engine with one acquired from WiredTiger (BerkeleyDB founders). One big difference from a corporate strategy perspective is that MySQL let the replacement storage engine (InnoDB) fall in to the hands of Oracle. MongoDB was smart enough to make sure that they were the…

> You're comparing ISAM/MySAM (storage engine) to the MongoDB replication protocol. As a more relevant parallel MongoDB also replaced its original storage engine with one acquired from WiredTiger (BerkeleyDB founders). My bad, MMAPv1 vs WiredTiger although I think it was obvious what I meant. > One big difference from a corporate strategy perspective is that MySQL let the replacement storage engine (InnoDB) fall in t…

Yes... its very relevant... If MySQL had acquired InnoDB instead of Oracle there is a good chance they would still exist as an independent entity. That's what I meant by MongoDB's acquisition of WT putting them in control of their own destiny.