Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jepsen Disputes MongoDB's Data Consistency Claims

infoq.com

Post reply on HN
Print view View original

291–300 of 416 posts

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#291

Post by akamaozu »

ianamartin wrote: post #212
erulabs wrote: post #186

I wonder if I'm the only sysadmin in the world who doesn't hate MongoDB. Yes, I wouldn't use it for new projects, and yes, I wish RethinkDB had taken its place, but it's not as horrible as people seem to think. Default configuration... If it weren't for RDS' doing PG-bouncer-style connection management, 95% of production postgres instances would probably fail. It innodb_buffer_pool_size wasn't set properly, plenty of…

RethinkDB is a better solution to every problem that MongoDB claims to solve. I wouldn't use it for everything. But once my need for a document store outgrows what's convenient and easy in Postgres with JSONB, I reach for Rethink. It's great. There's a Jepsen analysis of it a while back too that is quite positive. It's a shame that Rethink did so many things right and failed as a company while Mongo continues to do a…

My current data solution is layers of code on top of redis, trying really hard to be everything Rethink was.

Easy access to changelogs should be an "easy to access" feature in all databases. Event driven systems aren't rare: the data store needs to be done to tell interested parties that underlying data has changed.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#292

Post by stickupkid »

risyachka wrote: post #260
phyzome wrote: post #180

Earlier quoted context omitted.

It's great for small, read-only websites that aren't quite simple enough to just be static HTML sites. Also for desktop app config/data DBs.

Can you elaborate? I've seen benchmarks and from their website what I understood is that it can handle really massive reads and writes, tens (maybe hundreds) thousands of ops per second, but personally never tested to this extent.

It really can (LXD cloud setup from personal experience), the problem is that if you don't serialise your writes then yeah, fun times to be had. There are compromises for all databases. People just like telling others their opinion as fact, and how wrong everybody is apart from themselves of course.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#293

Post by nieve »

__Joker wrote: post #282
nieve wrote: post #209

Earlier quoted context omitted.

If memory serves the original EToys.com code treated the filesystem as tree-structured database using atomic operations (though no transactions). It worked just fine, then the rewrite with an RDBMS that should have been stabler and faster resulted in the famous meltdowns. Admittedly this is cheating a bit since you can name folders & files with semi-arbitrary or internally structured string keys. By 1997 standards pu…

Interesting, is there a blog around discussing this in detail ? If not would be kind enough to go more into detail.

I don't think anyone posted about their particular system, but it's not unknown now. If you google "filesystem as a database" there are some relevant hits. One super simple and probably not ideal, but at least balanced version uses a hash of some primary key like customer row id as the file index, then partitions the items into directories with all permutations at each level (or only populated ones) based on successive parts of the hash. For example an item key that hashes to a32c4214585e9cb7a55474133a5fc986 would be located somewhere like this:

  a32c/4214/585e/9cb7/a554/74133a5fc986
    a32c/
      4214/
        585e/
          9cb7/
            a554/
              74133/a5fc986
The advantage of this kind of structure is that you never need to manually scan a directory since you know exactly what path you're trying to open. You still incur the OS lookup time for the inode-equivalent in the directory entry, but a deeper heirarchy keeps that faster. You can trade off time to traverse the heirarchy versus number of entries in the final directories by adjusting the length of the hash chunk you use at each level. Two characters will put vastly fewer entries at a given level, but vastly increase your directory depth.

Basically if you're manually scanning the heirarchy for anything but a consistency check or garbage collection you've already lost.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#294

Post by pastacacioepepe »

thaumasiotes wrote: post #290
jen20 wrote: post #177

Earlier quoted context omitted.

Firstly let me point out that this response is neither intended as a defence of MongoDB defaults which are atrocious, or of the company, who are arguably duplicitous. However I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”.

> I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”. This kind of thing is a scourge. I had a Chinese friend respond to something I said once by saying "that's nice". It looks so innocent... but it's really hard to overlook the fact that "that's nice" is a serious insult coming from a native speaker. I had to ask them to please…

Did the joke just went over my head or "that's nice" is a bad thing for real?

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#295

Post by nieve »

nieve wrote: post #293
__Joker wrote: post #282

Earlier quoted context omitted.

Interesting, is there a blog around discussing this in detail ? If not would be kind enough to go more into detail.

I don't think anyone posted about their particular system, but it's not unknown now. If you google "filesystem as a database" there are some relevant hits. One super simple and probably not ideal, but at least balanced version uses a hash of some primary key like customer row id as the file index, then partitions the items into directories with all permutations at each level (or only populated ones) based on successi…

One important note: make sure you carefully consider using atomic renames and such for manipulating the files! Overwrite in place is a great way to end up with a corrupted item if something goes desperately wrong and you're not protected by COW or data journaling.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#296

Post by kuschku »

risyachka wrote: post #260
phyzome wrote: post #180

Earlier quoted context omitted.

It's great for small, read-only websites that aren't quite simple enough to just be static HTML sites. Also for desktop app config/data DBs.

Can you elaborate? I've seen benchmarks and from their website what I understood is that it can handle really massive reads and writes, tens (maybe hundreds) thousands of ops per second, but personally never tested to this extent.

We're using it in Quassel, and as soon as you go over ~3-4 parallel write/read threads, it starts locking up completely, sometimes taking 30 seconds for simple queries that should really take milliseconds.

The big issue is that sqlite does full db locking for any operation, so during any write you can't just easily read at all.

This can be fixed with WAL mode, but WAL mode is broken in uts early versions, and new versions of sqlite aren't in all disteos yet, despite being out for almost a decade. And even WAL mode gets abysmal performance.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#297

Post by tasuki »

chx wrote: post #64
madhadron wrote: post #27

In the circles I run in, MongoDB is regarded as a joke and the company behind it as basically duplicitous. For example, they still list Facebook as their first user of MongoDB on their website, for example, but there is no MongoDB use in Facebook hasn't been for years (it came in only via a startup acquisition). I had the misfortune to use MongoDB at a previous job. The replication protocol wasn't atomic. You would f…

There was a time when I advocated for MongoDB with the usual caveats. The ability to easily store and index complex data was of great value. And then in 2015 October, within a week of each other, SQLite and MySQL both learned how to index on expressions and store JSON (SQLite 3.9 2015-10-14, MySQL 5.7 2015-10-21). PostgreSQL added jsonb the year prior in 9.4. At that moment the value of MongoDB for me diminished grea…

Why is storing json in a database important to you? Whenever I see json fields in PostgreSQL/MySQL, I know I'm most likely in for inconsistent data and a world of pain.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#298

Post by cosmodisk »

pastacacioepepe wrote: post #294
thaumasiotes wrote: post #290

Earlier quoted context omitted.

> I can _quite easily_ see how a non-native English speaker could use the phrase “if you know what you are doing” to mean “if you are careful”. This kind of thing is a scourge. I had a Chinese friend respond to something I said once by saying "that's nice". It looks so innocent... but it's really hard to overlook the fact that "that's nice" is a serious insult coming from a native speaker. I had to ask them to please…

Did the joke just went over my head or "that's nice" is a bad thing for real?

I think it sits on the same level as ' it's interesting',when 9/10 times you could see that the person who said so is not even remotely interested.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#299

Post by Xylakant »

neeleshs wrote: post #276
vosper wrote: post #262

Earlier quoted context omitted.

Which is funny because one of the things hosted Mongo will do is constantly send you automated emails suggesting that you should create indexes (for tables that MySQL or Postgres would laugh at)

Sends emails when scanned vs returned record ratio is greater than a threshold. Not quite sure how something like this would be different for MySQL/postgres. If an index is missing for you query pattern, wouldn't you create an index in MySQL/postgres?

> If an index is missing for you query pattern, wouldn't you create an index in MySQL/postgres?

No. I’d consider adding an index. An index is not free, it comes at a cost and that cost may well be higher than the costs of not having that index. For example, if a reporting query that runs once every few hours is lacking an index, the cost of updating that index on every write (and the disk space/memory used) may well exceed its benefits.

Re: Jepsen Disputes MongoDB's Data Consistency Claims

#300

Post by Too »

neeleshs wrote: post #276
vosper wrote: post #262

Earlier quoted context omitted.

Which is funny because one of the things hosted Mongo will do is constantly send you automated emails suggesting that you should create indexes (for tables that MySQL or Postgres would laugh at)

Sends emails when scanned vs returned record ratio is greater than a threshold. Not quite sure how something like this would be different for MySQL/postgres. If an index is missing for you query pattern, wouldn't you create an index in MySQL/postgres?

MSSQL can automatically create indexes for frequently run queries, instead of sending emails about it.
Post reply on HN