Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB's Write Lock

blog.pythonisito.com

Post reply on HN
Print view View original

41–50 of 76 posts

Re: MongoDB's Write Lock

#41

Post by xxqs »

nknight wrote: post #22
xxqs wrote: post #17

Earlier quoted context omitted.

actually the number of ARM processors is growing, and not only in the mobile sector. There have been some efforts to bring ARM architecture into the server market. Also China is building its own MIPS-based supercomputer. Also the SPARC architecture is actually developing, although it's a pity to see it swallowed by Oracle. IBM is still shipping PowerPC servers. besides, there are huge SPARC-only datacenters still run…

> There have been some efforts to bring ARM architecture into the server market. None of which have really gone anywhere. > China is building its own MIPS-based supercomputer. You think a lot of supercomputers are going to be running Web-targeted NoSQL datastores? > Also the SPARC architecture is actually developing So its partisans have insisted for the past decade. The real world doesn't much care. Sun/Oracle was l…

> You think a lot of supercomputers are going to be running Web-targeted NoSQL datastores?

if they succeed in building a powerful architecture which beats Intel in performance per kilowatt, they will start selling commodity servers with it.

besides, come on, it's a database engine. What difference does it make if I have an enterprise or a web application?

Re: MongoDB's Write Lock

#42

Post by [deleted] »

DenisM wrote: post #36
milkshakes wrote: post #28

Earlier quoted context omitted.

In other words it will be surpassed by another system, aiming to solve the same set of problems, but designed with concurrency in mind from the start. MongoDB developers will end up spending all their efforts to retrofit concurrency, and the community of users will simply move on. Fine, I'll bite. Your assertion that MongoDB was not designed "with concurrency in mind" is simply wrong. They have in fact put a lot of t…

>that is hard and takes time. And I'm arguing that's going to be a hell of ride, which is more likely to break their backs than yield success. I read up on the link you posted elsewhere (thanks for that, by the way) and well, it's just as bad as I thought. Imagine, in pre-2.0 world, there's a greedy reader, and a subsequent writer queued up on the lock. All subsequent readers are blocked until writer quits, which can…

[deleted]

Re: MongoDB's Write Lock

#43

Post by latch »

JanezStupar wrote: post #33
ilaksh wrote: post #31

The thing is that you are still supposed to keep the whole working set in memory and use sharding if its larger than that. Which means that none of this is really relevant. Except that now with that new graph showing such good performance on reads during paging people are going to get confused. Anyway you can get 32GB of RAM for $232 or 48GB for $636. Which means that for 90% of applications, you actually don't need…

From multiple years of NoSQL experience. The Object/Relational impedance mismatch stays right there where you leave it. Using K/V stores will only help you write data. But the whole impedance nightmare is right there, waiting for you to try and make some sense of the data. Especially if you want to do relations. And you are dead wrong about relational DB's. It is either due to habit or because of being a better fit t…

> The Object/Relational impedance mismatch stays right there where you leave it.

Somewhat umbrella...but I do think that document-based storage does make a difference. It certainly doesn't erase it and, as you say, it varies based on the use-case at hand, but I think most people would consider the development experience to have less friction (and that's certainly been the overwhelming anecdotal evidence I've heard (and can give)).

Re: MongoDB's Write Lock

#44

Post by eternalban »

nknight wrote: post #25
DenisM wrote: post #21

Earlier quoted context omitted.

So one core is processing one request at a time, right? If it spends any time at all being queued up on disk IO or network traffic or anything else, that core is burning up XX watts for no good reason. A more efficient system, designed for concurrency, will have higher HW utilization and lower cost. For a sign of things to come, I invite you to take a look at how relational database vendors are fighting to squeeze si…

> queued up on disk IO And there's where your legacy understanding fails. You're really not supposed to be doing a lot of disk IO in modern datastores in the first place. Your working set should be in RAM. > beat benchmarks If my vendor is investing time in beating benchmarks instead of solving real problems, I'm finding a new vendor.

"solving real problems"

I am not at all addressing MongoDB here to be clear -- just your comment regarding worthwhile "problems" to "solve".

It is not too difficult to foresee a future where energy costs will trump all other considerations, including development time, sys ops, etc. Specially for aaS efforts, energy efficiency can clearly end up being a competitive edge.

Re: MongoDB's Write Lock

#45

Post by petrohi »

DenisM wrote: post #36
milkshakes wrote: post #28

Earlier quoted context omitted.

In other words it will be surpassed by another system, aiming to solve the same set of problems, but designed with concurrency in mind from the start. MongoDB developers will end up spending all their efforts to retrofit concurrency, and the community of users will simply move on. Fine, I'll bite. Your assertion that MongoDB was not designed "with concurrency in mind" is simply wrong. They have in fact put a lot of t…

>that is hard and takes time. And I'm arguing that's going to be a hell of ride, which is more likely to break their backs than yield success. I read up on the link you posted elsewhere (thanks for that, by the way) and well, it's just as bad as I thought. Imagine, in pre-2.0 world, there's a greedy reader, and a subsequent writer queued up on the lock. All subsequent readers are blocked until writer quits, which can…

In Microsoft SQL Server row/extent/table locks are of transactional semantics and often turned off with nolock option. What really matters for concurrency is page latch, which is per 8K page.

In SQLite readers actually do block writers by default. Writing transactions are committed with lock escalation steps. First shared lock is acquired, then reserved, then pending and finally exclusive. Pending blocks new shared locks and waits until all in-flight shared locks are released. Again this is default rollback journal behavior. As of 3.7 write-ahead log allows readers to be concurrent with writers, but AFAIK is still rarely used.

Re: MongoDB's Write Lock

#46

Post by steve8918 »

Does anyone know what the performance differences would be between MongoDB and SQL Server/Oracle if they all had enough RAM to hold the entire dataset in memory?

I'm only guessing but it seems to me that any database with their entire dataset in memory would be very fast, no?

Re: MongoDB's Write Lock

#47

Post by prodigal_erik »

latch wrote: post #43
JanezStupar wrote: post #33

Earlier quoted context omitted.

From multiple years of NoSQL experience. The Object/Relational impedance mismatch stays right there where you leave it. Using K/V stores will only help you write data. But the whole impedance nightmare is right there, waiting for you to try and make some sense of the data. Especially if you want to do relations. And you are dead wrong about relational DB's. It is either due to habit or because of being a better fit t…

> The Object/Relational impedance mismatch stays right there where you leave it. Somewhat umbrella...but I do think that document-based storage does make a difference. It certainly doesn't erase it and, as you say, it varies based on the use-case at hand, but I think most people would consider the development experience to have less friction (and that's certainly been the overwhelming anecdotal evidence I've heard (a…

I have to wonder whether that's because many devs started using schemaless databases for the first time over the last couple of years, and haven't yet really experienced the nightmare of data which was scribbled on by various forgotten buggy versions of the apps and never rigorously migrated (because they're self-selected to regard that as unimportant). I once worked on a Lotus Notes-based system with documents eventually reaching such nonsensical states that the dev team couldn't even say what app behavior would be appropriate, much less what the latest version of all our code would happen to do.

Re: MongoDB's Write Lock

#48

Post by peschkaj »

steve8918 wrote: post #46

Does anyone know what the performance differences would be between MongoDB and SQL Server/Oracle if they all had enough RAM to hold the entire dataset in memory? I'm only guessing but it seems to me that any database with their entire dataset in memory would be very fast, no?

You are correct - any database with the entire dataset in memory will be incredibly fast. SQL Server bypasses the Windows file system cache and will aggressively manage memory to keep frequently accessed data pages resident in RAM. The read/write performance is what you would expect for a database with fine-graned lock management - when you have to go to disk things get slower, otherwise I/O is only limited by RAM and the overhead of lock management.

Re: MongoDB's Write Lock

#49

Post by peschkaj »

fleitz wrote: post #6

Why not just go take the write lock out of the db all together if it doesn't need to be journalled? It's obvious at that point that missing / mangled data is acceptable. It's not particularly amazing that not writing data to disk is faster than writing to disk. What IS amazing is that the geniuses at 10gen have some how managed to make not writing to disk thousands of times slower than writing to disk. Who would desi…

MongoDB's journal is another collection that only syncs to disk 10 times a second. It's not a true journal like a write-ahead log. You can force that collection to fsync, but... yeah, I get get stupendous performance using a cheap RAID controller on commodity hardware with an old and busted relational database. Hooray for 40 years of technology!

Re: MongoDB's Write Lock

#50

Post by jhancock »

DenisM wrote: post #36
milkshakes wrote: post #28

Earlier quoted context omitted.

In other words it will be surpassed by another system, aiming to solve the same set of problems, but designed with concurrency in mind from the start. MongoDB developers will end up spending all their efforts to retrofit concurrency, and the community of users will simply move on. Fine, I'll bite. Your assertion that MongoDB was not designed "with concurrency in mind" is simply wrong. They have in fact put a lot of t…

>that is hard and takes time. And I'm arguing that's going to be a hell of ride, which is more likely to break their backs than yield success. I read up on the link you posted elsewhere (thanks for that, by the way) and well, it's just as bad as I thought. Imagine, in pre-2.0 world, there's a greedy reader, and a subsequent writer queued up on the lock. All subsequent readers are blocked until writer quits, which can…

My position on mongo is thus: Its goal is humongous data sets, hence the name. Until well proven, I'm not the type to use it for huge data, but will keep an eye out for case studies.

I have used Mongo on two projects with reasonably small data sets. My largest collection at the moment is 5 million, and that's basically a log. Other collections are less than 100,000. I've been running mongo 1.6 for a year on these two sites without so much as a hiccup. I do the normal very simple things to protect myself: a cron job to dump db and then copy it to a backup server. And that's it.

I enjoy using mongo for these projects because when I want to add a new feature to one of my domain models, I don't need to think much about retrofitting the data for all instances of that model. I just add an attribute where its needed for the new use case, ensure I have basic checking in my ruby model object and my system keeps incrementally improving.

I think the mongo folks are fantastic in their open dev process and maybe one day, some threshold will be crossed where I can say that for certain types of big data usage mongo is a clear solid choice.

Post reply on HN