MongoDB's Write Lock • Hacker News Board

Re: MongoDB's Write Lock

#71

Post by petrohi » Sun, Jan 01, 2012, 7:43 PM UTC

MichaelGG wrote: post #69

petrohi wrote: post #45

Earlier quoted context omitted.

In Microsoft SQL Server row/extent/table locks are of transactional semantics and often turned off with nolock option. What really matters for concurrency is page latch, which is per 8K page. In SQLite readers actually do block writers by default. Writing transactions are committed with lock escalation steps. First shared lock is acquired, then reserved, then pending and finally exclusive. Pending blocks new shared l…

I recently found out that turning on nolock is a horrible idea. It doesn't just take you out of transactions. That is, you won't just get uncommitted data, but apparently can get completely inconsistent data as internal structures are updated. That is, even rows not part of a current transaction might not be seen, if you use nolock. Edit: Turning ON nolock, that is.

My experience with nolock is that you may get inconsistent rows with some fields before and some after update. Or even seeing duplicated rows when b-tree is rebalanced. But I never seen single fields being partially updated. Per Microsoft, page latch protects atomicity of a single field update. This is why nolock was extremely useful for insert-only tables and in our database design we had many of them.

Re: MongoDB's Write Lock

#72

Post by orthecreedence » Sun, Jan 01, 2012, 11:14 PM UTC

mnutt wrote: post #57

orthecreedence wrote: post #51

> If you are able to do this, it turns out that the global write lock really doesn't affect you. Blocking reads for a few nanoseconds while a write completes turns out to be a non-issue. (I have not measured this, but I suspect that the acquisition of the global write lock takes significantly longer than the actual write.) Actually, it does affect you. I have worked with mongodb in production in a high-write scenario…

Were your writes changing the size of the documents so that mongo had to move them? I've had this happen and it'll cause mongo to grind to a halt.

Some of them were, but others were just updating a boolean or an integer value. For the most part we tried to pad our records, but I'm sure there was some moving along the way.

Re: MongoDB's Write Lock

#73

Post by ehthere » Sun, Jan 01, 2012, 11:52 PM UTC

philjackson wrote: post #63

ehthere wrote: post #54

Earlier quoted context omitted.

'I don't need to think much about retrofitting the data for all instances of that model. I just add an attribute where its needed for the new use case, ensure I have basic checking in my ruby model object and my system keeps incrementally improving.' That's exactly the same as adding a new column to your DB with NULL as the default value.

Not really. Mongo has the notion of undefined and Null. You can just start putting the new field on new records without having to backfill. Also, you don't have to do the migration thing, which can get messy in big teams (from my experience). Moving to a doc store from an RDMS really does bring with it an odd sense of freedom when it comes to the schema.

You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value? I don't see what you gain by having both 'undefined' and 'null'.

The 'odd sense of freedom' is not always a good thing either. It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea.

Re: MongoDB's Write Lock

#74

Post by philjackson » Mon, Jan 02, 2012, 12:51 AM UTC

ehthere wrote: post #73

philjackson wrote: post #63

Earlier quoted context omitted.

Not really. Mongo has the notion of undefined and Null. You can just start putting the new field on new records without having to backfill. Also, you don't have to do the migration thing, which can get messy in big teams (from my experience). Moving to a doc store from an RDMS really does bring with it an odd sense of freedom when it comes to the schema.

You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value? I don't see what you gain by having both 'undefined' and 'null'. The 'odd sense of freedom' is not always a good thing either. It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea.

"You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value?"

Taking the team I worked with at the BBC as an example:

1) There were staging, integration and production environments. Staging and integration would often not be aligned with production, or even one another, because we might find a bit of code turned out not to be production suitable/needed. If this happened we would have to drop the database back to a known, good state. You can't have columns with constraints left around when the code which might have satisfied those constraints is reverted. Doing it without migrations would have been idiotic to say the least.

2) Developers work on different features in different branches, often collaborating. Different features apply new attributes to the db schema. It's important for a developer to know his DB is in the correct state when he starts hacking. You do that with migrations.

Because you almost completely remove the need for schema definition (and what little of it you do, you can do in app. code) you simply don't need the migrations any more. Using mongo means you can pretty much just export your applications domain without having to coerce it into the relational model.

"I don't see what you gain by having both 'undefined' and 'null'."

They mean totally different things. Undefined means that the field has never been explicitly set, null means the field has been set. This means you know what's been backfilled and what hasn't - you can't tell without extra metadata in mysql. Also, in mysql if you provide a null default then every row has to be updated.

"It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea."

I don't know BASIC but you can put Perl into a certain configuration that allows this. That makes for horrible scoping issues that aren't analogous or applicable to what we're talking about.

Re: MongoDB's Write Lock

#75

Post by rick446 » Mon, Jan 02, 2012, 5:22 PM UTC

orthecreedence wrote: post #72

mnutt wrote: post #57

Earlier quoted context omitted.

Were your writes changing the size of the documents so that mongo had to move them? I've had this happen and it'll cause mongo to grind to a halt.

Some of them were, but others were just updating a boolean or an integer value. For the most part we tried to pad our records, but I'm sure there was some moving along the way.

I guess it depends on what % of your writes were simply updating a boolean or integer value. My benchmark shows that simple updates like that don't affect query performance much. Writes that take longer probably have different performance characteristics, YMMV, etc.

Re: MongoDB's Write Lock

#76

Post by ehthere » Tue, Jan 03, 2012, 3:29 AM UTC

philjackson wrote: post #74

ehthere wrote: post #73

Earlier quoted context omitted.

You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value? I don't see what you gain by having both 'undefined' and 'null'. The 'odd sense of freedom' is not always a good thing either. It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea.

"You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value?" Taking the team I worked with at the BBC as an example: 1) There were staging, integration and production environments. Staging and integration would often not be aligned with production, or even one another, because we might find a bit of code turned out not to be production suitable/needed. If this h…

If you added a column and you want to revert back, you just drop the column again! What's so hard about that? No 'migration' needed.

In BASIC you can 'declare' a variable by simply using it. The compiler will not warn you if you use an undeclared variable. That's the analogous situation here.