Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Year with MongoDB

blog.engineering.kiip.me

Post reply on HN
Print view View original

111–120 of 153 posts

Re: A Year with MongoDB

#111

Post by sausagefeet »

raverbashing wrote: post #109
birken wrote: post #94

Earlier quoted context omitted.

If you add a default value it locks the table and re-writes it. However, if you don't add a default value postgres will perform the operation instantly. Old rows will be untouched, and new values will be added with the shoe_size of either NULL or whatever you set it to be... IE exactly the same outcome and performance as the MongoDB case mentioned above. Adding a field in postgres while setting a default value would…

> Now I'm not anti-MongoDB, I'm just saying you shouldn't give it credit for something that relational database do just fine. No, it can't. Sorry, it absolutely can't. > If you add a default value it locks the table and re-writes it. > However, if you don't add a default value postgres will perform the operation instantly. And in MongoDB you don't have to change anything in the DB (hence, no downtime). It's all in yo…

> And that's why Facebook and others don't user relational anymore. Sure, you can use MySQL or PG, but the real data is stored in a text or binary blob

Is that why? I was under the impression that Facebook doesn't use the relational properties of MySQL because they couldn't get the performance out of it because of their scale, a problem a lot of advocates of the non-relational model dont' seem to have in my experience.

Re: A Year with MongoDB

#112

Post by sausagefeet »

pbiggar wrote: post #99
birken wrote: post #94

Earlier quoted context omitted.

If you add a default value it locks the table and re-writes it. However, if you don't add a default value postgres will perform the operation instantly. Old rows will be untouched, and new values will be added with the shoe_size of either NULL or whatever you set it to be... IE exactly the same outcome and performance as the MongoDB case mentioned above. Adding a field in postgres while setting a default value would…

> Adding a field in postgres while setting a default value would be exactly the same as adding a field Not true. Postgres locks the entire table for the duration of the update (which could be hours or days for a large enough dataset). Mongo will lock your entire databased for tiny fractions of a second, lots and lots of times. The difference is huge. The postgres implementation takes down your site, the Mongo impleme…

According to the Postgres documentation, the behavior the poster a few levels up defined is possible in Postgres with no downtime.

> When a column is added with ADD COLUMN, all existing rows in the table are initialized with the column's default value (NULL if no DEFAULT clause is specified).

> Adding a column with a non-null default or changing the type of an existing column will require the entire table to be rewritten. This might take a significant amount of time for a large table; and it will temporarily require double the disk space.

http://www.postgresql.org/docs/8.3/static/sql-altertable.htm...

Re: A Year with MongoDB

#113

Post by sausagefeet »

radicalbyte wrote: post #25
sirn wrote: post #13

> We changed the structure of our heaviest used models a couple times in the past year, and instead of going back and updating millions of old documents, we simply added a “version” field to the document and the application handled the logic of reading both the old and new version. This flexibility was useful for both application developers and operations engineers. Ugh, this sounds like a maintenance nightmare. How…

> oh, so you're implementing schema in your application? Isn't that where the schema belongs? Each document represents a conceptual whole. It doesn't contain fields which have to be NULL simply because they weren't in previous versions of the schema. I've been an rdbms guy (datawarehousing/ETL) for a long time now, I've seen a lot of large databases which have been in production for considerable time. They get messy.…

> They get messy. Really messy. They become basically unmaintainable. Apples, oranges and pears all squashed into a schema the shape of a banana.

Moving it to your application doesn't solve this problem, it just pushes it out of the DB.

Re: A Year with MongoDB

#114

Post by sausagefeet »

jasonmccay wrote: post #49
mitchellh wrote: post #47

Earlier quoted context omitted.

> Part of the lesson here is that if you're doing MongoDB on EC2, you should have more than enough RAM for your working set. We had more than enough RAM for our working set. Unfortunately, due to MongoDB's poor memory managed and non-counting B-trees, even our hot data would sometimes be purged out of memory for cold, unused data, causing serious performance degradation.

I understand your point, but the performance issues still stem off of poor IO performance on Amazon EBS. As we continue to use it, we continue to find it to be the source of most people's woes. If you have solid (even reasonable) IO, then moving things in and out of working memory is not painful. We have some customers on non-EBS spindles that have very large working sets (as compared to memory) ... faulting 400-500…

> but the performance issues still stem off of poor IO performance on Amazon EBS

But the point is it shouldn't need to do that I/O in the first place.

Re: A Year with MongoDB

#115

Post by jrussbowman »

mitchellh wrote: post #7
jrussbowman wrote: post #3

From the beginning I've understood mongodb to be built with it's approach for scaling, performance, redundancy and backup to be horizontal scaling. They recently added journaling for single server durability, but before that replication was how you made sure you data was safe. It seems to me when I see complaints about mongodb it's because people don't want to horizontally scale it and instead believe vertical scalin…

[Note: I wrote the blog post] I'm not at all against horizontally scaling. However, I don't believe that horizontally scaling should be necessary doing a mere 200 updates to per second to a data store that isn't even fsyncing writes to disk. Think of it in terms of cost per ops. Let's just say 200 update ops per second is the point at which you need to shard (not scientific, but let's just use that as a benchmark sin…

Interesting. Thanks for the reply and breaking it down the way you did. That provides some serious food for thought. Looking forward to your next post on the rationales for the other data stores.

Re: A Year with MongoDB

#116

Post by jrussbowman »

mitchellh wrote: post #7
jrussbowman wrote: post #3

From the beginning I've understood mongodb to be built with it's approach for scaling, performance, redundancy and backup to be horizontal scaling. They recently added journaling for single server durability, but before that replication was how you made sure you data was safe. It seems to me when I see complaints about mongodb it's because people don't want to horizontally scale it and instead believe vertical scalin…

[Note: I wrote the blog post] I'm not at all against horizontally scaling. However, I don't believe that horizontally scaling should be necessary doing a mere 200 updates to per second to a data store that isn't even fsyncing writes to disk. Think of it in terms of cost per ops. Let's just say 200 update ops per second is the point at which you need to shard (not scientific, but let's just use that as a benchmark sin…

Interesting. Thanks for the reply and breaking it down the way you did. That provides some serious food for thought. Looking forward to your next post on the rationales for the other data stores.

Re: A Year with MongoDB

#117

Post by charlieok »

rdtsc wrote: post #10

> Safe off by default I think that is fixed now. But this the single most appalling design decision they could have made while also claiming their product was a "database". (And this has been discussed here before so just do a search if you will). This wasn't a bug, it was a deliberate design decision. Ok, that that would have been alright if they put a bright red warning on their front page. "We disabled durability…

I think they just assumed that people would run the database in clusters, not single instances.

If you've done enough research to choose a relatively off-the-beaten-path DBMS such as MongoDB, the assumption is that you've carefully weighed the tradeoffs made by the various alternatives in the space, and learned the best practices for deploying the one you chose to use.

Re: A Year with MongoDB

#118

Post by [deleted] »

Philadelphia wrote: post #33

The impression I got after hearing some of the 10gen developers speak at a conference is that MongoDB has the same essential problem as PHP. It was written by people without a lot of formal knowledge who, for whatever reason, aren't interested in researching what's been tried before, what works, and what doesn't. Because of that, they're always trying to reinvent the wheel, and make flawed design decisions that keep…

[deleted]

Re: A Year with MongoDB

#119

Post by chad_walters »

taligent wrote: post #88
tolitius wrote: post #74

great post. direct and to the point, although there are many more flaws that I am sure you could have shared. we tried MongoDB to consume and analyze market feeds, and it failed miserably. I can add a couple of things to your list: * if there is a pending write due to an fsync lock, all reads are blocked: https://jira.mongodb.org/browse/SERVER-4243 * data loss + 10gen's white lies: https://jira.mongodb.org/browse/SER…

MongoDB is successful because of more than just marketing. It has great tool support, decent documentation, books and is accessible. Plus the whole transition from MySQL concept makes it easy to grab onto.

That all supports the marketing effort. Mongo is optimized for a good out-of-the-box experience for developers. It is basically the MySQL model -- hook developers first, fix the fundamentals later. Caveat emptor.

Re: A Year with MongoDB

#120

Post by fabioferrari »

sausagefeet wrote: post #113
radicalbyte wrote: post #25

Earlier quoted context omitted.

> oh, so you're implementing schema in your application? Isn't that where the schema belongs? Each document represents a conceptual whole. It doesn't contain fields which have to be NULL simply because they weren't in previous versions of the schema. I've been an rdbms guy (datawarehousing/ETL) for a long time now, I've seen a lot of large databases which have been in production for considerable time. They get messy.…

> They get messy. Really messy. They become basically unmaintainable. Apples, oranges and pears all squashed into a schema the shape of a banana. Moving it to your application doesn't solve this problem, it just pushes it out of the DB.

Exactly! In the best place you have to handle it. A nice API can make sense about the operation on the data and help it evolve.

These discussions always creates in my mind a picture of someone saying that we should be refilling all the forms that you have in a cabinet full of hand filled forms because someone decided that a new version of the form requires a new field like an e-mail address.

Post reply on HN