Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

12 Months with MongoDB

blog.wordnik.com

Post reply on HN
Print view View original

21–30 of 75 posts

Re: 12 Months with MongoDB

#21

Post by thibaut_barrere »

fehguy wrote: post #17
michaelchisari wrote: post #12

Alright, so, I fully understand the scalability reasons for using MongoDB, but I need someone to clearly explain to me when NoSQL would be a better solution than SQL from a development standpoint . Because like someone pointed out, Postgres without fsync can be just as fast. What is the advantage of giving up the ability to use SQL and the associated relational algebra that has long been established in that query lan…

Storing objects with any sort of hierarchy is so simple with Mongo that the LOC required to do so is ridiculously smaller. Querying them is also faster--for instance we can filter in our dictionary data with queries like {"entry.definitions.relatedWords":"cat"} instead of making some huge join and filtering against that.

I second that - a while back I wrote a MySQL application with a lot of interviews, made up of questions, answers and conditions etc.

I rewrote a very similar application with MongoDB this year and the code was way cleaner, and that's not caused by my increase in experience: the document-orientation is really helping here.

Re: 12 Months with MongoDB

#22

Post by danudey »

fehguy wrote: post #15
jrosoff wrote: post #2

Great writeup! Couple questions: - I'm curious why querying before a write makes such a big difference. I would have guessed that updating a document that's not in RAM would first load it into RAM, then perform the update. Does the write get applied to disk without loading the page into RAM first? If you do an update to a document that is not in RAM, is it in RAM following the update? - Can you elaborate on the corru…

Querying before the writes solved a lot of problems. It gets the object in the working RAM set. When doing an update, the database gets LOCKED when the statement hits the server--that means if your document is not in memory, you have to wait while it gets looked up. This was an easy, easy win for us. Regarding the corruption, I got an "invalido BSON object" or something on repair, which tells me some object was only…

This seems like the sort of optimization that should be occurring in MongoDB itself - instead of acquiring the lock, loading the record into memory (if it's not already), then making the change and releasing the lock, acquire the lock after the record has been loaded into memory (if it's not already).

Have you spoken with any of the MongoDB developers about why it's currently the way it is, vs. a more efficient update path?

Re: 12 Months with MongoDB

#23

Post by wslh »

Only had 5 days with MongoDB and I found it a good alternative for persistence of basic data structure in Python, my main concern was something that can $set individual elements of a JSON instead of retrieving the whole doc and modify it.

Re: 12 Months with MongoDB

#24

Post by danudey »

megaman821 wrote: post #7

It is kind of odd that speed is the main motivation to switch from MySQL. Horizontal scaling is the usually given reason. From what I have seen Mongo achieves most of its speed by not using fsync by default. There were some slides floating around a while ago that showed Postgres at about the same speed by turning off fsync.

I remember reading that when the developer of Sphinx was benchmarking MySQL's fulltext searches at Craigslist, most of the time spent performing a query was spent in locks and mutexes. The actual query time was very fast, but the overhead was what killed performance.

From what I understand, Postgres doesn't (necessarily) have those kinds of locking issues, but MongoDB does let you fetch documents (especially hierarchies) in a much more simple manner, rather than fetching them via potentially complicated join queries.

Re: 12 Months with MongoDB

#25

Post by fehguy »

danudey wrote: post #22
fehguy wrote: post #15

Earlier quoted context omitted.

Querying before the writes solved a lot of problems. It gets the object in the working RAM set. When doing an update, the database gets LOCKED when the statement hits the server--that means if your document is not in memory, you have to wait while it gets looked up. This was an easy, easy win for us. Regarding the corruption, I got an "invalido BSON object" or something on repair, which tells me some object was only…

This seems like the sort of optimization that should be occurring in MongoDB itself - instead of acquiring the lock, loading the record into memory (if it's not already), then making the change and releasing the lock, acquire the lock after the record has been loaded into memory (if it's not already). Have you spoken with any of the MongoDB developers about why it's currently the way it is, vs. a more efficient updat…

I think there are some possible timing issues with making that a general behavior in the server. 10gen did make it the default behavior on slaves, where the inserts are controlled by the oplog (http://jira.mongodb.org/browse/SERVER-1646).

For us, our DB abstraction layer made this behavior so simple to add that we didn't make much fuss about it.

Re: 12 Months with MongoDB

#26

Post by saurik »

emmett wrote: post #16
michaelchisari wrote: post #12

Alright, so, I fully understand the scalability reasons for using MongoDB, but I need someone to clearly explain to me when NoSQL would be a better solution than SQL from a development standpoint . Because like someone pointed out, Postgres without fsync can be just as fast. What is the advantage of giving up the ability to use SQL and the associated relational algebra that has long been established in that query lan…

For example, it's extremely awkward in SQL to find all the elements in a tree. There are at least 4 hacks I know of to fix this, none totally satisfactory. In a NOSQL context you can just store the entire tree - your implementation becomes straightforward and simple. In general though I agree with you. I can whip out SQL queries in seconds that would take me minutes to write against Mongo, even though they're all tec…

You usually want to query and update subtrees, often concurrently. Storing the entire tree is a horrible way to think about this problem. While some NoSQL databases try to help with this, they are not in any better position to solve the problem than a simple library over an SQL database.

Re: 12 Months with MongoDB

#27

Post by ergo98 »

mathias_10gen wrote: post #13
jasonjei wrote: post #5

Earlier quoted context omitted.

I think it's a bit short-sighted to assume everyone can use MongoDB if you're dealing with ACID type apps, or anything that deals with money. It's silly to say that they're developing at a much slower rate than someone that is. Use the right tool, or a combination of right tools, for the job.

While I don't support a blanket statements along the line of "every app should be using MongoDB," it is equally invalid to say that "anything that deals with money" has no use for MongoDB. If you look at http://www.mongodb.org/display/DOCS/Production+Deployments you will see a few financial and ecommerce sites. I can tell you that there are even more financial firms not on that list in various stages of production. E…

>you will see a few financial and ecommerce sites

Could you provide some examples? Scanning the list I see a couple that have a very periphery relation to financials, but the actual applications have very little financial applicability (and the implementations are trivial).

Though the person you responded to didn't actually say that `"anything that deals with money" has no use for MongoDB"', so you're setting up a strawman regardless.

Re: 12 Months with MongoDB

#28

Post by saurik »

thibaut_barrere wrote: post #14
michaelchisari wrote: post #12

Alright, so, I fully understand the scalability reasons for using MongoDB, but I need someone to clearly explain to me when NoSQL would be a better solution than SQL from a development standpoint . Because like someone pointed out, Postgres without fsync can be just as fast. What is the advantage of giving up the ability to use SQL and the associated relational algebra that has long been established in that query lan…

Hi Michael, from a development standpoint only, I like the fact that I don't have to write database migrations (as defined in Rails). It means I can iterate more quickly during the development. As well for data aggregation kind of jobs (such as http://www.toutpourmonipad.com/ where I munge different-formatted data streams), it's really convenient to be able to mix datas that are partly equal, partly different, when i…

This is because you are simply never updating your schema: if you actually want to rename a field, change a datatype, or reorganize your content, you are still going to need to run a migration, and now it won't even be possible to transanction lock the upgrade (better database servers, like PostgreSQL, can do multiple whole-daabase schema modifications within a transaction while still allowing non-conflicting access). In essence, your underlying schema is now "id->blob". ;(

Re: 12 Months with MongoDB

#29

Post by ergo98 »

michaelchisari wrote: post #12

Alright, so, I fully understand the scalability reasons for using MongoDB, but I need someone to clearly explain to me when NoSQL would be a better solution than SQL from a development standpoint . Because like someone pointed out, Postgres without fsync can be just as fast. What is the advantage of giving up the ability to use SQL and the associated relational algebra that has long been established in that query lan…

MongoDB is extraordinarily easy to develop for. It's a big (I would say the biggest) advantage of the tool.

Seriously, you owe it to yourself to do their superbly written tutorial -http://www.mongodb.org/display/DOCS/Tutorial.

That simplicity, however, is a front-end simplicity that -- if it isn't perfectly defined and appropriate for all future use -- can cause you tremendous pain in the future. A MongoDB document store tends to be very single purposed, and is horrendous to use outside of the narrow bands of that original intention (a simple aggregation is an exercise of extraordinary inefficiency). Which is fine for a straightforward, simple app like the one linked here, but isn't applicable for most projects. An RDBMS design encourages that you think about and abstract your data to the constituents, which often yields tremendous future flexibility, but with more front-end costs (and ongoing significant costs if you don't make the right decisions).

Once again, though, stories like this really...grind my gears (get off my lawn!). We have no idea how correctly or incorrectly they used their RDBMS, what their pain points are, etc, however they drank some of the magic elixir and all ills were cured. Anyone who questions the assumptions will be told the rote "Apples and Oranges!" quote (which is always humorous when the whole context is talking about moving from Apples to Oranges and how grand it is).

Re: 12 Months with MongoDB

#30

Post by saurik »

sbov wrote: post #18
michaelchisari wrote: post #12

Alright, so, I fully understand the scalability reasons for using MongoDB, but I need someone to clearly explain to me when NoSQL would be a better solution than SQL from a development standpoint . Because like someone pointed out, Postgres without fsync can be just as fast. What is the advantage of giving up the ability to use SQL and the associated relational algebra that has long been established in that query lan…

My only exposure to NoSQL in production is using document oriented databases for data that, if put in an SQL database, would require schema alterations over time. This might fall under scalability, but I've worked on a few projects where we just continually added new tables because applying an alter on the existing table in production would take an unknown amount of time. Another thing we sometimes did would be to ha…

Database servers like PostgreSQL can do the most common updates (add, delete, rename columns) instantly, S they Re abstracting over the underlying data storage. Only changing the type of a column should require reading and writing it, and you can do that change using a new temporary column and renaming it around, rather than using a whole new table. Even the alterations that take time can often be run with MVCC semantics, so existing users won't block. I don't even think MySQL (which is really bad at this) is as bad as the reality you are describing.
Post reply on HN