Live data from Hacker News

Guide to MongoDB for startups

optinidus.com

31–40 of 66 posts

Re: Guide to MongoDB for startups

#31
post #7

I find it somewhat depressing that the first comments on any post about MongoDB generally seem to be short, disparaging and dismissive without offering much if any substance. We use MongoDB in production, processing millions of new records every month and it works great. Our data has differing and evolving schemas and can be used as JavaScript objects or python dictionaries with little effort. It works and performs w…

> We use MongoDB in production, processing millions of new records every month and it works great.

It fails miserably in various ways when processing millions of new records per hour. On a very large scale MongoDB is unusable, but it is fine for problems 2-3 orders of magnitude smaller, so long as the data is not mission-critical.

Re: Guide to MongoDB for startups

#33
post #26
post #21

Earlier quoted context omitted.

Because some people don't use it correctly doesn't mean it can't be done. Here is a different random article about ServerDensity using it successfully https://blog.serverdensity.com/tech-behind-time-series-graph... This is nothing wrong with MongoDB.

I don't know that it's fair to say there's nothing wrong with it. We use it heavily, and generally enjoy it, but there are quite a few things wrong with it (write locks and ridiculous disk space usage come to mind up front). None of them are dealbreakers for us and can be worked around, but we also use it with the knowledge that it's a young product.

You might want to test out tokumx. Document level locking and compression are two of the features it has.

Re: Guide to MongoDB for startups

#34

Earlier quoted context omitted.

There was a spate of high profile "why we moved from MongoDB to $other_database" articles a year or so ago of startups getting burned when they realised that the all-signing all-dancing cure-all database they (thought they) had been sold, in fact wasn't. The issues encountered ranged from MongoDB failing to scale out as easily as promised, to significant loss of data. There was also some backlash because MongoDB didn…

Yes, I agree, however MongoDB is evolving This is probably what raised awareness of the issues of MongoDB http://www.slideshare.net/emiltamas/scaling-with-mongo-db-wi... and it's from 2011 It's a lot of time for a software project

It is a long time, but little has changed regarding the issues shown in that talk.

Re: Guide to MongoDB for startups

#35
post #7

I find it somewhat depressing that the first comments on any post about MongoDB generally seem to be short, disparaging and dismissive without offering much if any substance. We use MongoDB in production, processing millions of new records every month and it works great. Our data has differing and evolving schemas and can be used as JavaScript objects or python dictionaries with little effort. It works and performs w…

> Which means processing huge amount of data. Which SQL databases were never designed for.

When an article opens with a statement like it's difficult to give it much credence at that point. Unfortunately, this ignorance is very common among the MongoDB crowd.

Like your example? millions of new records every month? That's not a "huge amount of data". We're adding tens of millions new records each day, into mysql, and I would still consider our usage extremely intermediate if not amateur.

MongoDB has many benefits, they've improved on numerous oversights and flaws in their initial design, and it's becoming better all the time. What I'd love to actually see change is the naive belief of MongoDB users that it's a new, unique or vastly superior solution.

Re: Guide to MongoDB for startups

#36

Earlier quoted context omitted.

There was a spate of high profile "why we moved from MongoDB to $other_database" articles a year or so ago of startups getting burned when they realised that the all-signing all-dancing cure-all database they (thought they) had been sold, in fact wasn't. The issues encountered ranged from MongoDB failing to scale out as easily as promised, to significant loss of data. There was also some backlash because MongoDB didn…

Do you think it is MongoDB problem or all NoSQL dbs has the some problem?

Other NoSQL solutions were not marketed as a replacement for relational databases. MongoDB might be better now,but you cant take managers for idiots an expect them to buy into your product again. Right now MongoDB only lives because of javascript and nodejs ,since nodejs sucks big time with anything that is not MongoDB,library and driver wise.

Re: Guide to MongoDB for startups

#37
With the increasing data the search engines, e-commerce stores need to provide us with accurate results. Which means processing huge amount of data. Which SQL databases were never designed for.

What? There is no limit how much data you can stuff into a relational database and it mostly works really good. There are definitely some scenarios where relational databases are not optimal, for example aggregating huge datasets, but there are often solutions like materialized views or aggregates. The statement that RDBMSs are not designed to handle huge datasets is plain wrong.

With these growing needs the it was getting even more difficult to define a fixed structure to the data and the need to have a solution which handles unstructured data grew even more.

I don't understand how once inability to come up with a good data model is related. If you have no data model you will have a hard time working with your data anyway. You can stuff unstructured data into a string column and modern RDBMSs also support semistructured data like XML and JSON including operations to manipulate such data. And even with XML and JSON you still have a data model, just not a relational one.

I agree that the relational model is somewhat stiff and it is - somewhere between sometimes and often - a pain to design, implement or evolve a schema. But a fair amount of the complexity usually comes from the domain you are modeling and does not go away when you switch to a different data model - your database may not complain when you stuff documents with seven different schemas into it, but now the burden is on the code to deal with documents with seven different formats. Data modeling and model evolution is a difficult problem and does not go away by switching technologies. The major difference is the point in time when you recognize that you have a problem.

So most of the NoSql databases traded consistency for Availability and Partition Tolerance which was a fundamental shift from the relational world where in you had to design your schema perfectly so that there were no inconsistencies in the data ( 1NF, 2NF, 3NF, BCNF).

Consistency in the sense of the CAP theorem and in the sense of database schema normalization are almost completely unrelated. This gives me the impression that the author does not really know what he is talking about.

One of the biggest advantages claimed by NoSql databases is Horizontal scalability , which is the ability to handle more load by adding in more machines, since the computing power these days has gone cheap compared to the effort required in to fine tune the app and add more computation power and memory to an existing machine.

RDBMSs support partitioning across server as well. It is probably harder to set up and does not scale as well because the provided guarantees are stronger, but it is possible.

MongoDB uses reader-writer locking mechanism , it gives concurrent access to reads but exclusive rights for write operations which means it can handle concurrent read operations but if there is aright operation it will block the reads until it gets completed .

So do many RDBMSs. Or they use MVCC and allow even more parallelism.

Prior to mongodb version 2.2 mongodb had an instance level lock which means that whenever there was a write operation it used to lock the entire mongodb instance and even if there was a read queued for a different database it will have to wait as the write operation blocked the entire mongod instance.

This was changed in 2.2 where in the write operation locked the whole database instead of the complete instance. So the solution which was left to scale the write operations was to add in more shards and route the next write query to a separate mongodb shard instance.

This is still extremely inferior to modern RDBMSs which usually support row level looking.

For applications this is by far the most important point to take into consideration when choosing which database to use, as for write intensive application this might come in their way to scale.

RDBMSs allow you to opt out of consistency - if you want to read uncommited data, you are usually free to do so. I don't see why a RDBMSs should intrinsically allow less parallelism but admittedly dropping consistency guarantees is quite contrary to the reasons you usually choose a RDBMSs to begin with.

UPDATE: I probably misinterpreter the last part about locking and scalability - after reading it again, it sounds like the author actually warns that using MongoDB may cause scalability issues. I leave my comments as they are although they sound a bit strange in that light.

TO-THE-DOWN-VOTERS: I would love to hear where I am wrong, my opinion is neither set in stone nor absolute truth.

Re: Guide to MongoDB for startups

#38
post #7

I find it somewhat depressing that the first comments on any post about MongoDB generally seem to be short, disparaging and dismissive without offering much if any substance. We use MongoDB in production, processing millions of new records every month and it works great. Our data has differing and evolving schemas and can be used as JavaScript objects or python dictionaries with little effort. It works and performs w…

> We use MongoDB in production, processing millions of new records every month and it works great. It fails miserably in various ways when processing millions of new records per hour . On a very large scale MongoDB is unusable, but it is fine for problems 2-3 orders of magnitude smaller, so long as the data is not mission-critical.

One million new records a month is less than one new record a second. Ten million new records a month is almost four new records a second (slightly more than four if the month is February!). That is not a challenging level of performance for any database, relational or otherwise.

It is indeed the millions-per-hour order of magnitude where things get interesting.

Post reply on HN