Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't use MongoDB

pastebin.com

Post reply on HN
Print view View original

31–40 of 331 posts

Re: Don't use MongoDB

#31

Post by gfodor »

antirez wrote: post #26

I appreciate the "public service" intend of this blog post, however: 1) It is wrong to evaluate a system for bugs now fixed (but you can evaluate a software development process this way, however it is not the same as MongoDB itself, since the latter got fixed). 2) A few of the problems claimed are hard to verify, like subsystems crashing, but users can verify or deny this just looking at the mailing list if MongoDB h…

At the end of the post the author notes his concern isn't with the technical bugs per se, but with the deep rooted cultural problems and misplaced priorities the existence of those problems reveal.

Re: Don't use MongoDB

#32

Post by electic »

jdagostino wrote: post #25
electic wrote: post #12

We extensively tested this inside Viralheat with a write heavy load of over 30,000 writes per second and basically it failed our test. It is not robust for the analytics world is the conclusion we came to. Though, I hope it gets better one day...it has potential.

what did you end up going with?

Our company is a big data company. So our amazing engineers are responsible for storing hundreds of millions of pieces of data per week AND also crunching and analyzing that data. So basically we need a system where we can have incredible write and read performance but also a system that is elastic in nature. Most importantly, it has to be available.

Before I go into more details, MongoDB is great for most people who don't have a high transaction volume. It is easy to setup and easy to use. So if you are in this camp, MongoDB is probably a good fit for you.

We did about two months worth of extensive tests in our lab. Basically two things didn't bode well for us. One, the locking killed reading...we just had a hard time keeping the flow of writes and the flow of data to our statistics cluster alive. Yea, you could use replication but that too didn't work too well performance wise. Two, the sharding didn't seem that robust. As the cluster got bigger and bigger, we started noticing the overhead of keeping it up was getting to be too great. Rather than write in detail, I think this article covers some of the scaling issues we experienced:

http://blog.schmichael.com/2011/11/05/failing-with-mongodb/

We finally used a hybrid system. We went with Membase, now CouchBase, to handle immediate storage and we are now implementing Hadoop for our long term storage needs.

P.S. Our entire stack is a KV in nature.

Re: Don't use MongoDB

#33

Post by woodhull »

I couldn't agree more with this analysis, with the added addition that the single threaded nature of the JS interpreter can also cause really bad & unexpected performance things to happen.

Most of the people who are excited about mongo, have never used it in a high volume environment, or with a large dataset. We used it for a medium sized app at my last employer, with paid support from 10gen, and everyone on the project walked away wishing we had stayed with a more mature data store.

Of course things work well when traffic is low, everything fits in memory, and there are no shards.

Re: Don't use MongoDB

#34

Post by antirez »

gfodor wrote: post #31
antirez wrote: post #26

I appreciate the "public service" intend of this blog post, however: 1) It is wrong to evaluate a system for bugs now fixed (but you can evaluate a software development process this way, however it is not the same as MongoDB itself, since the latter got fixed). 2) A few of the problems claimed are hard to verify, like subsystems crashing, but users can verify or deny this just looking at the mailing list if MongoDB h…

At the end of the post the author notes his concern isn't with the technical bugs per se , but with the deep rooted cultural problems and misplaced priorities the existence of those problems reveal.

That's a fair problem, but I think It is true for other products as well and was true for things that we feel very solid today like MySQL. In other words there is a tention between stability and speed of development, a very "hard" tention indeed. It is up to the developers culture and sensibility to balance the two ingredients in the best way.

One of the reasons I don't want to create a company around Redis, but want to stay with VMware forever as an employee developing Redis, is that I don't want development pressures that are not drive by: users, technical arguments. So that I can balance speed of development and stability as I (and the other developers) feel right.

Without direct reference to 10gen I guess this is harder when there is a product-focused company around the product (but I don't know how true this is for 10gen as I don't follow very closely the development and behavior of other NoSQL products).

Re: Don't use MongoDB

#35

Post by gfodor »

antirez wrote: post #34
gfodor wrote: post #31

Earlier quoted context omitted.

At the end of the post the author notes his concern isn't with the technical bugs per se , but with the deep rooted cultural problems and misplaced priorities the existence of those problems reveal.

That's a fair problem, but I think It is true for other products as well and was true for things that we feel very solid today like MySQL. In other words there is a tention between stability and speed of development, a very "hard" tention indeed. It is up to the developers culture and sensibility to balance the two ingredients in the best way. One of the reasons I don't want to create a company around Redis, but want…

MySQL is a poor analogy because the history of MySQL is very similar to 10gen: a 'hacker' solution originally patched together by people who didn't take their responsibility as database engineers very seriously. It's only after years (decades) of work that MySQL has managed to catch up with database technology of the 80s in terms of reliability and stability (and it still has plenty of issues, as the most recent debacles with 5.5 show.)

On the other hand, commercial vendors like Oracle and open source projects like PostgreSQL recognize their role as database engineers is to first and foremost "do no harm." Ie, the database should never destroy data, period. Bugs that get released that do cause such things can be traced back to issues that are not related to a reckless pursuit of other priorities like performance. Watching the PostgreSQL engineers agonize over data integrity and correctness with any and all features that go out that are meant to improve performance is a re-assuring sight to behold.

This priority list goes without saying for professional database engineers. That there is such a 'tension' between stability and speed says less about a real phenomenon being debated by database engineers and more about the fact that many people who call themselves database engineers have about as much business doing so as so-called doctors who have not gone to medical school or taken the Hippocratic oath.

Re: Don't use MongoDB

#36

Post by StavrosK »

antimora wrote: post #8

I was planning adopt MongoDB for my big project and this post puts some doubts. Is this true? Could anyone confirm or deny this?

I've had version 1.6, if I remember correctly, just lose half my data with no warning, so I can believe this.

The fact that the 32-bit version also truncated data with no warning doesn't make me hopeful, either.

Re: Don't use MongoDB

#37

Post by ericflo »

But it does 8,000,000 operations per second! http://www.snailinaturtleneck.com/blog/2010/05/05/with-a-nam...

(Sorry, possibly excessive snark. That said, I think that blog post is a good example of one of this pastebin author's points: at least historically, benchmark numbers have been a big focus for Mongo developers.)

Re: Don't use MongoDB

#38

Post by mtkd »

Anyone using Mongo currently has to be aware there are likely to be some teething issues as it is very new technology.

I haven't used it in production (yet), but I would have no fear of using it today. I would run regular consistency monitoring and validation around critical data just like I do with our SQL databases.

I'm willing to take my part of the pain and inconvenience in making technology like this stable.

You could have written this about any adolescent SQL server BITD. All the tools you use today had to go through this process.

For me Mongo is awesome and getting more awesome. Mongo and technology like it is the reason I still get excited about writing new apps.

Re: Don't use MongoDB

#39

Post by antirez »

gfodor wrote: post #35
antirez wrote: post #34

Earlier quoted context omitted.

That's a fair problem, but I think It is true for other products as well and was true for things that we feel very solid today like MySQL. In other words there is a tention between stability and speed of development, a very "hard" tention indeed. It is up to the developers culture and sensibility to balance the two ingredients in the best way. One of the reasons I don't want to create a company around Redis, but want…

MySQL is a poor analogy because the history of MySQL is very similar to 10gen: a 'hacker' solution originally patched together by people who didn't take their responsibility as database engineers very seriously. It's only after years (decades) of work that MySQL has managed to catch up with database technology of the 80s in terms of reliability and stability (and it still has plenty of issues, as the most recent deba…

I agree with you but my comments are more about telling what is going on in my opinion, instead of telling what I think should be the right priority list. Even if I agree I still recognize that MySQL had a much bigger effect to the database world compared to PostgreSQL, so the success of a database can sometimes take strange paths.

But I think a major difference between MySQL and Redis, MongoDB, Cassandra, and all the other NoSQL solutions out there is that MySQL had an impressive test bed: all the GPL LAMP applications, from forums to blogs, shipped and users by a shitload of users. We miss this "database gym" so these new databases are evolving in small companies or other more serious production environments, and this creates all the sort of problems if they are not stable enough in the first place.

So what you say can be more important for the new databases than it was for MySQL indeed.

Re: Don't use MongoDB

#40

Post by chrissanz »

buff-a wrote: post #18
[deleted] wrote: post #15

Earlier quoted context omitted.

[deleted]

It works on my machine!

A lot of trolling here, I've never had any issues of missing data. When claiming a db (as big and popular as mongodb) doesn't work, you should include references, company, examples on how to reproduce, etc. Enough said: http://www.mongodb.org/display/DOCS/Production+Deployments
Post reply on HN