Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“MongoDB is dead. Long live Postgresql”

github.com

Post reply on HN
Print view View original

61–70 of 160 posts

Re: “MongoDB is dead. Long live Postgresql”

#61

Post by spellboots »

pilif wrote: post #46

The title is a bit misleading. This is basically an announcement of a fork of Errbit that has Postgres support. Additionally, the fork was announced as an issue on errbit with no discussion or as an official pull request. I would not consider this good etiquette. If you fork your project (especially without discussing the intention first), adding a bug to the original project isn't a very nice thing to do. An officia…

I disagree, I think that opening an issue on github is a good way to start a discussion about a feature. Many projects accept feature requests this way and if anyone did the same for one of my projects, this would be the way I would prefer them to handle it.

Re: “MongoDB is dead. Long live Postgresql”

#62

Post by mattquiros »

rlpb wrote: post #30
functional_test wrote: post #19

Seriously, another case of using Mongo incorrectly? I want to believe all the Mongo hate, but I can't because I always find out that the actual problem was one or more of: * didn't read the manual * poor schema * didn't maintain the database (compactions, etc.) In this case, they hit several: " Its volume on disk is growing 3-4 times faster than the real volume of data it store;" They should be doing compactions and…

> Seriously, another case of using Mongo incorrectly? If a large proportion of MongoDB users are using it incorrectly, then I'd argue that it is a MongoDB problem, if only a documentation and messaging one. Clarity on what is and is not an appropriate use should be prominent. So, what is this proportion?

> If a large proportion of MongoDB users are using it incorrectly, then I'd argue that it is a MongoDB problem

Hey, that sounds a lot like the logic of Java haters!

Kidding aside, I'm afraid I'm not sure your logic is convincing, but that's for another debate.

Re: “MongoDB is dead. Long live Postgresql”

#63

Post by remon »

lucian1900 wrote: post #25
functional_test wrote: post #19

Seriously, another case of using Mongo incorrectly? I want to believe all the Mongo hate, but I can't because I always find out that the actual problem was one or more of: * didn't read the manual * poor schema * didn't maintain the database (compactions, etc.) In this case, they hit several: " Its volume on disk is growing 3-4 times faster than the real volume of data it store;" They should be doing compactions and…

Mongo's disk format is extremely wasteful, the database files are gigantic. That is a real problem and there is no way to compact this to anywhere near the size something like Postgres would have for the same data. Mongo is very bad at managing used memory. In fact it doesn't actually manage memory since it just mmaps its database file. It also touches disk much more often than would be reasonable, especially for how…

Although it is true that MongoDB uses a lot of disk compared to your average RDMS there are reasons for that.

1) MongoDB (and various other NoSQL solution) are schemaless and thus have to store document fields along with the values for each document. This alone usually results in roughly twice as much actual disk space being used compared to an RDBMS.

2) MongoDB preallocates fairly large chunks of disk for their mmap based storage (2Gb per database files by default). This means there will be up to 2Gb * N where N is the number of logical databases in "wasted" (more accurately, unused) space. This can be addressed somewhat through the --smallfiles option.

3) The biggest issue that I actually consider an design flaw is the ineffective reuse of disk space previously occupied by deleted or, more commonly, moved documents. MongoDB reserves a bit of padding for each document but since a lot of documents can grow over time these documents will be moved around on disk leaving holes in the data files. These holes are not adequately re-used and a compaction is required to make that space available again. Compaction is NOT a background process at the moment and blocks writes. The "usePowerOf2Sizes" option will help with this issue at the expense of always using a power of 2 size in bytes per document.

The above are factual reasons why MongoDB uses a lot of disk space. It's certainly a relatively young database and some issues do need to be addressed but this whole polarizing "it's terrible booo!" nonsense has to stop. Inform yourself, choose the tech appropriate for your project and post mortem aftwards.

Small note on the mmap thing; a lot of people consider the mmap based storage engine a big issue (I tend to agree). Tokutek offers what seems to be a better storage engine but does lag behind a bit on releases. I'm not affiliated with them but if you're interested you can check out http://www.tokutek.com/products/tokumx-for-mongodb/

Re: “MongoDB is dead. Long live Postgresql”

#64

Post by stonewhite »

jvvlimme wrote: post #50

If you want to use MongoDB in a project and you don't intend to rely heavily on the aggregation framework, the consider TokuMX ( http://www.tokutek.com/products/tokumx-for-mongodb/ ) as it alleviates many of the shortcomings of MongoDB (data compression, document level locking for writes, ...) + it adds transactions. It's a drop in replacement so it will work with current drivers. (if you have a running mongo cluster…

It is basic an ops-friendly mongo fork with _obviously_ better engineering decisions. I hope mongodb will support pluggable storage engines soon.

Re: “MongoDB is dead. Long live Postgresql”

#65

Post by endijs »

I'm no MongoDB expert, but recently started to look into this db. Can anyone tell me (from experience, not from promo materials) - for which use cases MongoDB is good fit and for which ones it's not? It's clear that it can't fit for everyone. That's why it would be good to know in advance, for what it most likely to find and for what it's most likely not to fit.

Re: “MongoDB is dead. Long live Postgresql”

#66

Post by rbranson »

cullenking wrote: post #44

Maybe I am just incredibly lucky, but mongodb has worked fine for ridewithgps.com - we are sitting at 670gb of data in mongo (actual DB size, indexes included) and haven't had a problem. Replica sets have been fantastic, I wish there was another DB out there that did auto-failover as cleanly/easily as mongo does. We've had a few server crashes of our primary, and aside from 1-2 seconds or so of errors as requests com…

How many reads do you do on that server? 192GB + 8 SSDs is a pretty serious setup. Just the disks themselves should be able to push 500K+ random IOPS.

Re: “MongoDB is dead. Long live Postgresql”

#67

Post by remon »

memracom wrote: post #36

Lets just say that PostgreSQL answers the criticisms of relational databases that led to NoSQL. The complaints all boiled down to saying that the RDBMS forced you to do things one way and that it was cumbersome. PostgreSQL evolved and fixed the most annoying issues like JSON support and schemaless key-value store support. That's the way open source is supposed to work. Now folks are learning that throwing out the bab…

This is not true at all. The actual realization the past years is that strictly enforced relationality (is that a word?) and transactions are constructs that are not always or even rarely actually needed. Eventual consistency, schemaless data modelling and so on picked up steam and for good reasons. Every technology that survives the "Oh, new toy!" stage has a place or it wouldn't still exist. It is up to developers to choose the appropriate technology for them and their projects. That isn't to say that a lot of persistence problems cannot be solved by an RDBMS, a k/v store and a document store. In that case just base your decision on other drivers (comfort level, cost, and so on)

Re: “MongoDB is dead. Long live Postgresql”

#68

Post by remon »

jeffdavis wrote: post #51

"Its volume on disk is growing 3-4 times faster than the real volume of data it store[sic]" Are they saying that it has a high constant overhead to the data, or are they saying the storage grows in a super-linear fashion?

It's constant overhead with some spikiness to it. See my reply in this thread for details.

Re: “MongoDB is dead. Long live Postgresql”

#69

Post by efuquen »

remon wrote: post #63
lucian1900 wrote: post #25

Earlier quoted context omitted.

Mongo's disk format is extremely wasteful, the database files are gigantic. That is a real problem and there is no way to compact this to anywhere near the size something like Postgres would have for the same data. Mongo is very bad at managing used memory. In fact it doesn't actually manage memory since it just mmaps its database file. It also touches disk much more often than would be reasonable, especially for how…

Although it is true that MongoDB uses a lot of disk compared to your average RDMS there are reasons for that. 1) MongoDB (and various other NoSQL solution) are schemaless and thus have to store document fields along with the values for each document. This alone usually results in roughly twice as much actual disk space being used compared to an RDBMS. 2) MongoDB preallocates fairly large chunks of disk for their mmap…

"Small note on the mmap thing; a lot of people consider the mmap based storage engine a big issue (I tend to agree)."

This. Aside from general reliability issues I've had in the past, which are definitely fixable and might be now, this design decision is the thing that will cripple, and continue to cripple the db. The idea that some how the kernel is going to better at managing a db's memory is ridiculous. Kernel paging was designed for a completely different use case, it's not optimized to manage the memory of a database. The idea that somehow this line of reasoning: "oh hey guys, those Linux kernel hackers are smart and they can do it way better then use so let's just use their memory management system" was ok in the mind of the MongoDB creators really puts me off. It doesn't matter how many geniuses work on kernel memory management when they are solving a completely different problem then you need to solve, all memory management schemes aren't equal. When providing a db you need full control to be able to probably tune memory management to fit your needs, there is no magical fix for this.

Re: “MongoDB is dead. Long live Postgresql”

#70

Post by saraid216 »

mattquiros wrote: post #62
rlpb wrote: post #30

Earlier quoted context omitted.

> Seriously, another case of using Mongo incorrectly? If a large proportion of MongoDB users are using it incorrectly, then I'd argue that it is a MongoDB problem, if only a documentation and messaging one. Clarity on what is and is not an appropriate use should be prominent. So, what is this proportion?

> If a large proportion of MongoDB users are using it incorrectly, then I'd argue that it is a MongoDB problem Hey, that sounds a lot like the logic of Java haters! Kidding aside, I'm afraid I'm not sure your logic is convincing, but that's for another debate.

Actually, it sounds like the logic of Java proponents.
Post reply on HN