Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Year of MongoDB

speakerdeck.com

Post reply on HN
Print view View original

101–110 of 133 posts

Re: A Year of MongoDB

#101

Post by jshen »

marcosdumay wrote: post #94
jshen wrote: post #93

Earlier quoted context omitted.

"till it reached some kind of moderate scale where I realized it was a terrible choice going with a NoSQL db (Sometimes, I'd have to duplicate data because there were no Joins, etc)" That statement doesn't make sense to me. One of the first things you do with a sql database when you reach moderate scale is denormalize data so you don't have to do joins, or shard which prevents a lot of joins. Joins are super convenie…

You denormalize SOME joins. On nearly all of the cases, they are very usefull, but once in a while one of them gets too slow, so you make a bit more of work enforcing the consistence by hand, accepts a bit more of risk, and denormalize it. Throwing all of them away because a few may create problems later is a bit of an over reaction.

How many joins do you think this page uses? A denormalized version that doesn't scale probably uses 25-50. I'll bet the scalable implementation uses close to zero.

Re: A Year of MongoDB

#102

Post by jsemrau »

Well my 2 cents to the discussion.

We really tried to make Mongo work over the last 1.5 years. We decided to go for mongo mostly because of JSON and geospatial indexing.

We became to realize that for our business case it is not the right tool. That said, and I work with databases now for 20+ years, I don't see what problem Mongo solves.

Re: A Year of MongoDB

#103

Post by deno »

zimbatm wrote: post #76
jdoliner wrote: post #36

Earlier quoted context omitted.

Disclaimer, I work for RethinkDB a competitor of MongoDB. I completely agree with you that NoSQL is not a pure-white solution but I disagree that it's only useful for specific use-cases. NoSQL has frequently been touted as this magic concept that makes databases scale well. It doesn't as many of the poorly scaling NoSQL databases on the market today show you. The most you can say is that it maybe lets you ignore a fe…

Isn't Google BigQuery a SQL on top of a schemaless database ?

It’s a distributed near–real‐time querying system for data at rest that supports subset of SQL.

Google has scalable SQL database called Spanner[1]. They also have a proper NoSQL database called internally Megastore[2] that provides distributed transactions based on entity‐groups on top of BigTable. Also known as High Replication Datastore and available via AppEngine[3].

[1] https://en.wikipedia.org/wiki/Spanner_%28database%29

[2] https://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf‎

[3] And hopefully via Google Compute pending announcement in few days at I/O. That is if Google actually wants their Cloud to succeed.

Re: A Year of MongoDB

#104

Post by danbmil99 »

cobbzilla wrote: post #50
eranation wrote: post #29

Earlier quoted context omitted.

All I can say is this: if the saying "Always plan to throw away your MVP" is true, then I can't see any other storage solution other than MongoDB (or a similar schema-less document storage DB) for MVPs. The speed of development and flexibility are simply worth it. Yes, it is hard to refactor a live product and move it from MongoDB to MySQL / Postgre but was done before and you only do that if you get traction, so its…

You make a VERY big "if" in your first sentence, one that (admittedly anecdotally) I've very rarely seen hold true in tech companies. Much more often, the MVP becomes the product, and all those shortcuts and poor design decisions come back to kill your productivity when it becomes necessary to refactor foundational tech/designs that have metastasized throughout the codebase. I am curious if this is other folks' exper…

If NoSql's speed and flex gets you to MVP, and MVP gets you traction, then all the hell you may go through to rearchitect for SQL is probably worth it.

Premature scalability is as dumb as premature optimization and premature generality.

Re: A Year of MongoDB

#105

Post by ricardobeat »

malkia wrote: post #6

Why the URL are not working from the SpeakerDeck document? Captain Obvious here: But isn't that the purpose of the Web? It's the WEB... make those links work!

I think this is a fair comment. SpeakerDeck focuses on preserving the presentation layout with absolute fidelity, and does that by using static images. It's a trade-off. It's certainly possible to implement some kind of link-detection, I hope they are working or plan to work on it.

Re: A Year of MongoDB

#106

Post by moneypenny »

RethinkDB is a lovely piece of work doffs hat but the presentation author is correct when he says, "until Rethink is ready" - I can't figure out how to make Rethink go quickly, but I'm sure it'll all pan out in time.

Can anyone with more experience than I tell me one outstanding feature of MongoDB? Because I can't think of one.

Re: A Year of MongoDB

#107

Post by k_bx »

thedufer wrote: post #41
ericcholis wrote: post #16

Earlier quoted context omitted.

Agreed on all counts. Sounds like he is complaining more about issues with VM performance than MongoDB performance on VMs. The joins statement kills me. If somehow the database design requires joins, Mongo is fast enough to run two queries and then let you work with them in code.

> If somehow the database design requires joins, Mongo is fast enough to run two queries and then let you work with them in code. That's definitely not true. What if you were planning on filtering after the join? You may find yourself pulling millions of records. The bandwidth alone would bring you down. I work with MongoDB, and once in awhile I really miss joins. You can't emulate joins in any reasonable amount of t…

> That's definitely not true. What if you were planning on filtering after the join? You may find yourself pulling millions of records. The bandwidth alone would bring you down. I work with MongoDB, and once in awhile I really miss joins. You can't emulate joins in any reasonable amount of time in Mongo.

I agree that joins should be done by mongo (as a feature, similar to RethinkDB), but they can get you in trouble when you convert your collection from local-to-joined-one into sharded-into-multiple-servers. You'd then gain huge network load that can potentially bring your system down.

If you decided that data in some particular collection is potentially huge -- just don't use joins on it (unless REALLY once in a while).

Re: A Year of MongoDB

#108

Post by nasalgoat »

AlisdairO wrote: post #79
shin_lao wrote: post #69

Earlier quoted context omitted.

You are pretty violent against MongoDB. Any reason why?

I suspect the reason that Mongo attracts so much hate is that it's more heavily marketed/evangelised than other NoSQL DBs - so more in the public eye and people expect more from it. Since the reality is that Mongo is not yet a technically strong product (regardless of its merits in terms of ease of use and getting simple things done quickly), it's bound to attract more negative attention.

Having spent the last two years trying to make MongoDB work at large scale and only succeeding because I have a ton of resources (200+ db hosts), I can say it's more than just the spotlight.

It is a fundamentally broken product.

Re: A Year of MongoDB

#109

Post by nasalgoat »

As I clicked forward in the slideshow, I kept expecting to find something to disagree with but it never happened. Mostly I nodded to myself.

I have a huge investment in MongoDB at this point, both financially and in equipment - over 200 machines dedicated in various separate clusters - and a pivot to another datastore at this point would be a significant re-engineering effort.

All the people talking about changing databases after the MVP have clearly never had to deal with a typical hockey stick growth profile and having to allocate engineering resources based on need - either making the product better, or wasting time changing your database and losing traction.

Anyway, I'm hoping posts like this can dissuade people from choosing MongoDB for anything destined for high throughput Enterprise-level production environments.

I just wish there was something I could easily put in to replace MongoDB, but none of the available options quite fit the same document store model, but make better use of available resources and provide much better performance.

Re: A Year of MongoDB

#110

Post by nasalgoat »

dmytton wrote: post #14

A few comments on the problems: * CPU bottleneck. The mongod is no usually bound by CPU except for building indexes on existing data (which shouldn't really happen in production). The issue he's talking about is contention between the web server (or workers) and the mongos. This isn't anything unexpected. It's recommended to put the mongos onto the application server and then you scale this by adding CPUs initially b…

There are a number of very large bottlenecks in MongoDB, and one of the worst is the mongos process.

I can't run more than 100 processes on a server talking to a mongos process, because any more than that, mongos uses 70% of the CPU power on a 32-core box. 70% of all CPU for database transaction overhead! After that, it simply stops working.

The next one is the insane 20K connection limit per mongod process that is hardcoded into the binary. When asked about it, 10gen says they just decided years ago that that was the limit because no one would ever have more than 20GB in a server. So, I can't just run more mongos processes on other boxes - it hits the 20K limit.

I could go on but the point is, MongoDB is really badly designed.

Post reply on HN