Live data from Hacker News

Why Did So Many Startups Choose MongoDB?

nemil.com

151–160 of 164 posts

Re: Why Did So Many Startups Choose MongoDB?

#151
post #16

Back in 2012 I worked at a startup and we had a few backend data stores (Vertica, Postgres, Redis, Big Table, MySQL, Cassandra, ElasticSearch, Hadoop). When 10gen came along, they sold us an amazing vision. We were processing 500 million to billion social media messages a day and they sold us on this dream that we would have very fast writes, a cluster of 9 machines in master-slave-slave config for fail over. We woul…

But... Kafka isn't a database?...

No, it's not. We were trying to wrap our own system around Mongo to do the job of Kafka which is why I said that.

Re: Why Did So Many Startups Choose MongoDB?

#152
We are also one of those companies that bought into the promise (or hype) in 2011 and gave MongoDB a spin. After some very successful tests, I had long conversations with my CTO on why we should deploy MongoDB in prod and make it our primary OLTP store. Yes, MongoDB had it's challenges in 1.6 and it required some "faith". But it's 2017 now and MongoDB totally delivered on it’s vision, at least for us. Since 2012, that's when we upgraded to 2.0, we had 10 min of total downtime. That's 99.999% uptime in 5 years. Year after year, 99.999% uptime. And that 10 min is the total for scheduled (btw, we don't have scheduled downtime for our MongoDB clusters since there is no need for that) and un-scheduled downtime. We never take the service down. We perform all maintenance operations during regular business hours with no impact to end-users at all. So much about reliability! MongoDB as a product has bugs. What a surprise, what software product doesn't. But it's about you community (we are running the community edition) supports the product. We recently discovered an issue as well causing the primary to crash. As designed, another member in the repl set took over and there was no downtime at all. Even though we don't have a support contract, the issue was acknowledged by MongoDB in hours and fixed in a couple of days. I wish we had that type of support for products where we do have a support contract! And last but not least, if you can't figure out how to successfully use MongoDB, for god sake, take some free training classes on MongoDB University. MongoDB might not be a good fit for every use-case, but it's already an awesome platform for an incredible number of applications. Polyglot environments is the keyword here. SQL has it’s advantages and disadvantages and so does NoSQL. Use the best tool for the problem. One size fits all doesn't work anymore and MongoDB is clearly the best and most popular document store on the market. And even better, other platforms like postgres have acknowledged that the idea of storing JSON has its merits and have implemented its own version of that idea. However, IMO, if your use-case is a good fit for a document store, use MongoDB. We have done so and couldn't be happier with that decision.

Re: Why Did So Many Startups Choose MongoDB?

#153
I'd wanted to lift the burden of writing ORM layers and dealing with relational algebra and focus on persisting my intrinsic application models. I'd studied XML and other semi-structured web data at the graduate level, but was frustrated there wasn't a technology that was meeting that need. Companies like Microsoft and Oracle were adding XML functionality to their databases, but because it was bolted on an already constrained structure, it failed to address the mangling that relational structures impose.

I saw a demo of MongoDB in 2010 and was thoroughly impressed. The developer experience out of the box was exquisite. During a 1 hour presentation, I was able to download the binary, connect to the shell, create records in a familiar form (JSON), query the data from my preferred language, and all within a often second-class platform, Windows.

There was a certain _joy_ to using MongoDB, but I had my doubts, too. If this form of database interaction was so easy and fun to use, what was the catch?

Well, the catch was that MongoDB was re-thinking the issue from the ground up, re-envisioning the way we work with data, in an attempt to transcend the inertia that held back existing offerings.

When I started working with MongoDB, it felt like the same transcendence I experienced moving from C++ to Python in 1998--the approach was simpler, more intuitive, and powerful.

Sure, there were some growing pains. The concept came first without acknowledged writes or even journaling. Replication came in stages. Sharding came later.

But our operation was able to go from startup-scale to a global enterprise-scale operation backed by MongoDB, performing admirably, and giving developers the ability to rapidly innovate in a low-impedance environment. We've since developed dozens of applications against MongoDB.

I've considered emergent competing offerings like PostgreSQL with its JSON store, but when I did a few years ago, they were still dramatically behind in replication and sharding support, requiring extensive administration for aspects that MongoDB handled out of the box.

And in the meantime, MongoDB has continued to innovate, bringing in pluggable storage backends, advancing the query engine and aggregation framework, adding native geo-location support, and formalizing the concept of shard zones (a key feature for global scalability). And with Atlas, they've made server administration a near zero-cost proposition.

I'm a fan because when it comes to boots-on-the-ground experience, I see nothing else in the industry that even comes close. The most pressing deficiencies have been addressed and without compromising the key innovations. As a developer and technology leader, I choose MongoDB as my default database and would only consider a traditional database if the use case was intrinsically highly constrained and tabular in nature.

Re: Why Did So Many Startups Choose MongoDB?

#154

Earlier quoted context omitted.

Not only that but you can use PostgreSQL as a NoSQL and there are benchmarks showing that it outperforms MongoDB. And if you do not want to commit to either SQL or NoSQL, the PostgreSQL has you covered. You can put a JSONB column in any of your tables to handle the part of your data that is schemaless, and you can still index the data in that JSONB column. There are a very few applications that need distributed scala…

These PostgreSQL features are relatively recent. The jsonb column type dates back to 9.4, which was released in Dec 2014, less than three years ago. MongoDB had been around for well over five years by that point. There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets…

> These PostgreSQL features are relatively recent. The jsonb column type dates back to 9.4, which was released in Dec 2014, less than three years ago. MongoDB had been around for well over five years by that point.

It's much much harder to design a reliable engine than a new datatype.

> There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.

Yes they are, but chances are you don't have that need. You're not Google or Facebook.

You also picked up a really bad database to talk about scalability[1]

[1] https://www.datastax.com/wp-content/themes/datastax-2014-08/...

Re: Why Did So Many Startups Choose MongoDB?

#155

Earlier quoted context omitted.

My guess... SQL is a lot for newcomers to grasp on top of learning the rest of their stack. The appeal of "no schema" is most likely what gets them as it seems easier than SQL. Simple things at first seem daunting like: constraints, joins, and indexes. What they fail to see is that doing these database level things in the application layer becomes the final level hell and you get to live it. People also undervalue th…

Maybe I'm too old now... but SQL is relational algebra translated into some sort of code/language. I did a lot of SQL & relational algebra for my undergrad databases class [which was about 20 years ago now]. Has SQL/DBs been phased out of CS education? Just genuinely curious...

Nope, my college had at least 2 classes to cover databases. First one was to learn basics (although it had fun project to implement a DB engine and handle simple SQL), and second one was more about using it in real applications.

The classes were not mandatory so you could technically graduate with CS without knowing anything about databases.

Re: Why Did So Many Startups Choose MongoDB?

#156
post #97

I use Mongo & its been working great. Our data is not relational (imagine you were tasked with building "Powerpoint" on the web). Compared to mysql I like that it has automatic failover out of the box. I can also just add new replica set members without having to stop the master or bring a slave offline to rsync it over. With something like mysql I have to learn about & choose between row based replication or stateme…

Of course I don't know your product, but when I imagine Powerpoint-as-a service, do users not relate to decks which relate to slides which relate to the content on those slides? Content itself of course would probably be a pretty flexible tree structure. An app full of queries like "Get me all the content of slide Y of deck XXXX... or don't if the current user YYY doesn't have access to that" would make me itching fo…

> Get me all the content of slide Y of deck XXXX

But that's not a query I ever need. I load the user's document, which contains an array of slide (embedded documents). There is no relation between documents & slides, slides are part of the document.

But take it one step further & imagine you have a variety of widgets that can be placed on slides, with no common schema between them. Your choice is model out 100s of tables, use EAV, or go noSQL. We did the latter.

> would make me itching for indices and foreign constraints

Mongo has indexes. Regarding foreign keys it does not have that, but usually you can just embed the document in all the places you need it (de-normalization). Which can be a problem, so if you don't want de-normalization you should definitely use a SQL DB, not mongo.

> Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images"

Mongo has aggregation which has fortunately been able to perform all the reporting I've needed so far. I'm sure postgres is nice as well.

Re: Why Did So Many Startups Choose MongoDB?

#157
post #115

Earlier quoted context omitted.

Right; Facebook, Wikipedia, OpenStreetMap - all serious tech stacks that started and (in the case of the last two) continue to use PHP. Reminds me of: https://slack.engineering/taking-php-seriously-cf7a60065329

Wikipedia launched in 2001. Facebook launched in 2004. The choice was narrower then. Ruby and Python, while already active, did not have nice web and DB frameworks (both RoR and Django started in 2005). Java was also much less mature, and even more wordy. PHP, on the other had, was easy to get from any shared hoster.

There were many good web libraries/frameworks for Python before Django. One even had a transactional NoSQL object database.

Re: Why Did So Many Startups Choose MongoDB?

#158
post #52

Earlier quoted context omitted.

Facebook does not "normal" PHP that much, they use host of languages, including Hack, erlang and lots of Python.

Most of their code is Hack, and Hack is a superset of PHP (last time I checked all it really adds is static typing and async) https://news.ycombinator.com/item?id=14659729

> all it really adds is static typing and async

You make that sound like its not a big deal vs vanilla php.

Re: Why Did So Many Startups Choose MongoDB?

#159

Earlier quoted context omitted.

Have you ever worked on a significant project using MongoDB? I did. I wouldn't use it again. But reliability just wasn't an issue. There was a time when there were insane defaults that made this an issue, but that's a long time ago.

Any idea about Memsql? We are evaluating it at our company and all the info is kind of fuzzy.

MemSQL as in the distributed relational data warehouse product? If so, it's highly recommended, probably the best data warehouse solution you can run if you're not going with something managed (like redshift or bigquery).

Re: Why Did So Many Startups Choose MongoDB?

#160

Earlier quoted context omitted.

Have you ever worked on a significant project using MongoDB? I did. I wouldn't use it again. But reliability just wasn't an issue. There was a time when there were insane defaults that made this an issue, but that's a long time ago.

Any idea about Memsql? We are evaluating it at our company and all the info is kind of fuzzy.

Feel free to reach out to me on our public Slack channel to clear up anything that is fuzzy! I'm an engineer at MemSQL and want to know (1) how I can help and (2) how we can provide non-fuzzy information to the rest of the community.

Channel: chat.memsql.com My UN: eklhad

Post reply on HN