Live data from Hacker News

Startup Engineers and Our Mistakes with MongoDB

nemil.com

31–40 of 118 posts

Re: Startup Engineers and Our Mistakes with MongoDB

#31
post #25
post #4

Today in 2017, there is no real reason to use MongoDB other than in prototyping. I am happily waiting until the final nail is put on the coffin of this overhyped, flawed document store.

Earlier in 2017, Atlassian acquired Trello (MongoDB, Node.js, Redis tech stack) for 425 Million dollars

... So? Trello is a great product, and it will continue to be a great product no matter where it chooses to store its data.

Re: Startup Engineers and Our Mistakes with MongoDB

#32
post #20

Earlier quoted context omitted.

If data needs to be processed, an option would be to use MongoDB for collecting the data in bulk and later decide how you need to structure it for your needs. When it comes to "read" the data, you read it only from processed database that can be anything. Since you can do your processing completely independently of your web server, your are not necessarily pushing your computational load from the DB to the web server…

> If data needs to be processed, an option would be to use MongoDB for collecting the data in bulk and later decide how you need to structure it for your needs. OR, i can open a file stream, serialize my data to JSON, entity by entity, and dump all to a file. The good old file.

replied to the same question here: https://news.ycombinator.com/item?id=14805812

Re: Startup Engineers and Our Mistakes with MongoDB

#33
post #3

I'm a long-time MongoDB user and largely a fan of it because I believe the interface is superior to some text-based SQL statements. But whatever DB you like, if you move the state of your application to another application (i.e, a database) you better make sure you really understand how it works. For SQL databases, many people think they know how they work, but misconceptions seem widespread. Essentially, many beginn…

> Some databases might be easier to understand than others, but I feel MongoDB is on the 'easier' end here. YMMV. I agree with most of this, relational databases have a lot of moving parts that many developers (myself included) don't fully understand. However, I believe relational databases (Postgres is what I have experience with) has defaults that are basically correct, and unlikely to cause significant issues, whe…

> I believe relational databases (Postgres is what I have experience with) has defaults that are basically correct, and unlikely to cause significant issues, whereas in my experience MongoDB did not have this.

Strongly, strongly agreed. Learning every moving part in InnoDB (as an example) is quite a task. But not knowing about those features will rarely burn you, and it's usually apparent when you need to know something you don't.

Meanwhile Mongo... well, Mongo says a replicated write is complete as soon as it's queued to be written. They eventually updated the default settings to be slightly less catastrophic, but still far from good. What you don't know about Mongo can and will hurt you, which arguably makes the appearance of ease a drawback.

http://hackingdistributed.com/2013/01/29/mongo-ft/

Re: Startup Engineers and Our Mistakes with MongoDB

#34
post #20

> By far the most consistent mistake was choosing a non-relational database, when your data was strongly relational. Mongoose's ODM made this mistake surprisingly easy to make, which led to issues down the line. This mirrors my experiences with Mongo as well. The vast majority of data is relational. Mongoose allows people to make the mistake of structuring their data relationally. But all you are doing is pushing all…

If data needs to be processed, an option would be to use MongoDB for collecting the data in bulk and later decide how you need to structure it for your needs. When it comes to "read" the data, you read it only from processed database that can be anything. Since you can do your processing completely independently of your web server, your are not necessarily pushing your computational load from the DB to the web server…

Then again, if you're going to push into Postgres you might as well store the temp unstructured records there too

Re: Startup Engineers and Our Mistakes with MongoDB

#35
post #22

Earlier quoted context omitted.

>Do you have users? Do those users need to log in? Well, you now have access tokens related to users. Aren't those tokens supposed to be stored client side anyway? Aren't those tokens supposed to contain encoded information about the user that you decode server side? Why would you store the users token to begin with?

That's a matter of opinion and implementation. Not everyone uses JWTs. What if you want to invalidate all logged-in tokens for a user? You either need to store a blacklist (that's a relation) or set extremely short expiry times.

i think he is discussing standard cookie session ids here, not access tokens. in many web applications, all the information you need is stored within that token and signed by a server-side secret.

Re: Startup Engineers and Our Mistakes with MongoDB

#36
post #3

I'm a long-time MongoDB user and largely a fan of it because I believe the interface is superior to some text-based SQL statements. But whatever DB you like, if you move the state of your application to another application (i.e, a database) you better make sure you really understand how it works. For SQL databases, many people think they know how they work, but misconceptions seem widespread. Essentially, many beginn…

Couple of points: 1. ORMs I'm not sure why you dismiss ORMs. Have you ever used a good ORM? There are a lot of high-quality ORMs out there, that do the heavy lifting, provide type safety, and take care of the boilerplate. There is no reason to write text-based SQL anymore. 2. Relations: Foreign keys are incredibly useful. They're the biggest feature I miss out in relational databases. They help keep your data in a sa…

As someone who thinks both systems have their place, there are a few issues in your arguments:

1. ...There is no reason to write text-based SQL anymore.

There sure is! Unless your schema is dead simple your going to run into places where hand tuning is a requirement. Lots of ORM's are third rate at best, people swear by them but don't have complex data that would make one throw fits.

3. Normalization: NoSQL databases encourage you to denormalize your data. I've seen NoSQL databses frequently run into problems with stale data, with date that is duplicated and not kept updated, where you have multiple out-of-sync version of the same piece of information.

Nosql doesn't encourage you to de-normalize, and there are reasons to de-normalize in an RDBMS. Denormalization when done right should NOT result in duplication, if it does your doing it wrong. Both systems are afflicted by these poor choices. It is easier to screw it up in a document store like mongo.

4. Schemas: The worst thing about NoSQL is the absence of an enforced schema... If you are joining a new company, you have to sift through piles of code to figure what the structure of the data is...

Again were not talking about a problem that is exclusive to document stores. You can just as easily have queries all over the codebase that make changing your RDBMS hard. People also do very stupid things like throwing giant json blobs (and before that xml) into their RDBMS and expect it to work. However a schema makes dealing with data apart from your application a much simpler task, unless you throw all those rules out the window, and a shocking number of people do.

Re: Startup Engineers and Our Mistakes with MongoDB

#37
This brings back some memories from 2012. I was updating an internal tool at the company where I worked, and after a lot of thought I decided to replace a tried and true MySQL database with Mongo. It really seemed like a good idea at the time - we were storing loosely related documents where not having a fixed schema was an advantage.

Any advantage I got was lost by the time I built all the logic to handle the joins I did need to make (those join keys weren't so schemaless after all). And then I needed logic to make sure every record had the right keys, that they made sense, etc. This was all stuff that MySQL had been doing for me.

But the real kicker was that Mongo made my performance worse. When I migrated my database (roughly 10 million records), I compared before and after sizes of the actual files. Mongo's were 2-3x the size. I didn't realize it before starting, but Mongo's preallocation model gave me huge files with sparsely written records that had room for future updates. But I didn't need that extra room because my data rarely changed. So I ended up with larger files that took longer to scan for unindexed queries (mostly for reporting), which meant I had to index more stuff, which increased my memory usage, and forced me to upgrade to larger servers.

Had I stuck with MySQL, I would have been fine. Many expensive lessons were learned from this, which I guess was valuable.

Re: Startup Engineers and Our Mistakes with MongoDB

#38

It terrifies me to see this quote from their CTO: "MongoDB's CTO disagrees with this statement arguing that nearly 90% of database installations today would benefit from being replaced with MongoDB." I used to attend "office hours" at MongoDB's office where guests ask MongoDB employees for help. Most of my questions involved very complex aggregation queries (that would have been trivial in SQL) that even MongoDB empl…

MongoDB is web-scale.

Re: Startup Engineers and Our Mistakes with MongoDB

#39
post #17
post #6

I feel the best thing to come out of MongoDB is that Postgres now handles JSON.

Yes, this was an incredible development, and lead to hilarious projects like ToroDB (MongoDB API on top of Postgres), which ended up being faster and safer, with relations when you need them. JSON columns can be incredibly useful and easy to understand for things like user preferences, without the hassle of a metatable.

Call this hilarious if you want, I think it's rather cool, thanks for the pointer. Once this is out of beta it will definitely be useful, if not already.

Re: Startup Engineers and Our Mistakes with MongoDB

#40
post #34
post #20

Earlier quoted context omitted.

If data needs to be processed, an option would be to use MongoDB for collecting the data in bulk and later decide how you need to structure it for your needs. When it comes to "read" the data, you read it only from processed database that can be anything. Since you can do your processing completely independently of your web server, your are not necessarily pushing your computational load from the DB to the web server…

Then again, if you're going to push into Postgres you might as well store the temp unstructured records there too

Well, that's also an option :) But I believe this is a relatively new option.

Personally, I never got into the hype with the NoSQL and for me the only use is as a convenient API for storing JSON files during prototyping so that I can later decide how my data should be structured.

Definitely tech debt for the future but fast iterated design that needs the database stuff fixed has more chance for success than the rigid design with the perfect data architecture.

Post reply on HN