Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Startup Engineers and Our Mistakes with MongoDB

nemil.com

Post reply on HN
Print view View original

61–70 of 118 posts

Re: Startup Engineers and Our Mistakes with MongoDB

#61

Post by cmenge »

winter_blue wrote: post #15
cmenge wrote: post #3

I'm a long-time MongoDB user and largely a fan of it because I believe the interface is superior to some text-based SQL statements. But whatever DB you like, if you move the state of your application to another application (i.e, a database) you better make sure you really understand how it works. For SQL databases, many people think they know how they work, but misconceptions seem widespread. Essentially, many beginn…

Couple of points: 1. ORMs I'm not sure why you dismiss ORMs. Have you ever used a good ORM? There are a lot of high-quality ORMs out there, that do the heavy lifting, provide type safety, and take care of the boilerplate. There is no reason to write text-based SQL anymore. 2. Relations: Foreign keys are incredibly useful. They're the biggest feature I miss out in relational databases. They help keep your data in a sa…

Well

1.) It's always a text-based interface. The ORM merely hides the fact. Type safety is weak, as is with JSON, but both aren't un-typed.

2) Yes, they are, and there are a lot of good use cases for relational databases. But if you have a number of services that share nothing (especially not the database), what is the foreign key good for?

3) You can do any style with any database. Overly aggressive normalization in read-heavy environments can be bad, as is de-normalization in write-heavy environments. Choose the tool for the job..

4) Why all over the code? Can't you make it easy to find the model classes? Doesn't seem so hard. Also, is this so different from, say, EF code first? And anyway, doesn't every SQL database in fact, have multiple schemas (the one in the db, the one in the scripts, the one in the ORM and the one in the code?) Is that really so different?

Re: Startup Engineers and Our Mistakes with MongoDB

#62

Post by nemild »

eip wrote: post #59
nemild wrote: post #2

(I’m the author of this series) Eliot Horowitz (HN: @ehwizard), MongoDB’s current and founding CTO, reached out after my last post - and spent two hours providing feedback last week in Palo Alto. It was an expansive discussion, and Eliot was reflective and eager to understand the perspectives I had heard. He noted how much it mattered to him what HN thought. I left with tremendous empathy for the challenges of buildi…

> In that sense, he argues that Amazon RDS is a form of NoSQL today Lol. What? RDS is literally hosted RDBMS SQL databases.

I had a similar reaction.

Re: Startup Engineers and Our Mistakes with MongoDB

#63

Post by scarface74 »

MikeKusold wrote: post #7

> By far the most consistent mistake was choosing a non-relational database, when your data was strongly relational. Mongoose's ODM made this mistake surprisingly easy to make, which led to issues down the line. This mirrors my experiences with Mongo as well. The vast majority of data is relational. Mongoose allows people to make the mistake of structuring their data relationally. But all you are doing is pushing all…

But all you are doing is pushing all the joins to the web server (which in Node land is single threaded and compounds the mistake of choosing NoSQL).

Why is pushing the joins "to the web" inherently bad? It's a lot easier to scale web servers than database servers.

Re: Startup Engineers and Our Mistakes with MongoDB

#64

Post by tinix »

winter_blue wrote: post #15
cmenge wrote: post #3

I'm a long-time MongoDB user and largely a fan of it because I believe the interface is superior to some text-based SQL statements. But whatever DB you like, if you move the state of your application to another application (i.e, a database) you better make sure you really understand how it works. For SQL databases, many people think they know how they work, but misconceptions seem widespread. Essentially, many beginn…

Couple of points: 1. ORMs I'm not sure why you dismiss ORMs. Have you ever used a good ORM? There are a lot of high-quality ORMs out there, that do the heavy lifting, provide type safety, and take care of the boilerplate. There is no reason to write text-based SQL anymore. 2. Relations: Foreign keys are incredibly useful. They're the biggest feature I miss out in relational databases. They help keep your data in a sa…

Most of your points are ripe with folly.

1. ORMs are slow, but generally they are worth it, and it's trivial to dive into raw queries when needed for performance reasons.

2. Mongo supports relationships, not sure what you're trying to get at. Reverse delete rules are TRIVIAL to implement in your models. This is NOT the fault of the database engine, but fault of the developer, 100%.

3/4. Lack of normalization and schema is only a problem if you let it be. This is why model layers exist! If you are having to look "all over the code" to figure out the models, then the problem is you don't have any models.

This problem isn't even limited to Mongo either, this can happen on a SQL-backed project too if the developer is an amateur. Not having a model layer is the fault of the developer (or architect), not to be blamed on the DB engine of choice.

Re: Startup Engineers and Our Mistakes with MongoDB

#65

Post by scarface74 »

adjkant wrote: post #8

The reality of MongoDB is that it's a very specific use case of nonrelational data, which is very rare these days. I'm sad that so many people get looped into Mongo with a MEAN/MERN stack when those apps are almost always the CRUD apps that would benefit from SQL. Why force yourself to maintain a schema implicitly for relational data when you can get error messaging and explicit schemas? I think MongoDB is a great fi…

"I've been coding for 4 years and have yet to find a problem that fit Mongo best."

I'm going to leave that right there.....

Re: Startup Engineers and Our Mistakes with MongoDB

#66

Post by scarface74 »

winter_blue wrote: post #15
cmenge wrote: post #3

I'm a long-time MongoDB user and largely a fan of it because I believe the interface is superior to some text-based SQL statements. But whatever DB you like, if you move the state of your application to another application (i.e, a database) you better make sure you really understand how it works. For SQL databases, many people think they know how they work, but misconceptions seem widespread. Essentially, many beginn…

Couple of points: 1. ORMs I'm not sure why you dismiss ORMs. Have you ever used a good ORM? There are a lot of high-quality ORMs out there, that do the heavy lifting, provide type safety, and take care of the boilerplate. There is no reason to write text-based SQL anymore. 2. Relations: Foreign keys are incredibly useful. They're the biggest feature I miss out in relational databases. They help keep your data in a sa…

Couple of points: 1. ORMs I'm not sure why you dismiss ORMs. Have you ever used a good ORM? There are a lot of high-quality ORMs out there, that do the heavy lifting, provide type safety, and take care of the boilerplate. There is no reason to write text-based SQL anymore.

I use ORMs to do just what the name implies -- to map relational models to objects. If I'm working with objects anyway, either I'm going to end up using an ORM or creating my own object mappings.

2. Relations: Foreign keys are incredibly useful. They're the biggest feature I miss out in relational databases. They help keep your data in a sane state. If a user deletes his account, you can configure your database use a DELETE CASCADE to automatically remove all data associated with the user.

Or I can just as easily use my User management micro-service to enforce business rules.....

3. Normalization: NoSQL databases encourage you to denormalize your data. I've seen NoSQL databses frequently run into problems with stale data, with date that is duplicated and not kept updated, where you have multiple out-of-sync version of the same piece of information. With NoSQL database, you have to handle all the complications of denormalization. (You, the developer, have to remember to update/delete/etc from the multiple places the same piece of data lives in.) The database doesn't do it for you. (The database is a dumb key-value store, nothing more.) Relational databases encourage keeping the logical design of your databse normalized. To quote Wikipedia: "The preferred method is to keep the logical design normalised, but allow the database management system (DBMS) to store additional redundant information on disk to optimise query response. In this case it is the DBMS software's responsibility to ensure that any redundant copies are kept consistent. This method is often implemented in SQL as indexed views (Microsoft SQL Server) or materialised views (Oracle).".

Why would you have your business logic strewn all over your code base instead of using a common library/microservice that all of your code depends on? I wouldn't design a system even with Sql where all the code modifies the database willy-nilly.

4. Schemas: The worst thing about NoSQL is the absence of an enforced schema. Schemaless databases are a scourge. There's always a schema -- it's just that it's scattered all over the code. If you are joining a new company, you have to sift through piles of code to figure what the structure of the data is. Schemas are like types, and my dislike for dynamically typed languages carries over to schema-less database. Relational databases make you think carefully about the schema, and specify the schema explicitly.

Why is your schema "scattered all across your code? I use Mongo with C#. When I'm reading from writing to a collection, I'm not reading/writing BsonDocuments.

A typical code snippet from C# using RoboMongo is:

var collection = database.GetCollection("Users").AsQueryable();

All of my Linq queries, inserts, updates, etc. are strongly typed objects with autocomplete and type safety.

The "User" object is defined in one central place.

In the end, you end up needing to do a lot of extra work, likely end up with more unstable and buggy system, just to avoid the small amount of totally-worth-it upfront work that setting up a relational database requires.

"It's a poor carpenter that blames his tools"

Re: Startup Engineers and Our Mistakes with MongoDB

#67

Post by scarface74 »

cmenge wrote: post #61
winter_blue wrote: post #15

Earlier quoted context omitted.

Couple of points: 1. ORMs I'm not sure why you dismiss ORMs. Have you ever used a good ORM? There are a lot of high-quality ORMs out there, that do the heavy lifting, provide type safety, and take care of the boilerplate. There is no reason to write text-based SQL anymore. 2. Relations: Foreign keys are incredibly useful. They're the biggest feature I miss out in relational databases. They help keep your data in a sa…

Well 1.) It's always a text-based interface. The ORM merely hides the fact. Type safety is weak, as is with JSON, but both aren't un-typed. 2) Yes, they are, and there are a lot of good use cases for relational databases. But if you have a number of services that share nothing (especially not the database), what is the foreign key good for? 3) You can do any style with any database. Overly aggressive normalization in…

* Type safety is weak, as is with JSON, but both aren't un-typed."

How is type safety "weak" with an ORM? You are using strongly typed objects.

Re: Startup Engineers and Our Mistakes with MongoDB

#68

Post by bdcravens »

scarface74 wrote: post #63
MikeKusold wrote: post #7

> By far the most consistent mistake was choosing a non-relational database, when your data was strongly relational. Mongoose's ODM made this mistake surprisingly easy to make, which led to issues down the line. This mirrors my experiences with Mongo as well. The vast majority of data is relational. Mongoose allows people to make the mistake of structuring their data relationally. But all you are doing is pushing all…

But all you are doing is pushing all the joins to the web server (which in Node land is single threaded and compounds the mistake of choosing NoSQL). Why is pushing the joins "to the web" inherently bad? It's a lot easier to scale web servers than database servers.

Joins are typically a tiny intersection of 2 data sets. Multiply that by the number of web requests. Moreover, a query planner is far more optimized than the code you'd write on your web server.

Re: Startup Engineers and Our Mistakes with MongoDB

#69

Post by bdcravens »

threeseed wrote: post #54
flavio81 wrote: post #27

Earlier quoted context omitted.

> If data needs to be processed, an option would be to use MongoDB for collecting the data in bulk and later decide how you need to structure it for your needs. OR, i can open a file stream, serialize my data to JSON, entity by entity, and dump all to a file. The good old file.

Because we want to do partial updates, searches, indexing etc. Your position can be applied to all databases. Why not abandon them all and just use CSV ?

You could always do that, dump the data to s3, and use Athena :-)

https://aws.amazon.com/athena/

Re: Startup Engineers and Our Mistakes with MongoDB

#70

Post by bdcravens »

lowbloodsugar wrote: post #58
mrtksn wrote: post #30

Earlier quoted context omitted.

You definitely can but MongoDB provides convenience over storing and managing JSON files on you filesystem by your own efforts.

No, I really can't think of any situation where installing, managing, updating, maintaining, and crying over a mongodb cluster would be easier for a flat-file datastore than say, a file system, or S3. In fact if one is on AWS, then nothing beats just dumping them in S3 and processing them as necessary. If I'm a startup and I really wanted some querying then PostgreSQL looks great, with Amazon handling the maint and b…

You can even query flatfiles in s3 using SQL via AWS Athena.
Post reply on HN