Live data from Hacker News

Startup Engineers and Our Mistakes with MongoDB

nemil.com

101–110 of 118 posts

Re: Startup Engineers and Our Mistakes with MongoDB

#101
post #75

Earlier quoted context omitted.

If you're going to get on the "MongoDB was overhyped" bandwagon, you should probably not post links to performance comparisons you pulled off of an EDB Postgres website.

At least EDB used an inspectable and reproducable test. I think posting links to performance comparisons is fine when the performance comparison can be independently verified.

I suggest then that you get the latest versions of MongoDB and PostgreSQL and run that test and report back. Verifiable isn't the same as verified.

Re: Startup Engineers and Our Mistakes with MongoDB

#102
post #79

Earlier quoted context omitted.

Joins are typically a tiny intersection of 2 data sets. Multiply that by the number of web requests. Moreover, a query planner is far more optimized than the code you'd write on your web server.

also, scaling databases is not terribly difficult, it is just very "clumsy". Still, having a proper transaction manager and query planner will give you far better performance compared to doing it on webservers. Also, aren't bsically moving the complexity of the database to the webserver? Which makes your webserver far more complex and harder to scale in comparison. Especially if you need ACID. (which is a must if you…

You don't always need ACID. Sometimes eventual consistency is good enough. How would it make the webserver itself more complex? The application itself should just be able to run on n servers.

Re: Startup Engineers and Our Mistakes with MongoDB

#103

Earlier quoted context omitted.

Simplified, a JOIN on an SQL server will take advantage of its indexes to grab the data from the disk that matches the specified criteria. SQL databases are built for this sort of thing, and they're really freaking good at it. Sending that same JOIN to the web server means, at the least, shipping everything from both tables matching the criteria and matching up those two potentially huge blobs of data with each other…

If you are doing a lot of joins in Mongo - you're doing it wrong. The whole purpose of a document DB is storing your model with the related data as one document.

[deleted]

Re: Startup Engineers and Our Mistakes with MongoDB

#104

Earlier quoted context omitted.

Simplified, a JOIN on an SQL server will take advantage of its indexes to grab the data from the disk that matches the specified criteria. SQL databases are built for this sort of thing, and they're really freaking good at it. Sending that same JOIN to the web server means, at the least, shipping everything from both tables matching the criteria and matching up those two potentially huge blobs of data with each other…

If you are doing a lot of joins in Mongo - you're doing it wrong. The whole purpose of a document DB is storing your model with the related data as one document.

If we dump everything we might JOIN on to a single document then in many cases we're going to end up with a ridiculously nested document, and/or duplicate data everywhere. Let's try mapping out the project I'm working on right now.

We've got moves, and each move has multiple stops:

  move: {
    shipper: string,
    consignee: string,
    stops: [{
      address: string,
      appt: datetime,
    }] 
  }
So far this looks great. I like this. Yay MongoDB!

But I need to add a driver to each stop... Where do I put the driver? If I put him under the stop I'm going to duplicate his data all over the place. Do I put the move under the driver instead?

  driver: {
    name: string,
    id: string,
    moves: [{
      shipper: string,
      consignee: string,
      stops: [{
        address: string,
        appt: datetime,
    }]
  }
Ok, not bad. We've got an array of objects under an array of objects but I'm still perfectly comfortable.

Now I need to get a terminal-based look at all the drivers, so I know what every driver is doing in a particular city, also drivers can actually have multiple trucks too:

  terminal: {
    city: string,
    drivers: [{
      name: string,
      id: string,
      truck: [{
        plate: string,
        moves: [{
          shipper: string,
          consignee: string,
          stops: [{
            address: string,
            appt: datetime,
          }]
        }]
      }]
    }]
  }
That's getting a touch nested but... Drivers actually work for vendors, and vendors might work across multiple terminals. Do we duplicate the vendor across terminals, or the terminal across vendors? Also even the driver/truck relationship is many-to-many... Do we duplicate the truck info? What happens when we need to update a truck? Do we attempt to update across all drivers? Do we duplicate that data too?

  terminal: {
    city: string,
    vendor: [{ // many-to-many! duplicate vendor data?
      id: string,
      drivers: [{
        name: string,
        id: string,
        truck: [{ // many-to-many! duplicate truck data?
          plate: string,
          moves: [{
            shipper: string,
            consignee: string,
            stops: [{
              address: string,
              appt: datetime,
            }]
          }]
        }]
      }]
    }]
  }
Ok, this schema is starting to look like shit and it's only going to get worse. The shipper and consignee should actually be unique objects as well, and we might want to update the contact info on them. Do we duplicate that data and update across all moves?

Also we want to cross-reference the shipper, consignee, billing, and ordering entities for market analysis (the consignee isn't always the customer, and the customer doesn't even always get the bill!) Do we duplicate that information across moves? Do we duplicate moves across entities? More many-to-many relationships...

Maybe I'm just not creative enough to use MongoDB, but I'm really starting to feel like this is a square peg in a round hole.

Re: Startup Engineers and Our Mistakes with MongoDB

#105

Earlier quoted context omitted.

Couple of points: 1. ORMs I'm not sure why you dismiss ORMs. Have you ever used a good ORM? There are a lot of high-quality ORMs out there, that do the heavy lifting, provide type safety, and take care of the boilerplate. There is no reason to write text-based SQL anymore. 2. Relations: Foreign keys are incredibly useful. They're the biggest feature I miss out in relational databases. They help keep your data in a sa…

As someone who thinks both systems have their place, there are a few issues in your arguments: 1. ...There is no reason to write text-based SQL anymore. There sure is! Unless your schema is dead simple your going to run into places where hand tuning is a requirement. Lots of ORM's are third rate at best, people swear by them but don't have complex data that would make one throw fits. 3. Normalization: NoSQL databases…

> Denormalization when done right should NOT result in duplication

Isn't that what it is by definition? Can you give an example of how would you denormalize without causing some sort of data duplication?

Re: Startup Engineers and Our Mistakes with MongoDB

#106
post #87

Earlier quoted context omitted.

That's not the same thing at all; you're suggesting an approach that probably takes more code and definitely takes more setup.

You are underestimating the complexities of file management

Again, depends on scope and scale. If you have a handful of items to process it's not a big deal.

Re: Startup Engineers and Our Mistakes with MongoDB

#107
'One 10gen engineer made this point in analogizing SQL to Cobol, arguing that "SQL is Annoying":'

Why wasn't this the tagline for Mongo to begin with? At least it shows the arrogance, ignorance, and outright stupidity behind the database. This is not an engineer's comment. It cannot possibly be now, decades after SQL has become a de facto standard for some very good reasons (outlined in the article and elsewhere, won't rehash here). It is a marketer's comment, probably spoken by an engineer who isn't qualified to build simple demo apps, let alone anything as complex as a database. Mongo's CTO seems to fit this bill of marketer more than anything else or how would he be able to make the claims he makes with a straight face?

Our industry has a problem with fads and decisions based on feelings. Even if SQL is annoying, it's incredibly stupid to base your tech choices on feelings. It's even dumber to choose a product whose engineers build the product based on feelings. Last I checked, I thought we were an industry of engineers, trying to apply scientific principles and some human ingenuity to build software. Where does 'annoying' fit into that? Or other adjectives I often see thrown around on these boards like 'clean' and 'slick'. I know it's hard to quantify the quality of software and speak about it with any coherence, but this is beyond incoherent. 'Annoying' as applied to SQL tells me nothing. 'Annoying' as applied to the engineer making this incredibly stupid comment tells me that this engineer is either lazy, gullible, or just downright stupid. Have we seriously lost our ability to discern hype from reality that we let people and companies like this dictate our technology choices, throwing out decades of solid research in computer science for what some idiot at a company more specialized in marketing and PR than engineering tells us?

I'm willing and ready to hear well-thought out criticism of SQL and RDBMS. What I'm not willing and ready to hear is some idiot's feelings about SQL or RDBMS. You want to make a claim? Measure it. Present a report with data. But goddamn, if one of my reports came to me with this kind of stupidity, I'd give them a chance, but if they persisted, I'd fire them. This isn't revolutionary thought. It's not evolutionary thought. It's lazy idiots who don't want to learn SQL, a language so easy even business people with essentially no computer skills can pick up.

Re: Startup Engineers and Our Mistakes with MongoDB

#108

Earlier quoted context omitted.

If you are doing a lot of joins in Mongo - you're doing it wrong. The whole purpose of a document DB is storing your model with the related data as one document.

If we dump everything we might JOIN on to a single document then in many cases we're going to end up with a ridiculously nested document, and/or duplicate data everywhere. Let's try mapping out the project I'm working on right now. We've got moves, and each move has multiple stops: move: { shipper: string, consignee: string, stops: [{ address: string, appt: datetime, }] } So far this looks great. I like this. Yay Mon…

This isn't rocket scientist and people far smarter than us have already figured this stuff out.

https://martinfowler.com/bliki/DDD_Aggregate.html

If you look at this from a Domain Driven Design standpoint, you should model your business and think about your aggregate roots and each collection should be its own aggregate root, data access should be handled by one class/microservice for each aggregate root.

If you modeled your Domain correctly that wouldn't be an issue.

On the other hand, not modeling your domain correctly first would lead to an ungodly, untestable, tangle of stored procedures and 10 way joins with A relational database.

The Mongo docs goes into best practices which mirrors basically the concept of an aggregate root.

https://www.mongodb.com/blog/post/thinking-documents-part-2

Re: Startup Engineers and Our Mistakes with MongoDB

#109
post #25
post #4

Today in 2017, there is no real reason to use MongoDB other than in prototyping. I am happily waiting until the final nail is put on the coffin of this overhyped, flawed document store.

Earlier in 2017, Atlassian acquired Trello (MongoDB, Node.js, Redis tech stack) for 425 Million dollars

Funnily enough, the one feature I need to completely replace jira with trello is the ability to have the same card on multiple boards, but I guess they couldn't handle that kind of relational data.

Re: Startup Engineers and Our Mistakes with MongoDB

#110

Earlier quoted context omitted.

If we dump everything we might JOIN on to a single document then in many cases we're going to end up with a ridiculously nested document, and/or duplicate data everywhere. Let's try mapping out the project I'm working on right now. We've got moves, and each move has multiple stops: move: { shipper: string, consignee: string, stops: [{ address: string, appt: datetime, }] } So far this looks great. I like this. Yay Mon…

This isn't rocket scientist and people far smarter than us have already figured this stuff out. https://martinfowler.com/bliki/DDD_Aggregate.html If you look at this from a Domain Driven Design standpoint, you should model your business and think about your aggregate roots and each collection should be its own aggregate root, data access should be handled by one class/microservice for each aggregate root. If you mode…

The MongoDB document linked says "Referencing should be used to represent complex many-to-many relationships" ... "References are usually implemented by saving the _id field of one document in the related document as a reference. A second query is then executed by the application to return the referenced data."

So... It's like a join, but with two queries instead of one, and you push the join data to the webserver, which brings us back to the start of this comment thread...

Post reply on HN