Live data from Hacker News

MongoDB gets support for multi-document ACID transactions

techcrunch.com

191–200 of 245 posts

Re: MongoDB gets support for multi-document ACID transactions

#191

Earlier quoted context omitted.

How does MongoDB handle schema changes? For example, let's say I want to add a mobile phone field to a customer record type. How would I go about doing that in MongoDB?

The short answer is: just do it. You can add any field to any document at any time. That's the beauty of JSON documents without schema constraints. Then of course you need to let your application understand that. But it turns out it's almost trivial to make an application display a phone number field if it finds one, and not display a phone number if there isn't one in the document.

This is true

It hurts when the next requirement comes along something like...

"As a user I want to have a home, work, and mobile phone number"

Now you have 3 "versions" of your implicit "schema" to contend with

1) No phoneNumber 2) phoneNumber and mapping it into / out of one of the three phone numbers in the UI 3) objects with three properties homePhoneNumber, workPhoneNumber, mobilePhoneNumber etc

Then the business comes up with "As a user I want to have arbitrary phone numbers that I can label" now the developers start to squeal

RDBMS + SQL is no panacea but having DDL operations like the following (all probably syntactically invalid but you get the idea) out of the box is incredibly powerful. ALTER TABLE user RENAME COLUMN phone_number TO home_phone_number; ALTER TABLE user ADD COLUMN work_phone VARCHAR(32) NOT NULL; CREATE TABLE phone_number (id BIGINT NOT NULL, user_id BIGINT NOT NULL, name VARCHAR(64) NOT NULL, phone_number VARCHAR(32) NOT NULL);

I have had reasonable success using MongoDB as a store of "things that happened" and will never change

Re: MongoDB gets support for multi-document ACID transactions

#192

Earlier quoted context omitted.

> Having to resort to doing things like map/reduce for a simple group by / order is not the way to go IMO. Perhaps you are unaware, but that is a straw-man. That’s not how you’re supposed to with MongoDB. You’d use the aggregation framework: db.sales.aggregate( [ { $group : { _id : null, totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity" }, count: { $sum: 1 } } } ] )…

Aggregates carry with them a memory limit. We currently use MongoDB 3.4. It's definitely much improved. The replication protocol that came along with 3.4 has been very reassuring. But aggregate queries really do not give you the power of SQL. They are great for transforming stuff for a report, but I would avoid using them for anything else unless 1) it can be cached (and therefor properly invalidated) and 2) doesn't…

>Aggregates carry with them a memory limit.

I think you are referring to the 100MB RAM limit, but that’s not a hard limit, it’s more of a bad default. The `allowDiskUse` option lets MongoDB write intermediate results to the disk (which is exactly what SQL databases are doing).

> But aggregate queries really do not give you the power of SQL. They are great for transforming stuff for a report, but I would avoid using them for anything else unless 1) it can be cached (and therefor properly invalidated) and 2) doesn't need to be "real time"

I really don’t see a difference between what you can do with MongoDB and SQL. I can’t say much more without knowing specifically what impediments you have in mind, but I would certainly like to hear more. For example, why do you cite results caching and lack of real-time requirements?

> Being a document, schema-less database,

I guess if you’re on 3.4 you can’t take advantage of JSON Schema yet, but keep that in mind as a part of your upgrade plans. In the meanwhile you can still use document validation?

> your "quality of life" as you scale with MongoDB is going to be heavily dependent on how you structure your documents and the types of their fields.

This is completely true, but couldn’t we say that just as much about any database? I’d put money on there being way more grief out there over bad tabular schema than over bad document schema. I mean, who’s worked on large-scale systems that hasn’t put off implementing great ideas, or had to hack up app code to compensate for a restrictive schema, because you can’t take the pain of ALTER TABLE?

> Are you treating your collections like SQL tables?

Ouch. Please don't!

> Have any many-to-many relationships in hot code paths? Welcome to hell.

That's probably fair, but if you're in hell to a vastly greater than you would be with Postgres, I'm pretty sure that's a modeling problem. Again, can you tell me more about the particular example?

> Its type system is also limited compared to modern SQL DBs. Storing IP addresses, and want to query them based on a given CIDR range? Postgres makes this eas, MongoDB has you writing code that does sub-queries

That's 100% legit. MongoDB needs to do a ton better with that... types rule.

Re: MongoDB gets support for multi-document ACID transactions

#193
post #141

Earlier quoted context omitted.

The problem with that thinking is that the replication mattered. I'd argue it didn't, it was essentially a scam that people fell for. Who cares about failover when you're losing data due to a bad implementation? Who cares about replication when you can gain the same performance by using a performant database on a single node? Did mongodb truly allow anyone to really horizontally scale? Most places that need massive h…

The thing I don't understand about mongodb is that it makes a tradeoff for scalability. The secret ingredient in the horizontal scaling sauce is giving up inter node ACID transactions. Nothing prevents you from making the same tradeoff with mysql or postgresql.

Indeed, that's how youtube, twitter, and facebook use MySQL, among others.

Re: MongoDB gets support for multi-document ACID transactions

#194
post #6

MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. The RethinkDB retrospective[0] contains a lot of insight into how MongoDB has succeeded despite being vastly inferior on a technical level back when it first launched. I have to admit them a certain respect for executing their strategy so successfully. Choice quote: Every t…

A database which utterly fails the Jepsen test should not be considered for production. It might be good enough for a cache, but trusting it with real data is reckless.

They have since passed the Jepsen test to be fair. But before then I fully agree, why people trusted MongoDB with their critical data is beyond me.

Re: MongoDB gets support for multi-document ACID transactions

#195
post #184

Earlier quoted context omitted.

See this is the sort of crazy things I used to see people do and wonder why they had problems. MongoDB is a document database. You can't just take relational database tables, move them across and expect it to behave the same. And frankly I don't feel sympathy for bad engineering practice. You don't do system migrations without fully testing and understanding all of the systems. But for those of us that had document o…

If you never tracked what happened in production, it may have worked most of the time well enough that you never saw how bad it was. But read https://aphyr.com/posts/284-call-me-maybe-mongodb and https://aphyr.com/posts/284-call-me-maybe-mongodb for an idea of how the promises in MongoDB documentation compared to the reality of the software under stress. And it wasn't just hypothetical either - there are plenty of ho…

The EnterpriseDB benchmark is just a hit piece. See https://newbiedba.wordpress.com/2017/05/26/thoughts-on-postg...

Re: MongoDB gets support for multi-document ACID transactions

#196
post #66

Earlier quoted context omitted.

Honestly, the existence of Mongo is mostly an indictment about how user-hostile conventional RDBMS is. The fact that other DBs can do what Mongo does is not helpful when there is no easy workflow to do what Mongo does. The fact that I theoretically implement a web-based CMS in C and it could be more performant than all these web-language CMS products doesn't mean that C is better for making a CMS.

> "Honestly, the existence of Mongo is mostly an indictment about how user-hostile conventional RDBMS is." That's nonsense. I've taught people to use SQL before, even people with little to no programming experience. After the initial concepts were understood it was fairly easy to gradually expand knowledge over time. The basic SQL keywords to start getting useful information out of a RDBMS are: SELECT FROM WHERE INNE…

That's after you set up users, schemas, tables, columns, oddly-named datatypes with unexpected behaviors, etc.

The RDBMS is a lie. It's a beautiful kernel of relational theory wrapped in a 60-foot ball of hacks, tweaks, duct-tape, bubble-gum, and hate. The document store doesn't lie. It doesn't pretend. It's honest that it's stupid and it's a glorified hashtable with a string in it. It doesn't make any ridiculous pretenses of having a Sufficiently Smart Query Optimizer that will inevitably let you down and leave you pulling your hair out trying to figure out why on earth a simple, straightforward query is running so goddamned slow.

Then you build a complicated model and have to figure out from the query plan why your query is slow and deal with indices and foreign keys and all that nonsense.

Meanwhile an object DB may be inefficent and clumsy, but it gets all of that stuff out of the way. Also, if you don't want to join, you can work around that by duplicating the data all over the place. Something you can't do with an RDBMS because tables are fundamentally flat and so you can't stuff a parent-child relationship into a single table.

Re: MongoDB gets support for multi-document ACID transactions

#197

Earlier quoted context omitted.

The short answer is: just do it. You can add any field to any document at any time. That's the beauty of JSON documents without schema constraints. Then of course you need to let your application understand that. But it turns out it's almost trivial to make an application display a phone number field if it finds one, and not display a phone number if there isn't one in the document.

This is true It hurts when the next requirement comes along something like... "As a user I want to have a home, work, and mobile phone number" Now you have 3 "versions" of your implicit "schema" to contend with 1) No phoneNumber 2) phoneNumber and mapping it into / out of one of the three phone numbers in the UI 3) objects with three properties homePhoneNumber, workPhoneNumber, mobilePhoneNumber etc Then the business…

And I would still claim that this is easier in MongoDB because several versions of the phone number field(s) can happily coexist in the same collection. Those variants are usually trivial to understand for someone who even just looks at the data, and the application can be written to either accept the different formats, or adjust the format on the fly when it encounters a document that still uses an old schema. Or you could indeed write a batch job that bumps all your phone numbers to a new format, and you could put a JSON schema constraint on your collection that enforces the new schema for every future document. All those possibilities exist, and I truly see that as a big advantage.

Re: MongoDB gets support for multi-document ACID transactions

#198
post #184

Earlier quoted context omitted.

If you never tracked what happened in production, it may have worked most of the time well enough that you never saw how bad it was. But read https://aphyr.com/posts/284-call-me-maybe-mongodb and https://aphyr.com/posts/284-call-me-maybe-mongodb for an idea of how the promises in MongoDB documentation compared to the reality of the software under stress. And it wasn't just hypothetical either - there are plenty of ho…

The EnterpriseDB benchmark is just a hit piece. See https://newbiedba.wordpress.com/2017/05/26/thoughts-on-postg...

From your link, go to https://newbiedba.wordpress.com/2017/11/27/thoughts-on-postg... for the followup after he wrote those benchmarks. When he ran his benchmarks, he indeed got better throughput on MongoDB. But the 99% performance was massively worse - in fact slow enough to be unacceptable. To an extent that he concluded that you'd be better off using PostgreSQL.

And he's right. As pages like http://latencytipoftheday.blogspot.com/2014/06/latencytipoft... make clear, we have a lot of calls back to the application happening. Users will notice the occasional slow load surprisingly quickly, and it is worth a lot to get rid of them.

So even your chosen source agrees. A relational database is not orders of magnitude slower. In fact, a relational database is probably a better fit.

Re: MongoDB gets support for multi-document ACID transactions

#199
post #184

Earlier quoted context omitted.

See this is the sort of crazy things I used to see people do and wonder why they had problems. MongoDB is a document database. You can't just take relational database tables, move them across and expect it to behave the same. And frankly I don't feel sympathy for bad engineering practice. You don't do system migrations without fully testing and understanding all of the systems. But for those of us that had document o…

If you never tracked what happened in production, it may have worked most of the time well enough that you never saw how bad it was. But read https://aphyr.com/posts/284-call-me-maybe-mongodb and https://aphyr.com/posts/284-call-me-maybe-mongodb for an idea of how the promises in MongoDB documentation compared to the reality of the software under stress. And it wasn't just hypothetical either - there are plenty of ho…

Again. I have been personally involved the deployment and support of MongoDB clusters for very large datasets at very large companies. It does work if you use it for the right task i.e. highly nestable data not relational data. And let's be clear that if MongoDB was unusable then the company wouldn't still be here as successful as they are.

That EnterpriseDB link is completely ridiculous. Firstly, it predates WiredTiger which replaced the entire storage layer. Secondly, doing one for one comparisons with relational systems doesn't make sense. MongoDB is a document database. Compare it with other document databases.

Re: MongoDB gets support for multi-document ACID transactions

#200
post #13

Has anyone worked with MySQL JSON data types in production? For most projects, I prefer working in SQL via a query builder or ORM for abstraction, but find a few features that would benefit from denormalized JSON storage.

We switched over some non-essential data to json. All of it was for stuff that we don't index, join, or SQL search (elasticsearch still indexes it).

It is always just the display data on details page

Not sure if thats a good use case, but it cleaned up our tables and given the data fields can vary from customer to customer its a nice flexibility. Previously it was an Entity-attribute-value setup for these fields (which was very slow after we got into millions of rows) and we were considering a NoSQL option

Post reply on HN