Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB gets support for multi-document ACID transactions

techcrunch.com

Post reply on HN
Print view View original

161–170 of 245 posts

Re: MongoDB gets support for multi-document ACID transactions

#161

Post by philipkglass »

SCdF wrote: post #11
_wc0m wrote: post #6

MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. The RethinkDB retrospective[0] contains a lot of insight into how MongoDB has succeeded despite being vastly inferior on a technical level back when it first launched. I have to admit them a certain respect for executing their strategy so successfully. Choice quote: Every t…

> MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. I have no idea how capable MongoDB is these days, as I haven't used Mongo in years (and even then it was not for long). However, I do not know any developers who, after living through the "hype first, features later" strategy, have been left with a positive enough opinion…

I used it quite happily circa 2009 and at a different company in 2014. In both cases it was being added to systems that already had mature functionality built atop a RDBMS. In the first case it was used to store events that had started to overwhelm the main RDBMS with write volume. (Originally a system with one database as the monolithic data store.) Probably Kafka would have been even better for this use case, had Kafka been available at the time. But MongoDB did the job very well. I did a prototype in Cassandra too before settling on MongoDB, but MongoDB had much better docs, drivers, and single-node read performance at the time.

The second time I used MongoDB to automatically track templated email bodies that were being delivered through a third party mail platform. We had dozens of recurring templates and many more one-off templates for different curated campaigns. If somebody complained that a link or image or token was wrong in their email, we wanted to be able to look back at the history to see if the problem was in the template data or potentially a client issue on their side. Most of the queries were ad-hoc and not very performance-sensitive. This was where a flexible JSON document format came in handy. Modern Postgres would have worked well for it too, but that wasn't available in the company at the time. With MongoDB I got good flexibility, adequate speed, and I avoided reinventing wheels by not trying to shoehorn the data into another MySQL table. I was able to solve a customer support pain point in less than a week and the system has worked well for nearly 4 years now.

I'd be really frustrated if I had to use MongoDB as my only data store. I would guess that much of the hate for it comes from people who were forced into that position, or maybe from people who didn't take its documented limitations seriously enough before productionizing its use.

Re: MongoDB gets support for multi-document ACID transactions

#162

Post by ZenoArrow »

jasondc wrote: post #53
majidazimi wrote: post #49

Call me when they added Triggers, Stored Procedures, Common Table Expressions, Window Functions,...

Triggers?? Where you add some application logic to the database, 10 years go by, and no one has any idea how the triggers work? Or even how to test them? I've never seen triggers used successfully in any production application (maybe they work at first, but give them time, and a few code changes).

> "I've never seen triggers used successfully in any production application"

Triggers have one or two good use cases, and plenty of bad use cases.

I would suggest the strongest use case is for data validation. Databases don't have very sophisticated type systems, and custom data types can cause headaches. Database triggers allow you to ensure that the data stored in fields matches a set of criteria. To give a basic example, if you had a customer table with an email address field, you could validate that the email address had an @ symbol using a code in an insert and/or update trigger. Of course, you should have similar checks within the software that sits on top of a database, but by putting the validation at a low level by using database triggers you can be more confident that the data integrity will be protected.

As a second use case, if the triggers are being used to maintain data for audit or reporting purposes, that can be fine.

However, aside from the audit/reporting use case, I would recommend avoiding using triggers which span multiple tables (unless there are some exceptional circumstances where it's the best option). Things get messy when you have a chain of tables, each with their own triggers that can update other tables within that chain. If you spot lots of code like that, run to the hills!

Re: MongoDB gets support for multi-document ACID transactions

#163

Post by e12e »

da_chicken wrote: post #124
e12e wrote: post #59

Earlier quoted context omitted.

I love deriding mongodb as much as the next dev that hasn't used it much; but I'll just note that while I'd still be hard pressed to prefer mysql over postgres - there was a long period where mysql was put to tasks it was ill suited for, especially prior to around version 4.x. So while "hype first" might reap a deservedly abundant and bitter harvest of developer hatred - it doesn't preclude evolving into a genuinely…

Both true, although I can completely understand why devs went with MySQL over PostgreSQL at that time. I remember that during the same time period that MySQL was drawing seemingly endless criticism for generally poor RDBMS behavior (3.x and 4.x), PostgreSQL was notorious for having poor performance due to insanely undersized default settings. Like out of the box it was sized to run with at most 10 MB of RAM or simila…

At the time oob postgres probably wasn't the (wrong) competition, sybase, Ms sql, Oracle was...

Or maybe, probably managed postgres. Esp. In the mysql 3.x days and earlier.

Re: MongoDB gets support for multi-document ACID transactions

#164

Post by SamReidHughes »

frankpf wrote: post #158
SamReidHughes wrote: post #130

Earlier quoted context omitted.

It's probably better to go with Cockroach or TiDB. RethinkDB has problems of its own.

Can you clarify on those problems?

Performance, mostly. Too much CPU and disk usage. Change feeds don't scale well.

Re: MongoDB gets support for multi-document ACID transactions

#165

Post by spamizbad »

metheus wrote: post #118
jaequery wrote: post #93

Mongodb can be quite a nightmare once you start requiring anything more than a 1:1 relationship, which is pretty much any kind of app that is doing anything meaningful. Having to resort to doing things like map/reduce for a simple group by / order is not the way to go IMO. I think you later truly realize the beauty of SQL once you get far down that rabbit hole. Initially I rode on the Mongo's NoSQL bandwagon when I s…

> Having to resort to doing things like map/reduce for a simple group by / order is not the way to go IMO. Perhaps you are unaware, but that is a straw-man. That’s not how you’re supposed to with MongoDB. You’d use the aggregation framework: db.sales.aggregate( [ { $group : { _id : null, totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity" }, count: { $sum: 1 } } } ] )…

Aggregates carry with them a memory limit.

We currently use MongoDB 3.4. It's definitely much improved. The replication protocol that came along with 3.4 has been very reassuring. But aggregate queries really do not give you the power of SQL. They are great for transforming stuff for a report, but I would avoid using them for anything else unless 1) it can be cached (and therefor properly invalidated) and 2) doesn't need to be "real time"

Being a document, schema-less database, your "quality of life" as you scale with MongoDB is going to be heavily dependent on how you structure your documents and the types of their fields. Are you treating your collections like SQL tables? Have any many-to-many relationships in hot code paths? Welcome to hell. Its type system is also limited compared to modern SQL DBs. Storing IP addresses, and want to query them based on a given CIDR range? Postgres makes this eas, MongoDB has you writing code that does sub-queries.

Re: MongoDB gets support for multi-document ACID transactions

#166

Post by donttrack »

eksemplar wrote: post #142
SCdF wrote: post #11

Earlier quoted context omitted.

> MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. I have no idea how capable MongoDB is these days, as I haven't used Mongo in years (and even then it was not for long). However, I do not know any developers who, after living through the "hype first, features later" strategy, have been left with a positive enough opinion…

I work in a Danish muniplicity. Traditionally we've build everything on SQL because it's the world we function in, but we adopted the MEAN stack as a proof of concept a few years back and Mongo has been growing ever since. It does require building and maintaining schemas in a different manner, but when you do that, it's pretty great to work with, especially when we're doing design driven development that consists of…

I work on a mongodb installation in a danish municipality. From the technical side mongodb has been great to work with.

Re: MongoDB gets support for multi-document ACID transactions

#167

Post by fastest963 »

SamReidHughes wrote: post #130
nailer wrote: post #67

Earlier quoted context omitted.

Check out Rethink. It seems to be what you're after.

It's probably better to go with Cockroach or TiDB. RethinkDB has problems of its own.

We are in the process of evaluating CockroachDB vs Rethink internally and we've found CockroachDB to perform very poorly without obvious disk or CPU issues. I'm curious if you've seen different especially as it relates to Rethink.

Re: MongoDB gets support for multi-document ACID transactions

#168

Post by threeseed »

gaius wrote: post #152
threeseed wrote: post #133

Earlier quoted context omitted.

I have used MongoDB in production for a number of Fortune 100 sized companies. It has always been a unique database that was ideal for scenarios when your data model was document orientated. > was an overhyped steaming pile of shit for a very long time No it wasn't. This is something you heard from people who never really used it. It had its faults but it was never a pile of shit nor was it substantially worse than o…

This is something you heard from people who never really used it I used it at a previous job. Project to move a multi-tera dataset from an Oracle box (24 CPUs, 24G RAM, SAN) to a MongoDB cluster (10 boxes, each with 48 cores, 96G RAM and internal SSD). MongoDB couldn't perform for shit, and it couldn't stay up in a usable state for more than a few hours at a time. This is with 20x the processors and 40x the memory of…

See this is the sort of crazy things I used to see people do and wonder why they had problems. MongoDB is a document database. You can't just take relational database tables, move them across and expect it to behave the same. And frankly I don't feel sympathy for bad engineering practice. You don't do system migrations without fully testing and understanding all of the systems.

But for those of us that had document orientated data models it allowed for performance that was orders of magnitude faster than any SQL database.

Re: MongoDB gets support for multi-document ACID transactions

#169

Post by imtringued »

javiermaestro wrote: post #134
fnayr wrote: post #101

Earlier quoted context omitted.

Video games storing player data are a great example of nonrelational data. I'm intending on writing a blog post after I finish my game detailing the structure of data I store and why it was so perfect for mongodb.

Please do, because I'm interested in seeing how non-relational it is... I truly believe it has to be otherwise, so I'd love to read your findings!

You still need ACID transactions over multiple entries even if your data is nonrelational otherwise there is the potential for item and money duping bugs.

A simple example would be a marketplace.

Player buys item X with Y gold from another player.

1. Server checks that item X exists.

2. Server checks that the player has at least Y gold.

3. Server removes the gold from the player

4. Server gives gold to the seller.

5. Server removes item from marketplace.

6. Server adds item to inventory.

What if someone maliciously crafts two requests in a way that step 2 of the second request happens before step 3 of the first request? The money is deducted properly but the account can now have a negative balance and there are now two instances of the item.

Re: MongoDB gets support for multi-document ACID transactions

#170

Post by nemo44x »

aeorgnoieang wrote: post #149
nemo44x wrote: post #137

Earlier quoted context omitted.

Most meaningful data is in fact not relational. When you consider machine generated data, logs, metrics, network event data and all other types of data like this you find there are not relations in it. This data is meaningful because it allows to analyze what's going on over massive systems, detect when problems will happen, find bottle necks in applications and infrastructure, among many other use cases. Application…

> When you consider machine generated data, logs, metrics, network event data and all other types of data like this you find there are not relations in it. I find that hard to believe. Maybe not that the raw data isn't already relational, but that there are no relations real or implied . If logs contain info about 'things' and any of those things can be considered to be the 'same thing' for multiple entries, then the…

Well, it's all a point of view really. The data is in the form of an "event". An occurrence of a fact which occurred at a particular time and has data associated with it. So therefore, a "relation" as constructed in a relational database isn't appropriate. You aren't truly denormalizing the data when repeating ID's or tags or labels in this type of data. This is because, at that time that was in the fact the associated ID, tag, label, etc. It would make the stored event false if a field associated with it were to be changed as at the time of the event, that field did not have that value.

But it's all semantics really at that point.

Anyways, a relational database is a poor solution for this type of data. The stored data gains little to nothing, and may even negatively affect it's integrity (at time t, the event DID have this ID; it DID have this label), when stored relationally. Each event is discrete and there will be many of them which optimizes better for scale than relational organization.

I guess my point was there is vastly more useful data appropriate for a non-relational database than there is for relational databases. You might say it still has a "relation" in an abstract sense but this data does not need relational semantics within the database it resides in.

Post reply on HN