Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MongoDB gets support for multi-document ACID transactions

techcrunch.com

Post reply on HN
Print view View original

211–220 of 245 posts

Re: MongoDB gets support for multi-document ACID transactions

#211

Post by kureikain »

jeffdavis wrote: post #96
luckydata wrote: post #72

Earlier quoted context omitted.

If you're playing the long game and not looking to make a profit that's fine, but PostgreSQL as a company would have been doomed a long time ago. You have to keep in mind the timelines of the business and what they need to do to keep the lights on. MongoDB has identified a real pain point: many developers don't like to use SQL to interface with a transactional database. I'm not going into the merits of SQL vs. NoSQL,…

"PostgreSQL as a company would have been doomed a long time ago" PG has astonishing feature throughput . With each yearly release, they add 1-3 wow features, 6-10 major features, and countless smaller features still worthy of the release notes. That's really, really impressive for any database, commercial or otherwise. There's a perception that postgres is slow to add features because sometimes the feature latency is…

MongoDB supports 1) 2) 6) 7). Not sure what 3 and 4 is, but you can just add a new node and new data will be copied over, no need to restore data from snapshot, but you can restore from snapshot too, it shorten the time the replice become available.

Not sure what you mean by 5) though.

Anyway, replciation is strong point of MongoDB with oplog and I don't think Postgres can beat it.

Re: MongoDB gets support for multi-document ACID transactions

#212

Post by addicted »

takeda wrote: post #207
golergka wrote: post #103

Earlier quoted context omitted.

I don't know much about MongoDB. I am mostly a client-side developer after all. But every time I see a team transitioning from Mongo to something else, they transition to a relational database. May be their problem is not with MongoDB, but that their data is relational after all? Personally, I'd take a relational db over NoSQL for most of my needs, but all these stories don't really say anything about how Mongo compa…

That's because nearly all data is relational. At first it seems like you don't need a relational database, in fact NoSQL seems easier to use at first. As your data grows though you realize that your application become more and more complex. A single query might translate to multiple queries to the database, you need to handle scenarios where fields might not exist etc. With relational data you might have more work at…

Or maybe it's useful to get started off with a low overhead, easy to implement DB like Mongo and then as you grow larger, spin off uses that it doesn't serve well to other more specialized and complicated DBs?

One of the biggest problem with relational DBs is that once you decide on a schema, if it's the wrong one, you're gonna be in a lot of pain. Which makes a NoSQL DB a great fit for an early stage product where you are still figuring out what your product needs to do and contain. Once you have some more experience with it, and have a better understanding of your data, it's far easier to build the correct relationships.

Re: MongoDB gets support for multi-document ACID transactions

#213

Post by nnain »

SCdF wrote: post #11
_wc0m wrote: post #6

MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. The RethinkDB retrospective[0] contains a lot of insight into how MongoDB has succeeded despite being vastly inferior on a technical level back when it first launched. I have to admit them a certain respect for executing their strategy so successfully. Choice quote: Every t…

> MongoDB has successfully played the 'hype first, features later' strategy. Now it is well on the way to being a decent swiss-army-knife database. I have no idea how capable MongoDB is these days, as I haven't used Mongo in years (and even then it was not for long). However, I do not know any developers who, after living through the "hype first, features later" strategy, have been left with a positive enough opinion…

> However, I do not know any developers who, after living through the "hype first, features later" strategy, have been left with a positive enough opinion of MongoDB to ever want to use it again.

A new crop of developers is, always, just an year away though. I feel future adoption depends a lot on how well-suited the tools are for younger devs. That's where MongoDB found the initial audience!

Re: MongoDB gets support for multi-document ACID transactions

#214

Post by takeda »

Pxtl wrote: post #196
ZenoArrow wrote: post #159

Earlier quoted context omitted.

> "Honestly, the existence of Mongo is mostly an indictment about how user-hostile conventional RDBMS is." That's nonsense. I've taught people to use SQL before, even people with little to no programming experience. After the initial concepts were understood it was fairly easy to gradually expand knowledge over time. The basic SQL keywords to start getting useful information out of a RDBMS are: SELECT FROM WHERE INNE…

That's after you set up users, schemas, tables, columns, oddly-named datatypes with unexpected behaviors, etc. The RDBMS is a lie. It's a beautiful kernel of relational theory wrapped in a 60-foot ball of hacks, tweaks, duct-tape, bubble-gum, and hate. The document store doesn't lie. It doesn't pretend. It's honest that it's stupid and it's a glorified hashtable with a string in it. It doesn't make any ridiculous pre…

> That's after you set up users, schemas, tables, columns

You have software engineer you similarly should have a data engineer (i.e. a DBA). Nearly 50 years has passed and we still didn't find a better way to represent data, so perhaps it is the right model. The only difficult part is to bother enough to learn it.

> oddly-named datatypes with unexpected behaviors [...] 60-foot ball of hacks, tweaks, duct-tape, bubble-gum, and hate

That's only when using MySQL

> Meanwhile an object DB may be inefficent and clumsy, but it gets all of that stuff out of the way. Also, if you don't want to join, you can work around that by duplicating the data all over the place. Something you can't do with an RDBMS because tables are fundamentally flat and so you can't stuff a parent-child relationship into a single table.

You absolutely can store data inefficiently in RDBMS for example in Postgres you can create a table with two columns one named key, and another data. The data type for the second column can be JSONB. End you're essentially have equivalent of Mongo's collection.

But in that case you store data inefficiently and if your application starts evolving and you need to make different queries things will get more complex quickly.

Re: MongoDB gets support for multi-document ACID transactions

#215

Post by eksemplar »

jchb wrote: post #209
donttrack wrote: post #166

Earlier quoted context omitted.

I work on a mongodb installation in a danish municipality. From the technical side mongodb has been great to work with.

I am curious why a municipality needs custom software. I mean, the scandinavian countries had standardised paper forms for most municipal tasks (population register, ledgers etc) already in the 17-18th century, and those were used nationwide, or at least throughout a single province. Why can't the same be done with software?

Well there are 98 municipalities and 98 ways to operate in a thousand different ways.

I’ve worked on quite a few multi-municipalitiy open source projects, like handling employee refunds on driving.

Basically I drive x kilometers for a meeting, I get paid x and the taxman gets the report. Simple stuff.

Well in the 6 parties involved there were 6 ways to interpret tax laws, 4 different agreements with unions on what rates to pay, 3 different payment systems with 3 very different ways of taking the reported data from a flat file to a rest interface, at least one political decision to overrule tax laws for a certain set of employeees and several different ideas on how to host it and so single sign on, oh and 4 different ways to obtain employee data.

That’s for a simple system with basically 1 function. We have more than 350 it-systems.

Another example is in automation. We have a scanner software and we have an archiving system. They both have APIs but the APIs speak very differ languages. This meant that our local scanner people were tasked with distribution after they scanned things, a task taking several hours each week because putting files into many different areas of an archive sucks. What we did was ask the scanning company to build a QR reader into their software, and then we made a piece of software that put the archive recipient addresses into QR codes. We also made a MOX agent, that accepta the output of our scanning software and loads it into the archive through the API. So now the process of distributing is automated.

You can certainly run a municipality without developers, using standard software and outside hires, it’s just really expensive.

Re: MongoDB gets support for multi-document ACID transactions

#216

Post by takeda »

metheus wrote: post #118
jaequery wrote: post #93

Mongodb can be quite a nightmare once you start requiring anything more than a 1:1 relationship, which is pretty much any kind of app that is doing anything meaningful. Having to resort to doing things like map/reduce for a simple group by / order is not the way to go IMO. I think you later truly realize the beauty of SQL once you get far down that rabbit hole. Initially I rode on the Mongo's NoSQL bandwagon when I s…

> Having to resort to doing things like map/reduce for a simple group by / order is not the way to go IMO. Perhaps you are unaware, but that is a straw-man. That’s not how you’re supposed to with MongoDB. You’d use the aggregation framework: db.sales.aggregate( [ { $group : { _id : null, totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity" }, count: { $sum: 1 } } } ] )…

> db.sales.aggregate( [ { $group : { _id : null, totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity" }, count: { $sum: 1 } } } ] )

Now look at SQL version of it and tell me which one is easier:

SELECT SUM(price * quantity), AVG(quantity), COUNT(*) FROM sales;

Re: MongoDB gets support for multi-document ACID transactions

#217

Post by ZenoArrow »

Pxtl wrote: post #196
ZenoArrow wrote: post #159

Earlier quoted context omitted.

> "Honestly, the existence of Mongo is mostly an indictment about how user-hostile conventional RDBMS is." That's nonsense. I've taught people to use SQL before, even people with little to no programming experience. After the initial concepts were understood it was fairly easy to gradually expand knowledge over time. The basic SQL keywords to start getting useful information out of a RDBMS are: SELECT FROM WHERE INNE…

That's after you set up users, schemas, tables, columns, oddly-named datatypes with unexpected behaviors, etc. The RDBMS is a lie. It's a beautiful kernel of relational theory wrapped in a 60-foot ball of hacks, tweaks, duct-tape, bubble-gum, and hate. The document store doesn't lie. It doesn't pretend. It's honest that it's stupid and it's a glorified hashtable with a string in it. It doesn't make any ridiculous pre…

> "That's after you set up users, schemas, tables, columns, oddly-named datatypes with unexpected behaviors, etc."

You shouldn't jump in at the deep end when learning this stuff, getting a feel for how an existing database works before creating your own is helpful, and you can download sample databases to play around with when you start learning if you don't have access to one already. Aside from this recommendation, if you're starting out learning about RDBMS, then SQlite is a good starting point, and the setup of a SQlite database is fairly simple.

If you're starting from scratch with a brand new database, here are some of the SQL keywords you'll find useful:

CREATE TABLE

ALTER TABLE

DELETE TABLE

TRUNCATE TABLE

INSERT INTO

UPDATE

SET

DELETE FROM

VARCHAR

INT

DECIMAL

PRIMARY KEY

FOREIGN KEY

REFERENCES

Furthermore, if you're using a decent DB GUI frontend, you don't even need to remember most of the above, aside from VARCHAR (for strings), INT (for whole numbers) and DECIMAL (for numbers with a fractional element). Reason being, you can do all of the database setup graphically. Tools like SQL Server Management Studio help in reducing friction.

Re: MongoDB gets support for multi-document ACID transactions

#218

Post by takeda »

munk-a wrote: post #99
stevenwoo wrote: post #39

Earlier quoted context omitted.

Epic had a post mortem blog post here that mentioned in passing they had stumped all the experts they could find to look at unsolveable issues they had with MongoDB. https://news.ycombinator.com/item?id=16340462 I kind of assumed the fix is going to be a rewrite with Postgres or MySQL.

Yes, come over to the actually greener lawn of Postgres, it is amazing how much more powerful than MySQL it has become with recursive CTEs https://www.postgresql.org/docs/current/static/queries-with.... and some rather amazing JSON support https://www.postgresql.org/docs/current/static/functions-agg...

This is actually what I think the biggest power of JSONB is.

I can for example use jsonb_agg() and get a hierarchical response for 1:N joins. It returns JSON value even though neither of the columns contain JSON.

Previously in that scenario I would either need to make more than one query or get a response that has a lot of data repeated.

Re: MongoDB gets support for multi-document ACID transactions

#219

Post by takeda »

scarface74 wrote: post #78
riku_iki wrote: post #63

Earlier quoted context omitted.

I never used MognoDB, but read many stories that few years ago MognoDB default configuration permitted data loss: server returned write success response to client without syncing data to replica or disk, and if your server died then your data gone. Additionally some time ago MongoDB had single-threaded writes with collection level lock, which could cause poor write performance.

That's one thing I keep "hearing". The default configuration now is a WriteConcern of 1 - meaning at least the primary has to have written it successfully. You can choose 2 or majority among others.

That won't protect you, you can read the analysis for MongoDB from here: http://jepsen.io/analyses

The last analysis looks good, but you need to note that this is only true with the strongest settings (this means that Mongo will be configured with slowest settings) also this analysis did not include node crashes or restarts.

Re: MongoDB gets support for multi-document ACID transactions

#220

Post by takeda »

addicted wrote: post #212
takeda wrote: post #207

Earlier quoted context omitted.

That's because nearly all data is relational. At first it seems like you don't need a relational database, in fact NoSQL seems easier to use at first. As your data grows though you realize that your application become more and more complex. A single query might translate to multiple queries to the database, you need to handle scenarios where fields might not exist etc. With relational data you might have more work at…

Or maybe it's useful to get started off with a low overhead, easy to implement DB like Mongo and then as you grow larger, spin off uses that it doesn't serve well to other more specialized and complicated DBs? One of the biggest problem with relational DBs is that once you decide on a schema, if it's the wrong one, you're gonna be in a lot of pain. Which makes a NoSQL DB a great fit for an early stage product where y…

> Or maybe it's useful to get started off with a low overhead, easy to implement DB like Mongo and then as you grow larger, spin off uses that it doesn't serve well to other more specialized and complicated DBs?

Not really, converting to relational data is quite a bit of work.

Actually the reverse is the correct approach. You start with normalized data, when there's a bottleneck you start denormalizing it, if that's still not enough you move /subset/ of data to NoSQL database.

> One of the biggest problem with relational DBs is that once you decide on a schema, if it's the wrong one, you're gonna be in a lot of pain.

Not really from my experience all migrations were done through SQL. Also if multiple people (who understand relational databases) come with a schema they pretty much will arrive to the same normalized result.

Post reply on HN