Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Was MongoDB Ever the Right Choice?

simplethread.com

Post reply on HN
Print view View original

41–50 of 224 posts

Re: Was MongoDB Ever the Right Choice?

#41

Post by drbojingle »

jstimpfle wrote: post #38
giornogiovanna wrote: post #28

Earlier quoted context omitted.

Sorry for my ignorance, but what JSON craze?

As a serialization format, JSON is horrible. Way too loose regarding formatting and way too much noise. All that is typically needed is a standard text format for relational data. I.e. a fixed CSV. The only "advantage" to JSON is that it maps directly to the object trees most developers use in their scripted programs (which is misguided IMHO).

The advantage of JSON is that it was there. Horrible but immediately available > Amazing but currently unavailable.

Re: Was MongoDB Ever the Right Choice?

#42

Post by kevingadd »

leejo wrote: post #40

In my previous job, at a payment service provider, I wrote a "velocity" system. This was required to block transactions that met certain conditions within a certain time frame. A typical example was: block transactions that are from the same IP if that happens more than 5 times in a 10 minute period. Or: block transactions from the same card number if it is used more than 3 times in a minute. The problem was that the…

I'm not clear on why the datastore needed to be schemaless though? It sounds like the schema for the transaction data was well understood and only the blocking requirements were not, but the blocking requirements were formulas/assertions over the well-formed data.

Re: Was MongoDB Ever the Right Choice?

#43

Post by rwmj »

matthewmacleod wrote: post #30

I would argue that MongoDB is not—and has never been—the best choice for solving any particular technical problem. But it had some other "advantages" over other, better solutions – in that it was easier to set up, didn't require schema definition, had a passable clustering story etc. I have worked with at least one company that had been built using MongoDB as a primary data store from day one. This caused untold pain…

I once worked for a company where Lotus Notes fulfilled a similar role (this was back in the late 1990s). Eventually they ran an entire free to sign up web-based email system using 4 giant Notes instances. It was an absolute nightmare, but probably the company would never have happened otherwise because Notes was all that the founders knew. My job was managing the migration off to a normal SQL database, which took the best part of a year.

Re: Was MongoDB Ever the Right Choice?

#44

Post by tnolet »

matthewmacleod wrote: post #32
gnur wrote: post #21

Off course it was, prototyping speed on mongodb was (and probably still is) always excellent. Some features that don't scale are very nice to have when you don't have scaling issues. For example, if you add tags to your documents and you want to query on those tags (find all documents containing tag A and B), it's nice that's just a builtin. I haven't found a single datastore that is as developer friendly that suppor…

The other answers have confirmed that Postgres will do this with array fields, and it's good advice to follow. It's also in my view much easier to read than MongoDB's query language is! CREATE TABLE documents (name text, tags text[]); INSERT INTO documents VALUES ('Doc1', '{tag1, tag2}'); INSERT INTO documents VALUES ('Doc2', '{tag2, tag3}'); INSERT INTO documents VALUES ('Doc3', '{tag2, tag3, tag4}'); SELECT * FROM…

Using this almost 1:1 in a production app where customers can filter by tags. Works great.

Re: Was MongoDB Ever the Right Choice?

#45

Post by thosakwe »

In my experience, MongoDB shouldn’t be used in cases where you need relations, or the typical guarantees that ACID databases bring.

I think a lot of the “beef” people have towards MongoDB is from past experiences where it was used in this regard.

I usually use PostgreSQL at this point, but when I use MongoDB, it’s more as a “document store” than a “database.”

Re: Was MongoDB Ever the Right Choice?

#46

Post by rujuladanh »

leejo wrote: post #40

In my previous job, at a payment service provider, I wrote a "velocity" system. This was required to block transactions that met certain conditions within a certain time frame. A typical example was: block transactions that are from the same IP if that happens more than 5 times in a 10 minute period. Or: block transactions from the same card number if it is used more than 3 times in a minute. The problem was that the…

If you have almost no guarantees to provide, almost any solution would have worked.

In other words, you claim it worked for years, but anything would if you can lose all data, reboot it, have no real-time requirements, etc.

In summary, it is not really a useful data point on whether MongoDB is useful or not.

Re: Was MongoDB Ever the Right Choice?

#47

Post by a13n »

thosakwe wrote: post #45

In my experience, MongoDB shouldn’t be used in cases where you need relations, or the typical guarantees that ACID databases bring. I think a lot of the “beef” people have towards MongoDB is from past experiences where it was used in this regard. I usually use PostgreSQL at this point, but when I use MongoDB, it’s more as a “document store” than a “database.”

MongoDB is ACID compliant. https://www.mongodb.com/transactions

Re: Was MongoDB Ever the Right Choice?

#48

Post by leejo »

kevingadd wrote: post #42
leejo wrote: post #40

In my previous job, at a payment service provider, I wrote a "velocity" system. This was required to block transactions that met certain conditions within a certain time frame. A typical example was: block transactions that are from the same IP if that happens more than 5 times in a 10 minute period. Or: block transactions from the same card number if it is used more than 3 times in a minute. The problem was that the…

I'm not clear on why the datastore needed to be schemaless though? It sounds like the schema for the transaction data was well understood and only the blocking requirements were not, but the blocking requirements were formulas/assertions over the well-formed data.

The transaction data was well understood, but did change over time. It was well normalised in a relational backend that was (at the time) a few hundred GB, but it did not have any partitioning.

The problem we were trying to solve was that allowing, essentially, arbitrary queries against the main database had the potential to cause problems because we would have to ensure that those queries were optimal and all the relations were correct before we allowed those rules to be put into place - i.e. more dev and DBA work and potentially schema changes (addition of more indexes, etc).

It was far easier to have an ephemeral duplicate of the transaction data with a flat structure, a single key value document, as that mitigated the risks and removed the aforementioned problem.

The velocity system was essentially a firewall in front of the main database server - this was one of the main requirements, the velocity system must not have any affect on the load of the main transaction system. We could have added another db server, but that would have put us on the route to more servers, more complexity, etc, as we were self hosting at the time.

Re: Was MongoDB Ever the Right Choice?

#49

Post by leejo »

rujuladanh wrote: post #46
leejo wrote: post #40

In my previous job, at a payment service provider, I wrote a "velocity" system. This was required to block transactions that met certain conditions within a certain time frame. A typical example was: block transactions that are from the same IP if that happens more than 5 times in a 10 minute period. Or: block transactions from the same card number if it is used more than 3 times in a minute. The problem was that the…

If you have almost no guarantees to provide, almost any solution would have worked. In other words, you claim it worked for years, but anything would if you can lose all data, reboot it, have no real-time requirements, etc. In summary, it is not really a useful data point on whether MongoDB is useful or not.

> If you have almost no guarantees to provide, almost any solution would have worked.

Within reason. If the failure rate is 1 in 1000 then it's probably not as a big a concern as 1 in 10 depending on your use case

> In summary, it is not really a useful data point on whether MongoDB is useful or not.

As I said in the last sentence - it worked perfectly. In other words the "no guarantees to provide" never actually came up and I don't recall us having any data loss, response time, or write issues.

Re: Was MongoDB Ever the Right Choice?

#50

Post by jandrewrogers »

My anecdote: several years back I had a test lab for measuring the performance and scalability characteristics of various geospatial databases. We added MongoDB to the mix a couple years after they released geospatial support.

We always verified basic correctness with a new database by inserting several billion geometries as fast as the database would accept them, reading the entire data set back out, and comparing it to the original data set we inserted for any discrepancies. MongoDB never passed this test. It would apparently lose records semi-randomly every time, so we removed it from the test set. It was the only database we tested that had this issue.

Post reply on HN