Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to make MongoDB not suck for analytics

scaleapi.com

Post reply on HN
Print view View original

71–80 of 98 posts

Re: How to make MongoDB not suck for analytics

#71

Post by 198394549 »

stickfigure wrote: post #36

I tried using MongoDB for the customer-facing analytics of a large e-commerce marketplace. It didn't work very well. The problem is that at some point you end up wanting joins. MongoDB was actually the third try. My first two attempts were BigQuery and Keen, neither of which worked out because they support only one index - time. Users want to slice and dice by various axes! And there's an obvious additional index you…

>The problem is that at some point you end up wanting joins.

It can join with the $lookup function these days. Although it is only to a "non-sharded collection". I don't know why it can't join to a sharded collection when the join is on the same shard though.

There is also the option of using $in with a list of things you have pulled down in another query.

Then there are client-side joins.

Re: How to make MongoDB not suck for analytics

#72

Post by sb8244 »

threeseed wrote: post #68
GenericsMotors wrote: post #66

Earlier quoted context omitted.

> Because if you don't know what you want you are almost guaranteed to pick the wrong technology. That's one way to look at it...but a bit shortsighted. Requirements can and do change, and a well designed model in an RDBMS will be far more extensible than a similar one in NoSQL document store. So RDBMS' aren't the "wrong" technology, they the safest bet; not to mention most modern relational DBs already out-perform m…

How does one design a proper ER model without understanding the domain, query patterns etc ? Because that sounds like magic. Also MongoDB destroys any RDBMS (minimum 10x faster) if you have embedded structures instead of joining against 10 tables in a normalised design. Hence the importance of understanding your query patterns and domain before selecting the database.

The world is messy. The application will grow over time and those requirements can't be known. I think the point parent is making is that a rdms will allow for that future flexibility whereas NoSQL comes with a lot of caveats that make flexibility challenging.

Re: How to make MongoDB not suck for analytics

#73

Post by jrs95 »

Okay, we get it, Mongo sucks. Or at least that seems to be the consensus. From what I can tell it seems they've improved their tech a lot though, and I have to wonder if a lot of the "mongo sucks" sentiment comes from either 1. Using early versions of Mongo that really did suck or 2. people having used Mongo at companies where nobody really knew how to use Mongo that well.

Re: How to make MongoDB not suck for analytics

#75

Post by 198394549 »

sb8244 wrote: post #72
threeseed wrote: post #68

Earlier quoted context omitted.

How does one design a proper ER model without understanding the domain, query patterns etc ? Because that sounds like magic. Also MongoDB destroys any RDBMS (minimum 10x faster) if you have embedded structures instead of joining against 10 tables in a normalised design. Hence the importance of understanding your query patterns and domain before selecting the database.

The world is messy. The application will grow over time and those requirements can't be known. I think the point parent is making is that a rdms will allow for that future flexibility whereas NoSQL comes with a lot of caveats that make flexibility challenging.

What is this lack of flexibility you are speaking about? As in, actual specifics.

Re: How to make MongoDB not suck for analytics

#76

Post by alittletooraph »

larrydag wrote: post #12

This is a huge concern for me at my current organization. Dev has decided to put all data into mongoDB. Yet all decisions are based on that data and the tools we have do not allow for seamless flow (ETL) from mongoDB. That data is important for deriving decisions that affect revenue and costs. Where are solutions for the data analysts and scientists? Frankly I'm pretty sick of hearing it can just be automated. In my…

Beta of MongoDB Charts was announced at the annual conference. It lets you build visualizations on MongoDB data w/o moving data around / ETL. Or if you want to use your existing BI tools, use the BI Connector and the ODBC driver.

Re: How to make MongoDB not suck for analytics

#77

Post by 198394549 »

lomnakkus wrote: post #45
threeseed wrote: post #43

Earlier quoted context omitted.

You are completely contradicting yourself. On one hand you complain about using technologies before you have done a prototype and evaluated the product. Then you blindly tell startups to just use MySQL/PostgreSQL without having any idea of their use case or whether it matches their query patterns. If you are a startup the right way to go is to document your use case, understand what queries those use cases demand and…

Without wishing to put words in their mouth, I think parent poster's point might be that PostgreSQL will do at least a decent job at most things you'll want to throw at it. This is not the case for most of the NoSQL databases where you'll pay for lack of certain features either by a) having to write a lot of code, or b) bad-to-crippling performance for use cases it wasn't meant to solve. So, unless you're already ver…

>This is not the case for most of the NoSQL databases where you'll pay for lack of certain features either by a) having to write a lot of code, or b) bad-to-crippling performance for use cases it wasn't meant to solve.

Can you give a common example of these? This article is referring to issues related to row vs column data stores, not sql vs nosql.

Re: How to make MongoDB not suck for analytics

#78

Post by naranha »

198394549 wrote: post #71
stickfigure wrote: post #36

I tried using MongoDB for the customer-facing analytics of a large e-commerce marketplace. It didn't work very well. The problem is that at some point you end up wanting joins. MongoDB was actually the third try. My first two attempts were BigQuery and Keen, neither of which worked out because they support only one index - time. Users want to slice and dice by various axes! And there's an obvious additional index you…

>The problem is that at some point you end up wanting joins. It can join with the $lookup function these days. Although it is only to a "non-sharded collection". I don't know why it can't join to a sharded collection when the join is on the same shard though. There is also the option of using $in with a list of things you have pulled down in another query. Then there are client-side joins.

> Then there are client-side joins.

AKA what you are doing when writing your SPAs with their own state management. Server-side joins rarely make sense in that context.

For reporting/analytics.. yes. But these can be delegated to external system/databases optimized for that task. With elasticsearch for example you get very far very quickly without the need to write any SQL joins.

Re: How to make MongoDB not suck for analytics

#80

Post by nickserv »

198394549 wrote: post #69
nickserv wrote: post #65

Earlier quoted context omitted.

It's not too far from that joke. It's like if you ask "how do I drive my car downtown" and I answer, "Easy, just park at the station and take the train". To answer your other question, their marketing goes a long way. I recently started at a new company, and the lead was proudly telling me how the project was developed using Mongo... So I start explaining how it's basically shit after using it professionally for a fe…

Why is it basically shit? It appears to store and retrieve the data as per my instructions.

Except when it doesn't. We've had data corruption issues related to oplog, out of sync secondaries and excessive resource usage on the primary. As far as major problems. There were also a bunch of smaller problems but in fairness those were on the nodejs/mongoose side of things. Would not recommend.
Post reply on HN