Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to make MongoDB not suck for analytics

scaleapi.com

Post reply on HN
Print view View original

61–70 of 98 posts

Re: How to make MongoDB not suck for analytics

#61

Post by nostalgeek »

threeseed wrote: post #43
stickfigure wrote: post #36

I tried using MongoDB for the customer-facing analytics of a large e-commerce marketplace. It didn't work very well. The problem is that at some point you end up wanting joins. MongoDB was actually the third try. My first two attempts were BigQuery and Keen, neither of which worked out because they support only one index - time. Users want to slice and dice by various axes! And there's an obvious additional index you…

You are completely contradicting yourself. On one hand you complain about using technologies before you have done a prototype and evaluated the product. Then you blindly tell startups to just use MySQL/PostgreSQL without having any idea of their use case or whether it matches their query patterns. If you are a startup the right way to go is to document your use case, understand what queries those use cases demand and…

> Right tool for the right job.

I would argue that since both Mysql and PostgreSQL are JSON document stores with mostly the same capabilities when it comes to querying and aggregation I don't see the advantage of using MongoDB at all.

I wouldn't even use MongoDB for caching when redis does a better job at it. Logs? I don't see why logs cannot be shoved into a RDBMS. Prototyping? create a table with a JSON field and a primary key. Distributed file system? I don't know any business which uses gridFS as a CDN, full text search? PostgreSQL does it better. So what is the job your are talking about? PostgreSQL is so much powerful for analytics because of the power of SQL.

Re: How to make MongoDB not suck for analytics

#62

Post by tjpnz »

gaius wrote: post #56
larrydag wrote: post #12

This is a huge concern for me at my current organization. Dev has decided to put all data into mongoDB. Yet all decisions are based on that data and the tools we have do not allow for seamless flow (ETL) from mongoDB. That data is important for deriving decisions that affect revenue and costs. Where are solutions for the data analysts and scientists? Frankly I'm pretty sick of hearing it can just be automated. In my…

Dev has decided to put all data into mongoDB. Yet all decisions are based on that data and the tools we have do not allow Seems to me that it’s on your devs to explain to the business why their poor technical choices now necessitate a substantial additional investment to get a usable solution. When they could have just used Postgres, and they knew it.

In my experience these types of situations only come about through ignorance. This is anecdotal so YMMV but most people I've worked with who propose Mongo don't know SQL and don't generally want to take the time to learn it.

Quite often they will have also read somewhere that joins are slow and have managed to convince themselves that the solution is to avoid relational databases altogether. Or maybe that's just how they justify it.

Re: How to make MongoDB not suck for analytics

#63

Post by kockic »

I see that most of the `don't use mongodb for analytics` are being down-voted, however I tend to agree with them. For all the people out there looking for the database for analytics please check Clickhouse from Yandex, it's easy to get started, amazingly fast and open source.

Disclaimer: I am not affiliated with Yandex in anyway, just a happy customer

Re: How to make MongoDB not suck for analytics

#64

Post by eddd »

I kind of a hoped it'll end up a joke saying "Don't use mongo". Last time I used it was 2.4 and it was the worst db experience ever. Back then It was more sane to craft a solution with PG and HSTORE. Now, I think RedShift does the job, why would anyone use mongo on production for anything today?

Re: How to make MongoDB not suck for analytics

#65

Post by nickserv »

eddd wrote: post #64

I kind of a hoped it'll end up a joke saying "Don't use mongo". Last time I used it was 2.4 and it was the worst db experience ever. Back then It was more sane to craft a solution with PG and HSTORE. Now, I think RedShift does the job, why would anyone use mongo on production for anything today?

It's not too far from that joke.

It's like if you ask "how do I drive my car downtown" and I answer, "Easy, just park at the station and take the train".

To answer your other question, their marketing goes a long way. I recently started at a new company, and the lead was proudly telling me how the project was developed using Mongo... So I start explaining how it's basically shit after using it professionally for a few years. His answer? But SQL doesn't scale well enough!

Re: How to make MongoDB not suck for analytics

#66

Post by GenericsMotors »

threeseed wrote: post #59
lomnakkus wrote: post #45

Earlier quoted context omitted.

Without wishing to put words in their mouth, I think parent poster's point might be that PostgreSQL will do at least a decent job at most things you'll want to throw at it. This is not the case for most of the NoSQL databases where you'll pay for lack of certain features either by a) having to write a lot of code, or b) bad-to-crippling performance for use cases it wasn't meant to solve. So, unless you're already ver…

> So, unless you're already very clear on what your exact use case is going why the spend time analysing before even getting your project off the ground? Because if you don't know what you want you are almost guaranteed to pick the wrong technology.

> Because if you don't know what you want you are almost guaranteed to pick the wrong technology.

That's one way to look at it...but a bit shortsighted.

Requirements can and do change, and a well designed model in an RDBMS will be far more extensible than a similar one in NoSQL document store. So RDBMS' aren't the "wrong" technology, they the safest bet; not to mention most modern relational DBs already out-perform mongo, so the point is sort of moot anyway.

Re: How to make MongoDB not suck for analytics

#67

Post by codingdave »

I'm not aware of any analytics platform that runs directly from the source data. There is just about always some kind of ETL process, or at the very least, a data transformation process to shape the data as needed, to provide data that works well for the reporting. So while information on making MongoDB performant for such things is mildly interesting... it just isn't how analytics are generally architected.

Re: How to make MongoDB not suck for analytics

#68

Post by threeseed »

GenericsMotors wrote: post #66
threeseed wrote: post #59

Earlier quoted context omitted.

> So, unless you're already very clear on what your exact use case is going why the spend time analysing before even getting your project off the ground? Because if you don't know what you want you are almost guaranteed to pick the wrong technology.

> Because if you don't know what you want you are almost guaranteed to pick the wrong technology. That's one way to look at it...but a bit shortsighted. Requirements can and do change, and a well designed model in an RDBMS will be far more extensible than a similar one in NoSQL document store. So RDBMS' aren't the "wrong" technology, they the safest bet; not to mention most modern relational DBs already out-perform m…

How does one design a proper ER model without understanding the domain, query patterns etc ?

Because that sounds like magic.

Also MongoDB destroys any RDBMS (minimum 10x faster) if you have embedded structures instead of joining against 10 tables in a normalised design. Hence the importance of understanding your query patterns and domain before selecting the database.

Re: How to make MongoDB not suck for analytics

#69

Post by 198394549 »

nickserv wrote: post #65
eddd wrote: post #64

I kind of a hoped it'll end up a joke saying "Don't use mongo". Last time I used it was 2.4 and it was the worst db experience ever. Back then It was more sane to craft a solution with PG and HSTORE. Now, I think RedShift does the job, why would anyone use mongo on production for anything today?

It's not too far from that joke. It's like if you ask "how do I drive my car downtown" and I answer, "Easy, just park at the station and take the train". To answer your other question, their marketing goes a long way. I recently started at a new company, and the lead was proudly telling me how the project was developed using Mongo... So I start explaining how it's basically shit after using it professionally for a fe…

Why is it basically shit? It appears to store and retrieve the data as per my instructions.
Post reply on HN