Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I Migrated Away From MongoDB

svs.io

Post reply on HN
Print view View original

71–80 of 213 posts

Re: Why I Migrated Away From MongoDB

#71

Post by jeremyjh »

aneth4 wrote: post #23
stevencorona wrote: post #10

The downside, or challenge, with NoSQL (generally speaking) is that you need to handle your aggregations ahead of time - you need to know what queries you'll want to run in the future when you store your data. If you have some new aggregation you want to keep, you'll need to re-process the data (with Hadoop or something else). It's the trade-off of being able to scale reads and writes horizontally. And unless you nee…

This is the opposite of "agile." It is difficult to know where your product will be in 2 months let alone 12, so it seems the advice to use SQL first is sound - unless you enjoy long distractions to solve simple JOINs.

You should at least know if you will likely have any analytic use cases.

Re: Why I Migrated Away From MongoDB

#72

Post by stickfigure »

gregjor wrote: post #47
stickfigure wrote: post #43

Earlier quoted context omitted.

it's best just to start with a SQL database and go from there. This is bad advice. It's best to understand your problem domain and use the tools that are most appropriate. You see a lot of two types of posts on HN: * "I picked a NoSQL database for a problem domain with a better relational fit." Those posts look like this one. * "I picked an RDBMS for a problem domain with a better NoSQL fit." Those posts are usually…

This advice -- the most frequent reply in this thread -- is the same as saying you should be able to accurately predict the future. Database management is a big and complicated topic. Especially with emerging tools like Mongo there is no way reading a book is going to give you the expertise and experience you need to choose the right tool. You do your best and maybe make the wrong decision. It's not stupid to go down…

It doesn't require the ability to predict the future, just some familiarity with the tools. The OP's use case (small amounts of data, low traffic, lots of aggregation) is perfect for an RDBMS, and anyone with even casual familiarity with both SQL and NoSQL systems could easily prognosticate that.

Re: Why I Migrated Away From MongoDB

#73

Post by r00fus »

sureshv wrote: post #53
ericcholis wrote: post #37

Earlier quoted context omitted.

I've found that a simple {"lastname":/cholis/i} works great. However, trying to do the same thing for a multi-key search isn't ideal. Specifically, searching for 3 words in a title using $and with multiple regex queries on a collection with 100,000+ documents took about 520 ms. The mongodb documentation suggests that you could have an array with your keywords, generated from the field you wish to search. Using indexe…

Building over a full text search engine that indexes your db is going to scale much better in the long run. Plus you get stemming and other niceties to boot as long as consistency is not of the highest priority.

With tools like SOLR or Elasticsearch, I keep wondering why people try to re-invent the full-text indexing wheel - these tools make it easy to focus your app/datastore on your functionality and leave search as an additional service, to be used whenever you need it.

Re: Why I Migrated Away From MongoDB

#75

Post by gregjor »

blaines wrote: post #70
gregjor wrote: post #42

You were fortunate to recognize that MongoDB was the wrong tool for your job, and lucky to be able to move to Postgres instead of continuing to throw your time and effort away. I see the ad hominem "you're an ignorant idiot" attacks already started, along with advice like using regexes to do case-insensitive searches. Watching the NoSQL "movement" encounter the problems RDBMSs fixed 20 years ago and then hand-wave an…

So do you believe there is a use case for a document-oriented database? I feel like your comment writes off a huge swath of useful technology. I said this below, but I'll say it again. Data is malleable, and writing apps to fit around any datastore seems wrong. I write applications to fulfill their use case. When the needs of the application change, so could it's database (or other dependencies). http://gigaom.com/cl…

I don't know if there's a use case. I have never had one come up in my own projects. I do sometimes need to manage a large number of documents (unstructured data) but I have found the Unix file system sufficient for those, maybe augmented by metadata in the database.

I've worked on big applications with big databases where DB operations and aggregations started to get too slow but so far I or someone smarter on the team has always been able to figure out a solution without throwing the RDBMS out. One big advantage of working with RDBMSs is the roads are paved and well-mapped. By contrast you still need a machete and good luck in the NoSQL world, and even then you can get sick or lost in the jungle.

I haven't personally worked on a web app with the kinds of database scaling issues I've dealt with in logistics applications involving tens of millions of movements every day.

RDBMSs were invented to allow data malleability. Go back to the old textbooks and look at what preceded the relational model. Remember that data is more valuable and permanent than application code. When you have multiple applications accessing the same database, each with their own implementation of consistency rules and their own internal idea of the data schema, come back and explain how that is more fun than using an RDBMS and being able to sleep nights.

Re: Why I Migrated Away From MongoDB

#76

Post by ilaksh »

On your home page you imply that you can automatically OCR arbitrary handwritten receipts into an analyzable format.

No one can do that. That is your problem, not MongoDB.

As far as aggregation, use the new Aggregation Framework http://docs.mongodb.org/manual/tutorial/aggregation-examples...:

    db.zipcodes.aggregate
      $group:
        _id: "$state"
        totalPop:
          $sum: "$pop"
    ,
      $match:
        totalPop:
          $gte: 10 * 1000 * 1000
As far as "losing the independence of your data access paths", no you don't. You are free to use linking instead of embedding wherever you want. http://www.mongodb.org/display/DOCS/Schema+Design#SchemaDesi...

MongoDB doesn't have a built-in full text search? So what. Most systems with large amounts of text to search do not rely on the text search capabilities built into relational databases anyway. People use actual full-text search engines like Lucene/Solr, Sphinx, reds, etc. Having said that, if you just wanted to support lowercase keyword queries with MongoDB, would it really be so hard to extract and store lowercase keywords from your text, as suggested here? http://www.mongodb.org/display/DOCS/Full+Text+Search+in+Mong...

If you are trying to add four 1s and get '1111' instead of 4, that is an error in your application code which has nothing to do with MongoDB. Very common problem with JavaScript. If it is JavaScript, try finding the code where you are attempting numeric addition and change it so that instead of saying for example 'total += newNumber' it says 'total += (newNumber * 1)' .

Re: Why I Migrated Away From MongoDB

#77

Post by stickfigure »

yxhuvud wrote: post #68
stickfigure wrote: post #43

Earlier quoted context omitted.

it's best just to start with a SQL database and go from there. This is bad advice. It's best to understand your problem domain and use the tools that are most appropriate. You see a lot of two types of posts on HN: * "I picked a NoSQL database for a problem domain with a better relational fit." Those posts look like this one. * "I picked an RDBMS for a problem domain with a better NoSQL fit." Those posts are usually…

The problem with your argument is that all problem will look like it will have polymorphic and ambigous data until you have mapped it out properly. What the post you replied to was basically an advice for new projects and developers to fully map out the problem and see where things are going. It is a lot easier to start strict and organized and back down to less strictness with looser rules where you need it, than to…

Perhaps for "new developers". If the question is "as a neophyte developer, which should I learn first, Postgres or MongoDB?" then I would tend to lean towards SQL as well. But that isn't the proposal.

The problem is the old cliche that if all you have is a hammer, everything looks like a nail. "Just start with a hammer" is bad advice because with enough effort and willingness to cause collateral damage, you can almost always get a screw in with a hammer. If you've never seen a screwdriver, you're doomed to posting HN stories about "how I spent the last three weeks rebuilding my application architecture to deal with the fact that hammered-in screws tend to fall out a lot."

Re: Why I Migrated Away From MongoDB

#78

Post by nnnnnnnn »

ilaksh wrote: post #76

On your home page you imply that you can automatically OCR arbitrary handwritten receipts into an analyzable format. No one can do that. That is your problem, not MongoDB. As far as aggregation, use the new Aggregation Framework http://docs.mongodb.org/manual/tutorial/aggregation-examples... : db.zipcodes.aggregate $group: _id: "$state" totalPop: $sum: "$pop" , $match: totalPop: $gte: 10 * 1000 * 1000 As far as "losi…

"On your home page you imply that you can automatically OCR arbitrary handwritten receipts into an analyzable format. No one can do that. That is your problem"

Jeez, lay off the confrontational tone. He doesn't say anything about OCR. Maybe he's using humans to do data entry? In any event, it's completely irrelevant to the topic of databases.

Re: Why I Migrated Away From MongoDB

#79

Post by ilaksh »

nnnnnnnn wrote: post #78
ilaksh wrote: post #76

On your home page you imply that you can automatically OCR arbitrary handwritten receipts into an analyzable format. No one can do that. That is your problem, not MongoDB. As far as aggregation, use the new Aggregation Framework http://docs.mongodb.org/manual/tutorial/aggregation-examples... : db.zipcodes.aggregate $group: _id: "$state" totalPop: $sum: "$pop" , $match: totalPop: $gte: 10 * 1000 * 1000 As far as "losi…

"On your home page you imply that you can automatically OCR arbitrary handwritten receipts into an analyzable format. No one can do that. That is your problem" Jeez, lay off the confrontational tone. He doesn't say anything about OCR. Maybe he's using humans to do data entry? In any event, it's completely irrelevant to the topic of databases.

I notice you ignored all of my several very specific points directly related to his issues with the database system and your only comment was a criticism about the tone you perceived.

OK, maybe he is using humans to do data entry. The home page to me implies that the process is automatic, but I guess it doesn't rule out the possibility of humans doing data entry when he says 'tag and categorize'. But if he is using humans to do data entry instead of some automatic OCR, that is still his main business problem, rather than MongoDB. The application is relevant to the database discussion, and Hacker News is about all aspects of startups.

Re: Why I Migrated Away From MongoDB

#80

Post by nashequilibrium »

Taking all these comments into account, if startups have to prototype quickly while trying to find market fit, does it makes sense to start off using something like mongodb but with the plan to migrate to another database when you business starts growing? The database space is so confusing right now. It seems like Postgres is the safest choice and i also like this post fron Adam D'Angelo - 'http://www.quora.com/Quora-Infrastructure/Why-does-Quora-use...
Post reply on HN