Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I Migrated Away From MongoDB

svs.io

Post reply on HN
Print view View original

111–120 of 213 posts

Re: Why I Migrated Away From MongoDB

#111

Post by ilaksh »

nnnnnnnn wrote: post #107
ilaksh wrote: post #79

Earlier quoted context omitted.

I notice you ignored all of my several very specific points directly related to his issues with the database system and your only comment was a criticism about the tone you perceived. OK, maybe he is using humans to do data entry. The home page to me implies that the process is automatic, but I guess it doesn't rule out the possibility of humans doing data entry when he says 'tag and categorize'. But if he is using h…

It's because I have nothing to say about the database stuff. Why are you so adversarial? I'm not here to cross swords with you; I don't have an opinion on the matter. But I did notice your rudeness, and you're now being rude to me. Totally uncalled for.

I was not rude to the poster. I corrected him as far as his misguided complaints about MongoDB and the main problem for his business. That is the only way to help him.

I was definitely not rude to you either. You suggested that my comment was irrelevant, and I pointed out that my comment had a lot of relevant content in it whereas by your own definition of relevance your comment had none.

Re: Why I Migrated Away From MongoDB

#112

Post by taligent »

otterley wrote: post #65
trafficlight wrote: post #61

Earlier quoted context omitted.

>> "You have to put the whole dataset on RAM?" I'm pretty new to the whole database thing, but how is MongoDB different from Postgres or Mysql in this respect? In a traditional database, the data is pulled directly from the hard drive. Why does Mongo suffer a performance hit and Mysql doesn't?

Because MongoDB mmap(2)s its backing stores into its process memory space. It's a naive approach to persistence - it's very fast and simple, but if you overcommit (i.e. you store more in the database than you have memory available), page-thrashing results. MySQL's InnoDB table engine, on the other hand, uses direct I/O (in the recommended scenario) and manages the buffer pool independently of the kernel. Its buffer p…

It's not naive. It's a well understood and communicated design choice.

Remember that completely in memory databases are going to be how we all store our data in the decades to come and are already the standard for those who care about speed.

Re: Why I Migrated Away From MongoDB

#113

Post by gaius »

taligent wrote: post #101
gaius wrote: post #48

Earlier quoted context omitted.

The thing with the NoSQL guys is that many of them seem not to be in a position to make an educated comparison. For example, an, uhh, enthusiastic MongoDB advocate recently informed me that MongoDB was superior to Oracle because in Oracle you had to poll a table to see if it changed. Except, no, that isn't actually true: http://docs.oracle.com/cd/B19306_01/appdev.102/b14251/adfns_... - and that document is from 2005.…

This kind of rubbish really needs to stop. Just because you don't agree with or understand their choices does not mean that the majority of "NoSQL guys" are ignorant or uneducated. Some of the biggest companies e.g. Twitter, Foursquare, Google, Amazon all rely on NoSQL. The real issue I see is that by dismissing NoSQL as only for fools RDBMS developers are failing to see why they are popular to begin with. Take Postg…

I think we did "sharding" with relational databases... Back in the 80s. Then we got fast hash joins and partitioning and it turns out that the disadvantages of sharding just aren't worth it. The NoSQL crew will figure this out too around 2030 :-p

Re: Why I Migrated Away From MongoDB

#114

Post by taligent »

gregjor wrote: post #90
blaines wrote: post #82

Earlier quoted context omitted.

Respectfully, I believe your comment is not very open minded. I believe the obvious answer is "Of course there's a use case!", just like there's obviously a use case for RDBMS, NoSQL (Mongo, Cassandra, BigTable, Redis, etc) serves a purpose. I'll give you a couple examples. Google stores a huge amount of data in BigTable [1] which is fundamental for the operation of Gmail at scale. Another, Facebook initially used Ca…

I'll add that email and Facebook postings and Instagram photos don't require the same kind of data consistency and durability that medical records, bank transactions, or rail car movements require. If Google loses some old emails or Facebook loses the last thousand "likes" on a Jersey Shore fan page that's not much of a catastrophe, and they don't guarantee their data anyway. For those use cases Mongo or Cassandra et…

This is complete and utter nonsense.

Facebook and Instagram itself I imagine would care VERY much if data went missing as it would mean not just (a) user data going missing but (b) potentially huge inconsistencies in the social graph.

Unless you have some evidence that those companies do not care about data durability ?

Re: Why I Migrated Away From MongoDB

#115

Post by svs »

ilaksh wrote: post #76

On your home page you imply that you can automatically OCR arbitrary handwritten receipts into an analyzable format. No one can do that. That is your problem, not MongoDB. As far as aggregation, use the new Aggregation Framework http://docs.mongodb.org/manual/tutorial/aggregation-examples... : db.zipcodes.aggregate $group: _id: "$state" totalPop: $sum: "$pop" , $match: totalPop: $gte: 10 * 1000 * 1000 As far as "losi…

We don't do OCR, we have people who scan the documents if you snailmail them to us (or you can upload the scans directly to your account, email them in etc.) and then convert these images into queryable data with special emphasis on receipts (also invoices, bills, bank withdrawals, etc) and business cards. It's a service that our customers are happy to pay for, so I think I'm not quite getting your point about why this model won't work.

Had it been any one the issues I've pointed out, I would be quite happy to work around them. I do that routinely with other pieces of technology that do 95% of what I want. However, the hits kept coming. At the same time, I was using Postgres on another project and the question was begging to be asked - if I am going to run extra services like Lucene and do extra work to achieve full text search, what am I getting in return? And I am sorry to say - case insensitive search is quite a basic feature.

As for the 1111 issue - I admit it can be fixed. But the fact still remains - there's no way I would face this problem with Postgres. So why not just switch?

Postgres is a really nicely thought out and well executed database. And don't get me wrong - I am not building a system designed to operate at stratospheric scale. That would be premature in my opinion.

Thanks for taking the trouble to comment.

Re: Why I Migrated Away From MongoDB

#116

Post by taligent »

gaius wrote: post #113
taligent wrote: post #101

Earlier quoted context omitted.

This kind of rubbish really needs to stop. Just because you don't agree with or understand their choices does not mean that the majority of "NoSQL guys" are ignorant or uneducated. Some of the biggest companies e.g. Twitter, Foursquare, Google, Amazon all rely on NoSQL. The real issue I see is that by dismissing NoSQL as only for fools RDBMS developers are failing to see why they are popular to begin with. Take Postg…

I think we did "sharding" with relational databases... Back in the 80s. Then we got fast hash joins and partitioning and it turns out that the disadvantages of sharding just aren't worth it. The NoSQL crew will figure this out too around 2030 :-p

I guess so will the likes of Facebook, Google, Twitter, eBay etc.

Must be amazing to know more about databases than all of the world's biggest companies put together.

Re: Why I Migrated Away From MongoDB

#117

Post by gaius »

taligent wrote: post #116
gaius wrote: post #113

Earlier quoted context omitted.

I think we did "sharding" with relational databases... Back in the 80s. Then we got fast hash joins and partitioning and it turns out that the disadvantages of sharding just aren't worth it. The NoSQL crew will figure this out too around 2030 :-p

I guess so will the likes of Facebook, Google, Twitter, eBay etc. Must be amazing to know more about databases than all of the world's biggest companies put together.

Why? The guys at FB or Google are just guys like you and me, don't believe the hype, they are not superheroes or gurus despite their much talked about interview process (the Google interviews I did were a walk in the park compared to GS, btw). Some will know more than me sure, but some'll know less.

Re: Why I Migrated Away From MongoDB

#118

Post by mrinterweb »

I find this article to be more a reflection of a NoSQL newbie's failed foray with a document database that later realized that the grass is not as green as originally perceived. The developer realized that he does not like map-reduce and missed not having joins. I don't see how this person's failed experience with MongoDB is a reflection on MongoDB.

I think the recent popularity of MongoDB bashing is maybe a testament to MongoDBs popularity. I'd guess that because MongoDB is probably the closest NoSQL database to a RDBMS with its ad hoc queries, that it is attracting many newcomers.

Re: Why I Migrated Away From MongoDB

#119

Post by frio »

taligent wrote: post #101
gaius wrote: post #48

Earlier quoted context omitted.

The thing with the NoSQL guys is that many of them seem not to be in a position to make an educated comparison. For example, an, uhh, enthusiastic MongoDB advocate recently informed me that MongoDB was superior to Oracle because in Oracle you had to poll a table to see if it changed. Except, no, that isn't actually true: http://docs.oracle.com/cd/B19306_01/appdev.102/b14251/adfns_... - and that document is from 2005.…

This kind of rubbish really needs to stop. Just because you don't agree with or understand their choices does not mean that the majority of "NoSQL guys" are ignorant or uneducated. Some of the biggest companies e.g. Twitter, Foursquare, Google, Amazon all rely on NoSQL. The real issue I see is that by dismissing NoSQL as only for fools RDBMS developers are failing to see why they are popular to begin with. Take Postg…

This is a valid point. I dismissed NoSQL for years, and still (largely) do (I prefer to have a schema, documented data structures, deal with relational data etc.)... but I finally gave Mongo a go the other day, and am actually quite impressed. For a simple endpoint I can throw arbitrary persistent data in (rather than the filesystem), it's let me get a couple of proof-of-concept projects out the door much quicker.

Note that I'm not intending to comment on its scalability or how appropriate it is for other projects, just that I agree with you: dismissing it out of hand is hubris.

Re: Why I Migrated Away From MongoDB

#120

Post by jeltz »

voidr wrote: post #105

Relational databases are awesome if you are not dealing with huge amounts of data that your current hardware can't handle the relational way. There are some cases where you have a ridiculous amount of data(rows) and you simply can't store that in a relational database and you are happy to live without the benefits of relational databases. If you have millions of rows, you are probably better off with something like M…

A relational database at very modest hardware can handle millions of rows, and with a solid database, good hardware and a DBA who knows his shit you can handle billions.

OpenStreetMap has over a billion nodes stored in a PostgreSQL database.

http://www.openstreetmap.org/stats/data_stats.html

My point is that you can get very far with a classic relational database before you have to scale vertically.

Post reply on HN