Go Cry on Somebody Else's Shoulder: MongoDB is fine • Hacker News Board

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#21

Post by andrewf » Mon, Nov 07, 2011, 9:09 AM UTC

Can somebody recommend a database with an API like Mongo's, but performance and durability more like Postgresql or Oracle's?

What I want to do is throw semi-structured JSON data into a database, and define indexes on a few columns that I'd like to do equality and ranged queries on. Mongo seems ideal for this, but I don't needs its performance, and want durability and the ability to run the odd query which covers more data than fits into RAM, without completely falling over.

Right now, the alternative is to do something like the following in Postgres, and have the application code extract a few things from the JSON and duplicate them into database columns when I insert data.

  CREATE TABLE collected_data(
    source_node_id TEXT NOT NULL,
    timestamp INTEGER NOT NULL,
    json_data TEXT);
  CREATE INDEX collected_data_idx ON collected_data(source_node_id, timestamp);

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#22

Post by Luyt » Mon, Nov 07, 2011, 9:21 AM UTC

smacktoward wrote: post #14

pilif wrote: post #4

In my opinion, there is but one feature that a database really must have: whatever data I write into it, if I don't delete it, I want to read it back - unaltered (preferably without needing at least three machines, but I'm willing to compromise) Software that cannot provide this single feature just isn't something I would call database. If it's unsafe default configurations or just bugs. I don't care. Between these t…

But wouldn't you rather pin your mission-critical data to a table and give it a night it will never forget? :-D http://www.youtube.com/watch?v=b2F-DItXtZs

And with transcript: http://www.mongodb-is-web-scale.com/

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#23

Post by spectre » Mon, Nov 07, 2011, 10:02 AM UTC

josephcooney wrote: post #13

Anecdotes like the one from the article that ended with "The one thing that didn’t flinch was MongoDB" don't convince me one bit. When something else between the end user is a bottleneck it would be silly to assume that is the only problem in the entire system. Who is to say that if the load balancers hadn't been configured differently, or higher spec'd that their MongoDB wouldn't have become a smoldering crater?

While anecdotal evidence is always suspect, remember that the case in the article is MongoDB's optimal use case. That being of being extremely read heavy (There is no indication that they did more than one write that day).

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#24

Post by socratic » Mon, Nov 07, 2011, 10:10 AM UTC

andrewf wrote: post #21

Can somebody recommend a database with an API like Mongo's, but performance and durability more like Postgresql or Oracle's? What I want to do is throw semi-structured JSON data into a database, and define indexes on a few columns that I'd like to do equality and ranged queries on. Mongo seems ideal for this, but I don't needs its performance, and want durability and the ability to run the odd query which covers more…

This may be what you mean, but a common approach seems to be to create extra tables to create sort-of-implicit-schema-less-indexes, e.g.,:

http://news.ycombinator.com/item?id=496946

A PostgreSQL-specific alternative might be to write triggers in one of the provided procedural languages to turn your JSON into something indexed or materialized elsewhere.

Do either of those work for you?

Also, purely out of curiosity, do you have a design reason for only wanting to store schema-less JSON, or have you just been burned by slow database migrations in the past?

There seems to be a big community of people who really want to reject schema and use JSON for everything, and I'm really curious if they (a) don't understand relational databases, (b) are getting some surprising productivity gains somehow, (c) have been burned by slow database migrations in the past, or (d) some other reason.

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#25

Post by ubernostrum » Mon, Nov 07, 2011, 10:27 AM UTC

cmer wrote: post #9

Mongo is fine until it's not. It's been fine for us for many months, but once you hit its limitations, it's pretty horrible. We're in this situation right now and we're seriously considering moving back to MySQL or Postgres. Basically, "it doesn't scale" unless you throw tons of machines/shards at it. Once they fix a few of their main issues such as the global write lock and fix many of the bugs, it could become an o…

Mongo is fine until it's not.

My experience is that this is true of every database system (relational or non-). The thing is that they all break in different ways at different points, and so the smart thing to do is make choices based on that information.

The stupid thing to do is write blog posts about how Software Package X sucks and nobody should use it for anything.

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#26

Post by andrewf » Mon, Nov 07, 2011, 10:43 AM UTC

socratic wrote: post #24

andrewf wrote: post #21

Can somebody recommend a database with an API like Mongo's, but performance and durability more like Postgresql or Oracle's? What I want to do is throw semi-structured JSON data into a database, and define indexes on a few columns that I'd like to do equality and ranged queries on. Mongo seems ideal for this, but I don't needs its performance, and want durability and the ability to run the odd query which covers more…

This may be what you mean, but a common approach seems to be to create extra tables to create sort-of-implicit-schema-less-indexes, e.g.,: http://news.ycombinator.com/item?id=496946 A PostgreSQL-specific alternative might be to write triggers in one of the provided procedural languages to turn your JSON into something indexed or materialized elsewhere. Do either of those work for you? Also, purely out of curiosity, d…

All of the above would work, but feels less than ideal. I'm pretty comfortable using a well-schema'd relational database to manage data, but I don't think it fits something I'm working on atm.

I'm collecting and parsing data from a few different types of sources (think: some web page scrapers, Twitter, RSS feeds) for later analysis. I want an intermediate data store where I can throw all of the data together for querying in the short term (within days).

Some of the features I extract from it will probably be stored for longer-term use in a regular database. The JSON itself I expect won't ever be referred to in the long term.

If I can think of a new piece of data I might want to look at, it's very appealing to be able to just print it out in one of the data-gathering programs, without having to touch the entire stack top-to-bottom, deploy a new schema, etc.

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#27

Post by bigfun » Mon, Nov 07, 2011, 10:48 AM UTC

Actually, the default engine for MySQL 5.5 and later is InnoDB, not MyISAM (for some reason :>).

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#28

Post by zobzu » Mon, Nov 07, 2011, 10:59 AM UTC

I love articles that start with attacks in the title, specially cheap ones such as "go cry elsewhere" instead of giving real arguments.

6th grade all over again.

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#29

Post by zobzu » Mon, Nov 07, 2011, 11:03 AM UTC

ubernostrum wrote: post #25

cmer wrote: post #9

Mongo is fine until it's not. It's been fine for us for many months, but once you hit its limitations, it's pretty horrible. We're in this situation right now and we're seriously considering moving back to MySQL or Postgres. Basically, "it doesn't scale" unless you throw tons of machines/shards at it. Once they fix a few of their main issues such as the global write lock and fix many of the bugs, it could become an o…

Mongo is fine until it's not. My experience is that this is true of every database system (relational or non-). The thing is that they all break in different ways at different points, and so the smart thing to do is make choices based on that information. The stupid thing to do is write blog posts about how Software Package X sucks and nobody should use it for anything.

And everything is wonderful! Complains and criticism should be removed from the world. Ain't it?

Re: Go Cry on Somebody Else's Shoulder: MongoDB is fine

#30

Post by ubernostrum » Mon, Nov 07, 2011, 11:09 AM UTC

zobzu wrote: post #29

ubernostrum wrote: post #25

Earlier quoted context omitted.

Mongo is fine until it's not. My experience is that this is true of every database system (relational or non-). The thing is that they all break in different ways at different points, and so the smart thing to do is make choices based on that information. The stupid thing to do is write blog posts about how Software Package X sucks and nobody should use it for anything.

And everything is wonderful! Complains and criticism should be removed from the world. Ain't it?

And everything is wonderful! Complains and criticism should be removed from the world.

Nice job putting words in my mouth.

Look, I know it's fashionable right now to hate on Mongo for whatever reason, but the simple fact is that everything has a breaking point. Saying "I ran into this product's breaking point, therefore nobody should ever use it" (the gist of many of the recent posts) is frankly stupid; instead, we should be asking when and how and why something breaks when evaluating it, since that'll give us an idea of what fits specific use cases.

But maybe that's too practical.