Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“MongoDB is dead. Long live Postgresql”

github.com

Post reply on HN
Print view View original

151–160 of 160 posts

Re: “MongoDB is dead. Long live Postgresql”

#151

Post by gizzlon »

pilif wrote: post #46

The title is a bit misleading. This is basically an announcement of a fork of Errbit that has Postgres support. Additionally, the fork was announced as an issue on errbit with no discussion or as an official pull request. I would not consider this good etiquette. If you fork your project (especially without discussing the intention first), adding a bug to the original project isn't a very nice thing to do. An officia…

If you go back and look at it now, you'll see that this is a non-issue:

    @realmyst Will you put up a Pull Request?

    It sounds like MongoDB has no future indeed:   
    http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/



   realmyst commented 19 minutes ago

   @21croissants yes, I will.

Re: “MongoDB is dead. Long live Postgresql”

#152

Post by AlisdairO »

jeltz wrote: post #139
AlisdairO wrote: post #32

Earlier quoted context omitted.

Or you can use serialisable transaction isolation and retry on conflict.

For the simple cases you do not even need that.

Indeed. Depends on whether you need to do a read-update cycle or you can just do it in a single SQL command.

Re: “MongoDB is dead. Long live Postgresql”

#154

Post by 3pt14159 »

cullenking wrote: post #44

Maybe I am just incredibly lucky, but mongodb has worked fine for ridewithgps.com - we are sitting at 670gb of data in mongo (actual DB size, indexes included) and haven't had a problem. Replica sets have been fantastic, I wish there was another DB out there that did auto-failover as cleanly/easily as mongo does. We've had a few server crashes of our primary, and aside from 1-2 seconds or so of errors as requests com…

Which data centre are you hosting in? Even when I was colo'ing back at 500px that much RAM wasn't "cheap". "Fairly priced" is the way I would put it.

Re: “MongoDB is dead. Long live Postgresql”

#155

Post by cullenking »

3pt14159 wrote: post #154
cullenking wrote: post #44

Maybe I am just incredibly lucky, but mongodb has worked fine for ridewithgps.com - we are sitting at 670gb of data in mongo (actual DB size, indexes included) and haven't had a problem. Replica sets have been fantastic, I wish there was another DB out there that did auto-failover as cleanly/easily as mongo does. We've had a few server crashes of our primary, and aside from 1-2 seconds or so of errors as requests com…

Which data centre are you hosting in? Even when I was colo'ing back at 500px that much RAM wasn't "cheap". "Fairly priced" is the way I would put it.

We build our own machines and host them out of a carrier neutral facility in Portland, OR. $200/mo unmetered unlimited 100mbit connection and $1100 for a full rack with redundant 20 amp feeds. It would cost us something like $5k/mo to run our 9 servers on amazon.

Re: “MongoDB is dead. Long live Postgresql”

#156

Post by cullenking »

jacques_chester wrote: post #141
cullenking wrote: post #113

Earlier quoted context omitted.

See comment below. Definitely a small database size. You hit the nail on the head. It's going to grow fast this next year, and I'd like to put off sharding as long as possible, hence gzipped storage of the data. postgis isn't a good fit for the data we store in full fidelity, since it's not just geo data but also sensor data (heartrate, cadence, power in watts, temperature etc). However I'll be storing a point reduce…

> postgis isn't a good fit for the data we store in full fidelity, since it's not just geo data but also sensor data (heartrate, cadence, power in watts, temperature etc). Dumb question: why not?

because it's faster to serve the full fidelity track, with all sensor data for every lat/lng pair, straight from disk as a gzipped file. I can store a point reduced version in postgis for actual geo operations, since that only needs lat/lng pairs, and not at 1hz sampling.

Pulling 21k points (1.8mb of JSON) for a single recording out of disk into memory, turning it into JSON, sending it over the wire to the client, is slow (500ms just to query, pull off disk and turn into JSON). Serving the same data pre-compressed, is 2ms. This isn't even considering what it takes nginx to compress it on the fly for every request.

Re: “MongoDB is dead. Long live Postgresql”

#157

Post by nailer »

functional_test wrote: post #19

Seriously, another case of using Mongo incorrectly? I want to believe all the Mongo hate, but I can't because I always find out that the actual problem was one or more of: * didn't read the manual * poor schema * didn't maintain the database (compactions, etc.) In this case, they hit several: " Its volume on disk is growing 3-4 times faster than the real volume of data it store;" They should be doing compactions and…

The current stable node drivers silently throws away exceptions. Seriously, mongodb inc acknowledge it. Is this also a case of not using mongo correctly?

Re: “MongoDB is dead. Long live Postgresql”

#158

Post by jacques_chester »

cullenking wrote: post #156
jacques_chester wrote: post #141

Earlier quoted context omitted.

> postgis isn't a good fit for the data we store in full fidelity, since it's not just geo data but also sensor data (heartrate, cadence, power in watts, temperature etc). Dumb question: why not?

because it's faster to serve the full fidelity track, with all sensor data for every lat/lng pair, straight from disk as a gzipped file. I can store a point reduced version in postgis for actual geo operations, since that only needs lat/lng pairs, and not at 1hz sampling. Pulling 21k points (1.8mb of JSON) for a single recording out of disk into memory, turning it into JSON, sending it over the wire to the client, is…

So it's because you don't want to recreate the track from point data on each request. I can see why that's advantageous.

OK, new dumb question. It sounds like you're serving gzip'd JSON straight off the disk with nginx (serving gzip'd files direct from disk is one of my favourite nginx features). Where does Mongo come into the picture?

Re: “MongoDB is dead. Long live Postgresql”

#159

Post by mikegioia »

jeltz wrote: post #133
mikegioia wrote: post #85

Earlier quoted context omitted.

It has the benefits and ease of use of a json document store, it allows you to do SQL style where clauses, it takes about a minute to install and start using, there are a wide range of drivers available for many languages, and it has a simple javascript map/reduce. on the flip side, it implements database level locking, uses more disk/RAM than it probably should, and can start to give you headaches if you try to do a…

> to give you a real world example, we use mariadb for storing everything persistently. however, a lot of data like "number of teachers in school A" is aggregated and too difficult to run in real time when we render paged results. to get around that, we use mongo as a document store and use its SQL like querying to generate the paged search results. this lets us sort/filter on the data without having to do everything…

It definitely is possible with PostgreSQL and it's honestly probably better to do with Postgre, but we're using MariaDB for relational data and it was just quicker/easier to dump everything to mongodb.

Re: “MongoDB is dead. Long live Postgresql”

#160

Post by mikegioia »

jacques_chester wrote: post #142
mikegioia wrote: post #85

Earlier quoted context omitted.

It has the benefits and ease of use of a json document store, it allows you to do SQL style where clauses, it takes about a minute to install and start using, there are a wide range of drivers available for many languages, and it has a simple javascript map/reduce. on the flip side, it implements database level locking, uses more disk/RAM than it probably should, and can start to give you headaches if you try to do a…

> however, a lot of data like "number of teachers in school A" is aggregated and too difficult to run in real time when we render paged results. Does this mean you're using MongoDB as a kind of query cache? Was there a compelling reason to prefer it to other common caches? Or even building an ETL/DW into your existing database infrastructure?

Not a query cache so much as a document cache. I'm actually implementing a query cache in Redis which is, imo, the greatest tool on earth :P

To be honest the decision to use mongo as a document cache was made before I realized the scope of our data needs. Mongo was the quickest/easiest solution at the time (early 2012). If I could do this over again, and we actually will be in the next 6 months, I'd either set it up in PostgreSQL/json, or throw everything in a wide column store.

Post reply on HN