Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“MongoDB is dead. Long live Postgresql”

github.com

Post reply on HN
Print view View original

141–150 of 160 posts

Re: “MongoDB is dead. Long live Postgresql”

#141

Post by jacques_chester »

cullenking wrote: post #113
jlouis wrote: post #110

Earlier quoted context omitted.

670 gigabytes is a puny database size. You should be able to press so much power through a system with a disk system like the one you have. I would seriously consider a Postgres setup on a data set of that size. Additionally, I would probably just store the JSON data directly inside postgresql.

See comment below. Definitely a small database size. You hit the nail on the head. It's going to grow fast this next year, and I'd like to put off sharding as long as possible, hence gzipped storage of the data. postgis isn't a good fit for the data we store in full fidelity, since it's not just geo data but also sensor data (heartrate, cadence, power in watts, temperature etc). However I'll be storing a point reduce…

> postgis isn't a good fit for the data we store in full fidelity, since it's not just geo data but also sensor data (heartrate, cadence, power in watts, temperature etc).

Dumb question: why not?

Re: “MongoDB is dead. Long live Postgresql”

#142

Post by jacques_chester »

mikegioia wrote: post #85
endijs wrote: post #65

I'm no MongoDB expert, but recently started to look into this db. Can anyone tell me (from experience, not from promo materials) - for which use cases MongoDB is good fit and for which ones it's not? It's clear that it can't fit for everyone. That's why it would be good to know in advance, for what it most likely to find and for what it's most likely not to fit.

It has the benefits and ease of use of a json document store, it allows you to do SQL style where clauses, it takes about a minute to install and start using, there are a wide range of drivers available for many languages, and it has a simple javascript map/reduce. on the flip side, it implements database level locking, uses more disk/RAM than it probably should, and can start to give you headaches if you try to do a…

> however, a lot of data like "number of teachers in school A" is aggregated and too difficult to run in real time when we render paged results.

Does this mean you're using MongoDB as a kind of query cache? Was there a compelling reason to prefer it to other common caches? Or even building an ETL/DW into your existing database infrastructure?

Re: “MongoDB is dead. Long live Postgresql”

#143

Post by jacques_chester »

jcampbell1 wrote: post #18
seiji wrote: post #3

Earlier quoted context omitted.

PostgreSQL has native json support now. What else is missing? Just a protocol implementation? I'd love to see MongoDB give up and become a PostgreSQL consultancy. Everybody I talk to in the field has the exact same Mongo story: "We love JSON! We use JSON everywhere! We just wanted a DB with native JSON support. We didn't look at the implementation details. We only looked at their marketing. Now we wake up at 3am to f…

It is probably not as simple as "supports json now". Imagine if HN comments were stored as a JSON document: Client A: Read JSON. Client B: Read JSON. Client A: Append new comment to json document. Client B: Append new comment to json document. Client A: Save JSON Client B: Save JSON A's comment will get deleted. My understanding is that Mongo DB does have a way to append a record within a document, but Postgres does…

> I am just saying that I understand that MongoDB has much more sophisticated updates capability than Postgres.

How many isolation levels does MongoDB support?

Re: “MongoDB is dead. Long live Postgresql”

#144

Post by jacques_chester »

this_user wrote: post #21
busterarm wrote: post #10

Earlier quoted context omitted.

I really would love it if somebody could go back in HN-Time and track the data on Mongo posts & comments. I've always been slightly skeptical about it, but it always seemed to me that there was a long love affair with it overall. Then people started voicing their frustrations and the community was divided and now it looks acrimonious for everyone.

Isn't that true for most of those 'flavour of the month' technologies? First a shiny new piece of technology shows up that promises to solve all the issues you have with a mature and widely adopted solution. People get exited and at some point the media picks up on it und starts the hype cycle. More and more decision makers hear about the technology (probably aided by marketing) and decide to adopt it. Implementation…

http://en.wikipedia.org/wiki/Hype_cycle

Re: “MongoDB is dead. Long live Postgresql”

#145

Post by darkarmani »

functional_test wrote: post #118
jeffdavis wrote: post #43

Earlier quoted context omitted.

> They should be doing compactions and are not. https://jira.mongodb.org/browse/SERVER-11763 It looks like compaction is an offline process. That really puts the user between a rock and a hard place.

In a proper production environment, you just compact each slave one at a time because you have a replica set rather than a single instance. Of course, if you aren't replicating your business's production database, you have a whole world of problems.

> In a proper production environment, you just compact each slave one at a time because you have a replica set rather than a single instance.

That's the solution?? That sounds like a workaround in a production environment.

Re: “MongoDB is dead. Long live Postgresql”

#146

Post by jeffdavis »

AlisdairO wrote: post #124
remon wrote: post #67

Earlier quoted context omitted.

This is not true at all. The actual realization the past years is that strictly enforced relationality (is that a word?) and transactions are constructs that are not always or even rarely actually needed. Eventual consistency, schemaless data modelling and so on picked up steam and for good reasons. Every technology that survives the "Oh, new toy!" stage has a place or it wouldn't still exist. It is up to developers…

> Eventual consistency, ...may be possible, but almost always requires domain-specific concurrency-level understanding in your datastore, and is almost always harder to work with than strong consistency. Saying that transactions are 'rarely' needed boggles my mind. Working inside transactions (where feasible, which is in the large majority of situations) vastly simplifies data storage.

Agreed. The first rule of computing is that anything can do everything -- it's just a matter of how challenging it is to implement.

The notion of transactions were invented not for performance, but because they are easier to reason about. So much easier that it often means the difference between a project that never finishes and a project that finishes so early that you have time to spend on optimization and caching.

Re: “MongoDB is dead. Long live Postgresql”

#147

Post by keypusher »

cullenking wrote: post #44

Maybe I am just incredibly lucky, but mongodb has worked fine for ridewithgps.com - we are sitting at 670gb of data in mongo (actual DB size, indexes included) and haven't had a problem. Replica sets have been fantastic, I wish there was another DB out there that did auto-failover as cleanly/easily as mongo does. We've had a few server crashes of our primary, and aside from 1-2 seconds or so of errors as requests com…

> 192gb of ram with 8 RAID10 512gb SSDs probably masks performance issues

I would hope so.

Re: “MongoDB is dead. Long live Postgresql”

#148

Post by p4lindromica »

viraptor wrote: post #23
jcampbell1 wrote: post #18

Earlier quoted context omitted.

It is probably not as simple as "supports json now". Imagine if HN comments were stored as a JSON document: Client A: Read JSON. Client B: Read JSON. Client A: Append new comment to json document. Client B: Append new comment to json document. Client A: Save JSON Client B: Save JSON A's comment will get deleted. My understanding is that Mongo DB does have a way to append a record within a document, but Postgres does…

In that case you can (should) just "SELECT ... FOR UPDATE" in your transaction. This should prevent the issue. Client B will wait with the read until client A commits.

... or you can include an ETag (optimistic lock) in your rows/mongo documents and do a conditional update to ensure no writes are interposed with your reads.

Re: “MongoDB is dead. Long live Postgresql”

#149

Post by sergiosgc »

randomdata wrote: post #45
jtchang wrote: post #34

Earlier quoted context omitted.

> * didn't read the manual > * poor schema > * didn't maintain the database (compactions, etc.) The real world dictates that this happens more often than not. You know why I like Postgres? When I don't read the manual, create a crappy schema, and forgot to maintain the database it STILL seems to work okay.

To be fair, Postgres has automatic vacuuming now, but it is a relatively new feature. Both projects seem to agree that it is not a high-priority item, though there is certainly something to be said about using a mature product, which Mongo is most certainly not. Your comment has made me quite curious to know what people using mature databases of the time were saying about Postgres 19 years ago, when it was roughly th…

There's a huge difference. Postgresql was formally correct first, then it became fast, then it became easy to use. I always had the feeling I could trust it with my data. Postgresql, if wrong, would be wrong towards correctedness and towards data safety, at the expense of speed.

I definitely don't get the same vibe from MongoDB.

Re: “MongoDB is dead. Long live Postgresql”

#150

Post by lmm »

darkarmani wrote: post #145
functional_test wrote: post #118

Earlier quoted context omitted.

In a proper production environment, you just compact each slave one at a time because you have a replica set rather than a single instance. Of course, if you aren't replicating your business's production database, you have a whole world of problems.

> In a proper production environment, you just compact each slave one at a time because you have a replica set rather than a single instance. That's the solution?? That sounds like a workaround in a production environment.

Yes and no. Deleting is a hard problem that many databases don't handle well. Explicitly deciding to punt the problem like this is a much better approach than allowing performance to degrade unpredictably.
Post reply on HN