Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

121–130 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#121

Post by acjohnson55 »

csytan wrote: post #68
nemild wrote: post #30

The Guardian example was heavily used by MongoDB as a case study to pitch their database to others in 2011: https://www.mongodb.com/customers/guardian https://www.mongodb.com/presentations/mongodb-guardian https://www.slideshare.net/tackers/why-we-chose-mongodb-for-... And reupping my previous, three-part series on MongoDB: On MongoDB NoSQL databases were the future. MongoDB was the database for "modern" web engineer…

I think you're asking the wrong question. The question should be: How did MongoDB become so successful? IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. MongoDB was the natural choice because they excelled at being accessible to devs who were already familiar with Javascript and JSON. Not only that, their marketing/outreach efforts were also…

> When was the last time you saw a Postgres rep at a college tech event?

Is a Postgres rep a thing?

Re: Bye Bye Mongo, Hello Postgres

#122

Post by StreamBright »

mattbillenstein wrote: post #119
thejerz wrote: post #115

Their architectural choices are puzzling to me: 1) Why use Scala to write (a relatively simple) internal CMS? 2) Why use a clustered database for 2 million records? 3) Why write your own proxy? (in Akka, none the less) 4) Why would you migrate articles from Mongo to Postgres using a script that runs overnight in screen? The Guardian is, prima facie, a Wordpress blog. A simpler architecture would be: 1) Any CRUD web f…

Your architectural choices are puzzling to me - Rails/Django -- I've rescued more bad Django apps than I can count.

Exactly, I have moved companies from random web framework + random database to static site generators + CDN with high rate of success too. No point of using Rails/Django like stuff unless you have an extremely good case to, which is certainly not the Guardian use case.

Re: Bye Bye Mongo, Hello Postgres

#123

Post by anyfoo »

mooreds wrote: post #65
anyfoo wrote: post #44

Earlier quoted context omitted.

> By what metrics can we say the product is one worthy of passing a HN smell test? Common sense and formal education? I'm sorry, I'm aware of how incredibly snarky and arrogant that sounds, but in this case I always struggled to comprehend how MongoDB, or most of "NoSQL" in general, was considered viable to begin with. "Schemaless" just immediately means that instead of the database keeping consistency, you now essen…

> "Schemaless" just immediately means that instead of the database keeping consistency, you now essentially have to do all your type and constraint checking in the application... This is so true. Some component has to maintain integrity of the data. Should it be the application developer? Or the database software team? I know which one I'd choose, and which one is focused on data integrity and not business logic.

And it’s not just who you trust, it’s also at what layer you want to enforce your integrity and consistency. Not letting the db do that means that every part of the application that touches the database needs to do that enforcement. Even if you abstract that away into a strict layer directly above the database—which essentially means you’ve just implemented parts of a database on top of a database—you have to essentially invent and implement the methods and languages to do so yourself. At which point you’re pretty much guaranteed to be much worse of then if you had just let the DB do it. This might mean handling unexpected errors if you have a bug in your application logic, but that’s usually orders of magnitude better than introducing hard to fight inconsistencies.

Re: Bye Bye Mongo, Hello Postgres

#124

Post by h1d »

sitharus wrote: post #88
RDundon wrote: post #49

> “But postgres isn’t a document store!” I hear you cry. Well, no, it isn’t, but it does have a JSONB column type, with support for indexes on fields within the JSON blob. Interesting. I didn't know you could make indexes for things /within/ the JSON.

With a GIN index and a JSONB column you can index the entire json document. It's the same index type as Postgres uses for the full text search, but not super efficient. You can also project out specific subsets of the document to an index and Postgres will use them.

I take it this is Postgres specific and MySQL can't index JSON items.

Re: Bye Bye Mongo, Hello Postgres

#125

Post by sb8244 »

scrollaway wrote: post #90
mrsuprawsm wrote: post #80

>“But postgres isn’t a document store!” I hear you cry. Well, no, it isn’t, but it does have a JSONB column type, with support for indexes on fields within the JSON blob. > approximately 2.3m content items. I had a previous project where we did a similar thing (except with HSTORE instead of JSONB) and it exploded rather dramatically (very simple queries took multiple minutes or timed out entirely) after around 30m ro…

2.3m content items is tiny. So is 30m. You're at least 1-2 magnitudes away from something that will start to bother postgres. Anything before that is likely to be an index or IOPS issue.

On that note, one thing that the article points out is that managing the MongoDB setup was a full time job.....although managing Postgres will probably be as well. It would be easy and understandable to try to make it static and not needing attention, but that is a recipe for bad times.

Having issues accessing Xmillion rows seems like something that would be caught by someone focused on performance of the database full-time.

Re: Bye Bye Mongo, Hello Postgres

#126

Post by christkv »

The only why I saw was this comment

This is one of the reasons why building on top of the old API wasn’t an option. There was very little separation of concern in the original API and MongoDB specifics could be found even at the controller level. As a result the task of adding another database type in the existing API was too risky.

This seems like the problem was more related to the API layer code quality rather than where the data was stored.

Re: Bye Bye Mongo, Hello Postgres

#127

Post by sparkie »

zzzcpan wrote: post #102
michelpp wrote: post #76

Earlier quoted context omitted.

You cannot be old-school and hype. That's the reason why it is old school. And the people who can't make good database design choices are exactly the kind of people who should be using SQL. Postgres knows how to optimize and plan queries efficiently based on the actual distributions of values in your dataset. These poor choosers should be doing that... by hand? https://www.postgresql.org/docs/11/planner-optimizer.htm…

This is what I'm talking about. You think that some feature that makes performance unpredictable is that important. While it's the last thing you should care about.

What people should care about is the integrity and consistency of the data they're storing. RDBMS do an extremely good job of generalising this problem with reasonable performance, which suits the majority of real world problems. The ideas behind it are understood by most of the industry.

It should be the default choice (or plain filesystem storage), unless you have a specific requirement for something different. In the latter case, this should be people who are well informed about the database choices they're making.

Re: Bye Bye Mongo, Hello Postgres

#128

Post by h1d »

beering wrote: post #27

Did they ever publish the rationale for why they used MongoDB in the first place? They mention having 2.3 million content items. If we assume that they have 100 DB entries for each content item, that's still only 230 million DB items. In that case, was it important to run a sharded cluster vs a typical primary-secondary HA setup? (which they ended up switching to)

Considering they've been using it from 2011 or something, there can't be any good reason but a dumb choice by tech team or management bought Mongo's sales talk without good verification.

Re: Bye Bye Mongo, Hello Postgres

#129

Post by acjohnson55 »

orblivion wrote: post #107
nemild wrote: post #30

The Guardian example was heavily used by MongoDB as a case study to pitch their database to others in 2011: https://www.mongodb.com/customers/guardian https://www.mongodb.com/presentations/mongodb-guardian https://www.slideshare.net/tackers/why-we-chose-mongodb-for-... And reupping my previous, three-part series on MongoDB: On MongoDB NoSQL databases were the future. MongoDB was the database for "modern" web engineer…

I found it a little funny that NoSQL started becoming popular during at least some of the same years that static typing starting becoming popular (again).

I don't see it that way -- I feel like NoSQL's rise was more or less coincident with the adoption of Rails, Django, and Node over Java. The surge in interest in new static languages has mirrored the resurgence of Postgres, right around version 9.4 (and JSONB).

Re: Bye Bye Mongo, Hello Postgres

#130

Post by [deleted] »

thejerz wrote: post #115

Their architectural choices are puzzling to me: 1) Why use Scala to write (a relatively simple) internal CMS? 2) Why use a clustered database for 2 million records? 3) Why write your own proxy? (in Akka, none the less) 4) Why would you migrate articles from Mongo to Postgres using a script that runs overnight in screen? The Guardian is, prima facie, a Wordpress blog. A simpler architecture would be: 1) Any CRUD web f…

[deleted]
Post reply on HN