Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

131–140 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#131

While we are here. I know HN hates mongo. But lemme ask this question, what are the things in which mongo is better than postgres?

Subtle data loss...

The only database we lost data with is Mongo. We had support at the time but even them could not recover. Migrated to Postgres the next month and never looked back.

Re: Bye Bye Mongo, Hello Postgres

#132
post #112

I'm not sure why a database is required at all. Surely the main advantage is to support tables and joins. If you just want a key-value store surely S3 or similar would be simplest. Do they search the data? Surely Cassandra is designed for this. Any ideas?

Cassandra is more of a DB than Mongo is. Dynamodb is basically the same, and they wanted the hosted version so their approach is reasonable.

They definitely want more than a key value store. They at least want date and content indexing, editor, author, etc. I imagine they probably have a few internal requirements as well wrt analytics, tooling and more. S3 doesn't work for all this and another user's suggestion that this is all treatable like static content sounds way off to me. The best you can do is pregenerate some of the HTML.

Re: Bye Bye Mongo, Hello Postgres

#133
post #4
post #3

I’m not really sure they articulated why they had to switch very clearly. They didn’t like managing Mongo. They said they couldn’t use Mongo’s hosted solution BUT they switch to hosted Postgres. Why not just overcome the limitations preventing them from switching to hosted MongoDB?

It looks like they wanted to run Mongo inside their own Amazon account- under their direct control. AWS could do that with Postgres while MongoDB couldn't.

What's wrong with running a EC2 instance with Mongo in it?

Re: Bye Bye Mongo, Hello Postgres

#134

Seems like there's one of these articles every week. "Goodbye {Postgres|Mongo|MySQL}, Hello {MySQL|Mongo|Postgres}"

When was the last "Goodbye X, Hello Mongo" article?

Not sure, but there was Goodbye MySQL, Hello Postgres and Goodbye Postgres, Hello MySQL (same company did it actually) for sure. It is kind of weird why engineers are so obsessed with tools though.

Re: Bye Bye Mongo, Hello Postgres

#135
post #125

Earlier quoted context omitted.

2.3m content items is tiny. So is 30m. You're at least 1-2 magnitudes away from something that will start to bother postgres. Anything before that is likely to be an index or IOPS issue.

On that note, one thing that the article points out is that managing the MongoDB setup was a full time job.....although managing Postgres will probably be as well. It would be easy and understandable to try to make it static and not needing attention, but that is a recipe for bad times. Having issues accessing Xmillion rows seems like something that would be caught by someone focused on performance of the database fu…

RDS is really good at not having to be managed tbh, at the guardian's scale. And when shit does happen they get to be able to throw money at AWS's premium support. I definitely think their move makes a ton of sense there.

Re: Bye Bye Mongo, Hello Postgres

#136
post #35

Specific technology choices aside, this was an incredible write-up of their migration process: thorough, organized, readable prose about a technical topic. It is helpful to read how other teams handle these types of processes in real production systems. Perhaps most refreshing is the description of choices made for the various infrastructure pieces, because it is reasonable and real-world. Blog posts so often describ…

The BBC's Online and R&D departments have very interesting blogs, if you like this sort of thing.

http://www.bbc.co.uk/blogs/internet

https://www.bbc.co.uk/rd

Re: Bye Bye Mongo, Hello Postgres

#137

Fantastic write up article. A few things stood out at me. In no particular order: - Going with Mongo in the first place cost them dearly. CMS are a weird application for schemaless. Not necessarily wrong, but definitely weird. I wonder if they would benefit from moving to more structured schema and I'm willing to bet a lot of the migration complexity comes from that in the first place. - God damn that is a long migra…

> CMS are a weird application for schemaless.

That depends. If the CMS is managing documents that have flexible structures that are tree-like, it might not be such a horrible idea to model that structure in a document instead of relationally.

Re: Bye Bye Mongo, Hello Postgres

#139

While we are here. I know HN hates mongo. But lemme ask this question, what are the things in which mongo is better than postgres?

Subtle data loss...

I'm not trying to cover mongo but I hear stability has improved and random data loss is an old story.

Re: Bye Bye Mongo, Hello Postgres

#140

Earlier quoted context omitted.

Your architectural choices are puzzling to me - Rails/Django -- I've rescued more bad Django apps than I can count.

Exactly, I have moved companies from random web framework + random database to static site generators + CDN with high rate of success too. No point of using Rails/Django like stuff unless you have an extremely good case to, which is certainly not the Guardian use case.

Genuine question - because I'm currently doing CPR on an old Rails app with Mongo - what do you see as a "good use case" for Rails/Django and similar frameworks?
Post reply on HN