Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

301–310 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#301

Post by [deleted] »

nexuist wrote: post #291
AlexCoventry wrote: post #288

Earlier quoted context omitted.

It's a bit disturbing to me that they seem to be using AWS for confidential editorial work. > Due to editorial requirements, we needed to run the database cluster and OpsManager on our own infrastructure in AWS rather than using Mongo’s managed database offering.

>Since all our other services are running in AWS, the obvious choice was DynamoDB – Amazon’s NoSQL database offering. Unfortunately at the time Dynamo didn’t support encryption at rest. After waiting around nine months for this feature to be added, we ended up giving up and looking for something else, ultimately choosing to use Postgres on AWS RDS.

[deleted]

Re: Bye Bye Mongo, Hello Postgres

#302

Post by AlexCoventry »

nexuist wrote: post #291
AlexCoventry wrote: post #288

Earlier quoted context omitted.

It's a bit disturbing to me that they seem to be using AWS for confidential editorial work. > Due to editorial requirements, we needed to run the database cluster and OpsManager on our own infrastructure in AWS rather than using Mongo’s managed database offering.

>Since all our other services are running in AWS, the obvious choice was DynamoDB – Amazon’s NoSQL database offering. Unfortunately at the time Dynamo didn’t support encryption at rest. After waiting around nine months for this feature to be added, we ended up giving up and looking for something else, ultimately choosing to use Postgres on AWS RDS.

Anyone who gets control of the live server can still read a database, even if it encrypts its storage.

Re: Bye Bye Mongo, Hello Postgres

#303

Post by 6t6t6t6 »

StreamBright wrote: post #120
thejerz wrote: post #115

Their architectural choices are puzzling to me: 1) Why use Scala to write (a relatively simple) internal CMS? 2) Why use a clustered database for 2 million records? 3) Why write your own proxy? (in Akka, none the less) 4) Why would you migrate articles from Mongo to Postgres using a script that runs overnight in screen? The Guardian is, prima facie, a Wordpress blog. A simpler architecture would be: 1) Any CRUD web f…

I am not sure why you think that Rails or Django or any bloated web frameworks are necessary. The most trivial way to implement a website for high traffic is simple static content generator (like Jekyll) and use a CDN. There is incredible amount of CPU wasted on rendering the same exact content for every request. The content of the articles never (or very rarely) changes so you can put it in a CDN. CRUD, DMS, RDBMS a…

> bloated web frameworks

A bloated web framework makes your code simpler because there are many things that you don't need to reimplement by yourself.

The problem is when developers use a framework without understanding it well and:

1) Reimplement in the code features that are already present in the framework. 2) Fight against the framework because their business needs conflict with the conventions choosen by the framework. 3) Fight against the framework because they don't agree in the way the framework solved a particular problem and want to solve it their way.

In both cases, the origin of the issues is not the framework itself.

> The most trivial way to implement a website for high traffic is simple static content generator

Agree. But in any case, those kinds of sites are not hard to catch neither.

Re: Bye Bye Mongo, Hello Postgres

#304

Post by alexandre_m »

grumpydba wrote: post #189
alexandre_m wrote: post #91

If anyone is considering doing something similar today, I'd suggest to take a look at the recent FoundationDB Document layer which speaks MongoDB v3 ( https://foundationdb.github.io/fdb-document-layer/ ). I haven't tried it yet, and there are some limitations at the moment, but I find it extremely promising.

I would advise not to use such a baroque technology unless IT is your main activity. Most of the times a boring database is a better fit. Not everybody can be an early adopter, especially on the software which stores what is arguably your most precious asset.

I agree with the boring database, it's most often the good choice.

Elaborating on my comment though, it would make sense to consider a drop-in replacement for a technology for which you've already invested yourself, but got bitten enough time to think about moving away to a different model (operational burden, data loss, etc).

I'm not sure if "baroque" is a good qualifier for FDB.

It's recent in the open source community, but has been running in production for many companies (including Apple) for some years. It's based on sound architecture, compare to many others.

Re: Bye Bye Mongo, Hello Postgres

#305

Post by aprdm »

Would be interesting to share the different in ops cost (year support paid aside), for example, do you need a beefier server now? more servers? less? size on disk? IOPS of servers? also, any different in your metrics from an end user perspective? e.g: time to publish an article.

Re: Bye Bye Mongo, Hello Postgres

#306

Post by grogenaut »

AlexCoventry wrote: post #302
nexuist wrote: post #291

Earlier quoted context omitted.

>Since all our other services are running in AWS, the obvious choice was DynamoDB – Amazon’s NoSQL database offering. Unfortunately at the time Dynamo didn’t support encryption at rest. After waiting around nine months for this feature to be added, we ended up giving up and looking for something else, ultimately choosing to use Postgres on AWS RDS.

Anyone who gets control of the live server can still read a database, even if it encrypts its storage.

Encryption at rest is still important as it closes off a few attack/loss vectors: mis-disposed hard drives, re-allocated hosts. I'm probably missing a few others.

Re: Bye Bye Mongo, Hello Postgres

#307

Post by riku_iki »

user5994461 wrote: post #220
riku_iki wrote: post #186

Earlier quoted context omitted.

mongo is shared nothing architecture, which can linearly scale data-wise, postgres is not and can't.

bullshit. It's shared everything. if you have a 3 nodes cluster for redundancy, the 3 nodes keep the exact same data with only one node accepting reads and writes. It's literally the opposite of linear scalability.

in sharded setup each shard contains subset of data, each node doesn't contain all data: https://docs.mongodb.com/manual/core/sharded-cluster-shards/

You are talking about replica sets, which is different concept.

Re: Bye Bye Mongo, Hello Postgres

#308

Post by AlexCoventry »

grogenaut wrote: post #306
AlexCoventry wrote: post #302

Earlier quoted context omitted.

Anyone who gets control of the live server can still read a database, even if it encrypts its storage.

Encryption at rest is still important as it closes off a few attack/loss vectors: mis-disposed hard drives, re-allocated hosts. I'm probably missing a few others.

Yeah, but it doesn't really address my concern.

Re: Bye Bye Mongo, Hello Postgres

#309

Post by Noumenon72 »

KaiserPro wrote: post #169
pjmlp wrote: post #42

Earlier quoted context omitted.

Fashion driven development and not understanding how to actually make use of SQL.

At "large financial news company" we had a "designed for the CV" tag that applied to stupid architectural decisions (of which there were many) One of the biggest and most expensive was using Cassandra to store membership details. Something like 4 years of work, by a team of 40, wasted by stupid decisions. They included: o Using Cassandra to store 6 million rows of highly structured, mostly readonly data o hosting it…

"real tin"?

Re: Bye Bye Mongo, Hello Postgres

#310

Post by kikki »

Thaxll wrote: post #225

It's not a great article tbh, it's well written but it shows the clear lack of knowledge running a backend. The title should be "we didn't know what we were doing so we switched to a managed DB" I mean yeah who knew that blocking NTP therefore time drifting would break everything... For those criticizing MongoDB, Fortnite generates $3B/year and runs on MongoDB, you should tell them it's a mistake and that they should…

Epic have also had at least 2 major outages directly related to MongoDB

https://www.epicgames.com/fortnite/en-US/news/postmortem-of-...

https://www.epicgames.com/fortnite/en-US/news/postmortem-of-...

Post reply on HN