111–120 of 427 posts
Earlier quoted context omitted.
It's amazing how many insurmountable SQL performance problems can be surmounted by putting things in 3rd normal form.
But denormalization is one strategy to improve the performance of a rdbms.
The Guardian example was heavily used by MongoDB as a case study to pitch their database to others in 2011: https://www.mongodb.com/customers/guardian https://www.mongodb.com/presentations/mongodb-guardian https://www.slideshare.net/tackers/why-we-chose-mongodb-for-... And reupping my previous, three-part series on MongoDB: On MongoDB NoSQL databases were the future. MongoDB was the database for "modern" web engineer…
I found it a little funny that NoSQL started becoming popular during at least some of the same years that static typing starting becoming popular (again).
A few things stood out at me. In no particular order:
- Going with Mongo in the first place cost them dearly. CMS are a weird application for schemaless. Not necessarily wrong, but definitely weird. I wonder if they would benefit from moving to more structured schema and I'm willing to bet a lot of the migration complexity comes from that in the first place.
- God damn that is a long migration. Holy shit. I know tech isn't their core competency but they do seem to have very competent staff. I've worked in places this glacially slow and I get how it can get this bad but it really strikes me as having no right to be. 10 months to migrate .... from the point where they were ready, until they were done. And somehow integration tests got overlooked during all that.
- Close call with dynamodb. Bit of a wtf on waiting for the feature to be implemented for nine months though. I'm sure they have an account manager with aws... I definitely think blocking an internal process on such a fragile externality as a closed process upstream publishing a feature is the wrong move and a red flag. Their migration path would have probably been harder with dynamodb too.
- I feel the pain of their troubleshooting issues on the load testing step. I can completely see how this can happen. That said it also raises a few red flags to me. Letting something as simple as the load testing step get complex enough to require weeks of engineering is ... Eh.
Have more thoughts but I hate typing on mobile. This is a fantastic write up, I love when non-tech companies publish this sort of stuff. And hurrah for postgres.
1) Why use Scala to write (a relatively simple) internal CMS?
2) Why use a clustered database for 2 million records?
3) Why write your own proxy? (in Akka, none the less)
4) Why would you migrate articles from Mongo to Postgres using a script that runs overnight in screen?
The Guardian is, prima facie, a Wordpress blog. A simpler architecture would be:
1) Any CRUD web framework to build the CMS for reporters to draft their articles (Django, Rails, etc). Any basic RDBMS with read replication will do. Or, ditch the webapp entirely and just make a simple Markdown editor that commits to a git repo, a la Prose or Netlify.
2) When a reporter "publishes" an article, generate HTML for it and push to the CDN network. (I can't easily tell by looking at their HTTP headers, but I assume they're doing this already)
Okay, I'm being a little tongue-in-cheek. It's probably not that simple. But, one has to wonder, when you're serving up 100 million static HTML pages a day, if it really has to be this complicated.
Earlier quoted context omitted.
But denormalization is one strategy to improve the performance of a rdbms.
Denormalization as way of dealing with performance issues is like guilottin to cure headache. And more frequently than not the headache is still there even after that. It is true that for some narrow class of analytical workloads 20-25 years ago (behold BW of 199x) the denormalized case performance was better compare with straight non-optimized running of the same queries over normalized schema. Since then, the expon…
Did they ever publish the rationale for why they used MongoDB in the first place? They mention having 2.3 million content items. If we assume that they have 100 DB entries for each content item, that's still only 230 million DB items. In that case, was it important to run a sharded cluster vs a typical primary-secondary HA setup? (which they ended up switching to)
Great blogpost, though; well written, which is always a surprise.
Earlier quoted context omitted.
Couldn't they do this by using Atlas and choosing AWS as the host?
Atlas runs tne hosts inside mongodbs AWS account, we have the same restriant, becuase of client privacy isses and gdpr compliance we can let a 3rd party host the data, amazon is ok beacuse we control tbe data on tbe instances.
Their architectural choices are puzzling to me: 1) Why use Scala to write (a relatively simple) internal CMS? 2) Why use a clustered database for 2 million records? 3) Why write your own proxy? (in Akka, none the less) 4) Why would you migrate articles from Mongo to Postgres using a script that runs overnight in screen? The Guardian is, prima facie, a Wordpress blog. A simpler architecture would be: 1) Any CRUD web f…
Their architectural choices are puzzling to me: 1) Why use Scala to write (a relatively simple) internal CMS? 2) Why use a clustered database for 2 million records? 3) Why write your own proxy? (in Akka, none the less) 4) Why would you migrate articles from Mongo to Postgres using a script that runs overnight in screen? The Guardian is, prima facie, a Wordpress blog. A simpler architecture would be: 1) Any CRUD web f…