Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

341–350 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#341

Post by philjohn »

AlexCoventry wrote: post #288
dmix wrote: post #271

Earlier quoted context omitted.

It’s probably better if they didn’t answer this question...

It's a bit disturbing to me that they seem to be using AWS for confidential editorial work. > Due to editorial requirements, we needed to run the database cluster and OpsManager on our own infrastructure in AWS rather than using Mongo’s managed database offering.

They're using AWS VPC (Virtual Private Cloud) which isn't open to the world (you use a VPN to bridge the VPC into your internal network) and which you can spin up dedicated instances that don't share underlying hardware with other AWS customers.

Re: Bye Bye Mongo, Hello Postgres

#342

Post by glogla »

purple_ducks wrote: post #97
sandstrom wrote: post #84

Earlier quoted context omitted.

Well, we're kind of comparing MongoDB of ~2011 (when The Guardian started using them) with Postgres of today. One major change is that in 2010 you couldn't always run your whole DB in RAM, so there were some real performance benefits with MongoDB. Another difference is that MongoDB was early with great JSON-support. Something that Postgres has since gained. I think there are pros and cons with both. If I'd chose one…

If somebody is teleporting back then, the fact Mongo had a _global_ Mongo instance level lock for writes should be more than enough for people to run screaming. I worked with databases that only had table level locks(not row level) and there were more than enough occasions I cursed the creators. Instance level(& indeed DB level) is insanity unless your DB is a read only DB.

I don't think Mongo was used much by people who would know why instance level lock is bad, or even what it is. And if someone who knew came later, it was too late - Mongo's proprietary querying language is a vendor lock-in.

Re: Bye Bye Mongo, Hello Postgres

#343

Post by golergka »

Thaxll wrote: post #225

It's not a great article tbh, it's well written but it shows the clear lack of knowledge running a backend. The title should be "we didn't know what we were doing so we switched to a managed DB" I mean yeah who knew that blocking NTP therefore time drifting would break everything... For those criticizing MongoDB, Fortnite generates $3B/year and runs on MongoDB, you should tell them it's a mistake and that they should…

Having a successful product doesn't mean that all technical decisions that were involved in making that product were successful. It raises the bayesian estimate that they were, of course - but not to an absolute boolean value.

Re: Bye Bye Mongo, Hello Postgres

#344

Post by KaiserPro »

michaelcampbell wrote: post #255
KaiserPro wrote: post #185

Earlier quoted context omitted.

only if more than 0.3% of your workforce know it....

And thus, nothing new was ever developed again.

There is a time and place for new. This isn't it.

there were no problems of scale, speed or latency. It was migrating from one terrible system to something that should be smaller, simpler, cheaper and easier to run.

The API is/was supposed to do precisely four things:

  o provide an authentication flow for customers
  0 provide an authentication flow for enterprises to allow SSO
  o handle payment info
  o count the number of articles read for billing/freemium/premium
That is all. Its a solved problem. Don't innovate, do.

Spend the time instead innovating on the bits that are useful to the business and provide an edge: CRM, pattern recognition and data analytics.

Re: Bye Bye Mongo, Hello Postgres

#346

Post by mscasts »

I worked on a large site that used MongoDB. We hundreds of thousands of rows in one specific collection that were used as a log. We couldn't even query it without the whole database crashing.

We moved that specific collection to MySQL, no problem and there was virtually no change in the data structure. Both were of course indexed.

Re: Bye Bye Mongo, Hello Postgres

#347

Post by mscasts »

0xbadcafebee wrote: post #100

My takeaways: 1. Stop. Trying. To. Build. Your. Own. Cloud. A pizza shop doesn't build their own cars to deliver pizzas. 2. There's no such thing as hassle-free anything, unless you are paying someone else to deal with the hassle. Sales teams lie. 3. Justifying an untested idea with "but it's modern technology" is going to backfire. Follow established patterns with good track records. 4. Writing your own in-house beh…

Some issues with your statements:

1. Well, they probably own the car though which is a better parable. You don't need to rent your car, you can simply purchase it just as you can purchase servers. People build their own garages.

4. So you should only used already written software? Sometimes it is just faster and better to write it yourself. You get less dependencies, you know how the whole thing works etc.

5. Why not? That is exactly what one should do.

Re: Bye Bye Mongo, Hello Postgres

#348

Post by wftglf »

AlexCoventry wrote: post #312
nixgeek wrote: post #311

Earlier quoted context omitted.

Anyone who can control a server in any environment can potentially interact with the database powering applications running on that server. How is running on AWS different than Guardian Cloud in their basement?

The level of control over who has physical access, of course. Did reports of the Snowden revelations reside on the CMS?

Sadly we don't trust our security practices anywhere enough for that! Secret investigations happen in an air gapped room on computers with their network cards removed then get moved across to the main CMS when they're ready to publish.

Re: Bye Bye Mongo, Hello Postgres

#349

Post by wftglf »

reilly3000 wrote: post #321
wftglf wrote: post #242

Earlier quoted context omitted.

Hi! Thanks for your comments. I'm one of the authors of this post. It is the same platform at the moment (just not tagged with editorial tags so it stays away from the fronts), though sometimes the team that approves non-editorial posts to the site can be concerned about us writing about outages and things as it might carry a 'reputational risk', so we may end up migrating to a different platform in the future so we…

In our era of deplatforming a publisher publishing on something like Medium seems antithetical, even for a dev team that just wants to get words out. Should you spend some cycles on the dev blog? Probably, but you should also split test the donation copy and get the data warehouse moved forward for 2019 initiatives and fix 1200 other issues. Thanks for sharing a great post. I shared it with my team and we all learned…

Thanks to hacker news and reddit this piece got over 100,000 page views which should be enough to justify the blog staying on platform!

Re: Bye Bye Mongo, Hello Postgres

#350

Post by wftglf »

AlexCoventry wrote: post #288
dmix wrote: post #271

Earlier quoted context omitted.

It’s probably better if they didn’t answer this question...

It's a bit disturbing to me that they seem to be using AWS for confidential editorial work. > Due to editorial requirements, we needed to run the database cluster and OpsManager on our own infrastructure in AWS rather than using Mongo’s managed database offering.

In a happy world the guardian wouldn't rely on a company we spend a lot of time reporting on for unethical practices (tax avoidance, worker exploitation etc.) - but we decided it was the only way to compete. One of the big drivers was a denial of service attack on our datacentre in 2014 on boxing day - not an experience any of us want to have to deal with again.
Post reply on HN