Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

351–360 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#351

Post by untog »

AlexCoventry wrote: post #317
untog wrote: post #314

Earlier quoted context omitted.

Probably not, no, until they were about to be published. I imagine that the choice between "run an entire data centre ourselves, store everything there" and "use AWS, but keep high sensitivity stories on local machines" is an easy one. After all, the client computer that connects to the CMS is just as, or more likely to be compromised. I wouldn't be surprised if the coverage (or at least parts of it) were edited on a…

> the choice between "run an entire data centre ourselves, store everything there" If those were the only two choices, you might be right. But the resources needed for the actual CMS functionality sound modest enough to run independently of the main website. > the client computer that connects to the CMS is just as, or more likely to be compromised That's faulty reasoning.

> That's faulty reasoning.

Why? It's an obvious potential point of compromise.

Re: Bye Bye Mongo, Hello Postgres

#352

Post by wftglf »

fastftw wrote: post #209
petilon wrote: post #95

Disappointing article. There are no clear reasons mentioned for migrating away from MongoDB. "All these problems" and some issues with Ops Manager are mentioned, where "all these problems" is a couple of outages. As if other database technologies prevent outages. As soon as they experience a couple of outages with their new stack they will migrate to something else, presumably.

Thank you! I was reading through the comments wondering how no one has seen through the bullshit veil purported by the title, followed by a detailed outline of their migration which I think is the "shiny object" to distract people from an unfounded argument.

The main advantage of the migration was getting onto a fully managed DB. What we didn't mention was that there were also huge cost implications - savings of ~£40k - through switching to RDS - compared to paying for a mongo support contract.

Re: Bye Bye Mongo, Hello Postgres

#353

Post by wftglf »

OJFord wrote: post #149
wallflower wrote: post #106

Kudos to The Guardian for publishing this article on their top-level domain. It looked identical to one of their news articles (including ads). Only the content (excellent writeup) was different. Many tech companies silo their engineering teams writings into a “tech” part of their website. If this wasn’t an ad for The Guardian’s technology team, I think it could be. Putting the engineering team’s content in the same…

Have you tried to find it from the home page though? It's under `/info`, which un-suffixed redirects to `/about`, which I _can_ find from the home page (it's linked in the _footer_) but I can't find this engineering blog from `/about`, or anywhere else I can get to from `/`. I don't think it's so much a positive choice to use the same system as it is just using (a possibly separate instance of) what they already had,…

It's the same system, just without any tags on it that mix it in with editorial content. I think if we wanted it published somewhere like the technology section we'd have had to go through a much more rigorous editing process (we probably wouldn't have got away with 3000 words!)

Re: Bye Bye Mongo, Hello Postgres

#354

Post by ranman »

jack6e wrote: post #35

Specific technology choices aside, this was an incredible write-up of their migration process: thorough, organized, readable prose about a technical topic. It is helpful to read how other teams handle these types of processes in real production systems. Perhaps most refreshing is the description of choices made for the various infrastructure pieces, because it is reasonable and real-world. Blog posts so often describ…

I was actually a little confused by the article - it seems to go up and down in terms of technical depth. It feels like it was written by several people. The hyperlink to “a screen session” was odd as well ... ammonite hyperlink I get but... screen is a pretty ancient tool... people either know it or can find out about it. Like you link to screen but not “ELK” stack?

I like the article but it was a bit hard for me to consume with multiple voices in different parts.

Re: Bye Bye Mongo, Hello Postgres

#355

Post by wftglf »

AlexCoventry wrote: post #269
wftglf wrote: post #242

Earlier quoted context omitted.

Hi! Thanks for your comments. I'm one of the authors of this post. It is the same platform at the moment (just not tagged with editorial tags so it stays away from the fronts), though sometimes the team that approves non-editorial posts to the site can be concerned about us writing about outages and things as it might carry a 'reputational risk', so we may end up migrating to a different platform in the future so we…

Does SecureDrop run on your AWS infrastructure?

No, I think it's on our own infrastructure.

Re: Bye Bye Mongo, Hello Postgres

#356

Post by user5994461 »

riku_iki wrote: post #307
user5994461 wrote: post #220

Earlier quoted context omitted.

bullshit. It's shared everything. if you have a 3 nodes cluster for redundancy, the 3 nodes keep the exact same data with only one node accepting reads and writes. It's literally the opposite of linear scalability.

in sharded setup each shard contains subset of data, each node doesn't contain all data: https://docs.mongodb.com/manual/core/sharded-cluster-shards/ You are talking about replica sets, which is different concept.

In a sharded setup, every shard must be on a 3 nodes cluster for redundancy (a replica set), the 3 nodes keep the exact same data with only one node accepting reads and writes.

You have to add capacity (shards) 3 nodes at a time, two third of which sit unused. It's not scalable at all.

Re: Bye Bye Mongo, Hello Postgres

#357

Post by ownagefool »

da_chicken wrote: post #168
untog wrote: post #62

Earlier quoted context omitted.

IMO there is still a place for schema-less document databases. It's just that Postgres's JSON columns mean you can get the best of both worlds, which makes Mongo look weak by comparison.

I would rather say, "Postgres's JSONB provides a hybrid compromise that may meet the needs of many users." JSONB feels like it's closer to creating more complex data types that are less primitive than those that SQL currently allows. The real driver of the NoSQL movement, I believe, was that everybody wanted to be the next big social network or content aggregation site. Everybody wanted to be the next Facebook, Insta…

This likely isn't true for banking systems.

Many ATMs will still give you money when they're offline, and things become eventually consistent by comparing the ledger.

Shops also generally want to take orders and payments irregardless of the network availability, so whilst they might generally act as CP systems, they'll be AP in the event of network downtime, but will likely lose access to fraud checks, so may put limits of purchases, etc.

They're probably all CP locally and AP (or switchable from CP) from an entire system perspective.

Re: Bye Bye Mongo, Hello Postgres

#358

Post by Fiahil »

Was is most surprising (well, not really), is the apparent immaturity of Scala tooling (Ammonite), and Akka-based projects like Akka-HTTP and Akka-Stream. If you can get into very cryptic issues when using these project, it makes you reconsider the actual value you get from using a functional, type safe and on-paper concurrent language.

Their proxy is one example where Rust will give the same capabilities as Scala and will be more robust, in the future, if it's not already the case today.

Re: Bye Bye Mongo, Hello Postgres

#359

Post by thanatropism »

ranman wrote: post #354
jack6e wrote: post #35

Specific technology choices aside, this was an incredible write-up of their migration process: thorough, organized, readable prose about a technical topic. It is helpful to read how other teams handle these types of processes in real production systems. Perhaps most refreshing is the description of choices made for the various infrastructure pieces, because it is reasonable and real-world. Blog posts so often describ…

I was actually a little confused by the article - it seems to go up and down in terms of technical depth. It feels like it was written by several people. The hyperlink to “a screen session” was odd as well ... ammonite hyperlink I get but... screen is a pretty ancient tool... people either know it or can find out about it. Like you link to screen but not “ELK” stack? I like the article but it was a bit hard for me to…

screen is hard to google.

Re: Bye Bye Mongo, Hello Postgres

#360

Post by mukundmr »

idbentley wrote: post #161

I really don't get this as an indictment of MongoDB, or their OpsManager product really. They used the version of OpsManager that doesn't manage the deployment - is specifically not a deployment manager. Mongo does offer a managed version of this software, which the author mentions - with a justification for why they couldn't use that offering. However, I think this was the main mistake that The Guardian made. As the…

The pricing of hosted MongoDB solutions is high especially as volumes increase. The last I checked, disk space doesn't free up when you drop documents or collections and that adds to hosted cost unless you find time to fix the issue through manual intervention. This costing is moving us away from MongoDB in the future.
Post reply on HN