Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

281–290 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#281

Post by hnzix »

threeseed wrote: post #148
OJFord wrote: post #143

Earlier quoted context omitted.

Absolutely - if you don't know SQL and you do know JSON, postgres looks scary and Mongo looks familiar.

You make it sound like learning SQL is like learning Assembler. It's not that hard. And ORMs exist in every language to abstract it all away. PostgreSQL looks scary because it is a swiss army knife. It has a million different features and data structures. MongoDB does only one thing.

Learning SQL syntax isn't hard, but learning how to properly design relational databases is not something you can pick up from skimming blog posts.

Re: Bye Bye Mongo, Hello Postgres

#282

Post by karmakaze »

beering wrote: post #27

Did they ever publish the rationale for why they used MongoDB in the first place? They mention having 2.3 million content items. If we assume that they have 100 DB entries for each content item, that's still only 230 million DB items. In that case, was it important to run a sharded cluster vs a typical primary-secondary HA setup? (which they ended up switching to)

I was reading the article waiting to get to the part where they handle their shards can scale. It ended up just reading like a migration from one db to another.

I'm guessing their original choice for MongoDB was more for the schemaless development flexibility rather than horizontal scaling capabilities. This seems to have worked well enough for them. DynamoDB seems like a natural fit coming from a document store and they did very well not to choose it.

I think AWS is due for a managed NewSQL store comparable to Google Cloud Spanner or MS Cosmos DB.

Re: Bye Bye Mongo, Hello Postgres

#283

Post by dmix »

wftglf wrote: post #250
idbentley wrote: post #161

I really don't get this as an indictment of MongoDB, or their OpsManager product really. They used the version of OpsManager that doesn't manage the deployment - is specifically not a deployment manager. Mongo does offer a managed version of this software, which the author mentions - with a justification for why they couldn't use that offering. However, I think this was the main mistake that The Guardian made. As the…

Fair criticisms! It's true if we'd used Mongo Atlas or something similar it would likely have been a different story - often the MongoDB support spent half the time on the phone trying to work out what version of mongo, opsmanager etc. we were running. Re criticism of OpsManager - I think this is fair, given the sheer number of hoops we had to jump through to get a functioning OpsManager system running in AWS - no pr…

> often the MongoDB support spent half the time on the phone trying to work out what version of mongo, opsmanager etc. we were running.

That sounds awful. Reading these stories I'm happy I work with small companies without such a huge infrastructure.

Re: Bye Bye Mongo, Hello Postgres

#284

Post by bunderbunder »

michaelcampbell wrote: post #255
KaiserPro wrote: post #185

Earlier quoted context omitted.

only if more than 0.3% of your workforce know it....

And thus, nothing new was ever developed again.

It's not that you don't write anything in a new thing. It's that you start with small, less critical projects. Get your feet wet, give people a chance to get a feel for the pros and cons, that sort of thing.

Jumping right into writing "mission critical services" in the brand new language that few people at the company know well is asking for trouble.

Re: Bye Bye Mongo, Hello Postgres

#285

Post by _10x_dev »

“This is one of the reasons why building on top of the old API wasn’t an option. There was very little separation of concern in the original API and MongoDB specifics could be found even at the controller level. As a result the task of adding another database type in the existing API was too risky.“

This bit sounds like a cautionary tale about the importance of a layer and well structured API. There’s no reason why details about your data store should leak into an API Controller.

Re: Bye Bye Mongo, Hello Postgres

#286

Post by philliphaydon »

StreamBright wrote: post #134
bdcravens wrote: post #37

Earlier quoted context omitted.

When was the last "Goodbye X, Hello Mongo" article?

Not sure, but there was Goodbye MySQL, Hello Postgres and Goodbye Postgres, Hello MySQL (same company did it actually) for sure. It is kind of weird why engineers are so obsessed with tools though.

Oh which neither were recent. And their usage was just bizarre to begin with.

Re: Bye Bye Mongo, Hello Postgres

#287

Post by philliphaydon »

snacktaster wrote: post #245
Thaxll wrote: post #225

It's not a great article tbh, it's well written but it shows the clear lack of knowledge running a backend. The title should be "we didn't know what we were doing so we switched to a managed DB" I mean yeah who knew that blocking NTP therefore time drifting would break everything... For those criticizing MongoDB, Fortnite generates $3B/year and runs on MongoDB, you should tell them it's a mistake and that they should…

I'm sure mongodb is fine and powers a lot of high-profile sites and services. But I've never had a good experience running it or administering it. But I'm also one of the the people that still prefers to run MySQL over PostreSQL just because the tooling is still far superior

Can you give an example of superior tooling or tooling that isn’t good in PostgreSQL compared to MySQL. Serious question. I’m curious cos I mostly use PostgreSQL.

Re: Bye Bye Mongo, Hello Postgres

#288

Post by AlexCoventry »

dmix wrote: post #271
AlexCoventry wrote: post #269

Earlier quoted context omitted.

Does SecureDrop run on your AWS infrastructure?

It’s probably better if they didn’t answer this question...

It's a bit disturbing to me that they seem to be using AWS for confidential editorial work.

> Due to editorial requirements, we needed to run the database cluster and OpsManager on our own infrastructure in AWS rather than using Mongo’s managed database offering.

Re: Bye Bye Mongo, Hello Postgres

#289

Post by inferiorhuman »

quietbritishjim wrote: post #278
QuinnWilton wrote: post #237

Earlier quoted context omitted.

Elixir's primary database wrapper, Ecto [0], lets you dynamically build queries at runtime, and also isn't an ORM. Here's two examples directly from the docs: # Query all rows in the "users" table, filtering for users whose age is > 18, and selecting their name "users" |> where([u], u.age > 18) |> select([u], u.name) # Build a dynamic query fragment based on some parameters dynamic = false dynamic = if params["is_pub…

As I said to a sibling response, this is not a substitute for Mongo's aggregation pipeline unless it can do analogous things to Postgres's JSONB fields. For example, can it unwind an array field, match those subrecords where one field (like a "key") matches a value and another field (like a "value") exceeds an overall value, and then apply this condition to filter the overall rows in the table? Also, one of the benef…

> As I said to a sibling response, this is not a substitute for Mongo's aggregation pipeline

Huh? The aggregation framework is a solution to a mongo-only problem. Most other databases are performant, but Mongo suffers wildly from coarse locking and slow performance putting things into and retrieving things from the javascript VM.

> For example, can it unwind an array field, match those subrecords where one field (like a "key") matches a value and another field (like a "value") exceeds an overall value, and then apply this condition to filter the overall rows in the table?

This sounds suspiciously like a SQL view.

Edit: But if you actually need an array in a cell, Postgres has an array type that's also a first-class citizen with plenty of tooling around it.

Re: Bye Bye Mongo, Hello Postgres

#290

Post by inferiorhuman »

luhn wrote: post #279
quietbritishjim wrote: post #262

Earlier quoted context omitted.

For it to replace MongoDB's aggregation pipeline, it would need to play nicely with JSONB. Does it do that? This is the thing I'm really missing. For example, if documents in the JSONB column all look roughly like this: { "someArrayField": [ { "key": "steve", "value": 7 }, { "key": "bob", "value": 15 }, ], "someOtherField": [ "whatever" ] } * Can I count the number of entries in someArrayField, summed across all reco…

Yes, you can! The jsonb_array_elements function is roughly similar to Mongo’s $unwind pipeline op. It explodes a JSON array into a set of rows. From there it’s pretty simple aggregates to achieve what you’re looking for. I was evaluating Mongo a couple months back to solve roughly the same problems. Eventually discovered Postgres already had what I was looking for.

More the point Postgres has an actual array data type (and has for a while). You don't need to shove everything into a JSON/JSONB blob unless you absolutely cannot have any sort of schema.
Post reply on HN