Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

141–150 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#141

Post by astine »

zzzcpan wrote: post #102
michelpp wrote: post #76

Earlier quoted context omitted.

You cannot be old-school and hype. That's the reason why it is old school. And the people who can't make good database design choices are exactly the kind of people who should be using SQL. Postgres knows how to optimize and plan queries efficiently based on the actual distributions of values in your dataset. These poor choosers should be doing that... by hand? https://www.postgresql.org/docs/11/planner-optimizer.htm…

This is what I'm talking about. You think that some feature that makes performance unpredictable is that important. While it's the last thing you should care about.

The only way for query performance to be truly predictable is for your data to be static. If that's not your situation then a query optimizer is incredibly helpful because it means that you don't have to rewrite your queries every time you add an index.

Re: Bye Bye Mongo, Hello Postgres

#142

Post by femto113 »

> Dynamo didn’t support encryption at rest.

This requirement made some sense in a world where a rogue employee might yank your database server out of a rack and walk off with it, but I don't understand why this is still considered relevant in an AWS context. First of all your data is never really "at rest", a huge point of Dynamo is that the data is always available (at least 99.999% of "always" anyway). Second "at rest" where? Literally on a physical disk? Even if there was a single physical disk that held it I'm willing to bet that even if you were in the right AWS data center and had support of a willing AWS employee you couldn't find that disk and even if you could there'd be no way to remove it. I'll guess its literally millions of times more likely for you to fail to deprecate some AWS API key that's sitting in clear text on a laptop you sold as surplus than anybody gets their hands on your at rest Dynamo data.

Re: Bye Bye Mongo, Hello Postgres

#143

Post by OJFord »

csytan wrote: post #68
nemild wrote: post #30

The Guardian example was heavily used by MongoDB as a case study to pitch their database to others in 2011: https://www.mongodb.com/customers/guardian https://www.mongodb.com/presentations/mongodb-guardian https://www.slideshare.net/tackers/why-we-chose-mongodb-for-... And reupping my previous, three-part series on MongoDB: On MongoDB NoSQL databases were the future. MongoDB was the database for "modern" web engineer…

I think you're asking the wrong question. The question should be: How did MongoDB become so successful? IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. MongoDB was the natural choice because they excelled at being accessible to devs who were already familiar with Javascript and JSON. Not only that, their marketing/outreach efforts were also…

Absolutely - if you don't know SQL and you do know JSON, postgres looks scary and Mongo looks familiar.

Re: Bye Bye Mongo, Hello Postgres

#144

Post by bunderbunder »

h1d wrote: post #116
trhway wrote: post #99

Earlier quoted context omitted.

Denormalization as way of dealing with performance issues is like guilottin to cure headache. And more frequently than not the headache is still there even after that. It is true that for some narrow class of analytical workloads 20-25 years ago (behold BW of 199x) the denormalized case performance was better compare with straight non-optimized running of the same queries over normalized schema. Since then, the expon…

What do you suggest in place?

I would suggest not denormalizing at all.

The idea that joins are slow is a holdover from the bad old days when everyone used MySQL and MySQL sucked at joins. On a more robust DBMS, a normalized schema will often yield better performance than the denormalized one. Less time will be lost to disk I/O thanks to the more compact format, the working set will be smaller so the DBMS can make more efficient use of its cache, and the optimizer will have more degrees of freedom to work with when trying to work out the most efficient way to execute the query.

(edited to add: If you're having performance problems with a normalized schema, the first place to look isn't denormalization, it's the indexing strategy. And also making sure the queries aren't doing anything to prevent proper index usage. Use the Index, Luke! is a great, DB-agnostic resource for understanding how to go about this stuff: https://use-the-index-luke.com )

Re: Bye Bye Mongo, Hello Postgres

#145

Post by mr_tristan »

Drdrdrq wrote: post #77

> Since all our other services are running in AWS, the obvious choice was DynamoDB – Amazon’s NoSQL database offering. Unfortunately at the time Dynamo didn’t support encryption at rest. Whoa, that was close. I really don't see why anyone would choose DynamoDB as a general purpose data store, unless they enjoy wasting countless hours finding ways around the limitations it imposes about how data should be stored and a…

NoSQL like DynamoDB or Cassandra does make for good general purpose data stores, but you have to work within the concepts of NoSQL: typically, use 1 table with careful choices in partition and sort keys.

It's an epic shift in thinking to go from a schema of 50-60 tables to 1. Almost every dev I've worked with is very new to this.

The #1 sign a dev has no business using NoSQL: They chat about how flexible NoSQL is. Yeah, you can add attributes on the fly but I've found Dynamo to require much more careful planning then RDBMSes. Most devs I've met can understand when they need to use an index. But, almost all of them have issues predicting if their changes will lead to unbalanced requests against partitions.

Anyhow, since you can even connect the PostgreSQL WAL easily to a log stream like Kafka or Kinesis, I'm not sure why you'd ever start with a NoSQL DB, unless you just had master NoSQL data wonks.

Re: Bye Bye Mongo, Hello Postgres

#146

Post by threeseed »

csytan wrote: post #68
nemild wrote: post #30

The Guardian example was heavily used by MongoDB as a case study to pitch their database to others in 2011: https://www.mongodb.com/customers/guardian https://www.mongodb.com/presentations/mongodb-guardian https://www.slideshare.net/tackers/why-we-chose-mongodb-for-... And reupping my previous, three-part series on MongoDB: On MongoDB NoSQL databases were the future. MongoDB was the database for "modern" web engineer…

I think you're asking the wrong question. The question should be: How did MongoDB become so successful? IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. MongoDB was the natural choice because they excelled at being accessible to devs who were already familiar with Javascript and JSON. Not only that, their marketing/outreach efforts were also…

None of this makes sense.

ORMs have existed for decades so developers can use a SQL database just fine without knowing the language. So it's definitely not this.

It's more likely because Mongo is (a) is extremely fast, (b) the easiest database to manage and (c) has a flexible schema which aligns better with dynamic languages which are more popular amongst younger developers.

Re: Bye Bye Mongo, Hello Postgres

#147

Post by DRW_ »

jdietrich wrote: post #136
jack6e wrote: post #35

Specific technology choices aside, this was an incredible write-up of their migration process: thorough, organized, readable prose about a technical topic. It is helpful to read how other teams handle these types of processes in real production systems. Perhaps most refreshing is the description of choices made for the various infrastructure pieces, because it is reasonable and real-world. Blog posts so often describ…

The BBC's Online and R&D departments have very interesting blogs, if you like this sort of thing. http://www.bbc.co.uk/blogs/internet https://www.bbc.co.uk/rd

Also: https://medium.com/bbc-design-engineering

Re: Bye Bye Mongo, Hello Postgres

#148

Post by threeseed »

OJFord wrote: post #143
csytan wrote: post #68

Earlier quoted context omitted.

I think you're asking the wrong question. The question should be: How did MongoDB become so successful? IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. MongoDB was the natural choice because they excelled at being accessible to devs who were already familiar with Javascript and JSON. Not only that, their marketing/outreach efforts were also…

Absolutely - if you don't know SQL and you do know JSON, postgres looks scary and Mongo looks familiar.

You make it sound like learning SQL is like learning Assembler. It's not that hard. And ORMs exist in every language to abstract it all away.

PostgreSQL looks scary because it is a swiss army knife. It has a million different features and data structures. MongoDB does only one thing.

Re: Bye Bye Mongo, Hello Postgres

#149

Post by OJFord »

wallflower wrote: post #106

Kudos to The Guardian for publishing this article on their top-level domain. It looked identical to one of their news articles (including ads). Only the content (excellent writeup) was different. Many tech companies silo their engineering teams writings into a “tech” part of their website. If this wasn’t an ad for The Guardian’s technology team, I think it could be. Putting the engineering team’s content in the same…

Have you tried to find it from the home page though?

It's under `/info`, which un-suffixed redirects to `/about`, which I _can_ find from the home page (it's linked in the _footer_) but I can't find this engineering blog from `/about`, or anywhere else I can get to from `/`.

I don't think it's so much a positive choice to use the same system as it is just using (a possibly separate instance of) what they already had, and still squirreling it away in a corner.

Re: Bye Bye Mongo, Hello Postgres

#150

Post by kevin_thibedeau »

nemild wrote: post #30

The Guardian example was heavily used by MongoDB as a case study to pitch their database to others in 2011: https://www.mongodb.com/customers/guardian https://www.mongodb.com/presentations/mongodb-guardian https://www.slideshare.net/tackers/why-we-chose-mongodb-for-... And reupping my previous, three-part series on MongoDB: On MongoDB NoSQL databases were the future. MongoDB was the database for "modern" web engineer…

> What happened?

Text and still imagery based media companies aren't big data providers. NoSql was a bad choice from the start.

Post reply on HN