Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

331–340 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#331

Post by dsirola »

kstrauser wrote: post #264
dsirola wrote: post #214

Earlier quoted context omitted.

I'm pretty sure he wants to say that you should abstract your database away so that your business code / domain model doesn't depend on it. It becomes a "plugin" to your application and you can easily switch it just by writing another implementation.

I couldn't disagree more, at least in the context of this article. If you have an abstraction layer so high level that app developers can't tell whether they're using MongoDB or PostgreSQL, then they're not able to use any of the advantages of either of those systems. Sure, use an ORM to abstract away the differences between PostgreSQL and MySQL (up until you need to care about them). That's reasonable. But maintaini…

Why wouldn't you be able to take advantage of those systems? I think it's quite the opposite where you can take the advantages without even knowing about it / affecting other parts of the system.

Re: Bye Bye Mongo, Hello Postgres

#332

Post by xp84 »

quietbritishjim wrote: post #210
csytan wrote: post #68

Earlier quoted context omitted.

I think you're asking the wrong question. The question should be: How did MongoDB become so successful? IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. MongoDB was the natural choice because they excelled at being accessible to devs who were already familiar with Javascript and JSON. Not only that, their marketing/outreach efforts were also…

> IMO, the reason is that newer developers faced the choice of learning SQL or learning to use something with a Javascript API. The thing I dislike about this type of comment – although I now notice yours doesn't explicitly say this – is the implication that devs don't like SQL because they're lazy or stupid. Well, sometimes that is probably true! But there are some tasks where you need to build the query dynamically…

I'm confused by the implication that someone doing things like the above would be writing in SQL. SQL is a little like assembly language in a game: You may need to drop down to it for some key highly-optimized areas, but you rarely need to directly use it for most tasks. While it's true that you should understand how it works so you don't generate queries that suck performance-wise, the same goes for Mongo's intricacies too.

Every language I know of has great ORMs which do this for whatever SQL flavors people tend to use on that platform. I write things like this all the time, and it gets turned into SQL for Postgres:

```` Article.where(author_id: 37).order(:modified_date, :desc).where.not(published: false) ````

When using an ORM correctly (and indeed, the less I'm using any of my own bits of SQL the more this is true) I am also protected against injection attacks.

I'm not saying NoSQL has no value, but I believe it to be the wrong tool for data that lends itself to an RDBMS. If you have a bunch of documents who have deeply nested or inconsistent structures and where it makes no sense that you'd want to query by something other than the primary key, sure, it's a no-brainer to use a NoSQL system. For a CMS, which has been implemented thousands of times in RDBMSs, it is madness though. I cringe at realizing that apaprently there are developers out there who have avoided learning SQL entirely in their career out of fear, and as a result have to use Mongo for every application because that's the only thing they know how to do. I'm sure they're out there, but I wouldn't hire one.

Re: Bye Bye Mongo, Hello Postgres

#333

Post by BuckRogers »

indigodaddy wrote: post #170
0xbadcafebee wrote: post #100

My takeaways: 1. Stop. Trying. To. Build. Your. Own. Cloud. A pizza shop doesn't build their own cars to deliver pizzas. 2. There's no such thing as hassle-free anything, unless you are paying someone else to deal with the hassle. Sales teams lie. 3. Justifying an untested idea with "but it's modern technology" is going to backfire. Follow established patterns with good track records. 4. Writing your own in-house beh…

Point 5. I'm not a software/application person... can you expound on how else you would interact with a DB if not directly?

It could be an application frontend interfacing with a webservice or network service. That component handles the database. That could make implementations clean for migration from on-prem to cloud, or if you're ripping out the frontend stack semi-regularly as more web-oriented devs may do more often. I can't say I disagree, and moving an application to this model myself so that I can easily migrate it from a WinForms frontend to Blazor SPA at some point.

Re: Bye Bye Mongo, Hello Postgres

#334

Post by hbbio »

manigandham wrote: post #326
hbbio wrote: post #320

Earlier quoted context omitted.

Exactly. As I read the original article, which mentions "encryption-at-rest", there was a voice in my head crying: "No, what they need is E2EE". That would enable the authors to write confidential drafts of the articles, no matter where the data is stored (and AWS would be perfectly fine of course). Disclaimer: The voice is my head does not come out of nowhere. I am building a product which addresses this: https://gi…

In which case they could've just used a separate encryption layer with any database, including DynamoDB. The HSM security keys available from all the clouds makes this rather simple.

Yes, any db including Dynamo would have been fine.

Our software E2EE solution has advantages over HSM though: Cost obviously, and more features and extensibility.

Re: Bye Bye Mongo, Hello Postgres

#335

Post by dbrgn »

nemild wrote: post #30

The Guardian example was heavily used by MongoDB as a case study to pitch their database to others in 2011: https://www.mongodb.com/customers/guardian https://www.mongodb.com/presentations/mongodb-guardian https://www.slideshare.net/tackers/why-we-chose-mongodb-for-... And reupping my previous, three-part series on MongoDB: On MongoDB NoSQL databases were the future. MongoDB was the database for "modern" web engineer…

Nice writeup. I love this quote: "The first few times an engineer sees this kind of hype, they often think it's a structural shift. For engineers later in our career, we’ll often dismiss structural shifts as misplaced hype after getting burned too many times"

Re: Bye Bye Mongo, Hello Postgres

#336

Post by luhn »

quietbritishjim wrote: post #296
luhn wrote: post #279

Earlier quoted context omitted.

Yes, you can! The jsonb_array_elements function is roughly similar to Mongo’s $unwind pipeline op. It explodes a JSON array into a set of rows. From there it’s pretty simple aggregates to achieve what you’re looking for. I was evaluating Mongo a couple months back to solve roughly the same problems. Eventually discovered Postgres already had what I was looking for.

Allow me to restate this question: > Does it do that? It was supposed to be clear from the context that this meant: > Does building queries programmatically with SQLAlchemy do that? Maybe I'm misreading your comment, but you seem to just be talking about writing queries directly in SQL. If not, could you give an example/link of how to programmically build a query in SQLAlchemy that dynamically makes use of jsonb_arra…

I was speaking of SQL, but if you can write it in SQL you can usually map it to SQLAlchemy. If worse comes to worse, you can use text() to drop down to raw SQL for just a portion of the query.

SQLAlchemy’s Postgres JSONB type allows subscription, so you can do Model.col[‘arrayfield’]. You can also manually invoke the operator with Model.col.op(‘->’)(‘arrayfield’).

So you should be able to do something like:

func.sum(func.jsonb_array_elements(Model.col.op(‘->’)(‘arrayfield’)).op(‘->’)(‘val’))

(Writing on my mobile without reference, so may not be fully accurate)

Re: Bye Bye Mongo, Hello Postgres

#337

Post by emn13 »

scarface74 wrote: post #34
emn13 wrote: post #29

Earlier quoted context omitted.

By server management - do you mean VM? Sure, you can outsource that. I mean that what and how a home-grown application uses a database tends to mean that database (software/schema/optimization/whatever) management cannot be application agnostic. So if you outsource this, you're either effectively hiring a consultant that still needs to deal with and learn details of specifically your application, or you should assume…

Everything you are saying is true, but when people talk about managed services, for the most part they are referring to someone else managing the VM, the operating system and the server application running on the VM - in this case the database.

Exactly! That's why I was confused - they didn't replace mongodb with colloquially "hosted" postgres; they stayed at the same level of (partial) hosting. I.e., they didn't "switch to hosted postgres", in any normal sense.

Re: Bye Bye Mongo, Hello Postgres

#339

Post by stuartaxelowen »

0xbadcafebee wrote: post #100

My takeaways: 1. Stop. Trying. To. Build. Your. Own. Cloud. A pizza shop doesn't build their own cars to deliver pizzas. 2. There's no such thing as hassle-free anything, unless you are paying someone else to deal with the hassle. Sales teams lie. 3. Justifying an untested idea with "but it's modern technology" is going to backfire. Follow established patterns with good track records. 4. Writing your own in-house beh…

RE 5: I'd argue that with a relational database (barring extreme scale cases) is the perfect place for your business logic. Constraints, unique indexes, etc, all ensure that your data is always consistent with itself and with the business logic you've encoded.

Re: Bye Bye Mongo, Hello Postgres

#340

Post by brianmcc »

I enjoyed the write up. I love these kinds of semi-technical, semi-story-telling, pieces.

I am saddened by many of the comments here though which equate to: "never try anything until you know everything" - sorry but that's just not realistic and it's unfair to the people who - commendably - contribute to these write ups and hold their hands up to mistakes-made, decisions that went badly with hindsight, etc.

Bigger picture: the Guardian appears to be thriving, and is succeeding based on the efforts of the tech team here. So if you read this and come away with a sense of "failure" you're probably missing something important.

It's OK to try new things and fail - just get back up and keep on trying, and use the new wisdom you build!

Post reply on HN