Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bye Bye Mongo, Hello Postgres

theguardian.com

Post reply on HN
Print view View original

321–330 of 427 posts

Re: Bye Bye Mongo, Hello Postgres

#321

Post by reilly3000 »

wftglf wrote: post #242
viraptor wrote: post #103

Earlier quoted context omitted.

It's not their main platform though: > Digital Blog > A blog by the Guardian's internal Digital team. We build the Guardian website, mobile apps, Editorial tools, revenue products, support our infrastructure and manage all things tech around the Guardian

Hi! Thanks for your comments. I'm one of the authors of this post. It is the same platform at the moment (just not tagged with editorial tags so it stays away from the fronts), though sometimes the team that approves non-editorial posts to the site can be concerned about us writing about outages and things as it might carry a 'reputational risk', so we may end up migrating to a different platform in the future so we…

In our era of deplatforming a publisher publishing on something like Medium seems antithetical, even for a dev team that just wants to get words out. Should you spend some cycles on the dev blog? Probably, but you should also split test the donation copy and get the data warehouse moved forward for 2019 initiatives and fix 1200 other issues. Thanks for sharing a great post. I shared it with my team and we all learned a lot.

Re: Bye Bye Mongo, Hello Postgres

#323

Post by AlexCoventry »

hbbio wrote: post #320
AlexCoventry wrote: post #302

Earlier quoted context omitted.

Anyone who gets control of the live server can still read a database, even if it encrypts its storage.

Exactly. As I read the original article, which mentions "encryption-at-rest", there was a voice in my head crying: "No, what they need is E2EE". That would enable the authors to write confidential drafts of the articles, no matter where the data is stored (and AWS would be perfectly fine of course). Disclaimer: The voice is my head does not come out of nowhere. I am building a product which addresses this: https://gi…

Great idea.

Re: Bye Bye Mongo, Hello Postgres

#324

Post by quickthrower2 »

anyfoo wrote: post #44
elevenworth wrote: post #11

The creator of Envoy had some choice words about Mongo a few days ago. https://twitter.com/mattklein123/status/1074717204224999427 Plenty of people here like to dish on Mongo and the product seems to have been re-architected a few times since I used it seven years ago. By what metrics can we say the product is one worthy of passing a HN smell test? Passing Jepsen was seemingly not enough. https://www.mongodb.com/jeps…

> By what metrics can we say the product is one worthy of passing a HN smell test? Common sense and formal education? I'm sorry, I'm aware of how incredibly snarky and arrogant that sounds, but in this case I always struggled to comprehend how MongoDB, or most of "NoSQL" in general, was considered viable to begin with. "Schemaless" just immediately means that instead of the database keeping consistency, you now essen…

I also don’t see the benefit of schemaless. Schemes have been way too useful in my programming career.

But eventual consistency may be a viable trade off to help you scale, but the thing is you probably need to be at a massive scale before it is worth moving away from traditional SQL, which can have read replication and the app can have caching too. But there is probably some point at which eventually consistent nodes make sense.

In a sense that is what a CDN is, and we like to use those.

Re: Bye Bye Mongo, Hello Postgres

#325

Post by BuckRogers »

vonseel wrote: post #212
SonnyWortzik wrote: post #24

I never got around to using Mongo as my main doc db because it was incredibly hard to find a management tool. I now use json supported functions in SQL Server and do not have the need for a different type of database. SQL Server handles my small 'documents db' implementation with the infrastructure of a RDMS. Win win for me. To me Mongo just got popular by mistake way too early. It's like having a celebrity retweet y…

I have seen so many critical articles of Mongodb. Even if I did have a use-case particularly well-suited for it, I would never consider using Mongo. As an aside, I did consider using SQL Server until I looked at the licensing fees. Why would someone choose SQL Server when options like Postgres or MySQL/MariaDB exist? Is there a specific SQL feature (MSSQL or Oracle SQL) provides which is not available elsewhere and w…

SSRS is a big draw. There are other and probably better reporting tools, but the integration and support from MS is worth something to most companies. There's also simply the network effect at play and marketing, that MS has always backed that up with results. The money involved really isn't prohibitive for most small to mid-size companies that I've worked with. I've only seen the opposite, companies leaving their expensive UNIX licenses to move to an all-MS stack to save money. I have never seen large companies try to reduce their technology costs to zero with pure open source, because I think they intuitively know that you either pay with your licenses or you pay with maintaining a lot of expensive expertise, that also needs properly managed and effective. In the discussion of free as in freedom vs free as in beer, most companies core business is not technology, and they know there's no truly free beer and don't really care about the freedom. They want everything I've already mentioned, and the short and longterm support the license buys.

Outside of my professional career, with my personal projects I use SQLite, but if I were to build out something that intended to be larger scale business venture, I'd probably go with Azure SQL Database for multiple reasons. A large one being MS's overall integration, including their full control of the stack and CI/CD with .Net->Azure DevOps->github->Azure Pipelines->Azure PaaS/SQL Database. I admire what they're building, but a lot of people work outside of the MS realm. Companies don't tend to have a tech stack bone to pick as we do on HN, and many are already using part of the MS stack.

In sum, there's just a lot of business realities at play. I wouldn't personally run out and buy a SQL Server license myself, for what I do at home or with my (very) small side business, either.

Re: Bye Bye Mongo, Hello Postgres

#326

Post by manigandham »

hbbio wrote: post #320
AlexCoventry wrote: post #302

Earlier quoted context omitted.

Anyone who gets control of the live server can still read a database, even if it encrypts its storage.

Exactly. As I read the original article, which mentions "encryption-at-rest", there was a voice in my head crying: "No, what they need is E2EE". That would enable the authors to write confidential drafts of the articles, no matter where the data is stored (and AWS would be perfectly fine of course). Disclaimer: The voice is my head does not come out of nowhere. I am building a product which addresses this: https://gi…

In which case they could've just used a separate encryption layer with any database, including DynamoDB. The HSM security keys available from all the clouds makes this rather simple.

Re: Bye Bye Mongo, Hello Postgres

#327

Post by zaarn »

inferiorhuman wrote: post #290
luhn wrote: post #279

Earlier quoted context omitted.

Yes, you can! The jsonb_array_elements function is roughly similar to Mongo’s $unwind pipeline op. It explodes a JSON array into a set of rows. From there it’s pretty simple aggregates to achieve what you’re looking for. I was evaluating Mongo a couple months back to solve roughly the same problems. Eventually discovered Postgres already had what I was looking for.

More the point Postgres has an actual array data type (and has for a while). You don't need to shove everything into a JSON/JSONB blob unless you absolutely cannot have any sort of schema.

Not only arrays, you can, with some limitations, create proper types with field names, if your ORM supports that you should use that over JSONB if it fits.

Re: Bye Bye Mongo, Hello Postgres

#328

Post by manigandham »

StreamBright wrote: post #120
thejerz wrote: post #115

Their architectural choices are puzzling to me: 1) Why use Scala to write (a relatively simple) internal CMS? 2) Why use a clustered database for 2 million records? 3) Why write your own proxy? (in Akka, none the less) 4) Why would you migrate articles from Mongo to Postgres using a script that runs overnight in screen? The Guardian is, prima facie, a Wordpress blog. A simpler architecture would be: 1) Any CRUD web f…

I am not sure why you think that Rails or Django or any bloated web frameworks are necessary. The most trivial way to implement a website for high traffic is simple static content generator (like Jekyll) and use a CDN. There is incredible amount of CPU wasted on rendering the same exact content for every request. The content of the articles never (or very rarely) changes so you can put it in a CDN. CRUD, DMS, RDBMS a…

You can put a CDN in front of anything, including Wordpress or any framework. No need for a static site, and the homepages and section pages are rather dynamic now with user logins, story feeds, and personalization.

Re: Bye Bye Mongo, Hello Postgres

#329

Post by inferiorhuman »

inferiorhuman wrote: post #272
QuinnWilton wrote: post #237

Earlier quoted context omitted.

Elixir's primary database wrapper, Ecto [0], lets you dynamically build queries at runtime, and also isn't an ORM. Here's two examples directly from the docs: # Query all rows in the "users" table, filtering for users whose age is > 18, and selecting their name "users" |> where([u], u.age > 18) |> select([u], u.name) # Build a dynamic query fragment based on some parameters dynamic = false dynamic = if params["is_pub…

> Across all the different means of interacting with a database I have experience with (from full-fledged ORMs like ActiveRecord, to sprocs in ASP.NET), I've found that it offers the best compromise between providing an ergonomic abstraction over the database, and not hiding all of the nitty-gritty details you need to worry about in order to write performant queries or use database-specific features like triggers or…

Love the downvotes instead of comments. I've walked away from Elixir as the best practice deployment methodology (Distillery) is non-op on FreeBSD[1] and has been for a few months while the Distillery author is mum. All of this despite the vast love that the Elixir community seems to heap on FreeBSD.

Erlang and Elixir have plenty of promise but there simply is no good story for production deployments. Distillery and edeliver approximate capistrano, and that sounds great when it works (although I'd just as soon skip edeliver). But when it doesn't I'd much rather dig into the mess of ruby that is Capistrano than the mess of shell scripts, erlang, and god knows what else goes into a Distillery release.

Elixir is a really interesting language, but Phoenix seems to still be pretty wet behind the ears and very much in flux. Ecto too to a much smaller extent.

1: Some of the distillery scripts can communicate with epmd, some just give up.

Re: Bye Bye Mongo, Hello Postgres

#330

Post by inferiorhuman »

zaarn wrote: post #327
inferiorhuman wrote: post #290

Earlier quoted context omitted.

More the point Postgres has an actual array data type (and has for a while). You don't need to shove everything into a JSON/JSONB blob unless you absolutely cannot have any sort of schema.

Not only arrays, you can, with some limitations, create proper types with field names, if your ORM supports that you should use that over JSONB if it fits.

See also: PostGIS.
Post reply on HN