Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We migrated our PostgreSQL database with 11 seconds downtime

gds.blog.gov.uk

Post reply on HN
Print view View original

151–160 of 210 posts

Re: We migrated our PostgreSQL database with 11 seconds downtime

#151

Post by rjzzleep »

alerter wrote: post #148

Interesting, though I have no idea why the government is using AWS in the first place. This isn't a startup hacking away trying to find PMF, or dealing with unpredictable marketing-driven traffic spikes. We know we need these services running long term, and can make solid predictions about usage patterns. We could build a public sector cloud and/or adopt a sensible on-prem approach. This requires funding, coordinatio…

I don't know about the UK, but AWS has has GovCloud in the US for a long time, and to be honest compared to a lot of infrastructure I have seen there it's a blessing. On the flipside, I've met some really amazing infrastructure and ops people in a German gov healthcare institution running the in-house DC, where the problem was neither the tech, nor the people, but 100% the management and their processes, and their desire to be the bottleneck for every single interaction between infrastructure and engineering teams.

Re: We migrated our PostgreSQL database with 11 seconds downtime

#152

Post by draaglom »

globular-toast wrote: post #142

Is that a picture of them doing it? What is this weird fantasy that programming looks like this? It's always the same: a couple of people, always of diverse backgrounds, physically interacting, usually pointing and laughing at something, with perhaps another group, representing different demographics, looking on with a mixture of curiosity and awe. Do an image search for programming to see more examples of this. I gu…

The primary goal of most large organisations' dev blog is to attract potential candidates to the top of the recruitment funnel.

This picture appears to be (presumably) the actual team doing a team building activity.

This is both a reasonable choice given the inferred instrumental goals of the blog, and a honest reflection of (one part of) the team life, assuming it's a photo of the real team.

Re: We migrated our PostgreSQL database with 11 seconds downtime

#153

Post by globular-toast »

johneth wrote: post #149
globular-toast wrote: post #142

Is that a picture of them doing it? What is this weird fantasy that programming looks like this? It's always the same: a couple of people, always of diverse backgrounds, physically interacting, usually pointing and laughing at something, with perhaps another group, representing different demographics, looking on with a mixture of curiosity and awe. Do an image search for programming to see more examples of this. I gu…

Give it a rest, mate.

Nah, this is the thing that leads to open offices etc. I want people to understand what programming actually looks like and these silly pictures need to go.

Re: We migrated our PostgreSQL database with 11 seconds downtime

#154

Post by StupidOne »

alerter wrote: post #148

Interesting, though I have no idea why the government is using AWS in the first place. This isn't a startup hacking away trying to find PMF, or dealing with unpredictable marketing-driven traffic spikes. We know we need these services running long term, and can make solid predictions about usage patterns. We could build a public sector cloud and/or adopt a sensible on-prem approach. This requires funding, coordinatio…

I'm sure a lot of countries in EU did this. First hand I know, they did this in Croatia as I was one of the developer who had to use it to deploy on it. The worst thing I have ever seen in my life. And I worked on a lot legacy apps written in VB.NET, Web forms, old Sharepoint, Basic and even when the whole app was one big mess of store procedures.

AWS, Azure, GC are at least written with thought about end users (us, developers) while government cloud was architectured, designed and built by the lowest bidder whose first goal was to cut and cut and cut his costs whenever possible.

Re: We migrated our PostgreSQL database with 11 seconds downtime

#155

Post by jamietanna »

alerter wrote: post #148

Interesting, though I have no idea why the government is using AWS in the first place. This isn't a startup hacking away trying to find PMF, or dealing with unpredictable marketing-driven traffic spikes. We know we need these services running long term, and can make solid predictions about usage patterns. We could build a public sector cloud and/or adopt a sensible on-prem approach. This requires funding, coordinatio…

The UK government as a whole operates across the big names in the public cloud as well as some on-Prem/colo.

I'd very much recommend watching https://youtube.com/watch?v=mpY1lxkikqM&pp=ygUOUmljaGFyZCB0b... from September about Gov.UK's various iterations and some of the migrations across cloud that they've had to do.

One thing about (at least UK government) is that procurement requirements means that they go to market for quotes around usage every few years. If ie Oracle Cloud was 1/10th the price, it would likely mean they'd win the deal, and so would have to migrate to Oracle for the duration of the contract, and then potentially do the same to another cloud if that was cheaper

Re: We migrated our PostgreSQL database with 11 seconds downtime

#156

Post by kakoni »

bboygravity wrote: post #28
kunwon1 wrote: post #16

Earlier quoted context omitted.

AWS has a G-Cloud for UK just like they have one for US, no?

Why can't the UK government build there own cloud? It's just completely insane to me that they would make the gov internet infrastructure completely (geopolitically) dependent on another country AND just literally give all their (citizens') data away AND pay for that "privilege"?! I mean if the government can't host the government's websites using tech from the government's country, maybe it would be better to just f…

You know that UK's National Health Service did a deal with Palantir for "federated data platform"?

Re: We migrated our PostgreSQL database with 11 seconds downtime

#157

Post by rcxdude »

dijit wrote: post #24
lnxg33k1 wrote: post #13

Earlier quoted context omitted.

Its not an organisation, its a blucking government, it handles citizen data, and its sending them to a company of foreign country, because it can’t hire some system administrators? A GOVERNMENT? What are they doing? Still looking for their product market fit and can’t afford the headcount? Is it a joke? EDIT If they are looking for money id like to participate a bit in the seed round

Sysadmins are cheaper than many people seem to think. I had a person I trust a lot telling me that "if we go with a bare metal provider like GCore we'd have to hire someone", his reason for bringing that up was that the cost difference would be justified by not having to hire someone,. However a GCore €400,000k/y bill becomes a €6,000,000~ if you were to use a public cloud, even with the scaling up and down when not…

yeah, every company I know of that uses cloud has a team responsible for managing it anyway, and they don't seem much smaller than the team needed to manage on-prem. I don't really think this 'savings' exists in most cases.

Re: We migrated our PostgreSQL database with 11 seconds downtime

#158

Post by another2another »

juggertao wrote: post #113
justinclift wrote: post #93

Earlier quoted context omitted.

"Lets go with the easier approach" is a bit worrying when it means potentially breaking security assumptions / legal assurances. Though I guess it could be explained away by having something in the website's "Terms and Conditions for use". ;)

It's a gov website. What are they going to do, sue themselves?

Have a public enquiry, with lots of tea.

Re: We migrated our PostgreSQL database with 11 seconds downtime

#159

Post by rcxdude »

0xbadcafebee wrote: post #121
15457345234 wrote: post #98

Earlier quoted context omitted.

> is hiring experts 'moving to AWS' (or any cloud provider) is not 'hiring experts' it's just outsourcing the risk to an entity that you, in the event of a genuine crisis, have no leverage over beyond 'we're going to stop paying you (once we migrate away from you which will take ten years)'

AWS are not experts at providing computing services? Holy cow. This is news to me! I thought they were the most popular and highly regarded computing infrastructure and PaaS company in the world, managing both hardware and software and providing subject matter experts to work with customers on architecture and implementation, along with official partners and a marketplace of turn-key products. Boy, am I embarrassed!…

They may be experts, but you sure aren't hiring them. You are renting something from them.

Re: We migrated our PostgreSQL database with 11 seconds downtime

#160

Post by pbrooko »

pedrokost wrote: post #131

We recently completed the migration of a self-hosted 3 TB PostgreSQL database from version 12 to 16, transitioning from Ubuntu 18 to Ubuntu 22. Concurrently, we had to upgrade various extensions, most notably Timescale, for which a compatible version did not exist across all scenarios. We performed the upgrade by updating a replica in the following sequence: - Start: PG12, Ubuntu 18, TS2.9 - Step 1: Set up a read-onl…

We recently followed an almost identical upgrade path (however at the time PG16 wasn't yet supported by Timescale, so stopped at PG15 + TS 2.12).

We did look into using logical replication to reduce the downtime of the upgrade, but because database schema and DDL commands aren't replicated, it seems that it isn't recommended with Timescale in the loop.. (I suppose the underlying schema changes that Timescale needs to make under the hood are mostly a function of your hypertable chunk sizing and what your incoming writes look like, so this could be planned around / timed well, but we felt it added too much potential complexity & risk compared to simply opting for a small maintenance window while pg_upgrade completed).

Post reply on HN