161–170 of 210 posts
I'm quite negatively surprised that a government service is moving from their own platform to AWS for such an important service.
Earlier quoted context omitted.
That would indeed be a risk, but the circular logic of this means no new company could ever have any competence outside of its founders. Which feels shortsighted. Anyway, I am a former sysadmin. I am confident that I can identify competence in the requisite areas.
Governments tend to be far less competent at determining technical competence. Due to a wide variety of factors, governments tend to be completely uncompetitive in salary for technical positions meaning they're already hiring from the lower end of the pool (not including a few altruistic folks willing to forgo their market value). At a company if a department isn't working out you just restructure and move on, but in…
Lack of cost control or effective use of a cloud provider leads to spiralling uncontrollable costs.
Interesting, though I have no idea why the government is using AWS in the first place. This isn't a startup hacking away trying to find PMF, or dealing with unpredictable marketing-driven traffic spikes. We know we need these services running long term, and can make solid predictions about usage patterns. We could build a public sector cloud and/or adopt a sensible on-prem approach. This requires funding, coordinatio…
If you really think the public sector could build anything closely resembling any cloud, you are dreaming. Imagining that one cloud working for the entire public sector, we are entering delusional territory here.
Public sector projects are insanely expensive, take ages to develop and often flat out fail.
Not only that, we are starved for engineers even now. If we ran such a custom cloud, we would shoot ourselves in the leg by not being able to hire from a pool of experienced engineers.
Interesting, though I have no idea why the government is using AWS in the first place. This isn't a startup hacking away trying to find PMF, or dealing with unpredictable marketing-driven traffic spikes. We know we need these services running long term, and can make solid predictions about usage patterns. We could build a public sector cloud and/or adopt a sensible on-prem approach. This requires funding, coordinatio…
If you think government needs and demands are predictable, you don't follow politics (particularly uk politics in the last decade).
And then there are these things like pandemics that completely come out of left field. Being able to scale things on demand over the pandemic was one of the key demonstrators for use of the public commercial cloud by the public sector.
Earlier quoted context omitted.
There is no "just" about it. The absolute key takeaway is in "what we learned": > We chose to use DMS because it was well supported by the GOV.UK PaaS and we could also get support from AWS. If we were doing a PostgreSQL to PostgreSQL database migration in the future, we would invest more time in trying alternative tools such as pglogical. DMS potentially added more complexity, and an unfamiliar replication process t…
Even AWS in their own docs says to use the native tools when migrating from postgres to postgres[1]. They don't go into the details to much and points to pg_dump rather than pg_logical, but interesting to see that they don't recommend using DMS for it [1] https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source...
Interesting, though I have no idea why the government is using AWS in the first place. This isn't a startup hacking away trying to find PMF, or dealing with unpredictable marketing-driven traffic spikes. We know we need these services running long term, and can make solid predictions about usage patterns. We could build a public sector cloud and/or adopt a sensible on-prem approach. This requires funding, coordinatio…
Why would this be any different for a government?
We don't expect our government to build their own cars, but to buy them from Volkswagen or Renault. Even when the government has a clear need for transport. Why do we then insist they build their own IT infrastructure?
We did a similar migration (somewhat larger database) with ~20 seconds of downtime and much less work... using the magic of AWS RDS Blue-Green Deployments [1]. Surprised they aren't mentioned in the thread yet. Basically, you spin up a new Blue Green deployment with any desired changes (in our case, we were upgrading Postgres major from 13 to 15). While your blue configuration continues to serve traffic, AWS uses log…
How did you go about stopping and restarting applications which reach out to the database? We have a number of tasks running in ECS which can take a minute to spin down and a few minutes to spin back up.
We also had background worker services. For the very high throughput ones, we spun down the # of tasks to a bare minimum for <5 minutes and let the queue build up, rather than have a massive amount of errors and retries. For the other ones where throughput wasn't high, we just let them be, and during the downtime they errored and retried and the retries mostly succeeded.
Is that a picture of them doing it? What is this weird fantasy that programming looks like this? It's always the same: a couple of people, always of diverse backgrounds, physically interacting, usually pointing and laughing at something, with perhaps another group, representing different demographics, looking on with a mixture of curiosity and awe. Do an image search for programming to see more examples of this. I gu…
Earlier quoted context omitted.
Why? Do you deny the benefits of shared hosting in its entirety?
No. But I wish government agencies depend less on other entities. A government agency isn't in a business to make money. They should be concerned less with economic efficacy and more with accomplishing their goals. They have a fiduciary duty towards the citizens.
Additionally, a pandemic has shown that government services actually need to scale to arbitrary amounts, seemingly at random.
Don't you think that in this case using AWS is better to simply guarantee availability? Imagine tax payers money being spent on servers and then a pandemic happens and when the economy is screeching to a halt, more tax payers money is needed to expand the cluster, money that really would be useful to spend elsewhere.
Earlier quoted context omitted.
Well you’ve certainly introduced a teaching moment to me! What are the nature of writes you deal with that last 30+ minutes? What kind of data/what kind of people are involved with such DB writes where you need to rely on the DB engine to work so hard instead of something more split up by queues at a higher layer?
It's usually badly engineered things. However, just because it's badly engineered doesn't mean it's fine for it to break :-P Things like a super complex n^3 complexity query to find all users who were 3 hops away from a known fraudster, where a 'hop' means 'shared any piece of account data in common'