Is this related to supabase? There seems to be a link?
Actually no, this is a project by Xata. What link do you mean?
Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
11–20 of 55 posts
Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#12Here's a few other migration tooling options for Postgres: https://gist.github.com/cpursley/c8fb81fe8a7e5df038158bdfe0f...
Can you add https://github.com/shayonj/pg-osc? It's my favorite PG migration tool.
Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#13This seems nice at first glance, but lets think about it for a minute:
Before: You had a .sql file and if you messed up you had to revert manually. Maybe you would pre-write the revert script, maybe your site is down if you mess up. It's super easy to understand what is happening though.
Now: you use pgroll. An absolute heaping ton of magic is happening behind the scenes. Every table is replaced by a view and tables have tons of 'secret' columns holding old data. Every operation that made sense before (ALTER TABLE foo ADD COLUMN bar INTEGER NOT NULL) turns into some ugly json mess (look at the docs, it's horrible) that then drops into the magic of pgroll to get turned into who knows what operations in the db, with the PROMISE that it can be reversed safely. Since all software has bugs, pgroll has bugs. When pgroll leaves you in a broken state, which it will sooner or later, you are just FUCKED. You will have to reverse engineer all it's complicated magic and try to get it back on the rails live.
You're trading something you understand and is simple, but maybe not super convenient, for something that is magically convenient and which will eventually put a bullet in your head. It's a horrific product, don't use it.
Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#14Pgroll shines if you are doing slow rollouts.
Recently on the market for a tool to manage SQL migration patches with no need for slow rollouts, I reviewed many such tools and the one that impressed me was sqitch: https://github.com/sqitchers/sqitch
So if you are interrested in this field and if Pgroll is not quite what you are looking for, I recommand you have a look at sqitch.
Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#15This seems nice at first glance, but lets think about it for a minute: Before: You had a .sql file and if you messed up you had to revert manually. Maybe you would pre-write the revert script, maybe your site is down if you mess up. It's super easy to understand what is happening though. Now: you use pgroll. An absolute heaping ton of magic is happening behind the scenes. Every table is replaced by a view and tables…
What is good for the goose is not necessarily good for the gander.
Obviously for startups, you're 100% right. Just announce a brief downtime and/or do migrations after-hours. Keep it simple, no one will care if their requests timeout once every week for 30 seconds.
If your company has hundreds of developers making changes across every timezone and downtime (or developers being blocked waiting for scheduled merge windows) costs real money or creates real problems other than optics, something like this or Vitess (MySQL) is definitely worth it.
Engineering should not be a "one-size-fits-all" type of job, and while I do love postgres, my main gripe with the community is that the "keep it simple stupid" mentality persists well beyond its sell-by date in many cases.
Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#16Pgroll shines if you are doing slow rollouts. Recently on the market for a tool to manage SQL migration patches with no need for slow rollouts, I reviewed many such tools and the one that impressed me was sqitch: https://github.com/sqitchers/sqitch So if you are interrested in this field and if Pgroll is not quite what you are looking for, I recommand you have a look at sqitch.
> if Pgroll is not quite what you are looking for
If you don't need slow rollouts, what would you say the downsides of using Pgroll over Sqitch would be?
(I've used neither, but I got the impression from the op that slow rollouts was a feature, not a requirement)
Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#17Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#18This seems nice at first glance, but lets think about it for a minute: Before: You had a .sql file and if you messed up you had to revert manually. Maybe you would pre-write the revert script, maybe your site is down if you mess up. It's super easy to understand what is happening though. Now: you use pgroll. An absolute heaping ton of magic is happening behind the scenes. Every table is replaced by a view and tables…
What is good for the goose is not necessarily good for the gander. Obviously for startups, you're 100% right. Just announce a brief downtime and/or do migrations after-hours. Keep it simple, no one will care if their requests timeout once every week for 30 seconds. If your company has hundreds of developers making changes across every timezone and downtime (or developers being blocked waiting for scheduled merge wind…
The bigger and more important your company is, the less you should rely on a tool like this. You have more budget to invest in operations and less tolerance for being down for 3 days when a tool like this has a bug that takes your site down. You should hire DBA's and operations staff that understand how to apply db migrations in a safe way, and have them review them before and/or apply them during deployments. It's not hard to do manually, just a little bit of extra work (not that much extra). With a little bit of feedback engineers will learn the basics of how to write migrations that are safe and then the use case for this product is dramatically reduced anyway.
You keep things simple because you need to be able to understand what is going on to work with it later. pgroll is inherently complex and poorly designed, but even if it wasn't it is still bad to use something that you can't reasonably correct the problems it causes when it breaks.
Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#19Pgroll shines if you are doing slow rollouts. Recently on the market for a tool to manage SQL migration patches with no need for slow rollouts, I reviewed many such tools and the one that impressed me was sqitch: https://github.com/sqitchers/sqitch So if you are interrested in this field and if Pgroll is not quite what you are looking for, I recommand you have a look at sqitch.
Sqitch is an incredibly under appreciated tool. It doesn’t have a business pushing it like flyway and liquibase, so it isn’t as widely known, but I vastly prefer it to comparable migration tools.
Re: Pgroll – Zero-downtime, reversible, schema changes for PostgreSQL (new website)
#20This seems nice at first glance, but lets think about it for a minute: Before: You had a .sql file and if you messed up you had to revert manually. Maybe you would pre-write the revert script, maybe your site is down if you mess up. It's super easy to understand what is happening though. Now: you use pgroll. An absolute heaping ton of magic is happening behind the scenes. Every table is replaced by a view and tables…
I think it is fair criticism that this adds complexity. However, I do have a couple of counter-arguments:
In order to avoid downtime and locking, you generally need multiple steps (e.g. some variation of add another column, backfill the data, remove previous column). You can codify this in long guidebooks on how to do schema changes (for example this one from gitlab [1]). You also need to orchestrate your app deployments in between those steps, and you often need to have some backwards compatibility code.
This is all fine but: 1. it slows you down and 2. it's manual and error prone. With pgroll, the process is always the same (start pgroll migration, deploy code, complete/rollback migration) so the team can exercise it often.
Second, while any software has bugs, it's worth noting that the main reason roll-ing back is quick and safe with pgroll is that it only has to drop views and any hidden columns. While the physical schema is changed, it is in a backwards compatible way until with complete the migration, so you can always skip the views if you have to bypass whatever pgroll is doing.
[1]: https://docs.gitlab.com/ee/development/migration_style_guide...