I wish PostgreSQL would allow something as Oracle Flashback feature plus allowing to control how far history do you want to keep for specific tables, as typically only part of your data needs full auditing.
Versioning data in Postgres? Testing a Git like approach
11–20 of 92 posts
Re: Versioning data in Postgres? Testing a Git like approach
#12Re: Versioning data in Postgres? Testing a Git like approach
#13Neon does something like this I believe. They allow you to "branch" your databases: https://neon.tech
Re: Versioning data in Postgres? Testing a Git like approach
#14Better idea: just install the Postgres extension that enables ISO SQL Temporal Tables, just like what MSSQL and MariaDB have had for 8 years now: https://pgxn.org/dist/temporal_tables/
Re: Versioning data in Postgres? Testing a Git like approach
#15Better idea: just install the Postgres extension that enables ISO SQL Temporal Tables, just like what MSSQL and MariaDB have had for 8 years now: https://pgxn.org/dist/temporal_tables/
Separately though, it's great to see that there's still people trying to solve this problem at the extension level for Postgres. I was sad to discover that Postgres for a very long time had an official extension for time travel - https://www.postgresql.org/docs/11/contrib-spi.html#id-1.11.... - only to realize that it had been removed in https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit... - with a note that "it's easy to implement this separately" but no real hints that anyone was taking up the mantle.
As a simpler alternative, if you're only modifying data via Django and can tolerate some unreliability, https://django-simple-history.readthedocs.io/en/latest/ can get you much of the way there without a need for extensions. We've built on top of it with some domain-specific modifications to visualize and filter diffs, giving us at least a best-efforts audit-esque log system.
Re: Versioning data in Postgres? Testing a Git like approach
#16Re: Versioning data in Postgres? Testing a Git like approach
#17Better idea: just install the Postgres extension that enables ISO SQL Temporal Tables, just like what MSSQL and MariaDB have had for 8 years now: https://pgxn.org/dist/temporal_tables/
https://mariadb.com/kb/en/temporal-tables/
A quick Google turns up nothing for MySQL temporal.
Re: Versioning data in Postgres? Testing a Git like approach
#18Better idea: just install the Postgres extension that enables ISO SQL Temporal Tables, just like what MSSQL and MariaDB have had for 8 years now: https://pgxn.org/dist/temporal_tables/
Does this robustly support altering tables? What happens if a column is deleted - does it delete historical data, or mark the history column as nullable? As the OP article notes/hints, there are all sorts of performance and regulatory considerations around this. Separately though, it's great to see that there's still people trying to solve this problem at the extension level for Postgres. I was sad to discover that P…
Re: Versioning data in Postgres? Testing a Git like approach
#19I deployed NeonDB in a large enterprise client in Q2 of this year and, yes, the initial data migration is the hard part. We added 2x100gbe network cards directly on the VMware hosts where the data was so the migration to the Kubernetes cluster running NeonDB took hours instead of weeks.
Re: Versioning data in Postgres? Testing a Git like approach
#20Better idea: just install the Postgres extension that enables ISO SQL Temporal Tables, just like what MSSQL and MariaDB have had for 8 years now: https://pgxn.org/dist/temporal_tables/
Does MySQL have temporal? I thought only MariaDB implemented this. https://mariadb.com/kb/en/temporal-tables/ A quick Google turns up nothing for MySQL temporal.